Django, Meet Flume

This page an effort to merge Flume with Django. It's hypothesized that such a merger will help build a more compelling W5 prototype.

Django is one of these fancy new Web frameworks that makes life easier for Web programmers. It's basically the Python version of Ruby on Rails. It's a little hard to get ahold of what it's really doing, but here is my best guess. It is:

• A declarative language of sorts so that Web developers can specify their data model without writing SQL code. The point is that Django developers can play with objects stored in databases — loading them, saving them, altering them, filtering sets of them,etc — but without writing SQL. The SQL is all implied by the data model. That's a nice feature.
• An HTML templating system, that allows developers to split their HTML code from their Python logic. It has fancy features like iteration, to allow a developer to spit out a list of objects into an HTML table or whatnot.
• Fancy tools for dealing with HTML forms
• A fancy admin interface.

Of these aspects, I find the first most interesting, and it's what we've been focusing on to date. Also, a short list of what Django, in the current form, is not:

• A front-end that builds Python-friendly persistent objects over a generic storage backend. Rather, SQL seems pretty hardcoded in Django (and I believe the same is true for Ruby on Rails). It would be nice if a flatfile backend worked, or some sort of key-value store, but unfortunately, a SQL database backend is required.

However, it does look like people have gotten Django working with various “middleware” such as memcached, so it might be possible to build a high-traffic site with the toolkit. More exploration is required…

If you want to play along at home, here are my notes in installing django.

For example, let's try to specify something like a Facebook-style Web app,

from django.db import models
 
class User(models.Model):
    name = models.CharField(maxlength=200)
    join_date = models.DateTimeField('date joined')
    email = models.CharField(maxlength=200, unique=True)
    yob = models.IntegerField ()
 
class Institution(models.Model):
    name = models.CharField(maxlength=200)
    city = models.CharField(maxlength=20)
    state = models.CharField(maxlength=2)

The cool thing here is that the site developer can specify the data model in this syntax, without worrying about the SQL create commands. Behind the scenes, Django is creating primary keys for these objects, so they can be written to database, and read back out later, and linked to one another.

If you're playing along with the demo, put the above code in facebook/models.py. You can see the corresponding SQL statements with:

python manage.py sql polls

And you can actually run these command with:

python manage.py syncdb

You can write a little script to load and populate objects now:

import mysite.facebook.models as fb
print fb.User.objects.all ()
u = fb.User (name='Max K', 
             email='yo@bar.com', 
             join_date=datetime.datetime.now (), 
             yob=1988)
u.save ()
print "%s: %d" % (u.name, u.id)

I can run this from the shell (python manage.py shell) but for some reason I can't run it from the command line. Oh well.

The next thing I want to do is to associate Users with Institutions, as Facebook would do. Django has a many-to-many feature, but I think we actually want the label these associations, so I suggest a new data type:

class Affiliation (model.Model):
    desc = models.CharField(maxlength=200)
    startdate = models.DateField ()
    enddate = models.DateField ()
    user = models.ForeignKey (User)
    institution = models.ForeignKey (Institution)

Then I can do something like say that I'm getting my PhD from MIT:

import mysite.facebook.models as fb
mit = fb.Institution.objects.filter (id=1)[0]
me = fb.User.objects.filter (id=1)[0]
aff = fb.Affiliation (desc = 'phd',
                      enddate = datetime.datetime.now (),
                      startdate = datetime.datetime (2003,09,1),
                      user = me,
                      institution = mit)

There a whole bunch of other stuff that I don't know about, but I'll leave that out for now. This should be enough to start talking about what some viable data models might be for Django. This is all speculative right now….

In Django developers can express data models at a high level, using a simple syntax and using a declarative language. The idea here is to apply that same convenience to the security side of things (which Flume is going to provide via labels). Here are some preliminary thoughts:

First, let's tackle the User object from our earliest example:

from django.db import models
import flume.django.db.models as flmd
 
class User(models.Model):
 
    # These fields are as above.
    name = models.CharField(maxlength=200)
    join_date = models.DateTimeField('date joined')
    email = models.CharField(maxlength=200, unique=True)
    yob = models.IntegerField ()
 
    # whenever creating a new User object, also allocate a new
    # Flume capability group.  Grant this group to the global
    # user manager.  This is, very roughly, what FlumeWiki is
    # doing currently.
    caps = flmd.Group(name=name, umgr_grant=True)
 
    # Also when creating a new User object, create a new
    # export-protect and write-protect tag.  Grant corresponding
    # capabilities to the group 'caps' that was created above.
    ep = flmd.ExportProtectTag(name=name, rcap_grant=[caps])
    wp = flmd.WriteProtectTag(name=name, acap_grant=[caps])
 
    # The 'meta' subclass is something introduced in Django, to
    # specify meta properties for objects of type User.  In this
    # case we're specifying what secrecy and ownership labels to
    # apply to this object when stored as a row in the DB. Use
    # the export-protect and write-protect allocated above.    
    class Meta:
         S_label = [ ep ]
         O_label = [ wp ]

In this case, the user objects created of type facebook.User will be export-protected per user. Meaning all of the data in the User object is private. In a more fleshed out example, a User would have a several portions to his profile, some public, some totally private, and others selectively revealed.

I think things now start to get cool. Let's at first associate one Administrator with each Institution and play around with privileges:

class Institution(models.Model):
 
    # These 3 lines are as before!
    name = models.CharField(maxlength=200)
    city = models.CharField(maxlength=20)
    state = models.CharField(maxlength=2)
 
    # Appoint a user as an admin for this institution
    admin = models.ForeignKey (User)
 
    # Allocate export- and write-protect tags, but grant
    # the corresponding capabilities to the admin!
    ep = flmd.ExportProtectTag(name=name,
            rcap_grant = [ admin.caps ])
    wp = flmd.WriteProtectTag(name=name,
            acap_grant = [ admin.caps ])
 
    class Meta:
        # Institutions are public, but only the admin can
        # edit/update them
        S_label = []
        O_label = [ wp ]
 

Let's say that the default affiliation is public. That is, anyone can know that I'm getting my PhD at MIT. Of course, we said earlier that my profile was private, so there's a little bit of a conflict here. How about: assume for now that my profile is public up above.

class Affiliation (model.Model):
 
    # These 5 lines are as before!
    desc = models.CharField(maxlength=200)
    startdate = models.DateField ()
    enddate = models.DateField ()
    user = models.ForeignKey (User)
    institution = models.ForeignKey (Institution)
 
    class Meta:
        # Affiliations can be edited by the user or by the
        # the institution.  By default, affiliations are
        # public.
        O_label = [ user.wp, institution.wp ]
        S_label = []

But then we can start to subclass Affiliations if I want them to be totally private, or visible only to other people who have the same affiliation:

 
# If I go to MIT, only I can know that.
class PrivateAffiliation (Affiliation):
    class Meta:
        S_label = [ user.ep ]
 
# People also at MIT can know that I go to MIT
class SlightlyHiddenAffiliation (Affiliation):
     class Meta:
        S_label = [ institution.ep ]

• How do students at MIT get mit.ep when they become affiliated with MIT? I'm not sure how to express that. One thought is a on_insert hook in the affiliations object, with a lambda of some sort. Not ideal, I realize.
• How to do ACL-like operations such as: “I want my Photo Album from SOSP to be shown to all MIT affiliates and also a random UNM student of my choosing.” I fear that the mechanism to introduce such ACLs cannot possible be “declarative” like the other things we've seen so far.