April 30, 2011

My experience with django-mptt

Posted in Software at 00:57 by graham

In the past few months, I’ve inherited two projects which used django-mptt, a toolkit for adding trees to Django models. Here’s my experience so far:

mptt is full of magic

That’s both good and bad. Good because it does a lot for you. Bad because it’s difficult to find out what that is. By becoming an MPTTModel you magically get four new database fields, tens of methods, and a whole new manager, grafted on to your model.

The project is making an effort to reduce the magic, for example by switching from signals to method overrides, which simplifies things significantly.

You should always subclass MPTTModel, and never, never use the mptt.register(MyModel) approach. The docs recommend against it, and the core developers tried to remove it. To answer the example given for it’s necessity, if you need Django’s built-in Group to be hierarchical, create your own group which extends MPTTModel, and has a foreign key to the built-in Group.

Maybe you just wanted a foreign key to self

If you just want something like threaded comments, add this to your model:

parent = models.ForeignKey('self')

The intent will be immediately clear to those that come after you.

You should always start with the parent foreign key. If performance is a big problem later on, and you’re already de-normalized the important fields, then by all means add in mptt. It’s an optimization, and one that will cost you in maintainability, so be sure that cost is worth paying.

Do bulk imports in raw sql

Every time you save an MPTTModel, it will re-balance it’s tree. That makes saves slow and database intensive. If you’re doing bulk imports, you’ll need to switch to raw SQL, and then re-balance the tree afterwards.

If you try and create MPTTModel objects from many processes concurrently, you’ll get deadlocks in the tree re-balancing code. As above, do your heavy lifting in raw SQL.

Use MyModel.tree.rebuild() to rebuild your tree

There’s an undocumented method which will balance your tree. Use this after raw SQL inserts for example: TreeManager.rebuild.

Note that there’s probably a very good reason it’s undocumented. You’re on you’re own.

Conclusion so far

My experience with it so far tells me there’s a small class of problems where django-mptt would be fantastic. There’s also projects where it adds more complexity that it removes. Please start your project without it, and only add it when it’s really needed.

There three bigs thing I would love to see django-mptt do are:

1. Keep removing the magic, aggressively.

2. Provide a way to temporarily switch off tree balancing to speed up inserts.

3. Document TreeManager.rebuild, or document why it’s dangerous.

I realize I am being a bad open-source citizen here by providing suggestions as text instead of pull requests, and I apologize. Like everyone else, I am short on time.