April 30, 2011

My experience with django-mptt

Posted in Software at 00:57 by graham

In the past few months, I’ve inherited two projects which used django-mptt, a toolkit for adding trees to Django models. Here’s my experience so far:

mptt is full of magic

That’s both good and bad. Good because it does a lot for you. Bad because it’s difficult to find out what that is. By becoming an MPTTModel you magically get four new database fields, tens of methods, and a whole new manager, grafted on to your model.

The project is making an effort to reduce the magic, for example by switching from signals to method overrides, which simplifies things significantly.

You should always subclass MPTTModel, and never, never use the mptt.register(MyModel) approach. The docs recommend against it, and the core developers tried to remove it. To answer the example given for it’s necessity, if you need Django’s built-in Group to be hierarchical, create your own group which extends MPTTModel, and has a foreign key to the built-in Group.

Maybe you just wanted a foreign key to self

If you just want something like threaded comments, add this to your model:

parent = models.ForeignKey('self')

The intent will be immediately clear to those that come after you.

You should always start with the parent foreign key. If performance is a big problem later on, and you’re already de-normalized the important fields, then by all means add in mptt. It’s an optimization, and one that will cost you in maintainability, so be sure that cost is worth paying.

Do bulk imports in raw sql

Every time you save an MPTTModel, it will re-balance it’s tree. That makes saves slow and database intensive. If you’re doing bulk imports, you’ll need to switch to raw SQL, and then re-balance the tree afterwards.

If you try and create MPTTModel objects from many processes concurrently, you’ll get deadlocks in the tree re-balancing code. As above, do your heavy lifting in raw SQL.

Use MyModel.tree.rebuild() to rebuild your tree

There’s an undocumented method which will balance your tree. Use this after raw SQL inserts for example: TreeManager.rebuild.

Note that there’s probably a very good reason it’s undocumented. You’re on you’re own.

Conclusion so far

My experience with it so far tells me there’s a small class of problems where django-mptt would be fantastic. There’s also projects where it adds more complexity that it removes. Please start your project without it, and only add it when it’s really needed.

There three bigs thing I would love to see django-mptt do are:

  1. Keep removing the magic, aggressively.

  2. Provide a way to temporarily switch off tree balancing to speed up inserts.

  3. Document TreeManager.rebuild, or document why it’s dangerous.

I realize I am being a bad open-source citizen here by providing suggestions as text instead of pull requests, and I apologize. Like everyone else, I am short on time.

3 Comments »

  1. Matheus said,

    January 3, 2013 at 14:32

    Thanks for sharing your experience! I’ve a doubt if should I use or not, django-mptt. Here’s my case:

    class Animal: name = models.CharField(max_length=255) father = models.ForeignKey(self, null=True,blank=True,related_name=’father’) mother = models.ForeignKey(self, null=True, blank=True, related_name=’mother’)

    Do you think that django-mptt is necessary in this case?

    Thanks in advance!

  2. Gene said,

    March 23, 2012 at 16:33

    Hello,

    MyModel.tree.rebuild() pretty simple. Look in the github source in the manager.py module.

    rebuild() walks your tree based on the parent relations, and assigns lft, rght, level and tree_id values according to the standard MPTT method of doing that.

    What to watch out for with rebuild() is that children are ordered by “order_insertion_by”. If you have some order to your tree you want to preserve, but you don’t have a field to control that ordering, rebuild will probably scramble your ordering.

  3. h3 said,

    November 8, 2011 at 18:24

    Do you know about Django treebeard: https://tabo.pe/projects/django-treebeard/

    I found it while looking for something else than mptt because I use it and I often get weird problems with tree rebuilding. To the point where I sometime need to delete and reinsert the tree completely in order to get the correct ordering.

    Not sure if it’s better, but it seems pretty well documented, I might give it a try if mptt continues to be problematic.

Leave a Comment

Note: Your comment will only appear on the site once I approve it manually. This can take a day or two. Thanks for taking the time to comment.