September 15, 2011

Profiling Django for CPU bound apps

Posted in Software at 20:24 by graham

For most Django apps, indeed most webapps, the bottleneck is the database. The biggest gains usually come from reducing the number of queries used, and adding database indexes. django-debug-toolbar helps a lot here. After that, caching and de-normalization also help reduce database queries.

But what if your app is CPU bound? How do you find out where it’s spending it’s time? It’s easy with the runprofileserver from django-extensions – here’s how:

pip install django-extensions

Add django_extensions to your INSTALLED_APPS, then run the profiling server.

django-admin.py runprofileserver

In your browser, hit the url you’re interested in. runprofileserver will create hotshot files in /tmp/. Ctrl-C out of your server, and fire up ipython.

from hotshot import stats
p = stats.load('/tmp/<your filename here>')

# In order that functions got called
p.sort_stats('cumulative').print_stats(20)

# Slowest method first
p.sort_stats('time').print_stats(20)

Here p is a pstats object. I prefer the cumulative view. Look down it until you see a big drop in the cumulative time – that’s where the time was spent.

Leave a Comment

Note: Your comment will only appear on the site once I approve it manually. This can take a day or two. Thanks for taking the time to comment.