Easy SQL Query Counting in Django

I’ve become somewhat of a Django ninja during my time as GradeSolve’s developer (okay, a Django hacker…). One of the things that I have noticed as I have written several thousand lines of Django code is that SQL queries tend to be made pretty regularly–often, a little too regularly. Some of GradeSolve’s pages were loading a little slowly for my tastes, so I decided to do some basic SQL profiling to see where the trouble was.

The issue there was that django-debug-toolbar didn’t want to play nice, and I lacked the motivation to try to get it working. Instead, I re-invented the wheel a little bit and write a simple piece of middleware to print out the SQL execution time and number of SQL queries made for a particular request. Obviously, this is going to go away in the production version of GradeSolve (which is being released in eight days!), but for the time being, I like the extra information on my debug console.

Anyway, here’s the middleware class. Stick this in one of your project’s files:

from django.db import connection
class SqlPrintMiddleware(object):
    def process_response(self, request, response):
        sqltime = 0 # Variable to store execution time
        for query in connection.queries:
            sqltime += float(query["time"])  # Add the time that the query took to the total
 
        # len(connection.queries) = total number of queries
        print "Page render: " + unicode(sqltime) + "sec for " + unicode(len(connection.queries)) + " queries"
 
        return response

Now, in your settings.py file, add the path to the middleware class. For example, if your project’s name is gradesolve and your file’s name is middleware.py, your middleware classes setting would end up looking like:

MIDDLEWARE_CLASSES = (
    # ...
    'gradesolve.middleware.SqlPrintMiddleware',
    # ...
)

Happy (very simple) profiling!

How to Keep Unoconv + Apache From Making You Sad

Server-side conversion of documents is a fairly common task. On Linux/Apache, doing so should be easy: you should just be able to run LibreOffice or OpenOffice in headless mode. If you’re really adventurous, you can install unoconv, which makes document conversion better[citation needed]. Anyway, you’ll be delighted to know that you can invoke any of those programs from your terminal. You won’t be delighted to know that they will return error code 77 when Apache tries to invoke them to actually convert a document. That means your server will probably return a 500 error to your users, and that means you’ll get a phone call at 3:00 in the morning from a crying user. Nobody wants to get a 3 AM phone call from a crying user.

To save yourself from such punishment, there is actually a very simple fix. First, create a new home directory for your Apache user (www-data):

sudo mkdir /home/www-data && sudo chown www-data /home/www-data

Then, you’ll need to edit your /etc/passwd file (scary stuff, right) to change the Apache user’s home directory and default shell. Doing so magically gets rid of your error code 77 troubles (I can’t explain further than that).

sudo nano /etc/passwd

Scroll down to the line that starts with “www-data.” It will look something like “www-data:x:[a number]:[a number]:[a string]:[user's home directory]:[path to user's default shell].” Obviously, we only care about the last two parts. Change the default directory to /home/www-data and then change the default shell to /bin/bash. You may need to move some of your website’s files around–I’m not sure because I use Django with mod_wsgi for GradeSolve so my files are stored elsewhere and I’m still an Apache noob.

Good luck.