My Django

I used shared hosting for my Django-based Encyclopedia of Programming Languages (Russian version), but realized, that I need dedicated server for more performance and stability. For now, I chose cheap server with 1,8 GHz CPU and 512 Mb RAM, with CentOS 5 installed.

In this article I’ll share my experience for installing on this server Django with Apache mod_python, with memcached caching and lighttpd for serving static files.

So, we have a freshly installed CentOS 5 and want to host Django-based site (or several sites).

What should we install for this?

1. Django itself
2.  MySQL-python
3.  mod_python
4.  memcached
5.  lighttpd

All steps excluding installing Django itself are optional (for example, you can use Django with fcgi instead of mod_python and PostgreSQL or other DBMS instead of MySQL), but steps 1-3 are what you likely want to use in typical environment and steps 4-6 are for gaining better performance.

Most of the things in this article are CentOS-oriented, but to some extent these instructions will be applicable to any other Linux distribution.

In this article if there is no rpm-package with program or library that I need in the official CentOS repositories, I install that program or library from source. You may prefer searching for third-party rpms, but it may lead to dependency problems and even security issues. The ideal option, I guess, is to roll rpms yourself, but it will require some extra time and knowledge.

Some security considerations: as this is a freshly installed system, I’m not afraid of screwing up things, so I may work as root user. But it’s better to run most commands as unprivileged user and use “sudo” for commands that require root privileges.

So, lets start.

Django itself

Usually, you should use the latest svn-version of Django framework, not so-called “release” version. Currently, “official” release version is too old and lacks many important features (for instance, unicode support and auto-escaping).

So you should go to command line and type

svn co http://code.djangoproject.com/svn/django/trunk/ django-trunk

But before that you should install svn

yum install subversion

After fetching files from svn run

python setup.py install

After installing Django you should get your project files (via svn or simply by copying via ssh or ftp; you may also want to create a new system user to own these files) and database dump, and adapt your database settings in settings.py file to new environment (it is assumed that MySQL is already installed and configured, database and user are created, all data is in place – these things are far beyond the scope of this article).

Then go to your project directory and try running Django development server

python manage.py runserver 127.0.0.1:8000

On CentOS 5 you’ll get error message “Error loading MySQLdb module”.

MySQL-python version in CentOS distribution is too old for the latest Django svn-version, so you need to build it from sources.

MySQL-python

cd /usr/local/src
wget http://garr.dl.sourceforge.net/sourceforge/mysql-python/MySQL-python-1.2.2.tar.gz
tar xvzf MySQL-python-1.2.2.tar.gz
cd MySQL-python-1.2.2

gcc, python-devel and mysql-devel packages are required to build MySQL-python.

yum install gcc
yum install python-devel
yum install mysql-devel

cd /usr/local/src/MySQL-python-1.2.2
python setup.py build
python setup.py install

Done, MySQL-python is installed as .egg file in your site-packages directory.

Than again run

python manage.py runserver 127.0.0.1:8000

and look for any errors. For example, I use ImageFields in my projects, so I had to additionally install Python Imaging Library (yum install python-imaging).

If there are no errors, it’s generally a good idea to change 127.0.0.1 in runserver command to your IP-address and check if site works OK with your browser, but to do this you need to open 8000 port on CentOS firewall or just temporarily disable firewall (or you can temporarily shut down apache and run Django development server on 80 port).

If there are no errors, it’s time to shut down django development server and configure real web-server.

mod_python

It’s easy to configure apache with mod_python to work with Django. Just follow instructions in Django official documentation.

Just one note: since we installed MySQL-python as .egg-package, it’s important to create temp directory (set 777 permissions for it) for .egg cache and make a reference to it in the httpd.conf file:

<Location "/">
    ...
    SetEnv PYTHON_EGG_CACHE /var/tmp/egg
</Location>

And don’t forget to restart apache after editing httpd.conf:

/etc/init.d/httpd restart

Now your site should work via apache on standard http port. Check it with your browser.

memcached

Enabling memcached caching is very important for performance. Unfortunately, memcached isn’t in CentOS repository, so again we have to install from source. This chapter is partially based on this howto.

First of all, you need to install libevent:

cd /usr/local/src
wget http://monkey.org/~provos/libevent-1.4.2-rc.tar.gz 
tar xvzf  libevent-1.4.2-rc.tar.gz
cd libevent-1.4.2-rc
./configure
make
make install


vi /etc/ld.so.conf.d/libevent.conf
##add a line containing:
/usr/local/lib/

ldconfig -v

And then memcached itself:

cd /usr/local/src
wget http://danga.com/memcached/dist/memcached-1.2.5.tar.gz
tar zxvf memcached-1.2.5.tar.gz
cd memcached-1.2.5
./configure
make
make install

cp scripts/memcached.sysv /etc/init.d/memcached

ln /usr/local/bin/memcached /usr/bin/memcached
/etc/init.d/memcached start

chkconfig memcached on

And finally we need to install libmemcache and cmemcache (python extension for libmemcache):

cd /usr/local/src/
wget http://people.freebsd.org/~seanc/libmemcache/libmemcache-1.4.0.rc2.tar.bz2
bunzip2 libmemcache-1.4.0.rc2.tar.bz2
tar xf libmemcache-1.4.0.rc2.tar
cd libmemcache-1.4.0.rc2
./configure
make 
make install

cd /usr/local/src/
wget http://gijsbert.org/downloads/cmemcache/cmemcache-0.91.tar.bz2
bunzip2 cmemcache-0.91.tar.bz2
tar xf cmemcache-0.91.tar
cd cmemcache-0.91
python setup.py install

Then enable memcached caching in your project settings.py file

CACHE_BACKEND = 'memcached://127.0.0.1:11211/'

(check the official Django documentation for more information) and restart apache. Now django stuff should be cached with memcached.

To get even more performance benefits we shouldn’t use bloated Apache for such simple task as serving static files, so let’s install lighttpd.

lighttpd

cd /usr/local/src
wget http://www.lighttpd.net/download/lighttpd-1.4.19.tar.bz2
bunzip2 lighttpd-1.4.19.tar.bz2
tar xf lighttpd-1.4.19.tar
cd lighttpd-1.4.19
yum install glib2-devel openssl-devel pcre-devel bzip2-devel gzip-devel
./configure
make 
make install

cp doc/rc.lighttpd.redhat /etc/init.d/lighttpd
chkconfig lighttpd on

mkdir /etc/lighttpd
cp doc/lighttpd.conf /etc/lighttpd/

And now we should configure apache and lighttpd so that static files will be served by lighttpd (on 80 port), and other requests will be handled by apache on 81 port (thanks to nolancaudill.com for solution):

• tell apache to listen the 81 port: find Listen 80 and change it to Listen 127.0.0.1:81. Don’t forget to change port number in all VirtualHost directives, too. Then restart apache.

• edit /etc/lighttpd/lighttpd.conf. Uncomment ‘mod_alias’ and ‘mod_proxy’ modules and add something like this:

  $HTTP["host"] == "yourdomain.com" {
  
      # here we are mapping /media/ for admin media
      # and /static/ for the standard media_url
      alias.url = (
          "/media/" => "/usr/lib/python2.4/site-packages/django/contrib/admin/media/",
          "/static/" => "/var/www/html/media.yourdomain.com/",
      )
  
  
      $HTTP["url"] !~ "^/(static|media)/" {
          proxy.server = ( "" => (("host" => "127.0.0.1", "port" => 81 )) )
  
      }
  
  }
  

Also change server.document-root to “/var/www/html/” and link your project media directory to ”/var/www/html/media.yourdomain.com/”.

• Create directory for lighttpd log-file:

  mkdir /var/log/lighttpd/
  

• One more little edit to /etc/init.d/lighttpd: change path from /usr/sbin/lighttpd to /usr/local/sbin/lighttpd. Start lighttpd:

  /etc/init.d/lighttpd start
  

And finally some other performance tips:

• turn off all services you don’t use
• tune MySQL server (you may use tuning-primer.sh script, for example)
• turn off KeepAlive in httpd.conf
• more tips from Jacob Kaplan-Moss