The cake is a lie

Archive for September, 2008

Don’t put all your Python eggs in one basket

One of my favourite things about Python is how I can package up my module into an egg and easy_install it from a tarball, source export, or even an http link. All you need is setuptools and you’re off to the races, but there’s a few things to watch out for:

  • There’s no easy_uninstall. To remove an egg, you would need to go into /usr/lib/python2.5/site-packages/easy-install.pth, manually prune out the offending line, and finally rm -rf the directory container of that egg. Oh, and carefully pick out the various binary scripts the egg installed into /usr/bin. Good luck.
  • What if you’re working on two projects that require different versions of a specific egg? Having to re-easy_install a different version of an egg every time you switch contexts is no fun.
  • Didn’t your mother teach you that it’s always better to not do things as root?

The solution is virtualenv.

Virtualenv creates a local copy of everything you need for maintaining your eggs. The prescribed way to use it is to virtualenv ~/python1 and then execute ~/python1/bin/activate to enter into the environment of that local installation. From then on, anything you easy_install will go into your local installation instead of your root install. If zombies attack and you need to purge your site-packages of any disease, you can scrap the whole thing and start a-new — try to do that with your root install without losing hair.

The shazow method

The default use of virtualenv is great, but I like to take things a little further. I setup a permanent local install as soon as I log into a fresh Linux user. This way, I never have to install any eggs as root.

sudo easy_install virtualenv
virtualenv ~/local

And add these two lines to your ~/.bashrc or equivalent:

export PYTHONPATH=~/local/lib/python2.5:$PYTHONPATH
export PATH=~/local/bin:$PATH

Now I never have to activate anything, I just happily use my local install without worrying about making a mess in my root. If I need to have different versions of things, I create a separate virtualenv install and activate that whenever I need it as mentioned before.

It’s nice to have your very own contained egg basket. This is especially useful for deploying production code on servers. Production environments often require very specific versions of packages, so deploying multiple Python apps on the same server is made much easier with virtualenv.

Update: Another good idea is to use --no-site-packages with virtualenv, this way the root PYTHONPATH can be omitted and all of your package dependencies will live entirely in the virtual environment. (Thanks, Chris! Check out his Virtualenv and PasteScript screencast for more on the topic.)

IPython caveat!

For those of you who have discovered the incredible life-changing tool of never-ending code-gasms that is IPython, there’s one extra step you’ll need to perform: It turns out that IPython saves some hard-coded paths when it’s installed, so you’ll need to re-install it with your new virtualenv active before it will work properly.

5 comments