Most of my work is creating websites in Django (a Python web framework) and depl

Question

Most of my work is creating websites in Django (a Python web framework) and deploying them to my own or clients' servers. I work from a virtualenv to separate site from system packages and have perhaps 60-80 packages installed in there and that lot is shared between two-dozen sites.

This obvious limitation to this approach is needing to test every site if I upgrade a package it uses. I consider that a fair trade-off for not needing to keep on top of umpteen separate virtualenvs.

And that is essentially my whole problem. How on earth are you supposed to keep on top of virtualenv deployments? People just seem to treat them like a dumping ground but if the programming universe has learnt anything this past week from the Ruby on Rails explosion, using old versions of software is unacceptable.

I have a simple script that attempts to check for current package versions with the latest pipcounterpart but it's quite inacurrate. It also doesn't differentiate between security upgrades and feature upgrades (which require days of testing and fixing).

I am looking for something better. I am looking for something that can let me know if Django has a new security release out, or if something is end-of-life. I want something to help me (and other Python devops) not become the next batch of people crying after a wave of kids with scanners and scripts convert our servers into a botnet.

Does such a thing exist?

Explanation / Answer

We use buildout to create deployments that are reproduceable and isolated.

A buildout acts like a virtualenv in that it lets you install eggs, keeping them separate from your Python installation, and it acts like pip in that you can control what versions are used. But buildout is also a build tool, with recipes you can make it build arbitrary parts as needed to fit your application and testing.

My virtual env does not contain anything beyond the zc.buildout egg to create new bootstraps for new projects.

A typical buildout I build has several config files, to fit development, continuous integration, and production environments, but the base.cfg invariably looks something like:

[buildout]
extends =
versions.cfg

newest = false

eggs = customer_project_name

[django-base]
recipe = djangorecipe
settings = settings
eggs = ${buildout:eggs}
project = customer_project_name
projectegg = ${:project}

[python]
recipe = zc.recipe.egg
interpreter = python
eggs = ${buildout:eggs}
initialization =
import os
os.environ['DJANGO_SETTINGS_MODULE'] = '${django:projectegg}.settings'
with a versions.cfg to pin all versions to prevent accidental upgrades.

Because we use a versions.cfg file, it is easy to parse all versions files to see what projects are affected (it's just a ConfigParser file, after all), and upgrading the package is just a question of updating those pins and re-testing the project.

Once tested, push the updated versions updates to the customer server, rerun buildout.

Note that upgrading security fixes should be a dedicated maintenance task anyway, as you need to plan, test and find a upgrade window for your customers.

Related Questions

Navigate

• Browse (All)
• Browse M
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.