Bug 1146184 - Use virtualenv and peep for stage/prod package management

Prior to this change, on stage/production we didn't use virtualenvs
(unlike dev/the local Vagrant project) and instead pip installed
packages globally (when puppet ran periodically), using requirements
files that are not in the repo.

Now during deployment, a virtualenv is created and then populated using
peep (which uses hashes to verify the contents of packages before pip
installing them). The virtualenv is then made relocatable (as best as it
can, the feature isn't perfect), the lib64 symlinks are made relative,
and then the virtualenv is rsynced to all nodes, along with the source.

The one main remaining limitation of --relocatable is that the bash
activate script will not work on the other nodes - however the wrapper
scripts under treeherder-service/bin/ add venv/bin/ to PATH so using the
activate script is unnecessary for them. This just leaves running
manage.py commands locally on a node, for we can use:
|../venv/bin/python manage.py foo|, an alias or else we can fix up the
activate scripts in a follow-up bug.

deployment/update/update.py

@@ -47,6 +47,25 @@ def pre_update(ctx, ref=settings.UPDATE_REF):
 
 @task
 def update(ctx):
+    # Create/populate a virtualenv that will be rsynced later along with the source.
+    with ctx.lcd(settings.SRC_DIR):
+        activate_script = os.path.join(settings.SRC_DIR, 'venv', 'bin', 'activate_this.py')
+        # Peep doesn't yet cache downloaded files, so we reuse the virtualenv to speed up deploys.
+        if not os.path.exists(activate_script):
+            ctx.local('virtualenv --python=python2.7 venv')
+        # Activate virtualenv.
+        execfile(activate_script, dict(__file__=activate_script))
+        # Install requirements using peep, so hashes are verified.
+        with ctx.lcd(th_service_src):
+            ctx.local('python2.7 bin/peep.py install -r requirements/common.txt')
+            ctx.local('python2.7 bin/peep.py install -r requirements/prod.txt')
+        # Make the virtualenv relocatable since paths are hard-coded by default.
+        ctx.local('virtualenv --relocatable venv')
+        # Fix lib64 symlink to be relative instead of absolute.
+        with ctx.lcd('venv'):
+            ctx.local('rm -f lib64')
+            ctx.local('ln -s lib lib64')
+
     with ctx.lcd(th_service_src):
         # Collect the static files (eg for the Persona or Django admin UI)
         ctx.local("python2.7 manage.py collectstatic --noinput")