Since:
* Redis has additional features we need (eg for bug 1409679)
* the Redis server, python client and Django backend are more
actively maintained (so have persistent connections/pooling that
actually works, which gives a big perf win)
* we can use Heroku's own Redis addon rather than relying on a
third-party's (to hopefully prevent a repeat of the certificate
expiration downtime)
This commit:
* Switches pylibmc/django-pylibmc to redis-py/django-redis, and
configures the new backend according to:
http://niwinz.github.io/django-redis/latest/https://github.com/andymccurdy/redis-py
* Uses redis-py's native support for TLS to connect to the Heroku
Redis server's stunnel port directly, avoiding the complexity of
using a buildpack to create an stunnel between the dyno and server:
https://devcenter.heroku.com/articles/securing-heroku-redis#connecting-directly-to-stunnel
* Uses explicit `REDIS_URL` values on Travis/Vagrant rather than
relying on the django-environ `default` value (for parity with
how we configure `DATABASE_URL` and others).
* Removes the pylibmc connection-closing workaround from `wsgi.py`.
Note: Whilst the Heroku docs suggest using `django-redis-cache`, it's
not as actively maintained, so we're using `django-redis` instead:
https://devcenter.heroku.com/articles/heroku-redishttps://github.com/niwinz/django-redishttps://github.com/sebleier/django-redis-cache
Before this is merged, Heroku Redis instances will need to be created
for stage/production (prototype done) - likely on plan `premium-3`:
https://elements.heroku.com/addons/heroku-redis
We'll also need to pick an eviction policy:
https://devcenter.heroku.com/articles/heroku-redis#maxmemory-policy
Once deployed, the `memcachier-tls-buildpack` buildpack will no longer
be required and can be removed from prototype/stage/prod, along with
the `TREEHERDER_MEMCACHED` environment variable and Memcachier addon.
Now that we're using MySQL 5.7 server, there is no longer a conflict
between our desired choice of client library (which had to be 5.7 to
avoid security vulnerabilities) and the server version.
The library is still vendored on Heroku for now (in `bin/pre_compile`),
but that can stop too, once we switch to the Heroku-16 stack which is
based on Ubuntu 16.04 (bug 1365567).
Since the third-party PPA is no longer maintained, and doesn't have
Python 2.7.13 which is what is being used on Heroku. Instead we copy
the Heroku Python buildpack installation method for greater parity:
18c404f72d/bin/steps/python
Adds the virtualenv directory to `PATH` in `env.sh`, to save having
to activate the virtualenv (which is about to be removed) or set
`PATH` in each of the wrapper bin scripts.
Ed Morley
Dave Hunt
Ed Morley
Ed Morley