treeherder/bin
Ed Morley deb1fc86b7 Bug 1384518 - Switch from Memcached to Redis
Since:
* Redis has additional features we need (eg for bug 1409679)
* the Redis server, python client and Django backend are more
  actively maintained (so have persistent connections/pooling that
  actually works, which gives a big perf win)
* we can use Heroku's own Redis addon rather than relying on a
  third-party's (to hopefully prevent a repeat of the certificate
  expiration downtime)

This commit:
* Switches pylibmc/django-pylibmc to redis-py/django-redis, and
  configures the new backend according to:
  http://niwinz.github.io/django-redis/latest/
  https://github.com/andymccurdy/redis-py
* Uses redis-py's native support for TLS to connect to the Heroku
  Redis server's stunnel port directly, avoiding the complexity of
  using a buildpack to create an stunnel between the dyno and server:
  https://devcenter.heroku.com/articles/securing-heroku-redis#connecting-directly-to-stunnel
* Uses explicit `REDIS_URL` values on Travis/Vagrant rather than
  relying on the django-environ `default` value (for parity with
  how we configure `DATABASE_URL` and others).
* Removes the pylibmc connection-closing workaround from `wsgi.py`.

Note: Whilst the Heroku docs suggest using `django-redis-cache`, it's
not as actively maintained, so we're using `django-redis` instead:
https://devcenter.heroku.com/articles/heroku-redis
https://github.com/niwinz/django-redis
https://github.com/sebleier/django-redis-cache

Before this is merged, Heroku Redis instances will need to be created
for stage/production (prototype done) - likely on plan `premium-3`:
https://elements.heroku.com/addons/heroku-redis

We'll also need to pick an eviction policy:
https://devcenter.heroku.com/articles/heroku-redis#maxmemory-policy

Once deployed, the `memcachier-tls-buildpack` buildpack will no longer
be required and can be removed from prototype/stage/prod, along with
the `TREEHERDER_MEMCACHED` environment variable and Memcachier addon.
2018-01-21 14:41:35 +00:00
..
post_compile Bug 1414255 - Fix shellcheck warnings about missing quotes 2017-11-05 20:53:29 +00:00
pre_deploy Bug 1394128 - Heroku: Fail gracefully if revision.txt inaccessible (#2754) 2017-09-06 15:50:15 +01:00
run_celery_worker Bug 1419965 - Remove the estimated job time remaining backend (#2990) 2017-12-04 22:09:38 +00:00
run_celery_worker_buildapi Bug 1414255 - Fix shellcheck warnings about sourced files 2017-11-05 20:53:29 +00:00
run_celery_worker_hp Bug 1414255 - Fix shellcheck warnings about sourced files 2017-11-05 20:53:29 +00:00
run_celery_worker_log_parser Bug 1414255 - Fix shellcheck warnings about sourced files 2017-11-05 20:53:29 +00:00
run_celery_worker_pushlog Bug 1414255 - Fix shellcheck warnings about sourced files 2017-11-05 20:53:29 +00:00
run_celerybeat Bug 1414255 - Fix shellcheck warnings about sourced files 2017-11-05 20:53:29 +00:00
run_gunicorn Bug 1414255 - Fix shellcheck warnings about sourced files 2017-11-05 20:53:29 +00:00
run_read_pulse_jobs Bug 1414255 - Fix shellcheck warnings about sourced files 2017-11-05 20:53:29 +00:00
run_read_pulse_pushes Bug 1414255 - Fix shellcheck warnings about sourced files 2017-11-05 20:53:29 +00:00
travis-setup.sh Bug 1384518 - Switch from Memcached to Redis 2018-01-21 14:41:35 +00:00