Move to Heroku (#27)

* Update Newrelic to get some more features from it * Fix NewRelic app name * Fix path to NewRelic config file * Remove unused PROJECT_ROOT setting * Fix web container to have access to redis and db * Notify NewRelic about deploy Based on Treeherder files as discussed in https://bugzilla.mozilla.org/show_bug.cgi?id=1165229 * Disable user tracking via New Relic * Capture stuff in the RQ workers * Serve revision file from root URL path * Add security middleware Fix #26 * Add IS_HEROKU toggle * Properly set the Heroku specific settings * Use different strategy to set the environment * Ignore security.W009 * No soup for you * Show secret key * Remove duplicate newrelic config * Update newrelic config file * Use Codecov instead of Coveralls * Use URL conf for form URL
2016-10-07 16:03:31 +02:00 · 2016-10-07 16:03:31 +02:00 · 684d1c81cc
--- a/.env-dist
+++ b/.env-dist
@ -0,0 +1,12 @@
+PYTHONUNBUFFERED=1
+PYTHONDONTWRITEBYTECODE=1
+DATABASE_URL=postgres://postgres@db/postgres
+REDIS_URL=redis://redis:6379/0
+DEBUG=True
+PORT=8000
+ALLOWED_HOSTS=localhost,127.0.0.1,
+SECRET_KEY=
+AWS_ACCESS_KEY_ID=
+AWS_SECRET_ACCESS_KEY=
+AWS_DEFAULT_REGION=us-west-2
+SITE_URL=http://localhost:8000
--- a/.travis.yml
+++ b/.travis.yml
@ -5,7 +5,7 @@ python:
 services:
  - redis-server
 install:
-  - pip install tox coveralls
+  - pip install tox codecov
 env:
  - TOX_ENV=flake8
  - TOX_ENV=docs
@ -13,4 +13,4 @@ env:
 script:
  - tox -e $TOX_ENV
 after_success:
-  - coveralls
+  - codecov --env TRAVIS_OS_NAME,TOXENV
--- a/2
+++ b/2
@ -1,7 +1,6 @@
 FROM debian:jessie

 EXPOSE 8000
-CMD ["./bin/run-prod.sh"]

 RUN adduser --uid 1000 --disabled-password --gecos '' --no-create-home webdev

@ -25,6 +24,5 @@ COPY requirements.txt /app/requirements.txt
 RUN pip install --require-hashes --no-cache-dir -r requirements.txt

 COPY . /app
-RUN DEBUG=False SECRET_KEY=foo ALLOWED_HOSTS=localhost, DATABASE_URL= REDIS_URL= ./manage.py collectstatic --noinput -c
 RUN chown webdev.webdev -R .
 USER webdev
--- a/21
+++ b/21
@ -1,9 +1,23 @@
-.PHONY: build clean creds migrate shell static test up
+.PHONY: build clean creds migrate shell static stop test up
+
+help:
+	@echo "Welcome to the Telemetry Analysis Service\n"
+	@echo "The list of commands for local development:\n"
+	@echo "  build    Builds the docker images for the docker-compose setup"
+	@echo "  clean    Stops and removes all docker images and purge the collected static files"
+	@echo "  creds CLIENT_ID=<CLIENT_ID> CLIENT_SECRET=<CLIENT_SECRET>"
+	@echo "           Sets the Google Credentials required for authentication"
+	@echo "  migrate  Runs the Django database migrations"
+	@echo "  shell    Opens a Bash shell"
+	@echo "  static   Collects static files (only needed in rare circumstances such as DEBUG=False)"
+	@echo "  test     Runs the Python test suite"
+	@echo "  up       Runs the whole stack, served under http://localhost:8000/\n"
+	@echo "  stop     Stops the docker containers"

 build:
 	docker-compose build

-clean:
+clean: stop
 	docker-compose rm -f
 	rm -rf static/

@ -21,6 +35,9 @@ static:
 	# this is only necessary after adding/removing/editing static files
 	docker-compose run web ./manage.py collectstatic --noinput

+stop:
+	docker-compose stop
+
 test: static
 	docker-compose run web ./manage.py test

--- a/8
+++ b/8
@ -1 +1,7 @@
-web: ./bin/run-prod.sh
+web: newrelic-admin run-program gunicorn atmo.wsgi:application --workers 4 --log-file -
+worker: newrelic-admin run-program python manage.py rqworker default
+# django-rq doesn't support rqscheduler retry mode yet
+# so we need to use the original startup script
+scheduler: newrelic-admin run-program rqscheduler --url=$REDIS_URL
+
+release: ./bin/pre_deploy
--- a/README.md
+++ b/README.md
@ -3,7 +3,7 @@ atmo - The code for the Telemetry Analysis Service

 [![Build Status](https://travis-ci.org/mozilla/telemetry-analysis-service.svg?branch=master)](https://travis-ci.org/mozilla/telemetry-analysis-service)

-[![Coverage status](https://img.shields.io/coveralls/mozilla/telemetry-analysis-service/master.svg)](https://coveralls.io/r/mozilla/telemetry-analysis-service)
+[![codecov](https://codecov.io/gh/mozilla/telemetry-analysis-service/branch/master/graph/badge.svg)](https://codecov.io/gh/mozilla/telemetry-analysis-service)

 Run the tests
 -------------
@ -19,23 +19,28 @@ they are run by [travis](https://travis-ci.org)):
    pip install tox
    tox

-The `.travis.yml` file will also run [coveralls](https://coveralls.io) by
+The `.travis.yml` file will also run [codecov](https://codecov.io) by
 default.

-If you want to benefit from Travis and Coveralls, you will need to activate
+If you want to benefit from Travis and Codecov, you will need to activate
 them both for your project.

 Oh, and you might want to change the "Build Status" and "Coverage Status" links
-at the top of this file to point to your own travis and coveralls accounts.
+at the top of this file to point to your own travis and codecov accounts.

 Development Setup
 -----------------

-This application is packaged with Docker, which manages and maintains a consistent application environment.
+This application uses Docker for local development. Please make sure to
+[install Docker](https://docs.docker.com/mac/) and
+[Docker Compose](https://docs.docker.com/compose/install/).

-On a Debian-derived Linux distributions, run `./bin/build-deb.sh` to perform all
-the installation steps automatically. On other OSs, [install Docker](https://docs.docker.com/mac/) and
-[Docker Compose](https://docs.docker.com/compose/install/) manually.
+To set the application up, please copy the `.env-dist` file to one named `.env`
+and then update the variables starting with `AWS_` with the appropriate.
+
+Set the `SECRET_KEY` variable using the output of the following command:
+
+    python -c "from django.utils.crypto import get_random_string; print(get_random_string(50))"

 To start the application, run `make up`.

@ -44,21 +49,28 @@ Quick troubleshooting guide:
 * Docker gives an error message similar to `ERROR: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running?`
    * Run the command as administrator/superuser (for testing purposes, that is).
    * Make sure the user is in the `docker` group (use the `sudo usermod -aG docker ${USER}` command to do this). This allows the user to use Docker without superuser privileges. Note that this does not take effect until the user logs out and logs in again.
+
 * Docker-Compose gives an error message similar to `ERROR: client and server don't have same version (client : 1.21, server: 1.18)`
    * Make sure to install the latest versions of both Docker and Docker-Compose. The current versions of these in the Debian repositories might not be mutually compatible.
+
 * Docker gives an error message similar to `Err http://security.debian.org jessie/updates InRelease`
    * The installed Docker version is possibly too old. Make sure to use the latest available stable version.
    * Ensure that the DNS configuration is sane: see if `docker-compose run web ping security.debian.org` can connect successfully.
+
 * Django gives an error message similar to `OperationalError: SOME_TABLE doesn't exist`
    * The database likely isn't set up correctly.
    * Run `make migrate` to update it.
+
 * Django gives some other form of `OperationalError`, and we don't really care about the data that's already in the database (e.g., while developing or testing)
    * Most database issues can be resolved by just deleting the database, `telemetry_analysis.db`. It will be recreated on the next run.
+
 * Database errors are usually caused by an improper database configuration. For development purposes, recreating the database will often solve the issue.
+
 * Django gives an error message similar to `'NoneType' object has no attribute 'get_frozen_credentials'`.
    * The AWS credentials on the current machine are likely not correctly set.
-    * Set them in your **ENVIRONMENT VARIABLES** (these environment variables are transferred to the docker container, from definitions in `docker-compose.yml`).
+    * Set them in your **ENVIRONMENT VARIABLES** (these environment variables are transferred to the docker container, from definitions in `.env`).
    * See the [relevant section of the Boto3 docs](https://boto3.readthedocs.org/en/latest/guide/configuration.html#environment-variables) for more details.
+
 * Django raises a 404 when trying to login
    * Google Developer credentials are needed to get the Google authentication
      workflow running.
@ -73,21 +85,12 @@ Quick troubleshooting guide:
      to the django-allauth config system:
        * `CLIENT_ID=<client-id> CLIENT_SECRET=<client-secret> make creds`

-Production Setup
----------------
-
-1. Add your project in [Docker Registry](https://registry.hub.docker.com/) as [Automated Build](http://docs.docker.com/docker-hub/builds/)
-2. Prepare a 'env' file with all the variables needed by dev, stage or production.
-3. Run the image:
-
-    docker run --env-file env -p 80:8000 mozilla/atmo
-
 Heroku Setup
 ------------
-1. heroku create
-2. heroku config:set DEBUG=False ALLOWED_HOSTS=<foobar>.herokuapp.com, SECRET_KEY=something_secret
-   DATABASE_URL gets populated by heroku once you setup a database.
-3. git push heroku master
+1. `heroku create`
+2. `heroku config:set DEBUG=False ALLOWED_HOSTS=<foobar>.herokuapp.com, SECRET_KEY=something_secret`
+   `DATABASE_URL` gets populated by heroku once you setup a database.
+3. push to GitHub with `git push origin`

 NewRelic Monitoring
 -------------------
--- a/atmo/settings.py
+++ b/atmo/settings.py
@ -7,7 +7,7 @@ https://docs.djangoproject.com/en/1.9/topics/settings/
 For the full list of settings and their values, see
 https://docs.djangoproject.com/en/1.9/ref/settings/
 """
-
+from datetime import timedelta
 import os

 import dj_database_url
@ -15,8 +15,7 @@ from django.core.urlresolvers import reverse_lazy
 from decouple import Csv, config

 # Build paths inside the project like this: os.path.join(BASE_DIR, ...)
-BASE_DIR = os.path.dirname(os.path.dirname(__file__))
-ROOT = os.path.dirname(os.path.join(BASE_DIR, '..'))
+BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

 # Quick-start development settings - unsuitable for production
 # See https://docs.djangoproject.com/en/1.9/howto/deployment/checklist/
@ -29,6 +28,11 @@ DEBUG = config('DEBUG', cast=bool)

 ALLOWED_HOSTS = config('ALLOWED_HOSTS', cast=Csv())

+SITE_ID = 1
+
+# The URL under which this instance is running
+SITE_URL = config('SITE_URL', default='http://localhost:8000')
+
 # Application definition

 INSTALLED_APPS = [
@ -57,6 +61,7 @@ INSTALLED_APPS = [
 ]

 MIDDLEWARE_CLASSES = (
+    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
@ -144,10 +149,8 @@ LOGIN_REDIRECT_URL = reverse_lazy('dashboard')

 # django-allauth configuration
 ACCOUNT_LOGOUT_REDIRECT_URL = LOGIN_REDIRECT_URL
-if not DEBUG:
-    ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
 ACCOUNT_EMAIL_CONFIRMATION_EXPIRE_DAYS = 7
-ACCOUNT_EMAIL_SUBJECT_PREFIX = '[ATMO] '
+ACCOUNT_EMAIL_SUBJECT_PREFIX = '[Telemetry Analysis Service] '
 ACCOUNT_EMAIL_REQUIRED = True
 ACCOUNT_EMAIL_VERIFICATION = 'optional'
 ACCOUNT_LOGOUT_ON_GET = True
@ -182,13 +185,32 @@ STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'
 MEDIA_ROOT = config('MEDIA_ROOT', default=os.path.join(BASE_DIR, 'media'))
 MEDIA_URL = config('MEDIA_URL', '/media/')

-SESSION_COOKIE_SECURE = config('SESSION_COOKIE_SECURE', default=not DEBUG, cast=bool)
 SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
 SESSION_CACHE_ALIAS = 'default'

-SECURE_SSL_REDIRECT = True
+SECURE_BROWSER_XSS_FILTER = True
+SECURE_CONTENT_TYPE_NOSNIFF = True
+SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
+X_FRAME_OPTIONS = 'DENY'
+
+if SITE_URL.startswith('https://'):
+    ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
+    SECURE_SSL_REDIRECT = True
+    SECURE_HSTS_SECONDS = int(timedelta(days=365).total_seconds())
+    # Mark session and CSRF cookies as being HTTPS-only.
+    CSRF_COOKIE_SECURE = True
+    SESSION_COOKIE_SECURE = True
+
+
+SILENCED_SYSTEM_CHECKS = [
+    'security.W003',  # We're using django-session-csrf
+    # We can't set SECURE_HSTS_INCLUDE_SUBDOMAINS since this runs under a
+    # mozilla.org subdomain
+    'security.W005',
+    'security.W009',  # we know the SECRET_KEY is strong
+]
+

-PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
@ -252,5 +274,3 @@ CSP_STYLE_SRC = (

 # This is needed to get a CRSF token in /admin
 ANON_ALWAYS = True
-
-SITE_ID = 1
--- a/atmo/templates/atmo/detail-cluster.html
+++ b/atmo/templates/atmo/detail-cluster.html
@ -30,9 +30,9 @@
    <div class="col-md-10">{{cluster.end_date}}</div>
  </div>
  <p>
-    <form action="/clusters/delete/" method="POST" enctype="multipart/form-data">
+    <form action="{% url 'clusters-delete' %}" method="POST" enctype="multipart/form-data">
      {% csrf_token %}
-      <input name="cluster" value="{{cluster.id}}" type="hidden">
+      <input name="cluster" value="{{ cluster.id }}" type="hidden">
      <button type="submit" class="btn btn-sm btn-danger" title="Terminate Spark cluster">
        <span class="glyphicon glyphicon-stop" aria-hidden="true"></span>
        Terminate now
--- a/atmo/urls.py
+++ b/atmo/urls.py
@ -15,7 +15,8 @@ urlpatterns = [
    url(r'workers/', include('atmo.workers.urls')),

    # contribute.json url
-    url(r'^(?P<path>contribute\.json)$', static.serve, {'document_root': settings.ROOT}),
+    url(r'^(?P<path>contribute\.json)$', static.serve, {'document_root': settings.BASE_DIR}),
+    url(r'^(?P<path>revision\.txt)$', static.serve, {'document_root': settings.BASE_DIR}),

    url(r'^accounts/', include('allauth.urls')),
 ]
--- a/atmo/wsgi.py
+++ b/atmo/wsgi.py
@ -11,15 +11,4 @@ os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'atmo.settings')  # NOQA

 from django.core.wsgi import get_wsgi_application

-import newrelic.agent
-from decouple import config
-
-
 application = get_wsgi_application()
-
-# Add NewRelic
-newrelic_ini = config('NEW_RELIC_CONFIG_FILE', default='newrelic.ini')
-newrelic_license_key = config('NEW_RELIC_LICENSE_KEY', default=None)
-if newrelic_ini and newrelic_license_key:
-    newrelic.agent.initialize(newrelic_ini)
-    application = newrelic.agent.wsgi_application()(application)
--- a/bin/build-deb.sh
+++ b/bin/build-deb.sh
@ -1,16 +0,0 @@
-#!/usr/bin/env bash
-set -e
-
-DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
-
-# the Docker and Docker Compose packages in the repositories are too old to work with this project - install the latest versions instead
-sudo apt-get install --yes -qq curl
-sudo apt-get purge --yes -qq lxc-docker docker.io docker-compose # remove old packages that can cause conflicts
-curl -fsSL https://get.docker.com/ | sh # install Docker
-curl -L https://github.com/docker/compose/releases/download/1.6.0/docker-compose-`uname -s`-`uname -m` | sudo tee /usr/local/bin/docker-compose > /dev/null
-sudo chmod +x /usr/local/bin/docker-compose # install Docker Compose
-sudo usermod -aG docker ${USER} # add the current user to the Docker group (required in order to run the Docker daemon without sudo)
-
-# we still need sudo even though we've been added to the Docker group here, since the group change only takes effect after logging out and logging in again
-sudo docker-compose run web ${DIR}/../manage.py collectstatic # collect all the static files to /app/static, required by the tests
-sudo docker-compose run web ${DIR}/../manage.py test # run the test suite real quick
--- a/bin/post_compile
+++ b/bin/post_compile
@ -0,0 +1,15 @@
+#!/bin/bash -e
+# Tasks run by the Heroku Python buildpack after the compile step.
+
+# Make the current Git revision accessible at <site-root>/revision.txt
+echo $SOURCE_VERSION > revision.txt
+
+# The post_compile script is run in a sub-shell, so we need to source the
+# buildpack's utils script again, so we can use set-env/set-default-env:
+# https://github.com/heroku/heroku-buildpack-python/blob/master/bin/utils
+source $BIN_DIR/utils
+
+# Override the hostname that is displayed in New Relic, so the Dyno name
+# (eg "web.1") is shown, rather than "Dynamic Hostname". $DYNO is quoted
+# so that it's expanded at runtime on each dyno, rather than now.
+set-env NEW_RELIC_PROCESS_HOST_DISPLAY_NAME '$DYNO'
--- a/bin/pre_deploy
+++ b/bin/pre_deploy
@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+# Tasks run after the Heroku buildpack compile, but prior to the deploy.
+# Failures will block the deploy unless `IGNORE_PREDEPLOY_ERRORS` is set.
+
+echo "$SECRET_KEY"
+
+if [[ -v SKIP_PREDEPLOY ]]; then
+    echo "-----> PRE-DEPLOY: Warning: Skipping pre-deploy!"
+    exit 0
+fi
+
+if [[ -v IGNORE_PREDEPLOY_ERRORS ]]; then
+    echo "-----> PRE-DEPLOY: Warning: Ignoring errors during pre-deploy!"
+else
+    # Make non-zero exit codes & other errors fatal.
+    set -euo pipefail
+fi
+
+echo "-----> PRE-DEPLOY: Running Django migration..."
+newrelic-admin run-program ./manage.py migrate --noinput
+
+echo "-----> PRE-DEPLOY: Reporting deployment to New Relic..."
+# eg: "v750: Deploy 5d6b1f0"
+DESCRIPTION="$HEROKU_RELEASE_VERSION: $HEROKU_SLUG_DESCRIPTION"
+# Use the revision from the live site rather than a local file generated during
+# buildpack compile, so that in the case of deploy failures it's up to date.
+OLD_REVISION="$(curl --silent --show-error --fail --retry 5 --retry-max-time 15 $SITE_URL/revision.txt)"
+CHANGELOG="https://github.com/mozilla/telemetry-analysis-service/compare/$OLD_REVISION...$HEROKU_SLUG_COMMIT"
+# The author of the deploy isn't currently available to us. Have filed:
+# https://help.heroku.com/tickets/343783
+USER="Heroku"
+# Report the deploy to New Relic using their Python agent. In addition to
+# the passed arguments, record-deploy references the environment variables
+# `NEW_RELIC_APP_NAME` and `NEW_RELIC_API_KEY`.
+newrelic-admin record-deploy "$NEW_RELIC_CONFIG_FILE" \
+                             "$DESCRIPTION" \
+                             "$HEROKU_SLUG_COMMIT" \
+                             "$CHANGELOG" \
+                             "$USER"
+
+echo "-----> PRE-DEPLOY: Complete!"
--- a/bin/run-common.sh
+++ b/bin/run-common.sh
@ -1,3 +0,0 @@
-#!/bin/bash
-
-./manage.py migrate
--- a/bin/run-dev.sh
+++ b/bin/run-dev.sh
@ -1,4 +0,0 @@
-#!/bin/sh
-
-./bin/run-common.sh
-./manage.py runserver 0.0.0.0:8000
--- a/bin/run-prod.sh
+++ b/bin/run-prod.sh
@ -1,4 +0,0 @@
-#!/bin/sh
-
-./bin/run-common.sh
-gunicorn atmo.wsgi:application -b 0.0.0.0:${PORT:-8000} --log-file -
--- a/circle.yml
+++ b/circle.yml
@ -1,59 +0,0 @@
-# These environment variables must be set in CircleCI UI
-#
-# DOCKERHUB_REPO - docker hub repo, format: <username>/<repo>
-# DOCKER_EMAIL   - login info for docker hub
-# DOCKER_USER
-# DOCKER_PASS
-#
-machine:
-  services:
-    - docker
-
-dependencies:
-  pre:
-    - pip install tox
-  override:
-    - docker info
-
-    # create a version.json
-    - >
-        printf '{"commit":"%s","version":"%s","source":"https://github.com/%s/%s","build":"%s"}\n'
-        "$CIRCLE_SHA1"
-        "$CIRCLE_TAG"
-        "$CIRCLE_PROJECT_USERNAME"
-        "$CIRCLE_PROJECT_REPONAME"
-        "$CIRCLE_BUILD_URL"
-        > version.json
-
-    - docker build -t app:build .
-
-    # write the sha256 sum to an artifact to make image verification easier
-    - docker inspect --format '{{json .Config.Image}}' app:build | tr -d '"''"' | tee $CIRCLE_ARTIFACTS/docker-image-shasum256.txt
-
-test:
-  override:
-    - tox -e flake8
-    - tox -e docs
-    - tox -e tests
-
-
-# appropriately tag and push the container to dockerhub
-deployment:
-  hub_latest:
-    branch: "master"
-    commands:
-      - "[ ! -z $DOCKERHUB_REPO ]"
-      - docker login -e $DOCKER_EMAIL -u $DOCKER_USER -p $DOCKER_PASS
-      - "docker tag app:build ${DOCKERHUB_REPO}:latest"
-      - "docker push ${DOCKERHUB_REPO}:latest"
-
-  hub_releases:
-    # push all tags
-    tag: /.*/
-    commands:
-      - "[ ! -z $DOCKERHUB_REPO ]"
-      - docker login -e $DOCKER_EMAIL -u $DOCKER_USER -p $DOCKER_PASS
-      - "echo ${DOCKERHUB_REPO}:${CIRCLE_TAG}"
-      - "docker tag app:build ${DOCKERHUB_REPO}:${CIRCLE_TAG}"
-      - "docker images"
-      - "docker push ${DOCKERHUB_REPO}:${CIRCLE_TAG}"
--- a/docker-compose-base.yml
+++ b/docker-compose-base.yml
@ -1,14 +0,0 @@
-python_base:
-  build: .
-  volumes:
-    - .:/app
-  environment:
-    - DATABASE_URL=postgres://postgres@db/postgres
-    - REDIS_URL=redis://redis:6379/0
-    - PYTHONDONTWRITEBYTECODE=1
-    - DEBUG=True
-    - ALLOWED_HOSTS=localhost,127.0.0.1,
-    - SECRET_KEY=59114b6a-2858-4caf-8878-482a24ee9542
-    - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
-    - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
-    - AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -1,36 +1,43 @@
-db:
-  image: postgres:9.4
-redis:
-  image: redis:3.2
-web:
-  extends:
-    file: docker-compose-base.yml
-    service: python_base
-  ports:
-    - "8000:8000"
-  links:
-    - db
-    - redis
-  command:
-    ./bin/run-dev.sh
-rq_worker:
-  extends:
-    file: docker-compose-base.yml
-    service: python_base
-  links:
-    - db
-    - redis
-  command:
-    python manage.py rqworker default
-rq_scheduler:
-  extends:
-    file: docker-compose-base.yml
-    service: python_base
-  links:
-    - db
-    - redis
+version: '2'
+services:
+
+  db:
+    image: postgres:9.4
+
+  redis:
+    image: redis:3.2
+
+  app:
+    build: .
+    volumes:
+      - .:/app
+
+  web:
+    extends:
+      service: app
+    ports:
+      - "8000:8000"
+    depends_on:
+      - app
+    links:
+      - db
+      - redis
+    command: bash -c "python manage.py migrate && python manage.py runserver 0.0.0.0:${PORT}"
+
+  worker:
+    extends:
+      service: app
+    links:
+      - db
+      - redis
+    command: python manage.py rqworker default
+
+  scheduler:
+    extends:
+      service: app
+    links:
+      - db
+      - redis
    # django-rq doesn't support rqscheduler `retry` mode yet
-    # so we need to use the original startup script, which retrieves
-    # the
-  command:
-    rqscheduler --url=redis://redis:6379
+    # so we need to use the original startup script
+    command: rqscheduler --url=${REDIS_URL}
--- a/newrelic.ini
+++ b/newrelic.ini
@ -29,7 +29,7 @@
 # account in the New Relic service.
 # license_key = use NEW_RELIC_LICENSE_KEY environment variable

-# The appplication name. Set this to be the name of your
+# The application name. Set this to be the name of your
 # application as you would like it to show up in New Relic UI.
 # The UI will then auto-map instances of your application into a
 # entry on your home dashboard page.
@ -95,7 +95,7 @@ high_security = true
 # proxy. The "proxy_user" and "proxy_pass" settings should
 # additionally be set if proxy authentication is implemented by
 # the HTTP proxy. The "proxy_scheme" setting dictates what
-# protocol scheme is used in talking to the HTTP protocol. This
+# protocol scheme is used in talking to the HTTP proxy. This
 # would normally always be set as "http" which will result in the
 # agent then using a SSL tunnel through the HTTP proxy for end to
 # end encryption.
@ -105,17 +105,13 @@ high_security = true
 # proxy_user =
 # proxy_pass =

-# Tells the transaction tracer and error collector (when
-# enabled) whether or not to capture the query string for the
-# URL and send it as the request parameters for display in the
-# UI. When "true", it is still possible to exclude specific
-# values from being captured using the "ignored_params" setting.
-capture_params = false
-
-# Space separated list of variables that should be removed from
-# the query string captured for display as the request
-# parameters in the UI.
-ignored_params =
+# Capturing request parameters is off by default. To enable the
+# capturing of request parameters, first ensure that the setting
+# "attributes.enabled" is set to "true" (the default value), and
+# then add "request.parameters.*" to the "attributes.include"
+# setting. For details about attributes configuration, please
+# consult the documentation.
+# attributes.include = request.parameters.*

 # The transaction tracer captures deep information about slow
 # transactions and sends this to the UI on a periodic basis. The
@ -175,7 +171,7 @@ error_collector.ignore_errors =
 # For those Python web frameworks that are supported, this
 # setting enables the auto-insertion of the browser monitoring
 # JavaScript fragments.
-browser_monitoring.auto_instrument = true
+browser_monitoring.auto_instrument = false

 # A thread profiling session can be scheduled via the UI when
 # this option is enabled. The thread profiler will periodically
@ -201,10 +197,10 @@ monitor_mode = false
 monitor_mode = false

 [newrelic:staging]
-app_name = Python Application (Staging)
 monitor_mode = true

 [newrelic:production]
 monitor_mode = true

 # ---------------------------------------------------------------------------
+
--- a/requirements.txt
+++ b/requirements.txt
@ -37,8 +37,8 @@ pytz==2015.7 \
    --hash=sha256:fbd26746772c24cb93c8b97cbdad5cb9e46c86bbdb1b9d8a743ee00e2fb1fc5d \
    --hash=sha256:99266ef30a37e43932deec2b7ca73e83c8dbc3b9ff703ec73eca6b1dae6befea \
    --hash=sha256:8b6ce1c993909783bc96e0b4f34ea223bff7a4df2c90bdb9c4e0f1ac928689e3
-newrelic==2.60.0.46 \
-    --hash=sha256:17d598fccca0845c0337e5276d40d65f951afea6adf9bd9172cff3a87f2ef294
+newrelic==2.70.0.51 \
+    --hash=sha256:54e1da319d4670b4887b463309907cbefeff847a26e378609d04126a14d01b13
 django-session-csrf==0.7.0 \
    --hash=sha256:98670d093c86e8f1e64a2197dda84c7aa1ff32acacb559e18c3104b56c4a0804 \
    --hash=sha256:65791aca2c8c75b59c1e903e5bdd64d0e4fac7799367738ef96a0170ae2fd7e4
--- a/setup.py
+++ b/setup.py
@ -1,10 +1,9 @@
-#!/usr/bin/env python
-
-from distutils.core import setup
+from setuptools import setup

 setup(
    name='atmo',
-    version='0.1dev',
+    use_scm_version=True,
+    setup_requires=['setuptools_scm'],
    description='This is https://github.com/mozilla/telemetry-analysis-service',
    author='Mozilla Foundation',
    author_email='telemetry-analysis-service@mozilla.org',
--- a/tox.ini
+++ b/tox.ini
@ -5,9 +5,9 @@ envlist = tests, flake8, docs
 basepython = python2.7
 setenv =
    DEBUG=False
-    SECRET_KEY='FOO'
+    SECRET_KEY=bd2b6169a99f2561168eb6b39c3fe9ad4b184a2e485781a3c875a1d1052d469d
    ALLOWED_HOSTS=localhost
-    DATABASE_URL=sqlite:///tmp/test.db
+    DATABASE_URL=sqlite://:memory:
    REDIS_URL=redis://localhost:6379

 [testenv:tests]