From ca1fc1a3432111dd58d151e921633a49e6439d11 Mon Sep 17 00:00:00 2001 From: Stuart Colville Date: Fri, 2 Sep 2016 18:18:45 +0100 Subject: [PATCH] Add docs for docker for mac + windows and change port config --- .gitignore | 1 + docker-compose.override.yml.tmpl | 7 + docker-compose.yml | 4 - docs/topics/install/docker-machine.rst | 53 ++++++++ docs/topics/install/docker.rst | 173 ++++++++++++++++++------- docs/topics/install/index.rst | 1 + 6 files changed, 186 insertions(+), 53 deletions(-) create mode 100644 docker-compose.override.yml.tmpl create mode 100644 docs/topics/install/docker-machine.rst diff --git a/.gitignore b/.gitignore index 510f9446be..5e97931598 100644 --- a/.gitignore +++ b/.gitignore @@ -59,3 +59,4 @@ docker/artifacts/* logs/* supervisord.pid .ipython +docker-compose.override.yml diff --git a/docker-compose.override.yml.tmpl b/docker-compose.override.yml.tmpl new file mode 100644 index 0000000000..e5719124a2 --- /dev/null +++ b/docker-compose.override.yml.tmpl @@ -0,0 +1,7 @@ +nginx: + ports: + - "80:80" + +web: + ports: + - "8000:8000" diff --git a/docker-compose.yml b/docker-compose.yml index 22ea91df1a..953864816b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -28,13 +28,9 @@ worker: &worker web: <<: *worker command: supervisord -n -c /code/docker/supervisor.conf - ports: - - "8000:8000" nginx: image: addons/addons-nginx - ports: - - "80:80" volumes: - ./static:/srv/static - ./site-static:/srv/site-static diff --git a/docs/topics/install/docker-machine.rst b/docs/topics/install/docker-machine.rst new file mode 100644 index 0000000000..fe480010a7 --- /dev/null +++ b/docs/topics/install/docker-machine.rst @@ -0,0 +1,53 @@ +================================ +Installation with Docker machine +================================ + + +.. _creating-the-docker-vm: + +Creating the docker-machine vm +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Your first step is to create a vm - this step assumes we're using +virtualbox as the driver:: + + docker-machine create --driver=virtualbox addons-dev + +Then you can export the variables so that docker-compose can talk to +the docker service. This command will tell you how to do that:: + + docker-machine env addons-dev + +On a mac it's a case of running:: + + eval $(docker-machine env addons-dev) + +Now you have the vm running you can follow the standard docker +instructions: :ref:install-with-docker + +.. _accessing-the-web-server-docker-machine: + +Accessing the web-server with docker-machine +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you're using docker-machine, you can get the ip like so:: + + docker-machine ip addons-dev + +.. note:: + If you're still using boot2docker then the command is `boot2docker ip`. + At this point you can look at installing docker-toolbox and migrating + your old boot2docker vm across to running via docker-machine. See + docker-toolbox_ for more info. + +Now you can connect to port 80 of that ip address. Here's an example +(your ip might be different):: + + http://192.168.99.100/ + +.. note:: + ``docker-machine`` hands out IP addresses as each VM boots; your IP + addresses can change over time. You can find out which IP is in use using + ``docker-machine ip [machine name]`` + +.. _docker-toolbox: https://www.docker.com/toolbox diff --git a/docs/topics/install/docker.rst b/docs/topics/install/docker.rst index bfa90d841c..40457102cf 100644 --- a/docs/topics/install/docker.rst +++ b/docs/topics/install/docker.rst @@ -1,5 +1,5 @@ ==================== -Install via Docker +Install with Docker ==================== .. _install-with-docker: @@ -10,83 +10,78 @@ development environment. First you'll need to install docker_, please check the information for the installation steps specific to your operating system. -.. note:: - Docker recommends installing docker-toolbox_ if you're on OS X or - windows and that will provide you with the ``docker-machine`` and - ``docker-compose`` (Mac-only). +There are generally two options for running docker depending on the platform +you are running. + * Run docker on the host machine directly (recommended if supported) + * Run docker-machine which will run docker inside a virtual-machine -.. _creating-the-docker-vm: +Historically mac and windows could only run docker via a vm. That has +recently changed with the arrival of docker-for-mac_ and docker-for-windows_. -Creating the docker vm (mac/windows) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +If your platform can run docker directly either on Linux, with docker-for-mac_ +or docker-for-windows_ then this is the easiest way to install docker with the +most minimal set of moving parts. -If you go the docker-machine route your first step is to create a vm:: +If you have problems, due to not meeting the requirenents or you're on windows +and want to keep running docker-machine vms then docker-machine will still +work just fine. See the docs for creating the vm here :ref:`creating-the-docker-vm` - docker-machine create --driver=virtualbox addons-dev - -Then you can export the variables so that docker-compose can talk to -the docker service. This command will tell you how to do that:: - - docker-machine env addons-dev - -On a mac it's a case of running:: - - eval $(docker-machine env addons-dev) +.. note:: if you're on OSX and already have a working docker-machine setup you + can run that and docker-for-mac (*but not docker-for-windows*) side by side. + The only caveat here is that it's recommended that you keep the versions of + docker on the vm and the host in-sync to ensure compatibility when you switch + between them. Setting up the containers ~~~~~~~~~~~~~~~~~~~~~~~~~ +.. note:: + + docker-toolbox, docker-for-mac and docker-for-windows will install ``docker-compose`` + for you. If you're on linux and you need it, you can install it manually with:: + + pip install docker-compose + Next once you have docker up and running follow these steps on your host machine:: + # Checkout the addons-server sourcecode. git clone git://github.com/mozilla/addons-server.git cd addons-server - pip install docker-compose + # Create the docker-compose.override file with the default ports. + cp docker-compose.override.yml{.tmpl,} + # Download the containers docker-compose pull # Can take a while depending on your internet bandwidth. + # Start up the containers docker-compose up -d make initialize_docker # Answer yes, and create your superuser when asked. .. note:: - When using a virtual machine for docker the default behaviour is to mount - your home directory into the docker vm. Your code checkout will need - to be within your home directory so that docker-compose will find the necessary - files. + Generally docker requires the code checkout to exist within your home directory so + that docker can mount the source-code into the container. -Accessing docker -~~~~~~~~~~~~~~~~ +Accessing the web server +~~~~~~~~~~~~~~~~~~~~~~~~ -The last step is to grab the ip of the vm. If you're using docker-machine, -you can get the ip like so:: +By default our docker-compose config exposes the web-server on port 80 of localhost. - docker-machine ip addons-dev +We use olympia.dev as the default hostname to access your container server (e.g. for +Firefox Accounts). To be able access the development environment using http://olympia.dev +you'll need to edit your ``/etc/hosts`` file on your native operating system. +For example:: -.. note:: - If you're still using boot2docker then the command is `boot2docker ip`. - At this point you can look at installing docker-toolbox and migrating - your old boot2docker vm across to running via docker-machine. See - docker-toolbox_ for more info. + [ip-address] olympia.dev -Now you can connect to port 80 of that ip address. Here's an example -(your ip might be different):: - - http://192.168.99.100/ - -.. note:: - ``docker-machine`` hands out IP addresses as each VM boots; your IP - addresses can change over time. You can find out which IP is in use using - ``docker-machine ip [machine name]`` - -You may need to use a reliable hostname to access your container server (e.g. for -Firefox Accounts). You can set one by editing your ``/etc/hosts`` file on your -native operating system. For example:: - - 192.168.99.100 olympia.dev +Typically the ip address is localhost 127.0.0.1 if you're using docker-machine +see :ref:`accessing-the-web-server-docker-machine` for details of how to get the ip of +the docker vm. You can ensure your docker server is configured internally with this host by setting it in the environment and restarting the docker containers, like this:: + docker-compose stop # only needed if running export OLYMPIA_SITE_URL=http://olympia.dev docker-compose up -d @@ -140,6 +135,81 @@ migrations:: Gotchas! ~~~~~~~~ +Here's a list of a few of the issues you might face when using docker. + +Can't access the web server? +---------------------------- + +Check you've created an hosts file entry pointing ``olympia.dev`` to the +relevant ip address. + +Also make sure you've copied ``docker-compose.override.yml.tmpl`` to +``docker-compose.override.yml`` to get the default ports. If you haven't +stop the containers with ``docker-compose stop`` copy the file and restart +with ``docker-compose up -d``. + +Another tip is to use ``docker-compose ps`` to check the status of the +containers. If they are failing to start you should be able to tell here. + +Another way to find out what's wrong is to run ``docker-compose logs``. + +Getting "Programming error [table] doesn't exist"? +-------------------------------------------------- + +Check you've run the ``make initialize_docker`` step. + + +Port collisions (nginx container fails to start) +------------------------------------------------ + +Since by default the docker-compose file exposes the port to the nginx +server that sits in front of the web service on port 80 on your host +you might find it fails to start if you're already running a service on +port 80. + +This problem will manifest itself by the services failing to start, you'll +be able to see the error like so:: + + ERROR: for nginx Cannot start service nginx:..... + ...Error starting userland proxy: Bind for 0.0.0.0:80: unexpected error (Failure EADDRINUSE) + ERROR: Encountered errors while bringing up the project. + +There's a couple of ways to fix it. Simple one is to find out what's running on +port 80 and stop it:: + + sudo lsof -i :80 + +We now specify the ports nginx listens on in the ``docker-compose.override.yml`` +file that you copied from ``docker-compose.override.yml.tmpl`` when going through +the initial setup. The second solution to a port collision is to change the +default port that's bound on the host. + +If you need to change the ports you can do so by changing the defaults in +``docker-compose.override.yml``. + +For example if you want to run nginx on your host and still +access the development environment on port 80 you can change +``docker-compose.override.yml`` to this:: + + nginx: + ports: + - 8880:80 + +Now the container nginx is listening on 8880 on the host. You can now proxy +to the container nginx from the host nginx with the following nginx config:: + + server { + listen 80; + server_name olympia.dev; + location / { + proxy_pass http://olympia.dev:8880; + } + } + + +Persisting changes +------------------ + Please note: any command that would result in files added or modified outside of the ``addons-server`` folder (e.g. modifying pip or npm dependencies) won't persist, and thus won't survive after the @@ -150,6 +220,9 @@ running container exits. via the ``Dockerfile``. Commits to master will result in the Dockerfile being rebuilt on the docker hub. +Restarting docker-machine vms following a reboot +------------------------------------------------ + If you quit docker-machine, or restart your computer, docker-machine will need to start again using:: @@ -176,3 +249,5 @@ to Docker Hub for other developers to use after they pull image changes. .. _docker: https://docs.docker.com/installation/#installation .. _docker-toolbox: https://www.docker.com/toolbox +.. _docker-for-windows: https://docs.docker.com/engine/installation/windows/#/docker-for-windows +.. _docker-for-mac: https://docs.docker.com/engine/installation/mac/#/docker-for-mac diff --git a/docs/topics/install/index.rst b/docs/topics/install/index.rst index 59a2c65e81..a031418540 100644 --- a/docs/topics/install/index.rst +++ b/docs/topics/install/index.rst @@ -8,6 +8,7 @@ The following documentation covers how to install and develop the server using D :maxdepth: 2 docker + docker-machine The following documentation covers how to install the individual components, this documentation is deprecated in favour of using Docker. This documentation