2017-08-09 20:19:36 +03:00
# ![Logo](https://raw.githubusercontent.com/mozilla/experimenter/master/app/experimenter/static/imgs/favicon_24.png) Experimenter
2017-04-28 00:24:36 +03:00
[![CircleCI ](https://circleci.com/gh/mozilla/experimenter.svg?style=svg )](https://circleci.com/gh/mozilla/experimenter)
2017-08-09 20:19:36 +03:00
< p align = "center" >
< img src = "https://cdn1.iconfinder.com/data/icons/simple-arrow/512/arrow_20-128.png" > < br / >
< b > 1. Design 2. Launch 3. Analyze< / b >
< br > < br >
< / p >
Experimenter is a platform for managing experiments in [Mozilla Firefox ](https://www.mozilla.org/en-US/firefox/?utm_medium=referral&utm_source=firefox-com ).
## What is an experiment?
An experiment is a way to measure how a change to your product affects how people use it.
An experiment has three parts:
1. A new feature that can be selectively enabled
1. A group of users to test the new feature
1. Telemetry to measure how people interact with the new feature
## How do I run an experiment?
< p align = "center" >
< img src = "https://raw.githubusercontent.com/mozilla/experimenter/164/app/experimenter/static/imgs/architecture.png" > < br / >
< / p >
2017-04-11 22:45:02 +03:00
2017-08-09 20:19:36 +03:00
1. Build a new feature behind a pref flag
1. Define an experiment for that feature in Experimenter
1. Send it to Shield
1. After Shield reviews and approves it, it is sent to Firefox
1. Firefox clients check whether they should enroll in the experiment and configure themselves accordingly
1. Telemetry about the experiment is collected
1. Dashboards are created to visualize the telemetry
1. Analyze and collect the results to understand how the new feature impacted users
1. Do it again!
2017-04-11 22:45:02 +03:00
## Installation
2017-08-09 20:19:36 +03:00
1. Install [docker ](https://www.docker.com/ ) on your machine
2017-04-11 22:45:02 +03:00
2017-08-09 20:19:36 +03:00
1. Clone the repo
2017-04-11 22:45:02 +03:00
git clone < your fork >
2017-08-09 20:19:36 +03:00
1. Copy the sample env file
2017-04-11 22:45:02 +03:00
cp .env.sample .env
2017-08-09 20:19:36 +03:00
1. Create a new secret key and put it in .env
2017-04-11 22:45:02 +03:00
make secretkey
2017-08-09 20:19:36 +03:00
1. Run tests
2017-04-11 22:45:02 +03:00
make test
2017-08-09 20:19:36 +03:00
1. Run database migrations
2017-05-04 23:32:01 +03:00
make migrate
2017-08-09 20:19:36 +03:00
1. Make a local user
2017-05-04 23:32:01 +03:00
make createuser
2017-08-09 20:19:36 +03:00
1. Run a dev instance
2017-05-04 23:32:01 +03:00
make up
2017-04-11 22:45:02 +03:00
Done!
## Usage
Experimenter uses [docker ](https://www.docker.com/ ) for all development, testing, and deployment.
The following helpful commands have been provided via a Makefile:
### build
Build the application container by executing the [build script ](https://github.com/mozilla/experimenter/blob/master/scripts/build.sh )
### compose_build
Build the supporting services (nginx, postgresql) defined in the [compose file ](https://github.com/mozilla/experimenter/blob/master/docker-compose.yml )
### up
Start a dev server listening on port 80 using the [Django runserver ](https://docs.djangoproject.com/en/1.10/ref/django-admin/#runserver )
### test
Run the Django test suite with code coverage
### lint
Run flake8 against the code
### check
Run both test and lint
### migrate
Apply all django migrations
2017-05-04 23:32:01 +03:00
### createuser
Create an admin user in the local dev instance
2017-04-11 22:45:02 +03:00
### shell
Start an ipython shell inside the container (this lets you import and test code, interact with the db, etc)
### bash
Start a bash shell inside the container (this lets you interact with the containerized filesystem)
2017-05-04 23:32:01 +03:00
## API
2017-08-02 22:10:16 +03:00
### GET /api/v1/experiments/
List all of the started experiments.
2017-05-04 23:32:01 +03:00
2017-08-02 22:10:16 +03:00
#### Optional Query Parameters
project__slug - Return only the experiments for a given project, an invalid slug will raise 404
Example: GET /api/v1/experiments/?project__slug=project-slug
2017-05-04 23:32:01 +03:00
[
{
2017-07-11 20:49:23 +03:00
"active": true,
2017-05-04 23:32:01 +03:00
"name": "New Feature",
"slug": "new-feature",
2017-07-11 20:49:23 +03:00
"pref_key": "new.feature",
2017-05-18 23:01:31 +03:00
"addon_versions": [
"1.0",
"2.0"
],
2017-05-04 23:32:01 +03:00
"start_date": 1493928948000.0,
"end_date": 1495138548000.0,
"variant": {
"slug": "enabled",
"experiment_variant_slug": "new-feature:enabled",
2017-07-11 20:49:23 +03:00
"value": true,
"threshold": 0.2
2017-05-04 23:32:01 +03:00
},
"control": {
"slug": "disabled",
"experiment_variant_slug": "new-feature:disabled",
2017-07-11 20:49:23 +03:00
"value": false
2017-05-04 23:32:01 +03:00
}
}
]
2017-04-11 22:45:02 +03:00
2017-08-18 22:14:45 +03:00
### PATCH /api/v1/experiments/<experiment_slug>/accept
Body: None
Set the status of a Pending experiment to Accepted.
Example: PATCH /api/v1/experiments/my-first-experiment/accept
### PATCH /api/v1/experiments/<experiment_slug>/reject
content-type: application/json
Body: {message: "This experiment was rejected for reasons."}
Set the status of a Pending experiment to Rejected.
Example: PATCH /api/v1/experiments/my-first-experiment/accept
2017-04-11 22:45:02 +03:00
## Contributing
2017-08-18 22:14:45 +03:00
1. Fork the repo!
1. File a new issue describing the change you want to make: Change the things #123
1. Create your feature branch with the issue number: `git checkout -b 123`
1. Commit your changes: `git commit -am 'Changed the things fixes #123'`
1. Push to the branch: `git push origin 123`
1. Submit a pull request :D
2017-04-11 22:45:02 +03:00
## License
Experimenter uses the [Mozilla Public License ](https://www.mozilla.org/en-US/MPL/ )