fix #8360 feat(nimbus): Change root folder app to experimenter (#8375)

Because

- We will be adding cirrus in experimenter

This commit

- Rename `app` to `experimenter` and make necessary changes
This commit is contained in:
Yashika Khurana 2023-03-08 13:36:51 -08:00 коммит произвёл GitHub
Родитель a462dc03c0
Коммит 7855bbf1eb
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
1311 изменённых файлов: 316 добавлений и 287 удалений

Просмотреть файл

@ -37,7 +37,7 @@ jobs:
- run:
name: Check file paths
command: |
if git diff --name-only main HEAD | grep -E 'app/experimenter/targeting|app/experimenter/experiments|app/tests'
if git diff --name-only main HEAD | grep -E 'experimenter/experimenter/targeting|experimenter/experimenter/experiments|experimenter/tests'
then
echo "Continuing"
else
@ -65,7 +65,7 @@ jobs:
- run:
name: Check file paths
command: |
if git diff --name-only main HEAD | grep -E 'app/experimenter/targeting|app/experimenter/experiments|app/tests'
if git diff --name-only main HEAD | grep -E 'experimenter/experimenter/targeting|experimenter/experimenter/experiments|experimenter/tests'
then
echo "Continuing"
else
@ -228,7 +228,7 @@ jobs:
- run:
name: Check file paths
command: |
if git diff --name-only main HEAD | grep -E 'app/experimenter/targeting|app/experimenter/experiments|app/tests'
if git diff --name-only main HEAD | grep -E 'experimenter/experimenter/targeting|experimenter/experimenter/experiments|experimenter/tests'
then
echo "Continuing"
else
@ -277,10 +277,10 @@ jobs:
make build_dev build_test build_ui
./scripts/store_git_info.sh
make build_prod
docker tag app:dev ${DOCKERHUB_REPO}:build_dev
docker tag app:test ${DOCKERHUB_REPO}:build_test
docker tag app:ui ${DOCKERHUB_REPO}:build_ui
docker tag app:deploy ${DOCKERHUB_REPO}:latest
docker tag experimenter:dev ${DOCKERHUB_REPO}:build_dev
docker tag experimenter:test ${DOCKERHUB_REPO}:build_test
docker tag experimenter:ui ${DOCKERHUB_REPO}:build_ui
docker tag experimenter:deploy ${DOCKERHUB_REPO}:latest
docker push ${DOCKERHUB_REPO}:build_dev
docker push ${DOCKERHUB_REPO}:build_test
docker push ${DOCKERHUB_REPO}:build_ui
@ -415,7 +415,7 @@ jobs:
AS_VERSION=$(git -c 'versionsort.suffix=-' \
ls-remote --exit-code --refs --sort='version:refname' --tags https://github.com/mozilla/application-services.git \
'*.*.*' | tail --lines=1 | cut --delimiter='/' --fields=3)
docker build -t ${DOCKERHUB_REPO}:nimbus-rust-image -f app/tests/integration/nimbus/utils/Dockerfile-rust-image --build-arg as_version=$AS_VERSION --progress=plain .
docker build -t ${DOCKERHUB_REPO}:nimbus-rust-image -f experimenter/tests/integration/nimbus/utils/Dockerfile-rust-image --build-arg as_version=$AS_VERSION --progress=plain .
docker_id=$(docker run -t -d --name nimbus-rust-image ${DOCKERHUB_REPO}:nimbus-rust-image)
docker cp /home/circleci/experimenter/application-services-current.txt nimbus-rust-image:/code/application-services-old.txt
docker commit $docker_id ${DOCKERHUB_REPO}:nimbus-rust-image

51
.github/dependabot.yml поставляемый
Просмотреть файл

@ -1,26 +1,29 @@
version: 2
updates:
- package-ecosystem: npm
directory: "/app"
schedule:
interval: weekly
target-branch: main
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-major", "version-update:semver-patch"]
- package-ecosystem: pip
directory: "/app"
schedule:
interval: weekly
target-branch: main
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-major", "version-update:semver-patch"]
- package-ecosystem: pip
directory: "/app/tests"
schedule:
interval: weekly
target-branch: main
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-major", "version-update:semver-patch"]
- package-ecosystem: npm
directory: "/experimenter"
schedule:
interval: weekly
target-branch: main
ignore:
- dependency-name: "*"
update-types:
["version-update:semver-major", "version-update:semver-patch"]
- package-ecosystem: pip
directory: "/experimenter"
schedule:
interval: weekly
target-branch: main
ignore:
- dependency-name: "*"
update-types:
["version-update:semver-major", "version-update:semver-patch"]
- package-ecosystem: pip
directory: "/experimenter/tests"
schedule:
interval: weekly
target-branch: main
ignore:
- dependency-name: "*"
update-types:
["version-update:semver-major", "version-update:semver-patch"]

18
.gitignore поставляемый
Просмотреть файл

@ -76,9 +76,9 @@ target/
**/.vscode/*
# Hosted assets
app/experimenter/served/
app/experimenter/legacy/legacy-ui/assets/
app/experimenter/reporting/reporting-ui/assets/
experimenter/experimenter/served/
experimenter/experimenter/legacy/legacy-ui/assets/
experimenter/experimenter/reporting/reporting-ui/assets/
# Key files
*.pem
@ -87,7 +87,7 @@ app/experimenter/reporting/reporting-ui/assets/
.pytest_cache
.testmondata
.tmontmp
app/coverage_html_report/
experimenter/coverage_html_report/
# Dependencies
node_modules
@ -95,11 +95,11 @@ node_modules
google-credentials.json
# Jetstream config
app/experimenter/outcomes/metric-hub-main/*
!app/experimenter/outcomes/metric-hub-main/jetstream/
app/experimenter/outcomes/metric-hub-main/jetstream/*
!app/experimenter/outcomes/metric-hub-main/jetstream/outcomes/
app/experimenter/outcomes/metric-hub.zip
experimenter/experimenter/outcomes/metric-hub-main/*
!experimenter/experimenter/outcomes/metric-hub-main/jetstream/
experimenter/experimenter/outcomes/metric-hub-main/jetstream/*
!experimenter/experimenter/outcomes/metric-hub-main/jetstream/outcomes/
experimenter/experimenter/outcomes/metric-hub.zip
# Versioning assets generated on build
**/commit-description.txt

Просмотреть файл

@ -1,7 +1,7 @@
SHELL = /bin/bash
WAIT_FOR_DB = /app/bin/wait-for-it.sh -t 30 db:5432 &&
WAIT_FOR_RUNSERVER = /app/bin/wait-for-it.sh -t 30 localhost:7001 &&
WAIT_FOR_DB = /experimenter/bin/wait-for-it.sh -t 30 db:5432 &&
WAIT_FOR_RUNSERVER = /experimenter/bin/wait-for-it.sh -t 30 localhost:7001 &&
COMPOSE = docker-compose -f docker-compose.yml
COMPOSE_LEGACY = ${COMPOSE} -f docker-compose-legacy.yml
@ -68,16 +68,16 @@ auth_gcloud:
gcloud auth login --update-adc
jetstream_config:
curl -LJ -o app/experimenter/outcomes/metric-hub.zip $(JETSTREAM_CONFIG_URL)
unzip -o -d app/experimenter/outcomes app/experimenter/outcomes/metric-hub.zip
rm -Rf app/experimenter/outcomes/metric-hub-main/.script/
curl -LJ -o experimenter/experimenter/outcomes/metric-hub.zip $(JETSTREAM_CONFIG_URL)
unzip -o -d experimenter/experimenter/outcomes experimenter/experimenter/outcomes/metric-hub.zip
rm -Rf experimenter/experimenter/outcomes/metric-hub-main/.script/
feature_manifests:
curl -LJ --create-dirs -o app/experimenter/features/manifests/firefox-desktop.yaml $(FEATURE_MANIFEST_DESKTOP_URL)
curl -LJ --create-dirs -o app/experimenter/features/manifests/fenix.yaml $(FEATURE_MANIFEST_FENIX_URL)
curl -LJ --create-dirs -o app/experimenter/features/manifests/ios.yaml $(FEATURE_MANIFEST_FXIOS_URL)
curl -LJ --create-dirs -o app/experimenter/features/manifests/focus-android.yaml $(FEATURE_MANIFEST_FOCUS_ANDROID)
curl -LJ --create-dirs -o app/experimenter/features/manifests/focus-ios.yaml $(FEATURE_MANIFEST_FOCUS_IOS)
curl -LJ --create-dirs -o experimenter/experimenter/features/manifests/firefox-desktop.yaml $(FEATURE_MANIFEST_DESKTOP_URL)
curl -LJ --create-dirs -o experimenter/experimenter/features/manifests/fenix.yaml $(FEATURE_MANIFEST_FENIX_URL)
curl -LJ --create-dirs -o experimenter/experimenter/features/manifests/ios.yaml $(FEATURE_MANIFEST_FXIOS_URL)
curl -LJ --create-dirs -o experimenter/experimenter/features/manifests/focus-android.yaml $(FEATURE_MANIFEST_FOCUS_ANDROID)
curl -LJ --create-dirs -o experimenter/experimenter/features/manifests/focus-ios.yaml $(FEATURE_MANIFEST_FOCUS_IOS)
fetch_external_resources: jetstream_config feature_manifests
echo "External Resources Fetched"
@ -85,17 +85,20 @@ fetch_external_resources: jetstream_config feature_manifests
update_kinto:
docker pull mozilla/kinto-dist:latest
build_dev: ssl
DOCKER_BUILDKIT=1 docker build --target dev -f app/Dockerfile -t app:dev --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from mozilla/experimenter:build_dev $$([[ -z "$${CIRCLECI}" ]] || echo "--progress=plain") app/
compose_build:
$(COMPOSE) build
build_test: ssl
DOCKER_BUILDKIT=1 docker build --target test -f app/Dockerfile -t app:test --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from mozilla/experimenter:build_test $$([[ -z "$${CIRCLECI}" ]] || echo "--progress=plain") app/
build_dev: ssl compose_build
DOCKER_BUILDKIT=1 docker build --target dev -f experimenter/Dockerfile -t experimenter:dev --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from mozilla/experimenter:build_dev $$([[ -z "$${CIRCLECI}" ]] || echo "--progress=plain") experimenter/
build_ui: ssl
DOCKER_BUILDKIT=1 docker build --target ui -f app/Dockerfile -t app:ui --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from mozilla/experimenter:build_ui $$([[ -z "$${CIRCLECI}" ]] || echo "--progress=plain") app/
build_test: ssl compose_build
DOCKER_BUILDKIT=1 docker build --target test -f experimenter/Dockerfile -t experimenter:test --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from mozilla/experimenter:build_test $$([[ -z "$${CIRCLECI}" ]] || echo "--progress=plain") experimenter/
build_ui: ssl compose_build
DOCKER_BUILDKIT=1 docker build --target ui -f experimenter/Dockerfile -t experimenter:ui --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from mozilla/experimenter:build_ui $$([[ -z "$${CIRCLECI}" ]] || echo "--progress=plain") experimenter/
build_prod: build_ui ssl
DOCKER_BUILDKIT=1 docker build --target deploy -f app/Dockerfile -t app:deploy --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from mozilla/experimenter:latest $$([[ -z "$${CIRCLECI}" ]] || echo "--progress=plain") app/
DOCKER_BUILDKIT=1 docker build --target deploy -f experimenter/Dockerfile -t experimenter:deploy --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from mozilla/experimenter:latest $$([[ -z "$${CIRCLECI}" ]] || echo "--progress=plain") experimenter/
compose_stop:
$(COMPOSE) kill || true
@ -111,20 +114,20 @@ volumes_rm:
docker volume prune -f
static_rm:
rm -Rf app/node_modules
rm -Rf app/experimenter/legacy/legacy-ui/core/node_modules/
rm -Rf app/experimenter/nimbus-ui/node_modules/
rm -Rf app/experimenter/legacy/legacy-ui/assets/
rm -Rf app/experimenter/nimbus-ui/build/
rm -Rf experimenter/node_modules
rm -Rf experimenter/experimenter/legacy/legacy-ui/core/node_modules/
rm -Rf experimenter/experimenter/nimbus-ui/node_modules/
rm -Rf experimenter/experimenter/legacy/legacy-ui/assets/
rm -Rf experimenter/experimenter/nimbus-ui/build/
kill: compose_stop compose_rm volumes_rm
echo "All containers removed!"
check: build_test
$(COMPOSE_TEST) run app sh -c '$(WAIT_FOR_DB) (${PARALLEL} "$(NIMBUS_SCHEMA_CHECK)" "$(PYTHON_CHECK_MIGRATIONS)" "$(CHECK_DOCS)" "$(BLACK_CHECK)" "$(RUFF_CHECK)" "$(ESLINT_LEGACY)" "$(ESLINT_NIMBUS_UI)" "$(TYPECHECK_NIMBUS_UI)" "$(PYTHON_TYPECHECK)" "$(PYTHON_TEST)" "$(JS_TEST_LEGACY)" "$(JS_TEST_NIMBUS_UI)" "$(JS_TEST_REPORTING)") ${COLOR_CHECK}'
$(COMPOSE_TEST) run experimenter sh -c '$(WAIT_FOR_DB) (${PARALLEL} "$(NIMBUS_SCHEMA_CHECK)" "$(PYTHON_CHECK_MIGRATIONS)" "$(CHECK_DOCS)" "$(BLACK_CHECK)" "$(RUFF_CHECK)" "$(ESLINT_LEGACY)" "$(ESLINT_NIMBUS_UI)" "$(TYPECHECK_NIMBUS_UI)" "$(PYTHON_TYPECHECK)" "$(PYTHON_TEST)" "$(JS_TEST_LEGACY)" "$(JS_TEST_NIMBUS_UI)" "$(JS_TEST_REPORTING)") ${COLOR_CHECK}'
pytest: build_test
$(COMPOSE_TEST) run app sh -c '$(WAIT_FOR_DB) $(PYTHON_TEST)'
$(COMPOSE_TEST) run experimenter sh -c '$(WAIT_FOR_DB) $(PYTHON_TEST)'
up: build_dev
$(COMPOSE) up
@ -142,35 +145,35 @@ up_db: build_dev
$(COMPOSE) up db redis kinto autograph
up_django: build_dev
$(COMPOSE) up nginx app worker beat db redis kinto autograph
$(COMPOSE) up nginx experimenter worker beat db redis kinto autograph
up_detached: build_dev
$(COMPOSE) up -d
generate_docs: build_dev
$(COMPOSE) run app sh -c "$(GENERATE_DOCS)"
$(COMPOSE) run experimenter sh -c "$(GENERATE_DOCS)"
generate_types: build_dev
$(COMPOSE) run app sh -c "$(NIMBUS_TYPES_GENERATE)"
$(COMPOSE) run experimenter sh -c "$(NIMBUS_TYPES_GENERATE)"
code_format: build_dev
$(COMPOSE) run app sh -c '${PARALLEL} "$(RUFF_FIX);$(BLACK_FIX)" "$(ESLINT_FIX_CORE)" "$(ESLINT_FIX_NIMBUS_UI)"'
$(COMPOSE) run experimenter sh -c '${PARALLEL} "$(RUFF_FIX);$(BLACK_FIX)" "$(ESLINT_FIX_CORE)" "$(ESLINT_FIX_NIMBUS_UI)"'
makemigrations: build_dev
$(COMPOSE) run app python manage.py makemigrations
$(COMPOSE) run experimenter python manage.py makemigrations
migrate: build_dev
$(COMPOSE) run app sh -c "$(WAIT_FOR_DB) $(PYTHON_MIGRATE)"
$(COMPOSE) run experimenter sh -c "$(WAIT_FOR_DB) $(PYTHON_MIGRATE)"
bash: build_dev
$(COMPOSE) run app bash
$(COMPOSE) run experimenter bash
refresh: kill build_dev
$(COMPOSE) run -e SKIP_DUMMY=$$SKIP_DUMMY app bash -c '$(WAIT_FOR_DB) $(PYTHON_MIGRATE)&&$(LOAD_LOCALES)&&$(LOAD_COUNTRIES)&&$(LOAD_LANGUAGES)&&$(LOAD_FEATURES)&&$(LOAD_DUMMY_EXPERIMENTS)'
$(COMPOSE) run -e SKIP_DUMMY=$$SKIP_DUMMY experimenter bash -c '$(WAIT_FOR_DB) $(PYTHON_MIGRATE)&&$(LOAD_LOCALES)&&$(LOAD_COUNTRIES)&&$(LOAD_LANGUAGES)&&$(LOAD_FEATURES)&&$(LOAD_DUMMY_EXPERIMENTS)'
dependabot_approve:
echo "Install and configure the Github CLI https://github.com/cli/cli"
gh pr list --author app/dependabot | awk '{print $$1}' | xargs -n1 gh pr review -a -b "@dependabot squash and merge"
gh pr list --author experimenter/dependabot | awk '{print $$1}' | xargs -n1 gh pr review -a -b "@dependabot squash and merge"
# integration tests
integration_shell:
@ -186,18 +189,18 @@ integration_vnc_up_detached: build_prod
$(COMPOSE_INTEGRATION) up -d firefox
integration_test_legacy: build_prod
MOZ_HEADLESS=1 $(COMPOSE_INTEGRATION) run firefox sh -c "sudo apt-get -qqy update && sudo apt-get -qqy install tox;sudo chmod a+rwx /code/app/tests/integration/.tox;tox -c app/tests/integration -e integration-test-legacy $(TOX_ARGS) -- -n 2 $(PYTEST_ARGS)"
MOZ_HEADLESS=1 $(COMPOSE_INTEGRATION) run firefox sh -c "sudo apt-get -qqy update && sudo apt-get -qqy install tox;sudo chmod a+rwx /code/experimenter/tests/integration/.tox;tox -c experimenter/tests/integration -e integration-test-legacy $(TOX_ARGS) -- -n 2 $(PYTEST_ARGS)"
integration_test_nimbus: build_prod
MOZ_HEADLESS=1 $(COMPOSE_INTEGRATION) run firefox sh -c "if [ "$$UPDATE_FIREFOX_VERSION" = "true" ]; then sudo ./app/tests/integration/nimbus/utils/nightly-install.sh; fi; firefox -V; sudo apt-get -qqy update && sudo apt-get -qqy install tox;sudo chmod a+rwx /code/app/tests/integration/.tox;PYTEST_SENTRY_DSN=$(PYTEST_SENTRY_DSN) PYTEST_SENTRY_ALWAYS_REPORT=$(PYTEST_SENTRY_ALWAYS_REPORT) tox -c app/tests/integration -e integration-test-nimbus $(TOX_ARGS) -- $(PYTEST_ARGS)"
MOZ_HEADLESS=1 $(COMPOSE_INTEGRATION) run firefox sh -c "if [ "$$UPDATE_FIREFOX_VERSION" = "true" ]; then sudo ./experimenter/tests/integration/nimbus/utils/nightly-install.sh; fi; firefox -V; sudo apt-get -qqy update && sudo apt-get -qqy install tox;sudo chmod a+rwx /code/experimenter/tests/integration/.tox;PYTEST_SENTRY_DSN=$(PYTEST_SENTRY_DSN) PYTEST_SENTRY_ALWAYS_REPORT=$(PYTEST_SENTRY_ALWAYS_REPORT) tox -c experimenter/tests/integration -e integration-test-nimbus $(TOX_ARGS) -- $(PYTEST_ARGS)"
integration_test_nimbus_rust: build_prod
MOZ_HEADLESS=1 $(COMPOSE_INTEGRATION) run rust-sdk sh -c "chmod a+rwx /code/app/tests/integration/.tox;tox -c app/tests/integration -e integration-test-nimbus-rust $(TOX_ARGS) -- -n 2 $(PYTEST_ARGS)"
MOZ_HEADLESS=1 $(COMPOSE_INTEGRATION) run rust-sdk sh -c "chmod a+rwx /code/experimenter/tests/integration/.tox;tox -c experimenter/tests/integration -e integration-test-nimbus-rust $(TOX_ARGS) -- -n 2 $(PYTEST_ARGS)"
# cirrus
cirrus_up:
$(COMPOSE) up cirrus
cirrus_down:
$(COMPOSE) down cirrus

145
README.md
Просмотреть файл

@ -34,17 +34,20 @@ Check out the [🌩 **Nimbus Documentation Hub**](https://experimenter.info) or
### General Setup
1. Prerequisites
1. Prerequisites
On all platforms:
- Install [Node](https://nodejs.org/en/download/releases/) to match [current version](https://github.com/mozilla/experimenter/blob/main/app/Dockerfile#L29)
- Install [Node](https://nodejs.org/en/download/releases/) to match [current version](https://github.com/mozilla/experimenter/blob/main/experimenter/Dockerfile#L29)
On Linux:
- Install [Docker](https://www.docker.com/)
- Install [yarn](https://classic.yarnpkg.com/lang/en/docs/install)
- [Setup docker to run as non-root](https://docs.docker.com/engine/security/rootless/)
On MacOS:
- Install [Docker](https://docs.docker.com/desktop/mac/install/)
- Adjust resource settings
- CPU: Max number of cores
@ -53,45 +56,45 @@ Check out the [🌩 **Nimbus Documentation Hub**](https://experimenter.info) or
- Disk: 100gb+
- Install [yarn](https://github.com/yarnpkg)
1. Clone the repo
1. Clone the repo
git clone <your fork>
1. Copy the sample env file
1. Copy the sample env file
cp .env.sample .env
1. Set DEBUG=True for local development
1. Set DEBUG=True for local development
vi .env
1. Create a new secret key and put it in .env
1. Create a new secret key and put it in .env
make secretkey
vi .env
vi .env
```
...
SECRETKEY=mynewsecretkey
...
```
```
...
SECRETKEY=mynewsecretkey
...
```
1. Run tests
1. Run tests
make check
1. Setup the database
1. Setup the database
make refresh
#### Fully Dockerized Setup (continuation from General Setup 1-7)
1. Run a dev instance
1. Run a dev instance
make up
1. Navigate to it and add an SSL exception to your browser
1. Navigate to it and add an SSL exception to your browser
https://localhost/
@ -127,7 +130,8 @@ Notes:
sudo apt install libpq5=12.9-0ubuntu0.20.04.1
sudo apt install libpq-dev
_*Notes_
_\*Notes_
- _the specific libpq5 version shown here is required for libpq-dev at time of writing_
- _`poetry install` (next step) requires python 3.9, but there are multiple options for resolving this, see [here](https://python-poetry.org/docs/managing-environments/#switching-between-environments)_
@ -135,7 +139,7 @@ Notes:
source .env
cd app
cd experimenter
poetry install # see note above
yarn install
@ -153,7 +157,7 @@ Notes:
5. Django app
# in app
# in experimenter
poetry shell
@ -164,7 +168,7 @@ Notes:
yarn workspace @experimenter/nimbus-ui start
./manage.py runserver 0.0.0.0:7001
*Pro-tip*: we have had at least one large code refactor. You can ignore specific large commits when blaming by setting the Git config's `ignoreRevsFile` to `.git-blame-ignore-revs`:
_Pro-tip_: we have had at least one large code refactor. You can ignore specific large commits when blaming by setting the Git config's `ignoreRevsFile` to `.git-blame-ignore-revs`:
```
git config blame.ignoreRevsFile .git-blame-ignore-revs
@ -174,52 +178,55 @@ git config blame.ignoreRevsFile .git-blame-ignore-revs
1. If using VSCode, configure workspace folders
- Add `/experimenter/` and `/experimenter/app` folders to your workspace (File -> Add Folder to Workspace -> `path/to/experimenter/app`)
- From the `/experimenter/app` folder, run `yarn install`
- Make sure you are using the correct version of node
- Add `/experimenter/` and `/experimenter/experimenter` folders to your workspace (File -> Add Folder to Workspace -> `path/to/experimenter/experimenter`)
- From the `/experimenter/experimenter` folder, run `yarn install`
node -v
- Make sure you are using the correct version of node
- Troubleshooting:
- [Changing node version](https://stackoverflow.com/a/50817276/12178648)
- Clear npm cache: `npm cache clean --force`
node -v
- Troubleshooting:
- [Changing node version](https://stackoverflow.com/a/50817276/12178648)
- Clear npm cache: `npm cache clean --force`
### Google Credentials for Jetstream
On certain pages an API endpoint is called to receive experiment analysis data from Jetstream to display visualization tables. To see experiment visualization data, you must provide GCP credentials.
0. Prequisites
- Install GCP CLI
- Follow the instructions [here](https://cloud.google.com/sdk/docs/install)
- Project: `moz-fx-data-experiments`
- Install GCP CLI
- Follow the instructions [here](https://cloud.google.com/sdk/docs/install)
- Project: `moz-fx-data-experiments`
- Verify/request project permissions
- Check if you already have access to the storage bucket [here](https://console.cloud.google.com/storage/browser/mozanalysis)
- If needed, ask in `#nimbus-dev` for a project admin to grant `storage.objects.list` permissions on the `moz-fx-data-experiments` project
- Check if you already have access to the storage bucket [here](https://console.cloud.google.com/storage/browser/mozanalysis)
- If needed, ask in `#nimbus-dev` for a project admin to grant `storage.objects.list` permissions on the `moz-fx-data-experiments` project
1. Authorize CLI with your account
- `make auth_gcloud`
- this will save your credentials locally to a well-known location for use by any library that requests ADC
- **Note**: if this returns `Error saving Application Default Credentials: Unable to write file [...]: [Errno 21] Is a directory: ...`, delete the directory and try again (`rm -rf ~/.config/gcloud`)
- `make auth_gcloud`
- this will save your credentials locally to a well-known location for use by any library that requests ADC
- **Note**: if this returns `Error saving Application Default Credentials: Unable to write file [...]: [Errno 21] Is a directory: ...`, delete the directory and try again (`rm -rf ~/.config/gcloud`)
2. The next time you rebuild the docker-compose environment, your credentials will be loaded as a volume
- Note that this will require the existing volume to be removed (hint: run `make refresh`)
- Note that this will require the existing volume to be removed (hint: run `make refresh`)
3. (optional) Verify access
- `make refresh`
- `make bash`
- `./manage.py shell`
- ```
from django.core.files.storage import default_storage
default_storage.listdir('/')
```
- Confirm this second command prints a list instead of an error
- `make refresh`
- `make bash`
- `./manage.py shell`
- ```
from django.core.files.storage import default_storage
default_storage.listdir('/')
```
- Confirm this second command prints a list instead of an error
### Google Cloud Bucket for Media Storage
We support user uploads of media (e.g. screenshots) for some features.
In local development, the default is to store these files in `/app/media` using Django's `FileSystemStorage` class and the `MEDIA_ROOT` and `MEDIA_URL` settings.
In local development, the default is to store these files in `/experimenter/media` using Django's `FileSystemStorage` class and the `MEDIA_ROOT` and `MEDIA_URL` settings.
In production, a GCP bucket and credentials are required.
@ -238,6 +245,7 @@ In the real production deployment, credentials are configured via [workload iden
Experimenter uses [docker](https://www.docker.com/) for all development, testing, and deployment.
### Building
#### make build
Build the application container by executing the [build script](https://github.com/mozilla/experimenter/blob/main/scripts/build.sh)
@ -259,7 +267,7 @@ WARNING: this will remove your database and all data. Use this to reset your dev
#### make migrate
Apply all django migrations to the database. This must be run after removing database volumes before starting a dev instance.
Apply all django migrations to the database. This must be run after removing database volumes before starting a dev instance.
#### make load_dummy_experiments
@ -267,12 +275,13 @@ Populates the database with dummy experiments of all types/statuses using the te
#### make refresh
Run kill, migrate, load_locales_countries load_dummy_experiments. Useful for resetting your dev environment when switching branches or after package updates.
Run kill, migrate, load_locales_countries load_dummy_experiments. Useful for resetting your dev environment when switching branches or after package updates.
### Running a dev instance
#### make up
Start a dev server listening on port 80 using the [Django runserver](https://docs.djangoproject.com/en/1.10/ref/django-admin/#runserver). It is useful to run `make refresh` first to ensure your database is up to date with the latest migrations and test data.
Start a dev server listening on port 80 using the [Django runserver](https://docs.djangoproject.com/en/1.10/ref/django-admin/#runserver). It is useful to run `make refresh` first to ensure your database is up to date with the latest migrations and test data.
#### make up_db
@ -284,7 +293,7 @@ Start Django runserver, Celery worker, postgresql, redis, autograph, kinto on th
#### make up_detached
Start all containers in the background (not attached to shell). They can be stopped using `make kill`.
Start all containers in the background (not attached to shell). They can be stopped using `make kill`.
#### make update_kinto
@ -297,6 +306,7 @@ Pull in the latest Kinto Docker image. Kinto is not automatically updated when n
Run all test and lint suites, this is run in CI on all PRs and deploys.
##### Helpful UI Testing Tips
If you have a test failing to find an element (or finding too many, etc.) and the DOM is being cut off in the console output,
you can increase how much is printed by locally editing the `DEBUG_PRINT_LIMIT=7000` in the `Makefile` (line starts with `JS_TEST_NIMBUS_UI`).
@ -306,9 +316,10 @@ Run only the python test suite.
#### make bash
Start a bash shell inside the container. This lets you interact with the containerized filesystem and run Django management commands.
Start a bash shell inside the container. This lets you interact with the containerized filesystem and run Django management commands.
##### Helpful Python Tips
You can run the entire python test suite without coverage using the Django test runner:
```sh
@ -368,6 +379,7 @@ nimbus_push_experiment_to_kinto(experiment.id)
```
##### Helpful Yarn Tips
You can also interact with the yarn commands, such as checking TypeScript for Nimbus UI:
```sh
@ -382,9 +394,6 @@ yarn workspace @experimenter/nimbus-ui test:cov
For a full reference of all the common commands that can be run inside the container, refer to [this section of the Makefile](https://github.com/mozilla/experimenter/blob/main/Makefile#L16-L38)
#### make integration_test_legacy
Run the integration test suite for experimenter inside a containerized instance of Firefox. You must also be already running a `make up` dev instance in another shell to run the integration tests.
@ -412,7 +421,7 @@ make FIREFOX_VERSION integration_vnc_up
Then open your VNC client (Safari does this on OSX or just use [VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/)) and open `vnc://localhost:5900` with password `secret`. Right click on the desktop and select `Applications > Shell > Bash` and enter:
```bash
cd app
cd experimenter
sudo apt get update
sudo apt install tox
chmod a+rwx tests/integration/.tox
@ -429,6 +438,7 @@ To use NoVNC, navgate to this url `http://localhost:7902` with the password `sec
- `PYTEST_ARGS`: [Pytest](https://docs.pytest.org/en/6.2.x/usage.html#) commandline variables.
An example using PYTEST_ARGS to run one test.
```bash
make integration_test_legacy PYTEST_ARGS="-k test_addon_rollout_experiment_e2e"
```
@ -438,15 +448,19 @@ make integration_test_legacy PYTEST_ARGS="-k test_addon_rollout_experiment_e2e"
This builds and sets up the mobile sdk for use in testing.
### Testing Tools
#### Targeting test tool
Navigate to `app/tests/tools`
To test a targeting expression, first add an app context named `app_context.json` to the `app/tests/tools` directory.
#### Targeting test tool
Navigate to `experimenter/tests/tools`
To test a targeting expression, first add an app context named `app_context.json` to the `experimenter/tests/tools` directory.
You can then invoke the script with the `--targeting-string` flag:
```bash
python sdk_eval_check.py --targeting-string "(app_version|versionCompare('106.*') <= 0) && (is_already_enrolled)"
```
The script should return the results, either `True`, `False`, or an error.
Note that you can change the `app_context` live, and run the script again after.
@ -461,7 +475,7 @@ There are three accounts you can log into Kinto with depending on what you want
- `experimenter` / `experimenter` - This account is used by Experimenter to push its changes to Remote Settings and mark them for review.
- `review` / `review` - This account should generally be used by developers testing the workflow, it can be used to approve/reject changes pushed from Experimenter.
The `admin` and `review` credentials are hard-coded [here](https://github.com/mozilla/experimenter/blob/main/app/bin/setup_kinto.py#L7-L8), and the `experimenter` credentials can be found or updated in your `.env` file under `KINTO_USER` and `KINTO_PASS`.
The `admin` and `review` credentials are hard-coded [here](https://github.com/mozilla/experimenter/blob/main/experimenter/bin/setup_kinto.py#L7-L8), and the `experimenter` credentials can be found or updated in your `.env` file under `KINTO_USER` and `KINTO_PASS`.
Any change in remote settings requires two accounts:
@ -470,23 +484,22 @@ Any change in remote settings requires two accounts:
Any of the accounts above can be used for any of those two roles, but your local Experimenter will be configured to make its changes through the `experimenter` account, so that account can't also be used to approve/reject those changes, hence the existence of the `review` account.
For more detailed information on the Remote Settings integration please see the [Kinto module documentation](app/experimenter/kinto/README.md).
For more detailed information on the Remote Settings integration please see the [Kinto module documentation](experimenter/experimenter/kinto/README.md).
## Frontend
Experimenter has two front-end UIs:
- [`core`](./app/experimenter/legacy/legacy-ui/core) is the legacy UI used for Experimenter intake which will remain until `nimbus-ui` supersedes it
- [`nimbus-ui`](./app/experimenter/nimbus-ui) is the Nimbus Console UI for Experimenter that is actively being developed
- [`core`](./experimenter/experimenter/legacy/legacy-ui/core) is the legacy UI used for Experimenter intake which will remain until `nimbus-ui` supersedes it
- [`nimbus-ui`](./experimenter/experimenter/nimbus-ui) is the Nimbus Console UI for Experimenter that is actively being developed
Learn more about the organization of these UIs [here](./app/experimenter/legacy/legacy-ui/README.md).
Learn more about the organization of these UIs [here](./experimenter/experimenter/legacy/legacy-ui/README.md).
**Also see the [nimbus-ui README](https://github.com/mozilla/experimenter/tree/main/app/experimenter/nimbus-ui) for relevent Nimbus documentation.**
**Also see the [nimbus-ui README](https://github.com/mozilla/experimenter/tree/main/experimenter/experimenter/nimbus-ui) for relevent Nimbus documentation.**
## API
API documentation can be found [here](https://htmlpreview.github.io/?https://github.com/mozilla/experimenter/blob/main/app/experimenter/docs/swagger-ui.html)
API documentation can be found [here](https://htmlpreview.github.io/?https://github.com/mozilla/experimenter/blob/main/experimenter/experimenter/docs/swagger-ui.html)
## Contributing

Просмотреть файл

@ -17,46 +17,47 @@ Please note we have a [Code of Conduct](https://www.mozilla.org/en-US/about/gove
## Pull Request Process
1. Find the issue you want to work on. If no relevant issue exists, see above and file an
issue, then continue to the next step.
1. Find the issue you want to work on. If no relevant issue exists, see above and file an
issue, then continue to the next step.
1. Assign the issue to yourself.
1. Create a branch in your local fork with the issue number in the name.
1. Implement the code changes described by the issue. All Python code changes should
be accompanied by any relevant tests, either by modifying existing ones or adding new
ones. Front-end changes (HTML/CSS/JS) that involve React code should create or update
any relevant JS tests. Infrastructure changes (Docker related things)
do not require tests.
1. Implement the code changes described by the issue. All Python code changes should
be accompanied by any relevant tests, either by modifying existing ones or adding new
ones. Front-end changes (HTML/CSS/JS) that involve React code should create or update
any relevant JS tests. Infrastructure changes (Docker related things)
do not require tests.
1. When all your changes are complete to your satisfaction:
- Run `make check` to run the full test, linting, formatting, and coverage suites.
- Fix any linting issues by running `make code_format` or by hand if the auto formatter is unable to fix it.
- Fix any broken tests
- Ensure coverage remains at 100% by adding new tests for any uncovered lines.
- Run `make check` to run the full test, linting, formatting, and coverage suites.
- Fix any linting issues by running `make code_format` or by hand if the auto formatter is unable to fix it.
- Fix any broken tests
- Ensure coverage remains at 100% by adding new tests for any uncovered lines.
1. When all tests and checks are passing, commit all your changes into a single commit and follow the [Git Commit Guidelines](#git-commit-guidelines)
1. Push your branch up to your fork and submit a pull request on to main. Add any additional
information you'd like to the pull request body, including descriptions of changes, screenshots
of any UI changes, special instructions for testing, etc.
1. Push your branch up to your fork and submit a pull request on to main. Add any additional
information you'd like to the pull request body, including descriptions of changes, screenshots
of any UI changes, special instructions for testing, etc.
1. Find a reviewer. If you're not sure who should review, please contact us on #nimbus-dev on
the Mozilla Slack if you have access to it.
1. Find a reviewer. If you're not sure who should review, please contact us on #nimbus-dev on
the Mozilla Slack if you have access to it.
1. Any PRs that require changes to deployment infrastructure ie environment variable changes,
dependent services, etc also require a review by a member of the operations team. Please include
any relevant instructions about the changes in the PR. File a [Bugzilla ticket](https://bugzilla.mozilla.org/enter_bug.cgi?product=Data+Platform+and+Tools&component=Operations) with a link to the PR so that operations can track the change.
dependent services, etc also require a review by a member of the operations team. Please include
any relevant instructions about the changes in the PR. File a [Bugzilla ticket](https://bugzilla.mozilla.org/enter_bug.cgi?product=Data+Platform+and+Tools&component=Operations) with a link to the PR so that operations can track the change.
1. If you receive feedback that requires changes to your pull request, make the changes locally,
run `make check` again to ensure all tests and linting are passing, and then create a new commit
that describes what feedback was addressed. This commit can be formatted however you like, it will
be squashed before it is merged into main.
run `make check` again to ensure all tests and linting are passing, and then create a new commit
that describes what feedback was addressed. This commit can be formatted however you like, it will
be squashed before it is merged into main.
1. When your pull request is approved, it can be closed by using the 'Squash and Merge' button to
squash all of the commits into a single one that refers to both the issue and the pull request and
contains any additional descriptive information.
squash all of the commits into a single one that refers to both the issue and the pull request and
contains any additional descriptive information.
1. Thank you for submitting changes to Experimenter :D
@ -69,6 +70,7 @@ The subject should follow the this pattern:
`fixes #github_issue_number type(scope): Description`
#### Type
One of the following
- **feat**: A new feature
@ -116,15 +118,17 @@ This commit
Just as in the **subject**, use the imperative, present tense: "change" not "changed" nor "changes". Commits are expected to follow this format.
## Dependency Management
Dependencies are automatically updated by [Dependabot](https://dependabot.com/) which is now integrated
directly into GitHub. Each week Dependabot will create a large number of individual PRs that update
directly into GitHub. Each week Dependabot will create a large number of individual PRs that update
each dependency in each of the [Experimenter](https://github.com/mozilla/experimenter) and
[Experimenter Docs](https://github.com/mozilla/experimenter-docs) repos. To merge those into the `main` branch, use the following process:
[Experimenter Docs](https://github.com/mozilla/experimenter-docs) repos. To merge those into the `main` branch, use the following process:
### Merge Dependabot PRs
#### Manually
1. Dependabot will create many individual PRs against the `main` branch, which must pass all CI checks and be approved before merging. Approve each PR with the following comment and if they pass CI they will merge automatically:
1. Dependabot will create many individual PRs against the `main` branch, which must pass all CI checks and be approved before merging. Approve each PR with the following comment and if they pass CI they will merge automatically:
```
@dependabot squash and merge
@ -133,6 +137,7 @@ each dependency in each of the [Experimenter](https://github.com/mozilla/experim
All done!
#### Automatically
1. Install and configure the [Github CLI](https://github.com/cli/cli)
1. From your local Experimenter repo run
@ -141,29 +146,32 @@ make dependabot_approve
```
### Failed Dependabot PRs
If a Dependabot PR fails the CI checks you can either investigate the failure and see if it can be resolved quickly/easily, or close it altogether.
### Security Warnings
Dependabot will also produce [Security Advisories](https://github.com/mozilla/experimenter/security/dependabot) for packages that have registered [CVE](https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures) numbers. These can not be resolved automatically. To resolve the security warnings:
1. Copy the **Remediation** version from the security warning into the `"resolutions"` section of `app/package.json`, example:
Dependabot will also produce [Security Advisories](https://github.com/mozilla/experimenter/security/dependabot) for packages that have registered [CVE](https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures) numbers. These can not be resolved automatically. To resolve the security warnings:
```js
"resolutions": {
"postcss": "^7.0.36",
```
1. Copy the **Remediation** version from the security warning into the `"resolutions"` section of `experimenter/package.json`, example:
```js
"resolutions": {
"postcss": "^7.0.36",
```
1. Update the `yarn.lock` file by running
```sh
yarn install
```
```sh
yarn install
```
1. Commit your changes in a PR titled `chore(deps): Security <list affected packages>`
1. Create a PR and request review
1. Merge when approved
## Continuous Deployment Process
When a PR is merged into main it will automatically be deployed to the stage instance and if that
is successful then it will automatically be deployed to production.
@ -171,23 +179,23 @@ If the deployment fails at one of the stage/production steps, ops will automatic
If the deployment succeeds but the change inadvertently breaks stage/production, there are two options:
* Revert the change using a revert commit
- Revert the change using a revert commit
1. Create a branch from main called `revert-#123` where `#123` is the number of the issue, and then
revert the commit with `git revert <squashed commit hash>`. Fix any potential conflicts or lint/test
failures and finish the revert commit.
revert the commit with `git revert <squashed commit hash>`. Fix any potential conflicts or lint/test
failures and finish the revert commit.
1. If the original PR included any database migrations, they must be preserved and reverted by creating a new
subsequent migration
subsequent migration
* `git checkout origin/main -- app/experimenter/experiments/migrations/XXXX_migration_file.py`
- `git checkout origin/main -- experimenter/experimenter/experiments/migrations/XXXX_migration_file.py`
* `make makemigrations`
- `make makemigrations`
* `git add .;git commit -m 'Reverted migration'`
- `git add .;git commit -m 'Reverted migration'`
1. Push the branch and create a PR as normal following the above PR process
* Fix the change following the normal issue/PR processes above
- Fix the change following the normal issue/PR processes above
Because all changes will go to stage and prod automatically, you can use stage to validate your changes.

Просмотреть файл

@ -9,7 +9,7 @@ services:
- FIREFOX_VERSION
volumes:
- .:/code
- /code/app/tests/integration/.tox
- /code/experimenter/tests/integration/.tox
links:
- nginx
- kinto
@ -28,13 +28,12 @@ services:
- PYTHONPATH
volumes:
- .:/code
- /code/app/tests/integration/.tox
- /code/experimenter/tests/integration/.tox
working_dir: /code
links:
- nginx
- kinto
ping-server:
build: app/tests/integration/nimbus/utils/ping_server
build: experimenter/tests/integration/nimbus/utils/ping_server
ports:
- "5000:5000"

Просмотреть файл

@ -2,17 +2,17 @@ version: "3"
services:
yarn-legacy:
image: app:dev
image: experimenter:dev
env_file: .env
tty: true
volumes:
- ./app:/app
- media_volume:/app/experimenter/media
- /app/experimenter/legacy/legacy-ui/core/.cache/
- /app/experimenter/legacy/legacy-ui/core/node_modules/
- /app/experimenter/nimbus-ui/node_modules/
- /app/experimenter/served/
- /app/node_modules/
- ./experimenter:/experimenter
- media_volume:/experimenter/experimenter/media
- /experimenter/experimenter/legacy/legacy-ui/core/.cache/
- /experimenter/experimenter/legacy/legacy-ui/core/node_modules/
- /experimenter/experimenter/nimbus-ui/node_modules/
- /experimenter/experimenter/served/
- /experimenter/node_modules/
command: bash -c "yarn workspace @experimenter/core watch"
volumes:

Просмотреть файл

@ -1,8 +1,8 @@
version: "3"
services:
app:
image: app:deploy
experimenter:
image: experimenter:deploy
env_file: .env
environment:
- DEBUG=True
@ -16,30 +16,30 @@ services:
ports:
- "7001:7001"
volumes:
- media_volume:/app/experimenter/media
command: bash -c "python bin/setup_kinto.py;/app/bin/wait-for-it.sh db:5432 -- python manage.py collectstatic --noinput&&gunicorn -w 4 -b 0.0.0.0:7001 experimenter.wsgi"
- media_volume:/experimenter/experimenter/media
command: bash -c "python bin/setup_kinto.py;/experimenter/bin/wait-for-it.sh db:5432 -- python manage.py collectstatic --noinput&&gunicorn -w 4 -b 0.0.0.0:7001 experimenter.wsgi"
worker:
image: app:deploy
image: experimenter:deploy
env_file: .env
links:
- db
- redis
command: bash -c "/app/bin/wait-for-it.sh db:5432 -- celery -A experimenter worker -l DEBUG"
command: bash -c "/experimenter/bin/wait-for-it.sh db:5432 -- celery -A experimenter worker -l DEBUG"
beat:
image: app:deploy
image: experimenter:deploy
env_file: .env
links:
- db
- redis
command: bash -c "/app/bin/wait-for-it.sh db:5432 -- celery -A experimenter beat --pidfile /celerybeat.pid -s /celerybeat-schedule -l DEBUG"
command: bash -c "/experimenter/bin/wait-for-it.sh db:5432 -- celery -A experimenter beat --pidfile /celerybeat.pid -s /celerybeat-schedule -l DEBUG"
nginx:
build: ./nginx
env_file: .env
links:
- app
- experimenter
ports:
- "443:443"

Просмотреть файл

@ -1,8 +1,8 @@
version: "3"
services:
app:
image: app:test
experimenter:
image: experimenter:test
env_file: .env.sample
environment:
- DEBUG=False

Просмотреть файл

@ -1,8 +1,8 @@
version: "3"
services:
app:
image: app:dev
experimenter:
image: experimenter:dev
env_file: .env
stdin_open: true
tty: true
@ -13,60 +13,60 @@ services:
ports:
- "7001:7001"
volumes:
- ./app:/app
- media_volume:/app/experimenter/media
- /app/experimenter/legacy/legacy-ui/core/.cache/
- /app/experimenter/legacy/legacy-ui/core/node_modules/
- /app/experimenter/nimbus-ui/node_modules/
- /app/experimenter/served/
- /app/node_modules/
- ./experimenter:/experimenter
- media_volume:/experimenter/experimenter/media
- /experimenter/experimenter/legacy/legacy-ui/core/.cache/
- /experimenter/experimenter/legacy/legacy-ui/core/node_modules/
- /experimenter/experimenter/nimbus-ui/node_modules/
- /experimenter/experimenter/served/
- /experimenter/node_modules/
- ${GOOGLE_ADC_FILE}:${GOOGLE_APPLICATION_CREDENTIALS}:ro
command: bash -c "python bin/setup_kinto.py;/app/bin/wait-for-it.sh db:5432 -- python /app/manage.py runserver 0:7001"
command: bash -c "python bin/setup_kinto.py;/experimenter/bin/wait-for-it.sh db:5432 -- python /experimenter/manage.py runserver 0:7001"
yarn-nimbus-ui:
image: app:dev
image: experimenter:dev
env_file: .env
tty: true
ports:
- "3000:3000"
volumes:
- ./app:/app
- media_volume:/app/experimenter/media
- /app/experimenter/legacy/legacy-ui/core/.cache/
- /app/experimenter/legacy/legacy-ui/core/node_modules/
- /app/experimenter/nimbus-ui/node_modules/
- /app/experimenter/served/
- /app/node_modules/
- ./experimenter:/experimenter
- media_volume:/experimenter/experimenter/media
- /experimenter/experimenter/legacy/legacy-ui/core/.cache/
- /experimenter/experimenter/legacy/legacy-ui/core/node_modules/
- /experimenter/experimenter/nimbus-ui/node_modules/
- /experimenter/experimenter/served/
- /experimenter/node_modules/
- ${GOOGLE_ADC_FILE}:${GOOGLE_APPLICATION_CREDENTIALS}:ro
command: bash -c "FORCE_COLOR=true yarn workspace @experimenter/nimbus-ui start | cat"
worker:
image: app:dev
image: experimenter:dev
env_file: .env
links:
- db
- redis
volumes:
- ./app:/app
- ./experimenter:/experimenter
- ${GOOGLE_ADC_FILE}:${GOOGLE_APPLICATION_CREDENTIALS}:ro
command: bash -c "/app/bin/wait-for-it.sh db:5432 -- celery -A experimenter worker -l DEBUG"
command: bash -c "/experimenter/bin/wait-for-it.sh db:5432 -- celery -A experimenter worker -l DEBUG"
beat:
image: app:dev
image: experimenter:dev
env_file: .env
links:
- db
- redis
volumes:
- ./app:/app
- ./experimenter:/experimenter
- ${GOOGLE_ADC_FILE}:${GOOGLE_APPLICATION_CREDENTIALS}:ro
command: bash -c "/app/bin/wait-for-it.sh db:5432 -- celery -A experimenter beat --pidfile /celerybeat.pid -s /celerybeat-schedule -l DEBUG"
command: bash -c "/experimenter/bin/wait-for-it.sh db:5432 -- celery -A experimenter beat --pidfile /celerybeat.pid -s /celerybeat-schedule -l DEBUG"
nginx:
build: ./nginx
env_file: .env
links:
- app
- experimenter
ports:
- "443:443"

Просмотреть файл

Просмотреть файл

Просмотреть файл

Просмотреть файл

@ -4,16 +4,16 @@ FROM alpine:3.12.0 as file-loader
# such as permissions, timestamps, etc, all files are copied into a container and their
# permissions and timestamps are reset to consistent values
# Credit: https://gist.github.com/kekru/8ac61cd87536a4355220b56ae2f4b0a9
COPY . /app/
RUN chmod -R 555 /app \
&& chown -R root:root /app \
&& find /app -exec touch -a -m -t 201512180130.09 {} \;
COPY . /experimenter/
RUN chmod -R 555 /experimenter \
&& chown -R root:root /experimenter \
&& find /experimenter -exec touch -a -m -t 201512180130.09 {} \;
# Dev image
FROM python:3.11.1 AS dev
WORKDIR /app
WORKDIR /experimenter
# Disable python pyc files
@ -21,8 +21,8 @@ ENV PYTHONDONTWRITEBYTECODE 1
# Scripts for waiting for the db and setting up kinto
COPY --from=file-loader /app/bin/ /app/bin/
RUN chmod +x /app/bin/wait-for-it.sh
COPY --from=file-loader /experimenter/bin/ /experimenter/bin/
RUN chmod +x /experimenter/bin/wait-for-it.sh
# Install nvm with node and npm
@ -46,8 +46,8 @@ RUN apt-get --no-install-recommends install -y apt-utils ca-certificates postgre
RUN curl -sSL https://install.python-poetry.org | python3 -
ENV PATH "/root/.local/bin:$PATH"
RUN poetry config virtualenvs.create false
COPY --from=file-loader /app/pyproject.toml /app/pyproject.toml
COPY --from=file-loader /app/poetry.lock /app/poetry.lock
COPY --from=file-loader /experimenter/pyproject.toml /experimenter/pyproject.toml
COPY --from=file-loader /experimenter/poetry.lock /experimenter/poetry.lock
RUN poetry install
# If any package is installed, that is incompatible by version, this command
@ -56,19 +56,19 @@ RUN poetry check
# Node packages
COPY --from=file-loader /app/package.json /app/package.json
COPY --from=file-loader /app/yarn.lock /app/yarn.lock
COPY --from=file-loader /app/experimenter/legacy/legacy-ui/core/package.json /app/experimenter/legacy/legacy-ui/core/package.json
COPY --from=file-loader /experimenter/package.json /experimenter/package.json
COPY --from=file-loader /experimenter/yarn.lock /experimenter/yarn.lock
COPY --from=file-loader /experimenter/experimenter/legacy/legacy-ui/core/package.json /experimenter/experimenter/legacy/legacy-ui/core/package.json
RUN yarn install --frozen-lockfile
COPY --from=file-loader /app/experimenter/nimbus-ui/package.json /app/experimenter/nimbus-ui/package.json
COPY --from=file-loader /experimenter/experimenter/nimbus-ui/package.json /experimenter/experimenter/nimbus-ui/package.json
RUN yarn install --frozen-lockfile
FROM dev AS test
# Copy source
COPY --from=file-loader /app/ /app/
COPY --from=file-loader /experimenter/ /experimenter/
# Build image
@ -76,15 +76,15 @@ FROM dev AS ui
# Build assets
COPY --from=file-loader /app/experimenter/legacy/legacy-ui/ /app/experimenter/legacy/legacy-ui/
COPY --from=file-loader /experimenter/experimenter/legacy/legacy-ui/ /experimenter/experimenter/legacy/legacy-ui/
RUN yarn workspace @experimenter/core build
COPY --from=file-loader /app/experimenter/nimbus-ui/ /app/experimenter/nimbus-ui/
COPY --from=file-loader /experimenter/experimenter/nimbus-ui/ /experimenter/experimenter/nimbus-ui/
RUN yarn workspace @experimenter/nimbus-ui build
# Deploy image
FROM python:3.11.1-slim AS deploy
WORKDIR /app
WORKDIR /experimenter
EXPOSE 7001
# Disable python pyc files
@ -103,8 +103,8 @@ RUN apt-get --no-install-recommends install -y apt-utils ca-certificates postgre
# Copy source from previously built containers
COPY --from=dev /usr/local/bin/ /usr/local/bin/
COPY --from=dev /usr/local/lib/python3.11/site-packages/ /usr/local/lib/python3.11/site-packages/
COPY --from=dev /app/bin/ /app/bin/
COPY --from=file-loader /app/manage.py /app/manage.py
COPY --from=file-loader /app/experimenter/ /app/experimenter/
COPY --from=ui /app/experimenter/legacy/legacy-ui/assets/ /app/experimenter/legacy/legacy-ui/assets/
COPY --from=ui /app/experimenter/nimbus-ui/build/ /app/experimenter/nimbus-ui/build/
COPY --from=dev /experimenter/bin/ /experimenter/bin/
COPY --from=file-loader /experimenter/manage.py /experimenter/manage.py
COPY --from=file-loader /experimenter/experimenter/ /experimenter/experimenter/
COPY --from=ui /experimenter/experimenter/legacy/legacy-ui/assets/ /experimenter/experimenter/legacy/legacy-ui/assets/
COPY --from=ui /experimenter/experimenter/nimbus-ui/build/ /experimenter/experimenter/nimbus-ui/build/

Просмотреть файл

Просмотреть файл

Просмотреть файл

Просмотреть файл

Просмотреть файл

Просмотреть файл

@ -8,7 +8,7 @@
The front-end for Experimenter is currently composed of two applications. The application currently being used in production for legacy experimentation, survey, and delivery channels including Normandy pref-flips, roll-outs, and heartbeat surveys, lives at `app/experimenter/static/core`. It is composed of a mix of Django templates, vanilla JS, jQuery, and React components, and uses a combination of Bootstrap CSS, `react-bootstrap`, and custom CSS. A newer React application for Nimbus experiments (previously named Rapid experiments) can be found at `app/experimenter/static/rapid` and uses only Bootstrap CSS.
With the decision made to build the Nimbus console as part of the Experimenter codebase ([see ADR #0001](https://github.com/mozilla/experimenter/blob/main/app/experimenter/docs/adrs/0001-build-nimbus-console-in-experimenter.md)), it was determined we likely want a "new front-end." After that ADR was merged, through discussions it was agreed upon that [graphene-django](https://github.com/graphql-python/graphene-django) would be used in the back-end, allowing the use of GQL in the Nimbus UI.
With the decision made to build the Nimbus console as part of the Experimenter codebase ([see ADR #0001](https://github.com/mozilla/experimenter/blob/main/experimenter/experimenter/docs/adrs/0001-build-nimbus-console-in-experimenter.md)), it was determined we likely want a "new front-end." After that ADR was merged, through discussions it was agreed upon that [graphene-django](https://github.com/graphql-python/graphene-django) would be used in the back-end, allowing the use of GQL in the Nimbus UI.
This ADR, while slightly atypical as it does not serve to document a single architectural change, outlines a proposed directory tree change regarding the front-end, a proposed new front-end stack, how it compares to the current `rapid` stack, and how we will benefit from these changes.
@ -128,6 +128,6 @@ Ultimately, Bootstrap was chosen to stick around because Bootstrap CSS and `reac
## Links
[ADR #0001](https://github.com/mozilla/experimenter/blob/main/app/experimenter/docs/adrs/0001-build-nimbus-console-in-experimenter.md) - documented decision to build Nimbus in this repository with the existing Django back-end\
[ADR #0001](https://github.com/mozilla/experimenter/blob/main/experimenter/experimenter/docs/adrs/0001-build-nimbus-console-in-experimenter.md) - documented decision to build Nimbus in this repository with the existing Django back-end\
[graphene-django](https://github.com/graphql-python/graphene-django) - package to allow GraphQL in Python/Django\
[FxA Settings README](https://github.com/mozilla/fxa/tree/main/packages/fxa-settings) - well-documented project with a similar front-end stack that can be referenced for examples with other relevant ADRs (GQL, React Toolchain) linked at the top

Просмотреть файл

@ -142,7 +142,7 @@ No set up required. Doc maintainers will go to the Experimenter Wiki page and ma
[Storybook](https://github.com/storybookjs/storybook) is a versatile package that can not only be used to aid in UI development, but can also be used for various kinds of guides and documentation.
We currently use Storybook in [`nimbus-ui`](https://github.com/mozilla/experimenter/tree/main/app/experimenter/nimbus-ui), which at the time of writing has a small documentation section, and builds are automatically deployed to [`mozilla-storybooks-experimenter`](https://storage.googleapis.com/mozilla-storybooks-experimenter/index.html) per commit.
We currently use Storybook in [`nimbus-ui`](https://github.com/mozilla/experimenter/tree/main/experimenter/experimenter/nimbus-ui), which at the time of writing has a small documentation section, and builds are automatically deployed to [`mozilla-storybooks-experimenter`](https://storage.googleapis.com/mozilla-storybooks-experimenter/index.html) per commit.
Pros

Просмотреть файл

@ -6,7 +6,7 @@
## Context and Problem Statement
ADR #5, ["Use Docusaurus + GH Pages for the Nimbus User Doc Hub"](https://github.com/mozilla/experimenter/blob/main/app/experimenter/docs/adrs/0005-doc-hub.md), determined that our approach to setting up a new central location for Experimenter/Nimbus and related project documentation would use [Docusaurus](https://v2.docusaurus.io/), a static site generator, with the static files being served from [GitHub Pages](https://pages.github.com/). While we knew we would be introducing a new set of code we didn't explicitly decide where it would live and what the process for deploying would look like, beyond suggesting it could live in the existing Experimenter repository.
ADR #5, ["Use Docusaurus + GH Pages for the Nimbus User Doc Hub"](https://github.com/mozilla/experimenter/blob/main/experimenter/experimenter/docs/adrs/0005-doc-hub.md), determined that our approach to setting up a new central location for Experimenter/Nimbus and related project documentation would use [Docusaurus](https://v2.docusaurus.io/), a static site generator, with the static files being served from [GitHub Pages](https://pages.github.com/). While we knew we would be introducing a new set of code we didn't explicitly decide where it would live and what the process for deploying would look like, beyond suggesting it could live in the existing Experimenter repository.
In investigating how to set all of this up we realized there would be certain caveats to hosting and deploying this code from the Experimenter repository, and this lead to the consideration of using a dedicated repository. This ADR outlines our options and end decision.
@ -32,7 +32,7 @@ In investigating how to set all of this up we realized there would be certain ca
### Use the Experimenter repository, main branch
This decision would have us use the `mozilla/experimenter` repository, housing the code and documentation files in the `main` branch (likely in a nested `/app/docs` directory), and the built/static files would be committed to and served from a `gh-pages` branch.
This decision would have us use the `mozilla/experimenter` repository, housing the code and documentation files in the `main` branch (likely in a nested `/experimenter/docs` directory), and the built/static files would be committed to and served from a `gh-pages` branch.
Pros
@ -44,9 +44,9 @@ Cons
- There is a caveat to "pulling in the latest changes and adding new files": this doesn't necessarily apply to drive-by contributors. If you are someone who regularly works with the Experimenter codebase this should be a familiar process, but if you are someone who only wants to contribute simple documentation or perhaps suggest a change you'll need to first understand what you're looking at [in the repository] and where everything is located.
- The repository is set up to run a set of test and build CircleCI jobs any time a Pull Request is opened (initially, then on new pushes), and again when merging into `main`. They can be expensive with repeated runs, take a non-trivial amount of time, and are subject to hiccups and flaky tests. As such we would likely need to modify our existing CircleCI config to not run certain jobs if changes only occurred in documentation-related files, which could be a considerable amount of work.
- We _could_ alternatively ignore this altogether; we're under our CircleCI spend and in theory documentation-related runs wouldn't eat up too many credits. This would need to be monitored and evaluated at a later time.
- If we want to utilise Experimenter's existing Yarn workspaces it means we need to place the Docusaurus code underneath the code's `/app` directory as this is where the root Yarn project is. This isn't as ideal as having a `/docs-hub` directory at the root of the project, which would be better for contributors less familiar with the rest of Experimenter.
- It's also a possibility that we don't need to use Yarn workspaces for this. The downside to that is that we'd then have two distinct Yarn installation processes and `node_modules` directories.
- We _could_ alternatively ignore this altogether; we're under our CircleCI spend and in theory documentation-related runs wouldn't eat up too many credits. This would need to be monitored and evaluated at a later time.
- If we want to utilise Experimenter's existing Yarn workspaces it means we need to place the Docusaurus code underneath the code's `/experimenter` directory as this is where the root Yarn project is. This isn't as ideal as having a `/docs-hub` directory at the root of the project, which would be better for contributors less familiar with the rest of Experimenter.
- It's also a possibility that we don't need to use Yarn workspaces for this. The downside to that is that we'd then have two distinct Yarn installation processes and `node_modules` directories.
- This hub is intended to provide documentation for a variety of services, and is not just scoped to Experimenter Console. Given that we already have other separate familial repositories it might seem odd to house this hub inside `mozilla/experimenter`.
- Precludes us from using the repository's GitHub Pages path for another project that might be more specific to Experimenter Console.
@ -89,8 +89,8 @@ Cons
- Adds some cultural overhead as we are introducing Yet Another Repository for individuals to get familiar with, set up notifications for, etc.
- Using a repository separate from `mozilla/experimenter`, or any established repository, involves some technical overhead. This means ensuring correct permissions for contributors, setting up or finding alternate solutions to integrations (e.g. Sentry, Slack, Jira, Dependabot) we decide we need, and individuals needing to clone and install dependencies for the new codebase. We also run the risk of encountering unknowns that have been addressed over time with mature repository.
- An example of finding an alternate solution to an integration is Jira, and specifically filing issues. Instead of connecting this new repository with Jira we could disable issue filing on the repository and require any new issues be filed under the EXP project in Jira, having them sync to `mozilla/experimenter`.
- When it comes to repository permissions, granting edit access to the [`@mozilla/project-nimbus`](https://github.com/orgs/mozilla/teams/project-nimbus/members) team seems like the easiest approach.
- An example of finding an alternate solution to an integration is Jira, and specifically filing issues. Instead of connecting this new repository with Jira we could disable issue filing on the repository and require any new issues be filed under the EXP project in Jira, having them sync to `mozilla/experimenter`.
- When it comes to repository permissions, granting edit access to the [`@mozilla/project-nimbus`](https://github.com/orgs/mozilla/teams/project-nimbus/members) team seems like the easiest approach.
- You would not be able to merge changes to both documentation and the Experimenter codebase in the same Pull Request.
## Links

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 674 KiB

После

Ширина:  |  Высота:  |  Размер: 674 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 435 KiB

После

Ширина:  |  Высота:  |  Размер: 435 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 278 KiB

После

Ширина:  |  Высота:  |  Размер: 278 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 445 KiB

После

Ширина:  |  Высота:  |  Размер: 445 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 557 KiB

После

Ширина:  |  Высота:  |  Размер: 557 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 440 KiB

После

Ширина:  |  Высота:  |  Размер: 440 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 435 KiB

После

Ширина:  |  Высота:  |  Размер: 435 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 195 KiB

После

Ширина:  |  Высота:  |  Размер: 195 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 350 KiB

После

Ширина:  |  Высота:  |  Размер: 350 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 455 KiB

После

Ширина:  |  Высота:  |  Размер: 455 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 388 KiB

После

Ширина:  |  Высота:  |  Размер: 388 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 256 KiB

После

Ширина:  |  Высота:  |  Размер: 256 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 440 KiB

После

Ширина:  |  Высота:  |  Размер: 440 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 487 KiB

После

Ширина:  |  Высота:  |  Размер: 487 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 302 KiB

После

Ширина:  |  Высота:  |  Размер: 302 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 294 KiB

После

Ширина:  |  Высота:  |  Размер: 294 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 251 KiB

После

Ширина:  |  Высота:  |  Размер: 251 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 282 KiB

После

Ширина:  |  Высота:  |  Размер: 282 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 415 KiB

После

Ширина:  |  Высота:  |  Размер: 415 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 422 KiB

После

Ширина:  |  Высота:  |  Размер: 422 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 180 KiB

После

Ширина:  |  Высота:  |  Размер: 180 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 413 KiB

После

Ширина:  |  Высота:  |  Размер: 413 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 422 KiB

После

Ширина:  |  Высота:  |  Размер: 422 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 183 KiB

После

Ширина:  |  Высота:  |  Размер: 183 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 296 KiB

После

Ширина:  |  Высота:  |  Размер: 296 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 413 KiB

После

Ширина:  |  Высота:  |  Размер: 413 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 390 KiB

После

Ширина:  |  Высота:  |  Размер: 390 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 250 KiB

После

Ширина:  |  Высота:  |  Размер: 250 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 413 KiB

После

Ширина:  |  Высота:  |  Размер: 413 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 495 KiB

После

Ширина:  |  Высота:  |  Размер: 495 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 167 KiB

После

Ширина:  |  Высота:  |  Размер: 167 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 169 KiB

После

Ширина:  |  Высота:  |  Размер: 169 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 170 KiB

После

Ширина:  |  Высота:  |  Размер: 170 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 142 KiB

После

Ширина:  |  Высота:  |  Размер: 142 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 48 KiB

После

Ширина:  |  Высота:  |  Размер: 48 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 380 KiB

После

Ширина:  |  Высота:  |  Размер: 380 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 56 KiB

После

Ширина:  |  Высота:  |  Размер: 56 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 289 KiB

После

Ширина:  |  Высота:  |  Размер: 289 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 289 KiB

После

Ширина:  |  Высота:  |  Размер: 289 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 300 KiB

После

Ширина:  |  Высота:  |  Размер: 300 KiB

Просмотреть файл

До

Ширина:  |  Высота:  |  Размер: 252 KiB

После

Ширина:  |  Высота:  |  Размер: 252 KiB

Некоторые файлы не были показаны из-за слишком большого количества измененных файлов Показать больше