Because
* We have seen failures while pulling from dockerhub tags
* This may be caused by the tag being corrupted after being in use for
so long
* Let's try pushing to a new tag to see if we see the same failures
This commit
* Pushes to a new experimenter tag in dockerhub
fixes#9752
Because:
- #9712 broke integration tests on fork PRs by referencing
$DOCKERHUB_REPO, which is not available in fork PRs
this commit:
- hardcodes the dockerhub repo (mozilla/experimenter) for pulling the
image, which will allow fork PRs to run integration tests.
Fixes#9716
Because:
- #9694 broke integration tests;
This commit
- reverts #9694; and
- adds an explicit pull of our integration test image in our CI
workflow.
Fixes#9707
Because
- we don't want to manually build application-services megazords
ourselves for each container; and
- application-services publishes megazord artifacts on Taskcluster
This commit
- adds a new as-builder docker container to download cirrus and nimbus
megazords with the appropriate binaries for the platform as Python
packages;
- updates the cirrus and experimenter containers to copy the megazords
from the as-builder container;
- updates cirrus and experimenter to refer to the new megazord packages;
- updates cirrus to be based on Debian Bullseye so it can use the
generated binaries; and
- updates the Dockerfile-rust-image container to be built on top of
as-builder in CI.
Becuase
* We were evaluating Docker Hydrobuild as part of its beta program
* We did not find that it improved CI times
This commit
* Removes Docker Hydrobuild from CI
Co-authored-by: Yashika Khurana <yashikakhuranayashika@gmail.com>
Because
- We have a demo app that can test the integration end to end, it has
the frontend and backend.
- We could stand up the experimenter, launch the experiment or rollout,
and push to RS, Cirrus can sync to RS and then the frontend can send a
request to the backend and then backend can reach out to Cirrus and can
ask for the feature config
- When the backend receives the response, can send the response back to
the frontend
This commit
- Tests end-to-end integration between the experimenter, RS, Cirrus, and
Demo app
- Test both the experiment and rollout end to end
fixes#9387
Because
* As we add a wider variety of elements to the Nimbus project including
binary packages it becomes increasingly important to have a consistent
environment for each of our modules
* Using Docker allows us to control the environment each module runs in
and use a consistent base layer
* Dockerizing the schemas package will allow us to use identical Python,
Node, and Rust environments as the other modules in Nimbus
This commit
* Wraps the schemas module in a Dockerfile that is consistent with the
other modules in Nimbus
Because
* We've been seeing consistent intermittent failures with pulling images
from dockerhub
* An alternative is to use the Google Container Registry instead of
Dockerhub
* If we have better luck with Google Container Registry then we can use
that and deprecate Dockerhub
This commit
* Pushes to Google Container Registry in addition to Dockerhub
Because
* We enabled the experimental Docker Hydrobuild remote builder for our
CI jobs
* Hydrobuild experienced an outage that prevented our CI jobs from
running
* We disabled Hydrobuild until it could be rectified by the Docker team
* Fixes have been deployed and it should now be safe to re enable
This commit
* Re enables Docker Hydrobuild in CI
Because
* When we disabled hydrobuild we also disabled logging into docker
* Logging into docker is necessary at the deploy step
This commit
* Logs into docker if possible but disables hydrobuild
Because
* We disabled Hydrobuild in #9535
* We must also disable the multiplatform builds in the deploy step if
Hydrobuild is unavailable
This commit
* Disables multiplatform builds in the deploy steps of Experimenter and
Cirrus
Because
* Docker Hydrobuild has stopped functioning and is blocking our CI tasks
This commit
* Disables Docker Hydrobuild until it is functioning and we can re
enable it
Because
* We recently moved the FML parsing into the Experimenter container as
part of fetching external configs
* The Experimenter container requires an environment file to start
This commit
* Uses the .env.sample file as part of the fetch_external_configs circle
task
Because
* We now include binary assets that must target both amd64 and arm64
* We must verify that these are available and functioning in CI
This commit
* Adds an arm64 instance for cirrus and experimenter check to verify
that our containers will build and run on both platforms
* Disables path checks for changes in the root project dir or .circleci
folder or on main branch so that we run all tests if anything project
wide changes
* Removes now redundant branch/main copies of experimenter since path
checks will be disabled on main
Because
* We have many tasks in Circle that depend on a Docker account
* The docker account depends on which module is being tested ie
Experimenter, Cirrus
* We should explicitly parameterize the Docker setup step with the
credentials needed for that module
This commit
* Parameterizes the setup_docker step in circle
* Uses the appropriate credentials for each of Experimenter and Cirrus
in each circle task
Because
* The docker login step for deploying cirrus was removed in a previous
pr
* This login step is necessary becuase cirrus uses different credentials
than experimenter to deploy to dockerhub
This commit
* Restores the docker login command to the cirrus deploy step
Because
* Docker Hydrobuild requires access to environment secrets
* PRs from forks will not have access to environment secrets
This commit
* Disables hydrobuild for PRs that do not have access to secrets
* Re enables Circle docker layer caching
---------
Co-authored-by: Mike Williams <102263964+mikewilli@users.noreply.github.com>
Because
* We now have access to Docker Hydrobuild
* This allows us to use a shared cloud builder for our docker images so
we can share a build cache between CI and local dev
This commit
* Updates the makefile to explicitly use docker buildx
* Updates the circle config to install the latest docker client and
setup a cloud builder
Because
- we want to use the population sizing schemas for graphql and UI
This commit
- sets up generation of typescript schemas from the pydantic schemas
- sets up publishing the typescript schemas to npm
Because
- Enrollment metrics are necessary for experiment analysis
This commit
- Defines the method by which the Cirrus container records enrollment
metrics
---------
Co-authored-by: yashikakhurana <yashikakhuranayashika@gmail.com>
Because
* We recently renamed some commands in the circle config for consistency
* We accidentally broke the make command for checking the schemas
* It was missed because they didn't run becuase they failed the path
check
This commit
* Fixes the make check schemas command in the circle config
Because
* Now that the cirrus container is ready for use we need to deploy it to
dockerhub
This commit
* Adds a circle step to build and deploy the cirrus container to
dockerhub
Co-authored-by: yashikakhurana <yashikakhuranayashika@gmail.com>
Because
- twine isn't installed until poetry installs dependencies
- `poetry build` doesn't install dependencies
This commit
- adds the `schemas_install` to the deploy's prereqs
Because
- the circle config was assuming package directory
- the circle working dir was not set
- the circle image could not find the package to build/deploy
This commit
- uses make commands that drop into the correct directory before
building and deploying
Because
* We recently refactored the circleci config to update the path checks
* In the course of doing so we removed the command to actually run the
experimenter unit tests. Whoops!
This commit
* Restores the make check command to the experimenter unit tests in
circle
Becuase
* The experimenter repo now hosts many projects
* Not all tests will be related to changes in all of those folders
* We can conditionally check which tests to run based on the paths of
changes in a PR
This commit
* Updates all of the file path checks on all of the circleci tasks so we
don't run unnecessary tests in PRs
Because
- We want to have a better data contract between Jetstream and
Experimenter
This commit
- Creates schemas for the data products that Jetstream produces
- Adds a general python package structure with basic schema tests
Because
* We run make check on each branch but also on main before deploys
* This last check is to make sure that main is still in a healthy state
before it is deployed
* We added path filters to prevent certain circle tasks from running
depending on which files were changed
* This path filter was preventing make check from running on main
This commit
* Splits the check job into two, one for branches that includes the path
check, and one for main that excludes it
Because
* We've been seeing intermittent failures in the enrollment integration
tests
* It may be caused by parallel tests colliding when they reset the ping
server cache
This commit
* Disables parallel runs for the enrollment integration tests
Because:
* Diagnosing errors on CI can be hard without seeing what the page looks
like
This Commit:
* Adds HTML reporting which will include screenshots for debugging
errors.
Because
* We only need to run the targeting tests for changes in the experiments
or targeting apps
* We were checking those file paths in the release and beta circle runs
but not nightly
This commit
* Adds the correct file path checks to the nightly targeting run
Because:
* Having to add waits to each method can be time consuming
This Commit:
* Adds a default wait for every time we need to search for an element.
Because
- We want to add a linter on the Cirrus service and wanted to have CI
runs the cirrus check
This commit
- Add [Ruff linter](https://pypi.org/project/ruff/) 💪
- New circle ci job for `cirrus_check`
- Cirrus check - runs the test case and ruff linting
- New make command for check `make cirrus_check`
- New make command to fix formatting issues using `make
cirrus_code_format`
- Circle ci cirrus check will only run if the changes are in `/cirrus`
directory
- Experimenter check will only run if the changes are in `/experimenter`
directory
- Remove requirements.txt and uses poetry only
closes#8386#8388
Because
* Now that we have so many different integration tests
* We should try to keep the naming of all the tests and circle tasks
consistent
This commit
* Renames the integration tests and circle tasks to be more consistent
Because:
* Looking over the build logs on circleci I noticed that most of the
builds weren't building
This commit:
* Fixes the docker commands so that the logic for building the image has
the needed data.
ps: I still suck at docker
Because:
* Tracking down errors within flakey integration tests can be quite
difficult.
This commit:
* Adds a library to report test errors to sentry.
With sentrys rich logging and stacktraces, this will help us identify
what is causing a flakey test much quicker than before.
Because:
* Our nimbus-rust image was falling behind on its Application services
versioning
This Commit:
* Updates the image and adds its Dockerfile to this repo
* Adds a cron job to build weekly
Because:
* I noticed the desktop targeting tests were running tests against
non-desktop targeting configs.
This Commit:
* Changes the tests to run only on desktop
* Updates the amount of parallel test runs that are ran
Because:
* Currently we run a lot of parallel jobs on circleci and it can be a
bit hard to sort through which job does what tests
This Commit:
* Changes the workflow on circleci to be more descriptive so it is
easier to see which tests are passing or failing
* fixes#7592 type(test): Added end-to-end test for telemetry verification during enrollment and unenrollment
Because
* Making sure telemetry is sent when a client enrolls and unenrolls is very important. We did not have automated tests for this though
This Commit
* Adds automated checks for telemetry enroll/unenroll events.
Since this test can take a varying amount of time to complete, I utilized pytest-xdist to group tests so that this test can be run immediately, to save time.
I also had to write a ping server to collect and verify the pings, that is included in this PR.
Because
* The circle task to update external configs was repeatedly force pushing the same changes and triggering redundant CI runs
* We added a fix to check whether the local commit is different from the latest remote commit in the same branch
* However, after an external config PR is merged, its branch is deleted
* Later when the circle task wakes up and tries to detect if there's a change, it will fail when it tries to diff against the remote branch which was previously deleted
This commit
* Attempts to diff against the remote branch and if that fails, checks for a local diff
* This should prevent it from pushing duplicate commits when a pr exists, but still allow it to create a pr when none exists
Because
* When external configs change, we will automatically commit and create a pr
* If a pr already exists, we are force pushing changes on every run which forces CI to needlessly re run
This commit
* Checks if the new commit is different from the commit that exists before force pushing to prevent CI from re running
* fix#7519 chore(project): Re-enable Legacy Integration and JS tests on CI.
Because
* We are still supporting the use of legacy for the forseable future, so we should make sure that changes added to the codebase don't affect legacy in a negative way.
This commit
* Re-enables legacy integration tests on CI.
* Re-enables legacy JS tests as part of the check job on CI.
Because
* We only need to suppress the error code of the pr creation command in the external config circle task
* If we suppress the others they could fail for some unexpected reason and we'd never be notified
This commit
* Only suppresses the exit code of the pr creation command
Because
* The hourly circle task to check for changes to external config files will fail if there are no changes
* The failures are logged and notified as circle failures
* This can mask an actual failure
This commit
* Changes the task to exit 0 if no changes are detected
Because
* We want to update the external configs for jetstream and features automatically
* We want updates to those files to be tested as part of CI
This commit
* Adds a new circle task to check for updates in external configs hourly
* If changes are detected, automatically create a PR
Because
* Now that we have integration tests for both desktop and the sdk
* We should clearly label both with what they're testing
This commit
* Renames integration_nimbus to integration_nimbus_desktop
* Renames integration_nimbus_rust to integration_nimbus_sdk
Because
* Each call to make will trigger many remote fetches
* Having multiple subsequent make calls will repeat those same fetches
* Combining multiple make calls into a single call will only make the fetches once
This commit
* Moves multiple make calls into a single make call in circle where possible
Because
* Storybook is no longer used by anyone to test or validate UI changes
* Maintaining storybooks for components adds significant cost to UI changes
This commit
* Removes storybook
Because
* At some point we had to hard code a docker version in the docs because subsequent versions were broken
* This is no longer the case
This commit
* Updates docs to point to the latest docker desktop
Because
* The legacy workflow is in maintenance only mode and has not had any changes or failures in over a year
This commit
* Moves the legacy tests out of make check but preserves them in make check_legacy
* Moves the legacy dev services out of make up but preserves them in make up_legacy
* Removes the integration_legacy job from circle
Because
* We should add a simple integration test for each new major feature we add
This commit
* Adds an integration test for archiving
* Splits the integration tests into two groups
* Tests that run for each application (for remote settings integration)
* Tests that only run once (for things constrained just to experimenter UI)
* Adds a random identifier to the end of integration test experiments to allow rerunning tests locally
* Cleans up the page load wait locators
Because
* We previously used circle docker layer caching to speed up circle runs
* While migrating to the shared build cache, some confusing errors appeared
* We disabled it to see if that cleared up the issue
* Circle builds have been stable lately using the shared layer cache, but each circle run takes a significant amount of time to pull in remote images
* It's worth trying to re enable it to see if we can get both the circle caching and shared layer cache working together properly
This commit
* Re enables circle docker layer caching
Because
* We have a fourth layer in our Dockerfile that we're not remotely caching which slows down local builds/ci
This commit
* Builds and pushes the ui build target in our Dockerfile in circle
Because
* We recently started using the new buildkit shared caching to speed up our docker builds
* Today we saw strange obscure failures that seemed to mysteriously resolve by themselves
* Circle and Dockers shared layer caching strategies may not cooperate nicely
* In theory we should be able to get away with using only one
* The Circle layer caching costs money
This commit
* Disables Circle Docker layer caching
Because
* We recently reorganized our build steps
* We moved version.json generation from the deploy step to all builds
* version.json changes on every circle invocation which can invalidate shared docker caches
This commit
* Moves version.json generation back to only during deploy
Because
* It would save time if we push the docker build layers from circleci to dockerhub to reuse locally
This commit
* Uses the new buildkit --cache-from flag to pull in build layers from dockerhub when applicable
* Builds all stages and pushes them with their layers to dockerhub on every merge to main
* Configures local builds to pull those layers from dockerhub
* Removes the build circle stage since each circle task will now be able to pull from dockerhub rather than the local circle docker cache
Because
* We now have an integration test suite for nimbus we want to enable in circle
This commit
* Enables both the legacy and nimbus integration tests in circle
Because
* I just noticed while debugging another issue that circle only has app and db containers running while it runs tests, and if you make bash or up and have other containers running like redis or kinto or autograph, those can cause stateful changes to the test environment such as if a test is attempting to invoke a celery that hasn't been mocked out, having redis running in the background will allow that test to pass, so then it looks like the test is failing in circle and passing locally becuase you have redis up which can be confusing
This commit
* Runs make kill before starting any test so that if we hit a situation like that again, local testing should more closely replicate circle
Because
* The latest version of Docker breaks with nginx 1.9.9
This commit
* Updates our circle image to ubuntu-2004:202101-01 - Ubuntu 20.04, Docker v20.10.2, Docker Compose v1.28.2
* Updates our nginx image to 1.19.8
Becuase
* We recently added a dependency on checking out the git submodule for jetstream config to our build process
* We added that dependency to the makefile
* The circle config to build the deploy container was not using the makefile entry point to build the container
* Jetstream was not being pulled in before deploy build
* The system check we added to prevent starting worked correctly!
This commit
* Switches circle to use the makefile to build the deploy container
Because
* We increased the instance size in an attempt to reduce integration test errors we were seeing in Firefox
* The increase in size didn't mitigate the errors
This commit
* This reverts commit 1ab43e6494.
Because
* We've recently seen some intermittent integration test failures
* The errors are from firefox running out of file handlers, which might indicate a resource constraint issue
This commit
* Increases the integration test circle ci instance size to mitigate
Because
* We are now migrating all visualization work out of rapid and into nimbus
* All of the legacy/core ui is in maintenance mode
* Tests take up a lot of circle credits
This commit
* Disables all but the end to end integration tests for the legacy workflow
* Disables all of the rapid/visualization js tests/linting
* Disables the rapid/visualization dev yarn processes
* Decreases the size of the make check circle instance
* This should make the local dev lighter, make our tests faster, and reduce circle usage
Because
* We disabled the `up to date with main` branch protection, it's possible for changes that work as a pr to land on main and break main
This commit
* Gates deploying on running `make check` on main so if a pr lands on main that breaks we'll catch it before it gets deployed
Because:
* We'd like to be able to compare storybooks between main branch and
other branches (e.g. PRs)
This commit
* Enables publish_storybooks for main branch in CI
Because
* It would be nice to run the storybook publishing in parallel, and
* It would be nice to have a published storybook for a PR even if it hasn't yet got passing tests or complete test coverage
This commit
* Moves storybook publishing to a separate circle ci task
Because
* It would be nice to know if the backend changed that the frontend would break
This commit
* Adds make commands to check and fix that the graphql types match the nimbus-ui types
* This will cause CI to break if a backend change would break the frontend
Because
* I changed the layout of the docker containers to have separate dev/build/deploy stages
This commit
* Update the circle task to build the deploy image
Because
* We split the docker image into two images, one for local dev and one for deploying to prod so that the prod one would be small ~100mb
* We should find some way to test that the deploy image is working correctly
* A good candidate is to run the integration tests against it
This commit
* Adds a new make up_prod command that sets up an environment that more closely mirrors prod ie
- Use the prod image
- Disable all the yarn watchers
* Reorganizes the dockerfile to move frontend asset building out of the dev container to increase docker cache hits for local dev
Because
* The circle deploy task runs after every PR lands and it's not being cached it takes ~10m to rebuild everything from scratch every time
This commit
* Enables docker layer caching for the deploy task which means in the common case it should take <1m
* Bump to the latest circle docker image
Because
* We want to shrink the size of the deployable Docker image down from its current 2.71gb(1gb compressed)
This commit
* Adds a second stage to our Dockerfile for a deploy image which is a much more reasonable 480mb(133mb compressed)
* Use deploy image for Django container in local dev compose
Because
* We want to publish Storybook builds to demo and preview component work
in nimbus-ui
This commit
* Integrates mozilla-fxa/storybook-gcp-publisher into the CircleCI flow
so that Storybooks are built and published to a Google Cloud static
site for every test run
EXP-281
fixes#3427
Because
* Running circle jobs against each individual dependency PR makes it easier to know
if a single dependency change breaks the build rather than having to only run tests
against all dependencies combined
This commit
* Enables circle jobs on dependency PRs
Jared Lockhart
Barret Rennie
Yashika Khurana
Mike Williams
Charlie
Benjamin Forehand Jr
haroldwoo
yashika-khurana
Les Orchard