* fixes#7592 type(test): Added end-to-end test for telemetry verification during enrollment and unenrollment
Because
* Making sure telemetry is sent when a client enrolls and unenrolls is very important. We did not have automated tests for this though
This Commit
* Adds automated checks for telemetry enroll/unenroll events.
Since this test can take a varying amount of time to complete, I utilized pytest-xdist to group tests so that this test can be run immediately, to save time.
I also had to write a ping server to collect and verify the pings, that is included in this PR.
Because
* The circle task to update external configs was repeatedly force pushing the same changes and triggering redundant CI runs
* We added a fix to check whether the local commit is different from the latest remote commit in the same branch
* However, after an external config PR is merged, its branch is deleted
* Later when the circle task wakes up and tries to detect if there's a change, it will fail when it tries to diff against the remote branch which was previously deleted
This commit
* Attempts to diff against the remote branch and if that fails, checks for a local diff
* This should prevent it from pushing duplicate commits when a pr exists, but still allow it to create a pr when none exists
Because
* When external configs change, we will automatically commit and create a pr
* If a pr already exists, we are force pushing changes on every run which forces CI to needlessly re run
This commit
* Checks if the new commit is different from the commit that exists before force pushing to prevent CI from re running
* fix#7519 chore(project): Re-enable Legacy Integration and JS tests on CI.
Because
* We are still supporting the use of legacy for the forseable future, so we should make sure that changes added to the codebase don't affect legacy in a negative way.
This commit
* Re-enables legacy integration tests on CI.
* Re-enables legacy JS tests as part of the check job on CI.
Because
* We only need to suppress the error code of the pr creation command in the external config circle task
* If we suppress the others they could fail for some unexpected reason and we'd never be notified
This commit
* Only suppresses the exit code of the pr creation command
Because
* The hourly circle task to check for changes to external config files will fail if there are no changes
* The failures are logged and notified as circle failures
* This can mask an actual failure
This commit
* Changes the task to exit 0 if no changes are detected
Because
* We want to update the external configs for jetstream and features automatically
* We want updates to those files to be tested as part of CI
This commit
* Adds a new circle task to check for updates in external configs hourly
* If changes are detected, automatically create a PR
Because
* Now that we have integration tests for both desktop and the sdk
* We should clearly label both with what they're testing
This commit
* Renames integration_nimbus to integration_nimbus_desktop
* Renames integration_nimbus_rust to integration_nimbus_sdk
Because
* Each call to make will trigger many remote fetches
* Having multiple subsequent make calls will repeat those same fetches
* Combining multiple make calls into a single call will only make the fetches once
This commit
* Moves multiple make calls into a single make call in circle where possible
Because
* Storybook is no longer used by anyone to test or validate UI changes
* Maintaining storybooks for components adds significant cost to UI changes
This commit
* Removes storybook
Because
* At some point we had to hard code a docker version in the docs because subsequent versions were broken
* This is no longer the case
This commit
* Updates docs to point to the latest docker desktop
Because
* The legacy workflow is in maintenance only mode and has not had any changes or failures in over a year
This commit
* Moves the legacy tests out of make check but preserves them in make check_legacy
* Moves the legacy dev services out of make up but preserves them in make up_legacy
* Removes the integration_legacy job from circle
Because
* We should add a simple integration test for each new major feature we add
This commit
* Adds an integration test for archiving
* Splits the integration tests into two groups
* Tests that run for each application (for remote settings integration)
* Tests that only run once (for things constrained just to experimenter UI)
* Adds a random identifier to the end of integration test experiments to allow rerunning tests locally
* Cleans up the page load wait locators
Because
* We previously used circle docker layer caching to speed up circle runs
* While migrating to the shared build cache, some confusing errors appeared
* We disabled it to see if that cleared up the issue
* Circle builds have been stable lately using the shared layer cache, but each circle run takes a significant amount of time to pull in remote images
* It's worth trying to re enable it to see if we can get both the circle caching and shared layer cache working together properly
This commit
* Re enables circle docker layer caching
Because
* We have a fourth layer in our Dockerfile that we're not remotely caching which slows down local builds/ci
This commit
* Builds and pushes the ui build target in our Dockerfile in circle
Because
* We recently started using the new buildkit shared caching to speed up our docker builds
* Today we saw strange obscure failures that seemed to mysteriously resolve by themselves
* Circle and Dockers shared layer caching strategies may not cooperate nicely
* In theory we should be able to get away with using only one
* The Circle layer caching costs money
This commit
* Disables Circle Docker layer caching
Because
* We recently reorganized our build steps
* We moved version.json generation from the deploy step to all builds
* version.json changes on every circle invocation which can invalidate shared docker caches
This commit
* Moves version.json generation back to only during deploy
Because
* It would save time if we push the docker build layers from circleci to dockerhub to reuse locally
This commit
* Uses the new buildkit --cache-from flag to pull in build layers from dockerhub when applicable
* Builds all stages and pushes them with their layers to dockerhub on every merge to main
* Configures local builds to pull those layers from dockerhub
* Removes the build circle stage since each circle task will now be able to pull from dockerhub rather than the local circle docker cache
Because
* We now have an integration test suite for nimbus we want to enable in circle
This commit
* Enables both the legacy and nimbus integration tests in circle
Because
* I just noticed while debugging another issue that circle only has app and db containers running while it runs tests, and if you make bash or up and have other containers running like redis or kinto or autograph, those can cause stateful changes to the test environment such as if a test is attempting to invoke a celery that hasn't been mocked out, having redis running in the background will allow that test to pass, so then it looks like the test is failing in circle and passing locally becuase you have redis up which can be confusing
This commit
* Runs make kill before starting any test so that if we hit a situation like that again, local testing should more closely replicate circle
Because
* The latest version of Docker breaks with nginx 1.9.9
This commit
* Updates our circle image to ubuntu-2004:202101-01 - Ubuntu 20.04, Docker v20.10.2, Docker Compose v1.28.2
* Updates our nginx image to 1.19.8
Becuase
* We recently added a dependency on checking out the git submodule for jetstream config to our build process
* We added that dependency to the makefile
* The circle config to build the deploy container was not using the makefile entry point to build the container
* Jetstream was not being pulled in before deploy build
* The system check we added to prevent starting worked correctly!
This commit
* Switches circle to use the makefile to build the deploy container
Because
* We increased the instance size in an attempt to reduce integration test errors we were seeing in Firefox
* The increase in size didn't mitigate the errors
This commit
* This reverts commit 1ab43e6494.
Because
* We've recently seen some intermittent integration test failures
* The errors are from firefox running out of file handlers, which might indicate a resource constraint issue
This commit
* Increases the integration test circle ci instance size to mitigate
Because
* We are now migrating all visualization work out of rapid and into nimbus
* All of the legacy/core ui is in maintenance mode
* Tests take up a lot of circle credits
This commit
* Disables all but the end to end integration tests for the legacy workflow
* Disables all of the rapid/visualization js tests/linting
* Disables the rapid/visualization dev yarn processes
* Decreases the size of the make check circle instance
* This should make the local dev lighter, make our tests faster, and reduce circle usage
Because
* We disabled the `up to date with main` branch protection, it's possible for changes that work as a pr to land on main and break main
This commit
* Gates deploying on running `make check` on main so if a pr lands on main that breaks we'll catch it before it gets deployed
Because:
* We'd like to be able to compare storybooks between main branch and
other branches (e.g. PRs)
This commit
* Enables publish_storybooks for main branch in CI
Because
* It would be nice to run the storybook publishing in parallel, and
* It would be nice to have a published storybook for a PR even if it hasn't yet got passing tests or complete test coverage
This commit
* Moves storybook publishing to a separate circle ci task
Because
* It would be nice to know if the backend changed that the frontend would break
This commit
* Adds make commands to check and fix that the graphql types match the nimbus-ui types
* This will cause CI to break if a backend change would break the frontend
Because
* I changed the layout of the docker containers to have separate dev/build/deploy stages
This commit
* Update the circle task to build the deploy image
Because
* We split the docker image into two images, one for local dev and one for deploying to prod so that the prod one would be small ~100mb
* We should find some way to test that the deploy image is working correctly
* A good candidate is to run the integration tests against it
This commit
* Adds a new make up_prod command that sets up an environment that more closely mirrors prod ie
- Use the prod image
- Disable all the yarn watchers
* Reorganizes the dockerfile to move frontend asset building out of the dev container to increase docker cache hits for local dev
Because
* The circle deploy task runs after every PR lands and it's not being cached it takes ~10m to rebuild everything from scratch every time
This commit
* Enables docker layer caching for the deploy task which means in the common case it should take <1m
* Bump to the latest circle docker image
Because
* We want to shrink the size of the deployable Docker image down from its current 2.71gb(1gb compressed)
This commit
* Adds a second stage to our Dockerfile for a deploy image which is a much more reasonable 480mb(133mb compressed)
* Use deploy image for Django container in local dev compose
Because
* We want to publish Storybook builds to demo and preview component work
in nimbus-ui
This commit
* Integrates mozilla-fxa/storybook-gcp-publisher into the CircleCI flow
so that Storybooks are built and published to a Google Cloud static
site for every test run
EXP-281
fixes#3427
Because
* Running circle jobs against each individual dependency PR makes it easier to know
if a single dependency change breaks the build rather than having to only run tests
against all dependencies combined
This commit
* Enables circle jobs on dependency PRs
Benjamin Forehand Jr
Jared Lockhart
Les Orchard