Because:
* Mac M1 and M2 machines are slower to start up infrastructure like Redis and Firestore (which runs under emulation).
This commit:
* Doubles the RETRY period from 120 to 240s as most of the time, the __heartbeat__ response for fxa-auth-server and firestore processes responded between 140 - 190s.
Closes #No ticket
Because:
- Not all CI operations where taking advantage of NX caches
This Commit:
- Adjusts nx.json config to ensure correct caching behavior
- Adjusts nx.json config to ensure proper target chains. ie build is dependent on pre-build
- Uses nx to directly run integration tests
- Uses nx to directly run unit tests
- Phases out dependency on generated .list files
- Ensures 'start up' for services for functional tests is done with nx and uses nx caches.
- Creates nx tags to facilitate test commands
- Fixes linter errors encountered
- Updates docker images to nx for builds
- Updates nx ignore files
Because:
* On stack startup, services start before mysql patches have completed,
causing profile-server to fail on startup.
This commit:
* Adds script to check if patcher script has started and is still
running.
Closes #
Because:
- We want a poc that auth can be added to redis
This Commit:
- Configures all configs to support a redis password
- Sets the standard env for redis auth to REDIS_PASSWORD
- Creates a default redis password of 'fxa123' for local dev
- Starts redis container with --requirepass fxa123
Because:
- We want to clean up our docker build
- We no longer used our service specific images
This Commit:
- Switches to multi stage docker build
- Removes unused building of individual services
- Removes used script that sets environment variables.
- Removes `workspaces focus` call that resulted in redundant installs
- Removes trap on build logs.
- Uses circleci DLC to get cache hits for fxa-utils and fxa-build-utils
- Removes scripts that build service specific images
- Cleans up naming conventions to reflect that only one image is being built
Because:
- We want the package.json version to match the git tag being deployed.
- We don't want to commit a change on the package.json file to accomplish this.
This Commit:
- If there is a git tag, the code will send it into the docker build as a VERSION build arg.
- This build arg is then used to update the version field in all package.json files.
Because:
- We can build the docker image while tests are running
- This will cut down on deploy time
This commit:
- Builds the deployed docker images in parallel to tests
- Pushes temporary image to docker up
- Once tests finish re-tags temporary image and pushes to docker hub
- Removes temporary build image from docker hub
Because:
- We want to move our legal doc query to the server side so it can be made with a single call.
This Commit:
- Changes to clone script:
- Use the provided state.json provided by l10n repo
- Moves download to external folder
- Changes to settings:
- Use gql to fetch legal doc
- Changes to gql
- Adds ability to resolve a legal document.
Because:
- We are hitting race conditions using postinstall
- If two packages invoke l10n:clone at the same time in their post install git will error out
This Commit:
- Adds a git checkout hook, so the l10n folder is regularly updated
- Remove l10n:clone and l10n:prime commands from the post install
- Moves the l10n:prime command into build and tests commands
- Adds clean script so blank slates can be tested locally
- Adds respective l10n-bundle and l10n-prime scripts per workspace
- Moves the creation of the git-head.txt to the prime step and avoids creating an change on the l10n repo
Co-authored-by: Peter deHaan <peter@deseloper.com>
Because:
- We want to avoid redundant git clone operations
- We want to reduce the number of ftl bundles requested on the client side.
This Commit:
- Adds _scripts/l10n/*.sh for conducting l10n build operations
- Bundles ftl files together into ftl main
- Removes bundle prop on AppLocalizer, so that it uses the default, [main.ftl]
- Replace - with _ in locale names when resolving ftl files
- Removes clone-l10n.sh
Because:
* We want to standardize the branding used across fxa packages. FXA-5994 set up a global branding file that was initially used only for fxa-auth-server, and we now want to expand this file to be used for fxa-settings too.
This commit:
* Move branding terms from fxa-settings/src/components/en.ftl to the global branding file in fxa-shared/l10n/.
* Update clone-l10n script to include branding.ftl in the fxa-settings locales folders.
* Update fxa-setting to include "branding" in the Fluent bundle.
* Update ftl-test paths to include branding.ftl and react ftl files.
Closes #FXA-5995
Because:
* We're refactoring our legacy tech stack to our preferred stack by extending settings
This commit:
* Adds a clone-legal-docs script so we always fetch latest rather than pin to commit
* Reworks the 'react groups' into separate server and router files; router.js needs access, but cannot have any reference to route definitions since they now contain server-side logic and imports
* Moves available content-server routes into a separate file that does not import route definitions
* Uses and accounts for the existing regex for legal, legal/terms, and legal/privacy
* Adds two helper packages to render the MDs once fetched
Closes FXA-6131
Because:
- We wanted to run a few preliminary checks before proceeding to more
expensive CI jobs. Checks include:
- Compiling typescript in commonly referenced workspace packages
- Linting code that has changed
- Executing Unit Tests for code that has changed
- We wanted to partition test operations into unit tests, and
integration tests. Unit tests can be run relatively quickly and
require no additional infrastructure. Integration tests require
additional infrastructure and generally have longer execution
times. Now that jobs are blocked from running until preliminary
checks pass, one of which is unit tests, it is important to draw a
distinction between these two types of tests.
- We want to avoid unnecessary yarn installs and typescript
compilations, which are time consuming.
- We want to make sure that test results are published and failing tests
can be easily viewed in the CI.
This Commit:
- Creates a build-and-validate job in the CI that builds, lints, and
unit tests code prior to running any other jobs.
- Creates unit-test job in CI config
- Creates integration-test job in CI config
- Removes redundant calls to compile workspace packages. These
are now built up front, cached, and restored as needed for future
runs.
- Extends the create-lists script functionality to generate commands
that can be executed with the parallel command.
- Removes unnecessary yarn install operations. Invoking yarn workspace
focus results in a yarn install. In the case of running tests this is largely
unnecessary, because we already do a yarn install in the base-install
step.
- Make sure test results are exported as junit xml so the CI can report
back on tests that were failing. This was done for a couple workspace
packages, but many were lacking the capability. All test:unit and
test:integration npm scripts now export this data.
- Fixes the following issues encountered along the way:
- Adds logs to monitor heap usage of jest tests. Some
jest tests are still using a lot of memory.
- Moves a few slow / long running tests from unit test to
integration tests.
- Ensures that jest.transform for ts-jest is always instructed
to have the config option isolateModules is set to true. This
definitely decreases memory overhead and resolves some
of the OOM errors we were hitting. It was configured in
some places but not everywhere.
- Exports test results files for all tests
- Exports all test artifacts
- Uses gnu parallel to run tests in parallel. Turns out yarn
workspaces foreach would give a false positive when an OOM
was encountered. Fortunately, the parallel command offered an
acceptable work around, and even offers some nice features
like the load argument, which allows to control test execution a
bit more efficiently.
Because:
- We cannot add parameters to job triggered through circleci's API
This Commit:
- Removes parameters from the `test-content-server-remote` job
Because:
- Concatenating shared FTL strings into multiple packages before sending for localization created duplication of localization work, and we want each string to only be translated once.
This commit:
- Remove concatenation of shared files (branding.ftl, fxa-react ftl files) from grunttasks
- Add grunttasks in fxa-react to concatenate fxa-react FTL files into one react.ftl file
- Add grunt to fxa-react dev dependencies
- Update fxa-react scripts to run l10n and ftl tasks
- Update clone-l10n script to distribute react.ftl to packages
- Bundle branding and shared into AppLocalizationProvider
Closes #FXA-6388
Because
* Organizing l10n strings in per-component files aligns payments-server with the setup used for auth-server and settings. Splitting the FTL files also improves maintability by more tightling coupling the strings with the components where the strings are used.
This commit
* Create a temporary branding file in fxa-payments-server/src
* Create a gruntfile with tasks for FTL concatenation and watching
* Rename the destination file from main.ftl to payments.ftl
* Switch payments-server's default locale from en-US to en and update all references of en-US to en
* Create individual FTL files per component and move messages to their respective component FTL file
* Add a l10n entry in the package readme
* Update clone-l10n.sh to copy 'payments' ftl files instead of 'main'
* Update AppLocalizationProvider to use 'payments' bundles
* Replace setupFluentLocalizationTest with getFtlBundle/getFtlFromPackage from fxa-react
* Remove setupFluentLocalizationTest function (no longer used in fxa-settings)
* Add merge-ftl:test task to package.json start and test:frontend scripts
* Remove legacy strings from FTL files
* Update currency and date formats to use 'en' as default locale
* Remove legacy paths from .gitignore files in payments-server
* Move remaining paths to global .gitignore
* Delete .gitignore files in payments-server
Closes #FXA-5996 and #FXA-6255
Because:
- When starting tracing with open telemetry the network name would sometimes collide and result in error.
This Commit:
- Sets an 'fxa' network name
- Applies this network to services
- Allows services to communicate by their names
Because:
- We need to pull the latest pdfs from legal-docs into the fxa
assets folder.
This commit:
- Create a script to copy the latest pdfs into assets/legal.
- Create a Github Action to automatically pull in the pdfs from
legal-docs, and open a pull request to move pdfs into assets/legal.
Closes #fxa-5100
Because:
* Test all didn't run in parallel to take advantage of more cores and
a few older tests were flakey.
* Tests would hang as packages had a default test that was in watch
mode.
This commit:
* Updates several flakey tests and runs the test all in parallel for a
shorter completion time.
* Updates settings and payment-server packages to have new watch command
and deafult test command that runs the tests.
Closes #FXA-6096
Because:
* We updated the local mysql to a version that doesn't pass auth-server
tests.
This commit:
* Drops the local mysql to a version that the test passes.
Closes #FXA-6068.
Because:
- We want to start trace capture at client side
- We can't report trace data directly to cloud trace
This Commit:
- Adds support for client side tracing in tracing/browser-tracing.ts
- Some refactors to fxa-shared/tracing to ensure code is reused
- Cleans up tests
- Introduces traceparent header and tracestate html headers
- Opens up OTLP ports on jaeger
- Introduces a clientName to trace config
- Adds an open telemetry collector service (otel-collector)
- Uses this services for all trace capture
- Adds flow id to trace id headers so traces can be validated
Because:
- We want to start trace capture at client side
This Commit:
- Adds support for client side tracing in tracing/browser-tracing.ts
- Some refactors to fxa-shared/tracing to ensure code is reused
- Cleans up tests
- Introduces traceparent header and tracestate html headers
- Opens up OTLP ports on jaeger
- Introduces a clientName to trace config
Because:
- Subscription Services is prepareing for initial release
This commit:
- Adds initial set of Subscription Services legal documents.
Closes #FXA-6036
Because:
- When not enabled Jaeger was exiting with an 'error' status which was confusing.
This commit:
- Adjust infrastructure.config.js so that Jaeger exists cleanly with a 'stopped' status.
- Reports a better log message so that it's clear Jaeger is not required to run FxA.
Because:
- We want to be able to see trace information
- We want to be able to export trace information to google cloud
This Commit:
- Adds utility class to fxa-shared for configuring and initializing open telemetry
- Initializes open telemetry in the auth server
- Forces resolution of google-gax.
- Forces resolution of @grpc/grpc-js.
- Adds the jaeger docker container for viewing traces locally. This is accessible on localhost:16686.
Note, adding @google-cloud/opentelemetry-cloud-trace-exporter resulted
in a version conflicts for google-gax and @grpc/grpc-js, which caused
runtime failures when interacting with @google-cloud/firestore. After
many experiments, this seems to do the trick.
Because:
- Some affected packages were not being compiled
This pull request
- Includes the first line output by 'yarn workspaces list', so that the package is included in subsequent checks.
[skip ci]
Because:
* The fxa-content-server-l10n is still using master as default branch, and we want to update it to main.
This commit:
* Updates references from master to main in automation
* Fixes or removes master references in docs
* Fixes a few typos
Because:
- we want to upgrade to mysql 8
This commit:
- upgrade mysql for local dev
- start mysqld with default auth set to native password for backwards
compat with the 'mysql' module
Because:
* We should be informed about TS errors before a patch merges into main.
This commit:
* Adds a new script to run as the first task in the test-many job that compiles any modified back-end Typescript packages and any packages that depend on them.
Closes#12823
Co-authored-by: Barry Chen <bchen@mozilla.com>
Because:
* `gh-pages.sh` and `upload-assets-to-cdn.sh` are no longer used
This commit:
* removes them. Once this lands I will update the documentation as
well.
On M1 we currently sometimes need to 'yarn restart services' after a 'yarn start' before things start working. Maybe if we wait for the containers to be responsive before starting services we can avoid this. The important one seems to be mysql.
Bianca Danforth
dschom
Reino Muhl
Lisa Chan
Dan Schomburg
Wil Clouser
Barry Chen
Valerie Pomerleau
Lauren Zugai
Valerie Pomerleau
Ben Bangert
Julian Poyourow
Francesco Lodolo
郭佳慧
Vijay Budhram
Florian Dieminger
Danny Coates
Jody Heavener