Because:
* We want to use the new eligibilitymanager in the capability service
and compare its results to the existing check, reporting any
differences in the eligibility comparison.
This commit:
* Adds the eligibility manager to the capability service.
* Adds a wrapper to the results of the elgibility call to verify it
matches the existing comparison, and report to Sentry if it doesn't.
Closes FXA-8273
Because:
- Not all CI operations where taking advantage of NX caches
This Commit:
- Adjusts nx.json config to ensure correct caching behavior
- Adjusts nx.json config to ensure proper target chains. ie build is dependent on pre-build
- Uses nx to directly run integration tests
- Uses nx to directly run unit tests
- Phases out dependency on generated .list files
- Ensures 'start up' for services for functional tests is done with nx and uses nx caches.
- Creates nx tags to facilitate test commands
- Fixes linter errors encountered
- Updates docker images to nx for builds
- Updates nx ignore files
Because:
- We want to lock down our gql queries as part of good security practices.
This Commit:
- Creates task that extracts existing gql queries from our code
- Applies middleware that checks a allowlist of extracted queries.
- Creates a 403 if the query is not in the allowlist
- Makes all gql parsable by extraction utility. The extraction utility doesn't support string placeholders. e.g. gql` Account { ${ACCOUNT_FIELDS} }`
- Moves all the gql in the admin server into .ts files. Again for consistency and extraction utility support.
- Cleans up some config references at startup
- Adds nx config for extraction & copy tasks
Because:
- We want to clean up our docker build
- We no longer used our service specific images
This Commit:
- Switches to multi stage docker build
- Removes unused building of individual services
- Removes used script that sets environment variables.
- Removes `workspaces focus` call that resulted in redundant installs
- Removes trap on build logs.
- Uses circleci DLC to get cache hits for fxa-utils and fxa-build-utils
- Removes scripts that build service specific images
- Cleans up naming conventions to reflect that only one image is being built
Because:
- we want to use Glean for our metrics
This commit:
- integrates glean.js into content-server
- adds a couple placeholder registration Glean metrics events
Because:
- Content server would not return the l10n version
- Story books in auth server weren't building
This Commit:
- Makes sure l10n:prime is run prior to running tests.
- Makes sure l10n:prime is run prior to building storybooks.
Because:
- We want to avoid redundant git clone operations
- We want to reduce the number of ftl bundles requested on the client side.
This Commit:
- Adds _scripts/l10n/*.sh for conducting l10n build operations
- Bundles ftl files together into ftl main
- Removes bundle prop on AppLocalizer, so that it uses the default, [main.ftl]
- Replace - with _ in locale names when resolving ftl files
- Removes clone-l10n.sh
Because:
* We're refactoring our legacy tech stack to our preferred stack by extending settings
This commit:
* Adds a clone-legal-docs script so we always fetch latest rather than pin to commit
* Reworks the 'react groups' into separate server and router files; router.js needs access, but cannot have any reference to route definitions since they now contain server-side logic and imports
* Moves available content-server routes into a separate file that does not import route definitions
* Uses and accounts for the existing regex for legal, legal/terms, and legal/privacy
* Adds two helper packages to render the MDs once fetched
Closes FXA-6131
Because:
- We wanted to run a few preliminary checks before proceeding to more
expensive CI jobs. Checks include:
- Compiling typescript in commonly referenced workspace packages
- Linting code that has changed
- Executing Unit Tests for code that has changed
- We wanted to partition test operations into unit tests, and
integration tests. Unit tests can be run relatively quickly and
require no additional infrastructure. Integration tests require
additional infrastructure and generally have longer execution
times. Now that jobs are blocked from running until preliminary
checks pass, one of which is unit tests, it is important to draw a
distinction between these two types of tests.
- We want to avoid unnecessary yarn installs and typescript
compilations, which are time consuming.
- We want to make sure that test results are published and failing tests
can be easily viewed in the CI.
This Commit:
- Creates a build-and-validate job in the CI that builds, lints, and
unit tests code prior to running any other jobs.
- Creates unit-test job in CI config
- Creates integration-test job in CI config
- Removes redundant calls to compile workspace packages. These
are now built up front, cached, and restored as needed for future
runs.
- Extends the create-lists script functionality to generate commands
that can be executed with the parallel command.
- Removes unnecessary yarn install operations. Invoking yarn workspace
focus results in a yarn install. In the case of running tests this is largely
unnecessary, because we already do a yarn install in the base-install
step.
- Make sure test results are exported as junit xml so the CI can report
back on tests that were failing. This was done for a couple workspace
packages, but many were lacking the capability. All test:unit and
test:integration npm scripts now export this data.
- Fixes the following issues encountered along the way:
- Adds logs to monitor heap usage of jest tests. Some
jest tests are still using a lot of memory.
- Moves a few slow / long running tests from unit test to
integration tests.
- Ensures that jest.transform for ts-jest is always instructed
to have the config option isolateModules is set to true. This
definitely decreases memory overhead and resolves some
of the OOM errors we were hitting. It was configured in
some places but not everywhere.
- Exports test results files for all tests
- Exports all test artifacts
- Uses gnu parallel to run tests in parallel. Turns out yarn
workspaces foreach would give a false positive when an OOM
was encountered. Fortunately, the parallel command offered an
acceptable work around, and even offers some nice features
like the load argument, which allows to control test execution a
bit more efficiently.
Because:
- We want to speed up pipeline startup.
This Commit:
- Optimizes docker base image builds for the smallest images possible.
- Enables hard links for yarn cache
- Enables global yarn cache
- Avoids needlessly installing playwright browsers.
- Avoids needlessly running yarn install.
- Uses mozilla/fxa-circleci:ci-base-latest image for running test pipelines
- Uses mozilla/fxa-circleci:ci-base-browsers image for running functional tests pipelines
- Creates configurable executors that can be reused across pipelines
- Upgrades to yarn 3.3.0
- Enabled direct check out of PR code to test, which is faster than circle ci’s checkout command.
- Upgrades functional test to X-Large. This was already the case for playwright tests, but is now extended to content server tests too. This decision was made due to running lots of pipelines and realizing flakiness was largely due to CPU or memory hitting 100% for long periods of time.
- Turns off tracing, since it saves a bit of runtime.
Because:
- Concatenating shared FTL strings into multiple packages before sending for localization created duplication of localization work, and we want each string to only be translated once.
This commit:
- Remove concatenation of shared files (branding.ftl, fxa-react ftl files) from grunttasks
- Add grunttasks in fxa-react to concatenate fxa-react FTL files into one react.ftl file
- Add grunt to fxa-react dev dependencies
- Update fxa-react scripts to run l10n and ftl tasks
- Update clone-l10n script to distribute react.ftl to packages
- Bundle branding and shared into AppLocalizationProvider
Closes #FXA-6388
Because
* Organizing l10n strings in per-component files aligns payments-server with the setup used for auth-server and settings. Splitting the FTL files also improves maintability by more tightling coupling the strings with the components where the strings are used.
This commit
* Create a temporary branding file in fxa-payments-server/src
* Create a gruntfile with tasks for FTL concatenation and watching
* Rename the destination file from main.ftl to payments.ftl
* Switch payments-server's default locale from en-US to en and update all references of en-US to en
* Create individual FTL files per component and move messages to their respective component FTL file
* Add a l10n entry in the package readme
* Update clone-l10n.sh to copy 'payments' ftl files instead of 'main'
* Update AppLocalizationProvider to use 'payments' bundles
* Replace setupFluentLocalizationTest with getFtlBundle/getFtlFromPackage from fxa-react
* Remove setupFluentLocalizationTest function (no longer used in fxa-settings)
* Add merge-ftl:test task to package.json start and test:frontend scripts
* Remove legacy strings from FTL files
* Update currency and date formats to use 'en' as default locale
* Remove legacy paths from .gitignore files in payments-server
* Move remaining paths to global .gitignore
* Delete .gitignore files in payments-server
Closes #FXA-5996 and #FXA-6255
Because:
* We want better l10n testing across the codebase
This commit:
* Creates a Localized wrapper to use in place of Localized that requires children (fallback text)
* Adds a mock for the Localized wrapper in Jest's setupTests to avoid copying and pasting this into every test file
* Creates test-utils in fxa-react and exports functions to use for testing the mock that all IDs exist in the 'en' bundle, that fallback text matches the message in the bundle, and that the message doesn't contain straight apostrophes or quotes
* Adds tests and a test.ftl file for the test-utils l10n functions to ensure they test what we think they do
* Moves concatenating settings.ftl out of webpack and into grunt due to 1) needing a test version of settings.ftl at a different path, else tests may fail in CI with new or changed strings since the l10n repo won't have them immediately, 2) wanting to retest FTL changes without having the settings app running (jest does not run nicely with webpack), 3) consistency with the auth-server
* Uses this new test functionality in a few Settings tests covering various use cases
Closes FXA-5999
Because:
- We need to pull the latest pdfs from legal-docs into the fxa
assets folder.
This commit:
- Create a script to copy the latest pdfs into assets/legal.
- Create a Github Action to automatically pull in the pdfs from
legal-docs, and open a pull request to move pdfs into assets/legal.
Closes #fxa-5100
Because:
* We are starting a content-server reskin and want to adhere to the existing design guide for consistency and to make converting to React/TW later easier
This commit:
* Adds tailwind v3 to content-server, and includes tailwind.out.css only on the main index page for now
* Adds new script for building and pm2 process for watching
fixes#13480
Because:
* We are starting a content-server reskin and want to adhere to the existing design guide for consistency and to make converting to React/TW later easier
This commit:
* Adds tailwind v3 to content-server, and includes tailwind.out.css only on the main index page for now
* Adds new script for building and pm2 process for watching
fixes#13480
Because:
- There were issues starting the admin-server.
This Commit:
- Introduces a dev.json to solve issues with missing dev settings.
- Make sure the dist folder is actually cleaned. It was possible that a stale config.js file could cause problems.
- Make sure build is run and config/*.json files are copied over.
Because:
- We want to display account subscription data in the admin panel.
This Commit:
- Hooks up the Account React component to real subscription data.
- Fixes a long standing issue with Knex in the admin panel! Knex instance is now bound to BaseAuthModel and repurposed across derived classes.
- Introduces a subscription module that provides the nestjs services necessary to retrieve subscription data.
- Introduces a subscription service that acts as the primary point of request for subscription data.
- Introduces stripe service so that stripe can be dependency injected.
- Introduces firestore service so that firestore can be dependency injected.
- Introduces play store service so that play store accessor can be dependency injected.
- Introduces app store service so that app store accessor can be dependency injected.
- Introduces subscription formatters that unify subscription dtos into standard response format.
- Improves config setup to allow for local.json and secrets.json files to be used.
- Introduces configuration settings so that underlying services can be accessed.
- Introduces feature flags to disable queries to underlying stripe, google or apple apis.
- Hoists a couple joi validators up to fxa-shared for reuse.
- Breaks out an auxiliary method that can determine product ids given iapType and a plan. This exists in in stripe.ts in fxa-shared. (The previous code had zero dependence on the stripe class, which made this possible. This was primarily done for testing purposes.)
- Exposes a couple more fields on AppStoreSubscriptionPurchase to support subscription formatting.
- Fixes typing on MozSubscription.endedAt. Value is allowed to be null | undefined.
- Adds tests and achieves 90+ percent test coverage on all new code.
- Adds reusable mocks for standard services to facilitate testing.
- Updates readme with info about subscriptions, configuration, feature flags, and testing.
- Updates pm2.js with better defaults to support subscription service.
* chore(auth) - Phaseout mjml auth server
Because:
- A separate server was required to render templates needed for storybook.
This Commit:
- Creates some bindings to abstract away process context used by FluentLocalizer.
- These bindings support running in either a nodejs process or in the browser.
Fixes: #10218
* Improve ejs install
* Put mjml-browser helper under test
* Update documentation around static storybook build.
* PR feedback
* Try relative path
* Relative paths
* Switch to curl
* PR Feedback
Because:
- We want to be able to generate a single FTL file for fxa-content-server-l10n to pick up
- We want to be able to pull in existing auth locales from fxa-content-server-l10n
This commit:
- Adds a new grunt task for generating a single FTL file, placing it in a location fxa-content-server-l10n will find it
- Updates the clone-l10n script to check for auth server locales in the fxa-content-server-l10n repo
Because:
- we have multiple similar scripts that pull translations from the
fxa-content-server-l10n repo
- we need that functionality for new settings
This commit:
- combine the scripts into a single one
Because:
* We want to use NestJS for new apps, and graphql-api is still small
enough to feasibly convert to NestJS.
This commit:
* Converts graphql-api to a NestJS app.
Closes#6263
- We have two clients for the auth-server API, one in `content-server/app/scripts/lib/auth` and `fxa-js-client`. The content-server one was meant to replace fxa-js-client but I didn't finish the work to get it running on nodejs.
- Extracts the content-server implementation into `fxa-auth-client`
- Wraps the implementation with shims to work with nodejs in `server.ts`
- Updates references to fxa-js-client to use fxa-auth-client
- Removes fxa-js-client
Because:
* Currently fxa-settings imports SCSS into a Tailwind file, but outside of Create React App (which handles SCSS compilation in the React app itself) SCSS is not compiled.
This commit:
* Updates both Tailwind and Storybook's setup so that you can now compile down to CSS or SCSS depending on your needs, and ensure Storybook's SCSS loader picks it up
compile tailwind for use in fxa-react, fxa-settings, and fxa-admin-panel
add script to tailwind configs to only apply postcss output to fxa-react components being used in a given project
In order to keep the typescript shared modules up to date for their dependents to use while running locally this adds them to pm2 so that a `tsc --watch` can run alongside the other services.
Note, this DOES NOT automatically restart dependent javascript service processes.
This change refactors pm2-all.sh in order to start processes in dependency topological order. This requires an update to most pm2.config.js files in order to set PATH correctly while running under yarn.
@fxa-react is no longer used. content-server should reference
fxa-react through a workspace dependency.
This commit fixes references and build configs for this change.
Because:
We had to occassionally add to our list of !.* exceptions which was getting a little unwieldy. Instead, we'll manually add the ones we need ignored.
fixes#5250
This adds a base node image and builder image so
that all our service images can share the same
common base, be smaller, and require less customization.
why: several reasons...
- to have a unified approach and pattern for:
- debugging
- fs watching for local dev
- running services in dev
- configuring services in dev
- to improve the initial clone and subsequent `npm ci` experience
- to make future work on tooling easier
In order to reduce our CI resource usage we've combined
some jobs and changed what runs for each workflow.
Most packages have fast test runs. We've combined these
under one job. Packages with slower runs use their own
job so they can be run in parallel. Also, we've added
branch filters to the rarely changing, but resource
intensive jobs.
We now have three workflows that run at different times.
1. test_pull_request: runs only on PR branches and
only runs tests.
2. deploy_master: runs only on master and only builds
and deploys docker images
3. test_and_deploy_tag: runs only on tags and tests,
builds, and deploys
Ben Bangert
dschom
Lisa Chan
Barry Chen
Lauren Zugai
Valerie Pomerleau
Reino Muhl
Barry Chen
Danny Coates
Dan Schomburg
Jody Heavener
yogita
Vijay Budhram
Michael Kane Juncker
Ben Bangert
Les Orchard