Updating the code now that fenix lives in firefox-android. This
actually simplifies things quite a bit.
Renamed the parameter from `ff-android` to `firefox-android`. I don't
think we need to abbreviate it now that it's the only branch-build parameter.
Also, updating the branch build code to work with owner references with
a colon, like `bendk:my-branch`. This is what you get when clicking the
github copy branch name button.
Removed the support-migration component, which was removed on the a-c
side. Also added support-rusterrors and lib-auth.
Updated `update-branch-build-components.py` to pretty print the JSON
file. This should make the diffs nicer in the future.
- Made the android-components tests work more like the ones from android-components CI
- Split up the tests by project
- Updated which gradle tasks we run and allow the tasks to vary
per-project. My goal was to make this equivelent to the
android-components `taskcluster/ci/build/kind.yml` file.
- Updating android platform verison to 31
- Download robolectric and maven ahead of time, this seems to fix the
test errors.
- Build all architectures, not just x86
- Customize fetches based on which repo we're building from. This makes
each task fetch less total data, which seems to fix some intermittent
errors I was seeing.
- Added a `git rev-parse` to the pre-gradlew commands. This way we can
easily see the commit IDs.
- Update branch build regex:
- Allow specifying the repo owner
- allow '.' chars
- Support both `a-c: [branch]` and `ac: [branch]`
This didn't work out well in practice, since an index can only store one
task. So if we have all the branch build task using the same index,
whichever one runs last wins. I think what we really want is to index
the task group, but I don't think that's possible. Probably the best we
can do is what was already there by default, which is to index the
decision task, then find the task group from that.
Also fixed the spelling of "nightly" in the test param YAML file.
Added a `route` to our nightly branch build tasks. I think this should
mean we can view the latest branch builds via a taskcluster index.
Moved the code to calculate if we're in a nightly into the top-level
`get_decision_parameters()` function. It seems good to put this in one
place, especially since the way we determine if we're in a nightly build
is pretty hacky.
Added new CI tasks to run branch builds.
These can be triggered from a PR by adding `[ac: {branch-name}]` and/or
`[fenix: {branch-name}]` on the PR title.
I also added a nightly build cron-job that runs a branch build using the
main branch from all 3 repos.
Updated our Dockerfile so that it can build a-c.
Releng has tweaked the recommended way of bootstrapping Taskgraph from a
clone + checkout pinned revision, to Pypi and a requirements.txt.
This provides a few benefits:
1. Since Taskgraph uses SemVer, updating will provide a hint if there
will be backwards incompatible changes.
2. Using version numbers is nicer than revisions (e.g, Taskgraph's
CHANGELOG will be useful).
3. Provides a mechanism to pin other non-Taskgraph dependencies.
The reason I'm submitting this now is I'd like to migrate Taskgraph to a
new repository. By moving consumers over to Pypi first, the migration
can happen seamlessly without breaking anything.
Jira: RELENG-816
Created a rust toolchain task that installs the rust components we use
for building and uploads them as artifacts. This needs to run before
the other toolchain tasks, so I made a new kind called `early-toolchain`
for it.
Made the other tasks depend on this, and have them download and unpack
the artifact before starting.
This allows us to do this setup once rather than multiple times.
Moved functionality from `rustup-setup.sh` to other places:
- The Dockerfile now sets the ENV variables and installs `rustup`.
Note that we now always set `-Dwarnings`, before we didn't set if
for releases because of the corner case where we cherry-picked code
that had warnings from newer Rust versions (#3480). Now that we pin
the Rust version with our `rust-toolchain`, I don't think this is an
issue anymore.
- The task itself now runs `git submodule --init`
Moved the Rust installation code from `cross-compile-setup.sh` to the
new `rust.yml` task.
* Update to latest taskgraph and decision-mobile image
See also RELENG-658.
* Fixes#4393 - Run taskgraph generation with Python 3
See also RELENG-658.
* Run 'pyupgrade --py36-plus' on taskcluster/ directory
* Remove unused imports from taskcluster/ directory
It's reasonable for mobile-focused consumers to want to be able
to do a quick `cargo build` to test their local checkout, and it's
not reasonable for us to expect them to understand how or why this
is different from building inside Xcode or Android Studio. Let's
help make this Just Work (tm) by verifying the Desktop development
environment whenever we verify a Mobile development environment.
Prior to the commit, the Taskcluster job for each individual component
would execute `./gradlew component:test` and would thus need to build
the Rust code for the megazord.
With this commit, each of those tasks now depend on the existing
`android-build` task which knows how to run all the tests in a single
build. This should reduce the number of times we have to build the
megazord from once-per-components to just once. (Plus the independent
builds we do for actually publishin the megazord).
Prior to this commit, the gradle tasks for building and for testing
each component would depend on the "cargoBuild" gradle task of the
megazord, which means they would built it for all configured targets.
In TaskCluster we configure *seven* targets, so each `module-build-X`
TaskCluster job will build the full megazord seven times.
This commit makes two changes:
* Building the gradle package of a component no longer depends on
building the megazord at all. We doesn't actually need to the
Rust code built in order to package up the Kotlin files, and I
think this dependency was a hangover from when each component
used to build its own individual Rust library.
* Running the gradle tests of a component now depends only on
building the megazord for the native host platform, since that's
the only one that is actually needed for running the tests.
Given that we have 11 components that depend on the megazord,
each of which will now avoid building 6 useless copies of it
for non-native platforms, I expect this will save a total of
66 expensive `cargo build` invocations in a full TaskCluster run.
I've also included a few other tweaks to do less unnecessary
work in TaskCluster, which seem to save about a minute on the
much-more-common `android-build-pr` task.
This fixes#4108, getting our CircleCI tests passing again with
RUSTUP_TOOLCHAIN="beta". It's a bit of a circuitous fix but I
think it's worthwhile taking this approach.
As noted over in #4108, the `circleci/rust` docker image configures
rustup to use its "minimal" profile by default, which means it doesn't
automatically install `rustup` into non-default toolchains, which was
causing problems for our tests.
According to a Google search for "CircleCI Rust" and to the note at
the top of [1], the `circleci/rust` image is considered legacy and has
been replaced by `cimg/rust`. This new image uses the "default" rustup
profile, meaning it will more closely match the experience of executing
Rust tooling on your local machine, and in particular will install
`rustfmt` by default in all toolchains.
This new `cimg/rust` image does not support a `:latest` tag, and instead
requires you to specify an explicit version of the Rust toolchain to
use. To avoid duplicating this information in many places, I've used the
"executors" feature of CircleCI to consolidate the details of the docker
image in a single place.
Since I was defining some executors anyway, I also took the opportunity
to consolidate the details of the "macos" executor, so if we want to
bump the version of xcode in future it can be done in a single place.
Finally, I updated the version of `rustup-init` that is used to boostrap
the Rust tooling on the macos executor and in taskcluster. This isn't
necessary, but I did it as part of investigating the fix here and it
seemed worthwhile to keep.
[1] https://circleci.com/developer/images/image/cimg/rust
We originally landed this change in #3936 but had to revert it due
to release build issues. As far as I can tell, those were due to
mismatches between our use of a rust-toolchain file for repos-specific
config, and our CI scripts that try to configure the default rustup
toolchain.
This commit moves as much rustup config as possible into the toolchain
file, where we can maintain it in a single consistent place.
We are hitting some build errors on win32 with the latest release.
I think this may be due to the upgrade to Rust 1.50, and will see
what I can do to resolve it. But in the meantime we need to be
able to cut releases, and disabling the win32 artifacts seems like
the shortest path to that outcome.
In practise, this will mean that Android developers will not be
able to run unittests that use Application Services code if they
are on a Windows machine, since they won't have an appropriate
`.so` file from which to load the appservices code.
This is an attempt to simplify how we use `uniffi-bindgen`, by making
it easy to run the version used by the workspace rather than assuming
that the user has a correctly-matching version of it installed in their
system cargo.
This commit pulls in https://github.com/mozilla/nimbus-sdk as a
git submodule, and adds some build integration to publish it as
part of the application-services megazord. Instead of an all-in-one
nimbus package, we produce two separate packages:
* `org.mozilla.appservices:nimbus` with the Nimbus Kotlin code
* `org.mozilla.appservices:full-megazord` updated to include the
Nimbus Rust code.
Nimbus SDK and its dependencies are not currently compatible with
Rust v1.43.0, but application-services is pinned to that version
of rust for compatibility with mozilla-central.
This reverts commit 8388372259.
Ben Dean-Kawamura
Ryan VanderMeulen
Julien Cristau
Sammy Khamis
Michelle
Travis Long
Andrew Halberstadt
Ben Dean-Kawamura
Tarik Eshaq
Andrew Halberstadt
Ryan Kelly
Ryan Kelly
Dan Mosedale
Mark Hammond