Also update the table of Firefox requirements. It's worth noting that I
actually upgraded to 1.57 earlier than I should have (too close to
freeze), but what is done is done.
Differential Revision: https://phabricator.services.mozilla.com/D134124
This attribute isn't necessary as e10s is the default. Instead consumers can
check if the string '1proc' appears in the unittest_variant attribute.
Depends on D133225
Differential Revision: https://phabricator.services.mozilla.com/D133226
Tasks that set this key to false *will not* run under the standard no variant
configuration.
This is needed to replace the 'e10s=false' case (i.e tasks that *only* run with
1proc like mochitest-chrome or mochitest-a11y.
Depends on D133224
Differential Revision: https://phabricator.services.mozilla.com/D133225
Previously we we're validating the entire 'test_description_schema' again, even
though the vast majority of those keys were in fact no longer needed. This was
preventing us from ever removing keys from the 'task' object. Which I believe
is a good practice to keep things simple.
Differential Revision: https://phabricator.services.mozilla.com/D133222
We separate the patchset from the per-platform configuration, which
will ensure we keep the same patchset across them (spoiler alert: we
weren't).
Also, as most builds are PGO, use that in per-platform configurations,
and add an override config that sets it back to 2-stages for the
builds we don't want to PGO.
Differential Revision: https://phabricator.services.mozilla.com/D133317
I forgot that mozboot only pulls in updates when you run `./mach bootstrap`,
so some people got the new mozcrash.py locally without actually having the
new rust-minidump-based version. So now we first run the stackwalk binary
with -V to check what version it is.
The rest of the details can be found in the comments I added.
Also updates rust-minidump to 0.9.6 get some CLI parsing fixes
and better --help documentation (socorro staging is already updated to
this version).
Differential Revision: https://phabricator.services.mozilla.com/D133251
99% of the fetch tasks that pull from a git repo pull from a specific
revision, as specifically intended. There is however one exception,
which is the clang-trunk fetch, which is intended to always pull the
last revision on clang trunk. Unfortunately, the log also doesn't say
what specific commit it checked out.
Ironically, the log for all the other tasks does say which commit is
checked out, because git prints it out when it goes in detached head
state, which it does when checking out a specific sha1 (so in that
case, it's printing something we already know).
In the one instance where we checkout a branch, we don't rely on the
branch being checked out, so we just forcibly go in the same state of
detached head in that case too.
Differential Revision: https://phabricator.services.mozilla.com/D133063
Now that `use_fetches` runs after `make_task_description`, tasks that
come in look more like tasks from kind_dependencies, and the code to
extract the alias, artifact and environment info can now be shared.
Differential Revision: https://phabricator.services.mozilla.com/D132650
This will help to avoid the duplication of the code that takes artifact
names, environment and aliases for each fetch, and will allow changes to
toolchain.py that wouldn't be possible otherwise.
The only difference it makes to the resulting taskgraph is that scopes
are not in the same order as before, but that doesn't matter.
Differential Revision: https://phabricator.services.mozilla.com/D132649
This is being split to a new file mainly because the worker definitions take up
a lot of LOC. Getting this out of the main transforms makes it easier to
navigate.
Differential Revision: https://phabricator.services.mozilla.com/D132411
Chunking transforms should always go last since there should never be
configuration that is different between chunks (other than what is set by the
chunking transforms themselves).
Running them last also ensures we don't do extra work as the number of tasks
grows by 10x or more after splitting chunks.
Differential Revision: https://phabricator.services.mozilla.com/D132410
Future commits will tease this apart a bit more, but for now this helps
crystallize the order in which transforms are applied. The flow of the overall
test transforms goes something like this:
1. Enter 'transforms/test/__init__.py'
2. Validate all tasks against the test_description_schema
3. Run sibling transforms (starting with 'variant.py' and ending with 'other.py' for now)
4. Make the job description
As we pull more transforms out of 'other.py' and into their own smaller
transform files, it will be clear that the order in which these smaller files
run is important. Adding new transforms will no longer involve picking some
random spot to insert it.
Differential Revision: https://phabricator.services.mozilla.com/D132409
Many things need to be set "by variant", so therefore we need to resolve them
early on in the process. By pulling them out into a separate file that
explicitly runs before the "__init__.py" transforms, we A) make it harder
for someone to accidentally add their transform ahead of them, and B) reduce
some of the clutter in "__init__.py".
Differential Revision: https://phabricator.services.mozilla.com/D132408
Bug 1713258 added KEY to the beetmover manifest, but it gets excluded
from `upstreamArtifacts` in `generate_beetmover_upstream_artifacts`
because it's not in the signing task's `release-artifacts` attribute,
and so it doesn't actually work.
Differential Revision: https://phabricator.services.mozilla.com/D132470
This patch changes the viewport retries to 60. This means that we will now check the first 1 second of the video to try to find a viewport.
Differential Revision: https://phabricator.services.mozilla.com/D132379
This patch adds a new toolchain run property: toolchain-env which allows
specifying additional environment variables that will be added to the worker
when consuming the toolchain artifact.
Differential Revision: https://phabricator.services.mozilla.com/D131987
We have a 3 way circular dependency in the test transforms:
1. The 'raptor' transforms depend on 'split_variants'
2. The 'get_mobile_project' utility function depends on the 'raptor' transforms
3. The 'split_variants' transform depends on the 'get_mobile_project' utility function
The only reason things are working is that in step 3 we don't need the result
of 'get_mobile_project' to be perfect. This revision breaks the circular
dependency by checking for 'android' in the test-platform rather than relying
on 'get_mobile_project == geckoview'.
While this is admittedly not as precise, we only apply the particular variant
that needs this to geckoview tasks in the first place. In other words, this
revision does not cause any additional tasks to be added / removed.
Differential Revision: https://phabricator.services.mozilla.com/D132072
We'd like to break down the test transforms into smaller well-ordered pieces.
This will allow us to run any transforms defined in files under the
'transforms.test' directory.
Differential Revision: https://phabricator.services.mozilla.com/D132071
This ensures validation happens before we handle variants and raptor. It means
we no longer make any meaningful changes to the graph before validating.
Differential Revision: https://phabricator.services.mozilla.com/D132070
In the test transforms we currently have general 'test_description_schema' and
a schema for raptor / browsertime tasks. Prior, the raptor schema was validated
*before* the test schema, and we were doing a lot of validation twice.
This revision switches the order such that the test schema is first evaluated,
and then the raptor one is. To accomplish this, we create a 'raptor' subconfig
that the 'test_description_schema' completely ignores (and leaves up to the
raptor schema to evaluate).
The benefit of this change is that all test tasks will have a single entry point
when they get into the 'test' transforms. This makes reasoning about their
configuration much easier and lays the ground work to start splitting even more
transforms out into other modules.
Differential Revision: https://phabricator.services.mozilla.com/D132069
This includes:
transforms/tests.py -> transforms/test/__init__.py
transforms/raptor.py -> transforms/test/raptor.py
This is a pre-cursor to splitting the file up into multiple smaller files under
the new 'test' transform directory.
Differential Revision: https://phabricator.services.mozilla.com/D132068
Since we're going to enable EFT on Nightly and let it bake there a bit to
get feedback, we might face a situation where we have to disable EFT.
Having these test variants will help us to not regress non-EFT cases so if
we ever have to rollback EFT, there wouldn't be extra work.
Depends on D131477
Differential Revision: https://phabricator.services.mozilla.com/D131600
Mike Hommey
Michelle Goossens
Andrew Halberstadt
Tom Ritter
Ryan VanderMeulen
Gregory Mierzwinski
Noemi Erli
Agi Sferro
Alexis Beingessner
Joel Maher
Barret Rennie
Alexandru Michis
Heitor Neiva
Sandor Molnar
Sylvestre Ledru
Julien Cristau
Marco Castelluccio
Nicolas Chevobbe