Previously the OSX machines had a safari version (14.x.x) that was not
up to date (15.x.x) for the 10.15.7 machines. With this recent safari
version upgrade, the previous test set needs to be updated since now
*some* tests that used to permafail no longer do.
Since we can't upgrade safari any further on these machines, this is as
best as we currently can do. The test set will be revisited in tp7 work
and more generally speaking, the apple silicon machines will have the
more up to date Safari version soon.
Differential Revision: https://phabricator.services.mozilla.com/D187942
This should be useful for developers when they want to quickly iterate
on some changes and test it in CI. Currently we only have shippable
builds available on macosx, but both opt and shippable on windows and
linux in CI.
Differential Revision: https://phabricator.services.mozilla.com/D188311
Due to some changes upstream in depot_tools, pgrep is now a required cli
tool. In particular, we fail during the fetching of the source code due
to a pgrep check if the compiler proxy is running.
Differential Revision: https://phabricator.services.mozilla.com/D188827
Because we use an older version of Python than the one generally used on
the host system, this implied enforcing a few package requirements.
Differential Revision: https://phabricator.services.mozilla.com/D180057
I want to add a fetch for a rust crate archive from crates.io, and those
don't come with an extension in the url, so we can't rely on that.
Content sniffing is easy enough to put in place.
Differential Revision: https://phabricator.services.mozilla.com/D188152
And don't extract apks in webrender tasks.
This fixes enough of bug 1853865 to unblock the fetch-content change.
(technically, the exception for dmg is not needed, but it doesn't hurt)
Differential Revision: https://phabricator.services.mozilla.com/D188564
We never really had such tasks, so that they failed was not visible. The
problem is that the unpack code doesn't handle symbolic links properly,
and mac builds have a symbolic link for the icon that sends to
/Applications. What we really want to unpack, though, is the
Firefox*.app/Contents/Resources directory, so do that.
While doing that, we also change the path we unpack for other platforms,
which has the side effect of removing precomplete, so we don't need to
alter. It should have been this way all along.
Differential Revision: https://phabricator.services.mozilla.com/D188292
Actually, Fenix's minimum is 21 even if 32-bit and Google Play Service has
discontinued 4.4 support. We would like to increase Lite's minimum API to 21
even if 32-bit.
Differential Revision: https://phabricator.services.mozilla.com/D187861
Adds hardened signing config in ci/config.yml
Switches the signing behaviour for mac-signing to mac_sign_and_pkg_hardened
Depends on D187245
Differential Revision: https://phabricator.services.mozilla.com/D188115
These tests fail intermittently or permanently. The plan is to fix them in a
burndown. After they all pass, the tier can be changed to 2 or 1, and their
results will be monitored (sheriffed).
Differential Revision: https://phabricator.services.mozilla.com/D188362
I want to add a fetch for a rust crate archive from crates.io, and those
don't come with an extension in the url, so we can't rely on that.
Content sniffing is easy enough to put in place.
Differential Revision: https://phabricator.services.mozilla.com/D188152
For most suites this just creates the tasks so they can be scheduled on try.
For suites that are already running on autoland, it creates debug tasks there
as well. Luckily they all pass.
Differential Revision: https://phabricator.services.mozilla.com/D187055
I want to add a fetch for a rust crate archive from crates.io, and those
don't come with an extension in the url, so we can't rely on that.
Content sniffing is easy enough to put in place.
Differential Revision: https://phabricator.services.mozilla.com/D188152
Bug 1750820 (Make webgl-ipc the default configuration for mochitests) is
the goal, but we need more time to get that working, and we don't want
to lose coverage during this gap.
Differential Revision: https://phabricator.services.mozilla.com/D187607
Both the from_deps and release_deps transforms were adding the same task
as a dependency but under different keys. We don't actually need
release_deps for anything else here as far as I can tell, so skip it.
Differential Revision: https://phabricator.services.mozilla.com/D187691
Most of this patch is ripping `command-context` out from Gecko. The other parts are the fairly straightforward conversions from `command-context` to `task-context`.
Differential Revision: https://phabricator.services.mozilla.com/D186822
This is only sortof a new issue - it's come up because we longer allow `group_by` functions to be overridden, and the taskgraph version of the `single` group doesn't handle the `only-for` blocks that we use all over the place.
This patch provides a quick fix -- renaming the Gecko `single` group-by to `single-with-filters`, and using that in most places. (There were a couple of places that switching to `with-attributes` was simple enough - but in many cases we cannot yet replicate this functionality with `from-deps` alone AFAIK.)
Differential Revision: https://phabricator.services.mozilla.com/D186821
Most of this patch is ripping `command-context` out from Gecko. The other parts are the fairly straightforward conversions from `command-context` to `task-context`.
Differential Revision: https://phabricator.services.mozilla.com/D186822
This is only sortof a new issue - it's come up because we longer allow `group_by` functions to be overridden, and the taskgraph version of the `single` group doesn't handle the `only-for` blocks that we use all over the place.
This patch provides a quick fix -- renaming the Gecko `single` group-by to `single-with-filters`, and using that in most places. (There were a couple of places that switching to `with-attributes` was simple enough - but in many cases we cannot yet replicate this functionality with `from-deps` alone AFAIK.)
Differential Revision: https://phabricator.services.mozilla.com/D186821
We've created a new, dev bouncer instance as part of https://bugzilla.mozilla.org/show_bug.cgi?id=1819405 / https://mozilla-hub.atlassian.net/browse/SVCSE-1186. The main difference between it and the stage instance is that dev points at ftp.stage.mozaws.net. This happens to be the place that try releases are uploaded to - which means that we can get bouncer-check and final verify to finally pass on try releases by using it.
(bouncer stage points at archive.mozilla.org - so even though we can successfully submit releases to its admin endpoint, what we submit does not often line up with what is served when we query the public endpoint.)
Differential Revision: https://phabricator.services.mozilla.com/D186634
- use deb.debian.org instead of ftp.nl.debian.org and debug.mirrors.debian.org,
which should be much faster
- update the URL for the unofficial debian-multimedia archive
- also download symbols from the new debian-security-debug archive
Differential Revision: https://phabricator.services.mozilla.com/D187376
The new join/split node direction mode seems working well in the wild because
we have no regression reports about it. Therefore, we won't revert the pref
anymore, thus, the legacy mode behavior check is not required.
Differential Revision: https://phabricator.services.mozilla.com/D186680
As far as I can tell the only difference is that upstream checks the
repository_type param before considering pushlog id.
Differential Revision: https://phabricator.services.mozilla.com/D187116
`index-search` is already registered by upstream taskgraph, so this is a
no-op (`register_strategy` ignores the duplicate registration).
Differential Revision: https://phabricator.services.mozilla.com/D187115
Because it bumps the alignment requirement on aarch64, we make the
elfhack test create more relocations to make the .rela.dyn section large
enough for the test to pass.
Differential Revision: https://phabricator.services.mozilla.com/D181684
For most suites this just creates the tasks so they can be scheduled on try.
For suites that are already running on autoland, it creates debug tasks there
as well. Luckily they all pass.
Differential Revision: https://phabricator.services.mozilla.com/D187055
This is designed to reduce time that we spend processing variants.
Adding a `mozinfo` variable to variants.yml allows for making it easier
to find the variable we use in manifests. I found in a few cases it was
difficult to find how we could `skip-if` a specific variant. Doing this
also reduces the lookups and longer list of if/elif statements to guess
mozinfo properly.
Differential Revision: https://phabricator.services.mozilla.com/D186663
Also bumping the cache version, because mercurial 6.1 enabled
share-safe, and robustcheckout in the old image doesn't understand that,
so upon encountering a repo cloned with the new version it would delete
everything and re-clone from scratch.
Differential Revision: https://phabricator.services.mozilla.com/D186543
We're updating from ubuntu 20.04.3 / mercurial 5.8.1 to ubuntu 20.04.6 /
mercurial 6.4.3. We're also picking up recent changes to run-task and
robustcheckout.
Differential Revision: https://phabricator.services.mozilla.com/D186532
M2 capacity is small and with 3/4 workers out I am removing all tests for the firefox essential pageload tests (which should reduce testing on the M2s running osx13 by at a minimum of 90%)
Differential Revision: https://phabricator.services.mozilla.com/D186480
Changeset 498fdab9d21f "Migrate Android emulator tests from AWS to GCP" added
this alias and used it in one place, then a week later changeset 10f1675a7291
"Remove override that makes no sense" removed the use, but the alias stayed in
place.
The actual gecko-t/t-linux-metal worker pool was removed in bug 1818761
(https://hg.mozilla.org/ci/ci-configuration/rev/fb05add3bda9).
Differential Revision: https://phabricator.services.mozilla.com/D186350
The only differences between the linux64-aarch64 mozconfigs and linux64
mozconfigs are:
- unset NASM
- ac_add_options --target=aarch64
- unset MOZ_STDCXX_COMPAT
The mozconfig is only used once, so we can use a one-off config
instead.
While here, we remove `unset NASM`, which has been useless since bug
1686646.
Differential Revision: https://phabricator.services.mozilla.com/D185389
To create network tests that capture various codepaths and optimizations we need to fix network particulars such as latency and bandwidth.
This module provides access to the dev tools throttler for use in browsertime pageload tests.
Differential Revision: https://phabricator.services.mozilla.com/D183618
This is a bustage fix because I accidentally converted the
`partner_repackage.py` transforms to the new format without
converting a kind that used them. Normally I've been trying
to convert all kinds that use a given transform file at
once but I neglected this one.
Differential Revision: https://phabricator.services.mozilla.com/D185713
Thunderbird will be publishing MSIX packages on the MS Store using a separate
account. The repackaging command supports setting the vendor name, but the task
configuration does not.
Differential Revision: https://phabricator.services.mozilla.com/D185075
Due to some upstream changes, cipd binaries were not properly being
setup and necessary files were missing during the run hooks step (like dirmd).
This is only affecting macOS and Linux, so this patch just directly runs
the setup script before we proceed with the rest of the build sequence.
Differential Revision: https://phabricator.services.mozilla.com/D185497
While we're here, add the missing pkgconf dependency that leads to
unrelated error messages in the task log when other problems happen
(like in bug 1813506).
Differential Revision: https://phabricator.services.mozilla.com/D185240
Test tasks have contain a ReadOnlyDict from mozbuild.util (under
task["extra"]["test-setting"]) that PyYAML's safe_dump will not serialize
as it has no way of representing it.
TGDumper is a subclass that converts ReadOnlyDicts to regular dicts, and then
dumps accordingly. The assumption is that dumping a taskgraph to YAML is useful
for debugging, round-trips back to a taskgraph object are unnecessary, and
it's acceptable to lose the "read-only" status of these objects when serialized.
(This is what happens when serialized to JSON.)
Another quirk of dumping to YAML is objects are aliases and de-duplicated. This
results in things like `test-setting: *id1337`, which is a reference to
`test-setting: &id1337` (and the definition that follows). That's not too
readable, and again since round-trips are not needed TGDumper disables the use
of these aliases.
Differential Revision: https://phabricator.services.mozilla.com/D185239
Normally the `taskgraph_decision` code is not run outside of CI, but when using
`mach test-action-callback` locally it is used. The hardcoded path is problematic
when doing so.
Differential Revision: https://phabricator.services.mozilla.com/D185223
The move to `from-deps` changed the dependency edge name from `build` to
`system-symbols`. Apparently this is what the first path segment in an
`artifact-reference` is.
We should probably change the separator to a `:` or something here.
Differential Revision: https://phabricator.services.mozilla.com/D185297
Also:
* Drop it from the list of release promotion projects, because we won't be running release promotion there
* Add it to the nightly scope list for signing, so we can sign with real certs
Differential Revision: https://phabricator.services.mozilla.com/D184817
This results in an identical diff after running:
$ ./mach taskgraph full -p taskcluster/test/params --diff .~1 -J --fast --exclude-key attributes.primary-kind-dependency
Differential Revision: https://phabricator.services.mozilla.com/D181900
This patch adds the chromedrivers for version 115. It also removes multiple old versions. One thing to note is that this will trigger some expected failures on Android because we're removing the version 109 chromedriver, but there are still some devices that are running version 109. This adds variance to our results since we have multiple major versions being mixed together so we should have them fail instead.
Furthermore, the pathing for the artifacts have changed so we need to add some code to browsertime/base.py to handle this difference. We can't do this on the taskcluster side because we need to know the version.
Differential Revision: https://phabricator.services.mozilla.com/D183985
This changes the docker image used for:
- artifact-build-linux64-artifact/opt
- l10n-*-shippable/opt
- openh264-plugin-{linux,macosx}*
- repackage-{l10n,linux,macosx,msix,win}*
- repackage-shippable-l10n-msix*
- shippable-l10n*
While here, because builds are also mozharness-based, removed the
duplicate default from the build transform.
Differential Revision: https://phabricator.services.mozilla.com/D183928
When it's not built on the same kind of docker image that runs it,
there's a possibility of incompatibility because of the recent
change in openssl library version. This happens when crossing between
debian 11 and debian 12, for instance.
Differential Revision: https://phabricator.services.mozilla.com/D183944
Upstream taskgraph removed the special casing of `docker-image` tasks w.r.t
implicitly adding them to the graph, and instead switched over to relying on
the `always_target` feature to accomplish this.
However, in Gecko we usually disable the `always_target` feature entirely (only
enabling it on autoland and try). As a compromise, we implemented a feature in
Taskgraph to allow selectively enabling `always_target` to a specific list of
kinds.
This patch gets Gecko to start using this new feature.
Differential Revision: https://phabricator.services.mozilla.com/D184107
The task transform module keeps registries of payload and index
builders, identified by a "name" string; AFAICT nothing prevents
registering different functions under the same name, which seems
potentially confusing.
Differential Revision: https://phabricator.services.mozilla.com/D183960
Currently in mach try perf, the Benchmarks category is empty on android because the tasks don't have `benchmark` in their task label. This patch changes the task labels so that the benchmark tests get included in the category.
Differential Revision: https://phabricator.services.mozilla.com/D183864
`set_implementation` can get the implementation from the config just
fine, but we were insisting on `worker` being set on the task for no
good reason.
Differential Revision: https://phabricator.services.mozilla.com/D183874
This results in an identical diff after running:
$ ./mach taskgraph full -p taskcluster/test/params --diff .~1 -J --fast --exclude-key attributes.primary-kind-dependency
Differential Revision: https://phabricator.services.mozilla.com/D181900
Initiating a speculative connection via privileged js prior to pageload.
This an end-to-end test that will ensure the connection is made and used.
Differential Revision: https://phabricator.services.mozilla.com/D181812
The android jobs are currently run on a machine where some packages are
installed on the system and out of control of taskcluster.
Using the taskcluster-built python ignores those package, have it use
local packages instead.
Differential Revision: https://phabricator.services.mozilla.com/D182961
Release promotion reuses tasks generated by the on-push decision task,
to avoid duplicating work. For cached tasks however, that means reusing
index lookups that can be out of date by the time relpro runs, and
requiring a new push to pick up rebuilt cached tasks. By listing the
cached kinds in rebuild-kinds, we force new index lookups at release
promotion time, to pick up e.g. new docker images without an extra push.
Differential Revision: https://phabricator.services.mozilla.com/D182774
This has an important impact on compilation time, so we limit that to
shippable builds and under MOZ_AUTOMATION.
The benefit on speedometer time is not significant on macosx and android builds, so
we skip that platform.
Builds tend to stall on arm platforms, so we limit the target to
x86_64.
Differential Revision: https://phabricator.services.mozilla.com/D179912
clang trunk recently removed the granularity between fstream and
filesystem, and to have fstream, one needs to enable filesystem, now.
Differential Revision: https://phabricator.services.mozilla.com/D182135
Alexandre Lissy
Iulian Moraru
serge-sans-paille
Mike Hommey
KS
Sandor Molnar
Julien Cristau
Narcis Beleuzu
Makoto Kato
Heitor Neiva
Haik Aftandilian
Gabriele Svelto
Sebastian Hengst
Noemi Erli
Andrew Halberstadt
Joel Maher
Kelsey Gilbert
owlishDeveloper
alwu
Ben Hearsum
Logan Rosen
Alex Finder
Nicolas Chevobbe
Erich Gubler
Mark Banner
Masayuki Nakano
Ryan VanderMeulen
hsingh
Otto Länd
Norisz Fay
Glenn Watson
andrej
Andrew Creskey
Tom Marble
Amin Bandali
James Teh
Rob Lemley
Kershaw Chang
Jesse Schwartzentruber
Tom Ritter
Sylvestre Ledru
Greg Mierzwinski
Stanca Serban
Masatoshi Kimura
Natalia Csoregi
Andi-Bogdan Postelnicu
Steve Fink