Changes:
Applies the `filter_tasks_by_blacklist` method to try syntax pushes as well.
- moved `TARGET_TASK_BLACKLIST`and `filter_tasks_by_blacklist` method to live in `taskcluster/taskgraph/target_tasks.py`.
- removed existing filters against `ccov, windows10-aarch64` and `android-hw` filters against try syntax pushes.
- update imports for `fuzzy` and `chooser` selectors to refer to the new location of `filter_tasks_by_blacklist` method.
The reason for moving the logic (again) from `tools/tryselect` to `taskcluster/` is due to the placement of `try_option_syntax` and `target_tasks` files and both of those files handle the processing of `mach try syntax` pushes.
Differential Revision: https://phabricator.services.mozilla.com/D71698
The clang_rt.builtins lib files produced by the 1stage build do not include
functions necessary for 128bit integers. This functionality is required by the
OpenPGP libraries that Thunderbird is using, specifically Botan's bigint module.
When compiler-rt is built with MSVC, these functions are not included
by design.
Differential Revision: https://phabricator.services.mozilla.com/D71785
* For wrench builds, the clang toolchain must now be fetched and be
present in PATH for bindgen to work. When building OSMesa, we must
therefore set `LLVM_CONFIG=no` so that it does not attempt to build
llvmpipe.
* For wrench Mac cross-compiles, we must be careful to expose CFLAGS
and similar variables using the target-specific variable names, so
that host builds do not attempt to use flags intended for the
target. When building OSMesa we must use the generic variable `CC`,
so now we additionally set `HOST_CC`, so that host builds use the
host variable rather than the generic one.
* Similarily, for wrench android builds we must use a fork of
cargo-apk which sets the target-specific variables rather than the
generic ones. Otherwise we would attempt to use the NDK toolchain
for host builds.
Differential Revision: https://phabricator.services.mozilla.com/D70031
Firefox still does not default to Wayland when it's available (or when
X11 socket is not accessible). This can be changed by MOZ_ENABLE_WAYLAND=1
but as it's not the default, it's safer to just drop access to Wayland.
Differential Revision: https://phabricator.services.mozilla.com/D71448
Various updates to the custom retrigger action so that, without any custom changes to
parameters, the retriggered task runs with the same parameters as the original task.
Several issues were found and corrected, notably:
- parameters like --allow-software-gl-layers were ignored
- MOZHARNESS_TEST_PATHS was ignored
- many parameter customizations in the desktop mozharness configs were ignored
- mochitest suite/subsuite/flavor selection was not always correct
- using repeat=1 by default meant that each test ran twice
Differential Revision: https://phabricator.services.mozilla.com/D70457
Keeping the same for the currently chosen strategy for try auto, since we
don't want to decrease its regression detection rate.
We also add a new shadow scheduler which uses the reduced set with a higher
confidence threshold.
Differential Revision: https://phabricator.services.mozilla.com/D71205
This make four changes:
- use shippable/pgo builds on win64 and android platforms
- use linux64 instead of linux to get test packages for out-of-tree tasks
- consistently use `build` as the dependency name in tasks
- use the geckodriver toolchain, rather than the one packed in tasks
Differential Revision: https://phabricator.services.mozilla.com/D70943
Currently, we build android emulator packages manually and upload to
tooltool.
This patch switches it to be pulled from the toolchain built artifacts.
This also allows android tests to run in the staging environment.
Differential Revision: https://phabricator.services.mozilla.com/D68617
This patches fixes several problems found on Raptor and the condprof:
Raptor:
- Make sure the conditioned profile dir is removed after
it's been used, not before.
- Adds the --project option to raptor so we know if we're on try
autoland or mozilla-central.
- Both Fennec and Fenix are deactivated for now
- Use the allow-downgrade flag to be flexible on build ids (the next step will be bug 1628666)
Conditioned profiles, curation of the profile prefs:
- Fully deactivates Normandy during Raptor tests (app.normandy.enabled)
- Removes any GFX blacklisting (gfx.blacklist.*)
- Removes any marionette pref
- Enforce extensions sideloading (extensions.startupScanScopes)
Differential Revision: https://phabricator.services.mozilla.com/D70518
Changes:
- update the nodejs download path.
- save the file with a specific name and use that name going forward to reduce number of times the same filename is referenced.
Differential Revision: https://phabricator.services.mozilla.com/D71088
--HG--
extra : moz-landing-system : lando
We'll want some kind of backstop no matter what optimization algorithm we use.
We don't want to go too long without running any given task so we can find
regressions quickly and have a good merge candidate.
This pulls the logic that handles this out of the SETA strategy and into its
own strategy.
This will also make the SETA shadow scheduler more representative of what the
algorithm is doing.
Note in the future we may find ways to make this backstop more efficient (i.e
only run tasks that didn't run in the last 9 pushes for example).
Depends on D68621
Differential Revision: https://phabricator.services.mozilla.com/D68622
--HG--
extra : moz-landing-system : lando
This allows to nest strategies without having to register ever intermediate
composite strategy first. For example:
All(Any("skip-unless-schedules", "seta"), "backstop")
Prior to this patch, we'd need to register that 'Any' one first and then use it
in the 'All'.
Depends on D68620
Differential Revision: https://phabricator.services.mozilla.com/D68621
--HG--
extra : moz-landing-system : lando
I'd like to implement a 'backstop' strategy, such that it will prevent all other
optimizers from removing tasks under certain conditions (e.g every 10th push).
The nicest way to implement this seems to be an 'All' composite strategy
(similar to 'Either' which this patch renames to 'Any'). This means we could
do something like:
All("seta", "backstop")
which means we would only remove tasks if *all* substrategies say to remove
tasks.
Differential Revision: https://phabricator.services.mozilla.com/D68620
--HG--
extra : moz-landing-system : lando
We'll want some kind of backstop no matter what optimization algorithm we use.
We don't want to go too long without running any given task so we can find
regressions quickly and have a good merge candidate.
This pulls the logic that handles this out of the SETA strategy and into its
own strategy.
This will also make the SETA shadow scheduler more representative of what the
algorithm is doing.
Note in the future we may find ways to make this backstop more efficient (i.e
only run tasks that didn't run in the last 9 pushes for example).
Depends on D68621
Differential Revision: https://phabricator.services.mozilla.com/D68622
--HG--
extra : moz-landing-system : lando
This allows to nest strategies without having to register ever intermediate
composite strategy first. For example:
All(Any("skip-unless-schedules", "seta"), "backstop")
Prior to this patch, we'd need to register that 'Any' one first and then use it
in the 'All'.
Depends on D68620
Differential Revision: https://phabricator.services.mozilla.com/D68621
--HG--
extra : moz-landing-system : lando
I'd like to implement a 'backstop' strategy, such that it will prevent all other
optimizers from removing tasks under certain conditions (e.g every 10th push).
The nicest way to implement this seems to be an 'All' composite strategy
(similar to 'Either' which this patch renames to 'Any'). This means we could
do something like:
All("seta", "backstop")
which means we would only remove tasks if *all* substrategies say to remove
tasks.
Differential Revision: https://phabricator.services.mozilla.com/D68620
--HG--
extra : moz-landing-system : lando
These configs now have the same `fetches`/`toolchain`/`by-test-platform` values
as the `xpcshell` config in `xpcshell.yml`.
Differential Revision: https://phabricator.services.mozilla.com/D70650
--HG--
extra : moz-landing-system : lando
Currently the linux64-clang-9-win-cross toolchain depends on the win64-clang-cl toolchain for a few files. This causes very long lead times when toolchains are rebuilt, because of the un-parallelizable chain of tasks win64-clang-cl -> linux64-clang-9-win-cross -> builds.
As a partial mitigation, this patch adds a single-stage clang-cl build for consumption by the cross toolchain. It's not a very high quality build, but good enough for the purpose it serves, while being faster to build.
Differential Revision: https://phabricator.services.mozilla.com/D70254
--HG--
rename : build/build-clang/clang-win64.json => build/build-clang/clang-win64-1stage.json
extra : moz-landing-system : lando
The goal being to detect potential build failures for smoosh on a tier 1 build.
Differential Revision: https://phabricator.services.mozilla.com/D69621
--HG--
extra : moz-landing-system : lando
The goal being to detect potential build failures for smoosh on a tier 1 build.
Differential Revision: https://phabricator.services.mozilla.com/D69621
--HG--
extra : moz-landing-system : lando
Activates the conditioned profile by doing the following changes:
- make sure the conditioned profile dir is removed after
it's been used, not before.
- add the --project option to raptor so we know if we're on try
or mozilla-central.
- Both Fennec and Fenix are deactivated for now.
- Remove any gfx.blacklist.* prefs when using a conditioned profile
Differential Revision: https://phabricator.services.mozilla.com/D70518
--HG--
extra : moz-landing-system : lando
The `video_location` attribute (which was the URL or path of the video) was
removed from the `Job` type, due to it no longer being possible to be a URL.
This was because videos are now passed in through the browsertime-results.tgz
artifact, instead of specified separately in the job description.
This error message was missed in the refactor and was causing failures due to
visualmetrics.py to appear to be caused by our wrapper script. This will not
fix the underlying error causing the intermittents in the first place, but now
the real cause will appear in the log instead.
Differential Revision: https://phabricator.services.mozilla.com/D70062
--HG--
extra : moz-landing-system : lando
Make sure the conditioned profile dir is removed after
it's been used, not before. This patch also adds the
--project option to raptor so we know if we're on try
or mozilla-central. Both Fennec and Fenix are deactivated
for now.
Differential Revision: https://phabricator.services.mozilla.com/D70518
--HG--
extra : moz-landing-system : lando
Edwin Takahashi
Cosmin Sabou
Rob Lemley
Marian Raiciof
Jamie Nicol
Nicholas Nethercote
Tom Prince
Justin Wood
Bartłomiej Piotrowski
Stefan Hindli
Jesse Schwartzentruber
Geoff Brown
Marco Castelluccio
Andreea Pavel
Wander Lairson Costa
shravanrn@gmail.com
Tarek Ziadé
Sylvestre Ledru
Joel Maher
Andrew Halberstadt
Florin Strugariu
Mihai Tabara
Mihai Alexandru Michis
David Major
Matthew Gaudet
Dzmitry Malyshau
Bogdan Tara
Aki Sasaki
Barret Rennie
Csoregi Natalia