Various updates to the custom retrigger action so that, without any custom changes to
parameters, the retriggered task runs with the same parameters as the original task.
Several issues were found and corrected, notably:
- parameters like --allow-software-gl-layers were ignored
- MOZHARNESS_TEST_PATHS was ignored
- many parameter customizations in the desktop mozharness configs were ignored
- mochitest suite/subsuite/flavor selection was not always correct
- using repeat=1 by default meant that each test ran twice
Differential Revision: https://phabricator.services.mozilla.com/D70457
Keeping the same for the currently chosen strategy for try auto, since we
don't want to decrease its regression detection rate.
We also add a new shadow scheduler which uses the reduced set with a higher
confidence threshold.
Differential Revision: https://phabricator.services.mozilla.com/D71205
This make four changes:
- use shippable/pgo builds on win64 and android platforms
- use linux64 instead of linux to get test packages for out-of-tree tasks
- consistently use `build` as the dependency name in tasks
- use the geckodriver toolchain, rather than the one packed in tasks
Differential Revision: https://phabricator.services.mozilla.com/D70943
Currently, we build android emulator packages manually and upload to
tooltool.
This patch switches it to be pulled from the toolchain built artifacts.
This also allows android tests to run in the staging environment.
Differential Revision: https://phabricator.services.mozilla.com/D68617
This patches fixes several problems found on Raptor and the condprof:
Raptor:
- Make sure the conditioned profile dir is removed after
it's been used, not before.
- Adds the --project option to raptor so we know if we're on try
autoland or mozilla-central.
- Both Fennec and Fenix are deactivated for now
- Use the allow-downgrade flag to be flexible on build ids (the next step will be bug 1628666)
Conditioned profiles, curation of the profile prefs:
- Fully deactivates Normandy during Raptor tests (app.normandy.enabled)
- Removes any GFX blacklisting (gfx.blacklist.*)
- Removes any marionette pref
- Enforce extensions sideloading (extensions.startupScanScopes)
Differential Revision: https://phabricator.services.mozilla.com/D70518
Changes:
- update the nodejs download path.
- save the file with a specific name and use that name going forward to reduce number of times the same filename is referenced.
Differential Revision: https://phabricator.services.mozilla.com/D71088
--HG--
extra : moz-landing-system : lando
We'll want some kind of backstop no matter what optimization algorithm we use.
We don't want to go too long without running any given task so we can find
regressions quickly and have a good merge candidate.
This pulls the logic that handles this out of the SETA strategy and into its
own strategy.
This will also make the SETA shadow scheduler more representative of what the
algorithm is doing.
Note in the future we may find ways to make this backstop more efficient (i.e
only run tasks that didn't run in the last 9 pushes for example).
Depends on D68621
Differential Revision: https://phabricator.services.mozilla.com/D68622
--HG--
extra : moz-landing-system : lando
This allows to nest strategies without having to register ever intermediate
composite strategy first. For example:
All(Any("skip-unless-schedules", "seta"), "backstop")
Prior to this patch, we'd need to register that 'Any' one first and then use it
in the 'All'.
Depends on D68620
Differential Revision: https://phabricator.services.mozilla.com/D68621
--HG--
extra : moz-landing-system : lando
I'd like to implement a 'backstop' strategy, such that it will prevent all other
optimizers from removing tasks under certain conditions (e.g every 10th push).
The nicest way to implement this seems to be an 'All' composite strategy
(similar to 'Either' which this patch renames to 'Any'). This means we could
do something like:
All("seta", "backstop")
which means we would only remove tasks if *all* substrategies say to remove
tasks.
Differential Revision: https://phabricator.services.mozilla.com/D68620
--HG--
extra : moz-landing-system : lando
We'll want some kind of backstop no matter what optimization algorithm we use.
We don't want to go too long without running any given task so we can find
regressions quickly and have a good merge candidate.
This pulls the logic that handles this out of the SETA strategy and into its
own strategy.
This will also make the SETA shadow scheduler more representative of what the
algorithm is doing.
Note in the future we may find ways to make this backstop more efficient (i.e
only run tasks that didn't run in the last 9 pushes for example).
Depends on D68621
Differential Revision: https://phabricator.services.mozilla.com/D68622
--HG--
extra : moz-landing-system : lando
This allows to nest strategies without having to register ever intermediate
composite strategy first. For example:
All(Any("skip-unless-schedules", "seta"), "backstop")
Prior to this patch, we'd need to register that 'Any' one first and then use it
in the 'All'.
Depends on D68620
Differential Revision: https://phabricator.services.mozilla.com/D68621
--HG--
extra : moz-landing-system : lando
I'd like to implement a 'backstop' strategy, such that it will prevent all other
optimizers from removing tasks under certain conditions (e.g every 10th push).
The nicest way to implement this seems to be an 'All' composite strategy
(similar to 'Either' which this patch renames to 'Any'). This means we could
do something like:
All("seta", "backstop")
which means we would only remove tasks if *all* substrategies say to remove
tasks.
Differential Revision: https://phabricator.services.mozilla.com/D68620
--HG--
extra : moz-landing-system : lando
These configs now have the same `fetches`/`toolchain`/`by-test-platform` values
as the `xpcshell` config in `xpcshell.yml`.
Differential Revision: https://phabricator.services.mozilla.com/D70650
--HG--
extra : moz-landing-system : lando
Currently the linux64-clang-9-win-cross toolchain depends on the win64-clang-cl toolchain for a few files. This causes very long lead times when toolchains are rebuilt, because of the un-parallelizable chain of tasks win64-clang-cl -> linux64-clang-9-win-cross -> builds.
As a partial mitigation, this patch adds a single-stage clang-cl build for consumption by the cross toolchain. It's not a very high quality build, but good enough for the purpose it serves, while being faster to build.
Differential Revision: https://phabricator.services.mozilla.com/D70254
--HG--
rename : build/build-clang/clang-win64.json => build/build-clang/clang-win64-1stage.json
extra : moz-landing-system : lando
The goal being to detect potential build failures for smoosh on a tier 1 build.
Differential Revision: https://phabricator.services.mozilla.com/D69621
--HG--
extra : moz-landing-system : lando
The goal being to detect potential build failures for smoosh on a tier 1 build.
Differential Revision: https://phabricator.services.mozilla.com/D69621
--HG--
extra : moz-landing-system : lando
Activates the conditioned profile by doing the following changes:
- make sure the conditioned profile dir is removed after
it's been used, not before.
- add the --project option to raptor so we know if we're on try
or mozilla-central.
- Both Fennec and Fenix are deactivated for now.
- Remove any gfx.blacklist.* prefs when using a conditioned profile
Differential Revision: https://phabricator.services.mozilla.com/D70518
--HG--
extra : moz-landing-system : lando
The `video_location` attribute (which was the URL or path of the video) was
removed from the `Job` type, due to it no longer being possible to be a URL.
This was because videos are now passed in through the browsertime-results.tgz
artifact, instead of specified separately in the job description.
This error message was missed in the refactor and was causing failures due to
visualmetrics.py to appear to be caused by our wrapper script. This will not
fix the underlying error causing the intermittents in the first place, but now
the real cause will appear in the log instead.
Differential Revision: https://phabricator.services.mozilla.com/D70062
--HG--
extra : moz-landing-system : lando
Make sure the conditioned profile dir is removed after
it's been used, not before. This patch also adds the
--project option to raptor so we know if we're on try
or mozilla-central. Both Fennec and Fenix are deactivated
for now.
Differential Revision: https://phabricator.services.mozilla.com/D70518
--HG--
extra : moz-landing-system : lando
Since we don't look at the changesets, there is no need for hgmo to generate
them for us.
Differential Revision: https://phabricator.services.mozilla.com/D70567
--HG--
extra : moz-landing-system : lando
CLOSED TREE
Backed out changeset f42150bdee2b (bug 1626772)
Backed out changeset ab5b637f714a (bug 1626772)
Backed out changeset fd4026a9f380 (bug 1626772)
For building `macosx64-clang-tidy` with `linux64-clang-10` we need to build `cctools`
with `clang-10`.
Differential Revision: https://phabricator.services.mozilla.com/D70064
--HG--
extra : moz-landing-system : lando
Patches that are applied on top on `clang-10` repo are based on the original
patches from our trunk and have been rebased on top of `clang-10`.
Please see as an example: `find_symbolizer_linux.patch` copied to `find_symbolizer_linux_clang_10.patch`.
Differential Revision: https://phabricator.services.mozilla.com/D70063
--HG--
rename : build/build-clang/clang-linux64.json => build/build-clang/clang-10-linux64.json
rename : build/build-clang/find_symbolizer_linux.patch => build/build-clang/find_symbolizer_linux_clang_10.patch
rename : build/build-clang/llvmorg-11-init-4265-g2dcbdba8540.patch => build/build-clang/llvmorg-11-init-4265-g2dcbdba8540_clang_10.patch
rename : build/build-clang/rG7e18aeba5062.patch => build/build-clang/rG7e18aeba5062_clang_10.patch
rename : build/build-clang/rename_gcov_flush.patch => build/build-clang/rename_gcov_flush_clang_10.patch
rename : build/build-clang/tsan-hang-be41a98ac222.patch => build/build-clang/tsan-hang-be41a98ac222_clang_10.patch
rename : build/build-clang/unpoison-thread-stacks.patch => build/build-clang/unpoison-thread-stacks_clang_10.patch
extra : moz-landing-system : lando
Geoff Brown
Marco Castelluccio
Tom Prince
Andreea Pavel
Wander Lairson Costa
shravanrn@gmail.com
Tarek Ziadé
Edwin Takahashi
Sylvestre Ledru
Joel Maher
Andrew Halberstadt
Florin Strugariu
Mihai Tabara
Nicholas Nethercote
Mihai Alexandru Michis
David Major
Matthew Gaudet
Dzmitry Malyshau
Bogdan Tara
Bartłomiej Piotrowski
Aki Sasaki
Barret Rennie
Csoregi Natalia
Steve Fink
Andi-Bogdan Postelnicu