Previously source-tests requiring a build had a "global" mapping of
platform to build type in kind.yml. But this made it confusing to
figure out how to add task-specific configuration. To simplify things,
make the configuration for the dependent platforms also go in the task
definition.
Differential Revision: https://phabricator.services.mozilla.com/D68636
--HG--
extra : moz-landing-system : lando
The `linux64-clang-9-cross` toolchain was forked from `linux64-clang-9` in https://hg.mozilla.org/integration/autoland/rev/ca923afe3ed4 in order to pick up a patch for Windows compilation.
This is no longer necessary for two reasons:
1. Windows builds stopped depending on that patch in bug 1608460.
2. Even if we still needed that patch, the `linux64-clang-9` toolchain gained all of the Windows patches in bug 1618473.
Therefore the `clang-dist` task can go back to using unmodified `linux64-clang-9`.
Differential Revision: https://phabricator.services.mozilla.com/D70263
--HG--
extra : moz-landing-system : lando
For building `macosx64-clang-tidy` with `linux64-clang-10` we need to build `cctools`
with `clang-10`.
Differential Revision: https://phabricator.services.mozilla.com/D70064
--HG--
extra : moz-landing-system : lando
Patches that are applied on top on `clang-10` repo are based on the original
patches from our trunk and have been rebased on top of `clang-10`.
Please see as an example: `find_symbolizer_linux.patch` copied to `find_symbolizer_linux_clang_10.patch`.
Differential Revision: https://phabricator.services.mozilla.com/D70063
--HG--
rename : build/build-clang/clang-linux64.json => build/build-clang/clang-10-linux64.json
rename : build/build-clang/find_symbolizer_linux.patch => build/build-clang/find_symbolizer_linux_clang_10.patch
rename : build/build-clang/llvmorg-11-init-4265-g2dcbdba8540.patch => build/build-clang/llvmorg-11-init-4265-g2dcbdba8540_clang_10.patch
rename : build/build-clang/rG7e18aeba5062.patch => build/build-clang/rG7e18aeba5062_clang_10.patch
rename : build/build-clang/rename_gcov_flush.patch => build/build-clang/rename_gcov_flush_clang_10.patch
rename : build/build-clang/tsan-hang-be41a98ac222.patch => build/build-clang/tsan-hang-be41a98ac222_clang_10.patch
rename : build/build-clang/unpoison-thread-stacks.patch => build/build-clang/unpoison-thread-stacks_clang_10.patch
extra : moz-landing-system : lando
This patch changes the test that browsertime runs for speedometer from `raptor-speedometer-APP` to `speedometer` and changes some other small issues like the missing app name in the `extraOptions` field in the perfherder data.
Differential Revision: https://phabricator.services.mozilla.com/D69581
--HG--
extra : moz-landing-system : lando
Note that it only runs jit-tests (not jsapi-tests and the slower jstests) and
only two jit-test configurations. This makes the job very fast, I see it finish
in just 12-13 minutes on Try.
When this lands it should be green. For now it's tier 2 and only runs on central.
When WarpBuilder gets closer to shipping we can consider changing that (and probably
run more tests).
Differential Revision: https://phabricator.services.mozilla.com/D69512
--HG--
extra : moz-landing-system : lando
This patch disabled chrome release tests on mozilla-central and has them run through the cron task instead.
Differential Revision: https://phabricator.services.mozilla.com/D69570
--HG--
extra : moz-landing-system : lando
Currently, we build android emulator packages manually and upload to
tooltool.
This patch switches it to be pulled from the toolchain built artifacts.
This also allows android tests to run in the staging environment.
Differential Revision: https://phabricator.services.mozilla.com/D68617
--HG--
extra : moz-landing-system : lando
The bugbug scheduler currently chooses which manifests are important, and we
then run *every* task that contains those manifests. This is likely overkill
and we can reduce the number of configurations we run these manifests on.
Differential Revision: https://phabricator.services.mozilla.com/D68466
--HG--
extra : moz-landing-system : lando
This adds an optional routes key to the task_task_config schema,
which is a list of strings. Anything in this key is added to the list
of routes for tasks scheduled by the decision task.
Differential Revision: https://phabricator.services.mozilla.com/D67827
--HG--
extra : moz-landing-system : lando
This patch splits the browsertime tasks out of the raptor desktop tasks to keep the webext tasks separate from the browsertime ones and make it easier to configure each of them separately.
Differential Revision: https://phabricator.services.mozilla.com/D69085
--HG--
extra : moz-landing-system : lando
The definition for reftest/crashtest variants of web-platform-tests in web-platform.yml has a trailing `s` which is technically incorrect.
This causes problems when querying ActiveData for the runtimes for these subsuites since the suite name recorded in ActiveData has the trailing s.
Changes:
- remove the trailing `s` from the definitions
Differential Revision: https://phabricator.services.mozilla.com/D68069
--HG--
extra : moz-landing-system : lando
The bugbug scheduler currently chooses which manifests are important, and we
then run *every* task that contains those manifests. This is likely overkill
and we can reduce the number of configurations we run these manifests on.
Differential Revision: https://phabricator.services.mozilla.com/D68466
--HG--
extra : moz-landing-system : lando
Add nazgul counterpart for bouncer-locations
Add nazgul counterpart for partnew-repack-bouncer-sub
Differential Revision: https://phabricator.services.mozilla.com/D67887
--HG--
extra : moz-landing-system : lando
This patch adds the capability to run Google Chrome for Android tests through Raptor-Browsertime.
Chrome must be available on the device being tested, and there are no installation steps being added in this patch since CI already has the latest Chrome version available to test with. That said, this patch adds the ability to find the version of Chrome that is being tested and stores this in the Perfherder artifact. Getting this version is also necessary to be able to run Chrome with the correct chromedriver.
Two tests are initially be enabled for Chrome in this patch: Amazon, and YouTube. They will only run through a cron task three days a week. The other changes done in this patch are required for Chrome to work with Raptor-Browsertime.
Differential Revision: https://phabricator.services.mozilla.com/D65610
--HG--
extra : moz-landing-system : lando
This is required for llvm-mt, which building winchecksec will require.
We do a dummy change to build-clang.sh so as to change the toolchain
index hash.
Differential Revision: https://phabricator.services.mozilla.com/D68153
--HG--
extra : moz-landing-system : lando
This gets two improvements:
- Some useful notes on error messages that are likely to occur on automation,
which should help sheriffs.
- The update to `symbolic` v7.1.1.
Differential Revision: https://phabricator.services.mozilla.com/D68479
--HG--
extra : moz-landing-system : lando
Changes:
Migrate all of the currently running tsan test suites to run on linux1804 instead.
All future test suites should also be added under the linux1804 label.
Depends on D67125.
Differential Revision: https://phabricator.services.mozilla.com/D68087
--HG--
extra : moz-landing-system : lando
Ensure a minimum of 1 chunk for "per-file" tests like test-verify when run on try, even when
no local file changes are detected. --full is still required.
Differential Revision: https://phabricator.services.mozilla.com/D67694
--HG--
extra : moz-landing-system : lando
Use finer granularity for some reftest/mochitest SCHEDULES.exclusive entries,
so that reftest-plain does not run when only crashtests are modified, and
vice versa; similarly, break up mochitest into mochitest/browser-chrome/chrome/
a11y. Use schedules-component instead of category.
Differential Revision: https://phabricator.services.mozilla.com/D60085
--HG--
extra : moz-landing-system : lando
The former two are not used anymore, and the latter can't be generated
on cross-builds, and it was agreed in bug 1563564 (and reconfirmed on
Matrix) that we can go without it until it's generated from separate
tasks.
More formally removing the former two will be the subject of a followup.
Differential Revision: https://phabricator.services.mozilla.com/D67580
--HG--
extra : moz-landing-system : lando
Add test package mach support for gtest and hook into the custom retrigger
action. Some existing custom retrigger features, like setting gecko prefs,
are not (easily) applicable to gtest, which doesn't use mozprofile; for
this reason, use a separate action context with items suitable for gtest.
Differential Revision: https://phabricator.services.mozilla.com/D67384
--HG--
extra : moz-landing-system : lando
Changes:
While stepping through the debugger for Bug 1608837, it was found that for `platform = windows` the conditional was always evaluating to `False` thereby loading the fallback `unix` runtimes.
Differential Revision: https://phabricator.services.mozilla.com/D67390
--HG--
extra : moz-landing-system : lando
The diff jobs currently take a Firefox package and compares against the
one from another build. When that fails, it also makes the diff of the
generated files, which can be useful when there are differences, but
can also be useful on its own. Also, because sometimes, like right now,
there are differences in generated files that have no impact on Firefox
itself, when differences do show up for Firefox, the differences in
generated files are added noise that sheriffs can't work around.
Differential Revision: https://phabricator.services.mozilla.com/D67113
--HG--
extra : moz-landing-system : lando
Remove the remainder of automation.py.in and the build support for generating
automation.py.
Some of this functionality was in use, especially for android tests.
Some code was moved or re-implemented in remoteautomation.py or in the affected
harness(es). Some features were removed: There are some minor changes in behavior.
For instance, instead of using a different server startup timeout for debug builds,
one value is used for all builds (due to performance improvements over time,
the longer timeout is no longer needed).
Differential Revision: https://phabricator.services.mozilla.com/D66839
--HG--
extra : moz-landing-system : lando
Update the default values to avoid common pitfalls, such as trying to repeat
a 30-minute long tasks 30x times with extra logging!
The new defaults allow a simple re-run of most tasks with no changes.
While we are here, tweak the parameter descriptions.
Differential Revision: https://phabricator.services.mozilla.com/D66842
--HG--
extra : moz-landing-system : lando
Changes:
Remove references to `mochitest-browser-chrome-gpu` as this suite does not exist.
In `test-sets.yml`, split existing `mochitest-gpu` task into `plain` and `chrome` variants.
In `mochitest.yml` split the existing `mochitest-gpu` task into `plain` and `chrome` variants with separate symbols on treeherder.
Differential Revision: https://phabricator.services.mozilla.com/D66670
--HG--
extra : moz-landing-system : lando
This patch removes the chromedriver for Chrome 78, and adds a chromedriver for Chrome 81.
Differential Revision: https://phabricator.services.mozilla.com/D66987
--HG--
extra : moz-landing-system : lando
This is similar to the same-named commit in bug 1619837.
For those few configurations that don't run xpcshell tests, we need a
`fix-stacks` dependency in the build job. This is because xpcshell self tests
run during the build job, but the existing xpcshell test dependency doesn't
cover that case.
Differential Revision: https://phabricator.services.mozilla.com/D66927
--HG--
extra : moz-landing-system : lando
Changes:
Add a new configuration file for GDM that forces the use of X11 instead of Wayland.
This may help with reducing the number of crashes due to window managers.
Differential Revision: https://phabricator.services.mozilla.com/D67006
--HG--
extra : moz-landing-system : lando
This converts `--setenv` into `env` in `try_task_config` at parameter
generation time.
Differential Revision: https://phabricator.services.mozilla.com/D66537
--HG--
extra : moz-landing-system : lando
There are a number of settings that have equivalent expressions in
`try_options` (used for try syntax) and `try_task_config` (used for other try
selectors). Rather than requiring task generation code to understand both
formats, this converts the try syntax specification to `try_task_config` at
parameter generation time.
Differential Revision: https://phabricator.services.mozilla.com/D66536
--HG--
extra : moz-landing-system : lando
This allows us to change the default optimization strategy used in try pushes.
While probably not super useful to developers, it can help us easily test
changes to new and experimental optimizations on try.
This also changes the default to the 'bugbug_push_schedules' strategy, since
SETA is more or less random and shouldn't be used by 'mach try auto'. In the
future, we'll switch this back to simply using the default optimization as the
default will ideally be the best one that we have.
Differential Revision: https://phabricator.services.mozilla.com/D65746
--HG--
extra : moz-landing-system : lando
The 'auto' in 'mach try auto' stands for two things:
1. It automatically picks tasks for you.
2. It runs the same scheduling algorithms as autoland.
It accomplishes this by creating a new target_tasks method that spoofs the
'project' parameter to autoland.
Differential Revision: https://phabricator.services.mozilla.com/D60184
--HG--
extra : moz-landing-system : lando
Changes:
As documented in bug 1621483, `marionette` experiences a non-trivial amount of issues when run on ubuntu1804 docker image with GTK/GNOME desktop environment enabled.
GTK/GNOME has a higher degree of asynciness when manipulating window size/position and this leads to `marionette` and the derived suite `web-platform-tests-wdspec` reporting intermittent oranges for a number of tests.
While attempts were made to incorporate a fix for the marionette driver itself, the best attempts have only been able to achieve a ~50% reliability in green runs.
This patch reintroduces the use of bare `compiz` window manager exclusively for these two problematic test suites so that at least the tests are running on non-legacy software.
Differential Revision: https://phabricator.services.mozilla.com/D66482
--HG--
extra : moz-landing-system : lando
This creates a new toolchain artifact that repacks a combination of the
linux64-clang compiler along with parts of the win64-clang-cl compiler.
This has multiple advantages:
- It removes some convoluted parts of build task definitions (limiting
that to only occur on the win-cross toolchain itself).
- It simplifies the build setup by not requiring to prepare for where
clang-cl.exe is.
- It speeds up getting compiler artifacts because the win64-clang-cl
artifact is very large (due to there not being a llvm shared library)
and bzipped, which is slow to decompress. Here, we only take what we
need for the cross builds.
- It adds the runtime files that e.g. PGO will require, and that
linux clang-cl insists lives in the clang directory, not the
win64-clang-cl one, and that would require some convoluted setup to make
it work with the two separate toolchains.
Differential Revision: https://phabricator.services.mozilla.com/D66543
--HG--
extra : moz-landing-system : lando
The files have been removed in bug 1620158, so they won't appear as
differences anymore.
Differential Revision: https://phabricator.services.mozilla.com/D66545
--HG--
extra : moz-landing-system : lando
Add filename support for plain display
Update fzf version in tests
Update tooltool url for fzf
Differential Revision: https://phabricator.services.mozilla.com/D66571
--HG--
extra : moz-landing-system : lando
Creating the virtualenv prints an error message in the diffoscope docker
image because it doesn't have a compiler or the python development
headers, and that triggers an error setting up psutil after bug 1620513,
which in turn leads to a noisy and unrelated error message appearing in
the diffoscope jobs when they fail.
We don't need psutil set up really in those jobs, let alone a full
fledged virtualenv, so we can avoid the psutil error message by
disabling the virtualenv.
Differential Revision: https://phabricator.services.mozilla.com/D66361
--HG--
extra : moz-landing-system : lando
Changes:
Remove the unnecessary setup phase of `valgrind` on the docker image, given that we are not scheduling `mochitest-valgrind` [anywhere](https://searchfox.org/mozilla-central/source/taskcluster/ci/test/mochitest.yml#300) in the CI system.
Also, clean up `test-sets.yml` by renaming the `linux-common-tests` to `linux1604-legacy-tests` to better reflect that the set holds tests that haven't been migrated over to linux1804 yet.
Differential Revision: https://phabricator.services.mozilla.com/D65832
--HG--
extra : moz-landing-system : lando
This was disabled in bug 1599197 but this build is not really like the other
Linux32 builds: it runs ARM32 JIT code in SpiderMonkey's ARM32 simulator.
Differential Revision: https://phabricator.services.mozilla.com/D66228
--HG--
extra : moz-landing-system : lando
Nicholas Nethercote
James Graham
Mihai Alexandru Michis
Cosmin Sabou
David Major
Andi-Bogdan Postelnicu
Kartikaya Gupta
Henrik Skupin
Gregory Mierzwinski
Marco Castelluccio
Mihai Tabara
Jan de Mooij
jayati
Simon Fraser
Marian Raiciof
Tarek Ziadé
Florin Strugariu
Mike Hommey
Bogdan Tara
Wander Lairson Costa
Andrew Halberstadt
Edwin Takahashi
Csoregi Natalia
Ken VanDine
Christian Holler
Julien Cristau
Geoff Brown
Kershaw Chang
Noemi Erli
Michael Froman
Arthur Iakab
Bob Clary
Sebastian Hengst
Ciure Andrei
Tom Prince
Max
Lee Salzman
Bartłomiej Piotrowski
Steve Fink
Chris Hartjes
Anurag Aggarwal
Sylvestre Ledru
shindli