Changes:
As documented in bug 1621483, `marionette` experiences a non-trivial amount of issues when run on ubuntu1804 docker image with GTK/GNOME desktop environment enabled.
GTK/GNOME has a higher degree of asynciness when manipulating window size/position and this leads to `marionette` and the derived suite `web-platform-tests-wdspec` reporting intermittent oranges for a number of tests.
While attempts were made to incorporate a fix for the marionette driver itself, the best attempts have only been able to achieve a ~50% reliability in green runs.
This patch reintroduces the use of bare `compiz` window manager exclusively for these two problematic test suites so that at least the tests are running on non-legacy software.
Differential Revision: https://phabricator.services.mozilla.com/D66482
--HG--
extra : moz-landing-system : lando
This creates a new toolchain artifact that repacks a combination of the
linux64-clang compiler along with parts of the win64-clang-cl compiler.
This has multiple advantages:
- It removes some convoluted parts of build task definitions (limiting
that to only occur on the win-cross toolchain itself).
- It simplifies the build setup by not requiring to prepare for where
clang-cl.exe is.
- It speeds up getting compiler artifacts because the win64-clang-cl
artifact is very large (due to there not being a llvm shared library)
and bzipped, which is slow to decompress. Here, we only take what we
need for the cross builds.
- It adds the runtime files that e.g. PGO will require, and that
linux clang-cl insists lives in the clang directory, not the
win64-clang-cl one, and that would require some convoluted setup to make
it work with the two separate toolchains.
Differential Revision: https://phabricator.services.mozilla.com/D66543
--HG--
extra : moz-landing-system : lando
The files have been removed in bug 1620158, so they won't appear as
differences anymore.
Differential Revision: https://phabricator.services.mozilla.com/D66545
--HG--
extra : moz-landing-system : lando
Add filename support for plain display
Update fzf version in tests
Update tooltool url for fzf
Differential Revision: https://phabricator.services.mozilla.com/D66571
--HG--
extra : moz-landing-system : lando
Creating the virtualenv prints an error message in the diffoscope docker
image because it doesn't have a compiler or the python development
headers, and that triggers an error setting up psutil after bug 1620513,
which in turn leads to a noisy and unrelated error message appearing in
the diffoscope jobs when they fail.
We don't need psutil set up really in those jobs, let alone a full
fledged virtualenv, so we can avoid the psutil error message by
disabling the virtualenv.
Differential Revision: https://phabricator.services.mozilla.com/D66361
--HG--
extra : moz-landing-system : lando
Changes:
Remove the unnecessary setup phase of `valgrind` on the docker image, given that we are not scheduling `mochitest-valgrind` [anywhere](https://searchfox.org/mozilla-central/source/taskcluster/ci/test/mochitest.yml#300) in the CI system.
Also, clean up `test-sets.yml` by renaming the `linux-common-tests` to `linux1604-legacy-tests` to better reflect that the set holds tests that haven't been migrated over to linux1804 yet.
Differential Revision: https://phabricator.services.mozilla.com/D65832
--HG--
extra : moz-landing-system : lando
This was disabled in bug 1599197 but this build is not really like the other
Linux32 builds: it runs ARM32 JIT code in SpiderMonkey's ARM32 simulator.
Differential Revision: https://phabricator.services.mozilla.com/D66228
--HG--
extra : moz-landing-system : lando
The win64-aarch64 have a kind of a nasty trick that makes fetch-content
download artifacts of a dependent task directly as artifacts of the task
itself. For some reason, while this pattern works on native Windows
jobs, it doesn't on Linux. What happens is essentially that:
`pathlib.Path(path).joinpath('../foo').mkdir(parents=True, exist=ok=True)`
fails when path doesn't exist first. I guess the fetches directory
already exists on Windows worker or something.
Unfortunately, os.path.normpath doesn't take `pathlib.Path`s in
still-supported python 3.5, so we have to convert to str first.
Differential Revision: https://phabricator.services.mozilla.com/D66518
--HG--
extra : moz-landing-system : lando
First, with `fix_stacks.py` now being used on most test suites, those test
suites need a dependency on `fix-stacks`.
Second, for those few configurations that don't run xpcshell tests, we need a
`fix-stacks` dependency in the build job. This is because xpcshell self tests
run during the build job, but the existing xpcshell test dependency doesn't
cover that case.
Differential Revision: https://phabricator.services.mozilla.com/D65943
--HG--
extra : moz-landing-system : lando
Adjust runme to be executable
Rm sdk/pltf install.Manually add meta
Enforce firefox run instead of notify
Add policy to disable updates
Temp hack to default to firefox instead of notify-send
Fix mach linters
Remove firefox command hack. Proper fix
Remove duplicate cmd in runme
Fix indentantion in kind
Fix more linters
Differential Revision: https://phabricator.services.mozilla.com/D59561
--HG--
extra : moz-landing-system : lando
This duplicates the peculiarities of the win64-cross/opt task, while adding
a rust-size dependency it was lacking.
Differential Revision: https://phabricator.services.mozilla.com/D65483
--HG--
extra : moz-landing-system : lando
We were using the old cargo-vendor, which tries to fetch dependencies in a way
that crates.io dislikes, leading to spurious timeouts.
Use the built-in command instead. We no longer need the --relative-path flag
because the built-in one uses a relative path by default.
MANUAL PUSH: Fixing tree closure, so need to push on CLOSED TREE.
Differential Revision: https://phabricator.services.mozilla.com/D65882
Note: while we can use time.monotonic in fetch-content, we can't in
mach artifact toolchain yet because it's still python2.
Differential Revision: https://phabricator.services.mozilla.com/D65690
--HG--
extra : moz-landing-system : lando
They are going to be required by tier 1 tasks. Also move them around to
another file because they're not mingw-specific anymore.
Differential Revision: https://phabricator.services.mozilla.com/D65482
--HG--
extra : moz-landing-system : lando
Changes:
Replaces all instances of `itervlaues` and `iteritems` with `six` variant in try_option_syntax.py.
Differential Revision: https://phabricator.services.mozilla.com/D61055
--HG--
extra : moz-landing-system : lando
Bug 1602446 disabled coalescing for tasks which was designed to reduce load on
the pools which ran those tasks by skipping some of them. After coalescing was
affecting almost no tasks, it started to coalesce more tasks in November 2019
(bug 1602446) up to a level for which too few tasks ran and test coverage was
impacted. Coalescing got disabled and the conclusion is that coalescing is
currently not needed.
Differential Revision: https://phabricator.services.mozilla.com/D65733
--HG--
extra : moz-landing-system : lando
Simple update to strings and names for the custom retrigger action, in preparation
for the addition of more tasks.
Differential Revision: https://phabricator.services.mozilla.com/D65582
--HG--
rename : taskcluster/taskgraph/actions/retrigger_mochitest.py => taskcluster/taskgraph/actions/retrigger_custom.py
extra : moz-landing-system : lando
Define and use fuzzing-builds optimization strategy that allows either
skip-unless-schedules or seta strategies.
Differential Revision: https://phabricator.services.mozilla.com/D64628
--HG--
extra : moz-landing-system : lando
Changes:
Remove all tests other than `reftests`, `crashtests` and `mochitest-media` for windows10-aarch64.
Important changes are in `taskcluster/ci/test/test-platforms.yml` and `taskcluster/ci/test/test-sets.yml`.
I've gone ahead and removed the windows10-aarch64 specific entries in the YAML files as well.
Differential Revision: https://phabricator.services.mozilla.com/D65654
--HG--
extra : moz-landing-system : lando
Changes:
Pin the source docker image to a specific tag from canonical.
Remove unneeded packages that are either installed as part of the setup, or come preinstalled.
Silence the `dbus` startup message by redirecting the output to `2&>1`.
Remove unnecessary `locale-gen` call at the end of setup.
Differential Revision: https://phabricator.services.mozilla.com/D65434
--HG--
extra : moz-landing-system : lando
We don't ship openh264 builds on android, so there is no need
to build them in automation.
Differential Revision: https://phabricator.services.mozilla.com/D65669
--HG--
extra : moz-landing-system : lando
These jobs only use the workspace for obj directories, have some some logic to
cleanup the directory at the beginning of the run, so there is no reason to
cache the directory.
This also removes the now-unused common code for creating a workspace cache directory.
Differential Revision: https://phabricator.services.mozilla.com/D62486
--HG--
extra : moz-landing-system : lando
Since the workspace is no longer cached, but needs to be a volume for
performance reasons, add a check to ensure that is the case.
Differential Revision: https://phabricator.services.mozilla.com/D62484
--HG--
extra : moz-landing-system : lando
Since mozharness tasks are no longer caching the workspace directory, we don't
need a key for different tasks.
Differential Revision: https://phabricator.services.mozilla.com/D62483
--HG--
extra : moz-landing-system : lando
This moves the object directory and source directories around in all mozharness
jobs, to allow enabling caching on windows builders.
This makes a number of changes that all need to land at once:
- Move the source checkout for the workspace cache mount, to the checkouts
cache mount.
- Makes the object directoy from underneath the source directory, to directly
in the work directory (which is still under `workspace`).
- Sets the object directory to `obj-build` instead of `obj-firefox`.
- Stops caching the workspace directory (it is still a volume in docker workers,
so writes perform well; a followup revision add some checks around this).
- Removes one level of directory in the mozharness workdir (things were under
`workspace/build`, but are now just under `workspace/`.
- Adjust paths in environment variables and artifact specifications to match
the above changes.
Differential Revision: https://phabricator.services.mozilla.com/D62482
--HG--
extra : moz-landing-system : lando
Changes:
Replaces all instances of `itervlaues` and `iteritems` with `six` variant.
Differential Revision: https://phabricator.services.mozilla.com/D61055
--HG--
extra : moz-landing-system : lando
add_args_to_command in taskcluster/taskgraph/actions/util.py assumes that the
cmd_parts argument on OSX consists of a list containing a single list item.
This is not always the case in that it is possible for cmd_parts to contain
multiple command such as changing file permissions before executing a test command.
This patch adds a check for cmd_parts consisting of a list of two items, the
second of which is a list. The extra_args are appended to the second item. The
existing behavior for a cmd_parts argument consisting of a list containing a
single list item is preserved for backwards compatibility.
add_args_to_command also suffers from an additional problem in that it modifies
the cmd_parts argument which results in the object being modified in the caller.
create_isolate_failure_tasks in taskcluster/taskgraph/actions/isolate_test.py also
suffers from the modification of the task_description argument. This is fixed
by using deepcopied versions of the objects in the corresponding functions.
create_isolate_failure_tasks handling of empty failure groups and unknown
failure groups is also improved.
Differential Revision: https://phabricator.services.mozilla.com/D65430
--HG--
extra : moz-landing-system : lando
The new version has two improvements.
- It has slightly changed output in some cases -- more similar to the existing
stack fixers -- which avoids the need to change the DMD tests.
- It fixes some erroneous double-JSON-escaping that caused excess backslashes
in Windows paths.
Differential Revision: https://phabricator.services.mozilla.com/D65055
--HG--
extra : moz-landing-system : lando
Changes:
Remove `ubuntu-bionic` flag that was used during development to enable use of ubuntu1804-test docker image.
Remove unnecessary conditional and in the process rewrite how `runtests.py` checks the environment for `pactl` prior to running mochitests.
Differential Revision: https://phabricator.services.mozilla.com/D65389
--HG--
extra : moz-landing-system : lando
Changes:
Cleaned up a lot of the unnecessary environment variables that was inherited from previous build and test systems.
Differential Revision: https://phabricator.services.mozilla.com/D65361
--HG--
extra : moz-landing-system : lando
Changes to the firefox-snap Docker image to support building Thunderbird Snap
packages as well.
Replaced hardcoded "firefox" instances in runme.sh with $PRODUCT variable and
renamed "snapcraft.yaml.in" to "firefox.snapcraft.yaml.in".
Differential Revision: https://phabricator.services.mozilla.com/D63035
--HG--
rename : taskcluster/docker/firefox-snap/snapcraft.yaml.in => taskcluster/docker/firefox-snap/firefox.snapcraft.yaml.in
extra : moz-landing-system : lando
Calling the merge day automation requires an action so that we can pass in parameters such as source and destination repository and branch.
Differential Revision: https://phabricator.services.mozilla.com/D62763
--HG--
extra : moz-landing-system : lando
There's no way for Thunderbird to define a new task kind at the moment because
the verify_docs function only checks one base path.
This adds a 'documentation_paths' registry that holds a list of paths to check
for documentation files. Calling documentation_paths() with a filename later
will read all the files it finds with that name under all of the base paths,
returning them in one chunk. This minimizes the necessary changes to verify_docs
itself.
Differential Revision: https://phabricator.services.mozilla.com/D60298
--HG--
extra : moz-landing-system : lando
Adding 'try' to 'run-on-projects' is a no-op anyway, so we should avoid it.
Differential Revision: https://phabricator.services.mozilla.com/D64645
--HG--
extra : moz-landing-system : lando
The transform skips copying attributes from the dep job if attributes are specified. This means we skipped adding shipping_{phase,product}, nightly, and shippable attributes, and the poller task ended up getting filtered out of various target task methods.
Everything works properly if we just copy the attributes from the dependent job.
Differential Revision: https://phabricator.services.mozilla.com/D64684
--HG--
extra : moz-landing-system : lando
It currently is missing some cross-compilation stuff and so actually builds a
Linux binary.
The commit also avoids the unintentional installation of the `fix-stacks.d`
file, by introducing the `EXE` variable.
Differential Revision: https://phabricator.services.mozilla.com/D64528
--HG--
extra : moz-landing-system : lando
Future work should hopefully find a way to not assume tests are packaged with beetmover, and to have a solution for w64's emefree design.
Differential Revision: https://phabricator.services.mozilla.com/D64429
--HG--
extra : moz-landing-system : lando
Andrew Halberstadt
Max
Lee Salzman
Bartłomiej Piotrowski
Steve Fink
Sebastian Hengst
Tom Prince
Edwin Takahashi
Mike Hommey
Cosmin Sabou
Chris Hartjes
Anurag Aggarwal
Simon Fraser
Sylvestre Ledru
Nicholas Nethercote
Geoff Brown
Bob Clary
Jan de Mooij
shindli
Mihai Tabara
Dorel Luca
James Graham
Florin Strugariu
Gijs Kruitbosch
Emilio Cobos Álvarez
Mark Banner
Csoregi Natalia
Kris Taeleman
alexandru.ionescu
Ciure Andrei
Tarek Ziadé
Johan Lorenzo
Rob Lemley
Nick Thomas
Kartikaya Gupta
ffxbld
Nick Alexander
Tooru Fujisawa
Aki Sasaki
Chris Manchester
Razvan Maries
Justin Wood
Bogdan Tara
Brindusan Cristian
David Major