Changes:
Do not install text editors by default, since most of the time it won't be used.
Actively purge installed applications that are not necessary for tests.
Restructure the Dockerfile such that RUN commands are consolidated where applicable (reduces layers), COPY commands for files that are likely not changing are placed near top of the file in the hope that it will reduce the number of rebuilding steps.
Differential Revision: https://phabricator.services.mozilla.com/D54049
--HG--
extra : moz-landing-system : lando
Increase max-run-times to avoid intermittent task timeouts. Increasing chunks for
these tasks is not effective: chunks are very unbalanced due to a group of
long-running tests.
Differential Revision: https://phabricator.services.mozilla.com/D55182
--HG--
extra : moz-landing-system : lando
The run-visual-metrics.py script was opening the provided browsertime.json
files without specifying an encoding, which causes it to infer the encoding
from the environment. In our Docker containers, this results in the `ascii`
codec being chosen for opening text files. We now specifically open them as
UTF-8 to support non-ASCII content.
Differential Revision: https://phabricator.services.mozilla.com/D54653
--HG--
extra : moz-landing-system : lando
Instead of using a json file that contains URLs to task cluster artifacts, let's use a single zip file that contains all video files to analyze. This will reduce the number of artifacts we generate per raptor run and the number of network downloads.
The run-visual-metrics gets the tarball directly when it's used as a sub-task and produces its own result tarball
notice that at this point I don't see any reason to copy in the vismet results tarball all the files.
Maybe we should simply generate a *single* json file that contains a merge of all visualmetrics results?
But that could be done later. lmk
Differential Revision: https://phabricator.services.mozilla.com/D52907
--HG--
extra : moz-landing-system : lando
Fixes up ESR bouncer aliases now that ESR60 is EOL.
Adds check for latest products used by www.mozilla.org.
Enables the cron checks.
Fixes up a script error which meant we weren't checking all partial platforms in release automation.
Depends on D54656
Differential Revision: https://phabricator.services.mozilla.com/D54657
--HG--
extra : moz-landing-system : lando
Build a full sysroot by downloading the `wasi-sdk` repo, which has `llvm-project` and `wasi-libc` as submodules. `wasi-sdk` builds a `clang` in a unique configuration (with `wasm32-wasi` as a default target) and uses that `clang` to build the rest of the pieces for the wasm sysroot.
In principle it should be possible to build the sysroot using our in-house automation-built `clang`, but I kept running into strange, hard-to-diagnose issues when I attempted that. If someone else is able to straighten out all the compilation issues, we could replace this script and stop pulling in `wasi-sdk` entirely, which would result in a build that takes much less time overall. Until then, this will have to do to unblock the rest of the wasm sandboxing work.
Differential Revision: https://phabricator.services.mozilla.com/D54560
--HG--
extra : moz-landing-system : lando
In this patch I have adapted the Docker file to make use of core18 and snapcraft3.
This is the recommended approach as stated here.
https://snapcraft.io/docks/t/creating-docker-images-for-snapcraft/11739
Specifically the part talking about replacing likes FROM ubuntu:xenial with FROM ubuntu:bionic.
In snapcraft.yaml.in, I adjusted the snap by using the gnome-3-28 extensions as described in
https://forum.snapcraft.io/t/the-gnome-3-28-extension/13485.
In addition, I followed the instructions presented by removing the unnecessary plugs relating
to the desktop extension and adding the relevant dbus slot.
As part of the build process, it required a bunch of extra staged packages which I have documented.
I also used the magic part from Ken Vandine's thunderbird snapcraft for updating mime info.
https://bazaar.launchpad.net/~ubuntu-desktop/thunderbird/snap/view/head:/snapcraft.yaml
I also removed the xdg-open as that seemed to not be enabling the building of the snap.
Differential Revision: https://phabricator.services.mozilla.com/D53355
--HG--
extra : moz-landing-system : lando
Handles 'env' and 'chemspill-prio' configs in the transforms. The 'rebuild'
task config is purposefully excluded from the full_task_graph and instead
handled at the target_tasks stage. Otherwise if a user ran '--rebuild 20' then
retriggered one of those tasks, they'd instead get another 20 which is almost
certainly not what we want.
Differential Revision: https://phabricator.services.mozilla.com/D51417
--HG--
extra : moz-landing-system : lando
Also converts the 'artifact' from a Template to a TryConfig. So this changes the config from:
{
'templates': {
'artifact': {
'enabled': 1
}
}
}
to:
{
'use-artifact-builds': True
}
Differential Revision: https://phabricator.services.mozilla.com/D51416
--HG--
extra : moz-landing-system : lando
Changes:
Properly detect the distribution and release-version, and set the desktop session variable in `test-linux.sh` appropriately.
Use docker `entrypoint` for initializing `dbus`.
This change is critical for reducing the number of failures on GNOME-shell based systems, such as Ubuntu 18.04 as outlined in bug 1596526 comment 0.
Differential Revision: https://phabricator.services.mozilla.com/D53089
--HG--
extra : moz-landing-system : lando
This has the side effect of updating all debian10-based docker images
(except the ones based on debian10-test)
Differential Revision: https://phabricator.services.mozilla.com/D54069
--HG--
extra : moz-landing-system : lando
Previously we inspected the `TASKCLUSTER_WORKER_GROUP` environment variable,
which now only returns the cloud provider of the worker. This commit teaches
`run-task` to instead use the `TASKCLUSTER_WORKER_LOCATION` to gather
information about the location of the worker. We also use the extra data
about the cloud provider for the worker to construct a key for use in the
config, in the form `cloudprovider/region`, so GCP hgweb mirrors can be
amended to the `hgmointernal` config when they are ready.
While we're here we make the error handling for a missing environment
variable slightly nicer.
Differential Revision: https://phabricator.services.mozilla.com/D53209
--HG--
extra : moz-landing-system : lando
To get an accurate backtrace if the run task crashes, we need the
symbols from the instrumented build.
Depends on D53716
Differential Revision: https://phabricator.services.mozilla.com/D53717
--HG--
extra : moz-landing-system : lando
minidump_stackwalk is used to process any crash reports generated by the
PGO run task.
Depends on D52393
Differential Revision: https://phabricator.services.mozilla.com/D53716
--HG--
extra : moz-landing-system : lando
Since we may or may not have minidump files, it is not practical to
specifically name each file. We can just upload all files in the
directory instead.
Differential Revision: https://phabricator.services.mozilla.com/D52393
--HG--
extra : moz-landing-system : lando
Changes:
Add Ubuntu 18.04 `dockerfile`, support files and `ubuntu1804-test-system-setup.sh` that slightly differs from `ubuntu1604-test-system-setup.sh` in package contents.
Add a temporary flag to `try fuzzy` selector, taskcluster decision and taskgraphs to enable selection of Ubuntu 18.04 docker image to run linux tests.
Differential Revision: https://phabricator.services.mozilla.com/D53750
--HG--
extra : moz-landing-system : lando
Previously we inspected the `TASKCLUSTER_WORKER_GROUP` environment variable,
which now only returns the cloud provider of the worker. This commit teaches
`run-task` to instead use the `TASKCLUSTER_WORKER_LOCATION` to gather
information about the location of the worker. We also use the extra data
about the cloud provider for the worker to construct a key for use in the
config, in the form `cloudprovider/region`, so GCP hgweb mirrors can be
amended to the `hgmointernal` config when they are ready.
While we're here we make the error handling for a missing environment
variable slightly nicer.
Differential Revision: https://phabricator.services.mozilla.com/D53209
--HG--
extra : moz-landing-system : lando
Axel Hecht
Geoff Brown
Mihai Tabara
Justin Wood
Edwin Takahashi
octavian.negru
Marian Raiciof
Ricky Stewart
Barret Rennie
Tarek Ziadé
Nick Thomas
Sylvestre Ledru
Krish De Souza
Csoregi Natalia
Gurzau Raul
Aki Sasaki
ffxbld
Andrew Halberstadt
Stephen Donner
Mike Hommey
Nathan Froyd
Marco Castelluccio
Connor Sheehan
Daniel Varga
Mike Shal
Johan Lorenzo
Brindusan Cristian
Chris Manchester