We don't need to package tests for builds that we don't actually run
tests from, but it is tricky to align this correctly by setting
MOZ_AUTOMATION_PACKAGE_TESTS=0 in relevant mozconfigs. Instead we can
set the environment variable in the task definition, and use a full
taskgraph verification check to ensure that the flag is only set on
builds that have tests.
The one tricky area is the win64-aarch64 builds, which have a workaround
by specifying the new skip-verify-test-packaging attribute.
In one case, win64-aarch64-shippable has tests that run against it, but
it copies those tests from a win64-aarch64-shippable-no-eme task, which
itself has no tests. Both of those tasks need to skip the verify check
as a result.
In another case, the win64-aarch64-eme task is an artifact build that
grabs test packages from the win64-aarch64 build. Since the win64-aarch64
build doesn't have tests, it needs to skip the verify check.
Differential Revision: https://phabricator.services.mozilla.com/D59426
--HG--
extra : moz-landing-system : lando
In GCP we have the double the number of core compared to AWS
counterparts, but we use the same amount of memory. Request the builds
to be less parallel to avoid OOM.
Differential Revision: https://phabricator.services.mozilla.com//D62514
We are migrating the worker used by builds from AWS to GCP. We have had tier- 3 GCP builds for initial testing. We are replacing those with tier-3 AWS builds to ensure they keep working so we are able to switch back.
Differential Revision: https://phabricator.services.mozilla.com/D60042
--HG--
extra : moz-landing-system : lando
We are migrating the worker used by builds from AWS to GCP. We have had tier- 3 GCP builds for initial testing. We are replacing those with tier-3 AWS builds to ensure they keep working so we are able to switch back.
Differential Revision: https://phabricator.services.mozilla.com/D60042
--HG--
extra : moz-landing-system : lando
Changes:
Diffoscope linux32 (diff-reproducible-linux32) was triggering linux-shippable jobs on autoland.
For good measure, specify for `geckodriver-repack` that it should not run on autoland.
Also restrict `linux-shippable` build on gcp from taking place on autoland. It is a tier 3 job so not visible normally.
Differential Revision: https://phabricator.services.mozilla.com/D59444
--HG--
extra : moz-landing-system : lando
We don't need to package tests for builds that we don't actually run
tests from, but it is tricky to align this correctly by setting
MOZ_AUTOMATION_PACKAGE_TESTS=0 in relevant mozconfigs. Instead we can
set the environment variable in the task definition, and use a full
taskgraph verification check to ensure that the flag is only set on
builds that have tests.
The one tricky task is win64-aarch64-shippable/opt, which copies tests
from another build rather than building them itself. For this reason, it
explicitly sets MOZ_AUTOMATION_PACKAGE_TESTS: '0' in the environment
even though that is now the default. This is why the exception is only
raised if MOZ_AUTOMATION_PACKAGE_TESTS is not set at all, rather than
checking that it is set to 1.
Differential Revision: https://phabricator.services.mozilla.com/D59426
--HG--
extra : moz-landing-system : lando
We're going to build `lucetc` for other platforms; let's prefix this
task with the platform name like we do for other toolchains.
Differential Revision: https://phabricator.services.mozilla.com/D59498
--HG--
extra : moz-landing-system : lando
Changes:
Do not run `linux32` builds on `try` or `autoland` by default, but ensure it is able to run on all other repos (eg. `mozilla-beta`).
Filter out all tasks with `linux-shippable` in the name, including builds, auxiliary tasks (balrog, etc).
Differential Revision: https://phabricator.services.mozilla.com/D58982
--HG--
extra : moz-landing-system : lando
Changes:
Revert the changes to the treeherder name since the partials script does depend on the name as it appears in treeherder to the pre-D58406 state.
Differential Revision: https://phabricator.services.mozilla.com/D58897
--HG--
extra : moz-landing-system : lando
Changes:
Demote linux32-shippable/opt tasks to tier 2 on all repositories. Other linux32 jobs are defined but do not appear to run.
Limit the build of linux32-shippable/opt to `try`/`mozilla-central`.
Differential Revision: https://phabricator.services.mozilla.com/D58207
--HG--
extra : moz-landing-system : lando
Changes:
Demote linux32-shippable/opt tasks to tier 2 on all repositories. Other linux32 jobs are defined but do not appear to run.
Limit the build of linux32-shippable/opt to `try`/`mozilla-central`.
Differential Revision: https://phabricator.services.mozilla.com/D58207
--HG--
extra : moz-landing-system : lando
Changes:
Make available all variants of linux1804 for general use. Where appropriate, each variant is populated with `linux1804-tests`, which contain the shared tests that should run on all appropriate variants.
To deal with linux64-asan/opt and linux1804-64-asan/opt clashing in the taskgraph due to the translation service, remove `linux64/asan` entry from the translation dictionary and manually check in the `if/else` conditional later on.
Differential Revision: https://phabricator.services.mozilla.com/D58406
--HG--
extra : moz-landing-system : lando
Make sure mozharness does not attempt to download the condprof source
in win-aarch64 (not supported yet)
Differential Revision: https://phabricator.services.mozilla.com/D57082
--HG--
extra : moz-landing-system : lando
xvfb was used to create a virtual framebuffer for running Firefox during
build jobs to support PGO profile generation. That now runs in a
separate task, so we don't need this flag for builds anymore.
Note that other Linux builds still need xvfb in order to run xpcshell in
'make check'.
Differential Revision: https://phabricator.services.mozilla.com/D56111
--HG--
extra : moz-landing-system : lando
GCC8 happens not to generate the code that causes the crash, so do that for now
to unblock fuzzers from hitting this.
We still need to figure out what to do about the more general issue of course...
Differential Revision: https://phabricator.services.mozilla.com/D55985
--HG--
extra : moz-landing-system : lando
We need this for "full" C++17 support (everything is supported, but some
C++17 features still have bugs) and this change also brings Linux into
parity with our Mac requirements.
MANUAL PUSH: build toolchains on inbound to avoid clogging autoland
Differential Revision: https://phabricator.services.mozilla.com/D51450
With Bug 1579189 we are going to raise the minimum clang version to 5. But in clang 5
and clang 6 an issue has been introduced where the `Decl` nodes from the `AST` don't
contain all of the annotation attributes. The missing attributes can cause static
analysis failures. We are therefore going to disable the static analysis for the
base-toolchain clang builds as a workaround.
Differential Revision: https://phabricator.services.mozilla.com/D52025
--HG--
extra : moz-landing-system : lando
This does many things:
1) stops producing (and consuming) `FennecJNI*` JNI wrappers
2) removes the :app and :thirdparty Gradle projects
3) removes relevant pieces of the Gradle target configuration
4) updates lints
5) purges old configurations
After this commit, the `mobile/android` project/application builds
only GeckoView.
Differential Revision: https://phabricator.services.mozilla.com/D46536
--HG--
extra : moz-landing-system : lando
Our build toolchains don't contain libstdc++ headers for aarch64, so our
aarch64 builds rely on whatever libstdc++ headers the system has
installed. To bring in newer headers on our aarch64 builds, then, we
need to update the system images for those builds, which this patch does.
Depends on D45861
Differential Revision: https://phabricator.services.mozilla.com/D45862
--HG--
extra : moz-landing-system : lando
Mozharness no longer drives building with PGO; it is all handled in
Taskcluster and the build system.
Depends on D46070
Differential Revision: https://phabricator.services.mozilla.com/D46071
--HG--
extra : moz-landing-system : lando
The aarch64 cross toolchain is unused otherwise. The aarch64-linux
builds also exist for the express purpose of eventually standing up some
kind of fuzzing/ccov build, so we might as well start using a toolchain
that supports those use cases.
Differential Revision: https://phabricator.services.mozilla.com/D45210
--HG--
extra : moz-landing-system : lando
Increase max-run-time to avoid intermittent failures due to variance in robustcheckout
performance.
Differential Revision: https://phabricator.services.mozilla.com/D45200
--HG--
extra : moz-landing-system : lando
Currently, we have no real visibility on the time spent after the build
finished, despite the fact that a large chunk is actually happening via
make check (although thankfully more and more of it is moving out to
separate tasks).
Also, the mozharness machinery for make check dates from when we were
running in buildbot and takes care of turning builds orange instead of
red in case of failure, etc. which doesn't do anything useful anymore.
Differential Revision: https://phabricator.services.mozilla.com/D42806
--HG--
extra : moz-landing-system : lando
This part removes the use_toolchains transform, and adjusts all task
definitions accordingly.
Differential Revision: https://phabricator.services.mozilla.com/D41891
MANUAL PUSH: avoid closing autoland while docker images and toolchains
are rebuilt.
API lint is arguably the most valuable lint of all, but it's also hard
to fit into the Phab ecosystem, since there's no place to hang the
"API hash not correct" message in the case when the hash hasn't been
updated at all. Therefore, this commit doesn't convert it. See also
https://github.com/mozilla-mobile/gradle-apilint/issues/61 for adding
file/line information to API lint.
Differential Revision: https://phabricator.services.mozilla.com/D35277
--HG--
rename : mobile/android/config/mozconfigs/android-api-16-frontend/nightly => mobile/android/config/mozconfigs/android-api-16/nightly-android-lints
extra : moz-landing-system : lando
This build re-uses the PGO profile from the win64 build in the
win64-aarch64-shippable-no-eme part of the aarch64 build. Even though
the profile isn't generated on the smae platform, we still get enough of
a performance win to make this worthwhile.
Note that the pgo_flags() in configure need to be tweaked slightly since
we don't supprt the -fprofile-generate flag for aarch64 (we don't build
the clang_rt.profile lib there). So we always want to return the flags
namespace to make sure we get the use_* versions of flags, which we do
need.
Differential Revision: https://phabricator.services.mozilla.com/D38928
--HG--
extra : moz-landing-system : lando
In order to run tests against the win64-aarch64-shippable build, we need
the test packages from the aarch64-no-eme build since the tests aren't
compiled during the -shippable stage. Since packages like cppunittest
and the target.test_packages.json file won't be generated correctly in
the -shippable stage, we disable the package-step tests there by setting
MOZ_AUTOMATION_PACKAGE_TESTS to 0 in the environment.
Differential Revision: https://phabricator.services.mozilla.com/D37895
--HG--
extra : moz-landing-system : lando
We've been lucky that non-sanitizer cross-builds for macosx have not
required the clang runtime so far, but they soon will. And it's only
available in the mac-cross clang toolchain, so we need to use that on
all macosx builds.
Differential Revision: https://phabricator.services.mozilla.com/D38260
--HG--
extra : moz-landing-system : lando
In order to run tests against the win64-aarch64-shippable build, we need
the test packages from the aarch64-no-eme build since the tests aren't
compiled during the -shippable stage. Since packages like cppunittest
and the target.test_packages.json file won't be generated correctly in
the -shippable stage, we disable the package-step tests there by setting
MOZ_AUTOMATION_PACKAGE_TESTS to 0 in the environment.
Differential Revision: https://phabricator.services.mozilla.com/D37895
--HG--
extra : moz-landing-system : lando
Taskgraph needs to know the correct mar-channel, so allow it to pass it into the build,
rather than keying off the update-channel in configure. This will allow using a `mar`
binary that doesn't have the mar-channel configured in.
Differential Revision: https://phabricator.services.mozilla.com/D37480
--HG--
extra : moz-landing-system : lando
In newer docker versions the LeakSanitizer will fail unless it runs in a
privileged container. During the gecko build process in asan builds, the
support tools built as part of the build process will also run with
LeakSanitizer by default, but there is no need for such.
We then disable runtime leak sanitizer in asan builds to avoid erros in
builds inside a docker container.
Differential Revision: https://phabricator.services.mozilla.com/D37295
--HG--
extra : moz-landing-system : lando
Andreea Pavel
Mike Shal
Bob Clary
Justin Wood
Nathan Froyd
shindli
Wander Lairson Costa
Wander Lairson Costa
Chris Manchester
Marco Castelluccio
Tyson Smith
Daniel Varga
Edwin Takahashi
Ryan VanderMeulen
Adam Gashlin
Ciure Andrei
Oana Pop Rus
Aki Sasaki
Tarek Ziadé
Eric Rahm
Noemi Erli
Dorel Luca
Emilio Cobos Álvarez
Jesse Schwartzentruber
Ricky Stewart
Mike Hommey
Tom Prince
Christian Holler
Razvan Maries
Andi-Bogdan Postelnicu
Johan Lorenzo
Coroiu Cristina
Nick Alexander
David Major
Brindusan Cristian
Geoff Brown
Simon Sapin
Bogdan Tara
Tom Ritter