If the run task generates bad profile data, the merge step in the
profile-use task will fail. However, retrying the profile-use task
doesn't fix the problem, and there isn't a straightforward way to retry
the run task in this situation. Instead we can add a clang toolchain to
all the run tasks, and perform the merge there.
This means the output from the run task will always be a successfully
merged file called 'merged.profdata', and we no longer need to perform
the merge as part of the profile-use build as a GENERATED_FILES step.
Depends on D45262
Differential Revision: https://phabricator.services.mozilla.com/D45263
--HG--
extra : moz-landing-system : lando
I found that I did not restore the `set -e` flag after temporarily disabling it in the debian10-specific piece of experimental code in `test-linux.sh`, and this caused a bunch of my try pushes to register as successful despite having multiple unexpected failures.
Differential Revision: https://phabricator.services.mozilla.com/D44774
--HG--
extra : moz-landing-system : lando
There was quite a bit of discussion of this in `#build` on IRC,
and the consensus was that geckodriver should be built as a
stand-alone Rust crate and not as part of Firefox/Gecko (say, as a new
--enable-project target). This follows that approach, and the
expression, modeled off of cbindgen but updated to cross compile from
a Linux host to all targets, is pretty straight-forward.
A sparse profile would be nice, but the way that the Gecko Cargo
workspace works means that the profile must accumulate Rust code from
many locations.
If we want to, eventually testing/geckodriver can be removed from the
top-level Rust workspace, the geckodriver-signing tasks migrated to
these toolchain tasks, consumers migrated to the signing tasks, and
geckodriver removed from the "common" test archive.
Differential Revision: https://phabricator.services.mozilla.com/D43646
--HG--
rename : taskcluster/scripts/misc/vs-setup.sh => taskcluster/scripts/misc/vs-setup32.sh
extra : moz-landing-system : lando
Maybe back when .cargo/config.in was added, the directory indicated for
vendored crates needed to be absolute. That is at least not the case
with the current supported versions of rust.
The current setup has a few caveats:
- .cargo/config.in has shown to become stale (it currently contains
multiple unused entries)
- non-gecko build tasks have to generate a .cargo/config on their own if
they want to use vendored crates
- in turn, non-gecko build tasks that don't, may unknowingly get their
dependencies from crates.io (see the recent attempt at moving
geckodriver builds to a separate task).
By checking in a .cargo/config file, we can alleviate the last two, but
that comes at the price of `cargo update` not wanting to act when
.cargo/config exists, because of the source replacement configuration.
But rust vendor gently generates a suitable configuration on its own, so
we can use that to generate a .cargo/config automatically. Which
addresses the first caveat of the current setup. That leaves us with
`cargo update` not working out of the box, but that just requires people
running it to manually remove .cargo/config first. Which is arguably
what rust wants you to do in the first place. It's kind of incidental
that we started with a .cargo/config.in rather than .cargo/config.
Now, while a simple .cargo/config works, that's not enough for the case
where the objdir doesn't live inside the source directory. In that case
cargo looks for the configuration from the objdir, and fails to find it.
So we still need a .cargo/config.in, which we generate with a little
trick.
Differential Revision: https://phabricator.services.mozilla.com/D43012
--HG--
rename : .cargo/config.in => .cargo/config
extra : moz-landing-system : lando
The remaining uses all need adjustements to in-tree mozconfigs, so they
all need to be done at once.
However, to make things slightly more intelligible, we do this in two
steps. This is step 1: we modify the use_toolchain transform to take care of
the transformation, while keeping the task definitions intact, so that
we only deal with mozconfig and build script adjustements here.
Differential Revision: https://phabricator.services.mozilla.com/D41890
MANUAL PUSH: to allow docker images to build without closing autoland
Differential Revision: https://phabricator.services.mozilla.com/D41038
--HG--
extra : rebase_source : 60ae00549917411d1839b6e3f8e6ae962d217470
extra : amend_source : a2531b115f5732345f8c34c88669428510d100a4
Now that all GCC and related source tarballs extract to paths
independent of their version number, the scripts are all very
look-alike, so they can be consolidated.
Differential Revision: https://phabricator.services.mozilla.com/D40749
Bug 1479533 was proposing to add a similar functionality, but this
iteration avoids actually unpacking anything, and ensures
reproducibility by relying on the reproducible bits from the original
archives: file ordering, flags, etc. (since they are checksummed, those
are never going to change for a given archive).
Another notable difference is that this applies the repack on the fetch
task itself, rather than create a separate task to apply the repack. The
latter has advantages, in that it allows to change the repacking without
redownloading the original file from a third-party server, but in
practice, most changes to the repacking would trigger the download tasks
anyways.
This patch only takes care of changing the archive type (zip->tar), and
the compression type (anything->zstandard).
Differential Revision: https://phabricator.services.mozilla.com/D40740
What this means is that the sources for clang/llvm are downloaded
separately from the toolchain build (which also means we finally only
download a given version of clang once for all platforms).
In turn, this means the build-clang.py script needs to start with an
existing llvm-project tree, and we choose to make build-clang.py expect
that it's run from the llvm-project root directory.
This also means we don't need to download git for the windows toolchain
task.
Differential Revision: https://phabricator.services.mozilla.com/D40402
They are now all very similar, and there are only a few variations.
Differential Revision: https://phabricator.services.mozilla.com/D40365
MANUAL PUSH: avoid closing autoland while toolchains are rebuilt.
This removes the need for the hacks in wr-macos-cross-build-setup.sh and
wrench-windows-tests.sh, while keeping things working in other scripts.
Differential Revision: https://phabricator.services.mozilla.com/D40184
Mike Shal
Edwin Takahashi
Chris Manchester
Nick Alexander
Mike Hommey
Dustin J. Mitchell
Razvan Maries