Now that all GCC and related source tarballs extract to paths
independent of their version number, the scripts are all very
look-alike, so they can be consolidated.
Differential Revision: https://phabricator.services.mozilla.com/D40749
Bug 1479533 was proposing to add a similar functionality, but this
iteration avoids actually unpacking anything, and ensures
reproducibility by relying on the reproducible bits from the original
archives: file ordering, flags, etc. (since they are checksummed, those
are never going to change for a given archive).
Another notable difference is that this applies the repack on the fetch
task itself, rather than create a separate task to apply the repack. The
latter has advantages, in that it allows to change the repacking without
redownloading the original file from a third-party server, but in
practice, most changes to the repacking would trigger the download tasks
anyways.
This patch only takes care of changing the archive type (zip->tar), and
the compression type (anything->zstandard).
Differential Revision: https://phabricator.services.mozilla.com/D40740
What this means is that the sources for clang/llvm are downloaded
separately from the toolchain build (which also means we finally only
download a given version of clang once for all platforms).
In turn, this means the build-clang.py script needs to start with an
existing llvm-project tree, and we choose to make build-clang.py expect
that it's run from the llvm-project root directory.
This also means we don't need to download git for the windows toolchain
task.
Differential Revision: https://phabricator.services.mozilla.com/D40402
They are now all very similar, and there are only a few variations.
Differential Revision: https://phabricator.services.mozilla.com/D40365
MANUAL PUSH: avoid closing autoland while toolchains are rebuilt.
This removes the need for the hacks in wr-macos-cross-build-setup.sh and
wrench-windows-tests.sh, while keeping things working in other scripts.
Differential Revision: https://phabricator.services.mozilla.com/D40184
Plenty of places use `nproc`, and only a couple use `getconf
_NPROCESSORS_ONLN`. Use the former instead of the latter.
Differential Revision: https://phabricator.services.mozilla.com/D39999
--HG--
extra : moz-landing-system : lando
This is loosely based on what was in bug 1467359, but simplified to
handle git only, and simply using git-archive because, at least now,
it's deterministic (it uses the commit date as timestamp in tar
archives).
This also adds 4 tasks for some of the things we use for toolchains, but
doesn't hook them up yet.
This also upgrades the fetch docker image to Debian buster, and installs
the required packages in it.
Differential Revision: https://phabricator.services.mozilla.com/D39480
For some tasks, the workdir known to the decision task doesn't actually match
the workdir used in the task, which makes MOZ_FETCHES_DIR wrong when the
decision task derives it from the workdir.
On other tasks, MOZ_FETCHES_DIR is set to a relative directory, which
may work in some places where MOZ_FETCHES_DIR is used, but not in
others, that happen to be executed from a different directory.
To solve both problems, we set MOZ_FETCHES_DIR as a relative directory
everywhere, and we make run-task normalize it to an absolute path
before executing the task.
Differential Revision: https://phabricator.services.mozilla.com/D39152
--HG--
extra : moz-landing-system : lando
For some tasks, the workdir known to the decision task doesn't actually match
the workdir used in the task, which makes MOZ_FETCHES_DIR wrong when the
decision task derives it from the workdir.
On other tasks, MOZ_FETCHES_DIR is set to a relative directory, which
may work in some places where MOZ_FETCHES_DIR is used, but not in
others, that happen to be executed from a different directory.
To solve both problems, we set MOZ_FETCHES_DIR as a relative directory
everywhere, and we make run-task normalize it to an absolute path
before executing the task.
Differential Revision: https://phabricator.services.mozilla.com/D39152
--HG--
extra : moz-landing-system : lando
In browsertime.zip we should have:
browsertime/
package.json
package-lock.json
node_modules/
.bin/
browsertime -> ../browsertime/bin/browsertime.js
browsertime/
...
The idea is that we'll fetch browsertime.zip in a generic-worker
environment and be able to run Node.js from within the top level
browsertime/ directory.
Differential Revision: https://phabricator.services.mozilla.com/D38773
--HG--
extra : moz-landing-system : lando
Mike Hommey
Tom Prince
Razvan Maries
Sylvestre Ledru
Bogdan Tara
Andreea Pavel
shindli
Oana Pop Rus
Eric Rahm
Nick Alexander
Agi Sferro