These scripts are included by hazard-analysis.sh. That's their only
reference in repo.
We could probably inline these scripts. But let's start by moving them
out of mozharness since no active mozharness based task is using them.
MozReview-Commit-ID: 13oen42Txmh
--HG--
rename : testing/mozharness/scripts/spidermonkey/build.browser => taskcluster/scripts/builder/hazard-browser.sh
rename : testing/mozharness/scripts/spidermonkey/build.shell => taskcluster/scripts/builder/hazard-shell.sh
extra : rebase_source : 782f7b3f3537cfefb51b0e5f1b459c8ad0daca5b
The goal is to use a newer Android-Gradle build plugin version (2.3.3
is latest stable). That requires a modern Gradle (anything 3.3+, but
3.4.1 is the default from my Android Studio), and also a newer
build-tools (25.0.3 is latest stable).
The locations of lint output changed, and we want to use the standard
output location because it's difficult to accommodate variant details
in custom names. We change the location of findbugs output to follow
suit.
This requires either:
- fixing lint errors
- adding to the lint whitelist
- using the new lint baseline
It's best to use the new lint baseline, which will happen in the next commit.
MozReview-Commit-ID: D19FzIDCJrE
--HG--
extra : rebase_source : 12d132c0c3e0dbe2b8873b31360ea96d612de44c
This is simply diagnostic; it's nice to see what the versions of the
Android packages installed are, like:
Installed packages:
Path | Version | Description | Location
------- | ------- | ------- | -------
<snip>
platforms;android-23 | 3 | Android SDK Platform 23 | platforms/android-23/
tools | 26.0.1 | Android SDK Tools 26.0.1 | tools/
This is really useful because it's not possible to pin most packages
to specific versions; that is, we can only install latest "tools", not
"tools;26.0.1".
MozReview-Commit-ID: HgZLGCAObEs
--HG--
extra : rebase_source : 33e5c9f81d05551c9e167eac62009f1de023b410
I don't think (the output of) this script is used anywhere.
MozReview-Commit-ID: DwMFtpozjNL
--HG--
extra : rebase_source : 005086039f520ba116534ab47ee49616c6958e85
The only tricky piece here is that the resulting toolchain archive is
private, and uses a newly allocated Task Cluster scope
(queue:get-artifact:project/gecko/android-sdk/*) to restrict access to
the archive. All SCM levels (1, 2, 3) have been given the new scope:
see https://tools.taskcluster.net/auth/roles/moz-tree:level:1 and
friends.
MozReview-Commit-ID: CcDqDOHODpe
--HG--
extra : rebase_source : 81dbb065f2a3c4e7733e964be66adb1733db52c6
I don't think (the output of) this script is used anywhere.
MozReview-Commit-ID: DwMFtpozjNL
--HG--
extra : rebase_source : 36b3cbe1a6a9e5cd163782c1c13653be8558a03a
The only tricky piece here is that the resulting toolchain archive is
private, and uses a newly allocated Task Cluster scope
(queue:get-artifact:project/gecko/android-sdk/*) to restrict access to
the archive. All SCM levels (1, 2, 3) have been given the new scope:
see https://tools.taskcluster.net/auth/roles/moz-tree:level:1 and
friends.
MozReview-Commit-ID: CcDqDOHODpe
--HG--
extra : rebase_source : 062bca8c65556f0f46e9c9cc6cd81eb04cf2b522
When adding sccache toolchain jobs in bug 1381772, building with gcc
failed, and building with clang worked, so I just went with the path of
least resistance. That's however a suboptimal position in the dependency
graph, so it's still preferable to use gcc if possible.
Looking exactly how it fails, it turns out it's because without CC being
set, ring wants to build with "cc", which ends up being the system gcc
instead of ours (our gcc archive doesn't provide "cc", only "gcc"), and
it is too old to support the compiler flags ring uses.
So setting CC does the trick.
--HG--
extra : rebase_source : 4c657664957dff1f7aebe470e0440a52c9e280e5
Copy the repack_rust.py from the rust-build docker container
so it can be used more generally by other taskcluster jobs.
Add --host, --target, and --suffix switches, allowing control
of the packaged toolchain and standard library builds from
the command line.
This drops the previous default behaviour of packaging all
supported platforms and targets.
Add a hard-coded copy of the Rust release signing key to
the script and add it to the running user's gpg config
so we can validate downloaded artifacts from the project
in automation.
Remove the keybase artifact validation since it requires
out-of-project network services and doesn't provide much
advantage in automation.
Calculate the SHA-2 checksum during download and remove
the dependency on shasum/sha256sum command-line tools.
Use more python for filesystem an process interaction
in general.
Create a generic rustc.tar.* package to correctly match
the unversioned unpack dirctory name.
Add support for copying the package to an output directory
if the UPLOAD_DIR environment variable is set. This lets
us hook up the script to taskcluster toolchain jobs without
an external wrapper.
MozReview-Commit-ID: 68LmY3QVU8V
--HG--
extra : rebase_source : f6329056d518ad2cd25faa5c71b22130cbc65c8f
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
I also snuck in some last-minute assertions and minor fixes into this patch:
- don't stop reporting for a callee if we've seen it already (or rather, make the reachable set local to a root rather than global to all roots). This slows down runs with hundreds of hazards, but results in every problematic root being reported, for a more accurate count.
- annotate away some thread assertions
- special-case annotation for bug 1400435 since it's a whole family of hazards
--HG--
extra : rebase_source : ac7335d45e3e0772d34cb42cc6a3f628564fd3d1
CLOSED TREE
--HG--
extra : amend_source : 84120d6bacb5d72a9fbe41e4c3b405d63825da7c
extra : histedit_source : 8320c2193761b745f10850055ee74a3c9ac73615%2Cfbc2a28d8c5004a53305ef858ca5aea4245691e0
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
Our current sccache build links in openssl's libraries dynamically. The
sonames of the dynamic libraries linked in are specific to the
CentOS/Fedora-ish systems that we build on; attempting to run the
generated sccache binaries on different systems (e.g. Debian-ish) will
result in failure. All of our current automation images are
CentOS-based, but for various reasons, Debian-based images may be used
in the future, and it would be great to have an sccache binary to run on
such systems as well. (It might also be interesting to distribute the
sccache binary we use to local developers as well, but that's a bit
further off.)
Therefore, this patch alters the sccache build on Linux to use static
linking for openssl. We cannot use the system openssl we build on
because the system openssl links to libkrb5, and the distribution we use
for the system images does not provide static libraries of libkrb5.
Building openssl ourself enables us to eliminate the libkrb5 dependency.
An sccache binary from builds with this patch depends on the following
libraries:
froydnj@hawkeye:~$ ldd sccache2/sccache
linux-vdso.so.1 => (0x00007ffe02b39000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff0e7403000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007ff0e71fb000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff0e6fdd000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff0e6dc6000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff0e69fc000)
/lib64/ld-linux-x86-64.so.2 (0x0000557c8540b000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff0e66f2000)
which are standard on any Linux distribution.
Christian Holler
Tom Prince
Brindusan Cristian
Marco Castelluccio
Nick Alexander
Sebastian Hengst
David Major
Ralph Giles
Gregory Szorc
Nathan Froyd
Ted Mielczarek
Tom Ritter
Mike Hommey
Steve Fink
Stephen A Pohl
Nick Fitzgerald
Chris Manchester