This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
We'd like to install the NDK through the Android SDK manager. But we
can't pin versions of the NDK with the SDK manager, and so Google
can silently upgrade the NDK on us. Since that is undesirable, this is
the next best thing.
With the toolchain task in hand, we can make all the relevant tasks
depend on the toolchain task and remove the download of the NDK from
tooltool as well.
Android non-gradle tests run against an Android apk built without
gradle, "Bng" on treeherder; confusingly, this platform is labelled
"Android API16+ Gradle opt". This patch significantly reduces the
tests run on this platform: now only robocop tests are run.
The full set of Android tests, including mochitests and reftests,
continue to run on the "Android 4.3 API16+ opt" platform.
We don't actually go install the package, but if a one-click loaner user
goes on to apt-get install gdb, they will get a version that is useful,
rather than the version in wheezy that won't give them e.g. variables
information.
--HG--
extra : rebase_source : 1f1ba607a759fc3136c59513773a043e8e8680c0
The GDB version in Debian wheezy doesn't handle the DWARF data that the
GCC version we use to build Firefox and toolchains produce. So we take
the GDB version from Debian stretch and backport it.
--HG--
extra : rebase_source : dae0e9dcd5dde5a7c74b6cefd560480fccd9c5fa
When running setup_packages in a docker image that derives from another,
we're currently overwriting the file that contains the apt sources for
the package artifact repositories that were used for the parent docker
image.
This doesn't cause practical problems for the existing docker images,
but in some cases where a user gets a one-click loaner, it might cause
problems when they try to install a package that has a dependency that
can't be fulfilled once those sources are overwritten.
To give a practical example, installing the gdb package from wheezy
requires libpython2.7, but if you try to do that on a derivative of the
debian7-base image, you don't have the deb7-python artifact repository
in your sources.list, and would fail to install gdb because apt can't
install a version of libpython2.7 that can be installed alongside the
python2.7 that is installed.
By putting easy repository in a separate file, named after the task id
of the corresponding package task, we ensure each an every one of them
is uniquely represented in /etc/apt/sources.list.d.
--HG--
extra : rebase_source : efb83b8c292a28c43ede24d5a3879dfbbfe94af7
This will disable libxss-based code in the screenshot tool, like it was
before the switch to Debian-based build images in bug 1399679. There are
other places in the code that use libXss, but they don't require the
development headers, and use dlopen().
--HG--
extra : rebase_source : 4842361b680c70924b9fc0890a7e4f4dcbc7e338
This is basically the same deal as e331a3b9fae2. Caching is hard.
MozReview-Commit-ID: 9uWHHdnHgq1
--HG--
extra : rebase_source : fc7107a018f067d83dbee3796d8861c5187823b2
We start from the image used for Firefox builds, and add the debug
packages for all the system libraries.
--HG--
extra : rebase_source : 2c759975d9837beabdc08a15fd926a99fd1cecf8
The binutils we currently use as part of our GCC toolchain artifact
doesn't understand some relocations in the CRT objects on Debian
stretch, making the embedded CRT objects from bug 1427344, which we want
to remove in bug 1431251, necessary.
OTOH, there is no benefit from using our GCC toolchain artifact for host
compilations on those builds. In fact, Android builds, which are in a
similar position, being built on Debian stretch and being cross-builds,
don't care to use our GCC toolchain artifact.
It's arguably a good thing that those builds are not tied to the version
of GCC we use to build Firefox for linux, so let's remove this
dependency.
--HG--
extra : rebase_source : a80d4e4fb01a4862b844ebde0c521a635f462c0a
I tested this on automation and the build went on, though I couldn't test
the bindgen output because the build right now is busted on one dependent crate
with rust beta, which is the first toolchain that has this package, and will go
to release shortly.
This should work though! If I need more changes I'll adjust them in bug 1432153.
You can test the repackage manually with repack_rust.py --toolchain beta, for
example.
MozReview-Commit-ID: GI2f6vGVqTe
Don't build ucl when building upx, Debian stretch has a recent enough
version. In fact, the last upstream version doesn't build with GCC in
Debian stretch (http://bugs.debian.org/811707)
--HG--
extra : rebase_source : aae67773b9dd3b99f6ddf9ab7f59a628037e6925
This job requires cmake, which should be fixed, but in the meanwhile,
create a separate docker image with it installed, based on the image we
use for other spidermonkey builds.
--HG--
extra : rebase_source : da43a7999b6bd86dbba816358d907c902415bed4
We've observed apt failures multiple times where it apparently fails to
get a file in full from snapshot.debian.org. Making it retry
automatically rather than retriggering tasks seems better.
--HG--
extra : rebase_source : f3ffb415ccc30b7e7c44e6a48b29eb20e69efdd5
The apt in Debian stretch doesn't allow repositories with a Release file
not being GPG signed. Setting up GPG signatures on the
taskcluster-artifact-based repositories is a tricky process, and not
strictly necessary. It turns out not creating a Release file at all
works just as well, and works across all current Debian versions. The
packages priorities remain the same, such that packages from those
repositories are still prefered over the ones from the main Debian
repository (as long as versions are higher).
See comment in cloud-mirror-workaround.sh for that part.
--HG--
extra : rebase_source : df5af330859a314285a6c1922d899489997d2f19
That image is used to derive all the debian7-* images, and its
definition is parametrized, which will allow to create other images
based on other versions of Debian, from the same definition.
XZ_OPT is kept in each of those because we don't want to automatically
set it in all further derived images.
--HG--
extra : rebase_source : 7f4597c1ea4af83627a9373dbdc7945d20b7d996
The base images from docker hub actually contain a
/etc/apt/apt.conf.d/docker-clean that does the equivalent of an apt-get
clean after installing packages.
--HG--
extra : rebase_source : 190de9e3b10a0309cf9cfb3260a91477a5a93ba3
python-dev was required to build mercurial, but the need for that was
removed in bug 1429669.
The others were required for mingw32 toolchains, but they are using a
different docker image and will switch to another different docker image.
--HG--
extra : rebase_source : b65c586a325f220c565e79afb3d3c9acc9f922bc
So far, the best we've been able to do is to upload an image to the
docker hub, and point an image's Dockerfile's FROM to the version
uploaded onto the hub.
That is a cumbersome process, and makes the use of "layered" docker
images painful.
This change allows to declare a parent docker image in the
taskcluster/ci/docker-image/kind.yml definitions, which will be
automatically loaded before building the image. The Dockerfile can then
reference the image, using the DOCKER_IMAGE_PARENT argument, which will
contain the full image name:tag.
Some details are left off, for now, such as VOLUMEs. At this point,
VOLUMEs should all be defined in leaf docker images.
--HG--
extra : rebase_source : 221cff0ca5a91d694ff5c3626fe707c15ba45e23
Now that `mach taskcluster-build-image` can, we can avoid all the manual
handling based on curl and jq in the image builder.
An additional advantage on relying on `mach taskcluster-build-image`
doing more is that less changes to the build-image.sh script will be
necessary, and thus less updates of the image builder docker image.
--HG--
extra : rebase_source : dd174d60675e41e4391894f28235c674c1840829
This allows to avoid writing out a tar file to then extract it to feed
it to `docker build`. This is essentially what the image-builder docker
image does, except it uses a temporary file for the tar.
--HG--
extra : rebase_source : 8275d737e02714fc198d3ba3d3e62e3f18d8e0bf
While spawning `docker load` is likely to work on developer machines,
on automation, it requires a docker client that is the exact same
version as the server running on the taskcluster worker for
docker-in-docker, which is not convenient. The API required for `docker
load` is rather simple, though, and can be mimicked quite easily.
While this change in itself is not necessary for developer machines,
it will allow to re-use the same command for the image-builder to
load a parent docker images when deriving one from another. We could
keep a code branch using `docker load` but it seems wasteful to maintain
two branches when one can work for both use cases.
--HG--
extra : rebase_source : d72956d7dd329b92564cbaa3fbfe0687d4d5d994
The zstd command we spawn, if available at all, might be the wrong
version: zstd changed its stream format in an incompatible way at some
point, and the version shipped in e.g. Ubuntu 16.04 uses the old format,
while the version taskcluster relies on uses the new format.
Relying on gps's zstandard library allows to ensure we use the right
version. Another advantage is that we can trivially pip install it in a
virtualenv if it isn't available on the system running the command.
If we're ridding ourselves of the subprocess spawning for zstd, we might
as well cover curl as well. Especially considering the error handling
when subprocesses are involved is not trivial, such that the current
error handling code is actually broken and leads to dead-lock
conditions, when, for example, curl is still waiting for the python side
to read data, but the python side is not reading data anymore because
an exception was thrown in the tar reading loop.
--HG--
extra : rebase_source : 054c37cfaa68bf475b37545ebaa99144584b93d4
The used pattern:
except Exception:
error = sys.exc_info()[0]
finally:
...
if error:
raise error
actually loses everything that is interesting about the original
exception. Not catching the exception just makes it thrown up the stack,
except when a different exception is thrown from the finally block,
which is what that if error: raise error is attempting to do... except
it doesn't throw the original exception, but its type only.
--HG--
extra : rebase_source : 17601fcc90fcdfefd93c4267f3cd33425d5326fd
Now that we don't need to read the contents of a file to hash the
contents of a docker image context, we can avoid creating a file
in generate_context_hash.
--HG--
extra : rebase_source : 98abe9bfdc48b612a3d251296991d0f769b449fd
This will allow us, down the line, to avoid creating a file at all in
some cases.
--HG--
extra : rebase_source : e4ea40341836cf24aa6d61c905b2efa660ee13f2
It's been more than two weeks since the 1.23 stable release, and
we're making official builds with that toolchain release, so begin
requiring that version so new language features can be used in
development.
MozReview-Commit-ID: E6WuP41ceTn
--HG--
extra : rebase_source : 75850dd9edbf8e3f9beab394e4af7fad76ce3b17
When we branched to beta, the MinGW build started failing with a strange error
like "could not find the robustcheckout Mercurial extension". We disabled it
for a while, but now we're re-enabling it (and fixing the problem.)
The root culprit of this was that we were using the incorrect mozconfig. MinGW
does all sorts of stuff in the mozconfig, but the beta branch overrides the
mozconfig using platform_overrides in
testing/mozharness/configs/builds/branch_specifics.py
We avoid this override by changing the MinGW platform so it doesn't match
and the mozconfig doesn't get overridden.
MozReview-Commit-ID: JkETWCRHacO
--HG--
extra : rebase_source : 5ef7d96804acd3a4eaa6dd6cc8b483f3de438b61
New Android-Gradle plugins pin the build-tools version, and we want to
be consistent between Gradle and moz.build.
MozReview-Commit-ID: ApWS4rHzPuH
--HG--
extra : rebase_source : 22008e9333b15c594ce26c2a52f67396d6e3ab84
extra : source : f918500d9cf5112b70bc8e0a120df435b02252b7
Turns out Google's maven repository doesn't publish checksums. I
can't imagine why not, but there it is. We have to think more about
whether to trust the artifacts downloaded from maven.google.com.
MozReview-Commit-ID: CdWijorq1IV
--HG--
extra : rebase_source : 6c66cf1444876624f10409ea6437863e2c2ea9b0
extra : source : 0850b319efd43ac8f4d61485451722975da55ca1
New upstream release.
- Avoiding argument copies improves memory footprint.
- RwLock<T> no longer requires T to be Send.
- AsciiExt trait methods are now directly available
on str, [u8], u8, and char types without a `use`
statement.
MozReview-Commit-ID: 7Rx8uoNTMqH
--HG--
extra : rebase_source : 39d6297d61d19d710a1376557e4b4d81bdab02c9
In automation for both Linux and Linux64 the same docker image is used,
which is based on Ubuntu 16.04 64bit. To run the 32bit Firefox binary
the necessary libraries for the i386 architecture get installed.
What's missing is the creation of the font cache for i386 binaries. As
such it will be created at the first start of the appropriate application.
It means there is an additional ~30s delay before Firefox opens, which
causes several test failures especially for wdspec tests.
To prevent this delay the i386 version of fontconfig will be installed
and apt automatically creates the font cache for the i386 architecture
by leaving the x86_64 cache in-place.
So the cache creation will happen one time when the docker image is getting
build, and not for each and every test job, which saves us 30s for each.
MozReview-Commit-ID: 58xdOI4xYkP
--HG--
extra : rebase_source : f246fc47caa1acc8657c4963c9450c1ca7653fec
This patch disables the stdcxx-compat check for the sm-fuzzing build which
requires patching autospider as well. Furthermore, it switches the build
to linux64-clang-6-pre because the older clang 3.9 does not support trace-pc
instrumentation. Finally, it excludes fuzzing parts from the vanilla allocation
check.
MozReview-Commit-ID: FdhCIFdUore
--HG--
extra : rebase_source : c41bda01cb42f2ef0cd5a1675d88bdb55d9dc8c9
So we can change a string in this file to force the use of new caches,
thus ensuring a clean break from one changeset to the next.
MozReview-Commit-ID: EZsR23a1PZE
--HG--
extra : amend_source : bf6a3c53ab14f2eb249fdf57ff70dce97388f0a0
Switch almost all builds currently using the desktop-build image to use
the right debian7-*-build image instead. The only exception is the rust
bindgen spidermonkey builds, that require cmake being installed, but I
don't want to add it to the base images because that involves risking a
cmake dependency unwantedly slipping in Firefox (rust-bindgen ironically
requires cmake to build a single C++ file...)
Bump mingw version to get the newest commit and do not include the
un-needed dw-extras.h on MinGW (thanks Jacek!)
MozReview-Commit-ID: OjO93XHCxs
--HG--
extra : rebase_source : 933bbb385004988a23d1069c9cd3241b3a3b336e
It is no longer used as part of the build, now that publishing is handled in
separate tasks.
Differential Revision: https://phabricator.services.mozilla.com/D373
--HG--
extra : rebase_source : a6e31fe5d554076d4da584f0993c21b6c1a8ebd0
This makes the XZ_OPT environment variable set for every task that runs
on those docker images, effectively enabling parallel compression for
all of them.
For e.g. the gcc 4.9 toolchain build task, this makes the final
compression step take 23s instead of > 3 minutes.
--HG--
extra : rebase_source : 83809f1f72007cc0829573f1def29a9afdc300c9
This adds an 'override' for the default scheduling component for tests, which
is based on their suite.
MozReview-Commit-ID: 6vd8sb2zeuU
--HG--
extra : rebase_source : 25988c6790287e01fa7751effa72e8b924858948
llvm-symbolizer is necessary to get symbols in llvm-dsymutil crash
dumps. While we could use the one from clang during the build, it's
better if the llvm-dsymutil toolchain is standalone for local testing.
--HG--
extra : rebase_source : 5cd234a3e14ab52a4ce759821e0e756e68167797
When I originally wrote the llvm-dsymutil build script in bug 1430315,
I wasn't setting CMAKE_BUILD_TYPE to Release, and was ending up with
a very large binary (> 300MB), so I stripped it.
When I later set CMAKE_BUILD_TYPE to Release, I left the manual
stripping on, but that removes symbols that are useful for stacktraces
when dsymutil crashes (the Release type still leaves out debug info).
--HG--
extra : rebase_source : 802daadc24c0090574b1a44ea8b4e6c25735f703
By default, wget prints dots every 1k bytes. This can render a
lot of output for large files. We switch to the "mega" style, which
makes each dot represent 64k, thus reducing output by up to 64x.
We also force the use of dot display. By default, it uses "bar"
which attempts to use terminal formatting if possible. Since most
of this code executes in CI and terminal control characters can
interfere with logged output, we force the use of "dot." (Although
wget appears to automatically switch to dot in TC today. But
consistency is good.)
MozReview-Commit-ID: IpTWJdcauTV
--HG--
extra : rebase_source : 5c9aa1bbdcd78eaa0b31347ad026a2c1beaedc03
Now that we have a version of xz-utils that supports parallel compression,
we're not going to use it.
--HG--
extra : rebase_source : 7ac7840e979176b8d2042a4ef35f5868acd6f224
There were a few constraints in the choice of the version of dpkg to
backport:
- 1.17.20 is the first version that supports the debian source format
for that xz-utils package.
- versions >= 1.17.10 and <= 1.17.22 fail to build on wheezy.
- versions >= 1.17.21 depend on a version of patch not available on
wheezy.
All in all, the simpler choice was to go with version 1.17.20 with a
backport of the build failure fix.
That version of dpkg breaks the version of devscripts in wheezy, so the
version from wheezy-backports would be better to use, but we can't
unconditionally use it on all builds, because it happens that
mk-build-deps from that version is broken with the dpkg in wheezy.
In the end, it's simpler to build that backport and rely on package task
dependencies rather than selectively install the package from
wheezy-backports, so we do that. Except we can't use version
2.14.11~bpo70+1 because of bug 1419577.
--HG--
extra : rebase_source : 19ad1a44b770229fbc7e15bbcf01d3cb101315a8
In some cases, apt-get can't figure out how to install build
dependencies, and aptitude, with hints, is necessary instead. But
aptitude also doesn't necessarily work in some cases (e.g. it picks the
wrong automake version to build valgrind, although that's a problem in
valgrind's build dependencies). All in all, it's still better to use
apt-get for all the simple cases.
--HG--
extra : rebase_source : 5dcde3507f2642be8f284b6148d1ccaa4761164a
It can be necessary to use the artifacts from package tasks as build
dependencies for other package tasks.
--HG--
extra : rebase_source : a625670ae7af9a99c848924bdba9368c556e6766
Not all changes to debian_packages.py lead to actual changes to the
corresponding tasks. And since the tasks are pretty much self-contained,
we can consider that variations of them will be entirely represented in
the command used for the task. The only exception is the patch content
when there is an associated patch.
--HG--
extra : rebase_source : e70fa41a824106b5ceca09fd535c0c36bb0739ac
In many cases, building docker images starts on machines that don't have
a cached checkout, and it often takes forever to get a full clone. It
used to be worsened when 3 jobs could run at the same time because the
worker would start up clean, and 3 jobs would be doing a mercurial clone
at the same time, thrashing I/O, but that part is fortunately fixed.
It is still, however, appreciable not to waste time in the mercurial
clone part of image creation.
--HG--
extra : rebase_source : 8c76bc91e1d5102f68c43e1050d61971fef32e9f
The image builder image we use to build docker images is updated
manually, and not necessarily when changes occur in tree that should be
reflected by a new image builder image. For instance, its run-task is
currently outdated. Not enough that it's actually a problem, but it
could rapidly become a problem.
There is also a lot of friction when trying to make changes in how
docker images are built, and while last time I tried, I ended up not
being able to do the changes I wanted to make because the docker version
on the host is too old, but this is already the second time I've been
trying to make things better and hit a wall because the the image
builder is essentially fixed in stone on the docker hub.
So with this change, we make all the docker images use the in-tree image
builder image, except itself, obviously. That one uses the last version
that was uploaded. We may want to update it at some point, but not doing
so will only impact building the image builder image itself, not the
other ones.
--HG--
extra : rebase_source : 978cf033732cbbbb277d206dec69660175b82afa
Back in bug 1360609, we added `run-on-projects` to a list so that the
toolchain tasks wouldn't run on every push on release branches.
Fast forward to now, and they're depended upon by other tasks, meaning
they are triggered when appropriate, without resorting to that trick. In
fact, the commit message for bug 1360609 said we could switch to an
empty list once the jobs have dependencies.
The same is true from package tasks, which, in fact, I suspect would
happen on every push on release branches.
The only exception is for a few toolchains that are depended upon by
nothing, and that are produced for developer consumption with e.g. mach
artifact toolchain.
--HG--
extra : rebase_source : bb8624fed7490b85f4bd72b7ceb2db7a72b4c2ab
There are now only a handful of buildbot jobs remaining and the concern over
outdated treeherder exclusion profiles has largely been resolved.
This does remove the tc() group from a substantial number of tasks which will
now show up as top level tasks, potentially adding clutter. In some cases, we
might want to re-add a new group (e.g group builds or compiled tests together).
However rather than try to predict the best group names for tasks I'm unfamiliar
with, I think it's best to land this as is. Then if things are looking too
cluttered at the root namespace, file follow-up bugs as needed.
MozReview-Commit-ID: 8SMwjDwAOzV
--HG--
extra : rebase_source : 2f6d89d11c139bdcd404e7537db799d0e36ee4c3
When trying to remove an ubiquitous group like tc(), it's hard to tell where the
error was located without grepping my filesystem. This makes it a bit easier to
find and fix these errors.
MozReview-Commit-ID: 8NjvB5zOoqb
--HG--
extra : rebase_source : 167d3097f96548cf9c13b602d7d485cb69d48c2d
Relying on the various transforms setting it manually is error prone,
and, in fact, is why bug 1430037 busted beta. This change makes this
setting happen at a single place. This yields the same full task graph
as before, except for *more* chain-of-trust inputs being set now: they
were missing for toolchain tasks (which makes us closer to bug 1384430).
--HG--
extra : rebase_source : b6bf3a3b6da7174957c4c6814b853a51ee8a1e27
Add a geckoview-docs job that executes "./mach android geckoview-docs",
which takes care of calling gradle to generate the javadoc archive, and
uploading it to Github using given parameters.
MozReview-Commit-ID: DTWh4XdFZEO
--HG--
extra : rebase_source : 9d75be24cb553b3a773d3d34a2bdbdf4d4c8cd34
With the use of job-defaults, we can avoid a lot of repetition from
those definitions.
--HG--
extra : rebase_source : 932c2ed530aa8aec9a33da60cf652535fa0bd303
The mozharness transform is supposed to set the docker image to
desktop-build when not already set, but was not doing it properly.
I guess this is why some jobs were setting the image themselves, despite
using the mozharness transform.
Consequently, don't manually set the image to desktop-build when it's
the default.
--HG--
extra : rebase_source : 024bd10960bedaee3416785348a5c12498c5286f
At the same time, restrict the installed packages to the script
requirements to build Firefox. Toolchains have their own image so we
don't need to install packages for them.
--HG--
extra : rebase_source : c0e7aa178b1ce2ceb01f9dfe6af37bbb54d4d708
The one available in Debian wheezy is 1.7.10.4, which is really old, and
on our centos images, we're using 2.8.0rc3, which, while old too, is
more modern. While we may want to go with a more recent version, I'd
rather avoid differing from what we currently use, so use the exact same
version.
--HG--
extra : rebase_source : dfdf75a635073c248faef8a67648b2a83e4a1d84
... except libdmg-hfsplus. RedHat decided to patch libbz2 to have a
different soname, so a binary built on Debian can't run on
RedHat/CentOS. Ironically, a binary built on RedHat/CentOS can run
on Debian. While we could use some tricks to make libdmg-hfsplus built
on Debian work, at this point, it's not worth the effort. We can live
with libdmg-hfsplus being built on CentOS until the builds that use it
switch to Debian, which is imminent.
... and except mingw32-nsis. Sourceforce renewed their certificate last
week and somehow the corresponding CA is not yet recognized by the
ca-certificates in Debian wheezy (an update is underway but see below)
... and except wine, because it requires more 32-bits packages than can
be installed on the toolchain-build docker image. But all things
considered, the mingw32 builds don't need to be using the same docker
images as the linux builds, and they could be, like the android builds,
be based on a more recent build image. So the corresponding toolchains
can be built on a more recent version of Debian too.
Consequently, we keep all the mingw32 related toolchains on the
desktop-build image for now.
Because Ubuntu 16.04 changed in a way that busts gl3 tests and we can't
update the desktop1604-test image anymore.
--HG--
extra : amend_source : bfa07f9f77990dd6915b8c92d218227436bc6fc4
The install-mercurial.sh script currently installs a global mercurial
configuration after installing mercurial manually. In order to share
that configuration with docker images installing a mercurial package
through packages tasks, we move it to a separate file.
We however keep the part setting web.cacerts in install-mercurial.sh,
since it uses a path depending on what kind of environment the script is
run. Moreover, the instructions that come with mercurial to build
Debian packages come with web.cacerts set to the right path already, so
it's not needed in that case anyways.
At the same time, use multiple files in /etc/mercurial/hgrc.d/ instead
of a single /etc/mercurial/hgrc file.
--HG--
extra : rebase_source : 8140d8243cf012489025afe058f467c72224c891
Build dependencies won't be installed from backports unless they are not
satisfiable in the given Debian release. This is useful to get dh-python
on Wheezy.
--HG--
extra : rebase_source : 1f249b4ceae4fdd9ea37e9a9b9e9b62b48a1c9ed
In the case of mercurial, we don't want to use a .dsc as the original
source, but rather use the debian packaging scripts available in the
upstream mercurial tarball.
--HG--
extra : rebase_source : ec5b288f3994bc0bc1ec9ebce40def807bb7681f
The taskgraph.util.schema.check_schema function validates key names used
in schemas, ensuring they are dashed lower-case. However, it currently
assumes keys are either direct strings, Required or Optional entries,
and either ignores or fails to recognize other patterns.
For example, it ignores Any, and fails to recognize combinations like
Required(Any(...)), which we're going to use in next patch.
--HG--
extra : rebase_source : 4f6ff51a4a9dc9c7d9b6d070e03c6cc6e1befe80
I don't intend to install ccache in the Debian build images. Hazard
builds are the only builds running on the desktop-build image still
using ccache somehow, and that gains them nothing, since the ccache
directory is not set to a cached directory on taskcluster, meaning
the build always starts with an empty cache. If anything, this currently
makes the build slower.
Eventually, those builds should be able to use sccache, once the
necessary setup moves out of mozconfig.cache.
--HG--
extra : rebase_source : fba6dab78b25ea61892cbe6127ead36da395b0e0
taskcluster/taskgraph/transforms/task.py sets an expiry to 28 days for
tasks on try, vs. 1 year on other projects, but only do that when an
expiry is not already set, which docker images do. And they do always
set to 1 year.
But it doesn't make sense to keep large docker images from try for a
year. So use the same retention policy as the default one. We /could/
just remove the expiry from docker images and get the task.py default,
but it seems like whatever future change might happen to that default
shouldn't affect docker images, so it's better to duplicate the setting.
--HG--
extra : rebase_source : b5b46ca34a40ac82c5403b67d5b1aacf8cf8cceb
It was failing to build with the GCC/binutils on the CentOS-based docker
image, but it doesn't with the Debian-based one, so we can remove the
dependency on the gcc toolchain task. This allows sccache to remain
untouched when we change the gcc build scripts, and more importantly,
this allows it to depend on no toolchain that requires building things.
This makes it now possible to use sccache as a dependency for all other
toolchains jobs that compile, if that's beneficial (which might not be
the case, given the current sccache retention time, but at least it's a
viable option, now)
New Android-Gradle plugins pin the build-tools version, and we want to
be consistent between Gradle and moz.build.
MozReview-Commit-ID: ApWS4rHzPuH
--HG--
extra : rebase_source : 38a9781c472d858f3300cbbcbdc6d2311c465713
Turns out Google's maven repository doesn't publish checksums. I
can't imagine why not, but there it is. We have to think more about
whether to trust the artifacts downloaded from maven.google.com.
MozReview-Commit-ID: CdWijorq1IV
--HG--
extra : rebase_source : a884971e51ce7b1ff993754b130f462c476646ab
New Android-Gradle plugins pin the build-tools version, and we want to
be consistent between Gradle and moz.build.
MozReview-Commit-ID: ApWS4rHzPuH
--HG--
extra : rebase_source : 5a5730b4b9ce84af40a7c73c4f1abba017103f02
Turns out Google's maven repository doesn't publish checksums. I
can't imagine why not, but there it is. We have to think more about
whether to trust the artifacts downloaded from maven.google.com.
MozReview-Commit-ID: CdWijorq1IV
--HG--
extra : rebase_source : e4373273e7aea7df79d70b5fbc233233a84d2360
This marks **/docs/** as exclusively docs, and code that is autodoc'd as
inclusively docs.
That means that a change that purely modifies documentation files will *only*
run `docs` tasks, while a change that modifies autodoc'd source code will
*additionaly* run `docs` tasks. The tasks do not run by default.
MozReview-Commit-ID: G9tOK0AwtrI
--HG--
extra : rebase_source : 8dd971e5c9b0eb5f47895664a4ea49442f303ecb
extra : source : 0881de9b2b5e36ec37cc866f1d4af109da57a919
This makes use of pytest's generation feature. To add a new
template test, just add a new entry containing the input and
expected output to the dict in test_templates.py
MozReview-Commit-ID: 4qMefYHMjAp
--HG--
extra : rebase_source : ba3049885d1a2485048e1ff9913be43317559376
This is a minor cleanup of the python.yml source test tasks.
MozReview-Commit-ID: 6UanmbZHF8P
--HG--
extra : rebase_source : e06d310af9ca05bfdab1bc1e3bd2bc6aa3035cb9
This marks **/docs/** as exclusively docs, and code that is autodoc'd as
inclusively docs.
That means that a change that purely modifies documentation files will *only*
run `docs` tasks, while a change that modifies autodoc'd source code will
*additionaly* run `docs` tasks. The tasks do not run by default.
MozReview-Commit-ID: G9tOK0AwtrI
--HG--
extra : rebase_source : 8dd971e5c9b0eb5f47895664a4ea49442f303ecb
extra : source : 0881de9b2b5e36ec37cc866f1d4af109da57a919
And trigger a new Ubuntu 16.04 docker image with a different hash to get
the latest version of the GL drivers that were updated to llvmpipe 5
recently. Those new drivers make Firefox enabled
WEBGL_compressed_texture_s3tc, making the test pass.
--HG--
extra : rebase_source : 09a8995829f985aef29a8919fecaefaf90791584
We've had problems with crashes in llvm-dsymutil for a while, and while
they are, in essence, due to the fact that rustc produces bad debug
info, they are a hurdle to our builds. The tool comes along clang, and
updating clang is not necessarily easy (witness bug 1409265), so, so
far, we've relied on backporting fixes, which can be time confusing
(witness bug 1410148).
OTOH, llvm-dsymutil is a rather specific tool, that doesn't strictly
need to be tied to clang. It's only tied to it because it uses the llvm
code to do some of the things it does, and it's part of the llvm source
tree. But it could just as well be a separate tool, like it was(is?) on
OSX.
So, we add a toolchain job to build it from the llvm source,
independently from clang, so that we can update it separately, if we
hit new crashes that happen to already be fixed on llvm trunk. It will
also allow to more easily update after upstream fixes crashes after we
report them.
--HG--
extra : rebase_source : b814353b4b4632e46646a24b8f54c5300618ff49
In many cases, building docker images starts on machines that don't have
a cached checkout, and it often takes forever to get a full clone. It
used to be worsened when 3 jobs could run at the same time because the
worker would start up clean, and 3 jobs would be doing a mercurial clone
at the same time, thrashing I/O, but that part is fortunately fixed.
It is still, however, appreciable not to waste time in the mercurial
clone part of image creation.
--HG--
extra : rebase_source : bbe8b001849e59bb655bb0e9766a6071ad38a52c
The image builder image we use to build docker images is updated
manually, and not necessarily when changes occur in tree that should be
reflected by a new image builder image. For instance, its run-task is
currently outdated. Not enough that it's actually a problem, but it
could rapidly become a problem.
There is also a lot of friction when trying to make changes in how
docker images are built, and while last time I tried, I ended up not
being able to do the changes I wanted to make because the docker version
on the host is too old, but this is already the second time I've been
trying to make things better and hit a wall because the the image
builder is essentially fixed in stone on the docker hub.
So with this change, we make all the docker images use the in-tree image
builder image, except itself, obviously. That one uses the last version
that was uploaded. We may want to update it at some point, but not doing
so will only impact building the image builder image itself, not the
other ones.
--HG--
extra : rebase_source : 73e8fc51ea53af1e647fc1d5093c67d614dd009e
The one available in Debian wheezy is 3.81, but we're explicitly using
4.0 on CentOS, most notably because of its --output-sync option which
helps make logs better in some ways.
This takes the package from Debian jessie and builds it for Debian
wheezy.
--HG--
extra : rebase_source : 20bb550703fec41ed0175ef7f78c5b9a394160f3
This marks **/docs/** as exclusively docs, and code that is autodoc'd as
inclusively docs.
That means that a change that purely modifies documentation files will *only*
run `docs` tasks, while a change that modifies autodoc'd source code will
*additionaly* run `docs` tasks. The tasks do not run by default.
MozReview-Commit-ID: G9tOK0AwtrI
--HG--
extra : rebase_source : 8dd971e5c9b0eb5f47895664a4ea49442f303ecb
extra : source : 0881de9b2b5e36ec37cc866f1d4af109da57a919
This boils down to always setting worker.env to avoid a KeyError.
MozReview-Commit-ID: 1s4az9BFcc2
--HG--
extra : rebase_source : dabed2dedb00d176b829c6c0ff911e0236c5dec4
Sebastian Hengst
Brian Grinstead
Andrew Halberstadt
Gurzau Raul
Geoff Brown
Nathan Froyd
Johan Lorenzo
Marco Castelluccio
Ciure Andrei
Mike Hommey
Narcis Beleuzu
Cosmin Sabou
shindli
Ben Hearsum
Tom Prince
Mark Banner
Joel Maher
Aki Sasaki
Kim Moir
Rob Wood
Simon Fraser
Gregory Szorc
Nick Thomas
Tom Ritter
Tiberius Oros
Andreea Pavel
Brindusan Cristian
Sylvestre Ledru
Emilio Cobos Álvarez
Dorel Luca
Ralph Giles
Nick Alexander
Margareta Eliza Balazs
Csoregi Natalia
Jesse Schwartzentruber
Michael Kaply
Henrik Skupin
Christian Holler
Dustin J. Mitchell
Kartikaya Gupta
Ryan VanderMeulen
arthur.iakab
Noemi Erli
Jim Chen
Coroiu Cristina