Remove references to remove_executables from test task configurations for
mac mochitest-chrome, mochitest-clipboard, and test-verify. This fixes the
minidump_stackwalk definition, allowing proper crash reporting.
Summary:
We'll be adding the new periodic file updates task to run in parallel. This patch
moves the existing one to make it clear it's running on buildbot, so we don't get confused
later on.
Reviewers: Callek
Reviewed By: Callek
Bug #: 1436369
Differential Revision: https://phabricator.services.mozilla.com/D681
--HG--
rename : taskcluster/ci/repo-update/kind.yml => taskcluster/ci/repo-update-bb/kind.yml
extra : rebase_source : c1386a05c378589efdd508199d97cb8121308686
Ensure better determinism when creating rust toolchain packages
by rejecting generic channels like 'stable' or 'nightly'. Instead,
insist on a specific version or date.
The current valid dates for beta and nightly can be obtained with:
curl -s https://static.rust-lang.org/dist/channel-rust-beta.toml | grep ^date
curl -s https://static.rust-lang.org/dist/channel-rust-nightly.toml | grep ^date
MozReview-Commit-ID: I0DXw1KJGZz
--HG--
extra : rebase_source : 92e158193072582b8568d9c9f00ffdefa0af1a9c
This was backed out temporarily in cset 82c009dbf592 for high-frequency
intermittent failures. The source of those failures has been resolved in
bug 1442608 and so we can turn these tests back on.
This fix does several things:
* Removes the mime cache generated by the desktop-gtk3 remote part
* Installs a stub shared-mime-info database
* Set default association for all types to use xdg-open
Note: There is still work[1] to be completed in snapd, adding OpenFile
support to xdg-open. Landing this is harmless though, it will fail
silently just as it does today but will start working when the snapd
feature lands.
1. https://github.com/snapcore/snapd/pull/4766
opening requested files,
MozReview-Commit-ID: 1eeOLeVN8xQ
--HG--
extra : rebase_source : e948bafc583f91177530ce00ec26c69619a7f8f9
At the time tests were enabled they were green. However bug 1440664 which was
inflight at the same time made them have intermittent oranges. Rather than back
out bug 1440664 (which technically landed later) I'm turning off the tests until
we can figure out what's going on.
--HG--
extra : source : d3b20e603c5fe0c96c0d4a0b58a5b189dc7467b8
At the time tests were enabled they were green. However bug 1440664 which was
inflight at the same time made them have intermittent oranges. Rather than back
out bug 1440664 (which technically landed later) I'm turning off the tests until
we can figure out what's going on.
Note that static analysis was the only remaining user of the 32-bit toolchain, so I've removed win32-clang-cl (or rather, renamed it to win32-clang-cl-st-an).
--HG--
rename : build/build-clang/clang-win32.json => build/build-clang/clang-win32-st-an.json
rename : build/build-clang/clang-win64.json => build/build-clang/clang-win64-st-an.json
rename : taskcluster/scripts/misc/build-clang32-windows.sh => taskcluster/scripts/misc/build-clang32-st-an-windows.sh
rename : taskcluster/scripts/misc/build-clang64-windows.sh => taskcluster/scripts/misc/build-clang64-st-an-windows.sh
Require the current stable Rust release so new features can
be used in development.
MozReview-Commit-ID: 4NQNk3RfBkF
--HG--
extra : rebase_source : 9d88e6fdb823bd2e2ca8ac9940b1fafd420eebdc
Having both .*-qr/.* and windows.* as patterns in the run-on-projects
test platform list means that a platform like windows10-64-qr matches
both and the parser complains. Instead of doing this, we can remove the
windows patterns and take the talos tests out of the windows-talos
test-set which accomplishes the same thing.
MozReview-Commit-ID: 9kMooiNiHTb
--HG--
extra : rebase_source : e58d74c43531cb1f4b625b188be696d1e951eda0
With run-by-manifest, the debug reftests timeout on OSX. Increasing
the chunks by 1 seems to improve them.
MozReview-Commit-ID: 14xidhwXCue
--HG--
extra : rebase_source : b3042c3a65c67fc54af5d64624427593e48c8364
The webgl mochitests will run on all nightly branches (so inbound, autoland,
m-c, try) by default.
MozReview-Commit-ID: E3KPTbbxE3E
--HG--
extra : rebase_source : 4001221ba39b5b7f00dc70c8cd2b1e3a18599cd7
The Proguard dependency is now managed by Gradle.
MozReview-Commit-ID: EOvKSE5z28P
--HG--
extra : rebase_source : 760b117f500cc639cc8c24e9c02933990f358dd7
- add balrog submit-toplevel - this replaces the final portion of the updates builder.
- rename balrog transform to balrog_submit, because it's for balrog locale submission
- make this default to the 'promote' phase. balrog and beetmover currently take the current
phase, which isn't always the wanted behavior.
- rename balrog publish to balrog schedule
- add balrog secondary push and secondary scheduling, for RCs
- remove the release_updates transforms
- make the task.py balrog transforms smarter
- get rid of the release_balrog_publishing transforms; ad a generic worker_type transform
- add BALROG_ACTIONS to scriptworker.py
- add get_balrog_action_scope()
- remove the unused balrog channel scopes
MozReview-Commit-ID: 369ACiOAd5F
--HG--
rename : taskcluster/taskgraph/transforms/balrog.py => taskcluster/taskgraph/transforms/balrog_submit.py
extra : rebase_source : 9311522460ae6790af14a6b8b9600019702f8cbd
We now have access to workers running on EC2 instances with dozens of
vCPUs. gecko-<L>-b-linux-large is m4.10xlarge, m5.12xlarge, c5.9xlarge,
or c4.8xlarge. gecko-<L>-b-linux-xlarge is m5.24xlarge, m4.16xlarge,
or c5.18xlarge.
Experimentation reveals that Clang tasks are the only tasks that
are CPU efficient enough (read: cost effective) to run on these
larger worker types.
This commit defines the new worker types and switches Clang toolchain
tasks to run on the new workers. clang5 and clang6 tasks take ~30 minutes
on the -large variant but ~17 minutes on the -xlarge variant. All other
tasks don't show as linear of a speedup. So running them on the
-xlarge variant isn't justified.
As part of this change, Mac toolchain tasks have been converted
to run on gecko-<L>-b-linux* workers. The gecko-<L>-b-macosx64 workers
are actually Linux. IMO the b-macosx64 worker type is no longer needed.
Moving the toolchain tasks off the worker should hopefully not be very
controversial.
MozReview-Commit-ID: HynQPMWiWHo
--HG--
extra : rebase_source : 1142767e2a51c17880909ec6f15b694db8a43af2
Since comm-central doesn't have integration branches like autoland, we want to
always disable sccache on nightly builds, rather than being able to key off of
the tree to determine whether to enable it. This is currently done by not
including the sccache toolchain. However, the mozconfig files expect sccache to be
available if `SCCACHE_DISABLE` isn't set, which wasn't the case on windows.
Differential Revision: https://phabricator.services.mozilla.com/D603
--HG--
extra : rebase_source : 565d0b6a3b24a23f6ae844a8cd97b9824e9f5282
extra : amend_source : 8c1679a71a2baf04a0dc83074234c5101705b806
Virtual function declarations should specify only one of `virtual`, `final`, or `override`, as per the Mozilla C++ style guide:
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Coding_Style
This lint warns about:
virtual void Bad1() final
void Bad2() final override
void Bad3() override final
Caveats: This lint doesn't warn about `virtual void NotBad() override` at this time because there are 8000+ instances. It also doesn't warn about function declarations that span multiple lines because the regex can't match across line breaks.
MozReview-Commit-ID: LcBsOAKKgz7
--HG--
extra : rebase_source : 4da72ffac59acdc9796e3f540f24bb97af989cd0
Update sccache build description to use the latest stable
rust toolchain. We didn't upgrade earlier because of problems
on Windows.
Note that we can't just depend on the stable rust toolchain
alias, because the toolchain deps are resolved in a single
pass. Instead, we use the current stable version explicitly.
MozReview-Commit-ID: 4OVbFsYZZLZ
--HG--
extra : rebase_source : 5b65d05f646f061a4018e0fad2e31e48e884912a
The upload is handled by taskcluster, which is handled by copying files around,
so remove support for specifying alternative commands.
Differential Revision: https://phabricator.services.mozilla.com/D540
--HG--
extra : rebase_source : 968249c0a308178b62af77d48e6aa307d4192e5a
Run unit tests under geckoview/ when running 'mach android test'. This
also lets us run those tests on Taskcluster.
The test report parser for 'mach android test' had a bug where the input
directory was wrong. As a result, we weren't producing test output at
all. This patch fixes the input directory, and outputs an error if no
reports are found at all to avoid this bug in the future.
MozReview-Commit-ID: IiswQaSPCr0
Also add rc-{google-play-track,rollout-percentage} for RC pushapk.
One nice side effect of using the same push-apk kind: we don't re-run push-apk during the ship_fennec relpro flavor if we've run the ship_fennec_rc flavor with the same build. (Google Play would reject the same buildid.)
This is really for bug 1433536, but MozReview is forcing me to include this patch with the others for reasons.
MozReview-Commit-ID: 69ymqVLv9E2
--HG--
extra : rebase_source : b87d4dd2394788a5452ff3f52a8ca5022a15b9ee
extra : intermediate-source : 7a826d274a4828018a836cf1149df29d403a7c11
extra : source : a3c60b0370a3e08ce765f87c1d7d5dad24879881
This used to only be relevant to Devedition and Firefox releases.
In bug 1433536 we're going to add RC Fennec releases. Let's rename
the parameter now, for less parameters churn.
MozReview-Commit-ID: 28e1Y5FG4On
--HG--
extra : rebase_source : 59d41887d856481ab85bb8d2221dfcebca4430b0
extra : source : 4c96e312d88a3f7037c1eb39a3031d4baf997015
also clean up and move more config to the promotion config.
MozReview-Commit-ID: FmTWNNPcEaZ
--HG--
extra : rebase_source : 40431217fafb6796dbd65c7dfeab0e891ac1bbd4
extra : source : 0f5418a83477c1b6b221e4d28515792410e504d0
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
We'd like to install the NDK through the Android SDK manager. But we
can't pin versions of the NDK with the SDK manager, and so Google
can silently upgrade the NDK on us. Since that is undesirable, this is
the next best thing.
With the toolchain task in hand, we can make all the relevant tasks
depend on the toolchain task and remove the download of the NDK from
tooltool as well.
Android non-gradle tests run against an Android apk built without
gradle, "Bng" on treeherder; confusingly, this platform is labelled
"Android API16+ Gradle opt". This patch significantly reduces the
tests run on this platform: now only robocop tests are run.
The full set of Android tests, including mochitests and reftests,
continue to run on the "Android 4.3 API16+ opt" platform.
We don't actually go install the package, but if a one-click loaner user
goes on to apt-get install gdb, they will get a version that is useful,
rather than the version in wheezy that won't give them e.g. variables
information.
--HG--
extra : rebase_source : 1f1ba607a759fc3136c59513773a043e8e8680c0
The GDB version in Debian wheezy doesn't handle the DWARF data that the
GCC version we use to build Firefox and toolchains produce. So we take
the GDB version from Debian stretch and backport it.
--HG--
extra : rebase_source : dae0e9dcd5dde5a7c74b6cefd560480fccd9c5fa
When running setup_packages in a docker image that derives from another,
we're currently overwriting the file that contains the apt sources for
the package artifact repositories that were used for the parent docker
image.
This doesn't cause practical problems for the existing docker images,
but in some cases where a user gets a one-click loaner, it might cause
problems when they try to install a package that has a dependency that
can't be fulfilled once those sources are overwritten.
To give a practical example, installing the gdb package from wheezy
requires libpython2.7, but if you try to do that on a derivative of the
debian7-base image, you don't have the deb7-python artifact repository
in your sources.list, and would fail to install gdb because apt can't
install a version of libpython2.7 that can be installed alongside the
python2.7 that is installed.
By putting easy repository in a separate file, named after the task id
of the corresponding package task, we ensure each an every one of them
is uniquely represented in /etc/apt/sources.list.d.
--HG--
extra : rebase_source : efb83b8c292a28c43ede24d5a3879dfbbfe94af7
This will disable libxss-based code in the screenshot tool, like it was
before the switch to Debian-based build images in bug 1399679. There are
other places in the code that use libXss, but they don't require the
development headers, and use dlopen().
--HG--
extra : rebase_source : 4842361b680c70924b9fc0890a7e4f4dcbc7e338
This is basically the same deal as e331a3b9fae2. Caching is hard.
MozReview-Commit-ID: 9uWHHdnHgq1
--HG--
extra : rebase_source : fc7107a018f067d83dbee3796d8861c5187823b2
We start from the image used for Firefox builds, and add the debug
packages for all the system libraries.
--HG--
extra : rebase_source : 2c759975d9837beabdc08a15fd926a99fd1cecf8
The binutils we currently use as part of our GCC toolchain artifact
doesn't understand some relocations in the CRT objects on Debian
stretch, making the embedded CRT objects from bug 1427344, which we want
to remove in bug 1431251, necessary.
OTOH, there is no benefit from using our GCC toolchain artifact for host
compilations on those builds. In fact, Android builds, which are in a
similar position, being built on Debian stretch and being cross-builds,
don't care to use our GCC toolchain artifact.
It's arguably a good thing that those builds are not tied to the version
of GCC we use to build Firefox for linux, so let's remove this
dependency.
--HG--
extra : rebase_source : a80d4e4fb01a4862b844ebde0c521a635f462c0a
I tested this on automation and the build went on, though I couldn't test
the bindgen output because the build right now is busted on one dependent crate
with rust beta, which is the first toolchain that has this package, and will go
to release shortly.
This should work though! If I need more changes I'll adjust them in bug 1432153.
You can test the repackage manually with repack_rust.py --toolchain beta, for
example.
MozReview-Commit-ID: GI2f6vGVqTe
Don't build ucl when building upx, Debian stretch has a recent enough
version. In fact, the last upstream version doesn't build with GCC in
Debian stretch (http://bugs.debian.org/811707)
--HG--
extra : rebase_source : aae67773b9dd3b99f6ddf9ab7f59a628037e6925
This job requires cmake, which should be fixed, but in the meanwhile,
create a separate docker image with it installed, based on the image we
use for other spidermonkey builds.
--HG--
extra : rebase_source : da43a7999b6bd86dbba816358d907c902415bed4
We've observed apt failures multiple times where it apparently fails to
get a file in full from snapshot.debian.org. Making it retry
automatically rather than retriggering tasks seems better.
--HG--
extra : rebase_source : f3ffb415ccc30b7e7c44e6a48b29eb20e69efdd5
The apt in Debian stretch doesn't allow repositories with a Release file
not being GPG signed. Setting up GPG signatures on the
taskcluster-artifact-based repositories is a tricky process, and not
strictly necessary. It turns out not creating a Release file at all
works just as well, and works across all current Debian versions. The
packages priorities remain the same, such that packages from those
repositories are still prefered over the ones from the main Debian
repository (as long as versions are higher).
See comment in cloud-mirror-workaround.sh for that part.
--HG--
extra : rebase_source : df5af330859a314285a6c1922d899489997d2f19
That image is used to derive all the debian7-* images, and its
definition is parametrized, which will allow to create other images
based on other versions of Debian, from the same definition.
XZ_OPT is kept in each of those because we don't want to automatically
set it in all further derived images.
--HG--
extra : rebase_source : 7f4597c1ea4af83627a9373dbdc7945d20b7d996
The base images from docker hub actually contain a
/etc/apt/apt.conf.d/docker-clean that does the equivalent of an apt-get
clean after installing packages.
--HG--
extra : rebase_source : 190de9e3b10a0309cf9cfb3260a91477a5a93ba3
python-dev was required to build mercurial, but the need for that was
removed in bug 1429669.
The others were required for mingw32 toolchains, but they are using a
different docker image and will switch to another different docker image.
--HG--
extra : rebase_source : b65c586a325f220c565e79afb3d3c9acc9f922bc
So far, the best we've been able to do is to upload an image to the
docker hub, and point an image's Dockerfile's FROM to the version
uploaded onto the hub.
That is a cumbersome process, and makes the use of "layered" docker
images painful.
This change allows to declare a parent docker image in the
taskcluster/ci/docker-image/kind.yml definitions, which will be
automatically loaded before building the image. The Dockerfile can then
reference the image, using the DOCKER_IMAGE_PARENT argument, which will
contain the full image name:tag.
Some details are left off, for now, such as VOLUMEs. At this point,
VOLUMEs should all be defined in leaf docker images.
--HG--
extra : rebase_source : 221cff0ca5a91d694ff5c3626fe707c15ba45e23
Now that `mach taskcluster-build-image` can, we can avoid all the manual
handling based on curl and jq in the image builder.
An additional advantage on relying on `mach taskcluster-build-image`
doing more is that less changes to the build-image.sh script will be
necessary, and thus less updates of the image builder docker image.
--HG--
extra : rebase_source : dd174d60675e41e4391894f28235c674c1840829
This allows to avoid writing out a tar file to then extract it to feed
it to `docker build`. This is essentially what the image-builder docker
image does, except it uses a temporary file for the tar.
--HG--
extra : rebase_source : 8275d737e02714fc198d3ba3d3e62e3f18d8e0bf
While spawning `docker load` is likely to work on developer machines,
on automation, it requires a docker client that is the exact same
version as the server running on the taskcluster worker for
docker-in-docker, which is not convenient. The API required for `docker
load` is rather simple, though, and can be mimicked quite easily.
While this change in itself is not necessary for developer machines,
it will allow to re-use the same command for the image-builder to
load a parent docker images when deriving one from another. We could
keep a code branch using `docker load` but it seems wasteful to maintain
two branches when one can work for both use cases.
--HG--
extra : rebase_source : d72956d7dd329b92564cbaa3fbfe0687d4d5d994
The zstd command we spawn, if available at all, might be the wrong
version: zstd changed its stream format in an incompatible way at some
point, and the version shipped in e.g. Ubuntu 16.04 uses the old format,
while the version taskcluster relies on uses the new format.
Relying on gps's zstandard library allows to ensure we use the right
version. Another advantage is that we can trivially pip install it in a
virtualenv if it isn't available on the system running the command.
If we're ridding ourselves of the subprocess spawning for zstd, we might
as well cover curl as well. Especially considering the error handling
when subprocesses are involved is not trivial, such that the current
error handling code is actually broken and leads to dead-lock
conditions, when, for example, curl is still waiting for the python side
to read data, but the python side is not reading data anymore because
an exception was thrown in the tar reading loop.
--HG--
extra : rebase_source : 054c37cfaa68bf475b37545ebaa99144584b93d4
The used pattern:
except Exception:
error = sys.exc_info()[0]
finally:
...
if error:
raise error
actually loses everything that is interesting about the original
exception. Not catching the exception just makes it thrown up the stack,
except when a different exception is thrown from the finally block,
which is what that if error: raise error is attempting to do... except
it doesn't throw the original exception, but its type only.
--HG--
extra : rebase_source : 17601fcc90fcdfefd93c4267f3cd33425d5326fd
Now that we don't need to read the contents of a file to hash the
contents of a docker image context, we can avoid creating a file
in generate_context_hash.
--HG--
extra : rebase_source : 98abe9bfdc48b612a3d251296991d0f769b449fd
This will allow us, down the line, to avoid creating a file at all in
some cases.
--HG--
extra : rebase_source : e4ea40341836cf24aa6d61c905b2efa660ee13f2
It's been more than two weeks since the 1.23 stable release, and
we're making official builds with that toolchain release, so begin
requiring that version so new language features can be used in
development.
MozReview-Commit-ID: E6WuP41ceTn
--HG--
extra : rebase_source : 75850dd9edbf8e3f9beab394e4af7fad76ce3b17
Dorel Luca
Tiberius Oros
Ken VanDine
Johan Lorenzo
Geoff Lankow
Cosmin Sabou
Ben Hearsum
Geoff Brown
Csoregi Natalia
Simon Fraser
Ralph Giles
shindli
Joel Maher
Mihai Tabara
Kartikaya Gupta
Sebastian Hengst
Rob Wood
Steve Fink
David Major
Justin Wood
Rail Aliiev
Marco Castelluccio
arthur.iakab
Kim Moir
Andrew Halberstadt
Nick Alexander
Aki Sasaki
Ciure Andrei
Jordan Lund
aselagea@mozilla.com
Gurzau Raul
Bob Clary
Tom Ritter
Gregory Szorc
Tom Prince
Chris Peterson
Narcis Beleuzu
Andreea Pavel
Nick Thomas
Jesse Schwartzentruber
Noemi Erli
Brian Stack
Dustin J. Mitchell
Coroiu Cristina
Mike Hommey
Jim Chen
Brian Grinstead
Nathan Froyd
Mark Banner
Brindusan Cristian
Sylvestre Ledru
Emilio Cobos Álvarez
Margareta Eliza Balazs