This will get the js-bench tasks to run on physical hardware instead of AWS.
MozReview-Commit-ID: 44XavXAwxxn
--HG--
extra : rebase_source : ae1ba4c7f90b3a8526511a3f3c1dff913a334619
There is a superficial check in the run-task script which requires root. Simply
removing this check allows a native-engine task (which isn't running as root)
to proceed.
MozReview-Commit-ID: 44XavXAwxxn
--HG--
extra : rebase_source : bd1f01ce1c2feb4029838e07314493d449a4f46e
This adds an optional 'workdir' key to all job schemas. It still defaults to
/builds/worker, but can be overriden by individual tasks or schema
implementations.
MozReview-Commit-ID: LY20xfBhbCP
--HG--
extra : rebase_source : 7ac76ebf55d33d30c2aad73484421c6b4002cd33
Extends support of the use-artifacts key to native-engine based tasks.
MozReview-Commit-ID: FJILoyD5XVZ
--HG--
extra : rebase_source : 0cf8bf63f73d0fbb634f6b437bcc9bcce7821900
I've deliberately left as tier-3 the following tests:
- gtest (perma-OOM, likely from ASan malloc-meddling)
- xpcshell (builds need to be signed plus other failures too)
--HG--
extra : rebase_source : 812bf0de11e91c4e952cb5da9163241bd9386246
Summary:
the blocklist and remote-settings changes need to happen on beta,
but not the hsts/hpkp updates, so we have to split out the control of what
runs by project.
Reviewers: jlorenzo
Reviewed By: jlorenzo
Bug #: 1436369
Differential Revision: https://phabricator.services.mozilla.com/D1487
--HG--
extra : rebase_source : 19ccbb67b880ee7bd2dc2a37325dd70de635abad
for L10n jobs should run per-push based on the corresponding builds
Differential Revision: https://phabricator.services.mozilla.com/D1450
--HG--
extra : rebase_source : b2a6fe48ab031a3d7915bafe30fa8f603ec92d51
for L10n jobs should run per-push based on the corresponding builds
Differential Revision: https://phabricator.services.mozilla.com/D1449
--HG--
extra : rebase_source : f63e6c5a13904ba33fa2a46e05cfdd0997abd0fc
for L10n jobs should run per-push based on the corresponding builds
Differential Revision: https://phabricator.services.mozilla.com/D1409
--HG--
extra : rebase_source : 2798c5bc3e3153f8c293846d5a3d786e18bbdc34
for L10n jobs should run per-push based on the corresponding builds
Differential Revision: https://phabricator.services.mozilla.com/D1408
--HG--
extra : rebase_source : 6605d320082c767699e0c360cefa8a04e5525d10
for L10n jobs should run per-push based on the corresponding builds
Differential Revision: https://phabricator.services.mozilla.com/D1406
--HG--
extra : rebase_source : 207d1c25e37ab2619a09fb209282ffe55025de26
The crash reporter symbol files are the easiest cross-platform way to
find static initializers. While some types of static initializers (e.g.
__attribute__(constructor) functions) don't appear there in a notable
way, the static initializers we do care the most about for tracking do
(static initializers from C++ globals). As a matter of fact, there is
only a difference of 2 compared to the currently reported count of 125
on a linux64 build, so this is a good enough approximation. And allows
us to easily track the count on Android, OSX and Windows builds, which
we currently don't do.
The tricky part is that the symbol files are in
dist/crashreporter-symbols/$lib/$fileid/$lib.sym, and $fileid is hard to
figure out. There is a `fileid` tool in testing/tools, but it is a
target binary, meaning it's not available on cross builds (OSX,
Android).
So the simplest is just to gather the data while creating the symbol
files, which unfortunately requires to go through some hoops to make it
happen for just the files we care about.
--HG--
extra : rebase_source : 458fed1ffd6f9294eefef61f10ff7a284af0d986
Thunderbird releases need to look at comm-beta/comm-esr* branches for old
locale/version information.
Differential Revision: https://phabricator.services.mozilla.com/D1413
--HG--
extra : rebase_source : 76625ea5859d25f270b9fbec577f9075988bf2b7
PyYAML is not fast, so the fewer times we parse the same file, the better
MozReview-Commit-ID: KuYKFY7hFXp
--HG--
extra : rebase_source : 55df7c515db8864ee6d01895d444f7f26229bc2f
When running jobs on try, we shouldn't have access to the partials, so
don't try to give them.
Differential Revision: https://phabricator.services.mozilla.com/D1352
--HG--
extra : source : 90d19b52f956ce509525ca160ef5a2af80eea1a9
extra : histedit_source : b303ac9a4117a4e8e047d6459a7ba6477b74b762
For kind=hook, the spec doesn't include this value as it's untrustworthy.
For kind=task, it's still untrustworthy, but there is no privilege escalation
so that's not important. Still, it dramatically expands the size of the task
definition.
MozReview-Commit-ID: 6scQ2ZwxP10
--HG--
extra : rebase_source : 4dc34390a510091ddc26023755992995fe358e47
In a post-actions-as-hooks world, users will not have scopes to create tasks,
so this mode of action definition will not be possible. This is not currently
used from Treeherder (it links to
https://tools.taskcluster.net/tasks/<taskid>/interactive instead)
This drops support for the JSON-e-only interactive action; that action is not
currently used from treeherder, so that should have no impact for users.
MozReview-Commit-ID: 9i3POpjahAc
--HG--
extra : rebase_source : e6de03389a0c5c67d5332d2b1c97e1d4bf6a22d3
support_vcs_checkout() always sets the environment variables that
were set by this deleted function. In addition, support_vcs_checkout()
also adds caches and scopes - at least for docker-worker and
docker-engine. For generic-worker - which was used in all call sites
of docker_worker_add_gecko_vcs_env_vars() (yes, the "docker_worker"
bit of the name was completely wrong - probably a legacy holdover) -
support_vcs_checkout() was *almost* exactly equivalent to
docker_worker_add_gecko_vcs_env_vars(). The only difference is that
support_vcs_checkout() adds the
secrets:get:project/taskcluster/gecko/hgfingerprint scope in addition
to setting the environment variables.
MozReview-Commit-ID: 8fl3u9be5fT
--HG--
extra : rebase_source : 0eec2f143f903a3fcc5502b60026f5d8061100ea
This functionality was implemented at least 3 times. Let's consolidate
it to a central function.
Returning multiple command strings is kind of funky. I preserved
existing behavior and mozharness jobs are the only ones printing the
forensic logging. We should probably move this logging into
robustcheckout so we don't need to involve taskgraph with this. But
that can be deferred to another day.
MozReview-Commit-ID: I2LglJvfI6
--HG--
extra : rebase_source : 7cb413694aee4e46a6522febe9daa4b73b5307ca
extra : source : 096d7d374af427ee950c7a550878781eebad4135
The geckoview-junit tests require the OSS audio backend for the Android
4.3 ARM emulator, but mochitests don't work well with the OSS audio
backend. Therefore, use a different config file for the geckoview-junit
tests.
MozReview-Commit-ID: 20tzjtVdTuB
The email address used for notification on try was templated,
but nothing actually evaluated the template. This applies the same
templating that applies to the message to any emails specified.
Differential Revision: https://phabricator.services.mozilla.com/D1297
--HG--
extra : source : dd32a78ddf2196436f2098b4bc8bd3dc5c77b526
extra : amend_source : 8eac858e2b658bb2d8c3dacabe4c7fa3c077d9bc
A try push converting run-task to Python 3 seemed to complete without
error.
Since it is annoying writing code that needs to work on both Python
2 and 3, let's require Python 3 and remove code for supporting Python 2.
We implement a version check enforcing Python 3.5+. This is because
we're supposed to be standardizing on 3.5+ everywhere. I want to
prevent accidental usage of older Python 3 versions.
MozReview-Commit-ID: 4vATLZ6Si2e
--HG--
extra : source : 94a9641c5a018cfe729ebe748e75a7c4373e4322
Mostly normalization of str and bytes. Python 3 is annoying for
systems level code where most things are bytes.
MozReview-Commit-ID: KpvZGegBkYn
--HG--
extra : source : 4902cab3ce5dab2d1756cf0cd5c95f40603c0a0e
This required a lot of attention to bytes versus strings.
The hacks around handling process output are somewhat gross. Apparently
readline() doesn't work on bytes streams in Python 3?! So we install a
custom stream decoder so we can have nice things.
There are still some failures in run-task on Python 3. But we're a big
step closer.
MozReview-Commit-ID: 4FJlTn3q9Ai
--HG--
extra : source : 19fe5702cf6d018b743108b35e86d1750f205a76
The file failed to compile due to octal syntax and missing imports.
After this change, we get a run-time error, which is strictly better.
MozReview-Commit-ID: nY9A13Pt3E
--HG--
extra : source : ef477a048b575958be74287a2273830813b385f1
Our normal ubuntu 16.04 test image is suitable for hosting an Android x86
emulator with these minor updates: Install kvm and make sure /dev/kvm
rw permissions are open for everyone. Note that /dev/kvm is generally
only visible when running docker with --privileged; its permissions
cannot be modified in the Dockerfile, only at run-time: run-task is the
first opportunity.
download-and-compress isn't very complicated and should work on Python 3
with minimal effort. So let's switch it to use Python 3.
MozReview-Commit-ID: 9G1WfcbbKEY
--HG--
extra : rebase_source : 3a6bab06c8500a90413e8b7642a7bf7bdff04a46
python-zstandard 0.9 has an API that exposes a file object interface
for compression and decompression. This means we can remove our
stream wrapper in order to consume a zstandard compressed tar file.
MozReview-Commit-ID: DeWWKnigJVa
--HG--
extra : rebase_source : b510b9c7cf4471df835c755299a7842d13188b67
The latest python-zstandard uses a newer zstandard that is faster.
It also has wheels available, which means installation doesn't require
Python development headers, etc.
MozReview-Commit-ID: 5gRq81KYmX4
--HG--
extra : rebase_source : 96ccc64e9707c6b4815c1bfa5c1a98b9a428b387
Version 0.9.0 bundles a newer version of the zstandard library, which
is a little faster and has a few minor bug fixes (none that we were
likely hitting, however).
MozReview-Commit-ID: 9YgSZ0G41eg
--HG--
extra : rebase_source : 8f5a68323b1e1fe7e9f1dd1a92e132434972d21d
We want Python 3 available everywhere because it is 2018.
MozReview-Commit-ID: L3wufNXKdnp
--HG--
extra : rebase_source : c260923e3c13f8b28e30eaaf6e1bd38f79500052
Address Sanitizer builds use --disable-crashreporter so they don't have
symbols zip files to upload. Don't run symbol upload tasks for these builds.
MozReview-Commit-ID: GeQgRZF3m8t
--HG--
extra : rebase_source : 6ac59c5c96b5fb5ddbbe8c60af3a203d02ea9883
Mainly so searching "toolchain" + "gcc" yields something useful in the
taskgraph.
MozReview-Commit-ID: HWiT3AwwYQ2
--HG--
extra : rebase_source : b1ba0dfb4f99b6f8abe42506e8b37db68ed03590
Instead of having a special test-set for linux64-qr we can just use the
regular test set, and explicitly disable the individual tests that are
failing.
MozReview-Commit-ID: 8MUj1YdtOsH
--HG--
extra : rebase_source : 5b4398ccedd208c97fe2c58024d98bfdb759c932
Instead of having a special test-set for linux64-qr we can just use the
regular test set, and explicitly disable the individual tests that are
failing.
MozReview-Commit-ID: 8MUj1YdtOsH
--HG--
extra : rebase_source : 4d38dd3ea7a6934c84e57d6a20c7dc457f06c2da
Note in particular that tasks that were previously set to run on just
['mozilla-central', 'try'] will now also run on inbound and autoland, in
addition to mozilla-beta and other release branches. In some cases (e.g.
for talos tests) this might result in a significant increase in load on
CI infrastructure. For the tasks that were already running on ['trunk',
'try'] the extra load from the release branches should be relatively
small and will only take effect once 62 moves off nightly into beta.
MozReview-Commit-ID: 6sn9q6rCxOK
--HG--
extra : rebase_source : c228adf059a950aec3e311ae11915caf345e854f
Mainly so searching "toolchain" + "gcc" yields something useful in the
taskgraph.
MozReview-Commit-ID: HWiT3AwwYQ2
--HG--
extra : rebase_source : b1ba0dfb4f99b6f8abe42506e8b37db68ed03590
Summary:
In essence, we're allowing a new field in the task definition,
which is trusted over the one that's passed in with the config. This
wouldn't make much sense if it had a real date in, but allows us to
set an empty string for the kind that needs it
Reviewers: bhearsum
Reviewed By: bhearsum
Bug #: 1458854
Differential Revision: https://phabricator.services.mozilla.com/D1214
--HG--
extra : rebase_source : 82145a94fa91957ffe57112a1c0d327d99e32b23
This Dockerfile downloads non-deterministic remote content (by cloning a
Git repo) and then executes code from it. Part of that code is
executing Python package installs.
Since this Docker image was generated, it appears the remote code
requires new build dependencies. This commit adds those package
dependencies.
Not having deterministic Docker image builds is a bug. I'll file a
follow-up so we pin the Git commit used for building so this type
of failure doesn't occur again.
--HG--
extra : amend_source : 533a95abeb7cf7ddc9a1329549f5d294baf983f5
Johan Lorenzo
Tom Prince
Marco Castelluccio
Cosmin Sabou
Andrew Halberstadt
Chris AtLee
David Major
Simon Fraser
arthur.iakab
Justin Wood
Csoregi Natalia
Mathieu Leplatre
Mike Hommey
Nick Thomas
Ciure Andrei
Kartikaya Gupta
Joel Maher
Dorel Luca
Coroiu Cristina
Bogdan Tara
Brindusan Cristian
Aki Sasaki
Mike Shal
Dustin J. Mitchell
Margareta Eliza Balazs
Andreea Pavel
Ahilya Sinha
Sylvestre Ledru
Gregory Szorc
Raymond FOrbes
Sebastian Hengst
Jim Chen
Steve Fink
Geoff Brown
Andi-Bogdan Postelnicu
Ted Mielczarek
Maja Frydrychowicz
Narcis Beleuzu
Tom Ritter