Bug 1382564 made the `mach artifact toolchain` invocations from
mozharness use the MOZ_TOOLCHAINS environment variable when it's set by
taskcluster through the decision task, so that toolchain dependencies
from the task graph are used, but the mozharness code is still skipping
mach artifact toolchain when MOZ_TOOLCHAINS is set but there is no
tooltool manifest set. Most jobs today still have a tooltool manifest
set, but jobs shouldn't need a dummy tooltool manifest to use toolchain
dependencies automatically.
--HG--
extra : rebase_source : 0437a8f3d43a83ffe32c4192f86ee9a621977e3e
We need mozharness configurations and mozconfigs for rusttests. We are
explicitly not doing Windows debug configurations currently because of
peculiar link errors in such configurations.
This patch:
- removes the obsolete mozilla-aurora l10n-bumper config.
- adds both central and beta format desktop bumper configs to jamun for testing.
- updates the central and beta configs to add desktop.
- updates the script to support the desktop configs.
We now support an `ignore_config` which acts like the `ignore-platforms` attribute.
MozReview-Commit-ID: KGwo0bRibw4
--HG--
extra : rebase_source : 1014c8d46104fc3b05586aa64f207cf38f37f98f
The main bug this fixes is that on reftest, the objdir needs to be added to the
whitelist on Windows. However, this only happens when running on Linux for some
reason.
Changing the --work-path and --obj-path arguments to --sandbox-read-whitelist
was more of a drive-by cleanup than anything necessary.
MozReview-Commit-ID: Dq8ZLETMzeM
--HG--
extra : rebase_source : 3d2cdda125205e76f86235eb373074899fe0789a
This is to avoid hitting https://github.com/pypa/pip/issues/510.
MozReview-Commit-ID: 7TK4DdbpKRD
--HG--
extra : rebase_source : ffef899d80864a81a81a1a332a9da1f949a05551
This fixes Single Locale Repacks for Fennec 57.0 betas.
This broke because our merge scripts rewrote l10n-central to mozilla-beta expecting the l10n repo to be called 'mozilla-beta', which is no longer true with cross-channel l10n. When we patched the configs we didn't see a 'mozilla-beta' entry here, butmissed that the merge-day scripts change it on us.
MozReview-Commit-ID: F7BJzpZg0Xj
--HG--
extra : source : cc652dcb13dba40ae3f263ae89ce3e610a34165f
extra : amend_source : 9fa72e4b2a662a6a63ddfc59dbacf7f03e57dd67
Complications:
- had to copy ReftestManifest into a test zip
- reftest harness was emitting multiple suite_start log entries with --repeat
- some extra path manipulation required to find reftests
This uploads a JSON summary of the results, without extra logging or
expectation data or anything. It is mostly useful for comparing the
results from two runs e.g. in a dashboard.
MozReview-Commit-ID: Ac45NVBxhy8
--HG--
extra : rebase_source : 2c7a82d6e46b8c9f7af8ad34559e51d84ba4e4ba
Currently only the Mochitests on Linux use a startup_timeout of 180s
for Marionette. This would also be necessary for the other platforms
(MacOS and Windows).
MozReview-Commit-ID: TGO7XDLXdI
--HG--
extra : rebase_source : 29219cc51404cebf22e894966294eae687bb5d68
This change rides the train along with the change to build beta from
l10n-central.
Not changing 52 esr config, as that continues to build from the
repository it's currently building from.
ESR will be picked up when the next ESR is riding the trains from central.
MozReview-Commit-ID: F9Wo2mcZ3PA
--HG--
extra : rebase_source : dbf6c39cc5ab51bf4970caa4d62a80060b6b8c7c
Notable changes
* ensure we run dump-symbols and upload actions on all platforms
* On android:
* add configuration and support for aarch64
* set min_sdk levels to match Fennec builds
* use a full copy of the r11c ndk (our truncated one was missing toolchains we needed) and set NDKROOT when calling build
* ensure the tooltool provided sdk is on the PATH
* on linux copy tooltool.py into the mock environment, so we can get dump_syms from tt
* remove macosx32 config as we've deprecated that in Firefox builds
* update dump_syms to recent m-c, notably for aarch64 support on linux
* on linux rev e365137fa61bfd729617ba1ebf9f1ed79facd1f2 (via try 0f72a5c28be1cdc2f3bdfaafdf3826254f6ba077)
* on mac rev e365137fa61bfd729617ba1ebf9f1ed79facd1f2 (via compile on a bld-lin-r5)
* on windows rev a4a448ba7f187069fce916ee234a06cbb0d06f80 (via try dc8b121e3c08e8022d62c0fa1951dd3dc4d6f7cc)
* switch to Visual Studio 2015 Update 3 on win32/win64 to match Firefox
* many updates to environement variables
* painful to get win64 right to run win32 dump_syms.exe, but that's why the x86 redist is on teh PATH
* unwind the changes to get_output_from_command() in v1.6 patch to avoid affecting other builds, and use query_env() which has this support already
* add a scp_upload_directory since we don't have rsync on windows, use that to talk to the ffxbld upload host (not a long term solution but OK for now)
Applies on top of https://reviewboard.mozilla.org/r/64022/diff/4#index_header
MozReview-Commit-ID: B3NiWFvr2oR
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: 554IPMRWgzK
--HG--
extra : rebase_source : 00827d3f6bd705419bc801eb05b543af1ddc274f
From version-control-tools revision
0451f128550d8c9dcf83d6628dea5a8030d5cc08. Vendored without
modifications.
This should be a rubber stamp review since the changes to
robustcheckout were already reviewed.
MozReview-Commit-ID: 3J0db7Cja1D
--HG--
extra : rebase_source : 4e36dc83a486841d253cd365ce7a6ceeb50a8ccc
On Taskcluster machines, /tools/buildbot doesn't exist. It turns out, nothing
refers to exes['python'] anyway, so we can just remove that (the preference is
to use sys.executable instead).
The references to exes['buildbot'] were all for sendchanges from builds to
tests, and those too are all gone now.
Lines like
'virtualenv': ['/tools/buildbot/bin/python', '/tools/misc-python/virtualenv.py'],
Were committing two sins: first, using a python executable from a random
virtualenv; and second, using a virtualenv.py from another random directory (in
this case, it's a utility script for a PuppetAgain module). Such lines are
replaced with a reference to /tools/virtualenv/bin/virtualenv, which is
installed for the express purpose of providing a virtualenv binary on testers
(for builds, we use the vendored copy).
MozReview-Commit-ID: 4iHX3B3MLyK
--HG--
extra : rebase_source : e4b4902a19d688b148c136bd10c706fc127cbf2d
The robustcheckout extension from revision
134574b64ddfa4d7c31977d792761cceca67665a of the version-control-tools
repo is vendored without modifications.
Changes since last time include printing of the Mercurial version and
more robust handling repositories not using modern storage
requirements.
This patch was not explicitly reviewed by glob. But glob reviewed all
the robustcheckout changes since the last vendor. So by the transitive
property of code review...
MozReview-Commit-ID: CejuVVGpaEy
--HG--
extra : rebase_source : d24dd19357c44f50f7605e974d91434bac3c47f8
extra : source : 68cdc2d1184dec80455ba0ea1ebbcab232b8c119
We use `hg init` to create the directory. Because this is what
typically occurs.
We also remove the disabling of "dotencode" in the hgrc. This was added
in https://hg.mozilla.org/build/mozharness/rev/b1dbc0f56ff8 (bug 857853)
for reasons that are unclear to me. We should never disable dotencode
because it may make some repositories not clonable on Windows
filesystems.
Disabling dotencode will also interfere with the latest version of
robustcheckout, which requires its presence.
MozReview-Commit-ID: 35qBsgwp0uW
--HG--
extra : rebase_source : b786fb38f6b09da9614ac40f5de8231b8305bf5d
We use Mercurial 4.3.1 pretty much everywhere in CI now.
Mozharness should be testing with it as well.
MozReview-Commit-ID: HT2rocEvdIe
--HG--
extra : rebase_source : 148a9cd82b18e693ee570f31fc961373e8466a3c
The robustcheckout extension from revision
134574b64ddfa4d7c31977d792761cceca67665a of the version-control-tools
repo is vendored without modifications.
Changes since last time include printing of the Mercurial version and
more robust handling repositories not using modern storage
requirements.
This patch was not explicitly reviewed by glob. But glob reviewed all
the robustcheckout changes since the last vendor. So by the transitive
property of code review...
--HG--
extra : rebase_source : f511876ecc743e6b453118297d327e9a07bb9441
History suggests these have been in place for a long time, back to an era when
we used split-horizon DNS and DNS service was flaky. We don't see this sort of
error very often anymore.
Now we have docker images that do not have nslookup installed, and this is
causing retries to fail.
The other option is to use `socket.getaddrinfo` here, but that introduces new
failure mode and could prevent retries that would otherwise succeed. The least
failure-prone approach is just to remove the calls.
MozReview-Commit-ID: JDjhYxUrROS
--HG--
extra : rebase_source : 2e29e9ca19343feea9c505622753cbac1d91012e
This will change all build symbols to 'Ba' and set the USE_ARTIFACTS=1 environment variable.
Mozharness will detect this env to decide whether to perform an artifact build or not.
MozReview-Commit-ID: J8HVZzOt4mX
--HG--
extra : rebase_source : 453028d9be5cb2ad07e9a2a8b769cb6aac9893fe
Other related changes:
* Only target.{zip,tar.gz,tar.bz} get the widevine signature (not the various installers).
* Linux{32,64}-nightly are now repackaged. Their mar files are not signed during signed step anymore. It now happens after the repackage.
* As a consequence, funsize routes for linux are now set to repackage-signing (instead of signing)
* Signed upstream artifacts are now defined in a dedicated module (to avoid duplication)
* Platforms defined in beetmover_repackage now allow regex (to reduce duplication too)
* Mozharness configs: Delete unused (and misleading) `src_mozconfig` for windows. This value is actually not used when `run_configure` (in the same dict) is set to False.
MozReview-Commit-ID: COKqevW9Mzn
--HG--
extra : source : ffc2e43aa834e05f0d51d68dfb36317c1b408b08
From revision 19db5f4b5b10f639d24e69a4f21e4e80c6e5bbdd of
version-control-tools. Previous revision was
249a47720ddcf896a9f07600c429a1b4492b805e.
Changes include:
* Use new vfs APIs when available (bug 1359959)
* Mark as compatible with Mercurial 4.2 (bug 1361182)
* Retry after SSLError (bug 1354824)
* Mark as compatible with Mercurial 4.3
* Detect and recover from open locks (bug 1297153)
The most significant is the last one. TaskCluster can SIGKILL
`hg`, leading to orphaned transactions and locks. This can lead
to timeouts or use of a corrupt repository in some scenarios.
Those problems should no longer occur.
MozReview-Commit-ID: QAOSLyc0xD
--HG--
extra : rebase_source : c241626a5a59391b80baf34d50e90fc543e636f7
Since the buildbot-based Windows builds using releng.manifest are busted
anyways, there is no reason to keep clang entries in there. Which makes
those manifests identical to clang.manifest, so remote the latter.
--HG--
extra : rebase_source : eef7eca4bafc4e348eadc04d6da2bd17ea20deea
Currently manifestparser will only look for line continuations *after* looking for a key. This means
that line continuations cannot contain key separators. For example, this:
[test]
foo=
bar=baz
gets treated as:
{'name': 'test', 'foo': '', 'bar': 'baz'}
Here, bar=baz will be treated as a new key/value pair despite the indentation. This patch switches
the order around, so we look for a continuation first. Now, it is only treated as a continuation if
the indent is greater than the indent of the preceding key.
So this manifest:
[test]
foo=bar
baz=fleem
is a continuation and results in:
{'name': 'test', 'foo': 'bar\nbaz=fleem'}
But this manifest:
[test]
foo=bar
baz=fleem
is not a continuation, and yields:
{'name': 'test', 'foo': 'bar', 'baz': 'fleem'}
MozReview-Commit-ID: FAMP5TUIo9q
--HG--
extra : rebase_source : 624c53cfe0565374c1224dd86a3fffc8831279d3
Bug 1374748 removed Stylo-specific builds, but there are still a few config
files left behind that are now unused. This cleans them up.
MozReview-Commit-ID: EAUx7YKQBmN
--HG--
extra : rebase_source : 7e2124f7e5625d25efc5e868e151dbdc02cfba65
Ahem.ttf is copied to $(DIST)/bin/firefox/fonts, but this file is broken due to text mode copy. So we should use binary mode instead.
MozReview-Commit-ID: KP7yNyPiejU
--HG--
extra : rebase_source : 2de749f458a6d4650f9044f1912ff97835c5b795
mozharness used to call compare-locales with obj-l10n as MOZ_OBJDIR, while
calling configure and installers-% with obj-firefox.
Just make all the configs use the same objdir, instead of diving in
to mozharness.
MozReview-Commit-ID: 2CXTxVfYP82
--HG--
extra : rebase_source : 6a24927ef22701598f6807c4b7e0636172642655
The page load timeout of Marionette is 300s long. By using the same value
for the firefox ui harness invocation from mozharness it can cause a race
condition. Lets increase the value to what the Marionette jobs use to be
in sync.
MozReview-Commit-ID: FUevLmRIMSF
--HG--
extra : rebase_source : 43ee8b9009ba4ea89c8e8642d44eecf643b05ecc
Enable Stylo tests for win32, win64, and macosx64. Talos will be considered
separately.
MozReview-Commit-ID: BvTkddDX2Kk
--HG--
extra : rebase_source : 8d4df4469d4017b12bfbd33a98cc9f38111aa210
Flip around the logic of bug 1356122, so that the default from Stylo runs is the
parallel traversal, but we can opt in to single traversal as desired.
This ensures that for testing on other desktop platforms, we use parallel
traversal as the default.
MozReview-Commit-ID: KoBe1ltHP52
--HG--
extra : rebase_source : ad96f24c9760544c5812060f26e9ca18e5bf2ba8
This allows syntax like --try-test-paths web-platform-tests-reftests:path/to/test
MozReview-Commit-ID: uAet1ilPVy
--HG--
extra : rebase_source : 447277e47701435186ad87dfc089bd21f2bd1907
for: Land changes to flip windows nightlies on, and to tier 1 on central
MozReview-Commit-ID: H5Ysplr4NV7
--HG--
extra : rebase_source : 57b32b09d29ec3ced716dcd8e55bfa3bcb7c4475
Bug 1374940 adds a MOZ_TOOLCHAINS environment variable with a list of
path@task-id strings, where task-id is corresponding to the (possibly
optimized) toolchain job, and path corresponding to the
toolchain-artifact defined for that toolchain job.
We want to use that to pull artifacts instead of tooltool packages.
--HG--
extra : rebase_source : 277daa2c83d6d197975cb4ef36ee131176afa992
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: DUg6pXQ8OiI
--HG--
extra : rebase_source : cf91a7ef950d706daaf4ccc8d5631cc24156c7ff
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: 5un10JBaKrH
--HG--
extra : rebase_source : 45f61200a794f465637da657c0fa6e936a81bc09
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: 4Nxhbutjhfu
--HG--
extra : rebase_source : 2a180738584d6de77fe1b8fe81ecb004582000c5
Land date changes to support windows nightlies onto central
This patch also allows us to not reconfigure to run repackage on windows, which saves us both in potential pain points and execution time.
MozReview-Commit-ID: xnz98Z5N06
--HG--
extra : rebase_source : 1630a3b6b646a83abeb05000e77e0e3c3238250b
Land date changes to support windows nightlies onto central
This will be needed for windows, because if the artifact is output to a folder that is, itself, an artifact folder, we'll end up publishing the named artifact twice. Which is unhelpful.
MozReview-Commit-ID: 5S6SNul6Fm9
--HG--
extra : rebase_source : 5501d8d757c7408c9dd014bc04a8d36429bdd9b5
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: Jwk7c2W1QP7
--HG--
extra : rebase_source : c6f68a8b3621b3a743d1f1eb4a6857d23dd801b7
Rail Aliiev
Tom Ritter
Wes Kocher
Aki Sasaki
Mike Hommey
Nathan Froyd
Andrew Halberstadt
Tom Prince
Geoff Brown
Justin Wood
Kim Moir
Rob Wood
Joel Maher
James Graham
Henrik Skupin
Axel Hecht
Nick Thomas
Christian Holler
Johan Lorenzo
Kris Maglione
Cosm
Ganti Sai Sarath Chandra
J. Ryan Stinnett
Wander Lairson Costa
Rok Garbas
Gregory Szorc
Dustin J. Mitchell
Chris Manchester
Carsten "Tomcat" Book
Kartikaya Gupta
Mihai Tabara
Xidorn Quan
Chenxia Liu
Makoto Kato
Bob Clary
Gian-Carlo Pascutto
William Lachance
Marco Castelluccio
Chris AtLee
Maja Frydrychowicz
Francois Marier
Sebastian Hengst