This uploads a JSON summary of the results, without extra logging or
expectation data or anything. It is mostly useful for comparing the
results from two runs e.g. in a dashboard.
MozReview-Commit-ID: Ac45NVBxhy8
--HG--
extra : rebase_source : 2c7a82d6e46b8c9f7af8ad34559e51d84ba4e4ba
Currently only the Mochitests on Linux use a startup_timeout of 180s
for Marionette. This would also be necessary for the other platforms
(MacOS and Windows).
MozReview-Commit-ID: TGO7XDLXdI
--HG--
extra : rebase_source : 29219cc51404cebf22e894966294eae687bb5d68
This change rides the train along with the change to build beta from
l10n-central.
Not changing 52 esr config, as that continues to build from the
repository it's currently building from.
ESR will be picked up when the next ESR is riding the trains from central.
MozReview-Commit-ID: F9Wo2mcZ3PA
--HG--
extra : rebase_source : dbf6c39cc5ab51bf4970caa4d62a80060b6b8c7c
Notable changes
* ensure we run dump-symbols and upload actions on all platforms
* On android:
* add configuration and support for aarch64
* set min_sdk levels to match Fennec builds
* use a full copy of the r11c ndk (our truncated one was missing toolchains we needed) and set NDKROOT when calling build
* ensure the tooltool provided sdk is on the PATH
* on linux copy tooltool.py into the mock environment, so we can get dump_syms from tt
* remove macosx32 config as we've deprecated that in Firefox builds
* update dump_syms to recent m-c, notably for aarch64 support on linux
* on linux rev e365137fa61bfd729617ba1ebf9f1ed79facd1f2 (via try 0f72a5c28be1cdc2f3bdfaafdf3826254f6ba077)
* on mac rev e365137fa61bfd729617ba1ebf9f1ed79facd1f2 (via compile on a bld-lin-r5)
* on windows rev a4a448ba7f187069fce916ee234a06cbb0d06f80 (via try dc8b121e3c08e8022d62c0fa1951dd3dc4d6f7cc)
* switch to Visual Studio 2015 Update 3 on win32/win64 to match Firefox
* many updates to environement variables
* painful to get win64 right to run win32 dump_syms.exe, but that's why the x86 redist is on teh PATH
* unwind the changes to get_output_from_command() in v1.6 patch to avoid affecting other builds, and use query_env() which has this support already
* add a scp_upload_directory since we don't have rsync on windows, use that to talk to the ffxbld upload host (not a long term solution but OK for now)
Applies on top of https://reviewboard.mozilla.org/r/64022/diff/4#index_header
MozReview-Commit-ID: B3NiWFvr2oR
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: 554IPMRWgzK
--HG--
extra : rebase_source : 00827d3f6bd705419bc801eb05b543af1ddc274f
From version-control-tools revision
0451f128550d8c9dcf83d6628dea5a8030d5cc08. Vendored without
modifications.
This should be a rubber stamp review since the changes to
robustcheckout were already reviewed.
MozReview-Commit-ID: 3J0db7Cja1D
--HG--
extra : rebase_source : 4e36dc83a486841d253cd365ce7a6ceeb50a8ccc
On Taskcluster machines, /tools/buildbot doesn't exist. It turns out, nothing
refers to exes['python'] anyway, so we can just remove that (the preference is
to use sys.executable instead).
The references to exes['buildbot'] were all for sendchanges from builds to
tests, and those too are all gone now.
Lines like
'virtualenv': ['/tools/buildbot/bin/python', '/tools/misc-python/virtualenv.py'],
Were committing two sins: first, using a python executable from a random
virtualenv; and second, using a virtualenv.py from another random directory (in
this case, it's a utility script for a PuppetAgain module). Such lines are
replaced with a reference to /tools/virtualenv/bin/virtualenv, which is
installed for the express purpose of providing a virtualenv binary on testers
(for builds, we use the vendored copy).
MozReview-Commit-ID: 4iHX3B3MLyK
--HG--
extra : rebase_source : e4b4902a19d688b148c136bd10c706fc127cbf2d
The robustcheckout extension from revision
134574b64ddfa4d7c31977d792761cceca67665a of the version-control-tools
repo is vendored without modifications.
Changes since last time include printing of the Mercurial version and
more robust handling repositories not using modern storage
requirements.
This patch was not explicitly reviewed by glob. But glob reviewed all
the robustcheckout changes since the last vendor. So by the transitive
property of code review...
MozReview-Commit-ID: CejuVVGpaEy
--HG--
extra : rebase_source : d24dd19357c44f50f7605e974d91434bac3c47f8
extra : source : 68cdc2d1184dec80455ba0ea1ebbcab232b8c119
We use `hg init` to create the directory. Because this is what
typically occurs.
We also remove the disabling of "dotencode" in the hgrc. This was added
in https://hg.mozilla.org/build/mozharness/rev/b1dbc0f56ff8 (bug 857853)
for reasons that are unclear to me. We should never disable dotencode
because it may make some repositories not clonable on Windows
filesystems.
Disabling dotencode will also interfere with the latest version of
robustcheckout, which requires its presence.
MozReview-Commit-ID: 35qBsgwp0uW
--HG--
extra : rebase_source : b786fb38f6b09da9614ac40f5de8231b8305bf5d
We use Mercurial 4.3.1 pretty much everywhere in CI now.
Mozharness should be testing with it as well.
MozReview-Commit-ID: HT2rocEvdIe
--HG--
extra : rebase_source : 148a9cd82b18e693ee570f31fc961373e8466a3c
The robustcheckout extension from revision
134574b64ddfa4d7c31977d792761cceca67665a of the version-control-tools
repo is vendored without modifications.
Changes since last time include printing of the Mercurial version and
more robust handling repositories not using modern storage
requirements.
This patch was not explicitly reviewed by glob. But glob reviewed all
the robustcheckout changes since the last vendor. So by the transitive
property of code review...
--HG--
extra : rebase_source : f511876ecc743e6b453118297d327e9a07bb9441
History suggests these have been in place for a long time, back to an era when
we used split-horizon DNS and DNS service was flaky. We don't see this sort of
error very often anymore.
Now we have docker images that do not have nslookup installed, and this is
causing retries to fail.
The other option is to use `socket.getaddrinfo` here, but that introduces new
failure mode and could prevent retries that would otherwise succeed. The least
failure-prone approach is just to remove the calls.
MozReview-Commit-ID: JDjhYxUrROS
--HG--
extra : rebase_source : 2e29e9ca19343feea9c505622753cbac1d91012e
This will change all build symbols to 'Ba' and set the USE_ARTIFACTS=1 environment variable.
Mozharness will detect this env to decide whether to perform an artifact build or not.
MozReview-Commit-ID: J8HVZzOt4mX
--HG--
extra : rebase_source : 453028d9be5cb2ad07e9a2a8b769cb6aac9893fe
Other related changes:
* Only target.{zip,tar.gz,tar.bz} get the widevine signature (not the various installers).
* Linux{32,64}-nightly are now repackaged. Their mar files are not signed during signed step anymore. It now happens after the repackage.
* As a consequence, funsize routes for linux are now set to repackage-signing (instead of signing)
* Signed upstream artifacts are now defined in a dedicated module (to avoid duplication)
* Platforms defined in beetmover_repackage now allow regex (to reduce duplication too)
* Mozharness configs: Delete unused (and misleading) `src_mozconfig` for windows. This value is actually not used when `run_configure` (in the same dict) is set to False.
MozReview-Commit-ID: COKqevW9Mzn
--HG--
extra : source : ffc2e43aa834e05f0d51d68dfb36317c1b408b08
From revision 19db5f4b5b10f639d24e69a4f21e4e80c6e5bbdd of
version-control-tools. Previous revision was
249a47720ddcf896a9f07600c429a1b4492b805e.
Changes include:
* Use new vfs APIs when available (bug 1359959)
* Mark as compatible with Mercurial 4.2 (bug 1361182)
* Retry after SSLError (bug 1354824)
* Mark as compatible with Mercurial 4.3
* Detect and recover from open locks (bug 1297153)
The most significant is the last one. TaskCluster can SIGKILL
`hg`, leading to orphaned transactions and locks. This can lead
to timeouts or use of a corrupt repository in some scenarios.
Those problems should no longer occur.
MozReview-Commit-ID: QAOSLyc0xD
--HG--
extra : rebase_source : c241626a5a59391b80baf34d50e90fc543e636f7
Since the buildbot-based Windows builds using releng.manifest are busted
anyways, there is no reason to keep clang entries in there. Which makes
those manifests identical to clang.manifest, so remote the latter.
--HG--
extra : rebase_source : eef7eca4bafc4e348eadc04d6da2bd17ea20deea
Currently manifestparser will only look for line continuations *after* looking for a key. This means
that line continuations cannot contain key separators. For example, this:
[test]
foo=
bar=baz
gets treated as:
{'name': 'test', 'foo': '', 'bar': 'baz'}
Here, bar=baz will be treated as a new key/value pair despite the indentation. This patch switches
the order around, so we look for a continuation first. Now, it is only treated as a continuation if
the indent is greater than the indent of the preceding key.
So this manifest:
[test]
foo=bar
baz=fleem
is a continuation and results in:
{'name': 'test', 'foo': 'bar\nbaz=fleem'}
But this manifest:
[test]
foo=bar
baz=fleem
is not a continuation, and yields:
{'name': 'test', 'foo': 'bar', 'baz': 'fleem'}
MozReview-Commit-ID: FAMP5TUIo9q
--HG--
extra : rebase_source : 624c53cfe0565374c1224dd86a3fffc8831279d3
Bug 1374748 removed Stylo-specific builds, but there are still a few config
files left behind that are now unused. This cleans them up.
MozReview-Commit-ID: EAUx7YKQBmN
--HG--
extra : rebase_source : 7e2124f7e5625d25efc5e868e151dbdc02cfba65
Ahem.ttf is copied to $(DIST)/bin/firefox/fonts, but this file is broken due to text mode copy. So we should use binary mode instead.
MozReview-Commit-ID: KP7yNyPiejU
--HG--
extra : rebase_source : 2de749f458a6d4650f9044f1912ff97835c5b795
mozharness used to call compare-locales with obj-l10n as MOZ_OBJDIR, while
calling configure and installers-% with obj-firefox.
Just make all the configs use the same objdir, instead of diving in
to mozharness.
MozReview-Commit-ID: 2CXTxVfYP82
--HG--
extra : rebase_source : 6a24927ef22701598f6807c4b7e0636172642655
The page load timeout of Marionette is 300s long. By using the same value
for the firefox ui harness invocation from mozharness it can cause a race
condition. Lets increase the value to what the Marionette jobs use to be
in sync.
MozReview-Commit-ID: FUevLmRIMSF
--HG--
extra : rebase_source : 43ee8b9009ba4ea89c8e8642d44eecf643b05ecc
Enable Stylo tests for win32, win64, and macosx64. Talos will be considered
separately.
MozReview-Commit-ID: BvTkddDX2Kk
--HG--
extra : rebase_source : 8d4df4469d4017b12bfbd33a98cc9f38111aa210
Flip around the logic of bug 1356122, so that the default from Stylo runs is the
parallel traversal, but we can opt in to single traversal as desired.
This ensures that for testing on other desktop platforms, we use parallel
traversal as the default.
MozReview-Commit-ID: KoBe1ltHP52
--HG--
extra : rebase_source : ad96f24c9760544c5812060f26e9ca18e5bf2ba8
This allows syntax like --try-test-paths web-platform-tests-reftests:path/to/test
MozReview-Commit-ID: uAet1ilPVy
--HG--
extra : rebase_source : 447277e47701435186ad87dfc089bd21f2bd1907
for: Land changes to flip windows nightlies on, and to tier 1 on central
MozReview-Commit-ID: H5Ysplr4NV7
--HG--
extra : rebase_source : 57b32b09d29ec3ced716dcd8e55bfa3bcb7c4475
Bug 1374940 adds a MOZ_TOOLCHAINS environment variable with a list of
path@task-id strings, where task-id is corresponding to the (possibly
optimized) toolchain job, and path corresponding to the
toolchain-artifact defined for that toolchain job.
We want to use that to pull artifacts instead of tooltool packages.
--HG--
extra : rebase_source : 277daa2c83d6d197975cb4ef36ee131176afa992
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: DUg6pXQ8OiI
--HG--
extra : rebase_source : cf91a7ef950d706daaf4ccc8d5631cc24156c7ff
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: 5un10JBaKrH
--HG--
extra : rebase_source : 45f61200a794f465637da657c0fa6e936a81bc09
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: 4Nxhbutjhfu
--HG--
extra : rebase_source : 2a180738584d6de77fe1b8fe81ecb004582000c5
Land date changes to support windows nightlies onto central
This patch also allows us to not reconfigure to run repackage on windows, which saves us both in potential pain points and execution time.
MozReview-Commit-ID: xnz98Z5N06
--HG--
extra : rebase_source : 1630a3b6b646a83abeb05000e77e0e3c3238250b
Land date changes to support windows nightlies onto central
This will be needed for windows, because if the artifact is output to a folder that is, itself, an artifact folder, we'll end up publishing the named artifact twice. Which is unhelpful.
MozReview-Commit-ID: 5S6SNul6Fm9
--HG--
extra : rebase_source : 5501d8d757c7408c9dd014bc04a8d36429bdd9b5
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: Jwk7c2W1QP7
--HG--
extra : rebase_source : c6f68a8b3621b3a743d1f1eb4a6857d23dd801b7
Added cli_script attribute to TelemetryClientTests due to test failure
Added telemetry test requirements file to /testing/config
Added mozharness script to run telemetry tests from checkout
MozReview-Commit-ID: AJKM7b1OcVW
--HG--
extra : rebase_source : 8147ad3decaa94c28ba48b87310b4a00d5a90fd2
Added cli_script attribute to TelemetryClientTests due to test failure
Added telemetry test requirements file to /testing/config
Added mozharness script to run telemetry tests from checkout
MozReview-Commit-ID: AJKM7b1OcVW
--HG--
extra : rebase_source : 8147ad3decaa94c28ba48b87310b4a00d5a90fd2
Currently these are enabled for Linux and Windows, using the corresponding WPT
symbols with an "H" suffix to match the Mn->MnH convention followed by the
Marionette tests.
For Linux, the tests are set to run with a simulated 1600x1200 screen
resolution, which matches the Docker configuration. The default 1366x768 for
headless mode results in some unexpected failures due to windows not being
able to reach the full sizes the web platform tests expect.
For Windows, the simulated screen resolution is set to 1024x768, which is what
I've determined to be the resolution that Windows VMs run under within our
testing infrastructure. Setting a different screen resolution causes some web
platform tests to unexpectedly pass (in particular, several in
open-features-non-integer-height.html).
A "headless" variable is added to the environment that the WPT metadata
(.ini) files can access, indicating whether or not the test is running in
headless mode. This is used to mark the reftest bdi-neutral-wrapped.html as an
expected pass on Linux under headless mode: it normally fails (spuriously) due
to differences in scrollbar sizing, but scrollbars aren't rendered in headless
mode.
MozReview-Commit-ID: 3usazOoxx3q
--HG--
extra : rebase_source : c036070920391c30b13813e2d6282bd407a96715
Other related changes:
* Only target.{zip,tar.gz,tar.bz} get the widevine signature (not the various installers).
* Linux{32,64}-nightly are now repackaged. Their mar files are not signed during signed step anymore. It now happens after the repackage.
* As a consequence, funsize routes for linux are now set to repackage-signing (instead of signing)
* Signed upstream artifacts are now defined in a dedicated module (to avoid duplication)
* Platforms defined in beetmover_repackage now allow regex (to reduce duplication too)
* Mozharness configs: Delete unused (and misleading) `src_mozconfig` for windows. This value is actually not used when `run_configure` (in the same dict) is set to False.
MozReview-Commit-ID: COKqevW9Mzn
--HG--
extra : rebase_source : b9d18b73e288768c3e8f68c6a0b7059ebd73b166
Like we do for mochitests, set the MOZ_DEVELOPER_REPO_DIR and
MOZ_DEVELOPER_OBJ_DIR environment variables when running talos
tests so that the content sandbox permits access to test files
that are symlinks from the .app/ directory to the repo or
object directory.
MozReview-Commit-ID: J1bRqoB8a2i
--HG--
extra : rebase_source : 18f0bafa53dcd872d9bd647fc2e5ce143f5887bd
365731510976 (bug 1380391) added index routes for decision tasks by
pushlog id. This is a good idea. The pushlog id is guaranteed to always
be incrementing (except for repos that are periodically reset, which
we don't care about). It is useful to provide strict ordering for
pushes and is simpler for machines to consume and sort than dates.
So let's index all tasks by pushlog id.
MozReview-Commit-ID: BPqx4ARza1c
--HG--
extra : rebase_source : e952c636e5fcd2a0ddb5e14a0faaccb3761298bf
It be ideal to have |mach test {findbugs,test}| and |mach lint
{lint,checkstyle}|, but the |mach test| command is very difficult to
extend in a direction orthogonal to the existing direction. The
existing |mach test| is built around in-tree manifests, tagged and
divided into suites, intended to support |mach test
path/to/arbitrary/test|. The Android findbugs task is a global static
analysis that doesn't fit into the path/manifest model. The Android
test task is based on JUnit and not easy to build manifest support
for. The |mach lint| command is intended to be extended, but the
effort to extend it is non-trivial and not worth the effort (at this
time).
Therefore, I've taken the existing, little used |mach android| command
and added subcommands for use by local developers and automation. If
nothing else, this reduces the number of "special Gradle targets" --
the equivalent of "special Make targets" -- sprinkled throughout the
tree, which can only be a good thing!
MozReview-Commit-ID: 24b1vbgykpN
--HG--
extra : rebase_source : 8843a6e3840586fe05a1434484a848d48b2a6e8b
Wes Kocher
Joel Maher
Rob Wood
Tom Prince
James Graham
Henrik Skupin
Rail Aliiev
Axel Hecht
Geoff Brown
Nick Thomas
Christian Holler
Johan Lorenzo
Kris Maglione
Justin Wood
Cosm
Ganti Sai Sarath Chandra
J. Ryan Stinnett
Wander Lairson Costa
Aki Sasaki
Rok Garbas
Gregory Szorc
Dustin J. Mitchell
Chris Manchester
Andrew Halberstadt
Carsten "Tomcat" Book
Kartikaya Gupta
Mihai Tabara
Mike Hommey
Xidorn Quan
Chenxia Liu
Makoto Kato
Bob Clary
Gian-Carlo Pascutto
William Lachance
Marco Castelluccio
Chris AtLee
Maja Frydrychowicz
Francois Marier
Sebastian Hengst
Ryan VanderMeulen
John Dorlus
Michael Smith
Haik Aftandilian
Nick Alexander
Rob Thijssen