mozharness is Python. self.query_exe('python') could resolve to a
different Python interpreter from what mozharness is running as.
In order to promote consistency, always invoke python processes with
the Python being used to run mozharness.
In some cases, this may cause former `python` processes to run as
Python 2.7 instead of 2.6 (since `python` resolves to a 2.6 interpreter
on many systems). It may also result in slightly different Python
binaries being used. But I think sharing interpreters between the
mozharness script and launched processes is logical. So if this causes
problems, I'd like to flush those out.
MozReview-Commit-ID: KfawUvT5jgW
--HG--
extra : source : b6f04897fdda51e42612617a89a93f696edbdf92
extra : amend_source : 32dafc7c9dc2cec80bc289bd1a17cdbb8cde5025
Previously, mozharness defined a separate action to collect build
metrics. This required the script and/or config to define that
action.
Metrics collection for CI is important. So it should be enabled by
default.
This commit changes the "build" action/method to always call the
metrics collection function after successful build. References to
the "generate-build-stats" action have been removed because it is
redundant.
A side-effect of this change is we may generate build metrics where
we weren't before. This could lead to e.g. duplicate entries in some
Perfherder series. Let's see what breaks ;)
MozReview-Commit-ID: 42UQI5YQTMC
--HG--
extra : rebase_source : c57dc9ec6ac46003384edff098a0ad81c75539b7
extra : source : c9812dd7d27a174c0ee46d44ec595fbe29c9e1db
We're about to enable metrics collection for all builds. There are some
Android build configurations that use buildbase.py but don't create a
package. So we need a way to conditionally obtain package metrics.
We could change package metrics collection to no-op if a package file
can't be found. However, that has a risk that a future change could
break metrics collection and we wouldn't necessarily find out. I like
things that fail fast.
MozReview-Commit-ID: CzByf7yHVS8
--HG--
extra : rebase_source : 99ee18ed4dd61e5ea8f5eda1b810b573fe254158
A subsequent commit will make all this code conditional. Rather than
indent the world, it is easier to conditionally call a function.
A benefit of the new code is that we skip some code for debug builds,
which is one less thing that can break.
MozReview-Commit-ID: fiUNBbikmy
--HG--
extra : rebase_source : aeb151ea5864d0f97db20bee921b60afc00aee61
Previously, this ran during postflight_build(). The magic postflight_*
methods are called automagically by BaseScript.run_action() and are
only called if the main action method didn't raise. So there should
be no functional difference with this commit.
The reason I changed this is that a subsequent commit will perform
metrics generation from build() and without the build properties
file loaded, at least the OS X 64 opt buildbot build doesn't have
packageFilename defines, which breaks metrics collection.
MozReview-Commit-ID: 54ftuQqGKVi
--HG--
extra : rebase_source : c3c28426468474a7aa51a10787d01ebbba10dd82
extra : source : 387d8415d05e7f1dc96ed3adb441c54f232baf0d
mozharness is Python. self.query_exe('python') could resolve to a
different Python interpreter from what mozharness is running as.
In order to promote consistency, always invoke python processes with
the Python being used to run mozharness.
In some cases, this may cause former `python` processes to run as
Python 2.7 instead of 2.6 (since `python` resolves to a 2.6 interpreter
on many systems). It may also result in slightly different Python
binaries being used. But I think sharing interpreters between the
mozharness script and launched processes is logical. So if this causes
problems, I'd like to flush those out.
MozReview-Commit-ID: KfawUvT5jgW
--HG--
extra : rebase_source : 8babadc464ea4d8971e091d5446d86d2630e07b9
Servo reads the hosts from a file created at runtime, so this
configuration isn't required.
MozReview-Commit-ID: 20NoZyp3bJz
--HG--
extra : rebase_source : 955d15e9e43e1975fde856e5133fa2ff5786fec7
For Chrome and Edge we don't have any way to set the DNS configuration
to include web-platform.test, so we need to error if this isn't already set.
MozReview-Commit-ID: BHRsTiuV28x
--HG--
extra : rebase_source : cfd3c35a513f98b47a7ffc9328058e6d104d2b2e
This makes integration with other frontends a little easier.
MozReview-Commit-ID: 3gGeJqMPiZf
--HG--
extra : rebase_source : a9aa2a8d63cca80d3897149c2a0f7d3351031951
A number of developers find it convenient to build with
--disable-optimize --enable-debug for an improved debugging experience.
We don't currently have a configuration in CI that ensures this
combination of options works, so various changes break builds with this
configuration every so often. We should test such configurations to
ensure they build to provide a smooth experience for developers.
Before this patch, marionette.js starts its initialization right after receiving the "sessionstore-windows-restored" notification.
This is unfortunate for the browser_startup.js browser chrome mochitest added in bug 1358798, because it causes lots of marionette file to appear to be loaded before the browser is ready to handle user events.
This patch enables linux64-ccov to run JSVM code coverage collection at the same time GCOV code coverage is being collected. It uploads the code coverage that was created in a file called 'code-coverage-jsvm.zip'.
MozReview-Commit-ID: 4x5GrjRJRo4
--HG--
extra : rebase_source : 461eb6a0d6bf9f16a83899a1147c0f9cf2c02823
Testing :: geckodriver is a new Bugzilla component and it should be used
for the WPT wdspec tests located in testing/web-platform/tests/webdriver
because any test failures are likely to be implementation problems in
the testing/geckodriver program.
MozReview-Commit-ID: 75wG57PEsKl
--HG--
extra : rebase_source : 7d5c3bcb00cc00279ec30b5f4c90e2dd44540c3b
We regressed running entire directories when wptrunner was switched to
look up command parameters directly in the manifest rather than
iterating over prefix matches. For files that's OK, but for
directories the old behaviour is required.
MozReview-Commit-ID: HVL7rL1YuZx
--HG--
extra : rebase_source : d80150556ba5f07fb8c7ebbd7faab52ee63e2708
Several years ago there was a single zip file for all test files. Clients
would only extract the files they needed. Thus, zip was a reasonable
archive format because it allowed direct access to members without
having to decompress the entirety of the stream.
We have since split up that monolithic archive into separate,
domain-specific archives. e.g. 1 archive for mochitests and one
for xpcshell tests. This drastically cut down on network I/O
required on testers because they only fetched archives/data that
was relevant. It also enabled parallel generation of test archives,
we shaved dozens of seconds off builds due to compression being
a long pole.
Despite the architectural changes to test archive management, we
still used zip files. This is not ideal because we no longer access
specific files in test archives and thus don't care about single/partial
member access performance.
This commit implements support for generating tar.gz test archives.
And it switches the web-platform archive to a tar.gz file.
The performance implications for archive generation are significant:
before: 48,321,250 bytes; 6.05s
after: 31,844,267 bytes; 4.57s
The size is reduced because we have a single compression context
so data from 1 file can benefit compression in a subsequent file.
CPU usage is reduced because the compressor has to work less with
1 context than it does with N. While I didn't measure it, decompression
performance should also be improved for the same reasons. And of course
network I/O will be reduced.
mozharness consumers use a generic method for handling unarchiving.
This method automagically handles multiple file extensions. So as long
as downstream consumers aren't hard coding ".zip" this change should
"just work."
MozReview-Commit-ID: LQa5MIHLsms
--HG--
extra : rebase_source : 100092c2f2ff609362a724fff60f46dd6e49c94e
extra : intermediate-source : d10f5ccd882b965fcad39914f7c3c930d1301a41
extra : source : a0e257e346ccf3c1db332ec5903241f4eeb9a7ee
setTestName was used for logging which test was being run for the
JS Tests used in B2G.
MozReview-Commit-ID: FNF4Sm7vAYM
--HG--
extra : rebase_source : b12fd8ce04e7da739a8a5ec0e7b30b6734c58e4a
This is a remanent of the B2G code for injecting Mochitest style tests
into Gecko. This is no longer used by anything and is now dead code.
MozReview-Commit-ID: 4qaB3vxQzon
--HG--
extra : rebase_source : 3af2c7655ecd2d03f266d0a1ad02eadfea1b1a0b
setTestName was used for logging which test was being run for the
JS Tests used in B2G.
MozReview-Commit-ID: FNF4Sm7vAYM
--HG--
extra : rebase_source : 6ad739d2ff9bf3d6bafaed0450c8794a257657e7
This is a remanent of the B2G code for injecting Mochitest style tests
into Gecko. This is no longer used by anything and is now dead code.
MozReview-Commit-ID: 4qaB3vxQzon
--HG--
extra : rebase_source : b4b7e66452b7ce7335ef5f509957121f403d7043
This means they will be copied to $PROFILE/extensions, which the sandbox allows
access to; if they are installed as temporary addons, loading frame scripts in
the content process tries to read from wherever they happen to be on disk. This
breaks running tests with a packaged build once we have full read-restrictions
for the content process sandbox.
MozReview-Commit-ID: 7ZiiM9FMXfG
--HG--
extra : rebase_source : d2cf3a2d06df9099dc6056fae351200eaa1d0ca9
The main motivation behind this change is that going towards toolchain
tasks hooked up in the task graph (bug 1313111), we're going to end up
with jobs using both taskcluster toolchain job and tooltool artifacts
for their toolchain needs. With the current setup, this means the
toolchain dependencies will be spread between taskcluster task graph
definition and mozharness configuration.
It also makes things more complex to provide a command that pulls the
right toolchains from both taskcluster and tooltool (bug 1356529),
because one needs to find and parse the mozharness config (which also
happens to be python code that uses platform-specific things, so e.g.
reading windows mozharness config fails on other platforms).
All in all, moving the tooltool manifest path to the taskcluster task
definitions would make things simpler, and would also allow make patches
switching from tooltool to taskcluster artifacts more straightforward to
validate.
But since some build types still run on buildbot, we'll have to keep
part of the current setup using mozharness configs. So we allow to
override the tooltool manifest path from the environment, and we'll rely
on taskcluster task definitions being able to set environment variables.
Actually moving the relevant tooltool manifest paths from mozharness
config to taskcluster task definitions is left for a followup.
Another followup is to move the tooltool manifest paths declared in
some ad-hoc build scripts to taskcluster task definitions as well.
The immediate need for this, though, is to allow to have duplicated jobs
that only differ in their tooltool manifest, without duplicating a
complete mozharness config that will end up stale (the goal being that
really only the tooltool manifest differs, even when the original jobs
change).
--HG--
extra : rebase_source : 3622779926b1b5e86e809c1f6422bd55ef64eed7
Using the canonicalized path to start Firefox breaks the
browser on Windows because the "\\?\" prefix is not supported
yet. As result all components which rely on XCurProcD for
file handling are throwing JS errors, and do not initialize
correctly.
MozReview-Commit-ID: 5MWhDf1HCWf
--HG--
extra : rebase_source : da97482894eda970b6e6610e7462b927b57fb3a1
Gregory Szorc
Wes Kocher
James Graham
William Lachance
Ryan VanderMeulen
Sebastian Hengst
Jessica Jong
Dão Gottwald
Jan de Mooij
Joel Maher
Nathan Froyd
Florian Quèze
Geoff Brown
Greg Mierzwinski
Andreas Tolfsen
Marco Castelluccio
David Burns
Alex Gaynor
Tobias Schneider
Mike Hommey
Henrik Skupin
Carsten "Tomcat" Book