bc's original patch, but restricted to py3 (py2 already works - no need to change it!). I also updated
the six imports to suit my preference.
Differential Revision: https://phabricator.services.mozilla.com/D81561
The issue here is that the sort order of 2 or more elements with the same key
is indeterminate: One run (say, one test chunk) may order them one way and another
run (another test chunk) may order them another. The chunk_by_manifest() filter
relies on sort order in two places, in both cases using list length as the key
for sorting; as equal list lengths are not uncommon, multiple runs can assign
very different lists of tests to the chunks, even though all the inputs are the
same. Extending keys with unique data provides a simple fix.
I tried this before using the entire test dict as the second part of the sort key;
admittedly problematic. This version uses the test 'manifest', which is already
used earlier in the same function and should be unique.
Differential Revision: https://phabricator.services.mozilla.com/D79900
This should prevent unrelated tasks getting scheduled on try for
pushes with paths. The alternative would be to rename
web-platform-tests to web-platform-tests-testharness, but that seems
more likely to break other systems.
Differential Revision: https://phabricator.services.mozilla.com/D80079
This patch fixes the tests in testing/performance/ that hadn't been updated to
the new API, as well as fixing a couple issues with the `--proxy` option:
* `policy.json` wasn't being logged correctly, since mozlog was treating the
data as a format string
* Proxy prefs weren't being set correctly for the profile
Differential Revision: https://phabricator.services.mozilla.com/D79937
There is some remaining code in central originating from bug 1184405, which sought to associate source files with their "affected" test files. That ended up not panning out, and bug 1644228 removed a lot of that code, but left some remnants in the `Files` object which are still referenced in a couple different places. I'm deleting all of that code in `context.py` plus everything that references it for the following reasons:
1. Right now, `Files.{test_files,test_tags,test_flavors}` do get populated, but only ever with "default" values -- namely `moz.build` files that are above the files in question in the directory hierarchy. This is a heuristic that doesn't actually have anything to do with mapping source files to their corresponding test files, which is misleading.
2. Those attributes are accessed in two places. The first is in the `mach file-info dep-tests` command. This command isn't referenced anywhere else in tree and I don't have any evidence anyone ever uses it. Even if they do, I would claim that doing so is a mistake (because the results of the command aren't meaningful and are just populated by the "defaults" described above), and that person's workflow should be migrated to something else that *is* meaningful.
3. The second place where this metadata is accessed is in `testing/mozbase/moztest/moztest/resolve.py`; that method is invoked in `tools/tryselect/selectors/syntax.py`, but only if you pass `--detect-paths` to `mach try syntax`. This is [entirely broken](https://bugzilla.mozilla.org/show_bug.cgi?id=1614614), and even if we made an effort to fix it, it wouldn't do anything resembling what the documentation of `--detect-paths` suggests it would do (again, because the data isn't populated meaningfully). So I'm deleting the command line option entirely.
Differential Revision: https://phabricator.services.mozilla.com/D79711
Changes:
- when generating the `groups` in TestResolver for web-platform-tests, use the paths to the test files up to a maximum depth of 3 by default.
Differential Revision: https://phabricator.services.mozilla.com/D79355
Changes:
- when generating the `groups` in TestResolver for web-platform-tests, use the paths to the test files up to a maximum depth of 3 by default.
Differential Revision: https://phabricator.services.mozilla.com/D79355
Due to minor changes in D78159 and D77965 I figured it would be a good idea to release the bugfix version bump to pypi, despite our intention to discontinue using mozhttpd (Bug 1428709).
Differential Revision: https://phabricator.services.mozilla.com/D78163
For most test suites, the manifest id is the same as its relative path from
mozilla-central. But for WPT they are different. We decided to store the ids
in the 'manifest' attribute rather than 'manifest_relpath' since the latter
implies it is a file system path (which WPT manifest ids are not).
This patch takes this difference into account in
'TestResolver.tests_by_manifest'. It will consequently get the proper manifest
ids stored in the Decision task's 'tests-by-manifest.json.gz'.
Depends on D79338
Differential Revision: https://phabricator.services.mozilla.com/D79339
WPT loads test metadata after the fact (and it is the same regardless of the
loader being used). This adds a file called 'wpt_manifest_data.json' which,
combined with the related fixture, can mock out what WPT's manifestupdate
script returns.
Differential Revision: https://phabricator.services.mozilla.com/D79338
When you have a directory src and you want to move all the contents
to dst, this is surprisingly annoying. shutil.move doesn't operate
on globs. shutil.copytree does exactly what we need *except* it
requires the destination to not exist. In Python 3.7 it gained the
ability to not care about the destination existing but... we need
to handle the pre-Python 3.7 case.
Differential Revision: https://phabricator.services.mozilla.com/D76083
Depends on D75698
Note: the bug is fixed by the change in resolve.py.
manifestparser.py has no behavioral changes. It was refactored to make
the type (i.e. normalized absolute path) of `parentmanifest` more
obvious, and as a nice bonus the manifest_relpath value is generated
only once, instead of anew for every section in the test manifest.
Differential Revision: https://phabricator.services.mozilla.com/D78921
test_resolve.py spends too much time on running. This is because the
backend was regenerated using data from the whole tree, at every test
despite the test fixture.
Differential Revision: https://phabricator.services.mozilla.com/D78919
Rather than always printing instructions at the end of the bootstrap phase, we will now create a mozconfig
file if one doesn't exist and there's configuration to be written.
Differential Revision: https://phabricator.services.mozilla.com/D78417
In Py2, None compares as less than all strings and numbers.
In Py3, None isn't comparable with < to strings or numbers.
For unset variables, use "" as the value for the purposes of parsing manifest expressions.
Differential Revision: https://phabricator.services.mozilla.com/D76134
When people do Python 3 work and encounter an exception on this line, their first instinct is to do a `blame` and then NI me on the bug (which, fair enough). However when this particular issue is encountered it's usually that the `ProcessHandler` API is being invoked incorrectly and the process isn't being opened in the correct mode. Here we dump an error message to the screen immediately before the exception is thrown to hopefully point people working on dependencies of bug 1577599 on the right path.
Differential Revision: https://phabricator.services.mozilla.com/D77684
Changes:
- for web-platform-tests codepath, no longer perform filtering of the runtimes information to a list of key:value.
- for web-platform-tests codepath, load runtimes information into an OrderedDict in order from highest to lowest runtime.
- if any test paths remain after initial chunking by runtime information, distribute such paths across all chunks.
Differential Revision: https://phabricator.services.mozilla.com/D76538
Changes:
Update `writeruntimes` script so that each suite is in its own nested dictionary and add the ability to query `web-platform-tests-reftest`.
Differential Revision: https://phabricator.services.mozilla.com/D75229
CLOSED TREE
Backed out changeset 93efb53380c2 (bug 1637463)
Backed out changeset 5c8ccda5f1b1 (bug 1637463)
Backed out changeset a5fc1b604238 (bug 1637463)
At the beginning of the Python 3 migration (circa bug 1602540), we made an update to the interface of `mozpack/files.py` in the direction of aligning with Python 3's built-in `file` support; namely, that opening a file in text mode returns a stream of `str` (text), and that opening a file in binary mode returns a stream of `bytes`. This was deemed to be more trouble than it was worth. This patch undoes all of those changes to the interface in favor of moving back to the Python 2 style, where all files are bytestreams.
Differential Revision: https://phabricator.services.mozilla.com/D75424
I noticed one more moztest issue in bug 1638993: sorting an array of dict is
poorly defined on py2 and not tolerated on py3; adding the lambda clarifies
the behavior. With this resolved, I think we can bump the moztest version and
close this bug.
Differential Revision: https://phabricator.services.mozilla.com/D76486
Changes:
Update `writeruntimes` script so that each suite is in its own nested dictionary and add the ability to query `web-platform-tests-reftest`.
Differential Revision: https://phabricator.services.mozilla.com/D75229
This test has been failing, infrequently, mostly on osx/py2, for many months.
Increasing the time interval between events appears to reduce failure frequency.
Differential Revision: https://phabricator.services.mozilla.com/D76402
This patch changes over all of the logging to use the structured mozlog package.
I also ensured that the messages were logged in a way to be useful for end users
of mochitests. This means there were a few small changes in methods in order to
accommodate the logging. The behavior outside of logging should be the same.
Differential Revision: https://phabricator.services.mozilla.com/D74291
I need to add symbolication support for the mochitest Gecko Profiler command line
option. These profiles also need to be symbolicated. Unfortunately, there is not
a common place where I could use these files. Talos and Raptor each had their
own copy of the snappy symbolication server.
This commit consolidates these packages into a re-usable mozbase package that can
be used in mochitests, and eventually in other places like xpcshell tests.
I stubbed out a test file, but it doesn't do anything quite yet. This commit makes
it so that the tests still work in Raptor and Talos, but doesn't add any features.
It also doesn't try too hard to make the files look like a mozbase package.
Differential Revision: https://phabricator.services.mozilla.com/D74289
This sets things up to be a little bit easier and cleaner to modify going
forward. The taskgraph before/after this change is identical.
Differential Revision: https://phabricator.services.mozilla.com/D74452
This moves the creation of WPT groups into the TestResolver, where all kinds of
other consumers will be able to access them.
Differential Revision: https://phabricator.services.mozilla.com/D76085
`./mach run` or `./mach install` for GekcoView uses `android_device.py`, but
`android_device.py` isn't compatible with Python 3.
So, wWhen checking kvm (-accel-check), the following warning occurs.
```
WARNING: sequence item 0: expected str instance, int found
```
Also, when verifying emulator using telnet, the following exception occurs.
```
WARNING: Trying again after unexpected exception
```
Differential Revision: https://phabricator.services.mozilla.com/D75743
This is instead of tests and will make it easier to re-use in the taskgraph.
This commit is a straight refactor and results in zero differences in the
taskgraph.
Depends on D74448
Differential Revision: https://phabricator.services.mozilla.com/D74449
This adds a new property lsan-disabled to wpt metadata files, which takes
a boolean to determine if lsan should be entirely disabled for that directory.
Differential Revision: https://phabricator.services.mozilla.com/D73918
This patch reduces the redundancy in the mozproxy perfherder data names. It removes the `replay` suffix since it's already clear it's related to replay/not-replayed. The subtest names have the `mozproxy-replay` prefix removed through the addition of a new way of specifying supporting measurement values. Instead of only being able to submit the values with a `{measurement: values}` format, we can now submit it as `{measurement: {values: [], <SUBTEST-OPTIONS>}}`. The subtest options include perfherder-specific options, and some settings that change how the names are built. Furthermore, the full supporting data dictionary also includes two new fields: `summarize-values`, and `suffix-type`. The former dictates whether or not a summary value is produced, and the latter determines if the name should be suffixed with the data type.
Differential Revision: https://phabricator.services.mozilla.com/D72695
Because the web platform test harness doesn't recognize them and annotates them
with `CRITICAL`, which is disconcerting, even though those errors are typically
not harmful.
Differential Revision: https://phabricator.services.mozilla.com/D71485
Changes:
Use the value from `test.id` instead of `test.path_parts[-1]` for the test object attribute `name`.
In effect, this will change the value as follows:
`about-blank-cannot-host.html` -> `/portals/about-blank-cannot-host.html`
Differential Revision: https://phabricator.services.mozilla.com/D73141
Changes:
The patch at D70727 is now triggering the resolving of web-platform-test suites when `cram(try)` was being run.
It appears the easiest way to ensure green runs for `cram(try)` is to simply suppress the log output from `manifestupdate`. It can be argued that those log outputs were not providing any useful information for taskgraph generation.
Differential Revision: https://phabricator.services.mozilla.com/D73152
Because the web platform test harness doesn't recognize them and annotates them
with `CRITICAL`, which is disconcerting, even though those errors are typically
not harmful.
Differential Revision: https://phabricator.services.mozilla.com/D71485
This change will permit the `TestResolver` to add web-platform-tests into the list of supported tests.
Changes:
- add web-platform-test test objects to the list of supported tests.
- adjust value of attributes name, manifest, manifest_relpath.
Differential Revision: https://phabricator.services.mozilla.com/D70727
On a local Pixel 2 running Android 10, launching Geckoview test app via the
shell with the profile on external storage hangs and causes an ADBTimeoutError.
Using a profile in /data/local/tests does not exhibit this behavior.
In addition, attempting to push a directory tree directly to internal
storage can fail with "remote secure_mkdirs failed" errors when pushing
directories containing subdirectories to internal storage on /data.
The subdirectories were created with insufficient permissions for the
contents of the directories to be pushed. We can work around this problem
by catching the failure and falling back to using adb push to push the
source directory tree to a temporary location on the sdcard which does
not exhibit the error and then copying the temporary location to the
desired final destination.
These changes are necessary due to the change in Scoped Storage [1]
in Android 10 and later.
Since storing profiles on the sdcard is no longer supported the choices
for possible test roots has been reduced to using internal storage.
Since geckoview expects to be able to store content under /data/local/tmp
the default test root has been changed to /data/local/tmp/tests.
By default, the test root will be shared by all instances of ADBDevice
unless the share_test_root argument is set to False in the initializer.
This allows the test root to be defined globally with out requiring
each instance of ADBDevice to be modified.
The test_root default initial value should be None, not empty string.
The ADBDevice.test_root property is now read/write to support the new
shared test_root functionality.
Android emulators have exhibited an intermittent failure to clear logcat.
A ride-along change to retry the log clearing is included.
adb pull can fail if not all of the files have permission to be read.
This situation can appear when a profile is located on internal storage
and an app such as geckoview writes files to it. They will be readable
only by the app until the permissions are changed. Work around this
by calling chmod -R on any thing that is to be pulled from the device.
Bump mozdevice version to 3.1.2.
[1] https://developer.android.com/training/data-storage#scoped-storage
Differential Revision: https://phabricator.services.mozilla.com/D71722
Sarah Bird
Geoff Brown
James Graham
Edwin Takahashi
Mihai Alexandru Michis
Andrew Erickson
Narcis Beleuzu
Andrew Halberstadt
Sylvestre Ledru
Marco Castelluccio
Florin Strugariu
Gijs Kruitbosch
Jim Porter
Ricky Stewart
William Lachance
Bogdan Tara
Hamzah
Rob Wu
Tom Ritter
Mitchell Hentges
Noemi Erli
Sebastian Hengst
Bob Clary
Chris AtLee
Nils Ohlmeier [:drno]
Greg Tatum
Cosmin Sabou
Makoto Kato
Dzmitry Malyshau
Gregory Mierzwinski
Nicholas Nethercote
Csoregi Natalia
Joel Maher