This patch fixes the tests in testing/performance/ that hadn't been updated to
the new API, as well as fixing a couple issues with the `--proxy` option:
* `policy.json` wasn't being logged correctly, since mozlog was treating the
data as a format string
* Proxy prefs weren't being set correctly for the profile
Differential Revision: https://phabricator.services.mozilla.com/D79937
There is some remaining code in central originating from bug 1184405, which sought to associate source files with their "affected" test files. That ended up not panning out, and bug 1644228 removed a lot of that code, but left some remnants in the `Files` object which are still referenced in a couple different places. I'm deleting all of that code in `context.py` plus everything that references it for the following reasons:
1. Right now, `Files.{test_files,test_tags,test_flavors}` do get populated, but only ever with "default" values -- namely `moz.build` files that are above the files in question in the directory hierarchy. This is a heuristic that doesn't actually have anything to do with mapping source files to their corresponding test files, which is misleading.
2. Those attributes are accessed in two places. The first is in the `mach file-info dep-tests` command. This command isn't referenced anywhere else in tree and I don't have any evidence anyone ever uses it. Even if they do, I would claim that doing so is a mistake (because the results of the command aren't meaningful and are just populated by the "defaults" described above), and that person's workflow should be migrated to something else that *is* meaningful.
3. The second place where this metadata is accessed is in `testing/mozbase/moztest/moztest/resolve.py`; that method is invoked in `tools/tryselect/selectors/syntax.py`, but only if you pass `--detect-paths` to `mach try syntax`. This is [entirely broken](https://bugzilla.mozilla.org/show_bug.cgi?id=1614614), and even if we made an effort to fix it, it wouldn't do anything resembling what the documentation of `--detect-paths` suggests it would do (again, because the data isn't populated meaningfully). So I'm deleting the command line option entirely.
Differential Revision: https://phabricator.services.mozilla.com/D79711
Changes:
- when generating the `groups` in TestResolver for web-platform-tests, use the paths to the test files up to a maximum depth of 3 by default.
Differential Revision: https://phabricator.services.mozilla.com/D79355
Changes:
- when generating the `groups` in TestResolver for web-platform-tests, use the paths to the test files up to a maximum depth of 3 by default.
Differential Revision: https://phabricator.services.mozilla.com/D79355
Due to minor changes in D78159 and D77965 I figured it would be a good idea to release the bugfix version bump to pypi, despite our intention to discontinue using mozhttpd (Bug 1428709).
Differential Revision: https://phabricator.services.mozilla.com/D78163
For most test suites, the manifest id is the same as its relative path from
mozilla-central. But for WPT they are different. We decided to store the ids
in the 'manifest' attribute rather than 'manifest_relpath' since the latter
implies it is a file system path (which WPT manifest ids are not).
This patch takes this difference into account in
'TestResolver.tests_by_manifest'. It will consequently get the proper manifest
ids stored in the Decision task's 'tests-by-manifest.json.gz'.
Depends on D79338
Differential Revision: https://phabricator.services.mozilla.com/D79339
WPT loads test metadata after the fact (and it is the same regardless of the
loader being used). This adds a file called 'wpt_manifest_data.json' which,
combined with the related fixture, can mock out what WPT's manifestupdate
script returns.
Differential Revision: https://phabricator.services.mozilla.com/D79338
When you have a directory src and you want to move all the contents
to dst, this is surprisingly annoying. shutil.move doesn't operate
on globs. shutil.copytree does exactly what we need *except* it
requires the destination to not exist. In Python 3.7 it gained the
ability to not care about the destination existing but... we need
to handle the pre-Python 3.7 case.
Differential Revision: https://phabricator.services.mozilla.com/D76083
Depends on D75698
Note: the bug is fixed by the change in resolve.py.
manifestparser.py has no behavioral changes. It was refactored to make
the type (i.e. normalized absolute path) of `parentmanifest` more
obvious, and as a nice bonus the manifest_relpath value is generated
only once, instead of anew for every section in the test manifest.
Differential Revision: https://phabricator.services.mozilla.com/D78921
test_resolve.py spends too much time on running. This is because the
backend was regenerated using data from the whole tree, at every test
despite the test fixture.
Differential Revision: https://phabricator.services.mozilla.com/D78919
Rather than always printing instructions at the end of the bootstrap phase, we will now create a mozconfig
file if one doesn't exist and there's configuration to be written.
Differential Revision: https://phabricator.services.mozilla.com/D78417
In Py2, None compares as less than all strings and numbers.
In Py3, None isn't comparable with < to strings or numbers.
For unset variables, use "" as the value for the purposes of parsing manifest expressions.
Differential Revision: https://phabricator.services.mozilla.com/D76134
When people do Python 3 work and encounter an exception on this line, their first instinct is to do a `blame` and then NI me on the bug (which, fair enough). However when this particular issue is encountered it's usually that the `ProcessHandler` API is being invoked incorrectly and the process isn't being opened in the correct mode. Here we dump an error message to the screen immediately before the exception is thrown to hopefully point people working on dependencies of bug 1577599 on the right path.
Differential Revision: https://phabricator.services.mozilla.com/D77684
Changes:
- for web-platform-tests codepath, no longer perform filtering of the runtimes information to a list of key:value.
- for web-platform-tests codepath, load runtimes information into an OrderedDict in order from highest to lowest runtime.
- if any test paths remain after initial chunking by runtime information, distribute such paths across all chunks.
Differential Revision: https://phabricator.services.mozilla.com/D76538
Changes:
Update `writeruntimes` script so that each suite is in its own nested dictionary and add the ability to query `web-platform-tests-reftest`.
Differential Revision: https://phabricator.services.mozilla.com/D75229
CLOSED TREE
Backed out changeset 93efb53380c2 (bug 1637463)
Backed out changeset 5c8ccda5f1b1 (bug 1637463)
Backed out changeset a5fc1b604238 (bug 1637463)
At the beginning of the Python 3 migration (circa bug 1602540), we made an update to the interface of `mozpack/files.py` in the direction of aligning with Python 3's built-in `file` support; namely, that opening a file in text mode returns a stream of `str` (text), and that opening a file in binary mode returns a stream of `bytes`. This was deemed to be more trouble than it was worth. This patch undoes all of those changes to the interface in favor of moving back to the Python 2 style, where all files are bytestreams.
Differential Revision: https://phabricator.services.mozilla.com/D75424
I noticed one more moztest issue in bug 1638993: sorting an array of dict is
poorly defined on py2 and not tolerated on py3; adding the lambda clarifies
the behavior. With this resolved, I think we can bump the moztest version and
close this bug.
Differential Revision: https://phabricator.services.mozilla.com/D76486
Changes:
Update `writeruntimes` script so that each suite is in its own nested dictionary and add the ability to query `web-platform-tests-reftest`.
Differential Revision: https://phabricator.services.mozilla.com/D75229
This test has been failing, infrequently, mostly on osx/py2, for many months.
Increasing the time interval between events appears to reduce failure frequency.
Differential Revision: https://phabricator.services.mozilla.com/D76402
This patch changes over all of the logging to use the structured mozlog package.
I also ensured that the messages were logged in a way to be useful for end users
of mochitests. This means there were a few small changes in methods in order to
accommodate the logging. The behavior outside of logging should be the same.
Differential Revision: https://phabricator.services.mozilla.com/D74291
I need to add symbolication support for the mochitest Gecko Profiler command line
option. These profiles also need to be symbolicated. Unfortunately, there is not
a common place where I could use these files. Talos and Raptor each had their
own copy of the snappy symbolication server.
This commit consolidates these packages into a re-usable mozbase package that can
be used in mochitests, and eventually in other places like xpcshell tests.
I stubbed out a test file, but it doesn't do anything quite yet. This commit makes
it so that the tests still work in Raptor and Talos, but doesn't add any features.
It also doesn't try too hard to make the files look like a mozbase package.
Differential Revision: https://phabricator.services.mozilla.com/D74289
Jim Porter
Ricky Stewart
William Lachance
Edwin Takahashi
Bogdan Tara
Hamzah
Andrew Halberstadt
Rob Wu
Tom Ritter
Geoff Brown
Mitchell Hentges
Noemi Erli
James Graham
Sebastian Hengst
Bob Clary
Narcis Beleuzu
Chris AtLee
Nils Ohlmeier [:drno]
Sylvestre Ledru
Mihai Alexandru Michis
Greg Tatum