Changes:
Use the value from `test.id` instead of `test.path_parts[-1]` for the test object attribute `name`.
In effect, this will change the value as follows:
`about-blank-cannot-host.html` -> `/portals/about-blank-cannot-host.html`
Differential Revision: https://phabricator.services.mozilla.com/D73141
Changes:
The patch at D70727 is now triggering the resolving of web-platform-test suites when `cram(try)` was being run.
It appears the easiest way to ensure green runs for `cram(try)` is to simply suppress the log output from `manifestupdate`. It can be argued that those log outputs were not providing any useful information for taskgraph generation.
Differential Revision: https://phabricator.services.mozilla.com/D73152
This change will permit the `TestResolver` to add web-platform-tests into the list of supported tests.
Changes:
- add web-platform-test test objects to the list of supported tests.
- adjust value of attributes name, manifest, manifest_relpath.
Differential Revision: https://phabricator.services.mozilla.com/D70727
moztest assumes os.path but that fails for windows to compare against inputs that use mozpath commands with a posix slash.
Differential Revision: https://phabricator.services.mozilla.com/D71619
Changes:
- add build_flavor attribute in order to support writeruntimes and resolve().
- add subsuite attribute to web-platform-tests in resolve.py so that resolve() does not filter out test objects.
Differential Revision: https://phabricator.services.mozilla.com/D71086
The definition for reftest/crashtest variants of web-platform-tests in web-platform.yml has a trailing `s` which is technically incorrect.
This causes problems when querying ActiveData for the runtimes for these subsuites since the suite name recorded in ActiveData has the trailing s.
Changes:
- remove the trailing `s` from the definitions
Differential Revision: https://phabricator.services.mozilla.com/D68069
--HG--
extra : moz-landing-system : lando
Changes:
Remove references to `mochitest-browser-chrome-gpu` as this suite does not exist.
In `test-sets.yml`, split existing `mochitest-gpu` task into `plain` and `chrome` variants.
In `mochitest.yml` split the existing `mochitest-gpu` task into `plain` and `chrome` variants with separate symbols on treeherder.
Differential Revision: https://phabricator.services.mozilla.com/D66670
--HG--
extra : moz-landing-system : lando
There was only a single use of 'install-to-subdir' in all our manifests (which
was removed in the previous commit). All the comments around it suggest that it
was a giant hack. Removing it cleans up a lot of complexity and saves me the
trouble of needing to support it in the new sourcedir based TestResolver.
Differential Revision: https://phabricator.services.mozilla.com/D57170
--HG--
extra : moz-landing-system : lando
Also rename the key from 'ancestor-manifest' to 'ancestor_manifest' to be
consistent with other keys.
Differential Revision: https://phabricator.services.mozilla.com/D55283
--HG--
extra : moz-landing-system : lando
Also rename the key from 'ancestor-manifest' to 'ancestor_manifest' to be
consistent with other keys.
Differential Revision: https://phabricator.services.mozilla.com/D55283
--HG--
extra : moz-landing-system : lando
Changes:
Replace `<list>.items()` calls with `six.iteritems()`.
Remove `try/except` handling of `unittest` import as we have standardized on python2.7 in CI.
Use `six` to handle metaclass changes while python2/3 intercompatibility is required.
Differential Revision: https://phabricator.services.mozilla.com/D54376
--HG--
extra : moz-landing-system : lando
Build flavors are defined in 'python/mozbuild/mozbuild/testing.py'.
This change is needed by D52729 but it's also a good way to tell which suites
are integrated into the TestManifestBackend in the build system. So I'm landing
it here instead.
Depends on D53030
Differential Revision: https://phabricator.services.mozilla.com/D53698
--HG--
extra : moz-landing-system : lando
Changes:
These files do not appear to be referred to by anything as searched on searchfox.
I believe we are safe to remove them.
Differential Revision: https://phabricator.services.mozilla.com/D54201
--HG--
extra : moz-landing-system : lando
Build flavors are defined in 'python/mozbuild/mozbuild/testing.py'.
This change is needed by D52729 but it's also a good way to tell which suites
are integrated into the TestManifestBackend in the build system. So I'm landing
it here instead.
Depends on D53030
Differential Revision: https://phabricator.services.mozilla.com/D53698
--HG--
extra : moz-landing-system : lando
While not strictly necessary for this series, this patch allows:
./mach test path/to/manifest.ini
Which will be especially useful to have now that we are switching to running
tasks by manifest.
Depends on D52241
Differential Revision: https://phabricator.services.mozilla.com/D52242
--HG--
rename : testing/mozbase/moztest/tests/data/srcdir/apple/mochitest.ini => testing/mozbase/moztest/tests/data/srcdir/apple/a11y.ini
extra : moz-landing-system : lando
This gives us the ability to retrieve all browser-chrome tests (no flavor) but
not devtools-chrome (have a flavor).
Differential Revision: https://phabricator.services.mozilla.com/D52241
--HG--
extra : moz-landing-system : lando
This loader uses 'reader.find_variables_from_ast' to parse all *_MANIFESTS variables from
moz.build files using the abstract syntax tree. This means it will find all such variables
regardless of the current buildconfig.
Differential Revision: https://phabricator.services.mozilla.com/D51834
--HG--
extra : moz-landing-system : lando
Test test data looks like it was pulled from a live all_tests.pkl file. There
are way more path components than necessary.
This simplifies the test paths so they are easy to manipulate/add/inspect. It
will also make it easier to craft a fake "sourcedir" to test the
TestManifestLoader in the next commit such that the data from both matches.
I decided to use a fruit theme for directories because:
1. Using real directories will pollute grep/searchfox/etc queries with junk.
2. Using a 'dirA', 'dirB', 'dirC' scheme is hard to read.
3. Why not?
This change does not functionally modify what is being tested.
Depends on D51832
Differential Revision: https://phabricator.services.mozilla.com/D51833
--HG--
extra : moz-landing-system : lando
I'm making this refactor now because a future commit is going to completely re-write the
test data (so that it matches the data that a future TestManifestLoader is going to use).
With this method, updating the data will be a lot less tedious.
Differential Revision: https://phabricator.services.mozilla.com/D51278
--HG--
extra : moz-landing-system : lando
Previously there was a somewhat strange setup where we had both TestResolver
and TestMetadata classes. Both had 'resolve_tests' function and the separation
of concerns between the two were not clear.
With this change, all of the logic that is related to manipulating and
resolving the loaded tests has been moved to the TestResolver class. Also, the
TestMetadata class has been renamed to TestLoader, and it is solely responsible
for loading the metadata (from the build backend).
Future commits will add other types of TestLoaders.
Differential Revision: https://phabricator.services.mozilla.com/D49768
--HG--
extra : moz-landing-system : lando
A minor cleanup. Re-write paths will now automatically be joined to
self.topobjdir.
Depends on D49766
Differential Revision: https://phabricator.services.mozilla.com/D49767
--HG--
extra : moz-landing-system : lando
Similar to the vcs change, the MozbuildObject already has a reader attribute
available. So we can re-use that instead of creating our own.
Depends on D49765
Differential Revision: https://phabricator.services.mozilla.com/D49766
--HG--
extra : moz-landing-system : lando
Encapsulates all the logic around generating and loading the build backend
metadata on the TestMetadata class. Previously the TestResolver would trigger
the generation if necessary, and TestMetadata would load it. Now both
generation and loading happens in TestMetadata.load_tests.
This change also adds some convenience properties to make it easier to query
the loaded data.
Differential Revision: https://phabricator.services.mozilla.com/D49765
--HG--
extra : moz-landing-system : lando
This prevents us from adding the puppeteer tests over and over again. It
follows the wpt example.
Differential Revision: https://phabricator.services.mozilla.com/D49790
--HG--
extra : moz-landing-system : lando
Since TestResolver is a subclass of MozbuildObject, there's no need to create
separate repository object. It already has one.
Depends on D49761
Differential Revision: https://phabricator.services.mozilla.com/D49764
--HG--
extra : moz-landing-system : lando
Makes it possible for mach to resolve test paths for Puppeteer,
so that individual tests can be run from the command line using
"./mach test", as such:
% ./mach test remote/test/puppeteer/test/screenshot.spec.js
As the Puppeteer test suite is imported from upstream and we cannot
change this directory at will (i.e. to add test manifest files),
we take the same approach as for WPT and populate the manifest by
recursively walking the remote/test/puppeteer/test/**/*.spec.js file tree.
Differential Revision: https://phabricator.services.mozilla.com/D37013
--HG--
extra : moz-landing-system : lando
Makes it possible for mach to resolve test paths for Puppeteer,
so that individual tests can be run from the command line using
"./mach test", as such:
% ./mach test remote/test/puppeteer/test/screenshot.spec.js
As the Puppeteer test suite is imported from upstream and we cannot
change this directory at will (i.e. to add test manifest files),
we take the same approach as for WPT and populate the manifest by
recursively walking the remote/test/puppeteer/test/**/*.spec.js file tree.
Differential Revision: https://phabricator.services.mozilla.com/D37013
--HG--
extra : moz-landing-system : lando
Edwin Takahashi
Joel Maher
Bogdan Tara
Andrew Halberstadt
Kartikaya Gupta
Jeff Gilbert
James Graham
Mihai Alexandru Michis
Cosmin Sabou
Henrik Skupin
Andreas Tolfsen
Ciure Andrei
Alexandre Poirot
b4hand
Narcis Beleuzu