`try-test-paths` is set up to map anything under testing/web-platform
to the web-platform-tests flavour. By default, the web-platform-tests flavour
refers to the testharness test type for wptrunner, so we need to account for
reftest and wdspec test types.
This change causes mozharness to omit the test-type argument to wptrunner when
try-test-paths is being used, therefore making wptrunner determine the
appropriate test type for each requested test.
MozReview-Commit-ID: 7TDAShdDM4g
--HG--
extra : rebase_source : fde6ec219f574cd1e536764c0128a6816834f533
The GetNamedCookie command currently returns a JSON Array of one,
retained cookie, after it has removed the cookies that don't match by
name.
This is in violation of the WebDriver specification, which says it must
return the cookie serialisation directly.
MozReview-Commit-ID: 9yEiarEGBez
--HG--
extra : rebase_source : ec393eec712a6f957cee22d261bcd9ed13c6ec5b
The url fixture is used to access hosted files on the wptserve instance.
This patch makes it possible to choose between different wptserve HTTPDs
based on the protocol.
The default remains the HTTP protocol.
MozReview-Commit-ID: FvtMMUSlB4M
--HG--
extra : rebase_source : 3045dffd2c5c1ec5837b75252a4b148a5cc6e2b0
This is faster and works better in some cases (e.g. with SVG). However
it doesn't work correctly on Windows yet, so remains disabled there by default.
MozReview-Commit-ID: AXyeFUGfVgx
The internal reftest harness changes some results, but it's not clear
that it's overall better or worse than the old one, so just update the
expectation metadata for now.
MozReview-Commit-ID: KSE2z1Ct72t
This adds a metadata object associated with each test queue, and uses
it to pass cache information into the marionette internal reftest
implementation so that we are able to cache only those canvases that
will be reused.
MozReview-Commit-ID: zASrlvnri3
Initially wptrunner had a single test queue that was shared between
all processes. Then for --run-by-dir it changed to a queue of
queues. This change makes it a queue of deques, which is simpler,
since the test queues themselves are no longer shared between
processes. It also changes the implementation when we aren't using
--run-by-dir but are using multiple processes to pre-group the tests
into N queues rather than sharing a single queue between all
processes. This is necessary to use the deque of course, but
importantly anticipates a change in which we will pre-compute per
queue metdata for each queue; that doesn't work well with one shared
queue. The downside of this change is that there is no work stealing,
so it may be less efficient if we randomly assign many slow jobs to
one particular process.
MozReview-Commit-ID: 7e0Odk7yDwr
Add an InternalReftestImplmentation that runs reftests using the
built-in Marionette reftest primitives rather than screenshots. This
is actiivated using the --internal-reftest switch, although that may
become the default in the future.
MozReview-Commit-ID: 6HxGuBsTITf
This adds commands to start a reftest session, run a test, and end the
session. It as assumed that after you start a reftest session you will
just run reftests until you end the session. When starting a session
the user provides a string indicating when screenshots should be
taken, and an object mapping urls to a count of the number of times
that url is expected to be used in the session, to help with
caching. Running the tests takes a url to a test, an expected status,
a timeout, and a nested list of possible references, in which each
entry at a specific level is combined by OR and nested references are
combined by AND.
The implementation is heavilly inspired by the existing reftest
harness, starting a minimal window with no tabs, and loading the urls
directly in there. In order to get a screenshot in the e10s case we
have to pass the DRAW_VIEW and USE_WIDGET_LAYERS flags when taking the
screenshot.
For performance we heavily cache canvases; for references that will be
repeated we cache the full canvas with image, and we also cache a
single canvas to use for all other screenshots to avoid the overhead
of repeatedly creating a new canvas element.
MozReview-Commit-ID: JOFvtmH7tg
This is faster and works better in some cases (e.g. with SVG). However
it doesn't work correctly on Windows yet, so remains disabled there by default.
MozReview-Commit-ID: AXyeFUGfVgx
--HG--
extra : rebase_source : c7d33dc0c92624e05e9e62109606f1ca32bdd288
The internal reftest harness changes some results, but it's not clear
that it's overall better or worse than the old one, so just update the
expectation metadata for now.
MozReview-Commit-ID: KSE2z1Ct72t
--HG--
extra : rebase_source : f1810dd4b9f8dcc115528ceaac4894408d85f935
This adds a metadata object associated with each test queue, and uses
it to pass cache information into the marionette internal reftest
implementation so that we are able to cache only those canvases that
will be reused.
MozReview-Commit-ID: zASrlvnri3
--HG--
extra : rebase_source : feb714ec04c44f6b6d2709ae1acbb621f8b0771d
Initially wptrunner had a single test queue that was shared between
all processes. Then for --run-by-dir it changed to a queue of
queues. This change makes it a queue of deques, which is simpler,
since the test queues themselves are no longer shared between
processes. It also changes the implementation when we aren't using
--run-by-dir but are using multiple processes to pre-group the tests
into N queues rather than sharing a single queue between all
processes. This is necessary to use the deque of course, but
importantly anticipates a change in which we will pre-compute per
queue metdata for each queue; that doesn't work well with one shared
queue. The downside of this change is that there is no work stealing,
so it may be less efficient if we randomly assign many slow jobs to
one particular process.
MozReview-Commit-ID: 7e0Odk7yDwr
--HG--
extra : rebase_source : 8dbe7321a41d79f9bca1efa90494b514d16f07db
Add an InternalReftestImplmentation that runs reftests using the
built-in Marionette reftest primitives rather than screenshots. This
is actiivated using the --internal-reftest switch, although that may
become the default in the future.
MozReview-Commit-ID: 6HxGuBsTITf
--HG--
extra : rebase_source : 7b5416cfc1a26c54432346de3822a2685b41f9c9
This adds commands to start a reftest session, run a test, and end the
session. It as assumed that after you start a reftest session you will
just run reftests until you end the session. When starting a session
the user provides a string indicating when screenshots should be
taken, and an object mapping urls to a count of the number of times
that url is expected to be used in the session, to help with
caching. Running the tests takes a url to a test, an expected status,
a timeout, and a nested list of possible references, in which each
entry at a specific level is combined by OR and nested references are
combined by AND.
The implementation is heavilly inspired by the existing reftest
harness, starting a minimal window with no tabs, and loading the urls
directly in there. In order to get a screenshot in the e10s case we
have to pass the DRAW_VIEW and USE_WIDGET_LAYERS flags when taking the
screenshot.
For performance we heavily cache canvases; for references that will be
repeated we cache the full canvas with image, and we also cache a
single canvas to use for all other screenshots to avoid the overhead
of repeatedly creating a new canvas element.
MozReview-Commit-ID: JOFvtmH7tg
--HG--
extra : rebase_source : ab5a2ef2e450b9bbdc6bc3c9487ed5dfda2c1d4b
The GetNamedCookie command currently returns a JSON Array of one,
retained cookie, after it has removed the cookies that don't match by
name.
This is in violation of the WebDriver specification, which says it must
return the cookie serialisation directly.
MozReview-Commit-ID: 9yEiarEGBez
--HG--
extra : rebase_source : ad4acfb9510354a86a0b604cacf32cfd5c4f8e5d
The url fixture is used to access hosted files on the wptserve instance.
This patch makes it possible to choose between different wptserve HTTPDs
based on the protocol.
The default remains the HTTP protocol.
MozReview-Commit-ID: FvtMMUSlB4M
--HG--
extra : rebase_source : 42a25ebc1a0ca6a3d664ba4949711dd11a6f4007
Loading mozlog from within the executor process makes us prone to
deadlocks since we can't guarantee that the log mutex isn't acquired
when we fork() the parent process, and the python multiprocessing
module doesn't respect posix guidelines about execv()ing a new process
after a fork().
To avoid this specific instance of the probelm we stop pytest loading
the mozlog plugin which we aren't actually using but is loaded by
default when mozlog is available.
MozReview-Commit-ID: IIllNZVOUJz
--HG--
extra : rebase_source : 9281a4d72c82038589937a4f743e1a2447439e95
Emilio Cobos Álvarez
Phil Ringnalda
Christoph Kerschbaumer
Shawn Huang
Sebastian Hengst
Maja Frydrychowicz
Andreas Tolfsen
Andreas Tolfsen
Brad Werth
Andrea Marchesini
David Burns
James Graham
Tobias Schneider
Cameron McCormack
Wes Kocher
Joel Maher
Ho-Pang Hsu