Use test identifiers that look like posix-compliant relative file paths rather than URLs; identifier "paths" are relative to topsrcdir -- an intuitive, brief mapping from identifier to test file.
This patch adds (expands) a --topsrcdir option to runreftest.py and updates CI configuration to set it appropriately. runreftest.py calculates a manifest ID, a path relative to topsrcdir, for each requested manifest and passes those manifest IDs to manifest.jsm. In manifest.jsm, a new manifest ID is generated when a manifest is included from a new directory. Test identifiers are generated by adding the test file name to the manifest ID.
Examples (url -> identifier):
- file:///builds/worker/workspace/build/tests/reftest/tests/dom/svg/crashtests/398926-fill.svg -> dom/svg/crashtests/398926-fill.svg
- file:///Z:/task_1577755655/build/tests/reftest/tests/dom/svg/crashtests/398926-fill.svg -> dom/svg/crashtests/398926-fill.svg
- http://10.7.205.207:8854/tests/dom/svg/crashtests/398926-fill.svg -> dom/svg/crashtests/398926-fill.svg
- file:///builds/worker/workspace/build/tests/jsreftest/tests/js/src/tests/jsreftest.html?test=test262/built-ins/Array/prototype/indexOf/15.4.4.14-9-b-i-31.js -> js/src/tests/test262/built-ins/Array/prototype/indexOf/15.4.4.14-9-b-i-31.js
- chrome://reftest/content/bugs/299837-2.xhtml -> layout/reftests/bugs/299837-2.xhtml
- http://localhost:36067/1577754972876/1/289480.html#top -> layout/reftests/bugs/289480.html#top
- about:blank -> about:blank
- data:text/html,<body> -> data:text/html,<body>
Differential Revision: https://phabricator.services.mozilla.com/D58415
--HG--
extra : moz-landing-system : lando
We enable the pref in the harness so that local testing with --enable-fission will get the same results as treeherder.
Differential Revision: https://phabricator.services.mozilla.com/D56313
--HG--
extra : moz-landing-system : lando
The inclusions were removed with the following very crude script and the
resulting breakage was fixed up by hand. The manual fixups did either
revert the changes done by the script, replace a generic header with a more
specific one or replace a header with a forward declaration.
find . -name "*.idl" | grep -v web-platform | grep -v third_party | while read path; do
interfaces=$(grep "^\(class\|interface\).*:.*" "$path" | cut -d' ' -f2)
if [ -n "$interfaces" ]; then
if [[ "$interfaces" == *$'\n'* ]]; then
regexp="\("
for i in $interfaces; do regexp="$regexp$i\|"; done
regexp="${regexp%%\\\|}\)"
else
regexp="$interfaces"
fi
interface=$(basename "$path")
rg -l "#include.*${interface%%.idl}.h" . | while read path2; do
hits=$(grep -v "#include.*${interface%%.idl}.h" "$path2" | grep -c "$regexp" )
if [ $hits -eq 0 ]; then
echo "Removing ${interface} from ${path2}"
grep -v "#include.*${interface%%.idl}.h" "$path2" > "$path2".tmp
mv -f "$path2".tmp "$path2"
fi
done
fi
done
Differential Revision: https://phabricator.services.mozilla.com/D55443
--HG--
extra : moz-landing-system : lando
Guard against flaky results from the remote is_file and reduce adb traffic
by waiting once for the log file to be created rather than checking for it
throughout the test run.
(Minor ride-along change to reftest.jsm resolves javascript error introduced
by bug 1575266.)
Differential Revision: https://phabricator.services.mozilla.com/D55026
--HG--
extra : moz-landing-system : lando
This patch adds a new field to the `plugin-crashed` event that holds the list
of additional minidumps associated with a crash report. The test
infrastructure is modified to use it which also fixes a race when processing
the .extra file. The reftest machinery has also been modified to take the new
field into account.
Differential Revision: https://phabricator.services.mozilla.com/D54107
--HG--
extra : moz-landing-system : lando
With the fission changes everything is more async, meaning some tests can run longer. Some crashtests navigate to a new location meaning the original root element we have a variable for is no longer around, and chrome isn't allowed to keep content nodes alive, so we are left holding a dead wrapper that throws if we try to access anything on it.
So we check if contentRootElement has become a dead wrapper (and null it out) any time we try to access it and it could have become dead. Generally the existing code already handled a null contentRootElement.
Differential Revision: https://phabricator.services.mozilla.com/D52829
--HG--
extra : moz-landing-system : lando
I don't think this is strictly necessary but it lets us avoid a bunch of useless work, especially with webrender where these rects are always the full window size.
Differential Revision: https://phabricator.services.mozilla.com/D51346
--HG--
extra : moz-landing-system : lando
The code comment mostly explains the design. Basically, we force nothing to happen while we wait for the promises to finish and instead record what we need to do once the promise is finished, and do those pending tasks when it's finished.
Differential Revision: https://phabricator.services.mozilla.com/D51344
--HG--
extra : moz-landing-system : lando
This changes them to return a promise that resolves when the work is done, but we still need to change the callers to handle this new return type and do the right thing when these functions do their work async-ly.
To do this we add a JSWindowActor called ReftestFission. reftest-content.js communicates with this actor via reftest.jsm.
Differential Revision: https://phabricator.services.mozilla.com/D51343
--HG--
extra : moz-landing-system : lando
With the fission changes everything is more async, meaning some tests can run longer. Some crashtests navigate to a new location meaning the original root element we have a variable for is no longer around, and chrome isn't allowed to keep content nodes alive, so we are left holding a dead wrapper that throws if we try to access anything on it.
So we check if contentRootElement has become a dead wrapper (and null it out) any time we try to access it and it could have become dead. Generally the existing code already handled a null contentRootElement.
Differential Revision: https://phabricator.services.mozilla.com/D52829
--HG--
extra : moz-landing-system : lando
I don't think this is strictly necessary but it lets us avoid a bunch of useless work, especially with webrender where these rects are always the full window size.
Differential Revision: https://phabricator.services.mozilla.com/D51346
--HG--
extra : moz-landing-system : lando
The code comment mostly explains the design. Basically, we force nothing to happen while we wait for the promises to finish and instead record what we need to do once the promise is finished, and do those pending tasks when it's finished.
Differential Revision: https://phabricator.services.mozilla.com/D51344
--HG--
extra : moz-landing-system : lando
This changes them to return a promise that resolves when the work is done, but we still need to change the callers to handle this new return type and do the right thing when these functions do their work async-ly.
To do this we add a JSWindowActor called ReftestFission. reftest-content.js communicates with this actor via reftest.jsm.
Differential Revision: https://phabricator.services.mozilla.com/D51343
--HG--
extra : moz-landing-system : lando
This is basically what we did in bug 1593171 (Protect against the same test from calling RecordResult more than once in the reftest harness) where we early exit in SendInitCanvasWithSnapshot.
But now we do it in SendUpdateCanvasForEvent too because SendUpdateCanvasForEvent calls SynchronizeForSnapshot which calls setupAsyncScrollOffsets and setupAsyncZoom, both of which get the documentElement of the current doc and operate on it. The problem is that this could be SendUpdateCanvasForEvent call from the previous test operating on the dom of the current test.
I haven't actually observed this, just noticed it while implementing checking of contentRootElement to make sure all cases are covered.
Differential Revision: https://phabricator.services.mozilla.com/D52827
--HG--
extra : moz-landing-system : lando
Add test annotations to the ReftestManifest and TestResolver. For example, a
test description from the TestResolver might now contain 'skip-if': 'skiaContent';
similar to the content provided for manifestparser tests; this will allow
'mach test-info report' to filter tests based on reftest manifest test
annotations.
Also add the referenced-test field which identifies the test file associated
with test entries for reference files; this will allow test-verify to
run the correct reftest when only the reference file has been modified.
Differential Revision: https://phabricator.services.mozilla.com/D51113
--HG--
extra : moz-landing-system : lando
A speculative fix for intermittent android reftest failures in which the reftest log file
is incomplete but the missing logging is found in the logcat: I hope that closing the
log file explicitly will ensure logging is flushed before the application quits.
Differential Revision: https://phabricator.services.mozilla.com/D51695
--HG--
extra : moz-landing-system : lando
Some crashtests call window.location.reload one or more times. Each time the test document loads we call OnDocumentLoad, and each call of that function can lead to a RecordResult call, either directly or indirectly through WaitForTestEnd and its MakeProgress state machine.
Although the error when this happens ("program error managing timeouts") is confusing I found a couple intermittent instances of this happening in our bugzilla.
However with the fission changes to the reftest harness (which in general make things more async) this is much more common because we keep the tests alive longer before tearing down the test page, giving it more of a chance to happen.
To fix it we pass around the top level test url so it can make its way to RecordResult. This means we can ignore RecordResult calls for previous tests, and ignore the second or more RecordResult call for the current test. We reset the count when we start a new test.
Differential Revision: https://phabricator.services.mozilla.com/D51342
--HG--
extra : moz-landing-system : lando
This patch adds a menu item in "Toggle" -> "Paged Mode". If the item is
toggled, after finishing loading the document, the document will be
forced into paged mode.
The print settings in `nsLayoutDebuggingTools::SetPagedMode()` are
similar to `setupPrintMode()` in reftest-content.js.
Remove the hack in nsDocumentViewer::InitInternal() because we don't
want to set a new document when mIsPageMode = false.
Differential Revision: https://phabricator.services.mozilla.com/D49993
--HG--
extra : moz-landing-system : lando
The following tests are not removed and need to be ported to Shadow DOM:
- test_bug330925.xhtml
- test_bug372086.html
- test_bug319374.xhtml
Differential Revision: https://phabricator.services.mozilla.com/D50650
--HG--
extra : moz-landing-system : lando
Make reftest --run-until-failure consistent with mochitest:
if --run-until-failure requested without --repeat, repeat up to 30 times;
if --run-until-failure requested with --repeat=N, repeat up to N times.
Differential Revision: https://phabricator.services.mozilla.com/D50484
--HG--
extra : moz-landing-system : lando
These assertion counts were removed (accidentally?) by bug 1321127, effectively
not tracking assertion count mismatches in the reftest harness and instead
relying on mozharness to fail tasks based on the logged error messages.
Restoring the counts ensures:
- the reftest summary includes accurate assertion counts like
REFTEST INFO | Unexpected: 12 (..., 11 unexpected asserts, ...)
REFTEST INFO | Known problems: 64 (..., 31 known asserts, ...)
- assertion mismatches cause the harness to exit with an error code so
that the job fails even if the log parsing is broken (bug 1587139)
or the tests are being run locally with mach.
Differential Revision: https://phabricator.services.mozilla.com/D48594
--HG--
extra : moz-landing-system : lando
Adds a way for mochitest, reftest, and crashtests to skip XBL related
tests when XBL is disabled. Also, add an app constant so JS can
check whether XBL is enabled.
Depends on D45614
Differential Revision: https://phabricator.services.mozilla.com/D45615
--HG--
extra : moz-landing-system : lando
This means that you can use it as a very light-weight crashtest harness by
using:
MOZ_GDB_SLEEP=0 ./mach run -layoutdebug <file> -autoclose
Right now we just never exit otherwise.
Differential Revision: https://phabricator.services.mozilla.com/D47715
--HG--
extra : moz-landing-system : lando
This means that you can use it as a very light-weight crashtest harness by
using:
MOZ_GDB_SLEEP=0 ./mach run -layoutdebug <file> -autoclose
Right now we just never exit otherwise.
Differential Revision: https://phabricator.services.mozilla.com/D47715
--HG--
extra : moz-landing-system : lando
Geoff Brown
Justin Wood
Cameron McCormack
Ting-Yu Lin
Matt Woodrow
Gabriele Svelto
Gurzau Raul
Bert Peers
Botond Ballo
Emma Malysz
Timothy Nikkel
Alexander Surkov
Arthur Iakab
Jon Coppeard
Jamie Nicol
Brian Grinstead
Zibi Braniecki
Xidorn Quan
Andrew Halberstadt
Brendan Dahl
Brindusan Cristian
Emilio Cobos Álvarez
Noemi Erli
