This patch was written entirely by the following script:
#!/bin/bash
if [ ! -d "./.hg" ]
then
echo "Not in a source tree." 1>&2
exit 1
fi
find . -regex '.*\(ref\|crash\)test.*\.list' | while read FILENAME
do
echo "Processing ${FILENAME}."
# The following has four substitutions:
# * The first one replaces the *first* argument to fuzzy() when it doesn't
# have a - in it, by replacing it with an explicit 0-N range.
# * The second one does the same for the *second* argument to fuzzy().
# * The third does the same for the *second* argument to fuzzy-if().
# * The fourth does the same for the *third* argument to fuzzy-if().
#
# Note that this is using perl rather than sed because perl doesn't
# support non-greedy matching, which is needed for the first argument to
# fuzzy-if.
perl -pi -e 's/(fuzzy\()([^ ,()-]*)(,[^ ,()]*\))/${1}0-${2}${3}/g;s/(fuzzy\([^ ,()]*,)([^ ,()-]*)(\))/${1}0-${2}${3}/g;s/(fuzzy-if\([^ ]*?,)([^ ,()-]*)(,[^ ,()]*\))/${1}0-${2}${3}/g;s/(fuzzy-if\([^ ]*?,[^ ,()]*,)([^ ,()-]*)(\))/${1}0-${2}${3}/g' "${FILENAME}"
done
Differential Revision: https://phabricator.services.mozilla.com/D2974
--HG--
extra : moz-landing-system : lando
This covers all the reftests that have lower fuzz (or zero fuzz) and
were producing an UNEXPECTED-PASS result with webrender on windows. In
many cases I just adjusted the lower bound of the existing webrender
fuzz. In other cases existing fails-if conditions had to be tweaked to
exclude webrender.
MozReview-Commit-ID: 49LvS0vuYWR
--HG--
extra : rebase_source : d194e24affb87fe4560a127ff4016f9c38f414fd
Many of these could probably be fuzzed but in the interests of getting
the reftest suite turned on sooner I'm doing a blanket fails-if. This
covers all the reftests where there is more fuzz with webrender on
windows than any of existing annotations account for. In some cases the
fuzz is only a few pixels more than the equivalent Linux fuzz already
annotated, but I'll clean that up in a future bug.
MozReview-Commit-ID: IaKarbnL46d
--HG--
extra : rebase_source : 71889340305b0b12fa8eace722e42bb3faf14419
In these cases there is no human-visible difference in the WR renderings
of the test and reference images, so we can replace the failing
annotations with fuzzy ones instead.
MozReview-Commit-ID: Ioiem4ZwwI2
--HG--
extra : rebase_source : d2204d9674b4aba6eb78bc0d0007a40121564ac5
This patch:
- adds fails-if annotations for all the reftests that were consistently failing
with layers-free turned on.
- removes fails-if or reduces the range on fuzzy-if annotations for all
the reftests that were producing UNEXPECTED-PASS results with
layers-free turned on.
- adds skip-if, random-if, or fuzzy-if annotations to the reftests that
were intermittently failing due to timeout, obvious incorrectness, or
slight pixel differences, respectively.
MozReview-Commit-ID: A0Aknn6rnjj
--HG--
extra : rebase_source : 420d9cf43f23a5d654fa36eec69138937d13c173
Skip tests that are expected to fail in both Stylo and Gecko modes. They would unexpectedly "pass" in styloVsGecko mode when comparing the two failures, which is not a useful result.
MozReview-Commit-ID: 3mOpjU225Q1
--HG--
extra : rebase_source : 22bb5d4e3c5138ef832995eaf5716824f4707ffe
extra : source : d40fb20c9a49d0797c0eeae613a04912b12a28f7
Skip tests that are expected to fail in both Stylo and Gecko modes. They would unexpectedly "pass" in styloVsGecko mode when comparing the two failures, which is not a useful result.
MozReview-Commit-ID: 3mOpjU225Q1
--HG--
extra : rebase_source : 0c307639c3626af3b6b43e05d3ee73d08b3f47ce
Since -moz-element is moz only feature, the test case for -moz-element is
created under reftests/image-element folder.
MozReview-Commit-ID: G4V0G3cX7qr
--HG--
extra : rebase_source : f681d64fe4b9331a35fb082c7ed135f58c3f5a9d
L. David Baron
Matt Woodrow
Kartikaya Gupta
Andreea Pavel
Emilio Cobos Álvarez
Jonathan Kew
Ethan Lin
Sebastian Hengst
Chris Peterson
Phil Ringnalda
Alexis Beingessner
J. Ryan Stinnett
Shing Lyu
Nazım Can Altınova
Lee Salzman
Wes Kocher
Matt Brubeck
Hiroyuki Ikezoe
cku
Manish Goregaokar
Boris Zbarsky
Jeff Muizelaar