Test harnesses may use STYLO_FORCE_ENABLED, so we need to check this value when
building the reftest conditions.
MozReview-Commit-ID: 998UMZHNKLl
--HG--
extra : rebase_source : c52e2539c0384c2c300b729ac88fe3381e127509
This adds commands to start a reftest session, run a test, and end the
session. It as assumed that after you start a reftest session you will
just run reftests until you end the session. When starting a session
the user provides a string indicating when screenshots should be
taken, and an object mapping urls to a count of the number of times
that url is expected to be used in the session, to help with
caching. Running the tests takes a url to a test, an expected status,
a timeout, and a nested list of possible references, in which each
entry at a specific level is combined by OR and nested references are
combined by AND.
The implementation is heavilly inspired by the existing reftest
harness, starting a minimal window with no tabs, and loading the urls
directly in there. In order to get a screenshot in the e10s case we
have to pass the DRAW_VIEW and USE_WIDGET_LAYERS flags when taking the
screenshot.
For performance we heavily cache canvases; for references that will be
repeated we cache the full canvas with image, and we also cache a
single canvas to use for all other screenshots to avoid the overhead
of repeatedly creating a new canvas element.
MozReview-Commit-ID: JOFvtmH7tg
This adds commands to start a reftest session, run a test, and end the
session. It as assumed that after you start a reftest session you will
just run reftests until you end the session. When starting a session
the user provides a string indicating when screenshots should be
taken, and an object mapping urls to a count of the number of times
that url is expected to be used in the session, to help with
caching. Running the tests takes a url to a test, an expected status,
a timeout, and a nested list of possible references, in which each
entry at a specific level is combined by OR and nested references are
combined by AND.
The implementation is heavilly inspired by the existing reftest
harness, starting a minimal window with no tabs, and loading the urls
directly in there. In order to get a screenshot in the e10s case we
have to pass the DRAW_VIEW and USE_WIDGET_LAYERS flags when taking the
screenshot.
For performance we heavily cache canvases; for references that will be
repeated we cache the full canvas with image, and we also cache a
single canvas to use for all other screenshots to avoid the overhead
of repeatedly creating a new canvas element.
MozReview-Commit-ID: JOFvtmH7tg
--HG--
extra : rebase_source : ab5a2ef2e450b9bbdc6bc3c9487ed5dfda2c1d4b
This is needed before we can upgrade to flake8 3.3.0, as that version starts flagging these errors.
These files were modified by running:
autopep8 --select E305 --in-place -r <dir>
on the affected directories. I did it one dir at a time and verified the result after each.
MozReview-Commit-ID: FmlsfiKIbtr
--HG--
extra : rebase_source : 9df32258cadff5d27a0e72113c57f782756c0b18
This won't be an issue until the 57 release but at that time, the
reftest extensions will only load if the extensions.legacy.enabled
preference is set to true and Cu.isInAutomation, which depends on
another preference, is true. Set the appropriate prefs here for reftests.
MozReview-Commit-ID: JUWCPTsuJU
--HG--
extra : rebase_source : 042a5696d1926e7ca434a7c4213395176a080549
This catches a common problem where somebody adds a fuzzy annotation on
a test to work around some minor differences. Later the differences go
away, but since the test harness doesn't catch that, nobody is the
wiser. Subsequently a "real" regression can reintroduce differences
which are hidden by the stale fuzzy annotations.
With this patch, if the annotations are set up properly, the test
harness will flag tests as "UNEXPECTED-PASS" when the differences go
away. This will require the patch author to reduce the allowed fuzziness
parameters, and will make it easier to catch subsequent regressions.
MozReview-Commit-ID: B3rGPFLXkCu
This patch extends the "fuzzy" and "fuzzy-if" annotations so that they
accept a range of fuzziness values, much like the "asserts" and
"asserts-if" annotations. If the test produces differences that are
within the specified range, the test is considered as passing. Any
differences outside the ranges are considered failures. If the test is
marked as a "!=" test, then the opposite is true.
If the range given to "fuzzy" or "fuzzy-if" is a single value, it is
interpreted as being a range from zero to the specified value. This is
in contrast to "asserts" and "asserts-if" which match exactly the value
specified. The reason for this is mostly for backwards compatibility
with existing reftest annotation semantics.
MozReview-Commit-ID: 6qUU6FQ5mYP
This simply augments the logging in the reftest harness so that if a reftest
passes due to fuzzy matching, it reports both the actual fuzz numbers and the
maximum allowed fuzz numbers.
MozReview-Commit-ID: G356vBRXYUc
Remove the forbiddenURI pref which was removed in bug 1274893 as well
as browser.safebrowsing.enabled which got renamed in bug 1025965.
Set dummy URLs for all of the network endpoints.
MozReview-Commit-ID: Efk2fv6cC3g
--HG--
extra : rebase_source : 9fbb3eb0fa7f002fe24577a8a0870ec4d1b7cf31
Our linux32-debug build is very slow to startup when running mochitests on aws.
Sometimes we see similar behavior on other linux platforms. Intermittently,
in this environment, startup takes longer than the 120 seconds that marionette
waits, resulting in test failures in bug 1261598. Increasing the marionette
startup timeout to 180 seconds appears to effectively avoid these failures.
Remove suggested and enhanced tiles along with related campaign, frequency-cap, inadjacency, pings, preferences, strings, styles, tests.
MozReview-Commit-ID: FkjaSpSFQHu
--HG--
extra : rebase_source : 1c58ac542180f0abb290639ec1c61b9edf3d0a51
This prefix can't be produced by the standard structured logging
formatters because it doesn't know that the tests are
reftests. Therefore the reftest harness has a hack to add this prefix,
and the unstructured analyzer doesn't work with wpt reftests. This can
simply be solved by making the prefix optional, but looking for the
"INFO - " prefix that mozharness adds, when present, to identify the
start of the TEST- string.
MozReview-Commit-ID: GxWVWOSkAUZ
--HG--
extra : rebase_source : fbd8afa860da0fba4925b6bf2fc715cbf6c2688c
Non-ascii characters in error messages can cause exceptions when processing
structured log messages. The mochitest MessageLogger already handles such
cases; this is a simpler implementation for the reftest OutputHandler.
Previously the harness would check for navigator.plugins[Test Plug-in] in the browser window, but that is now blocked from working. Instead we can simply use an existing getTestPlugin function that enumerates the plugin through nsIPluginHost
MozReview-Commit-ID: Kp48u5iFkSa
Continue to allow non-multiprocessCompatible extensions in automation.
There are a ton of places that would need to be changed, many of which
will be changing soon anyway with the non-webextensions change in 57
so this is mostly the expedient route to keeping the tree green.
MozReview-Commit-ID: EZZoDVdhLfy
--HG--
extra : rebase_source : f83472bc1c88dd0deadbe485d9002499027ff07f
Continue to allow non-multiprocessCompatible extensions in automation.
There are a ton of places that would need to be changed, many of which
will be changing soon anyway with the non-webextensions change in 57
so this is mostly the expedient route to keeping the tree green.
MozReview-Commit-ID: EZZoDVdhLfy
--HG--
extra : rebase_source : 34aa762917566b052ade6372280caed72fbfbe9a
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.
r=haik,froydnj
MozReview-Commit-ID: KNvAoUs5Ati
--HG--
extra : rebase_source : 81ba8bfee0ca96979cf8e30d75cdd47f06bc10ea
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.
r=haik,froydnj
MozReview-Commit-ID: KNvAoUs5Ati
--HG--
extra : rebase_source : f637acff32fc8582732de932503dd696abc57877
This eliminates a 2 minute timeout seen at the end of Android mochitests
and reftests. Attempts to shutdown the web server were failing because
they were directed at IP 10.0.2.2 -- the loopback address for the
Android emulator.
The devicemanager killProcess() is updated to use force-stop first, then
use kill if force-stop does not work.
Browser test harnesses are updated to check if killProcess() worked, and
warn if it failed.
Pending crash reports are stored in UAppData, which is typically
outside of the test profile. Pending crash reports left by one test
may affect future tests, so it is important that they be deleted
between tests. However, since UAppData crash data may be important
to developers running tests outside of automation, the harness cannot
delete them unless requested with the --cleanup-crashes option.
The recommended Marionette preferences for use in automation are not so
useful when testing the browser itself. Many of the tests for Firefox
have highly specific requirements, and setting the recommended Marionette
preferences could for these mean a deterioration of test coverage.
The majority of test harnesses utilising Marionette use it to install
unsigned add-ons at runtime, and these preferences are not relevant in
this context.
To avoid an unfortunate and unintended regression in test coverage,
we skip using the recommended Marionette preferences when we test the
browser itself.
MozReview-Commit-ID: 4DeHZfQEX7d
--HG--
extra : rebase_source : bf31d336df7c4c28c04454338a912632087aa3ff
reftest cannot use testing-common, so we should include AsyncSpellCheckTestHelper.jsm in reftest.jar.
Previous fix has typo of chrome vs resource. It should be chrome://.
MozReview-Commit-ID: KeyDLjc9AUI
--HG--
extra : rebase_source : 5bf2e6f4105f3437fb3c88410a246e1b85b1bf1d
CLOSED TREE
Backed out changeset 941e0f9ff9a7 (bug 1351074)
Backed out changeset 4fdf3b87a70b (bug 1351074)
Backed out changeset 586428f69838 (bug 1351074)
reftest cannot use testing-common, so we should include AsyncSpellCheckTestHelper.jsm in reftest.jar.
MozReview-Commit-ID: KeyDLjc9AUI
--HG--
extra : rebase_source : 0c967b38faf506c2e93abc8707e27cfa5388de00
sutagent is no longer built or used; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and test harness options like --dm_trans are eliminated.
sutagent is no longer built or usedr; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and the --dmTrans and similar options have been removed
from test harnesses and mach commands.
This makes two changes:
* adds "on the line" to clarify what last means
* adds "when combining <failure-type> from the manifest include and the
test line" to clarify that the parenthetical only applies to combining
at different levels, and not within a line
DONTBUILD
--HG--
extra : rebase_source : 4e45753f11b20313ed010ec8d01e0403b89591fd
extra : amend_source : c1f4acb341f0cb2f713080e73c686a5e67aed521
This makes the code work the way it looks like it was intended to work
based on what was already in output.py.
MozReview-Commit-ID: 8LJHxlfiQQ8
--HG--
extra : rebase_source : 46b2435ddbd3e8485536806d9fc7987599227f16
Prior to this patch, dumping of style contexts in the layout
debugger would only dump the subtree associated with the root
frame, which just covers the viewport, canvas, and associated
scrollbars.
I think this has been broken since bug 236921 (2004) and
bug 378693 (2007) properly disconnected the root element from the
styles for the viewport and canvas.
This patch just manually dumps the root frame and the root
element's frame, since the style set no longer maintains a list
of all of the style context roots.
MozReview-Commit-ID: 5LEnbNtngN2
--HG--
extra : transplant_source : v%BB%09%C8M8%FE%25%A9%A6%3D%EA%BD%BE%DF7%08%9A%85m
With this pref turned on, channels with the LOAD_CLASSIFY_URI load flag
will return true from the nsIHttpChannel::IsTrackingResource() API after
being loaded if their URL appears on the TP list.
This is being disabled because of the leak in bug 1325215. It slipped in because leak
checking in reftest was temporarily failing to turn the job orange. We are disabling
leak checking so we can land the fix ASAP and prevent further bustage from slipping
through in the mean time. Once bug 1325215 has been fixed, this should be backed out.
MozReview-Commit-ID: 74VGhHvZtIG
--HG--
extra : rebase_source : c965aa1e229a15948d2d4b55dc16c5904c304ad9
This is being disabled because of the leak in bug 1325215. It slipped in because leak
checking in reftest was temporarily failing to turn the job orange. We are disabling
leak checking so we can land the fix ASAP and prevent further bustage from slipping
through in the mean time. Once bug 1325215 has been fixed, this should be backed out.
MozReview-Commit-ID: 74VGhHvZtIG
--HG--
extra : rebase_source : 176dd020bb13947198c52378be86463e69f7749b
This is intended as a structured replacement for the assertion checks
that previously used unstructured logs. It adds a log action
assertion_count, which takes the actual number of assertions observed
during a test, the minimum expeced number and the maximum expected
number. It also updates the reftest harness to use this logging.
MozReview-Commit-ID: JgjLlaYuvSG
Previously the run-wizard script would add a command to source the virtualenv in ~/.bashrc after
mozharness finished setting things up. This is fragile, assumes people are using bash, etc. Plus
it appeared to intermittently fail for some users.
Instead, this activates the virtualenv directly from individual mach commands that need it. This
guarantees we will always be using the virtualenv if required (and won't be using it if not). The
'activate_this.py' script is invoked the same way that we do it for in-tree mach commands:
https://dxr.mozilla.org/mozilla-central/rev/9c06e744b1befb3a2e2fdac7414ce18220774a1d/python/mozbuild/mozbuild/virtualenv.py#456
MozReview-Commit-ID: CfcoiVJXQTl
--HG--
extra : rebase_source : da01d1ce1bd9b41c89922e989f857c4de8c09341
On fennec we have the special workaround that is doesn't dispatch the "error" event when the error is
"MEDIA_ERR_SRC_NOT_SUPPORTED" because we will use an external app to open it.
But we don't want this behavior affect the tests we already have, so close the pref.
MozReview-Commit-ID: 9aoy1vnekvY
--HG--
extra : rebase_source : 9ac045b7595eadd36039bc6f42b32e4d3eac421b
This patch removes support for mozapp iframes, leaving support for
mozbrowser iframes intact. Some of the code has been rewritten in order
to phrase things in terms of mozbrowser only, as opposed to mozbrowser
or app. In some places, code that was only useful with apps has been
completely removed, so that the APIs consumed can also be removed. In
some places where the notion of appId was bleeding out of this API, now
we use NO_APP_ID. Other notions of appId which were restricted to this
API have been removed.
This patch removes support for mozapp iframes, leaving support for
mozbrowser iframes intact. Some of the code has been rewritten in order
to phrase things in terms of mozbrowser only, as opposed to mozbrowser
or app. In some places, code that was only useful with apps has been
completely removed, so that the APIs consumed can also be removed. In
some places where the notion of appId was bleeding out of this API, now
we use NO_APP_ID. Other notions of appId which were restricted to this
API have been removed.
This patch tries to do three things:
1) Replace the ENABLE_MARIONETTE entrypoint with --enable-marionette.
2) Fold the default value -- forced on unless building for target OS
Android or building with toolkit gonk -- into the flag, rather than
embedding that condition in the tree.
3) Stop using AC_DEFINE and instead use only AC_SUBST, so that no
compiled code needs to be rebuilt if the flag is flipped locally.
n.b., each installer/Makefile.in knows that ENABLE_MARIONETTE is set
(in order to set -DENABLE_MARIONETTE=1 for
*/installer/package-manifest.in) due to it being an AC_SUBST.
MozReview-Commit-ID: AkkmybyP1uI
--HG--
extra : rebase_source : c2c8b268c60350ff39d872cee357b53f17e79eef
WindowWatcher allows for either nsIArray or nsISupports array to be passed in
for the arguments param. This converts all internal usage to nsIArray.
MozReview-Commit-ID: DQjtIkobik0
This allows the logs to work with the structured reftest viewer.
MozReview-Commit-ID: CY71vSdDjLP
--HG--
extra : rebase_source : 6b83d98aff1c5e73ac0a802b5a83b8be95adf59a
This allows reftests from web-platform-tests to be analyzed using the
wpt_raw.log link on treeherder, and also supports the existing reftest harness.
MozReview-Commit-ID: I6TdGKQXpf8
--HG--
rename : layout/tools/reftest/reftest-analyzer.xhtml => layout/tools/reftest/reftest-analyzer-structured.xhtml
extra : rebase_source : be2718204d4dacd5f3353060b39010ab134fe68d
This adds the ability to run Android reftests from a taskcluster interactive loaner using
mach.
MozReview-Commit-ID: 5HMfm64wmK2
--HG--
extra : rebase_source : c17763e6916ef5dbaab25c6de6691aeb7541c3d9
When using an interactive loaner, we compute an explicit path to the adb binary. Passing it in
directly to mozdevice seems a bit cleaner than modifying the $PATH. This also does a minor
refactor around how the DM instances are instantiated.
MozReview-Commit-ID: 5HMfm64wmK2
--HG--
extra : rebase_source : a42afcc49f2b07892c2da083c5ce3589276cc765
Reftests hackily create a second argument parser and set defaults on it to get around the fact
that the mach command sends the harness a dict rather than a Namespace object. This is bad because:
1. It's much less hacky to just create a Namespace object directly (rather than making a second parser)
2. Most other mach commands actually *do* return a Namespace object, reftest is the odd one out here.
So this patch makes the reftest mach commands convert to the Namespace object, which allows us to get
rid of the hacks in the reftest harness. This also does some light refactoring of the entry points to
the reftest harness so make it more consistent with mochitest and xpcshell.
MozReview-Commit-ID: 5HMfm64wmK2
--HG--
extra : rebase_source : 7e9c0fc64e3aeced7662a9df13f94b1b5d751bc8
Android reftests still use os.chdir and imp to import reftest files. But there's a
_setup_objdir method (used by b2g) that does a similar thing instead. I decided to
try getting Android to use this as well and it seems to work. This is just a cleanup.
MozReview-Commit-ID: 5HMfm64wmK2
--HG--
extra : rebase_source : b9f52c01e0d1311a0c4511ffbc0a60e7d149ffe8
I noticed that if you request a loaner for a non-e10s job then run mach, it will be run with e10s
enabled. The mach command should accurately reflect the type of job that got requested. This patch
grabs the 'e10s' argument from the mozharness localconfig.json and uses that.
MozReview-Commit-ID: 4lsKGpizfH7
--HG--
extra : rebase_source : 6f7b45146bb7f3431c242d92b4206d41f7217c03
You can set attributes on a mach context by using the 'key' argument to the context_handler. Basically,
whatever gets returned by the handler when <key> is passed in, will get set (i.e cached) on the context
object for fast retrieval next time. This is a way to lazy load these attributes.
Previously I was setting functions like 'find_firefox()' on the context object, and then having the
mach_commands call that directly. But this way is much cleaner.
Now, the loaded 'mozharness_config' can be stored as an attribute on the context. Also 'find_firefox()'
is now an attribute called 'firefox_bin'.
MozReview-Commit-ID: 4lsKGpizfH7
--HG--
extra : rebase_source : af5e32e05a29b8e91d4cd0005689baa8079ec137
Ensure that the first line of the log for failing tests is identical to
one that would be produced by the tbplformatter from mozlog. This means
that treeherder will be able to sucessfully cross-match error lines.
Also make the data structures used for storing screenshots identical
between reftests and web-platform-tests so that we can start to move
toward removing the reftest-specific codepath here.
MozReview-Commit-ID: FZQXLjj9Ejv
--HG--
extra : rebase_source : 3e9bf70ff7915817b8c02032c8da0d0a659e4661
This fixes a regression from bug 1288827. It happened because I moved the logic that finds
the application path a little later on in the test harness. But there was an instance where
it was being used in the android mach command before that point.
As it turned out, we don't really *need* that value there. This patch grabs the same value
from build_obj.substs which is already an argument to the function.
MozReview-Commit-ID: 3IsI4VzEIIF
--HG--
extra : rebase_source : ceb718b265c7886b831e0378436baebf2b4e7598
This removes the unnecessary setting of c-basic-offset from all
python-mode files.
This was automatically generated using
perl -pi -e 's/; *c-basic-offset: *[0-9]+//'
... on the affected files.
The bulk of these files are moz.build files but there a few others as
well.
MozReview-Commit-ID: 2pPf3DEiZqx
--HG--
extra : rebase_source : 0a7dcac80b924174a2c429b093791148ea6ac204
This adds reftest support to the test package mach environment. It will allow
developers to easily run reftests after checking out an interactive worker.
MozReview-Commit-ID: fBAbfuG5XQ
--HG--
extra : rebase_source : 84b4a9fff7f2f27a325ffad4af1de7726bad296e
The idea of mozlog ids being tuples unfortunately didn't work so well
with external systems that found the tuple|string datatype difficult to
work with. Convert reftests to use simple ids of the form "<test url>
<comparison> <refurl>" e.g. "about:blank == data:text/html," instead of
tuples.
MozReview-Commit-ID: 18jufbssn4A
--HG--
extra : rebase_source : 542888186fecf3671847d1fca307770c7a989ad3
In bug 1271035 gps identified disk I/O as a major cause for reftest slowness.
He was able to fix most of it, but the new highest source of I/O in reftest is
reftest.log.
To note, we were only saving reftest.log when running via mach, so this won't
impact automation in any way:
https://hg.mozilla.org/mozilla-central/file/043082cb7bd8/layout/tools/reftest/mach_commands.py#l204
I don't know why we are doing this given that the same output goes to stdout.
And as dholbert pointed out, since bug 1034290 landed, that log contains raw
structured logs, which are not at all useful for debugging. Given that it is
no longer useful and causes slowness, we should stop saving it.
If anyone wishes to keep saving to a log, they can use:
./mach reftest --log-tbpl reftest.log
If they wish this to be the default behaviour they can make a machrc (or
.machrc) in either topsrcdir, ~/.mozbuild or $MACHRC. Then add:
[alias]
reftest = reftest --log-tbpl reftest.log
MozReview-Commit-ID: A3e2X7qF90H
--HG--
extra : rebase_source : 9286e3061c068c6fef55b503017b3f4a9588be46
It was a cold Friday night in San Francisco. Earlier in the day, I
informed Chris AtLee that I was going to start focusing on improving
the efficiency of Firefox automation and asked him where the biggest
capacity issues were. He said "we're hurting most on Windows tests."
As I was casually drinking a barleywine (note to reader: barleywines
are serious beers - there's nothing casual about them), I found myself
tediously clicking through Treeherder looking at logs for Windows jobs,
looking for patterns and other oddities. As I was clicking through,
something stood out to me: the sheer number of reftest jobs. I
recalled a random project I started a few years ago. Its aim was to
analyze buildbot job metadata so we could better understand where time
was spent in automation. I had mostly written off the side project as
a failure and a near complete waste of my time. Not even a stray
random thought of this project had entered my mind in the past year.
But clicking through Treeherder after a few glasses of barleywine
somehow reminded me of one of the few useful findings of that project:
reftest jobs consumed a seemingly disproportiate amount of machine time,
something like 35 or 40% IIRC of the time spent on all jobs.
Now, this finding was several years ago and almost certainly no longer
relevant. But, again, I had a few glasses of barleywine in me and was
bothered by the amount of reftest jobs and their duration, so I thought
"hey, why don't I run reftests and see why they take so long." So I
built Firefox on Windows - the platform Chris AtLee said we're "hurting
most on."
I decided to start my very casual profiling session by recording a
`mach reftest` run using Sysinternals Process Monitor. To my surprise,
it yielded a very obvious and unexpected result: the Places SQLite
database was incurring a lot of I/O. On my i7-6700K Windows 10 desktop
with a high performance SSD, `mach reftest` yielded the following:
File Time #Events #Reads #Writes RBytes WBytes Path
198s 980,872 243,270 669,231 7,971,471,360 20,667,084,080 places.sqlite-wal
165s 645,853 222,407 367,775 7,287,701,820 14.071,529,472 places.sqlite
2s 377,121 1 0 32,768 0 places.sqlite-shm
The Places SQLite database accounts for 2,003,846 of the total of
3,547,527 system calls (56.49%) recorded by procmon during `mach
reftest` execution. This represents a staggering 49,997,786,732 of the
50,307,660,589 (99.38%) bytes of I/O recorded! Yes, that's 50 GB.
I reckon the reason the Places database accumulates so much I/O load
during reftests is because the reftest suite essentially loads thousands
of pages as quickly as possible. This effectively performs a stress
test against the Places history recording service.
As effective as reftests are at stress-testing Places, it adds no value
to reftests because reftests are testing the layout features, not the
performance of history recording. So these 2M system calls and 50 GB
of I/O are overhead.
This commit disables Places when executing reftests and prevents
the overhead.
After this commit, `mach reftest` has significantly reduced interaction
with the Places SQLite database:
File Time #Events #Reads #Writes RBytes WBytes Path
0.09s 502 138 302 4,521,984 8,961,528 places.sqlite-wal
0.07s 254 20 140 524,604 8,126,464 places.sqlite
0.01s 3,289 1 0 32,768 0 places.sqlite-shm
Of the 948,033 system calls recorded with this change (26.7% of
original), 691,322 were related to I/O. The Places SQLite database
only consumed ~22MB of I/O, <0.01% of original. It's worth noting that
~half of the remaining I/O system calls are related to reftest.log,
which now accounts for the largest percentage of write I/O (only
~53 MB, however). It's worth noting that reftest.log appears to be
using unbuffered writes and is requiring an excessive amount of
system calls for writing. But that's for another bug and commit.
In terms of wall time, the drastic I/O reduction during `mach reftest`
appears to have minimal impact on my machine: maybe 30s shaved from a
~900s execution, or ~3%. But my machine with its modern SSD doesn't
struggle with I/O.
In automation, it is a different story.
I pushed this change to Try along with the base revision and triggered
4 runs of most reftest jobs. The runtime improvements in automation
are impressive. Here are the fastest reported times for various jobs:
Job Before After Delta
Linux Opt R1 31m 34m +3m
Linux Opt R2 43m 35m -8m
Linux Opt Ru1 40m 34m -6m
Linux Opt Ru2 43m 37m -6m
Linux Opt R E10s 89m 72m -17m
Linux Debug R1 52m 40m -12m
Linux Debug R2 49m 42m -7m
Linux Debug R3 60m 51m -9m
Linux Debug R4 42m 37m -5m
Linux Debug R1 E10s 84m 72m -12m
Linux Debug R2 E10s 97m 85m -12m
Linux64 Opt R1 35m 24m -11m
Linux64 Opt R2 37m 26m -11m
Linux64 Opt Ru1 32m 29m -3m
Linux64 Opt Ru2 37m 26m -12m
Linux64 Opt TC R1 12m 10m -2m
Linux64 Opt TC R2 10m 7m -3m
Linux64 Opt TC R3 11m 9m -2m
Linux64 Opt TC R4 11m 9m -2m
Linux64 Opt TC R5 13m 11m -2m
Linux64 Opt TC R6 11m 9m -2m
Linux64 Opt TC R7 9m 8m -1m
Linux64 Opt TC R8 11m 10m -1m
Linux64 Opt TC Ru1 30m 25m -5m
Linux64 Opt TC Ru2 36m 27m -11m
OS X 10.10 Opt 31m 27m -4m
OS X 10.10 Opt E10s 26m 25m -1m
OS X 10.10 Debug 68m 55m -13m
Win7 Opt R 30m 28m -2m
Win7 Opt Ru 28m 26m -2m
Win7 Opt R E10S 29m 27m -2m
Win7 Debug R 85m 76m -9m
Win7 Debug R E10S 75m 65m -10m
Win8 x64 Opt R 29m 26m -3m
Win8 x64 Opt Ru 27m 25m -2m
Win8 x64 Debug R 90m 71m -19m
Android 4.3 API15 Opt R1 89m 71m -18m
Android 4.3 API15 Opt R2 78m 64m -14m
Android 4.3 API15 Opt R3 75m 64m -11m
Android 4.3 API15 Opt R4 74m 68m -6m
Android 4.3 API15 Opt R5 75m 69m -6m
Android 4.3 API15 Opt R6 91m 86m -5m
Android 4.3 API15 Opt R7 87m 66m -21m
Android 4.3 API15 Opt R8 87m 82m -5m
Android 4.3 API15 Opt R9 80m 66m -14m
Android 4.3 API15 Opt R10 80m 67m -13m
Android 4.3 API15 Opt R11 73m 66m -7m
Android 4.3 API15 Opt R12 105m 91m -14m
Android 4.3 API15 Opt R13 72m 59m -13m
Android 4.3 API15 Opt R14 82m 61m -21m
Android 4.3 API15 Opt R15 73m 62m -11m
Android 4.3 API15 Opt R16 79m 78m -1m
The savings total 6+ *hours* or ~15% when running all reftests. I'd
say this isn't bad for a one-line code change!
MozReview-Commit-ID: H1LkACgSpVn
--HG--
extra : rebase_source : 891a5ce8e1f6c3d70fc646f116c2f49f897ad735
As of bug 1245092, reftest depends on marionette. Normally marionette client has an internal timeout of around
60 seconds where it waits for the server to become available. But when using debuggers (and especially valgrind),
it can take much much longer than a minute for Firefox to start. This patch adds a couple hidden command line
args to reftest to tweak the marionette timeouts.
MozReview-Commit-ID: 3xF0InBJNEf
--HG--
extra : rebase_source : f18092bc90a7d8aab34b527a15bd3b1dc6afc997
This extra window was initially left open because closing it was causing memory leaks
on debug e10s crashtests. There was pressure to get the regressing patch landed due
to addon signing, so it got landed with this extra window hanging around (as it didn't
impact test results).
But it is a UX wart for several reasons. Upon testing it again recently, the leaks all
seem to have vanished. I'm not sure why, possibly it was a bug fixed in e10s.
MozReview-Commit-ID: CEI2enKAOyv
--HG--
extra : rebase_source : da0e8889f67d57aa61670105c14c2235607d79a7
This is a internal-only syntax for guarding rules from a boolean
preference. Nothing causes @supports rules to be re-evaluated except
html.css registered in Part 2.
This is needed for rendering the disclosure triangle of the
summary element by using "display: list-item".
Usage example:
@supports -moz-bool-pref("dom.details_element.enabled") {
/* css rules */
}
MozReview-Commit-ID: HDCa8zHxYTA
--HG--
extra : rebase_source : b7a72a48166edf1d486014ff37363ed8be9127d9
Be warned. Do not attemp to change the .js "test" source code in ./js
They are meant to check
- the outdated 0666 octal constant is still parsed correctly,
- the outdated 0666 octal constant raises syntax error flag
in strict mode, etc.
So leave them alone.
There is an ImportError on Android, as well as a log related
regression from the structured log patch once that is fixed.
MozReview-Commit-ID: KxSEotr38qO
--HG--
extra : rebase_source : 15d8421aab813d9e0dbf6d00611f921aaa779a49
Structured logs bring many benefits. We can stop parsing the logs for magic strings, we
can modify the format without breaking things, and we can stream results into systems like
ActiveData. The structured logs originate primarily in reftest.js. StructuredLog.jsm is
used to generate the JSON-based log stream. Finally OutputHandler in the python harness
reads structured output from stdout, and formats it into human readable form.
MozReview-Commit-ID: G3ZLkMRl6p7
--HG--
extra : commitid : J3ui9XYWR3Q
extra : rebase_source : 77ed0ba842cc8e557141fb3494212b06868c728a
extra : amend_source : 735d48225a2e627e0fe45fc11b50b6c49a885a4b
extra : source : d1779fe421c3c7cd8e3d191816776390dc104f37
Structured logs bring many benefits. We can stop parsing the logs for magic strings, we
can modify the format without breaking things, and we can stream results into systems like
ActiveData. The structured logs originate primarily in reftest.js. StructuredLog.jsm is
used to generate the JSON-based log stream. Finally OutputHandler in the python harness
reads structured output from stdout, and formats it into human readable form.
--HG--
extra : commitid : J3ui9XYWR3Q
extra : rebase_source : 6bae978126dbd5beddc39332c7cbce0c1354cd87
extra : amend_source : 735d48225a2e627e0fe45fc11b50b6c49a885a4b
Also, add an opt-out for crashtest/reftest for the view-source thing so they don't all break, r=bz
--HG--
extra : commitid : 8NqvmbphSgh
extra : rebase_source : bbe0b6f11a77d7e6241a5733931d9baa95bb3fed
All the files modified are straightforward deletion except TouchManager
and ZoomConstraintsClient. I add some includes and wrap TouchManager by
mozilla namespace to fix build errors due to the removal of TouchCaret.
All the files modified are straightforward deletion except TouchManager
and ZoomConstraintsClient. I add some includes and wrap TouchManager by
mozilla namespace to fix build errors due to the removal of TouchCaret.
--HG--
extra : rebase_source : b31db322130f665e7dda53d1061cfc40f81ce411
This makes it clearer that really it's the same thing as FINAL_TARGET,
with preprocessing.
We still keep DIST_FILES in backend.mk because it's shorter and doesn't
really matter.
The bulk of this commit was generated with a script, executed at the top
level of a typical source code checkout. The only non-machine-generated
part was modifying MFBT's moz.build to reflect the new naming.
CLOSED TREE makes big refactorings like this a piece of cake.
# The main substitution.
find . -name '*.cpp' -o -name '*.cc' -o -name '*.h' -o -name '*.mm' -o -name '*.idl'| \
xargs perl -p -i -e '
s/nsRefPtr\.h/RefPtr\.h/g; # handle includes
s/nsRefPtr ?</RefPtr</g; # handle declarations and variables
'
# Handle a special friend declaration in gfx/layers/AtomicRefCountedWithFinalize.h.
perl -p -i -e 's/::nsRefPtr;/::RefPtr;/' gfx/layers/AtomicRefCountedWithFinalize.h
# Handle nsRefPtr.h itself, a couple places that define constructors
# from nsRefPtr, and code generators specially. We do this here, rather
# than indiscriminantly s/nsRefPtr/RefPtr/, because that would rename
# things like nsRefPtrHashtable.
perl -p -i -e 's/nsRefPtr/RefPtr/g' \
mfbt/nsRefPtr.h \
xpcom/glue/nsCOMPtr.h \
xpcom/base/OwningNonNull.h \
ipc/ipdl/ipdl/lower.py \
ipc/ipdl/ipdl/builtin.py \
dom/bindings/Codegen.py \
python/lldbutils/lldbutils/utils.py
# In our indiscriminate substitution above, we renamed
# nsRefPtrGetterAddRefs, the class behind getter_AddRefs. Fix that up.
find . -name '*.cpp' -o -name '*.h' -o -name '*.idl' | \
xargs perl -p -i -e 's/nsRefPtrGetterAddRefs/RefPtrGetterAddRefs/g'
if [ -d .git ]; then
git mv mfbt/nsRefPtr.h mfbt/RefPtr.h
else
hg mv mfbt/nsRefPtr.h mfbt/RefPtr.h
fi
--HG--
rename : mfbt/nsRefPtr.h => mfbt/RefPtr.h
Haik Aftandilian
Kartikaya Gupta
J. Ryan Stinnett
Sylvestre Ledru
Thomas Nguyen
James Graham
Alex Gaynor
Carsten "Tomcat" Book
David Anderson
Wes Kocher
Andrew Halberstadt
Olli Pettay
Xidorn Quan
Dão Gottwald
Andrew Swan
Shing Lyu
Mike Cooper
Francois Marier
Geoff Brown
Bobby Holley
Ed Lee
Benjamin Smedberg
Sebastian Hengst
Florian Queze
Andreas Tolfsen
Makoto Kato
Phil Ringnalda
Tooru Fujisawa
Mason Chang
L. David Baron
Matt Woodrow
Andrea Marchesini
Boris Zbarsky
Deepa
Ryan Hunt
Jet Villegas
Mark Banner
Ehsan Akhgari
Morris Tseng
Henrik Skupin
Gijs Kruitbosch
Masatoshi Kimura
Shing Lyu
Michelangelo De Simone
Cameron McCormack
Alastor Wu
Kris Maglione
Nick Alexander
Robert Strong
Ryan VanderMeulen
Eric Rahm
Jim Mathies
Dale Harvey
Terrence Cole
Xidorn Quan
Andrew McCreight
Seth Fowler
Tom Tromey
Brad Werth
Krishna
Jonathan Hao
Chris Peterson
Gregory Szorc
Mike Shal
Daniel Holbert
Hiroyuki Ikezoe
Joel Maher
Ting-Yu Lin
Tobias Schneider
Xidorn Quan
Stanley Mwangi
Felipe Gomes
Milan Sreckovic
chaithanya
Edwin Flores
ISHIKAWA, Chiaki
Kyle Huey
Jerry Shih
Benoit Girard
Christoph Kerschbaumer
Bob Clary
Lee Salzman
Nigel Babu
Geoffrey Sneddon
Mike Hommey
dlee
Julien Pagès
Nathan Froyd