The exact circumstances of how this is showing up in the wild aren't
clear - there seem to be a couple of ways we can get here. However it
all revolves around early shutdowns (i.e., from the select profile popup)
- before the StartupCache is ever initialized. In any case, the solution
shouldn't change based on the exact circumstances - if we don't have a
StartupCache, there's no need to write one. Also, don't bother lazy
initializing it if it doesn't exist yet.
Differential Revision: https://phabricator.services.mozilla.com/D63208
--HG--
extra : moz-landing-system : lando
Reordering the mWrittenOnce check should be sufficient to eliminate
the data race; however, I made mWrittenOnce an atomic just to reduce
the fragility of this since it is intended to be written from and
read to on multiple threads.
Differential Revision: https://phabricator.services.mozilla.com/D62949
--HG--
extra : moz-landing-system : lando
This was just left in, and does not need to be here. We want to be
spawning the background thread here which we will wait on from
xpcom-shutdown.
Differential Revision: https://phabricator.services.mozilla.com/D62848
--HG--
extra : moz-landing-system : lando
This was just left in, and does not need to be here. We want to be
spawning the background thread here which we will wait on from
xpcom-shutdown.
Differential Revision: https://phabricator.services.mozilla.com/D62848
--HG--
extra : moz-landing-system : lando
The initial thought for getting the StartupCache out of the shutdown
path was to simply not write it during shutdown, as it should write
60 seconds after startup, and the theory was that if the user shut
down within the first 60 seconds of use, they were likely updating or
something and it shouldn't matter. However, considering how many of
our users only ever open one tab, I think it's rather likely that
users are starting up firefox to go to a web site, then closing it
when done with that website, and then maybe opening up a new instance
in order to go to a different website. Accordingly it still makes
sense to continue writing it. However, we may as well leverage a
background thread for this and get it kicked off earlier during
shutdown, so we don't sit there blocking in the destructor late
during shutdown.
Differential Revision: https://phabricator.services.mozilla.com/D62294
--HG--
extra : source : 7b7b147b6955cee07e0c115993446bfbd59cf7e2
extra : histedit_source : 6990122d6b1ac4939b0e4b0a5e452183fb981e19
The initial thought for getting the StartupCache out of the shutdown
path was to simply not write it during shutdown, as it should write
60 seconds after startup, and the theory was that if the user shut
down within the first 60 seconds of use, they were likely updating or
something and it shouldn't matter. However, considering how many of
our users only ever open one tab, I think it's rather likely that
users are starting up firefox to go to a web site, then closing it
when done with that website, and then maybe opening up a new instance
in order to go to a different website. Accordingly it still makes
sense to continue writing it. However, we may as well leverage a
background thread for this and get it kicked off earlier during
shutdown, so we don't sit there blocking in the destructor late
during shutdown.
Differential Revision: https://phabricator.services.mozilla.com/D62294
--HG--
extra : moz-landing-system : lando
The initial thought for getting the StartupCache out of the shutdown
path was to simply not write it during shutdown, as it should write
60 seconds after startup, and the theory was that if the user shut
down within the first 60 seconds of use, they were likely updating or
something and it shouldn't matter. However, considering how many of
our users only ever open one tab, I think it's rather likely that
users are starting up firefox to go to a web site, then closing it
when done with that website, and then maybe opening up a new instance
in order to go to a different website. Accordingly it still makes
sense to continue writing it. However, we may as well leverage a
background thread for this and get it kicked off earlier during
shutdown, so we don't sit there blocking in the destructor late
during shutdown.
Differential Revision: https://phabricator.services.mozilla.com/D62294
--HG--
extra : moz-landing-system : lando
We can't guarantee uniqueness of the keys here because the file might
be corrupt. So we should check if the key exists and if it does, bail
out.
Differential Revision: https://phabricator.services.mozilla.com/D58387
--HG--
extra : moz-landing-system : lando
The inclusions were removed with the following very crude script and the
resulting breakage was fixed up by hand. The manual fixups did either
revert the changes done by the script, replace a generic header with a more
specific one or replace a header with a forward declaration.
find . -name "*.idl" | grep -v web-platform | grep -v third_party | while read path; do
interfaces=$(grep "^\(class\|interface\).*:.*" "$path" | cut -d' ' -f2)
if [ -n "$interfaces" ]; then
if [[ "$interfaces" == *$'\n'* ]]; then
regexp="\("
for i in $interfaces; do regexp="$regexp$i\|"; done
regexp="${regexp%%\\\|}\)"
else
regexp="$interfaces"
fi
interface=$(basename "$path")
rg -l "#include.*${interface%%.idl}.h" . | while read path2; do
hits=$(grep -v "#include.*${interface%%.idl}.h" "$path2" | grep -c "$regexp" )
if [ $hits -eq 0 ]; then
echo "Removing ${interface} from ${path2}"
grep -v "#include.*${interface%%.idl}.h" "$path2" > "$path2".tmp
mv -f "$path2".tmp "$path2"
fi
done
fi
done
Differential Revision: https://phabricator.services.mozilla.com/D55444
--HG--
extra : moz-landing-system : lando
There is no behavior change here - these asserts were previously being hit
inside the operator+ of RangedPtr, but this makes them explicit.
Differential Revision: https://phabricator.services.mozilla.com/D49985
--HG--
extra : moz-landing-system : lando
In bug 1550108 we added a thread to prefetch the contents of the
startup cache file, and as part of that we reduced its stack size
and the stack size of the thread which writes the file. However
it seems we set the write thread's stack size too low.
Differential Revision: https://phabricator.services.mozilla.com/D48570
--HG--
extra : moz-landing-system : lando
Does what it says on the tin. Once we have a central scheduling
system this should likely just consume that.
Differential Revision: https://phabricator.services.mozilla.com/D35454
--HG--
extra : moz-landing-system : lando
The first run loads more things into the StartupCache than are
used on the second and subsequent runs. This just ensures that if
the StartupCache diverges too far from its actual use that we will
rebuild it.
Differential Revision: https://phabricator.services.mozilla.com/D34654
--HG--
extra : moz-landing-system : lando
The signatures were updated in the previous patch to hand us the raw,
uncopied buffers. This just adjusts the callsites to match.
Differential Revision: https://phabricator.services.mozilla.com/D34653
--HG--
extra : moz-landing-system : lando
I am not aware of anything that depends on StartupCache being a
zip file, and since I want to use lz4 compression because inflate
is showing up quite a lot in profiles, it's simplest to just use
a custom format. This loosely mimicks the ScriptPreloader code,
with a few diversions:
- Obviously the contents of the cache are compressed. I used lz4
for this as I hit the same file size as deflate at a compression
level of 1, which is what the StartupCache was using previously,
while decompressing an order of magnitude faster. Seemed like
the most conservative change to make. I think it's worth
investigating what the impact of slower algs with higher ratios
would be, but for right now I settled on this. We'd probably
want to look at zstd next.
- I use streaming compression for this via lz4frame. This is not
strictly necessary, but has the benefit of not requiring as
much memory for large buffers, as well as giving us a built-in
checksum, rather than relying on the much slower CRC that we
were doing with the zip-based approach.
- I coded the serialization of the headers inline, since I had to
jump back to add the offset and compressed size, which would
make the nice Code(...) method for the ScriptPreloader stuff
rather more complex. Open to cleaner solutions, but moving it
out just felt like extra hoops for the reader to jump through
to understand without the benefit of being more concise.
Differential Revision: https://phabricator.services.mozilla.com/D34652
--HG--
extra : moz-landing-system : lando
This will not behave exactly the same if we had previously written bad
data for the entry that would fail to decompress. I imagine this is rare
enough, and the consequences are not severe enough, that this should be
fine.
Differential Revision: https://phabricator.services.mozilla.com/D30643
--HG--
extra : moz-landing-system : lando
I thought I had already written out the patch to remove these, but
apparently not. Per discussion in the startup cache telemetry bug,
there should be no reason for doing this.
Differential Revision: https://phabricator.services.mozilla.com/D31491
--HG--
extra : moz-landing-system : lando
Does what it says on the tin. Once we have a central scheduling
system this should likely just consume that.
Differential Revision: https://phabricator.services.mozilla.com/D35454
--HG--
extra : moz-landing-system : lando
The first run loads more things into the StartupCache than are
used on the second and subsequent runs. This just ensures that if
the StartupCache diverges too far from its actual use that we will
rebuild it.
Differential Revision: https://phabricator.services.mozilla.com/D34654
--HG--
extra : moz-landing-system : lando
The signatures were updated in the previous patch to hand us the raw,
uncopied buffers. This just adjusts the callsites to match.
Differential Revision: https://phabricator.services.mozilla.com/D34653
--HG--
extra : moz-landing-system : lando
I am not aware of anything that depends on StartupCache being a
zip file, and since I want to use lz4 compression because inflate
is showing up quite a lot in profiles, it's simplest to just use
a custom format. This loosely mimicks the ScriptPreloader code,
with a few diversions:
- Obviously the contents of the cache are compressed. I used lz4
for this as I hit the same file size as deflate at a compression
level of 1, which is what the StartupCache was using previously,
while decompressing an order of magnitude faster. Seemed like
the most conservative change to make. I think it's worth
investigating what the impact of slower algs with higher ratios
would be, but for right now I settled on this. We'd probably
want to look at zstd next.
- I use streaming compression for this via lz4frame. This is not
strictly necessary, but has the benefit of not requiring as
much memory for large buffers, as well as giving us a built-in
checksum, rather than relying on the much slower CRC that we
were doing with the zip-based approach.
- I coded the serialization of the headers inline, since I had to
jump back to add the offset and compressed size, which would
make the nice Code(...) method for the ScriptPreloader stuff
rather more complex. Open to cleaner solutions, but moving it
out just felt like extra hoops for the reader to jump through
to understand without the benefit of being more concise.
Differential Revision: https://phabricator.services.mozilla.com/D34652
--HG--
extra : moz-landing-system : lando
This will not behave exactly the same if we had previously written bad
data for the entry that would fail to decompress. I imagine this is rare
enough, and the consequences are not severe enough, that this should be
fine.
Differential Revision: https://phabricator.services.mozilla.com/D30643
--HG--
extra : moz-landing-system : lando
I thought I had already written out the patch to remove these, but
apparently not. Per discussion in the startup cache telemetry bug,
there should be no reason for doing this.
Differential Revision: https://phabricator.services.mozilla.com/D31491
--HG--
extra : moz-landing-system : lando
This limits us to 1 preloaded browser per window, in the top 3 normal windows + top 3 private windows.
If we try to create additional browsers beyond that, we instead move a pre-existing browser across.
Differential Revision: https://phabricator.services.mozilla.com/D21129
--HG--
extra : moz-landing-system : lando
This limits us to 1 preloaded browser per window, in the top 3 normal windows + top 3 private windows.
If we try to create additional browsers beyond that, we instead move a pre-existing browser across.
Differential Revision: https://phabricator.services.mozilla.com/D21129
--HG--
extra : moz-landing-system : lando
In bug 1264235 we have some indication that observed bugs with the
startup cache might have been resolved, but we don't really know
until we collect data. Collecting these stats will give us the
ability to have more certainty that the startup cache is functioning
correctly in the wild.
Differential Revision: https://phabricator.services.mozilla.com/D19573
--HG--
extra : moz-landing-system : lando
This is a best effort attempt at ensuring that the adverse impact of
reformatting the entire tree over the comments would be minimal. I've used a
combination of strategies including disabling of formatting, some manual
formatting and some changes to formatting to work around some clang-format
limitations.
Differential Revision: https://phabricator.services.mozilla.com/D13371
--HG--
extra : moz-landing-system : lando
If class A is derived from class B, then an instance of class A can be
converted to B via a static cast, so a slower QI is not needed.
Differential Revision: https://phabricator.services.mozilla.com/D6861
--HG--
extra : moz-landing-system : lando
This was done automatically replacing:
s/mozilla::Move/std::move/
s/ Move(/ std::move(/
s/(Move(/(std::move(/
Removing the 'using mozilla::Move;' lines.
And then with a few manual fixups, see the bug for the split series..
MozReview-Commit-ID: Jxze3adipUh
This switches over to using `LookupForAdd` which allows us to avoid a second
lookup when adding the entry. Addtionally `nsDependentCString` is used to
avoid copying the id string when looking up the entry.
--HG--
extra : rebase_source : 2dbb54fb992f4c79347150198af61438ee5b4eec
extra : source : 718961d941d4d495426e59c6b5044302059489f9
This avoids a redundant alloc and copy in `PutBuffer`. All existing callers
were destroying the passed in buffer after the call.
--HG--
extra : rebase_source : 065505219d70d26bad49c7eba2cec8edf0e9939d
extra : amend_source : 118eddad4dc901da02817c788fb98f6f4c85a3f0
extra : source : 7f0cedfb4bd85bfe1a523168019864c9c6c0e665
This switches over to using `LookupForAdd` which allows us to avoid a second
lookup when adding the entry. Addtionally `nsDependentCString` is used to
avoid copying the id string when looking up the entry.
--HG--
extra : rebase_source : e929431b4ab277345e455fb0b220a8aa6839b37f
This avoids a redundant alloc and copy in `PutBuffer`. All existing callers
were destroying the passed in buffer after the call.
--HG--
extra : rebase_source : 39a21686becedf32c38e58fa832ae47845b2f5e0
* changes call to use nsIURIMutator.setSpec()
* Add new NS_MutateURI constructor that takes new Mutator object
* Make nsSimpleNestedURI::Mutate() and nsNestedAboutURI::Mutate() return mutable URIs
* Make the finalizers for nsSimpleNestedURI and nsNestedAboutURI make the returned URIs immutable
MozReview-Commit-ID: 1kcv6zMxnv7
--HG--
extra : rebase_source : 99b13e9dbc8eaaa9615843b05e1539e19b527504
This class was needed for testing, but no longer.
MozReview-Commit-ID: AIk0kKlbScs
--HG--
extra : rebase_source : 35eed188e44dc4c9899479b35882461dd2cf4624
Right now, NS_GENERIC_FACTORY_SINGLETON_CONSTRUCTOR expects singleton
constructors to return already-addrefed raw pointers, and while it accepts
constructors that return already_AddRefed, most existing don't do so.
Meanwhile, the convention elsewhere is that a raw pointer return value is
owned by the callee, and that the caller needs to addref it if it wants to
keep its own reference to it.
The difference in convention makes it easy to leak (I've definitely caused
more than one shutdown leak this way), so it would be better if we required
the singleton getters to return an explicit already_AddRefed, which would
behave the same for all callers.
This also cleans up several singleton constructors that left a dangling
pointer to their singletons when their initialization methods failed, when
they released their references without clearing their global raw pointers.
MozReview-Commit-ID: 9peyG4pRYcr
--HG--
extra : rebase_source : 2f5bd89c17cb554541be38444672a827c1392f3f
Currently the Gecko Profiler defines a moderate amount of stuff when
MOZ_GECKO_PROFILER is undefined. It also #includes various headers, including
JS ones. This is making it difficult to separate Gecko's media stack for
inclusion in Servo.
This patch greatly simplifies how things are exposed. The starting point is:
- GeckoProfiler.h can be #included unconditionally;
- everything else from the profiler must be guarded by MOZ_GECKO_PROFILER.
In practice this introduces way too many #ifdefs, so the patch loosens it by
adding no-op macros for a number of the most common operations.
The net result is that #ifdefs and macros are used a bit more, but almost
nothing is exposed in non-MOZ_GECKO_PROFILER builds (including
ProfilerMarkerPayload.h and GeckoProfiler.h), and understanding what is exposed
is much simpler than before.
Note also that in BHR, ThreadStackHelper is now entirely absent in
non-MOZ_GECKO_PROFILER builds.
This patch makes the following changes to the macros.
- Removes PROFILER_LABEL_FUNC. It's only suitable for use in functions outside
classes, due to PROFILER_FUNCTION_NAME not getting class names, and it was
mostly misused.
- Removes PROFILER_FUNCTION_NAME. It's no longer used, and __func__ is
universally available now anyway.
- Combines the first two string literal arguments of PROFILER_LABEL and
PROFILER_LABEL_DYNAMIC into a single argument. There was no good reason for
them to be separate, and it forced a '::' in the label, which isn't always
appropriate. Also, the meaning of the "name_space" argument was interpreted
in an interesting variety of ways.
- Adds an "AUTO_" prefix to PROFILER_LABEL and PROFILER_LABEL_DYNAMIC, to make
it clearer they construct RAII objects rather than just being function calls.
(I myself have screwed up the scoping because of this in the past.)
- Fills in the 'js::ProfileEntry::Category::' qualifier within the macro, so
the caller doesn't need to. This makes a *lot* more of the uses fit onto a
single line.
The patch also makes the following changes to the macro uses (beyond those
required by the changes described above).
- Fixes a bunch of labels that had gotten out of sync with the name of the
class and/or function that encloses them.
- Removes a useless PROFILER_LABEL use within a trivial scope in
EventStateManager::DispatchMouseOrPointerEvent(). It clearly wasn't serving
any useful purpose. It also serves as extra evidence that the AUTO_ prefix is
a good idea.
- Tweaks DecodePool::SyncRunIf{Preferred,Possible} so that the labelling is
done within them, instead of at their callsites, because that's a more
standard way of doing things.
--HG--
extra : rebase_source : 318d1bc6fc1425a94aacbf489dd46e4f83211de4
This patch gives some structure and order to the profiler's API.
It also renames AutoProfilerRegister as AutoProfilerRegisterThread, to match
profiler_register_thread().
This removes the last uses of PR_smprintf from the tree (excluding the
security and nsprpub directories). It also fixes a related latent bug
in nsAppRunner.cpp (which was incorrectly freeing the pointer passed to
PR_SetEnv).
MozReview-Commit-ID: GynP2PhuWWO
--HG--
extra : rebase_source : c3b83c7bd08b1c222e137a00323caf5481352845
NS_SetCurrentThreadName() is added as an alternative to PR_SetCurrentThreadName()
inside libxul. The thread names are collected in the form of crash annotation to
be processed on socorro.
MozReview-Commit-ID: 4RpAWzTuvPs
As far as I can tell, this covers all the remaining threads which we start
using PR_CreateThread, except the ones that are created inside NSPR or NSS,
and except for the Shutdown Watchdog thread in nsTerminator.cpp and the
CacheIO thread. The Shutdown Watchdog thread stays alive past leak detection
during shutdown (by design), so we'd report leaks if we profiled it. The
CacheIO thread seems to stay alive past shutdown leak detection sometimes as
well.
This adds a AutoProfilerRegister stack class for easy registering and
unregistering. There are a few places where we still call
profiler_register_thread() and profiler_unregister_thread() manually, either
because registration happens conditionally, or because there is a variable that
gets put on the stack before the AutoProfilerRegister (e.g. a dynamically
generated thread name). AutoProfilerRegister needs to be the first object on
the stack because it uses its own `this` pointer as the stack top address.
MozReview-Commit-ID: 3vwhS55Yzt
--HG--
extra : rebase_source : 56dd27282e7bd09a7e7dc7ca09ccfe3a0198e7af
This patch removes checking of all the callback calls in memory reporter
CollectReport() functions, because it's not useful.
The patch also does some associated clean-up.
- Replaces some uses of nsIMemoryReporterCallback with the preferred
nsIHandleReportCallback typedef.
- Replaces aCallback/aCb/aClosure with aHandleRepor/aData for CollectReports()
parameter names, for consistency.
- Adds MOZ_MUST_USE/[must_use] in a few places in nsIMemoryReporter.idl.
- Uses the MOZ_COLLECT_REPORT macro in all suitable places.
Overall the patch reduces code size by ~300 lines and reduces the size of
libxul by about 37 KiB on my Linux64 builds.
--HG--
extra : rebase_source : e94323614bd10463a0c5134a7276238a7ca1cf23
Simon Giesecke
Doug Thayer
Cosmin Sabou
shindli
Gabriele Svelto
Tom Schuster
Razvan Maries
Mark Banner
Mike Conley
Andreea Pavel
Sylvestre Ledru
Gijs Kruitbosch
Alex Gaynor
Ehsan Akhgari
Andrew McCreight
Emilio Cobos Álvarez
Eric Rahm
Narcis Beleuzu
Masatoshi Kimura
Valentin Gosu
Kris Maglione
Nicholas Nethercote
Wes Kocher
Boris Zbarsky
Tom Tromey
Cervantes Yu
Sebastian Hengst
Carsten "Tomcat" Book
David Major
Joel Maher
Bill McCloskey
Barun Parruck
Markus Stange
Andi-Bogdan Postelnicu
Benjamin Smedberg
Iris Hsiao
Bob Clary