This backs out all work from bug 1627075 as well as all of its
descendents. There were a few conflicts when backing this out but
overall it was pretty clean, so I would say it's a fairly mild
level of risk. Historically Nathan Froyd has reviewed these patches,
but he is no longer at Mozilla, and no one else is particularly
familiar with the code, so I am passing this off to RyanVM who has
at least been familiar with the history of the bug.
Differential Revision: https://phabricator.services.mozilla.com/D90096
Opening our Omnijars can be expensive, and it should be deferrable until after
startup is completed, provided we have a startup cache. In a previous patch in this
stack, we implemented caching of the zip central directory for omnijars, but we
still have to open the file in order to hand the object off to various omnijar
consumers. In a later patch, we will wrap nsZipArchive access in a class which
will allow us to transparently cache nsZipArchive results. These two get us
most of the way to not needing to read from the underlying omnijar files during
startup, but there are still nontrivial pieces, like nsZipFind for instance,
which we don't want to just duplicate inside of a wrapper class, so we would
like to sort out a way in which we can use an nsZipArchive class, but not
actually back it up with the real underlying file until we really need data
from it which we can't find in a cache.
Depends on D77633
Differential Revision: https://phabricator.services.mozilla.com/D78584
We would like to be able to defer opening the omnijar files until after startup
if the StartupCache has already been populated. Opening the omnijar files takes
a nontrivial time, at least on Windows, and almost everything in the omnijar
should be fairly compressible, and thus makes sense to live in the StartupCache.
See the last patch in this series for a little more discussion on numbers, but
tl;dr: we saw a 12% improvement in time to about:home being finished on reference
hardware with these changes together with the changes from the descendant patches.
Differential Revision: https://phabricator.services.mozilla.com/D77632
Opening our Omnijars can be expensive, and it should be deferrable until after
startup is completed, provided we have a startup cache. In a previous patch in this
stack, we implemented caching of the zip central directory for omnijars, but we
still have to open the file in order to hand the object off to various omnijar
consumers. In a later patch, we will wrap nsZipArchive access in a class which
will allow us to transparently cache nsZipArchive results. These two get us
most of the way to not needing to read from the underlying omnijar files during
startup, but there are still nontrivial pieces, like nsZipFind for instance,
which we don't want to just duplicate inside of a wrapper class, so we would
like to sort out a way in which we can use an nsZipArchive class, but not
actually back it up with the real underlying file until we really need data
from it which we can't find in a cache.
Depends on D77633
Differential Revision: https://phabricator.services.mozilla.com/D78584
We would like to be able to defer opening the omnijar files until after startup
if the StartupCache has already been populated. Opening the omnijar files takes
a nontrivial time, at least on Windows, and almost everything in the omnijar
should be fairly compressible, and thus makes sense to live in the StartupCache.
See the last patch in this series for a little more discussion on numbers, but
tl;dr: we saw a 12% improvement in time to about:home being finished on reference
hardware with these changes together with the changes from the descendant patches.
Differential Revision: https://phabricator.services.mozilla.com/D77632
Opening our Omnijars can be expensive, and it should be deferrable until after
startup is completed, provided we have a startup cache. In a previous patch in this
stack, we implemented caching of the zip central directory for omnijars, but we
still have to open the file in order to hand the object off to various omnijar
consumers. In a later patch, we will wrap nsZipArchive access in a class which
will allow us to transparently cache nsZipArchive results. These two get us
most of the way to not needing to read from the underlying omnijar files during
startup, but there are still nontrivial pieces, like nsZipFind for instance,
which we don't want to just duplicate inside of a wrapper class, so we would
like to sort out a way in which we can use an nsZipArchive class, but not
actually back it up with the real underlying file until we really need data
from it which we can't find in a cache.
Depends on D77633
Differential Revision: https://phabricator.services.mozilla.com/D78584
We would like to be able to defer opening the omnijar files until after startup
if the StartupCache has already been populated. Opening the omnijar files takes
a nontrivial time, at least on Windows, and almost everything in the omnijar
should be fairly compressible, and thus makes sense to live in the StartupCache.
See the last patch in this series for a little more discussion on numbers, but
tl;dr: we saw a 12% improvement in time to about:home being finished on reference
hardware with these changes together with the changes from the descendant patches.
Differential Revision: https://phabricator.services.mozilla.com/D77632
Opening our Omnijars can be expensive, and it should be deferrable until after
startup is completed, provided we have a startup cache. In a previous patch in this
stack, we implemented caching of the zip central directory for omnijars, but we
still have to open the file in order to hand the object off to various omnijar
consumers. In a later patch, we will wrap nsZipArchive access in a class which
will allow us to transparently cache nsZipArchive results. These two get us
most of the way to not needing to read from the underlying omnijar files during
startup, but there are still nontrivial pieces, like nsZipFind for instance,
which we don't want to just duplicate inside of a wrapper class, so we would
like to sort out a way in which we can use an nsZipArchive class, but not
actually back it up with the real underlying file until we really need data
from it which we can't find in a cache.
Depends on D77633
Differential Revision: https://phabricator.services.mozilla.com/D78584
We would like to be able to defer opening the omnijar files until after startup
if the StartupCache has already been populated. Opening the omnijar files takes
a nontrivial time, at least on Windows, and almost everything in the omnijar
should be fairly compressible, and thus makes sense to live in the StartupCache.
See the last patch in this series for a little more discussion on numbers, but
tl;dr: we saw a 12% improvement in time to about:home being finished on reference
hardware with these changes together with the changes from the descendant patches.
Differential Revision: https://phabricator.services.mozilla.com/D77632
Opening our Omnijars can be expensive, and it should be deferrable until after
startup is completed, provided we have a startup cache. In a previous patch in this
stack, we implemented caching of the zip central directory for omnijars, but we
still have to open the file in order to hand the object off to various omnijar
consumers. In a later patch, we will wrap nsZipArchive access in a class which
will allow us to transparently cache nsZipArchive results. These two get us
most of the way to not needing to read from the underlying omnijar files during
startup, but there are still nontrivial pieces, like nsZipFind for instance,
which we don't want to just duplicate inside of a wrapper class, so we would
like to sort out a way in which we can use an nsZipArchive class, but not
actually back it up with the real underlying file until we really need data
from it which we can't find in a cache.
Depends on D77633
Differential Revision: https://phabricator.services.mozilla.com/D78584
We would like to be able to defer opening the omnijar files until after startup
if the StartupCache has already been populated. Opening the omnijar files takes
a nontrivial time, at least on Windows, and almost everything in the omnijar
should be fairly compressible, and thus makes sense to live in the StartupCache.
See the last patch in this series for a little more discussion on numbers, but
tl;dr: we saw a 12% improvement in time to about:home being finished on reference
hardware with these changes together with the changes from the descendant patches.
Differential Revision: https://phabricator.services.mozilla.com/D77632
Please double check that I am using this correctly. I believe we are
seeing the crash in the linked bug because we are not handling hardware
faults when reading from the memory mapped file. This patch just wraps
all accesses in the MMAP_FAULT_HANDLER_ macros.
Depends on D53042
Differential Revision: https://phabricator.services.mozilla.com/D53043
--HG--
rename : modules/libjar/MmapFaultHandler.cpp => mozglue/misc/MmapFaultHandler.cpp
rename : modules/libjar/MmapFaultHandler.h => mozglue/misc/MmapFaultHandler.h
extra : moz-landing-system : lando
Please double check that I am using this correctly. I believe we are
seeing the crash in the linked bug because we are not handling hardware
faults when reading from the memory mapped file. This patch just wraps
all accesses in the MMAP_FAULT_HANDLER_ macros.
Depends on D53042
Differential Revision: https://phabricator.services.mozilla.com/D53043
--HG--
rename : modules/libjar/MmapFaultHandler.cpp => mozglue/misc/MmapFaultHandler.cpp
rename : modules/libjar/MmapFaultHandler.h => mozglue/misc/MmapFaultHandler.h
extra : moz-landing-system : lando
A global instance of ZipArchiveLogger is used on multiple threads when the logging is turned on. This patch adds a lock to synchronize the access.
Differential Revision: https://phabricator.services.mozilla.com/D60537
--HG--
extra : moz-landing-system : lando
Temporarily just sidestep the issue in bug 1546498 (crash with latest SDK
on startup in Windows 7) by just continuing to use the old method in
Windows 7. We saw no wins in telemetry for Windows 7 anyway, so we should
investigate why that is, and why we see a mysterious crash in the fallback
code, in a followup bug.
Differential Revision: https://phabricator.services.mozilla.com/D29239
--HG--
extra : moz-landing-system : lando
Temporarily just sidestep the issue in bug 1546498 (crash with latest SDK
on startup in Windows 7) by just continuing to use the old method in
Windows 7. We saw no wins in telemetry for Windows 7 anyway, so we should
investigate why that is, and why we see a mysterious crash in the fallback
code, in a followup bug.
Differential Revision: https://phabricator.services.mozilla.com/D29239
--HG--
extra : moz-landing-system : lando
This is just to make it simpler to use PrefetchVirtualMemory in subsequent patches.
Differential Revision: https://phabricator.services.mozilla.com/D26016
--HG--
extra : moz-landing-system : lando
This is just to make it simpler to use PrefetchVirtualMemory in subsequent patches.
Differential Revision: https://phabricator.services.mozilla.com/D26016
--HG--
extra : moz-landing-system : lando
Re-enabling the PGO jarlog, which was unexpectedly disabled since Firefox 56
showed a regression on Windows 7, due to the use of mozilla::ReadAhead,
which on Windows 7 does explicit I/O on the caller thread.
While there may be some benefit from doing so, evidence says the
opposite, which is presumably due to the amount of data being loaded not
being relevant in every case: the jarlog is gathered from a first-run,
which has a different jar-loading profile from subsequent runs of
Firefox.
While we may want to improve the situation later on, the immediate thing
we can do is stop doing this explicit read, while keeping the OS
readahead hints on other platforms, which don't imply explicit I/O.
All this does is effectively get us back to the same state as if jarlogs
were disabled like it was since Firefox 56, for Windows 7 only.
aFd not being used anymore, the code could be cleaned up a lot, but we
may reintroduce the readahead later, so keep the status quo for now.
Differential Revision: https://phabricator.services.mozilla.com/D23642
--HG--
extra : moz-landing-system : lando
The jar log is used for optimization of the packaged jar files according
to their usage patterns during a profile run. The current content of the
file currently come with 2 caveats:
- it contains entries for jar archives that aren't relevant to
packaging, which is not a problem in itself, but see below.
- it contains full paths for jar archives that may not correspond to the
location of the packaged directory (on e.g. Android, where the build
almost certainly doesn't happen in the same directory on the host as
Fennec runs in the emulator/on the device).
The current JarLog code does somehow handle the various ways paths are
currently presented, but it's clearly missing code to map the paths in
the log to packaged paths. Instead of requiring manual work and extra
build options to handle this mapping, and considering the caveats above,
it's just simpler to log archive paths as if they were relative to the
packaged application directory in a build, and use that during
packaging.
Depends on D21655
Differential Revision: https://phabricator.services.mozilla.com/D21656
--HG--
extra : moz-landing-system : lando
Instead of checking the MOZ_JAR_LOG_FILE for each log entry, only check
it once, and only check whether to log once per archive rather than once
per item.
Differential Revision: https://phabricator.services.mozilla.com/D21655
--HG--
extra : moz-landing-system : lando
The jar log is used for optimization of the packaged jar files according
to their usage patterns during a profile run. The current content of the
file currently come with 2 caveats:
- it contains entries for jar archives that aren't relevant to
packaging, which is not a problem in itself, but see below.
- it contains full paths for jar archives that may not correspond to the
location of the packaged directory (on e.g. Android, where the build
almost certainly doesn't happen in the same directory on the host as
Fennec runs in the emulator/on the device).
The current JarLog code does somehow handle the various ways paths are
currently presented, but it's clearly missing code to map the paths in
the log to packaged paths. Instead of requiring manual work and extra
build options to handle this mapping, and considering the caveats above,
it's just simpler to log archive paths as if they were relative to the
packaged application directory in a build, and use that during
packaging.
Depends on D21655
Differential Revision: https://phabricator.services.mozilla.com/D21656
--HG--
extra : moz-landing-system : lando
Instead of checking the MOZ_JAR_LOG_FILE for each log entry, only check
it once, and only check whether to log once per archive rather than once
per item.
Differential Revision: https://phabricator.services.mozilla.com/D21655
--HG--
extra : moz-landing-system : lando
instead of ReadAhead, which does active reading on Windows.
The PrefetchVirtualMemory is a system call that is new to Windows 8.
Back when the readahead code for jar was added, in bug 810151, Windows 8
was still fresh out of the oven. 6 years later, we can do a runtime
check and use the PrefetchVirtualMemory function.
One downside is that the IOInterposer doesn't know about it (but it
doesn't know about madvise on other platforms anyways).
Differential Revision: https://phabricator.services.mozilla.com/D20807
--HG--
extra : moz-landing-system : lando
Consequently, since we don't readahead jars in child processes, we
don't record jar accesses from child processes. In fact, intertwining
jar accesses from child processes with those from the parent process was
arguably making things less efficient for the parent process. But this
code was written at a time e10s was barely a thing, so it wasn't really
thought for a multi-process world.
Differential Revision: https://phabricator.services.mozilla.com/D20752
--HG--
extra : moz-landing-system : lando
This backs out the main patch landed earlier in bug 1194856 and the
patch from bug 1225004.
Differential Revision: https://phabricator.services.mozilla.com/D14050
--HG--
extra : moz-landing-system : lando
Doug Thayer
Narcis Beleuzu
Mihai Alexandru Michis
Noemi Erli
Bas Schouten
Dorel Luca
anjali1903
Eric Rahm
Michal Novotny
Bogdan Tara
Gijs Kruitbosch
Sylvestre Ledru
Cosmin Sabou
Mike Hommey
Gurzau Raul
Ehsan Akhgari
Cameron McCormack
Andi-Bogdan Postelnicu
Sebastian Hengst
Tristan Bourvon