A global instance of ZipArchiveLogger is used on multiple threads when the logging is turned on. This patch adds a lock to synchronize the access.
Differential Revision: https://phabricator.services.mozilla.com/D60537
--HG--
extra : moz-landing-system : lando
Temporarily just sidestep the issue in bug 1546498 (crash with latest SDK
on startup in Windows 7) by just continuing to use the old method in
Windows 7. We saw no wins in telemetry for Windows 7 anyway, so we should
investigate why that is, and why we see a mysterious crash in the fallback
code, in a followup bug.
Differential Revision: https://phabricator.services.mozilla.com/D29239
--HG--
extra : moz-landing-system : lando
Temporarily just sidestep the issue in bug 1546498 (crash with latest SDK
on startup in Windows 7) by just continuing to use the old method in
Windows 7. We saw no wins in telemetry for Windows 7 anyway, so we should
investigate why that is, and why we see a mysterious crash in the fallback
code, in a followup bug.
Differential Revision: https://phabricator.services.mozilla.com/D29239
--HG--
extra : moz-landing-system : lando
This is just to make it simpler to use PrefetchVirtualMemory in subsequent patches.
Differential Revision: https://phabricator.services.mozilla.com/D26016
--HG--
extra : moz-landing-system : lando
This is just to make it simpler to use PrefetchVirtualMemory in subsequent patches.
Differential Revision: https://phabricator.services.mozilla.com/D26016
--HG--
extra : moz-landing-system : lando
Re-enabling the PGO jarlog, which was unexpectedly disabled since Firefox 56
showed a regression on Windows 7, due to the use of mozilla::ReadAhead,
which on Windows 7 does explicit I/O on the caller thread.
While there may be some benefit from doing so, evidence says the
opposite, which is presumably due to the amount of data being loaded not
being relevant in every case: the jarlog is gathered from a first-run,
which has a different jar-loading profile from subsequent runs of
Firefox.
While we may want to improve the situation later on, the immediate thing
we can do is stop doing this explicit read, while keeping the OS
readahead hints on other platforms, which don't imply explicit I/O.
All this does is effectively get us back to the same state as if jarlogs
were disabled like it was since Firefox 56, for Windows 7 only.
aFd not being used anymore, the code could be cleaned up a lot, but we
may reintroduce the readahead later, so keep the status quo for now.
Differential Revision: https://phabricator.services.mozilla.com/D23642
--HG--
extra : moz-landing-system : lando
The jar log is used for optimization of the packaged jar files according
to their usage patterns during a profile run. The current content of the
file currently come with 2 caveats:
- it contains entries for jar archives that aren't relevant to
packaging, which is not a problem in itself, but see below.
- it contains full paths for jar archives that may not correspond to the
location of the packaged directory (on e.g. Android, where the build
almost certainly doesn't happen in the same directory on the host as
Fennec runs in the emulator/on the device).
The current JarLog code does somehow handle the various ways paths are
currently presented, but it's clearly missing code to map the paths in
the log to packaged paths. Instead of requiring manual work and extra
build options to handle this mapping, and considering the caveats above,
it's just simpler to log archive paths as if they were relative to the
packaged application directory in a build, and use that during
packaging.
Depends on D21655
Differential Revision: https://phabricator.services.mozilla.com/D21656
--HG--
extra : moz-landing-system : lando
Instead of checking the MOZ_JAR_LOG_FILE for each log entry, only check
it once, and only check whether to log once per archive rather than once
per item.
Differential Revision: https://phabricator.services.mozilla.com/D21655
--HG--
extra : moz-landing-system : lando
The jar log is used for optimization of the packaged jar files according
to their usage patterns during a profile run. The current content of the
file currently come with 2 caveats:
- it contains entries for jar archives that aren't relevant to
packaging, which is not a problem in itself, but see below.
- it contains full paths for jar archives that may not correspond to the
location of the packaged directory (on e.g. Android, where the build
almost certainly doesn't happen in the same directory on the host as
Fennec runs in the emulator/on the device).
The current JarLog code does somehow handle the various ways paths are
currently presented, but it's clearly missing code to map the paths in
the log to packaged paths. Instead of requiring manual work and extra
build options to handle this mapping, and considering the caveats above,
it's just simpler to log archive paths as if they were relative to the
packaged application directory in a build, and use that during
packaging.
Depends on D21655
Differential Revision: https://phabricator.services.mozilla.com/D21656
--HG--
extra : moz-landing-system : lando
Instead of checking the MOZ_JAR_LOG_FILE for each log entry, only check
it once, and only check whether to log once per archive rather than once
per item.
Differential Revision: https://phabricator.services.mozilla.com/D21655
--HG--
extra : moz-landing-system : lando
instead of ReadAhead, which does active reading on Windows.
The PrefetchVirtualMemory is a system call that is new to Windows 8.
Back when the readahead code for jar was added, in bug 810151, Windows 8
was still fresh out of the oven. 6 years later, we can do a runtime
check and use the PrefetchVirtualMemory function.
One downside is that the IOInterposer doesn't know about it (but it
doesn't know about madvise on other platforms anyways).
Differential Revision: https://phabricator.services.mozilla.com/D20807
--HG--
extra : moz-landing-system : lando
Consequently, since we don't readahead jars in child processes, we
don't record jar accesses from child processes. In fact, intertwining
jar accesses from child processes with those from the parent process was
arguably making things less efficient for the parent process. But this
code was written at a time e10s was barely a thing, so it wasn't really
thought for a multi-process world.
Differential Revision: https://phabricator.services.mozilla.com/D20752
--HG--
extra : moz-landing-system : lando
This backs out the main patch landed earlier in bug 1194856 and the
patch from bug 1225004.
Differential Revision: https://phabricator.services.mozilla.com/D14050
--HG--
extra : moz-landing-system : lando
mZips key is used only for internal hashtable lookups, so GetPersistentDescriptor is suitable.
MozReview-Commit-ID: 48wDOSjyo3r
--HG--
extra : rebase_source : 03c4b47812dade1d3e321727aafacfbc12bcbf32
extra : intermediate-source : 85a0b6bc25a1f960767ac28ff23a8c26829946a2
extra : source : 544bf26e258d42c835c80672416b0e29a48ba33b
clang's -Wcomma warning warns about suspicious use of the comma operator such as between two statements or to call a function for side effects within an expression.
modules/libjar/nsZipArchive.cpp:651:25 [-Wcomma] possible misuse of comma operator here
MozReview-Commit-ID: 9PjB915D81f
--HG--
extra : rebase_source : c494c2f4a8291d6c08f02765e988c1fd14079e54
extra : source : 3643e37b615c4461b6a9359856731252acc36465
Modern compression algorithms are better than zlib both in terms of
space and time. The jar format, used for e.g. omni.ja, addons, etc.
could benefit from using such modern algorithms, but the format only
allows a limited set of compression algorithms.
However, the format in itself is flexible, in that it can be extended
with arbitrary compression algorithms. This breaks compatibility with
programs like unzip, obviously, but we've never promised the files
shipped with Firefox will always remain "valid" zips (which they already
aren't, but they currently work with most zip readers).
With this change, we allow those archives to contain brotli streams,
using an arbitrary large value for the compression type in the Zip local
file header. This only allows to read such archives, but not to produce
them, and, for now, support for brotli streams is kept Nightly-only,
until everything is pieced together and we're happy to ship it.
--HG--
extra : rebase_source : fa637251f460ad0d91d5f5bec392c6e59555e80d
Michal Novotny
Bogdan Tara
Gijs Kruitbosch
Sylvestre Ledru
Doug Thayer
Dorel Luca
Cosmin Sabou
Mike Hommey
Gurzau Raul
Ehsan Akhgari
Cameron McCormack
Andi-Bogdan Postelnicu
Sebastian Hengst
Tristan Bourvon
Jeff Walden
Masatoshi Kimura
Jonathan Kew
Petr Sumbera
Paul Bignier
Chris Peterson
Eric Rahm
Andrew McCreight
Nathan Froyd
Nicholas Nethercote
Kyle Huey
Patrick McManus
Kirk Steuber