Poison was setup at the start of xpcom init when that was assumed to be early enough.
Since then, Poison was added to Maybe, and Maybe has been used everywhere, including in
our channel implementation. As a result, poison was being used before it was initialized.
This basically meant our poison pointers were being replaced with null instead, which
dances into some more UB than accessing a page we have actually allocated. Also, tsan
noticed that accesses to the value were racing with the initializer actually being
called!
A (dynamic) static initializer forces the poison initialization as we can reasonably
hope without getting CallOnce or singleton patterns involved.
Other changes:
* Cleaned up the outdated documentation for mozWritePoison (the alignment
restriction was removed in Bug 1414901)
* Removed the poison supression from TSan
Differential Revision: https://phabricator.services.mozilla.com/D94251
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
We want to collect information on late writes via telemetry. We have been
doing this in Nightly for a while now, but want to do so in beta/release. I
was actually initially unaware of this limitation of the IOInterposer, but
we need the IOInterposer to collect information on late writes, so I would
like to enable it for just early beta, in the hopes that we can catch any
late writes that may be happening, without adding a performance tax onto
release.
Accordingly, is perf the only reason that this was restricted to Nightly?
And if so, did we measure a perf difference, or was this just general
caution regarding the performance impact? Is there anything else to look
out for?
Differential Revision: https://phabricator.services.mozilla.com/D90894
The RDD process gets shutdown following a NS_XPCOM_SHUTDOWN_OBSERVER_ID notification.
Notifications are processed in LIFO order, since the RDD process is started on demand it would have typically be registered after a content process.
We must ensure that the RDD get shutdown after all content processes so that it can receive notifications that the RemoteDecoderManagerChilds are shutting down.
Differential Revision: https://phabricator.services.mozilla.com/D90485
The RDD process gets shutdown following a NS_XPCOM_SHUTDOWN_OBSERVER_ID notification.
Notifications are processed in LIFO order, since the RDD process is started on demand it would have typically be registered after a content process.
We must ensure that the RDD get shutdown after all content processes so that it can receive notifications that the RemoteDecoderManagerChilds are shutting down.
Differential Revision: https://phabricator.services.mozilla.com/D90485
This backs out all work from bug 1627075 as well as all of its
descendents. There were a few conflicts when backing this out but
overall it was pretty clean, so I would say it's a fairly mild
level of risk. Historically Nathan Froyd has reviewed these patches,
but he is no longer at Mozilla, and no one else is particularly
familiar with the code, so I am passing this off to RyanVM who has
at least been familiar with the history of the bug.
Differential Revision: https://phabricator.services.mozilla.com/D90096
Mostly mechanical change, with some extra work where non-literal names are provided.
Also, when this is the only profiler call in a file, `#include "GeckoProfiler.h"` can be changed to `#include "mozilla/ProfilerMarkers.h"`.
Differential Revision: https://phabricator.services.mozilla.com/D89415
This patch was generated by running:
```
perl -p -i \
-e 's/^(\s+)([a-zA-Z0-9.]+) = NS_ConvertUTF8toUTF16\((.*)\);/\1CopyUTF8toUTF16(\3, \2);/;' \
-e 's/^(\s+)([a-zA-Z0-9.]+) = NS_ConvertUTF16toUTF8\((.*)\);/\1CopyUTF16toUTF8(\3, \2);/;' \
$FILE
```
against every .cpp and .h in mozilla-central, and then fixing up the
inevitable errors that happen as a result of matching C++ expressions with
regexes. The errors fell into three categories:
1. Calling the convert functions with `std::string::c_str()`; these were
changed to simply pass the string instead, relying on implicit conversion
to `mozilla::Span`.
2. Calling the convert functions with raw pointers, which is not permitted
with the copy functions; these were changed to invoke `MakeStringSpan` first.
3. Other miscellaneous errors resulting from over-eager regexes and/or the
replacement not being type-aware. These changes were reverted.
Differential Revision: https://phabricator.services.mozilla.com/D88903
This is a speculative fix for a crash we're seeing due to xul.css ostensibly
not existing. The theory is that xul.css does in fact exist and the cached
zip central for the omnijar is simply corrupt in some way. If it is corrupt in
this way, then there is a bigger issue, and we need to investigate deeper.
However, the benefit of this approach is that it is a very small and contained
patch which should be simple to uplift.
Differential Revision: https://phabricator.services.mozilla.com/D86829
This changes the StartupCache::PutBuffer call from Omnijar to use a
fallibly allocated buffer, to reduce OOM crashes. We can/should broaden the
scope of this, but this is a simple initial change to mitigate the OOM
crashed introduced with bug 1627075.
Differential Revision: https://phabricator.services.mozilla.com/D86067
CLOSED TREE
We don't need these macros anymore, for two reasons:
1. We have static analysis to provide the same sort of checks via `MOZ_RAII`
and friends.
2. clang now warns for the "temporary that should have been a declaration" case.
The extra requirements on class construction also show up during debug tests
as performance problems.
This change was automated by using the following sed script:
```
# Remove declarations in classes.
/MOZ_DECL_USE_GUARD_OBJECT_NOTIFIER/d
/MOZ_GUARD_OBJECT_NOTIFIER_INIT/d
# Remove individual macros, carefully.
{
# We don't have to worry about substrings here because the closing
# parenthesis "anchors" the match.
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_TO_PARENT)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_IN_IMPL)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_IN_IMPL)/)/g;
# Remove the longer identifier first.
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_TO_PARENT//g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM//g;
}
# Remove the actual include.
\@# *include "mozilla/GuardObjects.h"@d
```
and running:
```
find . -name \*.cpp -o -name \*.h | grep -v 'GuardObjects.h' |xargs sed -i -f script 2>/dev/null
mach clang-format
```
Differential Revision: https://phabricator.services.mozilla.com/D85168
We don't need these macros anymore, for two reasons:
1. We have static analysis to provide the same sort of checks via `MOZ_RAII`
and friends.
2. clang now warns for the "temporary that should have been a declaration" case.
The extra requirements on class construction also show up during debug tests
as performance problems.
This change was automated by using the following sed script:
```
# Remove declarations in classes.
/MOZ_DECL_USE_GUARD_OBJECT_NOTIFIER/d
/MOZ_GUARD_OBJECT_NOTIFIER_INIT/d
# Remove individual macros, carefully.
{
# We don't have to worry about substrings here because the closing
# parenthesis "anchors" the match.
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_TO_PARENT)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_IN_IMPL)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_IN_IMPL)/)/g;
# Remove the longer identifier first.
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_TO_PARENT//g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM//g;
}
# Remove the actual include.
\@# *include "mozilla/GuardObjects.h"@d
```
and running:
```
find . -name \*.cpp -o -name \*.h | grep -v 'GuardObjects.h' |xargs sed -i -f script 2>/dev/null
mach clang-format
```
Differential Revision: https://phabricator.services.mozilla.com/D85168
We're crashing occasionally here due to an assertion in Span.h. We would
also likely have problems down the road if we tried to cache nullptrs in
the startup cache. I think this is all we need to handle this as gracefully
as we are able - at least, this should make it so the recent StartupCache
changes are no longer making failure any less graceful.
Differential Revision: https://phabricator.services.mozilla.com/D83189
If the original `NtCreateFile()` call failed, the file handle may have been left untouched, and is therefore probably uninitialized or in another unusable state.
Differential Revision: https://phabricator.services.mozilla.com/D82647
This should be a relatively straightforward patch. Essentially, we implement
a wrapper class (and friends) around nsZipArchive (and friends), which transparently
caches entries from the underlying zip archive in the StartupCache. This will break
without changes to the StartupCache, made in the patch after this, which allow it
to be used off of the main thread, and outside the main process.
Depends on D77635
Differential Revision: https://phabricator.services.mozilla.com/D77634
Opening our Omnijars can be expensive, and it should be deferrable until after
startup is completed, provided we have a startup cache. In a previous patch in this
stack, we implemented caching of the zip central directory for omnijars, but we
still have to open the file in order to hand the object off to various omnijar
consumers. In a later patch, we will wrap nsZipArchive access in a class which
will allow us to transparently cache nsZipArchive results. These two get us
most of the way to not needing to read from the underlying omnijar files during
startup, but there are still nontrivial pieces, like nsZipFind for instance,
which we don't want to just duplicate inside of a wrapper class, so we would
like to sort out a way in which we can use an nsZipArchive class, but not
actually back it up with the real underlying file until we really need data
from it which we can't find in a cache.
Depends on D77633
Differential Revision: https://phabricator.services.mozilla.com/D78584
We need to be able to init StartupCache before the Omnijar in order to cache
all of the Omnijar contents we access. This patch implements that.
Depends on D77632
Differential Revision: https://phabricator.services.mozilla.com/D77633
We would like to be able to defer opening the omnijar files until after startup
if the StartupCache has already been populated. Opening the omnijar files takes
a nontrivial time, at least on Windows, and almost everything in the omnijar
should be fairly compressible, and thus makes sense to live in the StartupCache.
See the last patch in this series for a little more discussion on numbers, but
tl;dr: we saw a 12% improvement in time to about:home being finished on reference
hardware with these changes together with the changes from the descendant patches.
Differential Revision: https://phabricator.services.mozilla.com/D77632
This should be a relatively straightforward patch. Essentially, we implement
a wrapper class (and friends) around nsZipArchive (and friends), which transparently
caches entries from the underlying zip archive in the StartupCache. This will break
without changes to the StartupCache, made in the patch after this, which allow it
to be used off of the main thread, and outside the main process.
Depends on D77635
Differential Revision: https://phabricator.services.mozilla.com/D77634
Opening our Omnijars can be expensive, and it should be deferrable until after
startup is completed, provided we have a startup cache. In a previous patch in this
stack, we implemented caching of the zip central directory for omnijars, but we
still have to open the file in order to hand the object off to various omnijar
consumers. In a later patch, we will wrap nsZipArchive access in a class which
will allow us to transparently cache nsZipArchive results. These two get us
most of the way to not needing to read from the underlying omnijar files during
startup, but there are still nontrivial pieces, like nsZipFind for instance,
which we don't want to just duplicate inside of a wrapper class, so we would
like to sort out a way in which we can use an nsZipArchive class, but not
actually back it up with the real underlying file until we really need data
from it which we can't find in a cache.
Depends on D77633
Differential Revision: https://phabricator.services.mozilla.com/D78584
We need to be able to init StartupCache before the Omnijar in order to cache
all of the Omnijar contents we access. This patch implements that.
Depends on D77632
Differential Revision: https://phabricator.services.mozilla.com/D77633
We would like to be able to defer opening the omnijar files until after startup
if the StartupCache has already been populated. Opening the omnijar files takes
a nontrivial time, at least on Windows, and almost everything in the omnijar
should be fairly compressible, and thus makes sense to live in the StartupCache.
See the last patch in this series for a little more discussion on numbers, but
tl;dr: we saw a 12% improvement in time to about:home being finished on reference
hardware with these changes together with the changes from the descendant patches.
Differential Revision: https://phabricator.services.mozilla.com/D77632
This should be a relatively straightforward patch. Essentially, we implement
a wrapper class (and friends) around nsZipArchive (and friends), which transparently
caches entries from the underlying zip archive in the StartupCache. This will break
without changes to the StartupCache, made in the patch after this, which allow it
to be used off of the main thread, and outside the main process.
Depends on D77635
Differential Revision: https://phabricator.services.mozilla.com/D77634
Opening our Omnijars can be expensive, and it should be deferrable until after
startup is completed, provided we have a startup cache. In a previous patch in this
stack, we implemented caching of the zip central directory for omnijars, but we
still have to open the file in order to hand the object off to various omnijar
consumers. In a later patch, we will wrap nsZipArchive access in a class which
will allow us to transparently cache nsZipArchive results. These two get us
most of the way to not needing to read from the underlying omnijar files during
startup, but there are still nontrivial pieces, like nsZipFind for instance,
which we don't want to just duplicate inside of a wrapper class, so we would
like to sort out a way in which we can use an nsZipArchive class, but not
actually back it up with the real underlying file until we really need data
from it which we can't find in a cache.
Depends on D77633
Differential Revision: https://phabricator.services.mozilla.com/D78584
We need to be able to init StartupCache before the Omnijar in order to cache
all of the Omnijar contents we access. This patch implements that.
Depends on D77632
Differential Revision: https://phabricator.services.mozilla.com/D77633
We would like to be able to defer opening the omnijar files until after startup
if the StartupCache has already been populated. Opening the omnijar files takes
a nontrivial time, at least on Windows, and almost everything in the omnijar
should be fairly compressible, and thus makes sense to live in the StartupCache.
See the last patch in this series for a little more discussion on numbers, but
tl;dr: we saw a 12% improvement in time to about:home being finished on reference
hardware with these changes together with the changes from the descendant patches.
Differential Revision: https://phabricator.services.mozilla.com/D77632
Use `GetFileType(HANDLE)` on Windows.
Unlike `HandleToFilename`, `GetFileType` is fast enough that we don't need to use a `SmallArrayLRUCache` for it.
The pipe I/Os should not be visible anymore in the startup tests.
Differential Revision: https://phabricator.services.mozilla.com/D82303
Platform-specific observations will be able to specify a file type.
They will appear in distinct rows in the profiler.firefox.com Marker Chart.
The default type is "File", which is shown in markers as "FileIO" like before.
Differential Revision: https://phabricator.services.mozilla.com/D80399
Caching filenames in 32-entry LRU array covers >95% of calls, and makes the average `Filename()` call 5 to 10 times cheaper.
browser_start_content_mainthreadio.js needed to be updated to handle operations that now have a filename thanks to the cache.
Since `ClearPoisonIOInterposer()` is never called (see bug 1647107), during Firefox shutdown we put LRUCache in a shutdown mode, which bypasses the cache in case it is still used at that time.
Differential Revision: https://phabricator.services.mozilla.com/D79767
Bryce Seager van Dyk
Alexis Beingessner
Ricky Stewart
Bogdan Tara
Dorel Luca
Doug Thayer
Jean-Yves Avenard
Brindusan Cristian
Nika Layzell
Gerald Squelart
Gijs Kruitbosch
Nathan Froyd
Sylvestre Ledru
Mihai Alexandru Michis
Mike Hommey
Simon Giesecke
Kris Maglione
Csoregi Natalia
Narcis Beleuzu
Noemi Erli