profiler_is_locked_on_current_thread() is used to help users avoid calling `profiler_...` functions when the profiler may already have a lock in place, which would prevent a 2nd recursive lock (resulting in a crash or a never-ending wait).
So we must return `true` for any of:
- The main profiler mutex, used by most functions, and/or
- The buffer mutex, used directly in some functions without locking the main mutex, e.g., marker-related functions.
Differential Revision: https://phabricator.services.mozilla.com/D73789
Expose `ProfileChunkedBuffer`'s mutex (if present), so that potential callers can avoid recursive calls that would lock or crash.
Differential Revision: https://phabricator.services.mozilla.com/D73788
Make `BaseProfilerMutex::mOwningThreadId` non-optional, and use it in `IsLockedOnCurrentThread()`, which is similar to the one in Gecko Profiler.
Add related `BaseProfilerMaybeMutex::IsActivatedAndLockedOnCurrentThread()`.
Differential Revision: https://phabricator.services.mozilla.com/D73786
At the time this code was written, JavaScript JIT entry trampolines were
emitting EnterJIT label frames that carried a stack address. From this stack
address, register values could be recovered that would allow native stack
unwinding to resume after getting lost in JIT code.
The EnterJIT label frame was removed in bug 1057082.
Differential Revision: https://phabricator.services.mozilla.com/D73939
I apologize for mixing multiple types of changes in this patch. The mix is probably harder to review than necessary.
Because of the way these .cpp files are listed in mozglue/baseprofiler/moz.build,
they are only built when MOZ_GECKO_PROFILER is set. So the #ifdef MOZ_GECKO_PROFILER
wrapper is now unnecessary.
Removing this wrapper has many knock-on effects on other lines in these files
due to preprocessor line indentation.
As I was removing the #ifdefs, I was reordering includes as well, to follow the
include order laid out on https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes ;
namely, system headers should go before "your project's" .h files.
I also removed the BaseProfiler.h include from some files but not from all.
In the past, the BaseProfiler.h include served the purpose of picking up the
MOZ_BASE_PROFILER #define, and it's no longer needed for that reason.
There are probably lots of now-unnecessary BaseProfiler.h includes left.
Differential Revision: https://phabricator.services.mozilla.com/D73527
The profiler can be given any power of two, but there were no safety checks to ensure that the buffer had a minimum workable capacity (to hold at least 4 chunks, each capable of holding at least one stack sample), and also to prevent large buffers that could break the currently-supported 2GiB limit.
This fixes the issue with test_merged_stacks.js, which was requesting a too-small buffer. (This started when we switched to the chunk-based buffer, because the profiler was blindly using the provided number as maximum, and dividing that size by 4 for each chunk, which was not enough to hold a full sample in some builds.)
Differential Revision: https://phabricator.services.mozilla.com/D73212
This patch adds a boolean field `mIsDependent` indicating whether a module was
loaded via the executable's Import Directory Table or not.
This patch also partially reverts Bug 1587539, moving a logic to detect Import
Directory tampering to `PEHeaders`'s ctor. With this, we can skip generating
a map of the executable's dependent modules if no tampering is detected.
Differential Revision: https://phabricator.services.mozilla.com/D66274
When the browser process starts a sandbox process, we copy the executable's IAT
for ntdll.dll into the new process to prevent DLL injection via IAT tampering as
the launcher process does. However, if IAT has been modified by a module injected
via `SetWindowHookEx`, the browser process cannot copy IAT because a modified IAT
is invalid in a different process, failing to start any sandbox processes.
The proposed fix is to cache IAT before COM initialization which may load
modules via `SetWindowHookEx` for the first time in the process.
Differential Revision: https://phabricator.services.mozilla.com/D73303
This is how my mapfile looks like:
5587df936000-5587df96b000 r--p 00000000 fd:02 21584889 /home/emilio/src/moz/gecko/obj-debug/dist/bin/firefox
5587df96b000-5587df9ec000 r-xp 00035000 fd:02 21584889 /home/emilio/src/moz/gecko/obj-debug/dist/bin/firefox
5587df9ec000-5587df9ed000 r--p 000b6000 fd:02 21584889 /home/emilio/src/moz/gecko/obj-debug/dist/bin/firefox
5587df9ed000-5587df9ee000 rw-p 000b7000 fd:02 21584889 /home/emilio/src/moz/gecko/obj-debug/dist/bin/firefox
Note how the executable bit, which is the only one we look at, and which
is where we get exeExeAddr from (0x5587df96b000 in this case) is in the
middle of the executable, but the library will span all four ranges.
Check for whether the library _contains_ the start address of the
executable region instead of whether it starts with it.
Differential Revision: https://phabricator.services.mozilla.com/D72504
As opposed to `ProfileBufferIndex` (no "Block"), `ProfileBufferBlockIndex` is only supposed to point at a valid block start.
If we trust this assumption, it allows for quick access to the given block index inside the buffer, as we don't need to read blocks one by one until we reach the given position.
There are still safety checks (MOZ_ASSERTs in DEBUG builds) to verify that block indices are correctly used.
Differential Revision: https://phabricator.services.mozilla.com/D71502
Renamed some variables to be more generic. Their type is going to change in the next patch, and that type doesn't need to be in the names; also it will make the next patch easier to review.
Differential Revision: https://phabricator.services.mozilla.com/D71882
`explicit operator bool()` and `operator!()` were cute ways to make `InChunkBuffer` quack like a pointer when testing if it's effectively null.
But after some experience, and since `InChunkPointer` will not be used in generic code where pointers would be accepted, I now think that it's better to be clearer about it and use an explicit `IsNull()`.
Differential Revision: https://phabricator.services.mozilla.com/D71499
Same as with `BlocksRingBuffer`: Instead of a potentially-null pointer to a
`ProfileBufferEntryWriter`, we are now providing a
`Maybe<ProfileBufferEntryWriter>`, which is safer.
Differential Revision: https://phabricator.services.mozilla.com/D71287
Instead of a potentially-null pointer to a `ProfileBufferEntryWriter`, we are now providing a `Maybe<ProfileBufferEntryWriter>`, which is safer.
Differential Revision: https://phabricator.services.mozilla.com/D71286
This is needed to embed a small buffer (e.g., containing one backtrace attached
to a marker) into a bigger buffer (e.g., the main profiler buffer).
Differential Revision: https://phabricator.services.mozilla.com/D69499
--HG--
extra : moz-landing-system : lando
`InChunkPointer` is an internal accessor pointing at a position inside a chunk.
It can handle up to two groups of chunks (typically the extant chunks stored in
the chunk manager, and the current chunk.
Differential Revision: https://phabricator.services.mozilla.com/D69497
--HG--
extra : moz-landing-system : lando
To ensure that a spare chunk is ready to handle data that will eventually
overflow the current chunk, `ProfileChunkedBuffer` uses
`ProfileBufferChunk::RequestChunk()` to queue a request for a new chunk.
This request should be handled off-thread by the buffer user -- but a response
is not guaranteed, so the buffer does not rely on it and can get a new chunk
on the spot if really needed.
Because the request is asynchronous, and because either the buffer or the user
could be destroyed while a request is in flight, a shared
`RequestedChunkRefCountedHolder` object is used:
- When the request is handled, the new chunk (or nullptr) is given to the
holder.
- When the buffer needs a new chunk, it can retrieve the new chunk if the
request was successfully fulfilled.
If the requestee is destroyed first, the request won't be fulfilled and the
buffer will carry on without relying on requests.
If the requester is destroyed first, the holder (with a potential requested
chunk) will just get destroyed after the request is fulfilled or the requestee
is destroyed as well.
Differential Revision: https://phabricator.services.mozilla.com/D69495
--HG--
extra : moz-landing-system : lando
`ProfileChunkedBuffer` can handle zero or one `ProfileBufferChunkManager` at a
time, and can optionally take ownership of the manager.
Differential Revision: https://phabricator.services.mozilla.com/D69494
--HG--
extra : moz-landing-system : lando
ProfileChunkedBuffer simulates a near-infinite buffer over ProfileBufferChunks.
It uses a ProfileBufferChunkManager to get chunks and later release them.
Its use is similar to BlocksRingBuffer:
- It reserves blocks in chunks, adds some structure (just the size of the entry
that follows), and lets a user-provided writer write the entry.
- It allows reading past entries.
- It can be in an "out-of-session" state where APIs are still available but do
nothing.
It is intended to eventually replace BlocksRingBuffer.
This patch starts with the basic structure, following patches will add all
planned features.
Differential Revision: https://phabricator.services.mozilla.com/D69493
--HG--
extra : moz-landing-system : lando
`ProfileBuffer` used to check if a `BlocksRingBuffer` was in session by looking
at its buffer size.
Now `IsInSession()` should be used, it is clearer in intent, and will make the
transition to `ProfileChunkedBuffer` slightly easier.
Differential Revision: https://phabricator.services.mozilla.com/D69492
--HG--
extra : moz-landing-system : lando
Also move MOZ_MUST_USE before function declarations' specifiers and return type. While clang and gcc's __attribute__((warn_unused_result)) can appear before, between, or after function specifiers and return types, the [[nodiscard]] attribute must precede the function specifiers.
Differential Revision: https://phabricator.services.mozilla.com/D70631
--HG--
extra : moz-landing-system : lando
This commit removes `test_fix_stack_using_bpsyms.py`. That test can't easily be
modified to work with `fix_stacks.py` because it relies on internal
implementation details of `fix_stack_using_bpsym.py`. The unit testing done in
the `fix-stacks` repo provides test coverage that is as good or better.
Differential Revision: https://phabricator.services.mozilla.com/D66924
--HG--
extra : moz-landing-system : lando
With ASAN, GTest uses the old blocklist implemented in mozglue, where
the new blocklist type `RedirectToNoOpEntryPoint` behaves the same as
`DllBlocklistEntry`. The test needs to expect `LoadLibrary` to fail.
Differential Revision: https://phabricator.services.mozilla.com/D70578
--HG--
extra : moz-landing-system : lando
Please double check that I am using this correctly. I believe we are
seeing the crash in the linked bug because we are not handling hardware
faults when reading from the memory mapped file. This patch just wraps
all accesses in the MMAP_FAULT_HANDLER_ macros.
Depends on D53042
Differential Revision: https://phabricator.services.mozilla.com/D53043
--HG--
rename : modules/libjar/MmapFaultHandler.cpp => mozglue/misc/MmapFaultHandler.cpp
rename : modules/libjar/MmapFaultHandler.h => mozglue/misc/MmapFaultHandler.h
extra : moz-landing-system : lando
This patch introduces a new DLL blocklist type `RedirectToNoOpEntryPoint`
which hooks a DLL's entrypoint into a no-op function. With this technique,
we give the injected DLL no chance to run its code though we allow it to be
loaded into the process.
This new blocklist type is intended to block a DLL which is injected by IAT
patching which was planted by a kernel callback routine for LoadImage. It's
because blocking such a DLL makes a new process fail to launch.
Differential Revision: https://phabricator.services.mozilla.com/D68348
--HG--
extra : moz-landing-system : lando
This patch introduces `Kernel32ExportsSolver` which calculates RVAs of
kernel32's functions and transfers them to a target process, where the
transferred RVAs are resolved into function addresses.
Depends on D68346
Differential Revision: https://phabricator.services.mozilla.com/D68347
--HG--
extra : moz-landing-system : lando
This patch introduces a new DLL interceptor `WindowsDllEntryPointInterceptor`
which applies a hook to a target function without backing up the original
function code.
Depends on D68345
Differential Revision: https://phabricator.services.mozilla.com/D68346
--HG--
extra : moz-landing-system : lando
This patch introduces a new policy `MMPolicyInProcessEarlyStage` which does
not consume any functions imported from kernel32.dll so that we can use it
in a process's early stage i.e. before IAT is resolved.
Depends on D68344
Differential Revision: https://phabricator.services.mozilla.com/D68345
--HG--
extra : moz-landing-system : lando
`WindowsDllDetourPatcher::CreateTrampoline` does not only create a trampoline
region but also applies a patch on an original function. This patch extracts
the patching part as separate functions.
Differential Revision: https://phabricator.services.mozilla.com/D68344
--HG--
extra : moz-landing-system : lando
This patch moves the instantiation of `PEHeaders` from `CheckBlockInfo` to
`IsDllAllowed` so that `IsDllAllowed` can use an instance of `PEHeaders`.
Depends on D68342
Differential Revision: https://phabricator.services.mozilla.com/D68343
--HG--
extra : moz-landing-system : lando
This patch introduces `nt::VirtualQuery` which consumes only ntdll's functions
to reduce dependency in `MMPolicy` on kernel32.dll. With this, `MMPolicy` still
depends on kernel32.dll, that will be solved by a coming patch.
Differential Revision: https://phabricator.services.mozilla.com/D68342
--HG--
extra : moz-landing-system : lando
This patch introduces a new DLL blocklist type `RedirectToNoOpEntryPoint`
which hooks a DLL's entrypoint into a no-op function. With this technique,
we give the injected DLL no chance to run its code though we allow it to be
loaded into the process.
This new blocklist type is intended to block a DLL which is injected by IAT
patching which was planted by a kernel callback routine for LoadImage. It's
because blocking such a DLL makes a new process fail to launch.
Differential Revision: https://phabricator.services.mozilla.com/D68348
--HG--
extra : moz-landing-system : lando
This patch introduces `Kernel32ExportsSolver` which calculates RVAs of
kernel32's functions and transfers them to a target process, where the
transferred RVAs are resolved into function addresses.
Depends on D68346
Differential Revision: https://phabricator.services.mozilla.com/D68347
--HG--
extra : moz-landing-system : lando
This patch introduces a new DLL interceptor `WindowsDllEntryPointInterceptor`
which applies a hook to a target function without backing up the original
function code.
Depends on D68345
Differential Revision: https://phabricator.services.mozilla.com/D68346
--HG--
extra : moz-landing-system : lando
This patch introduces a new policy `MMPolicyInProcessEarlyStage` which does
not consume any functions imported from kernel32.dll so that we can use it
in a process's early stage i.e. before IAT is resolved.
Depends on D68344
Differential Revision: https://phabricator.services.mozilla.com/D68345
--HG--
extra : moz-landing-system : lando
`WindowsDllDetourPatcher::CreateTrampoline` does not only create a trampoline
region but also applies a patch on an original function. This patch extracts
the patching part as separate functions.
Depends on D68343
Differential Revision: https://phabricator.services.mozilla.com/D68344
--HG--
extra : moz-landing-system : lando
This patch moves the instantiation of `PEHeaders` from `CheckBlockInfo` to
`IsDllAllowed` so that `IsDllAllowed` can use an instance of `PEHeaders`.
Depends on D68342
Differential Revision: https://phabricator.services.mozilla.com/D68343
--HG--
extra : moz-landing-system : lando
This patch introduces `nt::VirtualQuery` which consumes only ntdll's functions
to reduce dependency in `MMPolicy` on kernel32.dll. With this, `MMPolicy` still
depends on kernel32.dll, that will be solved by a coming patch.
Differential Revision: https://phabricator.services.mozilla.com/D68342
--HG--
extra : moz-landing-system : lando
Similarly to changeset cfa64a6b5a87, a scope's enclosing scope is no longer updated by MovingTracer::onScopeEdge following the changes in bug 1625212. Update the suppression list accordingly.
Differential Revision: https://phabricator.services.mozilla.com/D69986
--HG--
extra : moz-landing-system : lando
RC should be non-copyable. It already is non-copyable in the atomic variant,
but not in the non-atomic variant. This ensures that RefCounted is also
non-copyable, which prevents that classes deriving from RefCounted get
accidentally copied.
Also, RC's operator= should be defined only in debug builds. As already
described in a code comment, it's only used in debug builds (to assign the
dead value), and otherwise no modifications other than incrementing &
decrementing should be possible.
Differential Revision: https://phabricator.services.mozilla.com/D69332
--HG--
extra : moz-landing-system : lando
Also move MOZ_MUST_USE before function declarations' specifiers and return type. While clang and gcc's attribute((warn_unused_result)) can appear before, between, or after function specifiers and return types, the [[nodiscard]] attribute must precede the function specifiers.
And removed a few unneded `#include "mozilla/Attributes.h"`.
Differential Revision: https://phabricator.services.mozilla.com/D69755
--HG--
extra : moz-landing-system : lando
Previously this ended up with MovingTracer doing the update itself, but now the pointer is stored in a CellHeader it's done through an overload of TraceEdge.
The TSAN stack doesn't appear to have the class for the unsafeSetPtr method that actually does the update so rather than make this apply to all uses of this method I added a supression for the next function up in the stack.
Differential Revision: https://phabricator.services.mozilla.com/D69524
--HG--
extra : moz-landing-system : lando
When our detour processes instructions, we pass `ReadOnlyTargetFunction` to
`CountPrefixBytes` to determine whether a lock prefix exists or not.
In that case, we don't need to pass both `ReadOnlyTargetFunction` and an offset
as a parameter because `ReadOnlyTargetFunction` has an offset as a member.
Differential Revision: https://phabricator.services.mozilla.com/D69360
--HG--
extra : moz-landing-system : lando
`ProfileBufferChunkManagerWithLocalLimit` is a chunk manager that enforces a
memory limit in each process.
It is meant to mimic the main way `BlocksRingBuffer` works, so that we can more
easily switch to the new buffer storage without introducing the extra complexity
of inter-process memory coordination yet.
`ProfileBufferChunkManagerWithLocalLimit` will still offer a benefit over
`BlocksRingBuffer`, in that it won't allocate the maximum buffer size
immediately -- speeding the initialization, and hopefully even reducing the
total Firefox memory consumption in short-lived processes.
Differential Revision: https://phabricator.services.mozilla.com/D68770
--HG--
extra : moz-landing-system : lando
Please double check that I am using this correctly. I believe we are
seeing the crash in the linked bug because we are not handling hardware
faults when reading from the memory mapped file. This patch just wraps
all accesses in the MMAP_FAULT_HANDLER_ macros.
Depends on D53042
Differential Revision: https://phabricator.services.mozilla.com/D53043
--HG--
rename : modules/libjar/MmapFaultHandler.cpp => mozglue/misc/MmapFaultHandler.cpp
rename : modules/libjar/MmapFaultHandler.h => mozglue/misc/MmapFaultHandler.h
extra : moz-landing-system : lando
Also adds missing includes in some files, these were previously only transivitely
included through mozilla/TypeTraits.h.
Differential Revision: https://phabricator.services.mozilla.com/D68561
--HG--
extra : moz-landing-system : lando
We still need it for Android < 6.0 (API 23) because otherwise we don't
have a way to hook up mozalloc.
Differential Revision: https://phabricator.services.mozilla.com/D66993
--HG--
extra : moz-landing-system : lando
A `ProfileBufferChunk` represents a single chunk of memory, with an optional
link to the next chunk.
In the new Fission-compatible profiler storage, chunks will be allocated by a
chunk manager, filled with data by the profiler, and then released back to the
chunk manager.
The chunk manager may decide to destroy or recycle old chunks based on memory
limits (per process, or for the entire Firefox app).
Differential Revision: https://phabricator.services.mozilla.com/D67272
--HG--
extra : moz-landing-system : lando
A `ProfileBufferChunk` represents a single chunk of memory, with an optional
link to the next chunk.
In the new Fission-compatible profiler storage, chunks will be allocated by a
chunk manager, filled with data by the profiler, and then released back to the
chunk manager.
The chunk manager may decide to destroy or recycle old chunks based on memory
limits (per process, or for the entire Firefox app).
Differential Revision: https://phabricator.services.mozilla.com/D67272
--HG--
extra : moz-landing-system : lando
We still need it for Android < 6.0 (API 23) because otherwise we don't
have a way to hook up mozalloc.
Differential Revision: https://phabricator.services.mozilla.com/D66993
--HG--
extra : moz-landing-system : lando
The old code was using `std::index_sequence_for` to call templated functions for
*every* possible Variant alternative.
Instead, using `Variant::match()` with a generic lambda (that also takes the
current Variant index, thanks to bug 1621865) means we're only calling the
function corresponding to the current alternative.
Differential Revision: https://phabricator.services.mozilla.com/D67241
--HG--
extra : moz-landing-system : lando
As far as I can tell there is no single entry-point into C++ code on Android.
Instead, GeckoThread and GeckoLibLoader call various functions to load libraries one-by-one.
We want to capture all that library loading in the profiler, so we need to kick off the base profiler at the beginning of whichever function is called first.
Differential Revision: https://phabricator.services.mozilla.com/D64999
--HG--
extra : moz-landing-system : lando
Stop including "Linker.h" and always include <link.h> instead, which also comes with ElfW() and other things that this code needs.
Also fix up "!/" path detection code.
Differential Revision: https://phabricator.services.mozilla.com/D64997
--HG--
extra : moz-landing-system : lando
This fixes the declaration of (BaseProfiler)LogTest. It also makes it so that the logs show up on Android.
In xpcom we have printf_stderr which does something similar and also handles Windows.
Differential Revision: https://phabricator.services.mozilla.com/D64994
--HG--
extra : moz-landing-system : lando
Markus Stange
Gerald Squelart
Greg V
Ricky Stewart
Toshihito Kikuchi
Bogdan Tara
Gabriele Svelto
Arash Fotouhi
Christian Holler
Jon Coppeard
Emilio Cobos Álvarez
Nazım Can Altınova
Chris Peterson
Nicholas Nethercote
Doug Thayer
Andreas Farre
Daniel Varga
Simon Giesecke
Andrea Marchesini
Sylvestre Ledru
Eric Rahm
Dorel Luca
André Bargull
James Willcox
Razvan Maries
Chris Fronk