We don't need these macros anymore, for two reasons:
1. We have static analysis to provide the same sort of checks via `MOZ_RAII`
and friends.
2. clang now warns for the "temporary that should have been a declaration" case.
The extra requirements on class construction also show up during debug tests
as performance problems.
This change was automated by using the following sed script:
```
# Remove declarations in classes.
/MOZ_DECL_USE_GUARD_OBJECT_NOTIFIER/d
/MOZ_GUARD_OBJECT_NOTIFIER_INIT/d
# Remove individual macros, carefully.
{
# We don't have to worry about substrings here because the closing
# parenthesis "anchors" the match.
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_TO_PARENT)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_IN_IMPL)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_IN_IMPL)/)/g;
# Remove the longer identifier first.
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_TO_PARENT//g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM//g;
}
# Remove the actual include.
\@# *include "mozilla/GuardObjects.h"@d
```
and running:
```
find . -name \*.cpp -o -name \*.h | grep -v 'GuardObjects.h' |xargs sed -i -f script 2>/dev/null
mach clang-format
```
Differential Revision: https://phabricator.services.mozilla.com/D85168
It is possible that some threads fail/forget to unregister themselves, in which case a registered thread id could get recycled by a later thread, which was not allowed before this patch.
Note: The thread name cannot currently be changed. We record a special marker with the new name, so the frontend could process it to split the thread track at that point.
We also record a marker when profiler_unregister_thread is called from an already-unregistered thread, this could help find reg/unreg mismatches or nesting in Firefox threads.
Differential Revision: https://phabricator.services.mozilla.com/D83293
On Linux (including Android), it was assumed that a registered thread could always be suspended through `tgkill`.
However in some cases a thread may not be correctly unregistered, in which case this would trigger `MOZ_ASSERT` or wait forever in the following loop.
This will especially be needed when `profiler_{,un}register_thread()` are made less strict in the following patch.
Windows and Mac already handle suspension failures.
Differential Revision: https://phabricator.services.mozilla.com/D83292
When the osclientcerts module attempts to use client certificates and keys from
certain tokens, the Windows APIs will attempt to load asepkcs.dll. If Firefox
blocks this library, the keys won't be available. Thus, it needs to be
unblocked.
Due to the architecture of osclientcerts (namely, its dedicated single thread),
using asepkcs.dll via the Windows APIs shouldn't cause the crashes that led to
this module being blocked.
Differential Revision: https://phabricator.services.mozilla.com/D82788
The file defines symbol renames to perform when linking. The list of
symbols to rename varies between an empty list and a full list depending
whether MOZ_MEMORY is set. Practically speaking, the variant with an
empty list is equivalent to not using a def-file at all.
This means we don't need to preprocess it any more, and we can just not
use a def-file when MOZ_MEMORY is not set.
As a side effect, this removes a rule from a Makefile.in.
Differential Revision: https://phabricator.services.mozilla.com/D83034
`nscore.h` includes `xpcom-config.h` which need not be generated for
non-XPCOM consumers. In additon, `nullptr` and `bool` are C++
keywords, so at least some of the comments were dated.
The added include lines address transitive consumers of `nscore.h`.
Differential Revision: https://phabricator.services.mozilla.com/D82640
This requires a workaround for the use of __wrap_dladdr, which can't be
used in logalloc-replay. The workaround involves making __wrap_dladdr
expand to dladdr, but that makes the definition ElfLinker.h conflict
with the one in the Android system headers, so we change it to match,
and adjust ElfLinker.cpp accordingly.
And while here, fix the condition in mozglue/misc to match the condition
around including Linker.h in StackWalk.cpp itself.
Differential Revision: https://phabricator.services.mozilla.com/D82648
To avoid deadlocks between the buffer main mutex and the externally-provided update callback code, the callback storage is now using a separate mutex, and both mutexes are never held at the same time.
Differential Revision: https://phabricator.services.mozilla.com/D82141
This is a rebase of a 7-year-old patch that was r=taras. Back then it
was waiting for breakpad changes that never were reviewed. However,
since then, bug 1486524 made the linker always map uncompressed files
directly, making it less necessary to report the library mappings to
the crash reporter, and bug 1291377 disabled the linker altogether on
Android 6.0+, which makes report_mapping unused at all in that case.
Differential Revision: https://phabricator.services.mozilla.com/D81025
AVG AntiVirus hooks ntdll!NtMapViewOfSection by planting two JMP instructions,
jumping to a trampoline area first, then jumping to aswhook.dll.
```
ntdll!NtMapViewOfSection:
00007ffa`6d77c560 e9d33cfebf jmp 00007ffa`2d760238
00007ffa`2d760238 ff25f2ffffff jmp qword ptr [00007ffa`2d760230] --> 00007ffa`541e2ad0
aswhook+0x2ad0:
00007ffa`541e2ad0 4055 push rbp
00007ffa`541e2ad2 53 push rbx
00007ffa`541e2ad3 56 push rsi
```
With this patch, our detour can detour on top of that pattern. The first part is
to remove the MEM_IMAGE check from IsPageAccessible. The second part is to introduce
a loop in ResolveRedirectedAddress to resolve a chain of jumps.
Differential Revision: https://phabricator.services.mozilla.com/D81582
This patch moves the logics of jump detection from ResolveRedirectedAddress to
ReadOnlyTargetFunction to simplify ReadOnlyTargetFunction.
Differential Revision: https://phabricator.services.mozilla.com/D81580
The profiler can be "paused", which stops sampling, and since bug 1578329 stops markers as well.
Some test suites use pausing between tests (to better differentiate the tests, to keep the profiler ready to run, and to lower the amount of recorded data). But this causes problems with some tracing markers, as their matching ends have not been recorded (e.g., an end marker is missing), which show up as very loooong markers.
To solve this, we need to be able to pause sampling only, but keep recording markers.
But we still need to be able to pause the whole profiler, in particular before capturing, to avoid recording anything around that time.
This big patch is mostly mechanical changes: Wherever there are "Pause" and "Unpause/Resume" profiler functions, we add matching "PauseSampling" and "UnpauseSampling/ResumeSampling" functions that only impact the periodic sampling loop; And existing "Pause/Unpause/Resume" imply pausing sampling as well.
Exceptions and extra work:
- nsIProfiler (the JS API) already had `Pause/ResumeSampling()`, which misleadingly paused everything! Now they do the right thing, and we have `Pause/Resume()` as well.
- All tests using `Pause/ResumeSampling()` now use `Pause/Resume()`, except for Talos tests that only pause sampling between tests; Added some extra `Pause()` calls to pause everything before capturing profiles.
- GeckoJavaSampler doesn't handle pausing/resuming everything, this should be done in a follow-up bug.
- Sampling-only pauses are not streamed into JSON. If needed, we should follow-up, with potential work on the front-end to deal with these.
Differential Revision: https://phabricator.services.mozilla.com/D81492
This header is using `MOZ_RAII` and `MFBT_ABI` from `Attributes.h` and
`Types.h`, respectively, so it should include those headers.
Differential Revision: https://phabricator.services.mozilla.com/D81600
This is a data race, but I don't think it is causing problems at the moment. It will go away when we get rid of ObjectGroups and is not trivial to fix, hence the decision to suppress it.
Differential Revision: https://phabricator.services.mozilla.com/D81327
This was previously r=jchen and landed, but was backed out because some
Android tests were running on an unrealistically old Linux kernel. These
tests have been retired, so this can reland.
Differential Revision: https://phabricator.services.mozilla.com/D81278
Bug 1486524 disabled the features from the custom linker that required
AutoObjectMapperFaultyLib. We can now rely on AutoObjectMapperPOSIX
instead.
Differential Revision: https://phabricator.services.mozilla.com/D81264
The Base Profiler is still recent and barely used, so it may contain some bugs.
With bug 1586939, the Base Profiler is now used more often because it is controlled the same way as the Gecko Profiler.
This has surfaced some intermittent issues, which pollute existing tests.
Until the root cause is found (see bug 1648325), setting `MOZ_PROFILER_STARTUP_NO_BASE=1` prevents the Base Profiler from running. This may be used where problems are visible, to diagnostic them and/or reduce them where needed.
Differential Revision: https://phabricator.services.mozilla.com/D81018
This function can be called whether the profiler mutex is locked or not. If locked, the provided pointer to the profiler mutex is used internally, otherwise the mutex will be locked as needed.
Differential Revision: https://phabricator.services.mozilla.com/D79412
PSAutoTryLock attempts to lock the mutex, but gives up immediately if it's already locked.
This can be used in profiler functions that may be called from unknown code, to avoid deadlocks involving the profiler mutex and another mutex that could be indirectly used by the profiler itself.
Differential Revision: https://phabricator.services.mozilla.com/D79410
In the JIT frame sampler, we apply the appropriate category in addition to
the "implementation" field. For JS frames (IS_JS_FRAME), we identify as
either BaselineInterpreter or Interpreter. Note that JS_Other still applies
to various places we enter SpiderMonkey outside of RunScript.
Differential Revision: https://phabricator.services.mozilla.com/D79524
Replace the duplicate lists in mozglue/baseprofiler/public and js/public with
a shared list. Add this list to both moz.build files so it is published twice
which simplifies supporting different standalone configurations.
Differential Revision: https://phabricator.services.mozilla.com/D79520
Introduce an IS_BLINTERP_FRAME flag to ProfilingStackFrame to distinguish C++
and Baseline interpreter frames. In the profile data this sets the
"implementation" to "blinterp".
Differential Revision: https://phabricator.services.mozilla.com/D78725
Round the number of reserved flag bits up to 16. This leaves 16-bits for the
category (so 64k subcategories). Also make the baseprofiler consistent.
Differential Revision: https://phabricator.services.mozilla.com/D78724
`BLOCK_WIN8_ONLY` was introduced by bug 1268470 to block klsihk64.dll only on
Win8. However, a new blocklist (bug 1445025) does wrong comparison on the OS
version, thus `BLOCK_WIN8_ONLY` has blocked modules on all platforms older than
Win10 including Win7 and Win8.1.
This patch corrects OS comparison and changes the flag to `BLOCK_WIN8_AND_OLDER`
to make it more handy. We also remove `BLOCK_WIN8PLUS_ONLY` which is never used.
Differential Revision: https://phabricator.services.mozilla.com/D78411
Also update documentation to suggest using the `GeneratedFile` template rather than directly referencing `GENERATED_FILES` where possible.
Differential Revision: https://phabricator.services.mozilla.com/D77496
Now both profilers are controlled by the same environment variables, e.g.
"MOZ_PROFILER_STARTUP" to run both profilers at startup.
The only remaining Base Profiler-specific commands are:
- MOZ_BASE_PROFILER_HELP, because the Base Profiler doesn't have the same
capabilities as Gecko, so it may display different available features.
- MOZ_BASE_PROFILER_LOGGING, because the Base Profiler doesn't have access to
MOZ_LOG (both the env-var and the C++ macros).
Differential Revision: https://phabricator.services.mozilla.com/D54449
Now both profilers are controlled by the same environment variables, e.g.
"MOZ_PROFILER_STARTUP" to run both profilers at startup.
The only remaining Base Profiler-specific commands are:
- MOZ_BASE_PROFILER_HELP, because the Base Profiler doesn't have the same
capabilities as Gecko, so it may display different available features.
- MOZ_BASE_PROFILER_LOGGING, because the Base Profiler doesn't have access to
MOZ_LOG (both the env-var and the C++ macros).
Differential Revision: https://phabricator.services.mozilla.com/D54449
If EAF+ is enabled for firefox.exe, the process does not launch because we parse
the PE headers of ntdll.dll at startup, which is prohibited by EAF+.
With this patch, we skip two operations when EAF+ is enabled.
The first one is to cache ntdll's IAT at startup. Because EAF+ is expected to
prevent an injected module from parsing PE headers and modifying IAT, we can skip
this caching safely.
The second one is to load ntdll's debug information for the profiler. With this
patch, the profiler's callstack will not show a raw address instead of a symbol
name. It's a bad side effect, but much better than startup crash.
Differential Revision: https://phabricator.services.mozilla.com/D76959
GetConstructedEventTarget and GetSpecificMessageEventTarget always return
null. The idea was that subclasses could override them, but nobody does any
more.
Differential Revision: https://phabricator.services.mozilla.com/D76984
This code was not being built any more.
GeckoView requires Android OS version 4.3 (API Level 18). I'm not sure what the
corresponding ANDROID_VERSION macro constant for API Level 18 is, but it's
definitely higher than 16.
Differential Revision: https://phabricator.services.mozilla.com/D73940
In addition to the existing "mainthreadio" feature, we now have:
- "fileio" to also capture file I/O from other profiled threads.
- "fileioall" to also capture file I/O from all threads (even unregistered threads).
- "noiostacks" to prevent capturing stack traces for "io" markers.
These are all off by default, except for `MOZ_PROFILER_STARTUP=1`.
Differential Revision: https://phabricator.services.mozilla.com/D75764
This is a simple compile-time check that the feature numbers go from 0 to N-1, to prevent some mistakes when updating feature lists.
Differential Revision: https://phabricator.services.mozilla.com/D75763
Some markers may be more useful when gathered into a single track, and the main thread is ideal because it's almost always present.
Differential Revision: https://phabricator.services.mozilla.com/D75760
Made StaticBaseProfilerStats a proper class, with more explicit types for measurements and printfs.
Also added the maximum duration in a set, which can be useful to see what the worst case of some code may be.
(Reminder: This code is not built by default, uncomment `# define PROFILER_RUNTIME_STATS` above to enable it.)
Differential Revision: https://phabricator.services.mozilla.com/D75758
Instead of accepting a `UniquePtr<ProfilerMarkerPayload>`, `profiler_add_marker_for_thread` now takes it by `const ProfilerMarkerPayload&`.
All callers can now create the payload on the stack, or as a temporary object. This saves a memory alloc+free for each call.
Differential Revision: https://phabricator.services.mozilla.com/D75912
This patch will do :
- add `Media` markers in related codes
- implement `MediaSampleMarkerPayload` to record the media sample
The advantage of doing so :
- using markers can help us know what happens on the media by a glance without expanding the call stack
- adding sample markers allows us compare the speed of decoding sample in `MediaDecoderStataMachine` and rendering sample in `VideoSink`
Differential Revision: https://phabricator.services.mozilla.com/D74174
This patch will do :
- create a sub-category `Cubeb`
- add `Cubeb` profiling labels in related codes
The advantage of doing so :
- allow us to know the percentage of time respectively we spend on cubeb and non-cubeb codes
More details :
The profiling code would include `<atomic>` which is C++ only, so I can't use the label in `cubeb.c` directly. Instead, I add labels on the `AudioStream` and `AudioCallbackDriver` where we would call cubeb related methods.
Differential Revision: https://phabricator.services.mozilla.com/D74172
This patch will do :
- create a profiling category `Media`
- add `Media` profiling labels in related codes
The advantage of doing so :
- allow us to easily see what operations are related to media playback from the profiled report
More details :
According to the description in the `ProfilingCategory.h`, `topmost profiler label frame in the label stack determines the category pair of that stack`. Therefore, most labels I added are the first task would run on the thread, in order to ensure all its following tasks can be marked as the media playback label as well.
Differential Revision: https://phabricator.services.mozilla.com/D74171
If a label contains a dynamic string that's too long (512 characters or more), instead of just replacing it with "(too long)", we now cut it down to the maximum size, with an ellipsis at the end.
Added test for that in gtest. Also added nearby test for empty strings.
Differential Revision: https://phabricator.services.mozilla.com/D74378
Whenever chunks are about to be destroyed, we try to keep one or two alive, to hopefully fulfill the next request, thereby avoiding a deallocation+allocation pair.
Differential Revision: https://phabricator.services.mozilla.com/D72370
The chunk metadata size is tiny (less than 100 bytes) compared to the buffer size (1MB by default), so it's fine to ignore it while dealing with cross-Firefox limits.
Differential Revision: https://phabricator.services.mozilla.com/D72558
Interface class for a chunk manager that can be controlled: It will provide updates about chunks, and release chunks on command.
Differential Revision: https://phabricator.services.mozilla.com/D72362
profiler_is_locked_on_current_thread() is used to help users avoid calling `profiler_...` functions when the profiler may already have a lock in place, which would prevent a 2nd recursive lock (resulting in a crash or a never-ending wait).
So we must return `true` for any of:
- The main profiler mutex, used by most functions, and/or
- The buffer mutex, used directly in some functions without locking the main mutex, e.g., marker-related functions.
Differential Revision: https://phabricator.services.mozilla.com/D73789
Expose `ProfileChunkedBuffer`'s mutex (if present), so that potential callers can avoid recursive calls that would lock or crash.
Differential Revision: https://phabricator.services.mozilla.com/D73788
Make `BaseProfilerMutex::mOwningThreadId` non-optional, and use it in `IsLockedOnCurrentThread()`, which is similar to the one in Gecko Profiler.
Add related `BaseProfilerMaybeMutex::IsActivatedAndLockedOnCurrentThread()`.
Differential Revision: https://phabricator.services.mozilla.com/D73786
At the time this code was written, JavaScript JIT entry trampolines were
emitting EnterJIT label frames that carried a stack address. From this stack
address, register values could be recovered that would allow native stack
unwinding to resume after getting lost in JIT code.
The EnterJIT label frame was removed in bug 1057082.
Differential Revision: https://phabricator.services.mozilla.com/D73939
I apologize for mixing multiple types of changes in this patch. The mix is probably harder to review than necessary.
Because of the way these .cpp files are listed in mozglue/baseprofiler/moz.build,
they are only built when MOZ_GECKO_PROFILER is set. So the #ifdef MOZ_GECKO_PROFILER
wrapper is now unnecessary.
Removing this wrapper has many knock-on effects on other lines in these files
due to preprocessor line indentation.
As I was removing the #ifdefs, I was reordering includes as well, to follow the
include order laid out on https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes ;
namely, system headers should go before "your project's" .h files.
I also removed the BaseProfiler.h include from some files but not from all.
In the past, the BaseProfiler.h include served the purpose of picking up the
MOZ_BASE_PROFILER #define, and it's no longer needed for that reason.
There are probably lots of now-unnecessary BaseProfiler.h includes left.
Differential Revision: https://phabricator.services.mozilla.com/D73527
The profiler can be given any power of two, but there were no safety checks to ensure that the buffer had a minimum workable capacity (to hold at least 4 chunks, each capable of holding at least one stack sample), and also to prevent large buffers that could break the currently-supported 2GiB limit.
This fixes the issue with test_merged_stacks.js, which was requesting a too-small buffer. (This started when we switched to the chunk-based buffer, because the profiler was blindly using the provided number as maximum, and dividing that size by 4 for each chunk, which was not enough to hold a full sample in some builds.)
Differential Revision: https://phabricator.services.mozilla.com/D73212
This patch adds a boolean field `mIsDependent` indicating whether a module was
loaded via the executable's Import Directory Table or not.
This patch also partially reverts Bug 1587539, moving a logic to detect Import
Directory tampering to `PEHeaders`'s ctor. With this, we can skip generating
a map of the executable's dependent modules if no tampering is detected.
Differential Revision: https://phabricator.services.mozilla.com/D66274
When the browser process starts a sandbox process, we copy the executable's IAT
for ntdll.dll into the new process to prevent DLL injection via IAT tampering as
the launcher process does. However, if IAT has been modified by a module injected
via `SetWindowHookEx`, the browser process cannot copy IAT because a modified IAT
is invalid in a different process, failing to start any sandbox processes.
The proposed fix is to cache IAT before COM initialization which may load
modules via `SetWindowHookEx` for the first time in the process.
Differential Revision: https://phabricator.services.mozilla.com/D73303
This is how my mapfile looks like:
5587df936000-5587df96b000 r--p 00000000 fd:02 21584889 /home/emilio/src/moz/gecko/obj-debug/dist/bin/firefox
5587df96b000-5587df9ec000 r-xp 00035000 fd:02 21584889 /home/emilio/src/moz/gecko/obj-debug/dist/bin/firefox
5587df9ec000-5587df9ed000 r--p 000b6000 fd:02 21584889 /home/emilio/src/moz/gecko/obj-debug/dist/bin/firefox
5587df9ed000-5587df9ee000 rw-p 000b7000 fd:02 21584889 /home/emilio/src/moz/gecko/obj-debug/dist/bin/firefox
Note how the executable bit, which is the only one we look at, and which
is where we get exeExeAddr from (0x5587df96b000 in this case) is in the
middle of the executable, but the library will span all four ranges.
Check for whether the library _contains_ the start address of the
executable region instead of whether it starts with it.
Differential Revision: https://phabricator.services.mozilla.com/D72504
As opposed to `ProfileBufferIndex` (no "Block"), `ProfileBufferBlockIndex` is only supposed to point at a valid block start.
If we trust this assumption, it allows for quick access to the given block index inside the buffer, as we don't need to read blocks one by one until we reach the given position.
There are still safety checks (MOZ_ASSERTs in DEBUG builds) to verify that block indices are correctly used.
Differential Revision: https://phabricator.services.mozilla.com/D71502
Renamed some variables to be more generic. Their type is going to change in the next patch, and that type doesn't need to be in the names; also it will make the next patch easier to review.
Differential Revision: https://phabricator.services.mozilla.com/D71882
`explicit operator bool()` and `operator!()` were cute ways to make `InChunkBuffer` quack like a pointer when testing if it's effectively null.
But after some experience, and since `InChunkPointer` will not be used in generic code where pointers would be accepted, I now think that it's better to be clearer about it and use an explicit `IsNull()`.
Differential Revision: https://phabricator.services.mozilla.com/D71499
Same as with `BlocksRingBuffer`: Instead of a potentially-null pointer to a
`ProfileBufferEntryWriter`, we are now providing a
`Maybe<ProfileBufferEntryWriter>`, which is safer.
Differential Revision: https://phabricator.services.mozilla.com/D71287
Instead of a potentially-null pointer to a `ProfileBufferEntryWriter`, we are now providing a `Maybe<ProfileBufferEntryWriter>`, which is safer.
Differential Revision: https://phabricator.services.mozilla.com/D71286
This is needed to embed a small buffer (e.g., containing one backtrace attached
to a marker) into a bigger buffer (e.g., the main profiler buffer).
Differential Revision: https://phabricator.services.mozilla.com/D69499
--HG--
extra : moz-landing-system : lando
`InChunkPointer` is an internal accessor pointing at a position inside a chunk.
It can handle up to two groups of chunks (typically the extant chunks stored in
the chunk manager, and the current chunk.
Differential Revision: https://phabricator.services.mozilla.com/D69497
--HG--
extra : moz-landing-system : lando
To ensure that a spare chunk is ready to handle data that will eventually
overflow the current chunk, `ProfileChunkedBuffer` uses
`ProfileBufferChunk::RequestChunk()` to queue a request for a new chunk.
This request should be handled off-thread by the buffer user -- but a response
is not guaranteed, so the buffer does not rely on it and can get a new chunk
on the spot if really needed.
Because the request is asynchronous, and because either the buffer or the user
could be destroyed while a request is in flight, a shared
`RequestedChunkRefCountedHolder` object is used:
- When the request is handled, the new chunk (or nullptr) is given to the
holder.
- When the buffer needs a new chunk, it can retrieve the new chunk if the
request was successfully fulfilled.
If the requestee is destroyed first, the request won't be fulfilled and the
buffer will carry on without relying on requests.
If the requester is destroyed first, the holder (with a potential requested
chunk) will just get destroyed after the request is fulfilled or the requestee
is destroyed as well.
Differential Revision: https://phabricator.services.mozilla.com/D69495
--HG--
extra : moz-landing-system : lando
`ProfileChunkedBuffer` can handle zero or one `ProfileBufferChunkManager` at a
time, and can optionally take ownership of the manager.
Differential Revision: https://phabricator.services.mozilla.com/D69494
--HG--
extra : moz-landing-system : lando
ProfileChunkedBuffer simulates a near-infinite buffer over ProfileBufferChunks.
It uses a ProfileBufferChunkManager to get chunks and later release them.
Its use is similar to BlocksRingBuffer:
- It reserves blocks in chunks, adds some structure (just the size of the entry
that follows), and lets a user-provided writer write the entry.
- It allows reading past entries.
- It can be in an "out-of-session" state where APIs are still available but do
nothing.
It is intended to eventually replace BlocksRingBuffer.
This patch starts with the basic structure, following patches will add all
planned features.
Differential Revision: https://phabricator.services.mozilla.com/D69493
--HG--
extra : moz-landing-system : lando
`ProfileBuffer` used to check if a `BlocksRingBuffer` was in session by looking
at its buffer size.
Now `IsInSession()` should be used, it is clearer in intent, and will make the
transition to `ProfileChunkedBuffer` slightly easier.
Differential Revision: https://phabricator.services.mozilla.com/D69492
--HG--
extra : moz-landing-system : lando
Also move MOZ_MUST_USE before function declarations' specifiers and return type. While clang and gcc's __attribute__((warn_unused_result)) can appear before, between, or after function specifiers and return types, the [[nodiscard]] attribute must precede the function specifiers.
Differential Revision: https://phabricator.services.mozilla.com/D70631
--HG--
extra : moz-landing-system : lando
This commit removes `test_fix_stack_using_bpsyms.py`. That test can't easily be
modified to work with `fix_stacks.py` because it relies on internal
implementation details of `fix_stack_using_bpsym.py`. The unit testing done in
the `fix-stacks` repo provides test coverage that is as good or better.
Differential Revision: https://phabricator.services.mozilla.com/D66924
--HG--
extra : moz-landing-system : lando
With ASAN, GTest uses the old blocklist implemented in mozglue, where
the new blocklist type `RedirectToNoOpEntryPoint` behaves the same as
`DllBlocklistEntry`. The test needs to expect `LoadLibrary` to fail.
Differential Revision: https://phabricator.services.mozilla.com/D70578
--HG--
extra : moz-landing-system : lando
Please double check that I am using this correctly. I believe we are
seeing the crash in the linked bug because we are not handling hardware
faults when reading from the memory mapped file. This patch just wraps
all accesses in the MMAP_FAULT_HANDLER_ macros.
Depends on D53042
Differential Revision: https://phabricator.services.mozilla.com/D53043
--HG--
rename : modules/libjar/MmapFaultHandler.cpp => mozglue/misc/MmapFaultHandler.cpp
rename : modules/libjar/MmapFaultHandler.h => mozglue/misc/MmapFaultHandler.h
extra : moz-landing-system : lando
This patch introduces a new DLL blocklist type `RedirectToNoOpEntryPoint`
which hooks a DLL's entrypoint into a no-op function. With this technique,
we give the injected DLL no chance to run its code though we allow it to be
loaded into the process.
This new blocklist type is intended to block a DLL which is injected by IAT
patching which was planted by a kernel callback routine for LoadImage. It's
because blocking such a DLL makes a new process fail to launch.
Differential Revision: https://phabricator.services.mozilla.com/D68348
--HG--
extra : moz-landing-system : lando
This patch introduces `Kernel32ExportsSolver` which calculates RVAs of
kernel32's functions and transfers them to a target process, where the
transferred RVAs are resolved into function addresses.
Depends on D68346
Differential Revision: https://phabricator.services.mozilla.com/D68347
--HG--
extra : moz-landing-system : lando
This patch introduces a new DLL interceptor `WindowsDllEntryPointInterceptor`
which applies a hook to a target function without backing up the original
function code.
Depends on D68345
Differential Revision: https://phabricator.services.mozilla.com/D68346
--HG--
extra : moz-landing-system : lando
This patch introduces a new policy `MMPolicyInProcessEarlyStage` which does
not consume any functions imported from kernel32.dll so that we can use it
in a process's early stage i.e. before IAT is resolved.
Depends on D68344
Differential Revision: https://phabricator.services.mozilla.com/D68345
--HG--
extra : moz-landing-system : lando
`WindowsDllDetourPatcher::CreateTrampoline` does not only create a trampoline
region but also applies a patch on an original function. This patch extracts
the patching part as separate functions.
Differential Revision: https://phabricator.services.mozilla.com/D68344
--HG--
extra : moz-landing-system : lando
This patch moves the instantiation of `PEHeaders` from `CheckBlockInfo` to
`IsDllAllowed` so that `IsDllAllowed` can use an instance of `PEHeaders`.
Depends on D68342
Differential Revision: https://phabricator.services.mozilla.com/D68343
--HG--
extra : moz-landing-system : lando
This patch introduces `nt::VirtualQuery` which consumes only ntdll's functions
to reduce dependency in `MMPolicy` on kernel32.dll. With this, `MMPolicy` still
depends on kernel32.dll, that will be solved by a coming patch.
Differential Revision: https://phabricator.services.mozilla.com/D68342
--HG--
extra : moz-landing-system : lando
This patch introduces a new DLL blocklist type `RedirectToNoOpEntryPoint`
which hooks a DLL's entrypoint into a no-op function. With this technique,
we give the injected DLL no chance to run its code though we allow it to be
loaded into the process.
This new blocklist type is intended to block a DLL which is injected by IAT
patching which was planted by a kernel callback routine for LoadImage. It's
because blocking such a DLL makes a new process fail to launch.
Differential Revision: https://phabricator.services.mozilla.com/D68348
--HG--
extra : moz-landing-system : lando
This patch introduces `Kernel32ExportsSolver` which calculates RVAs of
kernel32's functions and transfers them to a target process, where the
transferred RVAs are resolved into function addresses.
Depends on D68346
Differential Revision: https://phabricator.services.mozilla.com/D68347
--HG--
extra : moz-landing-system : lando
This patch introduces a new DLL interceptor `WindowsDllEntryPointInterceptor`
which applies a hook to a target function without backing up the original
function code.
Depends on D68345
Differential Revision: https://phabricator.services.mozilla.com/D68346
--HG--
extra : moz-landing-system : lando
This patch introduces a new policy `MMPolicyInProcessEarlyStage` which does
not consume any functions imported from kernel32.dll so that we can use it
in a process's early stage i.e. before IAT is resolved.
Depends on D68344
Differential Revision: https://phabricator.services.mozilla.com/D68345
--HG--
extra : moz-landing-system : lando
`WindowsDllDetourPatcher::CreateTrampoline` does not only create a trampoline
region but also applies a patch on an original function. This patch extracts
the patching part as separate functions.
Depends on D68343
Differential Revision: https://phabricator.services.mozilla.com/D68344
--HG--
extra : moz-landing-system : lando
This patch moves the instantiation of `PEHeaders` from `CheckBlockInfo` to
`IsDllAllowed` so that `IsDllAllowed` can use an instance of `PEHeaders`.
Depends on D68342
Differential Revision: https://phabricator.services.mozilla.com/D68343
--HG--
extra : moz-landing-system : lando
This patch introduces `nt::VirtualQuery` which consumes only ntdll's functions
to reduce dependency in `MMPolicy` on kernel32.dll. With this, `MMPolicy` still
depends on kernel32.dll, that will be solved by a coming patch.
Differential Revision: https://phabricator.services.mozilla.com/D68342
--HG--
extra : moz-landing-system : lando
Similarly to changeset cfa64a6b5a87, a scope's enclosing scope is no longer updated by MovingTracer::onScopeEdge following the changes in bug 1625212. Update the suppression list accordingly.
Differential Revision: https://phabricator.services.mozilla.com/D69986
--HG--
extra : moz-landing-system : lando
RC should be non-copyable. It already is non-copyable in the atomic variant,
but not in the non-atomic variant. This ensures that RefCounted is also
non-copyable, which prevents that classes deriving from RefCounted get
accidentally copied.
Also, RC's operator= should be defined only in debug builds. As already
described in a code comment, it's only used in debug builds (to assign the
dead value), and otherwise no modifications other than incrementing &
decrementing should be possible.
Differential Revision: https://phabricator.services.mozilla.com/D69332
--HG--
extra : moz-landing-system : lando
Also move MOZ_MUST_USE before function declarations' specifiers and return type. While clang and gcc's attribute((warn_unused_result)) can appear before, between, or after function specifiers and return types, the [[nodiscard]] attribute must precede the function specifiers.
And removed a few unneded `#include "mozilla/Attributes.h"`.
Differential Revision: https://phabricator.services.mozilla.com/D69755
--HG--
extra : moz-landing-system : lando
Previously this ended up with MovingTracer doing the update itself, but now the pointer is stored in a CellHeader it's done through an overload of TraceEdge.
The TSAN stack doesn't appear to have the class for the unsafeSetPtr method that actually does the update so rather than make this apply to all uses of this method I added a supression for the next function up in the stack.
Differential Revision: https://phabricator.services.mozilla.com/D69524
--HG--
extra : moz-landing-system : lando
When our detour processes instructions, we pass `ReadOnlyTargetFunction` to
`CountPrefixBytes` to determine whether a lock prefix exists or not.
In that case, we don't need to pass both `ReadOnlyTargetFunction` and an offset
as a parameter because `ReadOnlyTargetFunction` has an offset as a member.
Differential Revision: https://phabricator.services.mozilla.com/D69360
--HG--
extra : moz-landing-system : lando
`ProfileBufferChunkManagerWithLocalLimit` is a chunk manager that enforces a
memory limit in each process.
It is meant to mimic the main way `BlocksRingBuffer` works, so that we can more
easily switch to the new buffer storage without introducing the extra complexity
of inter-process memory coordination yet.
`ProfileBufferChunkManagerWithLocalLimit` will still offer a benefit over
`BlocksRingBuffer`, in that it won't allocate the maximum buffer size
immediately -- speeding the initialization, and hopefully even reducing the
total Firefox memory consumption in short-lived processes.
Differential Revision: https://phabricator.services.mozilla.com/D68770
--HG--
extra : moz-landing-system : lando
Please double check that I am using this correctly. I believe we are
seeing the crash in the linked bug because we are not handling hardware
faults when reading from the memory mapped file. This patch just wraps
all accesses in the MMAP_FAULT_HANDLER_ macros.
Depends on D53042
Differential Revision: https://phabricator.services.mozilla.com/D53043
--HG--
rename : modules/libjar/MmapFaultHandler.cpp => mozglue/misc/MmapFaultHandler.cpp
rename : modules/libjar/MmapFaultHandler.h => mozglue/misc/MmapFaultHandler.h
extra : moz-landing-system : lando
Also adds missing includes in some files, these were previously only transivitely
included through mozilla/TypeTraits.h.
Differential Revision: https://phabricator.services.mozilla.com/D68561
--HG--
extra : moz-landing-system : lando
We still need it for Android < 6.0 (API 23) because otherwise we don't
have a way to hook up mozalloc.
Differential Revision: https://phabricator.services.mozilla.com/D66993
--HG--
extra : moz-landing-system : lando
A `ProfileBufferChunk` represents a single chunk of memory, with an optional
link to the next chunk.
In the new Fission-compatible profiler storage, chunks will be allocated by a
chunk manager, filled with data by the profiler, and then released back to the
chunk manager.
The chunk manager may decide to destroy or recycle old chunks based on memory
limits (per process, or for the entire Firefox app).
Differential Revision: https://phabricator.services.mozilla.com/D67272
--HG--
extra : moz-landing-system : lando
A `ProfileBufferChunk` represents a single chunk of memory, with an optional
link to the next chunk.
In the new Fission-compatible profiler storage, chunks will be allocated by a
chunk manager, filled with data by the profiler, and then released back to the
chunk manager.
The chunk manager may decide to destroy or recycle old chunks based on memory
limits (per process, or for the entire Firefox app).
Differential Revision: https://phabricator.services.mozilla.com/D67272
--HG--
extra : moz-landing-system : lando
We still need it for Android < 6.0 (API 23) because otherwise we don't
have a way to hook up mozalloc.
Differential Revision: https://phabricator.services.mozilla.com/D66993
--HG--
extra : moz-landing-system : lando
The old code was using `std::index_sequence_for` to call templated functions for
*every* possible Variant alternative.
Instead, using `Variant::match()` with a generic lambda (that also takes the
current Variant index, thanks to bug 1621865) means we're only calling the
function corresponding to the current alternative.
Differential Revision: https://phabricator.services.mozilla.com/D67241
--HG--
extra : moz-landing-system : lando
As far as I can tell there is no single entry-point into C++ code on Android.
Instead, GeckoThread and GeckoLibLoader call various functions to load libraries one-by-one.
We want to capture all that library loading in the profiler, so we need to kick off the base profiler at the beginning of whichever function is called first.
Differential Revision: https://phabricator.services.mozilla.com/D64999
--HG--
extra : moz-landing-system : lando
Stop including "Linker.h" and always include <link.h> instead, which also comes with ElfW() and other things that this code needs.
Also fix up "!/" path detection code.
Differential Revision: https://phabricator.services.mozilla.com/D64997
--HG--
extra : moz-landing-system : lando
This fixes the declaration of (BaseProfiler)LogTest. It also makes it so that the logs show up on Android.
In xpcom we have printf_stderr which does something similar and also handles Windows.
Differential Revision: https://phabricator.services.mozilla.com/D64994
--HG--
extra : moz-landing-system : lando
This removes most dependencies on BlocksRingBuffer, to ease the transition to
the upcoming Fission-friendly profile buffer, including:
- Length type,
- SumBytes(),
- Gecko extensions of serialization.
Differential Revision: https://phabricator.services.mozilla.com/D66722
--HG--
rename : tools/profiler/public/BlocksRingBufferGeckoExtensions.h => tools/profiler/public/ProfileBufferEntrySerializationGeckoExtensions.h
extra : moz-landing-system : lando
Fold expressions are more expressive and probably produce better code than the
older template-recursion and initializer_list patterns.
Differential Revision: https://phabricator.services.mozilla.com/D66535
--HG--
extra : moz-landing-system : lando
When we are running from a network drive the new feature in part 1 doesn't work.
So this uses DuplicateHandle instead of OpenThread to get the thread handle used
by the profiler.
It also removes a DuplicateHandle THREAD_ALL_ACCESS call that also fails and a
DuplicateHandle to get a real process handle, which only seems to have been to
fix something on Windows XP.
The handle passed in is always the profiler one, so already has the necessary
permissions. If no thread handle is passed then the pseudo handle is used.
Differential Revision: https://phabricator.services.mozilla.com/D66611
--HG--
extra : moz-landing-system : lando
When we are running from a network drive the new feature in part 1 doesn't work.
So this uses DuplicateHandle instead of OpenThread to get the thread handle used
by the profiler.
It also removes a DuplicateHandle THREAD_ALL_ACCESS call that also fails and a
DuplicateHandle to get a real process handle, which only seems to have been to
fix something on Windows XP.
The handle passed in is always the profiler one, so already has the necessary
permissions. If no thread handle is passed then the pseudo handle is used.
Depends on D66610
Differential Revision: https://phabricator.services.mozilla.com/D66611
--HG--
extra : moz-landing-system : lando
The new ProfileBufferEntryReader/Writer are now used everywhere, including in
the profilers and tests.
The old EntryReader/Writer have been removed.
Differential Revision: https://phabricator.services.mozilla.com/D65697
--HG--
extra : moz-landing-system : lando
Actual implementation of the entry reader and writer.
They expose a very similar API as the original BlocksRingBuffer entry reader&
writer.
(No tests here; In the next patch they will get used instead of the old reader&
writer, and will hence inherit all relevant tests.)
Differential Revision: https://phabricator.services.mozilla.com/D65696
--HG--
extra : moz-landing-system : lando
ProfileBufferEntrySerialization.h will contain the entry reader and writer.
This patch creates the file with renamed copies of the (de)serializers from
BlocksRingBuffer; there shouldn't be significant code changes, this `hg cp` will
help keep their history. See next patch for how they will actually be used.
Differential Revision: https://phabricator.services.mozilla.com/D65695
--HG--
rename : mozglue/baseprofiler/public/BlocksRingBuffer.h => mozglue/baseprofiler/public/ProfileBufferEntrySerialization.h
extra : moz-landing-system : lando
The blocklist variable `BROWSER_PROCESS` did not work as expected. Entries
defined there were blocked not only in the browser process but also in the
child process.
This patch makes sure entries in `BROWSER_PROCESS` are blocked only in the
browser process.
Differential Revision: https://phabricator.services.mozilla.com/D65248
--HG--
extra : moz-landing-system : lando
Apps targeting SDK 29 are not allowed to open /dev/ashmem directly, and
instead must use NDK functions. Those functions are only available in
SDK 26 and higher, so we need this shim to use the functions if they
are available, else fallback to opening /dev/ashmem directly.
Differential Revision: https://phabricator.services.mozilla.com/D61012
--HG--
extra : moz-landing-system : lando
MOZ_PROFILE_GENERATE is already defined in mozilla-config.h and doesn't
need to be re-defined by the moz.build files.
Differential Revision: https://phabricator.services.mozilla.com/D65210
--HG--
extra : moz-landing-system : lando
In the current Win 8.0, these functions both start with a RIP-relative JMP (6 bytes) followed by 6 nops (6-bytes), which does not give us the 13-bytes we need for a trampoline so we require the trampoline to fit into 10 bytes.
Differential Revision: https://phabricator.services.mozilla.com/D63260
--HG--
extra : moz-landing-system : lando
We copy IAT for ntdll.dll into a new process so that our hook code can use
ntdll's functions even in the early stage. However, IAT can be modified and
some entries may point to an address which is not valid in the child process.
In such a case, we should not copy IAT. One example is Windows compat mode
which redirects some ntdll functions into AcLayers.dll via IAT.
With this patch, we verify each IAT entry and if any of them is outside ntdll,
we give up using the launcher process and start the browser process.
Differential Revision: https://phabricator.services.mozilla.com/D62852
--HG--
extra : moz-landing-system : lando
To facilitate the upcoming transition to a new Fission-friendly storage, all
uses of `BlocksRingBuffer::{,Block}Index` are replaced with
`ProfileBuffer{,Block}Index`.
`BlocksRingBuffer::{,Block}Index` are not needed anymore.
Differential Revision: https://phabricator.services.mozilla.com/D64516
--HG--
extra : moz-landing-system : lando
ProfileBufferIndex.h contains:
- `ProfileBufferIndex`, a generic index into a Profiler buffer.
- `ProfileBufferBlockIndex`, an almost-opaque index that should only point at
valid (or expired) blocks.
These will soon be used instead of `BlocksRingBuffer::{,Block}Index`, to make
the transition away from `BlocksRingBuffer` easier.
Differential Revision: https://phabricator.services.mozilla.com/D64515
--HG--
extra : moz-landing-system : lando
In x86, our detour handles opcode 83 only when the Mod bits is 3.
When working on another project, I hit the instruction `cmp [ebp+0Ch],1`
where the Mod bits is 1, and it can be easily handled by a small fix.
It turned out my project does not need it, but it'd be good to have this.
Differential Revision: https://phabricator.services.mozilla.com/D64196
--HG--
extra : moz-landing-system : lando
Some of the BaseProfile headers are always available, even when BaseProfiler
itself is not built, so we should test these in all cases.
Differential Revision: https://phabricator.services.mozilla.com/D63230
--HG--
extra : moz-landing-system : lando
Some of the BaseProfile headers are always available, even when BaseProfiler
itself is disabled, so we should test these in all cases.
Differential Revision: https://phabricator.services.mozilla.com/D63230
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
Apps targeting SDK 29 are not allowed to open /dev/ashmem directly, and
instead must use NDK functions. Those functions are only available in
SDK 26 and higher, so we need this shim to use the functions if they
are available, else fallback to opening /dev/ashmem directly.
Differential Revision: https://phabricator.services.mozilla.com/D61012
--HG--
extra : moz-landing-system : lando
Nathan Froyd
Byron Campen [:bwc]
Aaron Klotz
Gerald Squelart
Tyson Smith
Randell Jesup
Sylvestre Ledru
Dana Keeler
Mike Hommey
Csoregi Natalia
Christian Holler
Nick Alexander
Toshihito Kikuchi
Simon Giesecke
Cameron Kaiser
Jon Coppeard
Ted Campbell
Paul Adenot
Botond Ballo
Kevin Jacobs
Ricky Stewart
Razvan Maries
Greg Tatum
Andrew McCreight
Markus Stange
Gabriele Svelto
alwu
Jason Kratzer
Jared Wein
Chris Fronk
Greg V
Bogdan Tara
Arash Fotouhi
Emilio Cobos Álvarez
Nazım Can Altınova
Chris Peterson
Nicholas Nethercote
Doug Thayer
Andreas Farre
Daniel Varga
Andrea Marchesini
Eric Rahm
Dorel Luca
André Bargull
James Willcox
Bob Owen
Narcis Beleuzu
shindli
Mike Shal
David Parks
Ciure Andrei
Arthur Iakab
Michal Novotny
Cosmin Sabou
Coroiu Cristina
Brindusan Cristian