`BlocksRingBuffer` had an "entry destructor" to make it a more generic
container, and it was useful during early prototyping of the new profiler
storage (so that we could store owning pointers).
But this entry destructor is stored in an `std::function`, which gets marked as
a potential GC caller by the js rooting hazard analyzer; and as this bug found,
it's not obvious where to place `JS::AutoSuppressGCAnalysis`, because profiler
entries (including stacks) could be added on one thread while GC happens
elsewhere, which triggers the embedded `AutoAssertNoGC` check.
Since we don't actually use the entry destructor facility in the profiler, it's
easier to just get rid of it. As a bonus, it's a small optimization.
Tests that were using an entry destructor now use the `State` instead, to verify
that entries are pushed and cleared as expected.
If needed in the future outside of the profiler, `BlocksRingBuffer` could again
include an entry destructor, but it would have to be through templating, so that
the class used in the profiler wouldn't contain an `std::function`.
Differential Revision: https://phabricator.services.mozilla.com/D46738
--HG--
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
This patch adds the following:
* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
owns its buffer. The buffers are null-terminated so that they may be used as
C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
current thread ID directly from the `TEB` when we do not yet have access to
kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
MFBT `Vector` in code that might not yet have access to Win32 heap functions.
Differential Revision: https://phabricator.services.mozilla.com/D43155
--HG--
extra : moz-landing-system : lando
Some objects are copied byte-by-byte to/from `ModuloBuffer`s.
E.g., serialized `BlocksRingBuffer`s, or duplicate stacks. (And more to come.)
`Iterator::ReadInto(Iterator&)` optimizes these copies by using the minimum
number of `memcpy`s possible.
Differential Revision: https://phabricator.services.mozilla.com/D45838
--HG--
extra : moz-landing-system : lando
Since payloads are not kept alive long after they have been serialized, we can
just create them on the stack and pass a reference to their base (or pointer,
`nullptr` representing "no payloads") to `profiler_add_marker()`.
Differential Revision: https://phabricator.services.mozilla.com/D43430
--HG--
extra : moz-landing-system : lando
Copy the full contents of BlocksRingBuffer into another one.
This is mainly useful to use a temporary buffer to store some data without
contentions, and then integrate the temporary buffer into the main one.
Differential Revision: https://phabricator.services.mozilla.com/D45306
--HG--
extra : moz-landing-system : lando
Payloads will serialize themselves into a `BlocksRingBuffer` entry when first
captured.
Later they will be deserialized, to stream JSON for the output profile.
Differential Revision: https://phabricator.services.mozilla.com/D43428
--HG--
extra : moz-landing-system : lando
Add basic testing of `profiler_get_backtrace` before working on serializing it.
Differential Revision: https://phabricator.services.mozilla.com/D43424
--HG--
extra : moz-landing-system : lando
In some situations we will *need* to use a `BlocksRingBuffer` that absolutely
does not use a mutex -- In particular in the critical section of
`SamplerThread::Run()`, when a thread is suspended, because any action on any
mutex (even a private one that no-one else interacts with) can result in a hang.
As a bonus, `BlocksRingBuffer`s that are known not to be used in multi-thread
situations (e.g., backtraces, extracted stacks for duplication, etc.) will waste
less resources trying to lock/unlock their mutex.
Differential Revision: https://phabricator.services.mozilla.com/D45305
--HG--
extra : moz-landing-system : lando
Some objects are copied byte-by-byte to/from `ModuloBuffer`s.
E.g., serialized `BlocksRingBuffer`s, or duplicate stacks. (And more to come.)
`Iterator::ReadInto(Iterator&)` optimizes these copies by using the minimum
number of `memcpy`s possible.
Differential Revision: https://phabricator.services.mozilla.com/D45838
--HG--
extra : moz-landing-system : lando
Since payloads are not kept alive long after they have been serialized, we can
just create them on the stack and pass a reference to their base (or pointer,
`nullptr` representing "no payloads") to `profiler_add_marker()`.
Differential Revision: https://phabricator.services.mozilla.com/D43430
--HG--
extra : moz-landing-system : lando
Copy the full contents of BlocksRingBuffer into another one.
This is mainly useful to use a temporary buffer to store some data without
contentions, and then integrate the temporary buffer into the main one.
Differential Revision: https://phabricator.services.mozilla.com/D45306
--HG--
extra : moz-landing-system : lando
Payloads will serialize themselves into a `BlocksRingBuffer` entry when first
captured.
Later they will be deserialized, to stream JSON for the output profile.
Differential Revision: https://phabricator.services.mozilla.com/D43428
--HG--
extra : moz-landing-system : lando
Add basic testing of `profiler_get_backtrace` before working on serializing it.
Differential Revision: https://phabricator.services.mozilla.com/D43424
--HG--
extra : moz-landing-system : lando
In some situations we will *need* to use a `BlocksRingBuffer` that absolutely
does not use a mutex -- In particular in the critical section of
`SamplerThread::Run()`, when a thread is suspended, because any action on any
mutex (even a private one that no-one else interacts with) can result in a hang.
As a bonus, `BlocksRingBuffer`s that are known not to be used in multi-thread
situations (e.g., backtraces, extracted stacks for duplication, etc.) will waste
less resources trying to lock/unlock their mutex.
Differential Revision: https://phabricator.services.mozilla.com/D45305
--HG--
extra : moz-landing-system : lando
Backtraces (that are kept in some marker payloads) are stored in a small
ProfileBuffer, we will need to store that data, which will happen to be inside
a BlockRingBuffer, so BlockRingBuffer needs to be able to (de)serialize itself!
This is done by storing the contents in the active buffer range, and some extra
data, to later reconstruct a BlocksRingBuffer that looks like the original.
Depends on D42496
Differential Revision: https://phabricator.services.mozilla.com/D42634
--HG--
extra : moz-landing-system : lando
Markers and their payloads contain all kinds of objects that we'll need to
serialize into a BlocksRingBuffer (new ProfileBuffer storage).
This patch will add functions to:
- Compute the size needed to store objects,
- Write multiple objects into a BlockRingBuffer entry,
- Read objects back from an entry.
And it will provide a number of useful de/serialization helpers for:
- Trivially-copyable objects,
- Strings of different types,
- Raw pointers (with some safety guards to avoid surprises),
- Tuples (to store multiple sub-objects),
- Spans,
- Maybe (for optional objects),
- Variant.
This should be enough to store most kinds of data. Further specializations
can&will be written as necessary for more complex or obscure types.
Differential Revision: https://phabricator.services.mozilla.com/D42496
--HG--
extra : moz-landing-system : lando
This allows its use in std algorithms and types that require a real iterator
(like `template <typename InputIt> std::string::string(InputIt, InputIt)`).
Differential Revision: https://phabricator.services.mozilla.com/D42452
--HG--
extra : moz-landing-system : lando
`Reader::At()` can be used to get a `BlockIterator` at a given `BlockIndex`,
clamped between `begin()` and `end()`.
This will be useful when we want to iterate starting at a given index, e.g.,
when duplicating stacks.
Differential Revision: https://phabricator.services.mozilla.com/D42449
--HG--
extra : moz-landing-system : lando
The main goal of these bugs is to move markers to a new storage, so I'm adding
lots of markers to TestBaseProfiler.
Also adding labels, easier to read unsymbolicated profiles, and gives a bit more
coverage too.
And adding a separate "fibonacci canceller" thread, which is needed on some
slower platforms (e.g., Linux 64 ASAN times out otherwise); as a bonus, this
tests AUTO_BASE_PROFILER_REGISTER_THREAD.
Differential Revision: https://phabricator.services.mozilla.com/D42448
--HG--
extra : moz-landing-system : lando
In practice the Reader doesn't need to be copied/moved/reassign.
BlocksRingBuffer::Read() can just instantiate one on the stack, and pass it by
reference to callbacks.
Differential Revision: https://phabricator.services.mozilla.com/D42118
--HG--
extra : moz-landing-system : lando
The point of the EntryReserver was mainly to have an object that represented a
writing lock on BlocksRingBuffer, so potentially perform multiple consecutive
writes.
After some experience implementing bug 1562604, there's actually no need for it.
So instead of having `Put()` create an `EntryReserver`, we now have
`ReserveAndPut()` that does the whole work in one function.
Differential Revision: https://phabricator.services.mozilla.com/D42116
--HG--
extra : moz-landing-system : lando
EntryWriter doesn't even need to be moveable, as BlocksRingBuffer can just
create one on the stack, and pass it by reference to callbacks.
This removes risks, and potential data copies.
Differential Revision: https://phabricator.services.mozilla.com/D42115
--HG--
extra : moz-landing-system : lando
`BlocksRingBuffer` will be used both inside and outside `ProfileBuffer`:
- Inside to serve as `ProfileBuffer`'s main storage for stack traces,
- Outside to allow marker storage even when `ProfileBuffer` is locked during
stack sampling.
`ProfileBuffer` only exists while `ActivePS` is alive, but because of the
potential outside accesses above (due to small races between ProfileBuffer
shutdown, and thread-local IsBeingProfiled() flags), we cannot just do the same
for BlocksRingBuffer, and it must remain alive to gracefully deny these accesses
around the profiler startup and shutdown times.
To accomplish this, `BlocksRingBuffer` may be in different states:
- "In-session", we have a real buffer to write to and read from,
- "Out-of-session", without buffer so reads&writes do nothing.
This is implemented by enclosing the underlying `ModuloBuffer` and the entry
deleter in a `Maybe`, which may be `Nothing` when the profiler is not running
and the `ProfileBuffer`'s `BlocksRingBuffer` is out-of-session.
Differential Revision: https://phabricator.services.mozilla.com/D41519
--HG--
extra : moz-landing-system : lando
`BlocksRingBuffer` will be used both inside and outside `ProfileBuffer`:
- Inside to serve as `ProfileBuffer`'s main storage for stack traces,
- Outside to allow marker storage even when `ProfileBuffer` is locked during
stack sampling.
`ProfileBuffer` only exists while `ActivePS` is alive, but because of the
potential outside accesses above (due to small races between ProfileBuffer
shutdown, and thread-local IsBeingProfiled() flags), we cannot just do the same
for BlocksRingBuffer, and it must remain alive to gracefully deny these accesses
around the profiler startup and shutdown times.
To accomplish this, `BlocksRingBuffer` may be in different states:
- "In-session", we have a real buffer to write to and read from,
- "Out-of-session", without buffer so reads&writes do nothing.
This is implemented by enclosing the underlying `ModuloBuffer` and the entry
deleter in a `Maybe`, which may be `Nothing` when the profiler is not running
and the `ProfileBuffer`'s `BlocksRingBuffer` is out-of-session.
Differential Revision: https://phabricator.services.mozilla.com/D41519
--HG--
extra : moz-landing-system : lando
After some bad experiences, I think EntryReader should be move-only:
- It needs to be moveable so it can be created from a function, and move-
constructed into a Maybe<> if needed.
- It can be passed around as a reference.
Previously, it could be passed by value, but it was too easy to create bugs,
e.g.: A function delegates to a sub-function to read something at the beginning,
then the first function wants to read more past that, but if the reader was
passed by value the first function would not see past what the sub-function did
read.
As a bonus, `mRing` can now be a reference instead of a pointer, and other
members can be const.
Differential Revision: https://phabricator.services.mozilla.com/D40958
--HG--
extra : moz-landing-system : lando
This makes it easier to grab all BlocksRingBuffer state variables:
- Range start and end.
- Number of pushed blocks/entries, number of cleared blocks/entries.
The function is thread-safe, and the returned values are consistent with each
other, but they may become stale straight after the function returns (and the
lock is released).
They are still valuable to statistics, and to know how far the range has at
least reached (but may go further soon).
Differential Revision: https://phabricator.services.mozilla.com/D40621
--HG--
extra : moz-landing-system : lando
cppunittest TestBaseProfiler and gtest GeckoProfiler.Markers now show overhead
stats.
(Separate patch, because we may want to remove them after a while.)
Differential Revision: https://phabricator.services.mozilla.com/D39642
--HG--
extra : moz-landing-system : lando
`ClearBefore()` with a past-the-end `BlockIndex` was calling `Clear()`, which
tried to take the lock again! Also we didn't return after that.
Fixed, and added corresponding test.
Also: Removed ambiguous "delete" word, now using more precise "destroy" or
"entry destructor".
Differential Revision: https://phabricator.services.mozilla.com/D38846
--HG--
extra : moz-landing-system : lando
This is a similar concept as `nullptr` is to a pointer.
`BlocksRingBuffer` now skips the first byte in the buffer, so that no entries
start at 0 (the internal default `BlockIndex` value).
All `BlocksRingBuffer` public APIs handle this default value, and do nothing
and/or return Nothing (as if it pointed at an already-deleted entry).
Added tests for this, and for all BlockIndex operations.
Differential Revision: https://phabricator.services.mozilla.com/D38667
--HG--
extra : moz-landing-system : lando
Without declaring them, ModuloBuffer had its copy&move constructor&assignments
defaulted. This means it could have been copied, and then both objects would now
own the same resource and attempt to free it on destruction!
So now:
- Copy construction&assignment are now explicitly disallowed.
- Move assignment is disallowed, to keep some members `const`.
- Move construction is allowed (so a function can return a ModuloBuffer), and
ensures that the moved-from object won't free the resource anymore.
Bonus: `mBuffer` is now `const`, to ensure that it cannot point at something
else, but note the pointed-at bytes are *not* const.
So ModuloBuffer is like an unchanging resource, but it allows to be moved-from
as an xvalue that should not be used after the move.
Differential Revision: https://phabricator.services.mozilla.com/D38665
--HG--
extra : moz-landing-system : lando
By default `ModuloBuffer` allocates its own buffer on the heap.
Now `ModuloBuffer` adds two alternatives:
- Take ownership of a pre-allocated `UniquePtr<uint8_t>` buffer.
- Work over an unowned `uint8_t*` array. The caller is responsible for
ownership, and ensuring that the array lives at least as long as the
`ModuloBuffer`/`BlocksRingBuffer`.
`BlocksRingBuffer` can pass along these new options to its underlying
`ModuloBuffer`.
The main use will be for small on-stack `BlocksRingBuffer` that can store a
stack trace, or to more easily collect data (without allocating anything on the
heap) that can then go into the upcoming `ProfileBuffer`'s `BlocksRingBuffer`.
Differential Revision: https://phabricator.services.mozilla.com/D38285
--HG--
extra : moz-landing-system : lando
This adds to the byte-oriented ModuloBuffer from bug 1563425:
- Thread-safety: All APIs may be called at any time from any thread.
- Structure: The buffer will be divided in "blocks" of different size, with some
block meta-data and space for the user "entry".
- Capable of handling user resources: The user may provide a "deleter" that will
be informed about soon-to-be-destroyed entries; so if some entries reference
outside resources, these references may be properly released.
Note: This first implementation still only allows the user to manipulate bytes
and trivially-copyable objects (same as with the ModuloBuffer iterators). A
follow-up bug will introduce better serialization capabilities, with the aim to
eventually store everything that current Profiler Markers and their payloads
contain.
Differential Revision: https://phabricator.services.mozilla.com/D37702
--HG--
extra : moz-landing-system : lando
Since higher-level APIs that we test may depend on lower-level APIs that we
also test, and since those higher-level APIs may spin up background threads
that call those lower-level APIs, we should ensure that tests are ordered
such that the lower-level APIs are hooked first, thus preventing races where
higher-level background threads call lower-level APIs while the test's main
thread is in the midst of hooking a lower-level API.
I also added some fflush calls to the test so that, the next time we see lots
of crashes in this test, the log output is more complete.
Differential Revision: https://phabricator.services.mozilla.com/D37497
--HG--
extra : moz-landing-system : lando
Basic usage:
- Create buffer: `ModuloBuffer mb(PowerOfTwo);`
- Get iterator: `auto writer = mb.WriterAt(Index);` (or `ReaderAt()`)
- Basic iterator functions on bytes: `*++writer = 'x';`
- Write: `writer.WriteULEB128(sizeof(int)); writer.WriteObject<int>(42);`
- Comparisons, move: `while (writer > reader) { --writer; }`
- Read: `size_t s = reader.ReadULEB128<size_t>(); int i = ReadObject<int>();`
There are no safety checks, it will be up to the caller to ensure thread-safety,
and data safety when wrapping around the buffer.
Differential Revision: https://phabricator.services.mozilla.com/D36869
--HG--
extra : moz-landing-system : lando
The new ProfileBuffer data structure will need to store block sizes (usually
small, LEB128 uses fewer bytes for small numbers), and the circular buffer will
provide iterators that hide the wrapping-around.
Differential Revision: https://phabricator.services.mozilla.com/D36473
--HG--
extra : moz-landing-system : lando
PowerOfTwo stores a power of 2 value, i.e., 2^N.
PowerOfTwoMask stores a mask corresponding to a power of 2, i.e., 2^N-1.
These should be used in places where a power of 2 (or its mask) is stored or
expected.
`% PowerOfTwo{,Mask}` and `& PowerOfTwoMask` operations are optimal.
MakePowerOfTwo{,Mask}<T, Value>() may be used to create statically-checked
constants.
{,Make}PowerOfTwo{,Mask}{32,64} shortcuts for common 32- and 64-bit types.
Differential Revision: https://phabricator.services.mozilla.com/D36026
--HG--
extra : moz-landing-system : lando
PowerOfTwo stores a power of 2 value, i.e., 2^N.
PowerOfTwoMask stores a mask corresponding to a power of 2, i.e., 2^N-1.
These should be used in places where a power of 2 (or its mask) is stored or
expected.
`% PowerOfTwo{,Mask}` and `& PowerOfTwoMask` operations are optimal.
MakePowerOfTwo{,Mask}<T, Value>() may be used to create statically-checked
constants.
{,Make}PowerOfTwo{,Mask}{32,64} shortcuts for common 32- and 64-bit types.
Differential Revision: https://phabricator.services.mozilla.com/D36026
--HG--
extra : moz-landing-system : lando
Gerald Squelart
Aaron Klotz
Brindusan Cristian
Ciure Andrei
Andreea Pavel
Bogdan Tara
Toshihito Kikuchi
Sylvestre Ledru