Unique strings are used to encode all markers' 'name' field, SpliceableJSONWriter::UniqueStringElement can be used for that (instead of a caller-provided callback, which was necessary before UniqueJSONStrings was de-duplicated).
Differential Revision: https://phabricator.services.mozilla.com/D95513
Some markers (e.g., Screenshot) use unique strings in their data.
The UniqueJSONStrings used during streaming is attached to the SpliceableJSONWriter, and StreamJSONMarkerData can use pass-through functions UniqueStringProperty() and UniqueStringElement().
Differential Revision: https://phabricator.services.mozilla.com/D95512
Some markers (e.g., GC major/minor/slice) need to splice JSON strings in their data.
So now, instead of JSONWriter, StreamJSONMarkerData functions will be given a mozilla::baseprofiler::SpliceableJSONWriter.
Differential Revision: https://phabricator.services.mozilla.com/D95511
`UniqueJSONStrings` constructors now accept a `JSONWriter::CollectionStyle` to pass to the internal JSON writer.
The default won't change how the unique string list is output (with friendly indented lines), but tests can use `JSONWriter::SingleLineStyle` to make comparison strings more readable and maintainable.
Differential Revision: https://phabricator.services.mozilla.com/D95832
One of the scenarios in TestMMPolicy.exe is to reserve all regions but one free
region, and then FindRegion can find that free region. However, it intermittently
failed to find a free region on 32bit. Probably a different thread invoked by
the system reserved it before the main thread does. A proposed fix is to limit
the address range to reserve.
Differential Revision: https://phabricator.services.mozilla.com/D95188
The latest launcher process showed one of the top failures was `WriteProcessMemory` in
`CopyStubToChildProcess` failed with `ERROR_INVALID_ADDRESS` or `ERROR_NOACCESS`, that
is to store a trampoline address to the global variable of firefox.exe failed. Its root
cause should be the same as bug 1662560, the executable was loaded onto a different
address from the browser process.
The fix is to to expand the usage of `CrossExecTransferManager` to `FuncHookCrossProcess`
and `Kernel32ExportsSolver`.
Depends on D94652
Differential Revision: https://phabricator.services.mozilla.com/D94653
This commit uses the new Markers 2.0 API for FileIO Markers. I had to
create another option for the MarkerStack class in order to conditionally
capture a backtrace inside of the Macro. Otherwise the macro invocation
failed.
Differential Revision: https://phabricator.services.mozilla.com/D93848
This commit uses the new Markers 2.0 API for FileIO Markers. I had to
create another option for the MarkerStack class in order to conditionally
capture a backtrace inside of the Macro. Otherwise the macro invocation
failed.
Differential Revision: https://phabricator.services.mozilla.com/D93848
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
The latest launcher process ping showed one of the reasons why we failed to
detour `NtMapViewOfSection` is that `MMPolicyBase::FindRegion` failed to find
a free region. Inspecting the function carefully, there were three problems.
Firstly, `FindRegion` did not fully scan the given range. To randomize
the address of a free region we use, we start scanning from a random address
within the given range. The problem is we scan only addresses bigger than
that random address, without scanning smaller addresses. Probably this is
the reason why `FindRegion` fails.
Secondly, `FindRegion` may return an address not aligned with the allocation
granularity because `VirtualQueryEx` returns such an address. If that happens,
the subsequent mapping API fails with the alignment error.
Lastly, when we randomize an address to start scanning from, we divide a random
number by `maxOffset`, but with that, we never start scanning from the last
region. It does not affect the product's behavior, but to have fair randomization,
a divisor should be `maxOffset + 1`.
This patch fixes all of these three problems along with a new test program.
Differential Revision: https://phabricator.services.mozilla.com/D94110
Everything related to Base Profiler legacy markers can now be removed, only the new API from BaseProfilerMarkers.h should now be used.
Depends on D93737
Differential Revision: https://phabricator.services.mozilla.com/D93738
Since the main thread is almost always being profiled, this makes it easy to direct important markers there (e.g., FileIO markers from unregistered threads).
Tech note: The `#include "BaseProfiler.h"` line was moved to BaseProfilerMarkersPrerequesites.h so that `MarkerThreadId::MainThread()` there can access `profiler_main_thread_id()`.
Depends on D93735
Differential Revision: https://phabricator.services.mozilla.com/D93736
These functions will be useful to get the main thread id, or check if we're in it, in some public code (e.g., markers).
Differential Revision: https://phabricator.services.mozilla.com/D93735
These unit tests were going through chunks very quickly in tight loops, which on some platforms could cause successive chunks to be marked "done" with the same timestamps.
Distinct timestamps are needed to uniquely identify chunks (in each process), and related assertions caught that. In real Firefox code, chunks are much bigger (around 1MB), and fill at a slower rate, so timestamps are always different as expected there.
To fix this artificial issue in our unit tests, delays are introduced before each `MarkDone()`, to ensure that they'll get a different timestamp every time.
Differential Revision: https://phabricator.services.mozilla.com/D93479
Add `static mozilla::MarkerSchemaWriter MarkerTypeDisplay()` for each existing marker type.
Because all markers of a given type now must have the same payload and display schema, the DOM event marker has changed from type=tracing with category=DOMEvent, to its own type=DOMEvent, which is now accepted on the front-end.
Based on c9692715f2/src/profile-logic/marker-schema.js
Differential Revision: https://phabricator.services.mozilla.com/D90658
This is the third attempt to investigate the launcher failure of our detour.
The previous commits d8315e4ed18d and 1b81ea85c43d added the assembly bytes
of a detour target and a special error code `DetourResultCode` to the launcher
failure ping.
In the latest telemetry data, however, the most common value of `hresult`
is still `ERROR_UNIDENTIFIED_ERROR`, meaning the previous commit missed to
set an error code in the common fallible codepath we wanted to know.
Besides `ERROR_UNIDENTIFIED_ERROR`, we're seeing `DETOUR_PATCHER_DO_RESERVE_ERROR`
in the telemetry, but having that code is not enough to pinpoint a falling
operation.
For further investigation, this patch adds ten more values to `DetourResultCode`.
`FUNCHOOKCROSSPROCESS_COPYSTUB_ERROR` is the last codepath we forgot to cover
in the previous commit. The values of `MMPOLICY_RESERVE_*` are to investigate
`DETOUR_PATCHER_DO_RESERVE_ERROR` in the MMPolicy level. In both cases, we add
the last Windows error code to `DetourError::mOrigBytes`.
Differential Revision: https://phabricator.services.mozilla.com/D92974
The latest Windows Insider Preview (version 20226.1000) changes the machine code for BaseThreadInitThunk to have a preamble like the following:
00007FFDBF244C40 48 83 EC 28 sub rsp,28h
00007FFDBF244C44 85 C9 test ecx,ecx
00007FFDBF244C46 75 25 jne 00007FFDBF244C6D
00007FFDBF244C48 49 BA 70 A2 DC 12 6A 97 99 B0 mov r10,0B099976A12DCA270h
This patch adds "MOV r64, imm64" capability to the DLL interceptor so that we can hook this.
Differential Revision: https://phabricator.services.mozilla.com/D92146
This makes it clearer where marker-type-specific payload arguments start, just after the marker type object.
Also improved the main API documentation.
Differential Revision: https://phabricator.services.mozilla.com/D91681
The `category.WithOptions(...)` syntax was a bit strange and difficult to explain.
Now the category and options are separate parameters. Default options can be specified with `MarkerOptions{}` or just `{}`.
As a special case, defaulted-NoPayload functions don't need `<>`, and defaulted-NoPayload functions and macros don't even need `{}` for default options, e.g.:
`profiler_add_marker("name", OTHER); PROFILER_MARKER_UNTYPED("name", OTHER);`
Differential Revision: https://phabricator.services.mozilla.com/D91680
Gerald Squelart
Razvan Maries
Mihai Alexandru Michis
Toshihito Kikuchi
Greg Tatum
Butkovits Atila
Ricky Stewart
Bogdan Tara
Dorel Luca
David Parks