This patch supports a skeleton UI for default, light, and dark themes.
It is not enabled for apenglow or any custom themes.
This also takes into account the system theme. If the user has the default
theme selected and is in dark mode, we override the theme and present the
dark theme skeleton UI.
Differential Revision: https://phabricator.services.mozilla.com/D96230
This patch adds a list of the executable's dependent module's path to SharedSection
as an array of the offset to a string and a string buffer. A following patch will
use this data from the browser process and the sandboxed processes.
Depends on D96283
Differential Revision: https://phabricator.services.mozilla.com/D96284
We transfer several ntdll's function addresses to a child process directly via
`WriteProcessMemory`. This patch changes the way to transfer data to using
a section object as Chromium sandbox does, so that we can transfer more data
with the same cost as transferring a single handle value.
Depends on D96282
Differential Revision: https://phabricator.services.mozilla.com/D96283
If a searchbar is present, draw that similar to a urlbar.
This also changes how we store the urlbar, as we know will use our custom struct
to cleanly store the width and height.
Differential Revision: https://phabricator.services.mozilla.com/D95029
Unique strings are used to encode all markers' 'name' field, SpliceableJSONWriter::UniqueStringElement can be used for that (instead of a caller-provided callback, which was necessary before UniqueJSONStrings was de-duplicated).
Differential Revision: https://phabricator.services.mozilla.com/D95513
Some markers (e.g., Screenshot) use unique strings in their data.
The UniqueJSONStrings used during streaming is attached to the SpliceableJSONWriter, and StreamJSONMarkerData can use pass-through functions UniqueStringProperty() and UniqueStringElement().
Differential Revision: https://phabricator.services.mozilla.com/D95512
Some markers (e.g., GC major/minor/slice) need to splice JSON strings in their data.
So now, instead of JSONWriter, StreamJSONMarkerData functions will be given a mozilla::baseprofiler::SpliceableJSONWriter.
Differential Revision: https://phabricator.services.mozilla.com/D95511
`UniqueJSONStrings` constructors now accept a `JSONWriter::CollectionStyle` to pass to the internal JSON writer.
The default won't change how the unique string list is output (with friendly indented lines), but tests can use `JSONWriter::SingleLineStyle` to make comparison strings more readable and maintainable.
Differential Revision: https://phabricator.services.mozilla.com/D95832
The shutdown code of BackgroundReadFiles races with BeginBackgroundRead.
This is paritally obfuscated by the usage of the initialEvent convenience
of NS_NewNamedThread, so this change also removes that in favour of explicit
dispatch. (xpcom devs want to remove the convenience anyway)
Differential Revision: https://phabricator.services.mozilla.com/D94877
The previous patch removed the only two uses of ProfilerStringView::String().
Since it can be potentially expensive (creating a `std::string` object, sometimes allocating a buffer, and copying the string contents), it's best to remove it completely.
Differential Revision: https://phabricator.services.mozilla.com/D95116
For consistency with `JSONWriter` (which UniqueJSONStrings' functions use), and for added safety and some efficiency, UniqueJSONStrings now takes `Span<const char`> arguments instead of raw pointers to null-terminated strings.
Differential Revision: https://phabricator.services.mozilla.com/D95114
Document the class and methods.
`GetOrAddIndex` is only used internally, so it can be private.
`SpliceStringTableElements` can now only work on rvalue UniqueJSONStrings, this emphasizes that it shouldn't be used anymore after this call.
Differential Revision: https://phabricator.services.mozilla.com/D95113
Most of this patch is a dance to avoid size flickering of the skeleton UI
window. We change all Resize/Move/SetSizeMode calls from before the first
nsWindow::Show call. Normally those have no effect, since the window isn't
shown yet, and if the window is not maximized, they typically match the
sizes we've gotten out of the registry anyway. However, if we are maximized,
then they produce a lot of visual noise. We can however achieve the desired
effect by just calling SetWindowPlacement.
Similarly, we switch the window styles of the skeleton UI window to match those
of the toplevel Windows window, and adjust the client rect from our window proc
in a way that matches the adjustments in nsWindow in the WM_NCCALCSIZE handler.
We do this because otherwise we get a flicker as soon as we change the styles
and nonclient margins as the fake chrome pops up and then back down.
Lastly we also change the extended window styles so that they match. We
historically added WS_EX_TOOLWINDOW here to hide the toolbar entry, because it
would otherwise switch out to a new toolbar entry when we changed the window
styles. However since our new styles match, we no longer need to do this. It
was also causing the maximized window to paint over the Windows taskbar.
Differential Revision: https://phabricator.services.mozilla.com/D93534
For a better user experience during slow startups (and to match the design
by Markus Jaritz), we want to animate the placeholder elements (the grey
rectangles which hold the place of text and icons) with a light moving
gradient.
A question may arise regarding whether the performance cost of running this
animation is worth the improved user experience. My claim is yes, hinging
mostly on the observation that the performance cost is small.
On my machine, one frame of the animation takes at most 0.15ms, and runs
every 16.15ms. This means we eat less than 1% CPU time on one core of the
system. It should also be noted that this animation runs primarily during
the window wherein we are prefetching dlls, AKA while we are blocked on IO
and the CPU is more likely to be idle.
On slower systems, one frame may take longer - however, on slower systems
we should also be blocked *more* by IO, making the trade favorable.
For further anecdotal evidence of this being okay, when I run this on slow
reference hardware shortly after OS startup, I do not see any dropped frames,
indicating that this has very little trouble completing, and is thus quite
cheap.
Regarding testing, I will invoke the same logic as for the rest of the
skeleton UI patches - it would involve quite a bit of work to test this in
automation given our existing frameworks. It may be worth it at some point,
but certainly not while this is a feature that we are just experimenting
with and which is not enabled by default.
Differential Revision: https://phabricator.services.mozilla.com/D91453
One of the scenarios in TestMMPolicy.exe is to reserve all regions but one free
region, and then FindRegion can find that free region. However, it intermittently
failed to find a free region on 32bit. Probably a different thread invoked by
the system reserved it before the main thread does. A proposed fix is to limit
the address range to reserve.
Differential Revision: https://phabricator.services.mozilla.com/D95188
Most of this patch is a dance to avoid size flickering of the skeleton UI
window. We change all Resize/Move/SetSizeMode calls from before the first
nsWindow::Show call. Normally those have no effect, since the window isn't
shown yet, and if the window is not maximized, they typically match the
sizes we've gotten out of the registry anyway. However, if we are maximized,
then they produce a lot of visual noise. We can however achieve the desired
effect by just calling SetWindowPlacement.
Similarly, we switch the window styles of the skeleton UI window to match those
of the toplevel Windows window, and adjust the client rect from our window proc
in a way that matches the adjustments in nsWindow in the WM_NCCALCSIZE handler.
We do this because otherwise we get a flicker as soon as we change the styles
and nonclient margins as the fake chrome pops up and then back down.
Lastly we also change the extended window styles so that they match. We
historically added WS_EX_TOOLWINDOW here to hide the toolbar entry, because it
would otherwise switch out to a new toolbar entry when we changed the window
styles. However since our new styles match, we no longer need to do this. It
was also causing the maximized window to paint over the Windows taskbar.
Differential Revision: https://phabricator.services.mozilla.com/D93534
For a better user experience during slow startups (and to match the design
by Markus Jaritz), we want to animate the placeholder elements (the grey
rectangles which hold the place of text and icons) with a light moving
gradient.
A question may arise regarding whether the performance cost of running this
animation is worth the improved user experience. My claim is yes, hinging
mostly on the observation that the performance cost is small.
On my machine, one frame of the animation takes at most 0.15ms, and runs
every 16.15ms. This means we eat less than 1% CPU time on one core of the
system. It should also be noted that this animation runs primarily during
the window wherein we are prefetching dlls, AKA while we are blocked on IO
and the CPU is more likely to be idle.
On slower systems, one frame may take longer - however, on slower systems
we should also be blocked *more* by IO, making the trade favorable.
For further anecdotal evidence of this being okay, when I run this on slow
reference hardware shortly after OS startup, I do not see any dropped frames,
indicating that this has very little trouble completing, and is thus quite
cheap.
Regarding testing, I will invoke the same logic as for the rest of the
skeleton UI patches - it would involve quite a bit of work to test this in
automation given our existing frameworks. It may be worth it at some point,
but certainly not while this is a feature that we are just experimenting
with and which is not enabled by default.
Differential Revision: https://phabricator.services.mozilla.com/D91453
Poison was setup at the start of xpcom init when that was assumed to be early enough.
Since then, Poison was added to Maybe, and Maybe has been used everywhere, including in
our channel implementation. As a result, poison was being used before it was initialized.
This basically meant our poison pointers were being replaced with null instead, which
dances into some more UB than accessing a page we have actually allocated. Also, tsan
noticed that accesses to the value were racing with the initializer actually being
called!
A (dynamic) static initializer forces the poison initialization as we can reasonably
hope without getting CallOnce or singleton patterns involved.
Other changes:
* Cleaned up the outdated documentation for mozWritePoison (the alignment
restriction was removed in Bug 1414901)
* Removed the poison supression from TSan
Differential Revision: https://phabricator.services.mozilla.com/D94251
If the urlbar is zoomed (defined by attribute breakout-extend), then we
scale it down to the unfocused coordinate values.
Differential Revision: https://phabricator.services.mozilla.com/D92303
The latest launcher process showed one of the top failures was `WriteProcessMemory` in
`CopyStubToChildProcess` failed with `ERROR_INVALID_ADDRESS` or `ERROR_NOACCESS`, that
is to store a trampoline address to the global variable of firefox.exe failed. Its root
cause should be the same as bug 1662560, the executable was loaded onto a different
address from the browser process.
The fix is to to expand the usage of `CrossExecTransferManager` to `FuncHookCrossProcess`
and `Kernel32ExportsSolver`.
Depends on D94652
Differential Revision: https://phabricator.services.mozilla.com/D94653
This patch introduces a class `CrossExecTransferManager` to manage the data
transfer from the current process to a remote process via `WriteProcessMemory`.
The class also encapsulates a logic to bridge the gap between two executable's
imagebase.
Differential Revision: https://phabricator.services.mozilla.com/D94652
This commit uses the new Markers 2.0 API for FileIO Markers. I had to
create another option for the MarkerStack class in order to conditionally
capture a backtrace inside of the Macro. Otherwise the macro invocation
failed.
Differential Revision: https://phabricator.services.mozilla.com/D93848
The new Markers 2.0 code had missed one detail:
Backtraces in markers were serialized as just the `ProfileChunkedBuffer`, which doesn't expose the original thread id like `ProfilerBacktrace` did.
Then when outputting the profile, the marker code would use the marker's thread id (where the marker should be displayed in the frontend, which *could* be different from where the backtrace came) to deserialize and stream the attached marker, and a special check in the streaming code meant that the mismatched id would ignore the stored sample, and the displayed marker would show no stack.
With this patch, when streaming stacks from markers, the given thread id is 0 (an impossible thread id), which indicates that whatever sample is present should be streamed.
`ProfilerBacktrace` doesn't need to store the thread id anymore.
This solves the above problem.
As a bonus, the streaming code now reports the original thread of the sample(s) it found. This could be used in the future, to better show in the frontend that some stacks may come from other threads.
Differential Revision: https://phabricator.services.mozilla.com/D94264
This commit uses the new Markers 2.0 API for FileIO Markers. I had to
create another option for the MarkerStack class in order to conditionally
capture a backtrace inside of the Macro. Otherwise the macro invocation
failed.
Differential Revision: https://phabricator.services.mozilla.com/D93848
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
The latest launcher process ping showed one of the reasons why we failed to
detour `NtMapViewOfSection` is that `MMPolicyBase::FindRegion` failed to find
a free region. Inspecting the function carefully, there were three problems.
Firstly, `FindRegion` did not fully scan the given range. To randomize
the address of a free region we use, we start scanning from a random address
within the given range. The problem is we scan only addresses bigger than
that random address, without scanning smaller addresses. Probably this is
the reason why `FindRegion` fails.
Secondly, `FindRegion` may return an address not aligned with the allocation
granularity because `VirtualQueryEx` returns such an address. If that happens,
the subsequent mapping API fails with the alignment error.
Lastly, when we randomize an address to start scanning from, we divide a random
number by `maxOffset`, but with that, we never start scanning from the last
region. It does not affect the product's behavior, but to have fair randomization,
a divisor should be `maxOffset + 1`.
This patch fixes all of these three problems along with a new test program.
Differential Revision: https://phabricator.services.mozilla.com/D94110
Everything related to Base Profiler legacy markers can now be removed, only the new API from BaseProfilerMarkers.h should now be used.
Depends on D93737
Differential Revision: https://phabricator.services.mozilla.com/D93738
TextMarkerPayload is the only legacy marker type that is still used in the Base Profiler. We're converting it to the new BASE_PROFILER_MARKER_TEXT.
Depends on D93736
Differential Revision: https://phabricator.services.mozilla.com/D93737
Since the main thread is almost always being profiled, this makes it easy to direct important markers there (e.g., FileIO markers from unregistered threads).
Tech note: The `#include "BaseProfiler.h"` line was moved to BaseProfilerMarkersPrerequesites.h so that `MarkerThreadId::MainThread()` there can access `profiler_main_thread_id()`.
Depends on D93735
Differential Revision: https://phabricator.services.mozilla.com/D93736
These functions will be useful to get the main thread id, or check if we're in it, in some public code (e.g., markers).
Differential Revision: https://phabricator.services.mozilla.com/D93735
These unit tests were going through chunks very quickly in tight loops, which on some platforms could cause successive chunks to be marked "done" with the same timestamps.
Distinct timestamps are needed to uniquely identify chunks (in each process), and related assertions caught that. In real Firefox code, chunks are much bigger (around 1MB), and fill at a slower rate, so timestamps are always different as expected there.
To fix this artificial issue in our unit tests, delays are introduced before each `MarkDone()`, to ensure that they'll get a different timestamp every time.
Differential Revision: https://phabricator.services.mozilla.com/D93479
Along with the deserializer, we now also store other marker functions (that output the marker type name and schema).
These will be used in the following patch to output the combined schema list for all used marker types.
Differential Revision: https://phabricator.services.mozilla.com/D90659
Add `static mozilla::MarkerSchemaWriter MarkerTypeDisplay()` for each existing marker type.
Because all markers of a given type now must have the same payload and display schema, the DOM event marker has changed from type=tracing with category=DOMEvent, to its own type=DOMEvent, which is now accepted on the front-end.
Based on c9692715f2/src/profile-logic/marker-schema.js
Differential Revision: https://phabricator.services.mozilla.com/D90658
This class collects all the information necessary to stream the JSON schema that informs the front-end how to display a type of markers.
It will be created and populated in `MarkerTypeDisplay()`, functions in each marker type definition, with Add/Set functions (see following patch).
Based on c9692715f2/src/profile-logic/marker-schema.js
Differential Revision: https://phabricator.services.mozilla.com/D90657
`AutoArraySchemaWriter::FreeFormElement()` is never used, we can remove it.
When constructed, `AutoArraySchemaWriter` was optionally taking a `UniqueStrings` reference, which was stored as a pointer.
Then `AutoArraySchemaWriter::StringElement()` would do a dangerous `MOZ_RELEASE_ASSERT(mStrings);`.
The class is now split in two:
- `AutoArraySchemaWriter`, which does not deal with strings at all, so we don't need a pointer to `UniqueStrings`.
- `AutoArraySchemaWithStringsWriter` that can also deal with strings, in which we store a (non-null) `UniqueStrings` reference.
This is both:
- Safer, because it's not possible to instantiate the non-string writer and try to write a string.
- More efficient, because we don't need to pass&store a `UniqueStrings` reference/pointer when we don't deal with strings.
Differential Revision: https://phabricator.services.mozilla.com/D93191
This is the third attempt to investigate the launcher failure of our detour.
The previous commits d8315e4ed18d and 1b81ea85c43d added the assembly bytes
of a detour target and a special error code `DetourResultCode` to the launcher
failure ping.
In the latest telemetry data, however, the most common value of `hresult`
is still `ERROR_UNIDENTIFIED_ERROR`, meaning the previous commit missed to
set an error code in the common fallible codepath we wanted to know.
Besides `ERROR_UNIDENTIFIED_ERROR`, we're seeing `DETOUR_PATCHER_DO_RESERVE_ERROR`
in the telemetry, but having that code is not enough to pinpoint a falling
operation.
For further investigation, this patch adds ten more values to `DetourResultCode`.
`FUNCHOOKCROSSPROCESS_COPYSTUB_ERROR` is the last codepath we forgot to cover
in the previous commit. The values of `MMPOLICY_RESERVE_*` are to investigate
`DETOUR_PATCHER_DO_RESERVE_ERROR` in the MMPolicy level. In both cases, we add
the last Windows error code to `DetourError::mOrigBytes`.
Differential Revision: https://phabricator.services.mozilla.com/D92974
We blocked the older versions of database.dll as Bug 1566109 in 2019,
but the same crash is still happening with the newer versions.
We decided to block all versions because crashing in the middle of printing
or file uploading is not acceptable.
Differential Revision: https://phabricator.services.mozilla.com/D92988
It is possible to add options to a NoPayload marker, but stack and inner window ids would be lost because they are normally stored in the payload 'data' JSON object, which doesn't exist for NoPayload markers.
So in this case, we automatically change the marker to a new (internal) "NoPayloadUserData" type, which has a payload in which we can store options.
This is temporary, until bug 1646714 moves these options into the top-level marker JSON object.
Differential Revision: https://phabricator.services.mozilla.com/D93059
This is the third attempt to investigate the launcher failure of our detour.
The previous commits d8315e4ed18d and 1b81ea85c43d added the assembly bytes
of a detour target and a special error code `DetourResultCode` to the launcher
failure ping.
In the latest telemetry data, however, the most common value of `hresult`
is still `ERROR_UNIDENTIFIED_ERROR`, meaning the previous commit missed to
set an error code in the common fallible codepath we wanted to know.
Besides `ERROR_UNIDENTIFIED_ERROR`, we're seeing `DETOUR_PATCHER_DO_RESERVE_ERROR`
in the telemetry, but having that code is not enough to pinpoint a falling
operation.
For further investigation, this patch adds ten more values to `DetourResultCode`.
`FUNCHOOKCROSSPROCESS_COPYSTUB_ERROR` is the last codepath we forgot to cover
in the previous commit. The values of `MMPOLICY_RESERVE_*` are to investigate
`DETOUR_PATCHER_DO_RESERVE_ERROR` in the MMPolicy level. In both cases, we add
the last Windows error code to `DetourError::mOrigBytes`.
Differential Revision: https://phabricator.services.mozilla.com/D92974
The latest Windows Insider Preview (version 20226.1000) changes the machine code for BaseThreadInitThunk to have a preamble like the following:
00007FFDBF244C40 48 83 EC 28 sub rsp,28h
00007FFDBF244C44 85 C9 test ecx,ecx
00007FFDBF244C46 75 25 jne 00007FFDBF244C6D
00007FFDBF244C48 49 BA 70 A2 DC 12 6A 97 99 B0 mov r10,0B099976A12DCA270h
This patch adds "MOV r64, imm64" capability to the DLL interceptor so that we can hook this.
Differential Revision: https://phabricator.services.mozilla.com/D92146
This makes it clearer where marker-type-specific payload arguments start, just after the marker type object.
Also improved the main API documentation.
Differential Revision: https://phabricator.services.mozilla.com/D91681
The `category.WithOptions(...)` syntax was a bit strange and difficult to explain.
Now the category and options are separate parameters. Default options can be specified with `MarkerOptions{}` or just `{}`.
As a special case, defaulted-NoPayload functions don't need `<>`, and defaulted-NoPayload functions and macros don't even need `{}` for default options, e.g.:
`profiler_add_marker("name", OTHER); PROFILER_MARKER_UNTYPED("name", OTHER);`
Differential Revision: https://phabricator.services.mozilla.com/D91680
This makes it clearer where marker-type-specific payload arguments start, just after the marker type object.
Also improved the main API documentation.
Differential Revision: https://phabricator.services.mozilla.com/D91681
The `category.WithOptions(...)` syntax was a bit strange and difficult to explain.
Now the category and options are separate parameters. Default options can be specified with `MarkerOptions{}` or just `{}`.
As a special case, defaulted-NoPayload functions don't need `<>`, and defaulted-NoPayload functions and macros don't even need `{}` for default options, e.g.:
`profiler_add_marker("name", OTHER); PROFILER_MARKER_UNTYPED("name", OTHER);`
Differential Revision: https://phabricator.services.mozilla.com/D91680
The previous commit d8315e4ed18d introduced a new telemetry field
in the launcher process ping to collect the assembly pattern of
a target function on detour failure, but most of the crash instances
do not have a value in the field. This means the failure happens
before or after `CreateTrampoline`.
To narrow down the root cause, this patch puts a fine-grained error value
in the "hresult" field instead of the hardcoded ERROR_UNIDENTIFIED_ERROR.
This patch also adds `IsPageAccessible` check before fetching data from
a different process because fetching data from an invalid address hits
`MOZ_RELEASE_ASSERT` in `EnsureLimit`, resulting in crash without sending
the launcher process failure.
Differential Revision: https://phabricator.services.mozilla.com/D91881
The last Avast Antivirus's hook function contains `CALL [disp32]` instruction.
Our detour needs to be able to handle that pattern.
Differential Revision: https://phabricator.services.mozilla.com/D91155
This patch optimizes our detour's code handling Opcode 0xFF, expanding
its coverage to INC and DEC reg64 as well as PUSH and CALL.
Testcases for these scenarios are of course included.
Differential Revision: https://phabricator.services.mozilla.com/D91154
In the initial patches for bug 1656526, mhowell noticed that for startups which
take a very long time, if the user interacts with the skeleton UI window, the OS
will flag us as not responsive, which could be a poorer user experience than
seeing nothing. Since our UI is designed to look non-interactive anyway, we
assume that a better experience would be to simply squash the not responsive
response from the OS by trivially processing native events. It's not perfect in,
say, the event that startup is hung for some reason, but it's arguably preferable
to our old model of startup being hung, which was just nothing being displayed at
all.
Differential Revision: https://phabricator.services.mozilla.com/D91005
This saves 1 byte when serializing each marker (and removes all the code that was related to the 2nd byte).
Also it will be easier to use it in legacy code that only knows about the category pair.
Added unit tests for the whole of MarkerCategory.
Differential Revision: https://phabricator.services.mozilla.com/D91110
This patch uses the prefs relevant to the pre-xul skeleton UI to control the
registry value. There are three prefs:
- A new pref intended to be the global toggle once all themes can be handled.
- browser.startup.blankWindow - we currently depend on the code in BrowserGlue.jsm,
gated on this pref, which immediately specifies the width and height of the
window and ultimately causes XUL to consume and take ownership of our pre-xul
window. Without this, we resize to fit a small content area, and then resize
again back to the correct size.
- extensions.activeThemeID - Given that we have hardcoded layout and colors, we
need to ensure that we're only presenting the skeleton UI for the default theme.
We're hoping to not need to gate on any prefs other than the new pref that we're
adding, but this allows us to provide a simple direction for users who may want
to dogfood our changes in the mean time: use the default theme, ensure the
browser.startup.blankWindow pref is set, and turn our new pref on.
Differential Revision: https://phabricator.services.mozilla.com/D90884
This patch bypasses `-Wunused-lambda-capture` warnings by using `Unused` as
Bug 1375386 did. Having two definitions with `#ifdef` confuses clang-format.
Using `Unused` seems like the easiest approach.
Differential Revision: https://phabricator.services.mozilla.com/D90610
Many instances of the launcher failure ping indicate hooking NtMapViewOfSection
or LdrLoadDll failed. This is most likely caused by a third-party application
applying a hook onto the same target earlier than we do.
This patch is to add a new field "detour_orig_bytes" in the laucnher failure ping
to collect the first sixteen bytes of a detour target function. With this,
we can know whether those detour failures were caused by a third-party hook or not,
and if yes, what was the actual binary pattern.
Differential Revision: https://phabricator.services.mozilla.com/D89836
This is currently failing the tier 2 MinGW-Clang builds, complaining
that DPI_AWARENESS_CONTEXT is not defined. Accordingly I pulled out the
logic from WinUtils.h which ensures it is defined into a shared header.
Differential Revision: https://phabricator.services.mozilla.com/D90467
Many instances of the launcher failure ping indicate hooking NtMapViewOfSection
or LdrLoadDll failed. This is most likely caused by a third-party application
applying a hook onto the same target earlier than we do.
This patch is to add a new field "detour_orig_bytes" in the laucnher failure ping
to collect the first sixteen bytes of a detour target function. With this,
we can know whether those detour failures were caused by a third-party hook or not,
and if yes, what was the actual binary pattern.
Differential Revision: https://phabricator.services.mozilla.com/D89836
The earlier fix ea452bb92e6a proved the executable's imagebase in a child
process is not always the same as the local imagebase. This patch applies
the new approach to retieve the imagebase from a handle to all channels.
Interestingly, we observed the launcher failures at `VirtualProtectEx` only
when launching a sandboxed process, not when launching the browser process.
In the long term, we may need to take care of all `WriteProcessMemory` calls
for a child process for greater safety, but given that observation, this
patch only updates `RestoreImportDirectory` and `InitializeDllBlocklistOOP`.
Differential Revision: https://phabricator.services.mozilla.com/D90316
We need this because otherwise we load user32, which fails the check in
WindowsDllBlocklist.cpp (line 649). It sounds like this check is non-
negotiable, so this is the only solution I can come up with. Obviously
please let me know if there is some reason we cannot do this, but it
seems to function fine.
Depends on D89670
Differential Revision: https://phabricator.services.mozilla.com/D90271
Hopefully the comments in the actual code are enough to explain what is going
on here.
NOTE: this patch does not represent a finished skeleton UI. There are some
questions in comments within the code, and generally I'm seeking feedback on
whether the overall approach seems sane or not. Gijs, I'm including you for
feedback on whether you think this is maintainable by more frontend-oriented
folks, and Molly, I'm including you for feedback on whether the justification
for writing to a raw pixel buffer seems sound or not, and a general review of
the approach.
Differential Revision: https://phabricator.services.mozilla.com/D86447
See bug for justification. This patch aims to display a blank window prior to
loading/prefetching xul.dll. It also has a placeholder for drawing a
skeleton UI into that window. Note that this is disabled by default based on
a registry value, as there are still kinks to work out (for instance, what
happens if we aren't actually going to display a window, because, say, Firefox
is already running.) This just gives a basic implementation to dogfood, and
facilitates distributing work across multiple contributors.
Onto the details. The patch achieves its goal by creating a window and
assigning its handle to a static variable, which will be consumed inside
nsWindow::Create by the first toplevel window we want to make. nsWindow::Create
will take ownership of the window handle, restyle it to its own liking, and
then proceed as if everything is normal and it had created the window itself.
Differential Revision: https://phabricator.services.mozilla.com/D86263
In most situations, JSONWriter users already know string lengths (either directly, or through `nsCString` and friends), so we should keep this information through JSONWriter and not recompute it again.
This also allows using JSONWriter with sub-strings (e.g., from a bigger buffer), without having to create null-terminated strings.
Public JSONWriter functions have overloads that accept literal strings.
Differential Revision: https://phabricator.services.mozilla.com/D86192
Hopefully the comments in the actual code are enough to explain what is going
on here.
NOTE: this patch does not represent a finished skeleton UI. There are some
questions in comments within the code, and generally I'm seeking feedback on
whether the overall approach seems sane or not. Gijs, I'm including you for
feedback on whether you think this is maintainable by more frontend-oriented
folks, and Molly, I'm including you for feedback on whether the justification
for writing to a raw pixel buffer seems sound or not, and a general review of
the approach.
Differential Revision: https://phabricator.services.mozilla.com/D86447
See bug for justification. This patch aims to display a blank window prior to
loading/prefetching xul.dll. It also has a placeholder for drawing a
skeleton UI into that window. Note that this is disabled by default based on
a registry value, as there are still kinks to work out (for instance, what
happens if we aren't actually going to display a window, because, say, Firefox
is already running.) This just gives a basic implementation to dogfood, and
facilitates distributing work across multiple contributors.
Onto the details. The patch achieves its goal by creating a window and
assigning its handle to a static variable, which will be consumed inside
nsWindow::Create by the first toplevel window we want to make. nsWindow::Create
will take ownership of the window handle, restyle it to its own liking, and
then proceed as if everything is normal and it had created the window itself.
Differential Revision: https://phabricator.services.mozilla.com/D86263
`MarkerOptions::Set()` returns the same reference type as the object it's invoked on, i.e.: & -> &, and && -> &&.
`MarkerOptions::NAME()` now always returns a reference to a `const` member, so it's clear it cannot be modified (even if the object at hand is not `const`).
`MarkerOptions::NAMERef()` must be used when non-`const` access is needed.
Differential Revision: https://phabricator.services.mozilla.com/D89715
The name `AUTO_PROFILER_MARKER_TEXT` is more consistent with the equivalent non-`AUTO` macro, and similarly arguments have been re-ordered to be the same, i.e.: Name, category&options, text.
The different macros with different argument sets can now be collapsed into one macro, and the optional arguments (timing, inner window id, backtrace) can easily be added to the `MarkerOptions` where needed.
As a bonus, a specific start time can optionally be provided at construction time.
Differential Revision: https://phabricator.services.mozilla.com/D89588
Mostly mechanical changes, with some work needed to convert the different payloads (with optional timestamps, inner window id, and/or backtrace) to the equivalent MarkerOptions.
Differential Revision: https://phabricator.services.mozilla.com/D89587
Mostly mechanical change, with some extra work where non-literal names are provided.
Also, when this is the only profiler call in a file, `#include "GeckoProfiler.h"` can be changed to `#include "mozilla/ProfilerMarkers.h"`.
Differential Revision: https://phabricator.services.mozilla.com/D89415
Our data indicates when the browser process populates a newly-created child process,
`VirtualProtectEx` may fail with `ERROR_INVALID_ADDRESS` for some unknown reason.
One possible cause is the parameter `aRemoteExeImage` of `RestoreImportDirectory`
was wrong i.e. pointing to an invalid address. We simply pass the local process's
imagebase as `aRemoteExeImage` based on the assumption that the same executable is
mapped onto the same address in a different process, but it may not be guaranteed.
To deal with that potential case, we could retrieve a correct imagebase from the handle
of a remote process as we do for the plugin process. Since we're not so sure about
the root cause or the effectiveness of this fix, we run it only when the first
attempt to `VirtualProtectEx` failed in Nightly. Once it's confirmed, we promote this
to a permanent fix.
Differential Revision: https://phabricator.services.mozilla.com/D89502
`ProfileChunkedBuffer` needed to be fully defined, because its destructor is needed to define `UniquePtr<ProfileChunkedBuffer>`.
It can just be empty, because it won't actually be used anyway.
Added non-`MOZ_GECKO_PROFILER` tests around this.
Differential Revision: https://phabricator.services.mozilla.com/D89351
This patch ports existing ProfilerMarkerPayload types to draft struct definitions that may be used with the new markers API.
This is just a starting point, they may be changed later on as needed, see meta-bug 1661394.
Differential Revision: https://phabricator.services.mozilla.com/D87259
This is the main public marker API:
- `AddMarkerToBuffer` can be used to store a marker to any buffer. This could be useful to code that wants to store markers outside of the default profiler buffers.
- `baseprofiler::AddMarker`/`profiler_add_marker` store a marker in the appropriate profiler buffer.
- BASE_PROFILER_MARKER and PROFILER_MARKER do the same, but are also defined (and empty) when MOZ_GECKO_PROFILER is not #defined.
All these take a name, marker options, a marker type, and the type's expected arguments if any (as expected by the `StreamJSONMarkerData` function).
Extra helpers for the most common types:
- BASE_PROFILER_MARKER_UNTYPED and PROFILER_MARKER_UNTYPED store a marker with no data payload.
- BASE_PROFILER_MARKER_TEXT and PROFILER_MARKER_TEXT store a text marker. `baseprofiler::markers::Text` is an example of how new marker types can be defined.
Differential Revision: https://phabricator.services.mozilla.com/D87257
Main marker serialization function, which accepts the same arguments (with implicit conversions) as the marker type's `StreamJSONMarkerData(JSONWriter&, ...)` function, and stores them in a ProfileChunkedBuffer after the marker name and options.
Differential Revision: https://phabricator.services.mozilla.com/D87255
`DeserializeAfterKindAndStream()` is the main function that extracts all the marker data (past the already-read entry kind), and streams it to JSON using the user-provided `Stream(JSONWriter&, ...)` function in the appropriate marker type definition.
It currently requires two external functions to stream the name and the optional backtrace, because these are different between the two profilers. This may change in the future.
(Deserialization is implemented before serialization, because the `Deserialize()` function is needed during serialization to get a marker type tag.)
Differential Revision: https://phabricator.services.mozilla.com/D87254
`NoPayload` will be mostly used internally when adding markers without payload data.
It has an empty specialization of the MarkerTypeHelper (mainly to catch misuses), and the add-marker code will need to have different compile-time paths to handle it.
Differential Revision: https://phabricator.services.mozilla.com/D87252
Marker types will be defined with a simple struct that contains (amongst other things) one `StreamJSONMarkerData(JSONWriter&, ...)` function.
This patch introduces a type-traits-like helper to examine that function. This will be used to check the arguments given to the upcoming add-marker function, to serialize and deserialize them.
Differential Revision: https://phabricator.services.mozilla.com/D87251
This is similar to the existing deserializer table in ProfilerMarkerPayload.*:
Each type of marker (except the `NoPayload` one) has its corresponding deserializer in a table, and the index is used as a tag in the serialization buffer.
Differential Revision: https://phabricator.services.mozilla.com/D87250
A `MarkerOptions` must be constructed from a MarkerCategory, e.g.: `mozilla::baseprofiler::category::OTHER`. This will be required for all markers.
`MarkerOptions` also contains defaulted `MarkerThreadId`, `MarkerTiming`, `MarkerStack`, and `MarkerInnerWindowId`. They may be set by calling `WithOptions()` on the MarkerCategory, e.g.:
`PROFILER_MARKER<...>("name", OTHER.WithOptions(MarkerThreadId(otherThread), MarkerStack::Capture()), ...);`
Differential Revision: https://phabricator.services.mozilla.com/D87249
This marker option allows three cases:
- By default, no stacks are captured.
- The caller can request a stack capture, and the add-marker code will take care of it in the most efficient way.
- The caller can still provide an existing backtrace, for cases where a marker reports something that happened elsewhere.
Differential Revision: https://phabricator.services.mozilla.com/D87247
`profiler_capture_backtrace(ProfileChunkedBuffer&)` renamed to `profiler_capture_backtrace_into(ProfileChunkedBuffer&)` (notice "_into"), which is clearer.
New function `profiler_capture_backtrace()` creates a buffer, uses `profiler_capture_backtrace_into()`, and returns a `UniquePtr<ProfileChunkedBuffer>`, which can later be given to `MarkerStack::TakeBacktrace`.
`profiler_get_backtrace()` (returning a `UniqueProfilerBacktrace`) now uses `profiler_capture_backtrace()`.
This patch reduces most duplicate code between these functions.
Differential Revision: https://phabricator.services.mozilla.com/D88280
`IsEmpty()` makes it easier for users to determine if there is anything stored in the `ProfileChunkedBuffer`.
And `ProfileChunkedBuffer` can now be serialized from a raw pointer, which will be useful when adding markers with an optional stack.
Deserialization should be done to a `UniquePtr<ProfileChunkedBuffer>`, which is already implemented.
Differential Revision: https://phabricator.services.mozilla.com/D87246
This moves the existing MarkerTiming class introduced in bug 1640969 to the BaseProfilerMarkersPrerequesites.h header, and can be used as a marker option.
Some minor clarifying changes:
- `Instant()` is split into two functions: `InstantNow()` and `InstantAt(TimeStamp)`.
- `Interval(TimeStamp, TimeStamp)` must be given both start and end, otherwise `IntervalUntilNowFrom(TimeStamp)` takes the start only and ends "now".
Also the default construction is now reserved for internal marker usage, the private member function `IsUnspecified()` will be used by the add-marker code will replace it with `InstantNow()`.
The serialization contains the phase, and only one or two timestamps as needed, to save space for non-interval timings.
Differential Revision: https://phabricator.services.mozilla.com/D87245
This marker option captures a given thread id.
If left unspecified (by default construction) during the add-marker call, the current thread id will be used then.
Differential Revision: https://phabricator.services.mozilla.com/D87244
The main, and only compulsory, marker option is the `MarkerCategory`, which captures the "category pair", as well as the parent category.
Differential Revision: https://phabricator.services.mozilla.com/D88154
The upcoming profiler-specific add-marker function will need to know which `ProfileChunkedBuffer` to serialize to, `profiler_get_core_buffer()` give access to the profiler's buffer, and `CachedCoreBuffer()` keeps it stored in a function-static object.
Differential Revision: https://phabricator.services.mozilla.com/D87256
These string views are similar to `std::string_view`, but they are optimized to be serialized in the profiler buffer, and later deserialized and streamed to JSON.
They accept literal strings, and keep them as unowned raw pointers and sizes.
They also accept any substring reference, assuming that they will only be used as parameters during function calls, and therefore the dependent string will live during that call where these `StringView`'s are used.
Internally, they also allow optional string ownership, which is only used during deserialization and streaming.
This is hidden, so that users are not tempted to use potentially expensive string allocations during profiling; it's only used *after* profiling, so it's less of an impact to allocate strings then. (But it could still be optimized later on, as part of bug 1577656.)
Differential Revision: https://phabricator.services.mozilla.com/D87242
This patch introduces all new files that contain the new markers C++ API and implementation.
They are mostly empty at this time, only including each other as eventually needed, and with `#ifdef MOZ_GECKO_PROFILER` guards.
Rough inclusion diagram: (header <-- includer)
BaseProfilerMarkerPrerequesites.h <-- ProfilerMarkerPrerequesites.h (Useful types: Input string view, marker options)
^ ^
BaseProfilerMarkerDetail.h <-- ProfilerMarkerDetail.h (Implementation details)
^ ^
BaseProfilerMarkers.h <-- ProfilerMarkers.h (Main API)
^ ^--------- ^ ^---------
BaseProfilerMarkerTypes.h | <-- ProfilerMarkerTypes.h | (Common marker types)
^ BaseProfiler.h <-- GeckoProfiler.h (Existing main profiler headers)
Differential Revision: https://phabricator.services.mozilla.com/D87241
This patch ports existing ProfilerMarkerPayload types to draft struct definitions that may be used with the new markers API.
This is just a starting point, they may be changed later on as needed, see meta-bug 1661394.
Differential Revision: https://phabricator.services.mozilla.com/D87259
This is the main public marker API:
- `AddMarkerToBuffer` can be used to store a marker to any buffer. This could be useful to code that wants to store markers outside of the default profiler buffers.
- `baseprofiler::AddMarker`/`profiler_add_marker` store a marker in the appropriate profiler buffer.
- BASE_PROFILER_MARKER and PROFILER_MARKER do the same, but are also defined (and empty) when MOZ_GECKO_PROFILER is not #defined.
All these take a name, marker options, a marker type, and the type's expected arguments if any (as expected by the `StreamJSONMarkerData` function).
Extra helpers for the most common types:
- BASE_PROFILER_MARKER_UNTYPED and PROFILER_MARKER_UNTYPED store a marker with no data payload.
- BASE_PROFILER_MARKER_TEXT and PROFILER_MARKER_TEXT store a text marker. `baseprofiler::markers::Text` is an example of how new marker types can be defined.
Differential Revision: https://phabricator.services.mozilla.com/D87257
Main marker serialization function, which accepts the same arguments (with implicit conversions) as the marker type's `StreamJSONMarkerData(JSONWriter&, ...)` function, and stores them in a ProfileChunkedBuffer after the marker name and options.
Differential Revision: https://phabricator.services.mozilla.com/D87255
`DeserializeAfterKindAndStream()` is the main function that extracts all the marker data (past the already-read entry kind), and streams it to JSON using the user-provided `Stream(JSONWriter&, ...)` function in the appropriate marker type definition.
It currently requires two external functions to stream the name and the optional backtrace, because these are different between the two profilers. This may change in the future.
(Deserialization is implemented before serialization, because the `Deserialize()` function is needed during serialization to get a marker type tag.)
Differential Revision: https://phabricator.services.mozilla.com/D87254
`NoPayload` will be mostly used internally when adding markers without payload data.
It has an empty specialization of the MarkerTypeHelper (mainly to catch misuses), and the add-marker code will need to have different compile-time paths to handle it.
Differential Revision: https://phabricator.services.mozilla.com/D87252
Marker types will be defined with a simple struct that contains (amongst other things) one `StreamJSONMarkerData(JSONWriter&, ...)` function.
This patch introduces a type-traits-like helper to examine that function. This will be used to check the arguments given to the upcoming add-marker function, to serialize and deserialize them.
Differential Revision: https://phabricator.services.mozilla.com/D87251
This is similar to the existing deserializer table in ProfilerMarkerPayload.*:
Each type of marker (except the `NoPayload` one) has its corresponding deserializer in a table, and the index is used as a tag in the serialization buffer.
Differential Revision: https://phabricator.services.mozilla.com/D87250
A `MarkerOptions` must be constructed from a MarkerCategory, e.g.: `mozilla::baseprofiler::category::OTHER`. This will be required for all markers.
`MarkerOptions` also contains defaulted `MarkerThreadId`, `MarkerTiming`, `MarkerStack`, and `MarkerInnerWindowId`. They may be set by calling `WithOptions()` on the MarkerCategory, e.g.:
`PROFILER_MARKER<...>("name", OTHER.WithOptions(MarkerThreadId(otherThread), MarkerStack::Capture()), ...);`
Differential Revision: https://phabricator.services.mozilla.com/D87249
This marker option allows three cases:
- By default, no stacks are captured.
- The caller can request a stack capture, and the add-marker code will take care of it in the most efficient way.
- The caller can still provide an existing backtrace, for cases where a marker reports something that happened elsewhere.
Differential Revision: https://phabricator.services.mozilla.com/D87247
`profiler_capture_backtrace(ProfileChunkedBuffer&)` renamed to `profiler_capture_backtrace_into(ProfileChunkedBuffer&)` (notice "_into"), which is clearer.
New function `profiler_capture_backtrace()` creates a buffer, uses `profiler_capture_backtrace_into()`, and returns a `UniquePtr<ProfileChunkedBuffer>`, which can later be given to `MarkerStack::TakeBacktrace`.
`profiler_get_backtrace()` (returning a `UniqueProfilerBacktrace`) now uses `profiler_capture_backtrace()`.
This patch reduces most duplicate code between these functions.
Differential Revision: https://phabricator.services.mozilla.com/D88280
`IsEmpty()` makes it easier for users to determine if there is anything stored in the `ProfileChunkedBuffer`.
And `ProfileChunkedBuffer` can now be serialized from a raw pointer, which will be useful when adding markers with an optional stack.
Deserialization should be done to a `UniquePtr<ProfileChunkedBuffer>`, which is already implemented.
Differential Revision: https://phabricator.services.mozilla.com/D87246
This moves the existing MarkerTiming class introduced in bug 1640969 to the BaseProfilerMarkersPrerequesites.h header, and can be used as a marker option.
Some minor clarifying changes:
- `Instant()` is split into two functions: `InstantNow()` and `InstantAt(TimeStamp)`.
- `Interval(TimeStamp, TimeStamp)` must be given both start and end, otherwise `IntervalUntilNowFrom(TimeStamp)` takes the start only and ends "now".
Also the default construction is now reserved for internal marker usage, the private member function `IsUnspecified()` will be used by the add-marker code will replace it with `InstantNow()`.
The serialization contains the phase, and only one or two timestamps as needed, to save space for non-interval timings.
Differential Revision: https://phabricator.services.mozilla.com/D87245
This marker option captures a given thread id.
If left unspecified (by default construction) during the add-marker call, the current thread id will be used then.
Differential Revision: https://phabricator.services.mozilla.com/D87244
The main, and only compulsory, marker option is the `MarkerCategory`, which captures the "category pair", as well as the parent category.
Differential Revision: https://phabricator.services.mozilla.com/D88154
The upcoming profiler-specific add-marker function will need to know which `ProfileChunkedBuffer` to serialize to, `profiler_get_core_buffer()` give access to the profiler's buffer, and `CachedCoreBuffer()` keeps it stored in a function-static object.
Differential Revision: https://phabricator.services.mozilla.com/D87256
These string views are similar to `std::string_view`, but they are optimized to be serialized in the profiler buffer, and later deserialized and streamed to JSON.
They accept literal strings, and keep them as unowned raw pointers and sizes.
They also accept any substring reference, assuming that they will only be used as parameters during function calls, and therefore the dependent string will live during that call where these `StringView`'s are used.
Internally, they also allow optional string ownership, which is only used during deserialization and streaming.
This is hidden, so that users are not tempted to use potentially expensive string allocations during profiling; it's only used *after* profiling, so it's less of an impact to allocate strings then. (But it could still be optimized later on, as part of bug 1577656.)
Differential Revision: https://phabricator.services.mozilla.com/D87242
This patch introduces all new files that contain the new markers C++ API and implementation.
They are mostly empty at this time, only including each other as eventually needed, and with `#ifdef MOZ_GECKO_PROFILER` guards.
Rough inclusion diagram: (header <-- includer)
BaseProfilerMarkerPrerequesites.h <-- ProfilerMarkerPrerequesites.h (Useful types: Input string view, marker options)
^ ^
BaseProfilerMarkerDetail.h <-- ProfilerMarkerDetail.h (Implementation details)
^ ^
BaseProfilerMarkers.h <-- ProfilerMarkers.h (Main API)
^ ^--------- ^ ^---------
BaseProfilerMarkerTypes.h | <-- ProfilerMarkerTypes.h | (Common marker types)
^ BaseProfiler.h <-- GeckoProfiler.h (Existing main profiler headers)
Differential Revision: https://phabricator.services.mozilla.com/D87241
This patch ports existing ProfilerMarkerPayload types to draft struct definitions that may be used with the new markers API.
This is just a starting point, they may be changed later on as needed, see meta-bug 1661394.
Differential Revision: https://phabricator.services.mozilla.com/D87259
This is the main public marker API:
- `AddMarkerToBuffer` can be used to store a marker to any buffer. This could be useful to code that wants to store markers outside of the default profiler buffers.
- `baseprofiler::AddMarker`/`profiler_add_marker` store a marker in the appropriate profiler buffer.
- BASE_PROFILER_MARKER and PROFILER_MARKER do the same, but are also defined (and empty) when MOZ_GECKO_PROFILER is not #defined.
All these take a name, marker options, a marker type, and the type's expected arguments if any (as expected by the `StreamJSONMarkerData` function).
Extra helpers for the most common types:
- BASE_PROFILER_MARKER_UNTYPED and PROFILER_MARKER_UNTYPED store a marker with no data payload.
- BASE_PROFILER_MARKER_TEXT and PROFILER_MARKER_TEXT store a text marker. `baseprofiler::markers::Text` is an example of how new marker types can be defined.
Differential Revision: https://phabricator.services.mozilla.com/D87257
Main marker serialization function, which accepts the same arguments (with implicit conversions) as the marker type's `StreamJSONMarkerData(JSONWriter&, ...)` function, and stores them in a ProfileChunkedBuffer after the marker name and options.
Differential Revision: https://phabricator.services.mozilla.com/D87255
`DeserializeAfterKindAndStream()` is the main function that extracts all the marker data (past the already-read entry kind), and streams it to JSON using the user-provided `Stream(JSONWriter&, ...)` function in the appropriate marker type definition.
It currently requires two external functions to stream the name and the optional backtrace, because these are different between the two profilers. This may change in the future.
(Deserialization is implemented before serialization, because the `Deserialize()` function is needed during serialization to get a marker type tag.)
Differential Revision: https://phabricator.services.mozilla.com/D87254
`NoPayload` will be mostly used internally when adding markers without payload data.
It has an empty specialization of the MarkerTypeHelper (mainly to catch misuses), and the add-marker code will need to have different compile-time paths to handle it.
Differential Revision: https://phabricator.services.mozilla.com/D87252
Marker types will be defined with a simple struct that contains (amongst other things) one `StreamJSONMarkerData(JSONWriter&, ...)` function.
This patch introduces a type-traits-like helper to examine that function. This will be used to check the arguments given to the upcoming add-marker function, to serialize and deserialize them.
Differential Revision: https://phabricator.services.mozilla.com/D87251
This is similar to the existing deserializer table in ProfilerMarkerPayload.*:
Each type of marker (except the `NoPayload` one) has its corresponding deserializer in a table, and the index is used as a tag in the serialization buffer.
Differential Revision: https://phabricator.services.mozilla.com/D87250
A `MarkerOptions` must be constructed from a MarkerCategory, e.g.: `mozilla::baseprofiler::category::OTHER`. This will be required for all markers.
`MarkerOptions` also contains defaulted `MarkerThreadId`, `MarkerTiming`, `MarkerStack`, and `MarkerInnerWindowId`. They may be set by calling `WithOptions()` on the MarkerCategory, e.g.:
`PROFILER_MARKER<...>("name", OTHER.WithOptions(MarkerThreadId(otherThread), MarkerStack::Capture()), ...);`
Differential Revision: https://phabricator.services.mozilla.com/D87249
This marker option allows three cases:
- By default, no stacks are captured.
- The caller can request a stack capture, and the add-marker code will take care of it in the most efficient way.
- The caller can still provide an existing backtrace, for cases where a marker reports something that happened elsewhere.
Differential Revision: https://phabricator.services.mozilla.com/D87247
`profiler_capture_backtrace(ProfileChunkedBuffer&)` renamed to `profiler_capture_backtrace_into(ProfileChunkedBuffer&)` (notice "_into"), which is clearer.
New function `profiler_capture_backtrace()` creates a buffer, uses `profiler_capture_backtrace_into()`, and returns a `UniquePtr<ProfileChunkedBuffer>`, which can later be given to `MarkerStack::TakeBacktrace`.
`profiler_get_backtrace()` (returning a `UniqueProfilerBacktrace`) now uses `profiler_capture_backtrace()`.
This patch reduces most duplicate code between these functions.
Differential Revision: https://phabricator.services.mozilla.com/D88280
`IsEmpty()` makes it easier for users to determine if there is anything stored in the `ProfileChunkedBuffer`.
And `ProfileChunkedBuffer` can now be serialized from a raw pointer, which will be useful when adding markers with an optional stack.
Deserialization should be done to a `UniquePtr<ProfileChunkedBuffer>`, which is already implemented.
Differential Revision: https://phabricator.services.mozilla.com/D87246
This moves the existing MarkerTiming class introduced in bug 1640969 to the BaseProfilerMarkersPrerequesites.h header, and can be used as a marker option.
Some minor clarifying changes:
- `Instant()` is split into two functions: `InstantNow()` and `InstantAt(TimeStamp)`.
- `Interval(TimeStamp, TimeStamp)` must be given both start and end, otherwise `IntervalUntilNowFrom(TimeStamp)` takes the start only and ends "now".
Also the default construction is now reserved for internal marker usage, the private member function `IsUnspecified()` will be used by the add-marker code will replace it with `InstantNow()`.
The serialization contains the phase, and only one or two timestamps as needed, to save space for non-interval timings.
Differential Revision: https://phabricator.services.mozilla.com/D87245
This marker option captures a given thread id.
If left unspecified (by default construction) during the add-marker call, the current thread id will be used then.
Differential Revision: https://phabricator.services.mozilla.com/D87244
The main, and only compulsory, marker option is the `MarkerCategory`, which captures the "category pair", as well as the parent category.
Differential Revision: https://phabricator.services.mozilla.com/D88154
The upcoming profiler-specific add-marker function will need to know which `ProfileChunkedBuffer` to serialize to, `profiler_get_core_buffer()` give access to the profiler's buffer, and `CachedCoreBuffer()` keeps it stored in a function-static object.
Differential Revision: https://phabricator.services.mozilla.com/D87256
These string views are similar to `std::string_view`, but they are optimized to be serialized in the profiler buffer, and later deserialized and streamed to JSON.
They accept literal strings, and keep them as unowned raw pointers and sizes.
They also accept any substring reference, assuming that they will only be used as parameters during function calls, and therefore the dependent string will live during that call where these `StringView`'s are used.
Internally, they also allow optional string ownership, which is only used during deserialization and streaming.
This is hidden, so that users are not tempted to use potentially expensive string allocations during profiling; it's only used *after* profiling, so it's less of an impact to allocate strings then. (But it could still be optimized later on, as part of bug 1577656.)
Differential Revision: https://phabricator.services.mozilla.com/D87242
This patch introduces all new files that contain the new markers C++ API and implementation.
They are mostly empty at this time, only including each other as eventually needed, and with `#ifdef MOZ_GECKO_PROFILER` guards.
Rough inclusion diagram: (header <-- includer)
BaseProfilerMarkerPrerequesites.h <-- ProfilerMarkerPrerequesites.h (Useful types: Input string view, marker options)
^ ^
BaseProfilerMarkerDetail.h <-- ProfilerMarkerDetail.h (Implementation details)
^ ^
BaseProfilerMarkers.h <-- ProfilerMarkers.h (Main API)
^ ^--------- ^ ^---------
BaseProfilerMarkerTypes.h | <-- ProfilerMarkerTypes.h | (Common marker types)
^ BaseProfiler.h <-- GeckoProfiler.h (Existing main profiler headers)
Differential Revision: https://phabricator.services.mozilla.com/D87241
This patch ports existing ProfilerMarkerPayload types to draft struct definitions that may be used with the new markers API.
This is just a starting point, they may be changed later on as needed, see meta-bug 1661394.
Differential Revision: https://phabricator.services.mozilla.com/D87259
This is the main public marker API:
- `AddMarkerToBuffer` can be used to store a marker to any buffer. This could be useful to code that wants to store markers outside of the default profiler buffers.
- `baseprofiler::AddMarker`/`profiler_add_marker` store a marker in the appropriate profiler buffer.
- BASE_PROFILER_MARKER and PROFILER_MARKER do the same, but are also defined (and empty) when MOZ_GECKO_PROFILER is not #defined.
All these take a name, marker options, a marker type, and the type's expected arguments if any (as expected by the `StreamJSONMarkerData` function).
Extra helpers for the most common types:
- BASE_PROFILER_MARKER_UNTYPED and PROFILER_MARKER_UNTYPED store a marker with no data payload.
- BASE_PROFILER_MARKER_TEXT and PROFILER_MARKER_TEXT store a text marker. `baseprofiler::markers::Text` is an example of how new marker types can be defined.
Differential Revision: https://phabricator.services.mozilla.com/D87257
Main marker serialization function, which accepts the same arguments (with implicit conversions) as the marker type's `StreamJSONMarkerData(JSONWriter&, ...)` function, and stores them in a ProfileChunkedBuffer after the marker name and options.
Differential Revision: https://phabricator.services.mozilla.com/D87255
`DeserializeAfterKindAndStream()` is the main function that extracts all the marker data (past the already-read entry kind), and streams it to JSON using the user-provided `Stream(JSONWriter&, ...)` function in the appropriate marker type definition.
It currently requires two external functions to stream the name and the optional backtrace, because these are different between the two profilers. This may change in the future.
(Deserialization is implemented before serialization, because the `Deserialize()` function is needed during serialization to get a marker type tag.)
Differential Revision: https://phabricator.services.mozilla.com/D87254
`NoPayload` will be mostly used internally when adding markers without payload data.
It has an empty specialization of the MarkerTypeHelper (mainly to catch misuses), and the add-marker code will need to have different compile-time paths to handle it.
Differential Revision: https://phabricator.services.mozilla.com/D87252
Marker types will be defined with a simple struct that contains (amongst other things) one `StreamJSONMarkerData(JSONWriter&, ...)` function.
This patch introduces a type-traits-like helper to examine that function. This will be used to check the arguments given to the upcoming add-marker function, to serialize and deserialize them.
Differential Revision: https://phabricator.services.mozilla.com/D87251
This is similar to the existing deserializer table in ProfilerMarkerPayload.*:
Each type of marker (except the `NoPayload` one) has its corresponding deserializer in a table, and the index is used as a tag in the serialization buffer.
Differential Revision: https://phabricator.services.mozilla.com/D87250
A `MarkerOptions` must be constructed from a MarkerCategory, e.g.: `mozilla::baseprofiler::category::OTHER`. This will be required for all markers.
`MarkerOptions` also contains defaulted `MarkerThreadId`, `MarkerTiming`, `MarkerStack`, and `MarkerInnerWindowId`. They may be set by calling `WithOptions()` on the MarkerCategory, e.g.:
`PROFILER_MARKER<...>("name", OTHER.WithOptions(MarkerThreadId(otherThread), MarkerStack::Capture()), ...);`
Differential Revision: https://phabricator.services.mozilla.com/D87249
This marker option allows three cases:
- By default, no stacks are captured.
- The caller can request a stack capture, and the add-marker code will take care of it in the most efficient way.
- The caller can still provide an existing backtrace, for cases where a marker reports something that happened elsewhere.
Differential Revision: https://phabricator.services.mozilla.com/D87247
`profiler_capture_backtrace(ProfileChunkedBuffer&)` renamed to `profiler_capture_backtrace_into(ProfileChunkedBuffer&)` (notice "_into"), which is clearer.
New function `profiler_capture_backtrace()` creates a buffer, uses `profiler_capture_backtrace_into()`, and returns a `UniquePtr<ProfileChunkedBuffer>`, which can later be given to `MarkerStack::TakeBacktrace`.
`profiler_get_backtrace()` (returning a `UniqueProfilerBacktrace`) now uses `profiler_capture_backtrace()`.
This patch reduces most duplicate code between these functions.
Differential Revision: https://phabricator.services.mozilla.com/D88280
`IsEmpty()` makes it easier for users to determine if there is anything stored in the `ProfileChunkedBuffer`.
And `ProfileChunkedBuffer` can now be serialized from a raw pointer, which will be useful when adding markers with an optional stack.
Deserialization should be done to a `UniquePtr<ProfileChunkedBuffer>`, which is already implemented.
Differential Revision: https://phabricator.services.mozilla.com/D87246
This moves the existing MarkerTiming class introduced in bug 1640969 to the BaseProfilerMarkersPrerequesites.h header, and can be used as a marker option.
Some minor clarifying changes:
- `Instant()` is split into two functions: `InstantNow()` and `InstantAt(TimeStamp)`.
- `Interval(TimeStamp, TimeStamp)` must be given both start and end, otherwise `IntervalUntilNowFrom(TimeStamp)` takes the start only and ends "now".
Also the default construction is now reserved for internal marker usage, the private member function `IsUnspecified()` will be used by the add-marker code will replace it with `InstantNow()`.
The serialization contains the phase, and only one or two timestamps as needed, to save space for non-interval timings.
Differential Revision: https://phabricator.services.mozilla.com/D87245
This marker option captures a given thread id.
If left unspecified (by default construction) during the add-marker call, the current thread id will be used then.
Differential Revision: https://phabricator.services.mozilla.com/D87244
The main, and only compulsory, marker option is the `MarkerCategory`, which captures the "category pair", as well as the parent category.
Differential Revision: https://phabricator.services.mozilla.com/D88154
The upcoming profiler-specific add-marker function will need to know which `ProfileChunkedBuffer` to serialize to, `profiler_get_core_buffer()` give access to the profiler's buffer, and `CachedCoreBuffer()` keeps it stored in a function-static object.
Differential Revision: https://phabricator.services.mozilla.com/D87256
These string views are similar to `std::string_view`, but they are optimized to be serialized in the profiler buffer, and later deserialized and streamed to JSON.
They accept literal strings, and keep them as unowned raw pointers and sizes.
They also accept any substring reference, assuming that they will only be used as parameters during function calls, and therefore the dependent string will live during that call where these `StringView`'s are used.
Internally, they also allow optional string ownership, which is only used during deserialization and streaming.
This is hidden, so that users are not tempted to use potentially expensive string allocations during profiling; it's only used *after* profiling, so it's less of an impact to allocate strings then. (But it could still be optimized later on, as part of bug 1577656.)
Differential Revision: https://phabricator.services.mozilla.com/D87242
This patch introduces all new files that contain the new markers C++ API and implementation.
They are mostly empty at this time, only including each other as eventually needed, and with `#ifdef MOZ_GECKO_PROFILER` guards.
Rough inclusion diagram: (header <-- includer)
BaseProfilerMarkerPrerequesites.h <-- ProfilerMarkerPrerequesites.h (Useful types: Input string view, marker options)
^ ^
BaseProfilerMarkerDetail.h <-- ProfilerMarkerDetail.h (Implementation details)
^ ^
BaseProfilerMarkers.h <-- ProfilerMarkers.h (Main API)
^ ^--------- ^ ^---------
BaseProfilerMarkerTypes.h | <-- ProfilerMarkerTypes.h | (Common marker types)
^ BaseProfiler.h <-- GeckoProfiler.h (Existing main profiler headers)
Differential Revision: https://phabricator.services.mozilla.com/D87241
In Win7 and later, some exported functions in kernel32.dll are just a stub
jumping to a function in kernelbase.dll. After the fix for Bug 1642626,
our detour resolves such a stub in kernel32.dll and detours a corresponding function in
kernelbase.dll. This new behavior caused a problem in Win8 when we detour
`DuplicateHandle` because our detour cannot handle the assembly pattern of
`KERNELBASE!DuplicateHandle`.
Win8's `KERNELBASE!DuplicateHandle` has jump instructions whose destination is
within the region where we move instructions to a trampoline.
In the example below, the address `000007f954ad271c` is a destination of the
`JMP` instructions, but when we detour `KERNELBASE!DuplicateHandle`, we move
the original instructions to a trampoline, and that address will point to
an invalid instruction, jumping to which address causes a crash.
A proposed fix is to detour `KERNEL32!DuplicateHandle` without resolving redirection,
that is the behavior before bug 1642626.
```
KERNEL32!DuplicateHandle:
000007f9`54cd2d5c ff2556b61100 jmp qword ptr [KERNEL32!_imp_DuplicateHandle] --> KERNELBASE!DuplicateHandle
```
```
KERNELBASE!DuplicateHandle:
000007f9`54ad2710 4883ec48 sub rsp,48h
000007f9`54ad2714 4c8bd1 mov r10,rcx
000007f9`54ad2717 83faf4 cmp edx,0FFFFFFF4h
000007f9`54ad271a 733b jae KERNELBASE!DuplicateHandle+0x43 (000007f9`54ad2757)
000007f9`54ad271c 8b842480000000 mov eax,dword ptr [rsp+80h]
...
000007f9`54b8f0de 65488b042560000000 mov rax,qword ptr gs:[60h]
000007f9`54b8f0e7 488b5020 mov rdx,qword ptr [rax+20h]
000007f9`54b8f0eb 488b5220 mov rdx,qword ptr [rdx+20h]
000007f9`54b8f0ef e92836f4ff jmp KERNELBASE!DuplicateHandle+0xc (000007f9`54ad271c)
000007f9`54b8f0f4 65488b042560000000 mov rax,qword ptr gs:[60h]
000007f9`54b8f0fd 488b5020 mov rdx,qword ptr [rax+20h]
000007f9`54b8f101 488b5228 mov rdx,qword ptr [rdx+28h]
000007f9`54b8f105 e91236f4ff jmp KERNELBASE!DuplicateHandle+0xc (000007f9`54ad271c)
000007f9`54b8f10a 65488b042560000000 mov rax,qword ptr gs:[60h]
000007f9`54b8f113 488b5020 mov rdx,qword ptr [rax+20h]
000007f9`54b8f117 488b5230 mov rdx,qword ptr [rdx+30h]
000007f9`54b8f11b e9fc35f4ff jmp KERNELBASE!DuplicateHandle+0xc (000007f9`54ad271c)
```
Differential Revision: https://phabricator.services.mozilla.com/D88136
This patch adds a new property `process_type` to the launcher process failure
ping, indicating which process type the browser process failed to initialize
as a sandboxed process.
Depends on D83639
Differential Revision: https://phabricator.services.mozilla.com/D83640
This patch adds winlauncher's HandleLauncherError to DllServices
along with InitializeDllBlocklistOOPInternal so that SandboxBroker
can call HandleLauncherError.
Differential Revision: https://phabricator.services.mozilla.com/D83639
Normally I believe it's good to keep functionally-distinct code in separate functions when appropriate.
However in this case, this `ChunkedJSONWriteFunc::GetTotalLength()` function was only used internally, so it's easy to hide it. It is not very big, so it's less important to keep it separate. And its code (going through all chunks to collect sizes) is similar to the code that follows (going through all chunks to collect their content), so it makes some sense to have these similar things in the same place; if one day this chunking change, both loops would probably have to be modified at the same time.
More importantly to justify this change: It was including the final null terminator, which was a bit inconsistent with the usual `Length()` functions in string containers.
Now that code is only used where absolutely necessary (before allocating an output buffer), so it rightly accounts for the null terminator that is added at the end of the contents.
And in upcoming bug 1612799, `ChunkedJSONWriteFunc` will have a new "retired" state (e.g., after an internal memory allocation failed) in which case a public `GetTotalLength()` function could give misleading results -- should it be 0 (nothing to output), 1 (only the null terminator), 5 ("null"), or the length of some error message? So I think it's better not to expose such an ambiguous function.
It could of course be re-introduced if new needs arise in the future.
Differential Revision: https://phabricator.services.mozilla.com/D87833
In most calls to `SpliceableJSONWriter::Splice(const char*)`:
- The data comes from a `ChunkedJSONWriteFunc` and is copied to a new buffer, which is then copied again through `Write()`. Instead we can copy the data directly from the `ChunkedJSONWriteFunc`; and this is a nice complement to `TakeAndSplice()` below.
- Or the length is already known, so we can pass it to a new `Splice(const char*, size_t)`, which forwards it to `Write(const char*, size_t)`, saving one `strlen` call.
Differential Revision: https://phabricator.services.mozilla.com/D87703
`SpliceableChunkedJSONWriter::ChunkedWriteFunc` returns a `ChunkedJSONWriteFunc*`, which is never null and is either used to:
1. Copy data.
2. Or take ownership of the chunks.
In the first case, `ChunkedWriteFunc()` now returns a `const ChunkedJSONWriteFunc&` (notice "const &"), so only const members may be used to copy the data.
In the second case, a new function `TakeChunkedWriteFunc()` returns `ChunkedJSONWriteFunc&&` (notice "&&"), so it's clear that its chunks can be taken away. Some `DEBUG` assertions help ensure that it's not used anymore after that.
`TakeAndSplice()` now takes a `ChunkedJSONWriteFunc&&`.
All callers have been updated to the more appropriate functions.
Differential Revision: https://phabricator.services.mozilla.com/D87702
This patch makes it possible to serialize and deserialize std::basic_string's of different character types, not just std::string. This will be useful when dealing with char16_t from nsString and others.
We also expose the internal read-contents functions, to allow the user to read a sequence of characters of already-known length.
Differential Revision: https://phabricator.services.mozilla.com/D87237
Hopefully the comments in the actual code are enough to explain what is going
on here.
NOTE: this patch does not represent a finished skeleton UI. There are some
questions in comments within the code, and generally I'm seeking feedback on
whether the overall approach seems sane or not. Gijs, I'm including you for
feedback on whether you think this is maintainable by more frontend-oriented
folks, and Molly, I'm including you for feedback on whether the justification
for writing to a raw pixel buffer seems sound or not, and a general review of
the approach.
Differential Revision: https://phabricator.services.mozilla.com/D86447
See bug for justification. This patch aims to display a blank window prior to
loading/prefetching xul.dll. It also has a placeholder for drawing a
skeleton UI into that window. Note that this is disabled by default based on
a registry value, as there are still kinks to work out (for instance, what
happens if we aren't actually going to display a window, because, say, Firefox
is already running.) This just gives a basic implementation to dogfood, and
facilitates distributing work across multiple contributors.
Onto the details. The patch achieves its goal by creating a window and
assigning its handle to a static variable, which will be consumed inside
nsWindow::Create by the first toplevel window we want to make. nsWindow::Create
will take ownership of the window handle, restyle it to its own liking, and
then proceed as if everything is normal and it had created the window itself.
Differential Revision: https://phabricator.services.mozilla.com/D86263
This change makes everything compile and presumably has some tiny effect on
performance. We also take the opportunity to ensure that the compiler won't
optimize out our inline asm by making the asm volatile and indicating that
it touches memory.
Differential Revision: https://phabricator.services.mozilla.com/D86594
The stack sampling can be abstracted to only use a reference to a `ProfileBuffer`, and the existing `locked_profiler_get_backtrace` can provide its stack-based `ProfileBuffer` that points at a heap-based `ProfileChunkedBuffer` (the one that will be stored in the returned `ProfilerBacktrace`).
And we can now add a public `profiler_capture_backtrace` that only takes a reference to a `ProfileChunkedBuffer`, and fills it with a backtrace.
This will be used by the new marker API, to optionally capture a backtrace in stack-based buffers at the user's request.
Differential Revision: https://phabricator.services.mozilla.com/D86514
A heap-allocate ProfileBuffer is not really needed, so it's more efficient to have one on the stack during capture, and we don't need to keep it in the `ProfilerBacktrace`.
Differential Revision: https://phabricator.services.mozilla.com/D86513
Instead of always taking ownership of both heap-allocated `ProfileBuffer` and `ProfileChunkedBuffer`, `ProfilerBacktrace` can now accept:
- Unique pointers to both or either, similar to what it was before, so a ProfilerBacktrace can be kept for later use.
- Non-owning pointers to both or either, to allow callers to use stack-based buffer(s); null pointers are allowed for totally empty backtraces.
Only the `ProfileChunkedBuffer` contains the actual data, we can create a `ProfileBuffer` on the spot if not provided.
Differential Revision: https://phabricator.services.mozilla.com/D86512
Instead of keeping a pointer to a null-terminated string, it's simpler to keep a proper `std::string`, and it helps to keep the length ready for streaming.
Differential Revision: https://phabricator.services.mozilla.com/D86511
Bogdan Tara
Emma Malysz
Toshihito Kikuchi
Doug Thayer
Gerald Squelart
Narcis Beleuzu
Alexis Beingessner
Florian Quèze
Gabriele Svelto
Butkovits Atila
Sebastian Hengst
Byron Campen [:bwc]
Greg Tatum
Ricky Stewart
sanketh
Dorel Luca
Christian Holler
Andreea Pavel
David Parks
Markus Stange
Jon Coppeard
Cristina Coroiu
Paul Bone
Brindusan Cristian
Sylvestre Ledru
Mihai Alexandru Michis
Razvan Maries
Nathan Froyd