This patch is to improve the way to detect an injected dependent module for
automatic DLL blocking (bug 1659438).
In the previous version, we created a list of dependent modules in the launcher
process and shared it with other processes via the shared section. However, it
was not compatible with third-party applications who tamper the Import Table and
revert it in the injected module's DllMain (bug 1682834) because we parsed the
Import Table in the launcher process after it was reverted.
With this patch, we check the Import Table in `patched_NtMapViewOfSection`,
so we can see tampering before it's reverted. More specifically, we create
a list of dependent modules in the browser process as below.
1. The launcher process creates a section object and initializes
the kernel32.dll's functions in it.
2. The launcher process transfers a writable handle of the shared
section to the browser process.
3. In the browser process, if an injected dependent module is being
mapped by `NtMapViewOfSection`, we add its NT path to the shared
section and block it with `REDIRECT_TO_NOOP_ENTRYPOINT`.
4. The `main` function of the browser process converts the writable
handle of the shared section into a readonly handle.
5. The browser process transfers a readonly handle of the shared
section to a sandbox process.
Since automatic DLL blocking may still cause a compat issue like bug 1682304,
we activate it only in Nightly for now.
Differential Revision: https://phabricator.services.mozilla.com/D101460
Currently, printf_stderr doesn't show up when running with ./mach run.
This is because we run with -attach-console and that redirects stderr
to a different file descriptor using freopen in UseParentConsole.
The change from just using stderr directly happened in bug 340443 and was done
to avoid some linking issues. That problem doesn't seem to apply anymore so you'd
expect we'd be able to go back to the straightforward implemention that works even
if stderr has been redirected. Unforunately, Windows takes not buffering
stderr very seriously and fprintf will write out the results character
by character. This can cause log output lines to be intermixed which
breaks log parsing in CI. We keep using fdopen to create a new FILE*
that's buffered but instead of hard coding fd 2, we get the actual fd
that corresponds to stderr using fileno.
The mozglue implementation was cargo culted from xpcom, so we update it
as well.
Differential Revision: https://phabricator.services.mozilla.com/D98550
CLOSED TREE
Backed out changeset f82f5070bee5 (bug 1205985)
Backed out changeset 89b03879ce7d (bug 1205985)
Backed out changeset 9ba60febbcf8 (bug 1205985)
This eliminates all of the thread leaks we had on record, while also increasing
our coverage on the code that's used for testing firefox.
Differential Revision: https://phabricator.services.mozilla.com/D100264
`ImageBridgeChild` was turned into a one-off thread due to a deadlock, but when bug 1672255 is finished we can revert the changes back to a background task queue without any issues and get rid of these suppressions.
Differential Revision: https://phabricator.services.mozilla.com/D100257
The `RunningTimes` class stores CPU measurements. It may seem overkill for only one value, but in the future more measurements will be added.
During sampling, CPU measurements are collected by platform-specific code. This patch doesn't produce anything yet, see later patches.
These are stored with the samples.
Note that for duplicated samples (when a thread is known to be "asleep"), we still need to collect new measurements, because there could potentially be some activity happening, e.g. in system calls.
Finally the measurements are output as extra "samples" values.
Units for these values may platform-specific, so they are stored in the top-level JSON "meta" object.
We don't collect running times in the Base Profiler (yet), but we still need to add the appropriate field names in the samples' "schema", as expected by profiler.firefox.com.
Differential Revision: https://phabricator.services.mozilla.com/D99413
This patch adds "CPU Utilization" ("cpu" for short) as a new feature that will control the upcoming still-experimental CPU measurements.
Differential Revision: https://phabricator.services.mozilla.com/D99054
Instead of only capturing one feature (NoStackSampling), the sampler thread now stores all features so that any feature can be quickly looked at during sampling.
Currently this is still limited to NoStackSampling, a later patch will start using another feature.
Differential Revision: https://phabricator.services.mozilla.com/D99053
Now is already supported when CLOCK_MONOTONIC is supported, but
ComputeProcessUptime is not. This shares the code with other BSDs, and
makes it look like the implementation in Timestamp_darwin.cpp.
Eventually, we'll remove the one from Timestamp_darwin.cpp.
Differential Revision: https://phabricator.services.mozilla.com/D100069
Some sites do have stacks that require more than 64KB to store in the profiler buffer.
Note that this only affects one semi-permanent buffer per process during profiling, and short-lived buffers when capturing stacks in markers.
Differential Revision: https://phabricator.services.mozilla.com/D99981
The "expected maximum stack size" (currently 64KiB) value was present in multiple places.
Now it's accessible from everywhere as ProfileBufferChunkManager::scExpectedMaximumStackSize, so it's easier to modify as needed.
Differential Revision: https://phabricator.services.mozilla.com/D100222
Instead of discarding released chunks, we keep them as "next" chunks, and we make sure there's a valid "current" chunk when possible.
Recycling released chunks means that when using the `ProfileBufferChunkManagerSingle`, the one chunk, in whichever state it may be, will be kept alive and reused.
Differential Revision: https://phabricator.services.mozilla.com/D99979
When a "Put" operation fails (most probably because no chunk was available to store the data), we remember the number of bytes that couldn't be stored.
This can be useful to give an indication of how much more memory would have been needed for successful puts.
Differential Revision: https://phabricator.services.mozilla.com/D99977
This bug has revealed some issues when the single chunk gets filled, and there are different paths depending on whether the chunk is filled right to the end, or past it.
Later patches will fix these issues and update these tests accordingly.
Differential Revision: https://phabricator.services.mozilla.com/D99976
In DEBUG builds, instead of only testing for the expected chunk state(s), there are now separate assertions for each *un*expected state, which makes it much easier to track the source of failures.
Differential Revision: https://phabricator.services.mozilla.com/D99974
So this is an ugly solution, but it was the best I could come up with. We do
not want to show the skeleton UI if we're going to show the profile manager,
and we *will* show the profile manager if StartWithLastProfile=0 is under
[General] in profiles.ini. Accordingly the only ways to do the correct thing
here are to try to mirror edits that firefox makes to the profiles.ini file
to the registry, or to simply read the profiles.ini file ourselves. There are
many ways that profiles.ini could get out of sync with the registry if we
tried to mirror its state there, so going straight to the source of truth
seemed the best option.
There is one case which is still not covered here: if there is no profile for
our install marked as Default=1, then we will show the profile manager. This
should only be possible if the user manually edits their profiles.ini file,
however, and then it should resolve itself after one run, so I don't consider
it a significant enough problem to jump through all the hoops we would need
to jump through to solve it.
Depends on D98525
Differential Revision: https://phabricator.services.mozilla.com/D98936
Note that we can probably use mLastRefreshDriverTime directly in
DocumentTimeline::GetCurrentTimeStamp(), i.e. we don't need to use the refresh
driver there, but I'd preserve the current behavior.
Differential Revision: https://phabricator.services.mozilla.com/D97823
Note that we can probably use mLastRefreshDriverTime directly in
DocumentTimeline::GetCurrentTimeStamp(), i.e. we don't need to use the refresh
driver there, but I'd preserve the current behavior.
Differential Revision: https://phabricator.services.mozilla.com/D97823
Add `mWaitingForDecode` to indicate that main-thread is blocked on the
Monitor and should be woken. This avoids generating some of the unused
Runnables. Also ensure `mFinishDecodeRunnablePending` is only accessed while
holding the lock.
Depends on D99403
Differential Revision: https://phabricator.services.mozilla.com/D99404
The `mToken` is used across threads without full locking so it must be marked
as atomic. At the same time, we can use `exchange` operations to read or
write outside of the mMonitor lock.
Depends on D99402
Differential Revision: https://phabricator.services.mozilla.com/D99403
Be consistent about always calling `MaybeFinishOffThreadDecode` without
holding the lock to simplify code. This lets us remove a TSAN deadlock
exception.
Differential Revision: https://phabricator.services.mozilla.com/D99402
The menubar is permanently shown if autohide is false. If that is the case, we
insert space above the tab and ensure the tab does not have a left margin.
It's height can change, so we store the height in our registry.
Differential Revision: https://phabricator.services.mozilla.com/D97195
Some env vars have effects similar to command line arguments which present
problems for the skeleton UI, and we want to treat these env vars similarly.
Differential Revision: https://phabricator.services.mozilla.com/D98475
Previously, we implemented arg checking with `marionette` just carrying a free
pass, so we could let the arguments which typically come when running tests.
However, some marionette tests do like to play with arguments which we do not
want to get a free pass, such as -safe-mode. These changes allow just the
-profile argument through, as that is necessary for running tests.
Differential Revision: https://phabricator.services.mozilla.com/D98474
Some env vars have effects similar to command line arguments which present
problems for the skeleton UI, and we want to treat these env vars similarly.
Differential Revision: https://phabricator.services.mozilla.com/D98475
Previously, we implemented arg checking with `marionette` just carrying a free
pass, so we could let the arguments which typically come when running tests.
However, some marionette tests do like to play with arguments which we do not
want to get a free pass, such as -safe-mode. These changes allow just the
-profile argument through, as that is necessary for running tests.
Differential Revision: https://phabricator.services.mozilla.com/D98474
Even though the Glean dispatcher was shutting down, the SDK was not
properly joining on the thread, triggering a TSAN error. This has been
addressed in a new version of the Glean SDK.
Differential Revision: https://phabricator.services.mozilla.com/D98889
We don't want to show the skeleton UI if there is already an instance of
Firefox running for that install. Accordingly, we implement something
similar to the profile lock, acquiring exclusive access to
~/AppData/Local/Mozilla/Firefox/SkeletonUILock-<installHash>. If we do not do
this, then when a user clicks firefox.exe while an existing instance is
running, under default conditions we will open the skeleton UI, then
almost immediately terminate and send a message to the existing instance to
open a new window.
Differential Revision: https://phabricator.services.mozilla.com/D98525
TSan found races on these flags that are ostensibly benign but this way there's no UB.
This code is a bit weird. These bitfields are seemingly pointless as
they're squeezed between a uint64_t and an Atomic<bool>. There's plenty
of space for 2 more bools there.
Also the Atomic<bool> could theoretically be merged into the flags. For
now, here's the version of this patch that preserves the semantics of
this code as closely as possible, for review by the code owners.
Differential Revision: https://phabricator.services.mozilla.com/D93416
TSan found races between mResolveAgain and mGetTtl. This makes them non-UB.
The code is a bit weird. Although the values are typed as uint16_t's, they're
used as bools (even assigned true/false). In addition, the atomic bool mTRRUsed
could be folded into these fields, as there is a spare bit. I decided not to
change these things, as network code can have weird representation requirements
that I'd prefer the owners of the code chime in on first.
Differential Revision: https://phabricator.services.mozilla.com/D93417
TSan found races on these flags that are ostensibly benign but this way there's no UB.
This code is a bit weird. These bitfields are seemingly pointless as
they're squeezed between a uint64_t and an Atomic<bool>. There's plenty
of space for 2 more bools there.
Also the Atomic<bool> could theoretically be merged into the flags. For
now, here's the version of this patch that preserves the semantics of
this code as closely as possible, for review by the code owners.
Differential Revision: https://phabricator.services.mozilla.com/D93416
TSan found races between mResolveAgain and mGetTtl. This makes them non-UB.
The code is a bit weird. Although the values are typed as uint16_t's, they're
used as bools (even assigned true/false). In addition, the atomic bool mTRRUsed
could be folded into these fields, as there is a spare bit. I decided not to
change these things, as network code can have weird representation requirements
that I'd prefer the owners of the code chime in on first.
Differential Revision: https://phabricator.services.mozilla.com/D93417
clang 12 (specifically https://reviews.llvm.org/D91379) made some refactorings to libc++ that exposed a problem in the MinGW headers. That has now been fixed upstream.
In the meantime the headers also gained definitions for ProcessPayloadRestrictionPolicy, so we can remove our workaround for that.
Differential Revision: https://phabricator.services.mozilla.com/D98945
No idea why this didn't show up in earlier try runs, but this is a known false-positive that we need to suppress in the newly-added EncryptedClientHelloServer.
Differential Revision: https://phabricator.services.mozilla.com/D98771
Currently, printf_stderr doesn't show up when running with ./mach run.
This is because we run with -attach-console and that redirects stderr
to a different file descriptor using freopen in UseParentConsole.
The change from just using stderr directly happened in bug 340443 and was done
to avoid some linking issues. That problem doesn't seem to apply anymore so we
should be able to go back to the straightforward implemention that works even
if stderr has been redirected. The mozglue implementation was cargo culted from
xpcom, and there wasn't a reason other than that for the fdopen(dup()) there.
Differential Revision: https://phabricator.services.mozilla.com/D98550
TimeStamps in markers must now be streamed through `SpliceableJSONWriter::TimeProperty(name, timestamp)`.
This is consistent with all other JSON-writing functions being in `SpliceableJSONWriter` (and base class `JSONWriter`).
Depends on D97556
Differential Revision: https://phabricator.services.mozilla.com/D97557
Expand `profiler_is_locked_on_current_thread()` to also return true if the ProfilerChild mutex is locked on the current thread.
This will prevent some profiler-re-entrant operations (like native allocation markers) from running.
In particular, this removes the potential deadlock found in bug 1671403:
- SamplerThread: In the sampling loop, lock the main profiler mutex, run a ProfilerChild update, which attempts to lock the ProfilerChild mutex.
- ProfilerChild thread: While processing an IPC message with an update, lock the ProfilerChild mutex, then resolve the update, which records a native allocation with a backtrace that attempts to lock the main profiler mutex.
With this patch, the native allocation won't record a marker while the ProfilerChild mutex is locked.
Differential Revision: https://phabricator.services.mozilla.com/D96969
In this case, the same marker type "CONTENT_FULL_PAINT_TIME" is used in separate places, so it makes sense to put the marker type definition in a common location.
Differential Revision: https://phabricator.services.mozilla.com/D96042
To be able to deserialize raw pointers (when streaming markers), `ProfileBufferRawPointer` must be used (to prevent uncontrolled raw pointers), but with the new API implementation the deserializer must accept references to `ProfilerBufferRawPointer` instead of just a raw pointer.
Differential Revision: https://phabricator.services.mozilla.com/D96032
In this case, the same marker type "CONTENT_FULL_PAINT_TIME" is used in separate places, so it makes sense to put the marker type definition in a common location.
Differential Revision: https://phabricator.services.mozilla.com/D96042
To be able to deserialize raw pointers (when streaming markers), `ProfileBufferRawPointer` must be used (to prevent uncontrolled raw pointers), but with the new API implementation the deserializer must accept references to `ProfilerBufferRawPointer` instead of just a raw pointer.
Differential Revision: https://phabricator.services.mozilla.com/D96032
In this case, the same marker type "CONTENT_FULL_PAINT_TIME" is used in separate places, so it makes sense to put the marker type definition in a common location.
Differential Revision: https://phabricator.services.mozilla.com/D96042
To be able to deserialize raw pointers (when streaming markers), `ProfileBufferRawPointer` must be used (to prevent uncontrolled raw pointers), but with the new API implementation the deserializer must accept references to `ProfilerBufferRawPointer` instead of just a raw pointer.
Differential Revision: https://phabricator.services.mozilla.com/D96032
* Bumps the tsan toolchain to rust-nightly-2020-11-14 that has my patches to make -Zbuild-std work in vendored environments:
* https://github.com/rust-lang/cargo/pull/8834
* https://github.com/rust-lang/rust/pull/78790
* Passes -Zbuild-std to cargo when MOZ_TSAN is defined (mk_add_options --enable-thread-sanitizer)
* Removes generic Rust supressions and adds much more specific ones
* One presumed upstream false positive from tsan not understanding the code
* One actual upstream bug tsan found (yay!)
* One new real issue uncovered
* One issue that probably already existed intermittently but I happened to hit
Differential Revision: https://phabricator.services.mozilla.com/D97165
I'm hoping that the comments in the code are sufficient documentation of this,
but here is a quick overview. The mockup we got from design has rounded rects
in it, and bordered rounded rects in it, so we need to support drawing those.
Initially we tried using the RoundRect function in GDI. However, 1) GDI doesn't
support anti-aliasing, which is unfortunately noticeable, 2) we were
observing strange issues with only some of the corners being rounded with
RoundRect at higher radii / stroke widths. 3) drawing on top of things drawn
with RoundRect would be complicated and inefficient unless we switched
everything over to GDI calls.
As it stands this drawing code is platform agnostic, assuming we have a way of
blitting a raw bitmap to the screen on a given platform, which is a nice trait
to have and makes me think twice about switching all of the drawing over to
direct GDI calls.
Differential Revision: https://phabricator.services.mozilla.com/D95317
I'm hoping that the comments in the code are sufficient documentation of this,
but here is a quick overview. The mockup we got from design has rounded rects
in it, and bordered rounded rects in it, so we need to support drawing those.
Initially we tried using the RoundRect function in GDI. However, 1) GDI doesn't
support anti-aliasing, which is unfortunately noticeable, 2) we were
observing strange issues with only some of the corners being rounded with
RoundRect at higher radii / stroke widths. 3) drawing on top of things drawn
with RoundRect would be complicated and inefficient unless we switched
everything over to GDI calls.
As it stands this drawing code is platform agnostic, assuming we have a way of
blitting a raw bitmap to the screen on a given platform, which is a nice trait
to have and makes me think twice about switching all of the drawing over to
direct GDI calls.
Differential Revision: https://phabricator.services.mozilla.com/D95317
This patch supports a skeleton UI for default, light, and dark themes.
It is not enabled for apenglow or any custom themes.
This also takes into account the system theme. If the user has the default
theme selected and is in dark mode, we override the theme and present the
dark theme skeleton UI.
Differential Revision: https://phabricator.services.mozilla.com/D96230
This patch supports a skeleton UI for default, light, and dark themes.
It is not enabled for apenglow or any custom themes.
This also takes into account the system theme. If the user has the default
theme selected and is in dark mode, we override the theme and present the
dark theme skeleton UI.
Differential Revision: https://phabricator.services.mozilla.com/D96230
This patch adds a list of the executable's dependent module's path to SharedSection
as an array of the offset to a string and a string buffer. A following patch will
use this data from the browser process and the sandboxed processes.
Depends on D96283
Differential Revision: https://phabricator.services.mozilla.com/D96284
We transfer several ntdll's function addresses to a child process directly via
`WriteProcessMemory`. This patch changes the way to transfer data to using
a section object as Chromium sandbox does, so that we can transfer more data
with the same cost as transferring a single handle value.
Depends on D96282
Differential Revision: https://phabricator.services.mozilla.com/D96283
If a searchbar is present, draw that similar to a urlbar.
This also changes how we store the urlbar, as we know will use our custom struct
to cleanly store the width and height.
Differential Revision: https://phabricator.services.mozilla.com/D95029
Unique strings are used to encode all markers' 'name' field, SpliceableJSONWriter::UniqueStringElement can be used for that (instead of a caller-provided callback, which was necessary before UniqueJSONStrings was de-duplicated).
Differential Revision: https://phabricator.services.mozilla.com/D95513
Some markers (e.g., Screenshot) use unique strings in their data.
The UniqueJSONStrings used during streaming is attached to the SpliceableJSONWriter, and StreamJSONMarkerData can use pass-through functions UniqueStringProperty() and UniqueStringElement().
Differential Revision: https://phabricator.services.mozilla.com/D95512
Some markers (e.g., GC major/minor/slice) need to splice JSON strings in their data.
So now, instead of JSONWriter, StreamJSONMarkerData functions will be given a mozilla::baseprofiler::SpliceableJSONWriter.
Differential Revision: https://phabricator.services.mozilla.com/D95511
`UniqueJSONStrings` constructors now accept a `JSONWriter::CollectionStyle` to pass to the internal JSON writer.
The default won't change how the unique string list is output (with friendly indented lines), but tests can use `JSONWriter::SingleLineStyle` to make comparison strings more readable and maintainable.
Differential Revision: https://phabricator.services.mozilla.com/D95832
Alexis Beingessner
Molly Howell
Emma Malysz
Toshihito Kikuchi
Razvan Maries
Brindusan Cristian
Gerald Squelart
Paul Adenot
Jeff Muizelaar
Mihai Alexandru Michis
Kris Wright
Narcis Beleuzu
Masatoshi Kimura
Mike Hommey
Jon Coppeard
Doug Thayer
smolnar
Hiroyuki Ikezoe
Andreea Pavel
Steve Fink
Ted Campbell
Andrew McCreight
Alessio Placitelli
Bogdan Tara
David Major
Simon Giesecke
Kevin Jacobs
Geoff Brown
Florian Quèze
Csoregi Natalia
Sylvestre Ledru
Christian Holler