The ports library import came with 2 gtests. This patch gets them running under
our gtest runner to ensure we don't break the ports functionality.
Differential Revision: https://phabricator.services.mozilla.com/D112768
This removes the last form of unique link between two MessageChannels so that
all MessageChannels communicate using PortLink, as it is fairly straightforward
to use PortLink to communicate between two threads in-process.
Differential Revision: https://phabricator.services.mozilla.com/D116672
This adjusts how all actors created using `Endpoint` behave so that they now
use ports instead of creating a unique native channel connection between each
pair of processes.
Differential Revision: https://phabricator.services.mozilla.com/D116670
These port attachments are stored directly on the IPC::Message until the
message is ready to be routed to another process, at which point they will be
attached to the port in WillBeRoutedExternally. When the message is then
received on the other side, the ports will be re-extracted from the
UserMessageEvent before it is discarded and re-added to the IPC::Message so
that serializers only need to interact directly with the IPC::Message type.
Differential Revision: https://phabricator.services.mozilla.com/D116669
This is used because, unlike in Mojo, we cannot get from the IPC::Message
object to its enclosing UserMessageEvent object to attach more ports to it, and
this extra parameter makes that easy to do.
Differential Revision: https://phabricator.services.mozilla.com/D116668
This unfortunately requires a new method to be added to BufferList to
support truncating the buffer to a particular iterator.
Differential Revision: https://phabricator.services.mozilla.com/D116666
This extends on the changes in part 12a and consumes the new PortRef-based API
in all existing process types other than the fork server. The IPDL C++ unit
tests were already broken before this change, and were not updated.
Differential Revision: https://phabricator.services.mozilla.com/D112777
This also consumes the existing channel created when launching a process to
create the the conneciton required by NodeController for communicating between
processes. In part 12b, consumers of the broken APIs will be adjusted to use
the new interface.
The new routing approach is not used for the fork server process, as an IO
thread and the NodeController object cannot be initialized before the fork has
been performed, and the IPC requirements of that process are fairly minimal.
Differential Revision: https://phabricator.services.mozilla.com/D112776
The NodeController and NodeChannel types act as the backbone connecting the
existing IPC logic and driving the ports routing code. Individual NodeChannel
objects wrap and respond to messages from IPC::Channel, and the NodeController
orchestrates all messaging for a process.
The design of these types are inspired by the types with the same names from
Mojo but have been simplified and streamlined to only support features used by
Gecko.
Support for attaching ports or handles to messages hasn't been added yet, but
can be added in follow-up patches.
Differential Revision: https://phabricator.services.mozilla.com/D112775
These will be used to serialize extra event metadata into IPC messages when
they're sent over the ports infrastructure. In the future better integration
may be used to reduce the overhead of this if necessary.
Differential Revision: https://phabricator.services.mozilla.com/D112773
The ports library import came with 2 gtests. This patch gets them running under
our gtest runner to ensure we don't break the ports functionality.
Differential Revision: https://phabricator.services.mozilla.com/D112768
* We make `EnsureMTA`'s default constructor `private`, and `ProcessRuntime` a friend.
`ProcessRuntime` calls this to eagerly create the MTA.
* The default constructor uses the new-ish `CoIncrementMTAUsage` to create the
MTA without requiring a dedicated thread (when available). Otherwise we
fall back to the traditional method. In the latter case, we synchronously
wait for the initialization to complete so that we are guaranteed to have
an MTA when we return.
* Some minor refactoring to make it easier to do the sync wait in the
default constructor. I also renamed a couple of things just to make them
more clear.
Differential Revision: https://phabricator.services.mozilla.com/D113562
This patch does the following:
* General cleanup:
* More explicit restrictions of how/when the various constructors are available.
* `InitializeSecurity` is made `static`, since it doesn't really manipulate instance variables.
* We move some logic for resolving `CoInitializeEx` flags into `GetDesiredApartmentType`.
* Addition of `PostInit`:
* This doesn't do anything at the moment, but since I'm already making a bunch
of changes, I wanted to add this too. `PostInit` is a static method that
is invoked once the `ProcessRuntime` is finished initializing, and, when
present, the sandbox is fully enabled.
* We call `EnsureMTA`'s default constructor to eagerly bring up the MTA. This
causes all background threads to implicitly become members of the MTA, which
means that we can eliminate `CoInitializeEx` calls throughout our codebase,
since they're slow and most developers do not have a clear understanding of
what those functions actually do.
* This also simplifies the COM initialization in sandboxed content processes
with Win32K lockdown, since if our main thread is in the implicit MTA, we
can immediately initialize ourselves without needing to punt that work
over to the persistent MTA thread.
Differential Revision: https://phabricator.services.mozilla.com/D113560
This patch sets up a few different things that will be used by the WER runtime
exception module when it needs to notify the main process of a child process
crash.
For every child process we allocate a structure in the main process called
WindowsErrorReportingData that contains three things:
- The address of the function used to notify the main process that there's a
pending minidump for a given child process
- The PID of said child process
- The name of the minidump that has been generated
The first field is filled up by the main process and will be read by the WER
process when running the runtime exception module, the second and third fields
on the other hand start empty and will be written into by the runtime exception
module after it has generated a minidump.
I know this sounds scary. It is. But bear with me please.
When we register the runtime exception module we can pass it a single
pointer-sized parameter but we need to pass it at least another pointer that
includes data coming from the child process itself (this one is called
InProcessWindowsErrorReportingData). This data currently includes only the
process type but will also include certain annotations in the future
(e.g. bug 1711418). So here's what we do: we store a pointer to the parent
data structure in the child process command-line (cringe) and we read it
from the runtime exception module by reading the crashed process command-line
arguments and parsing them (double-cringe).
Armed with this information the WER runtime exception module can populate
the info for the generated minidump and then push it into the main process
by calling CreateRemoteThread() (which creates a new thread in the main
process, triple-cringe at this point).
Differential Revision: https://phabricator.services.mozilla.com/D115379
This patch sets up a few different things that will be used by the WER runtime
exception module when it needs to notify the main process of a child process
crash.
For every child process we allocate a structure in the main process called
WindowsErrorReportingData that contains three things:
- The address of the function used to notify the main process that there's a
pending minidump for a given child process
- The PID of said child process
- The name of the minidump that has been generated
The first field is filled up by the main process and will be read by the WER
process when running the runtime exception module, the second and third fields
on the other hand start empty and will be written into by the runtime exception
module after it has generated a minidump.
I know this sounds scary. It is. But bear with me please.
When we register the runtime exception module we can pass it a single
pointer-sized parameter but we need to pass it at least another pointer that
includes data coming from the child process itself (this one is called
InProcessWindowsErrorReportingData). This data currently includes only the
process type but will also include certain annotations in the future
(e.g. bug 1711418). So here's what we do: we store a pointer to the parent
data structure in the child process command-line (cringe) and we read it
from the runtime exception module by reading the crashed process command-line
arguments and parsing them (double-cringe).
Armed with this information the WER runtime exception module can populate
the info for the generated minidump and then push it into the main process
by calling CreateRemoteThread() (which creates a new thread in the main
process, triple-cringe at this point).
Differential Revision: https://phabricator.services.mozilla.com/D115379
This patch sets up a few different things that will be used by the WER runtime
exception module when it needs to notify the main process of a child process
crash.
For every child process we allocate a structure in the main process called
WindowsErrorReportingData that contains three things:
- The address of the function used to notify the main process that there's a
pending minidump for a given child process
- The PID of said child process
- The name of the minidump that has been generated
The first field is filled up by the main process and will be read by the WER
process when running the runtime exception module, the second and third fields
on the other hand start empty and will be written into by the runtime exception
module after it has generated a minidump.
I know this sounds scary. It is. But bear with me please.
When we register the runtime exception module we can pass it a single
pointer-sized parameter but we need to pass it at least another pointer that
includes data coming from the child process itself (this one is called
InProcessWindowsErrorReportingData). This data currently includes only the
process type but will also include certain annotations in the future
(e.g. bug 1711418). So here's what we do: we store a pointer to the parent
data structure in the child process command-line (cringe) and we read it
from the runtime exception module by reading the crashed process command-line
arguments and parsing them (double-cringe).
Armed with this information the WER runtime exception module can populate
the info for the generated minidump and then push it into the main process
by calling CreateRemoteThread() (which creates a new thread in the main
process, triple-cringe at this point).
Differential Revision: https://phabricator.services.mozilla.com/D115379
* We make `EnsureMTA`'s default constructor `private`, and `ProcessRuntime` a friend.
`ProcessRuntime` calls this to eagerly create the MTA.
* The default constructor uses the new-ish `CoIncrementMTAUsage` to create the
MTA without requiring a dedicated thread (when available). Otherwise we
fall back to the traditional method. In the latter case, we synchronously
wait for the initialization to complete so that we are guaranteed to have
an MTA when we return.
* Some minor refactoring to make it easier to do the sync wait in the
default constructor. I also renamed a couple of things just to make them
more clear.
Differential Revision: https://phabricator.services.mozilla.com/D113562
This patch does the following:
* General cleanup:
* More explicit restrictions of how/when the various constructors are available.
* `InitializeSecurity` is made `static`, since it doesn't really manipulate instance variables.
* We move some logic for resolving `CoInitializeEx` flags into `GetDesiredApartmentType`.
* Addition of `PostInit`:
* This doesn't do anything at the moment, but since I'm already making a bunch
of changes, I wanted to add this too. `PostInit` is a static method that
is invoked once the `ProcessRuntime` is finished initializing, and, when
present, the sandbox is fully enabled.
* We call `EnsureMTA`'s default constructor to eagerly bring up the MTA. This
causes all background threads to implicitly become members of the MTA, which
means that we can eliminate `CoInitializeEx` calls throughout our codebase,
since they're slow and most developers do not have a clear understanding of
what those functions actually do.
* This also simplifies the COM initialization in sandboxed content processes
with Win32K lockdown, since if our main thread is in the implicit MTA, we
can immediately initialize ourselves without needing to punt that work
over to the persistent MTA thread.
Differential Revision: https://phabricator.services.mozilla.com/D113560
* We make `EnsureMTA`'s default constructor `private`, and `ProcessRuntime` a friend.
`ProcessRuntime` calls this to eagerly create the MTA.
* The default constructor uses the new-ish `CoIncrementMTAUsage` to create the
MTA without requiring a dedicated thread (when available). Otherwise we
fall back to the traditional method. In the latter case, we synchronously
wait for the initialization to complete so that we are guaranteed to have
an MTA when we return.
* Some minor refactoring to make it easier to do the sync wait in the
default constructor. I also renamed a couple of things just to make them
more clear.
Differential Revision: https://phabricator.services.mozilla.com/D113562
This patch does the following:
* General cleanup:
* More explicit restrictions of how/when the various constructors are available.
* `InitializeSecurity` is made `static`, since it doesn't really manipulate instance variables.
* We move some logic for resolving `CoInitializeEx` flags into `GetDesiredApartmentType`.
* Addition of `PostInit`:
* This doesn't do anything at the moment, but since I'm already making a bunch
of changes, I wanted to add this too. `PostInit` is a static method that
is invoked once the `ProcessRuntime` is finished initializing, and, when
present, the sandbox is fully enabled.
* We call `EnsureMTA`'s default constructor to eagerly bring up the MTA. This
causes all background threads to implicitly become members of the MTA, which
means that we can eliminate `CoInitializeEx` calls throughout our codebase,
since they're slow and most developers do not have a clear understanding of
what those functions actually do.
* This also simplifies the COM initialization in sandboxed content processes
with Win32K lockdown, since if our main thread is in the implicit MTA, we
can immediately initialize ourselves without needing to punt that work
over to the persistent MTA thread.
Differential Revision: https://phabricator.services.mozilla.com/D113560
This patch adds a UnstrippedURI into the LoadInfo. This attribute
represents the channel's URI has been stripped if this attributes is not
a nullptr.
Having this attribute allows us to be able to revert the query stripping
in the case where the loading channel is in the content blocking allow
list in the parent process.
In addition, this patch removes the main thread assertion in URIUtils
given that we've made the URL construction thread-safe. This will allow
us to be able to use nsIURI directly in ParentLoadInfoForwarderArgs.
Differential Revision: https://phabricator.services.mozilla.com/D116108
To collect session storage data for session store, we make it possible
to query the background session storage managar for data.
Depends on D111432
Differential Revision: https://phabricator.services.mozilla.com/D111433
To collect session storage data for session store, we make it possible
to query the background session storage managar for data.
Depends on D111432
Differential Revision: https://phabricator.services.mozilla.com/D111433
This is almost certainly a very small optimization, and likely won't address
the frequency of hangs from bug 1677509. However, it still should be an
improvement, and will work on anything Vista or later.
We observed as part of trying to diagnose the somewhat mysterious bug 1677509
that sometimes multiple messages would be queued, and yet the IPC I/O thread
would go idle in between servicing them in GetQueuedCompletionStatusEx. Given
that we send frequent sync ipc messages for mouse move events, it seems prudent
to be able to service multiple at a time.
Differential Revision: https://phabricator.services.mozilla.com/D114287
This change adds the ground work to share content provided by the JS engine of
the Parent process to initialize the JS engine of other threads and Content
processes.
The singleton class xpc::SelfHostedShmem is used to wrap the logic behind
holding the memory. The memory is initialized with `InitFromParent` or
`InitFromChild`. The memory is accessible using either the `Content` or
`Handle`.
The shared memory is transfered through the command line using
`mozilla::ipc::ExportSharedJSInit` and read using
`mozilla::ipc::ImportSharedJSInit` functions. The command line is used, as we
need the shared memory to be avilable for the JS engine initialization. The
command line is composed of a single command named `-jsInit` which is followed
by the handle (on Windows) and the length of the shared content.
The memory associated with the shared memory is cleared in `ShutdownXPCOM` after
closing all threads, and shuting down the JS engine. This is necessary as we
expect the JS engine to borrow content from the shared memory.
Differential Revision: https://phabricator.services.mozilla.com/D110576
We'd like to know if there are any problems with starving content processes
of cleaning up memory in a timely way. Add some telemetry to get a sense of
this.
Differential Revision: https://phabricator.services.mozilla.com/D113275
In order to avoid concurrency issues when dumping or to be able to have a ccov report per process, when GCOV_CHILD_PREFIX is set, set child processes' GCOV_PREFIX to GCOV_CHILD_PREFIX + child process PID
Differential Revision: https://phabricator.services.mozilla.com/D114239
This both helps IDE integration and will help avoid build issues which would've
otherwise cropped up when adding new files to the directory for bug 1706374.
Differential Revision: https://phabricator.services.mozilla.com/D112764
`High` priority is being used for vsync tasks, so we should rename it to
make it clear, and renaming it also makes our priority naming less
confusing.
Differential Revision: https://phabricator.services.mozilla.com/D109536
Eliminates the NPAPI plugin process type from the GeckoChildProcess enum as part of NPAPI removal. In order to avoid altering enum values when updating the process list, the GECKO_PROCESS_TYPE macro has been updated to include the desired enum value. We want to resist altering the values as they need to be consistent e.g. in telemetry reports.
We also remove plugins from adjacent spots that need to maintain consistency with GeckoChildProcess -- most notably the nsICrashService.
Differential Revision: https://phabricator.services.mozilla.com/D108689
Removes Windows NPAPI process sandboxing code, including the code to establish a viable temp directory that was accessible by the sandboxed process.
Differential Revision: https://phabricator.services.mozilla.com/D108688
Removes async (windowless) NPAPI plugin rendering methods used to render a plugin to an offscreen surface in the GPU process. None of this code is used since we have removed all NPAPI plugin support.
Differential Revision: https://phabricator.services.mozilla.com/D107152
This is the first of two patches in this series that removes a large amount of now dead code from dom/plugins as part of removing all NPAPI plugin support. This patch removes re-entrancy guards we have for Windows OnPaint messages, as the guards were only needed for windowed plugins.
Differential Revision: https://phabricator.services.mozilla.com/D107144
Removes the mac plugin_interposer (and the related NSCursorInfo behavior), as part of removing all of NPAPI plugin support, since it has no other clients.
Differential Revision: https://phabricator.services.mozilla.com/D107142
Removes the PPluginSurface actor used for windowed plugins, as part of removing all of NPAPI plugin support. SharedDIB is then unused and is also removed.
Differential Revision: https://phabricator.services.mozilla.com/D107140
`High` priority is being used for vsync tasks, so we should rename it to
make it clear, and renaming it also makes our priority naming less
confusing.
Differential Revision: https://phabricator.services.mozilla.com/D109536
Eliminates the NPAPI plugin process type from the GeckoChildProcess enum as part of NPAPI removal. In order to avoid altering enum values when updating the process list, the GECKO_PROCESS_TYPE macro has been updated to include the desired enum value. We want to resist altering the values as they need to be consistent e.g. in telemetry reports.
We also remove plugins from adjacent spots that need to maintain consistency with GeckoChildProcess -- most notably the nsICrashService.
Differential Revision: https://phabricator.services.mozilla.com/D108689
Removes Windows NPAPI process sandboxing code, including the code to establish a viable temp directory that was accessible by the sandboxed process.
Differential Revision: https://phabricator.services.mozilla.com/D108688
Removes async (windowless) NPAPI plugin rendering methods used to render a plugin to an offscreen surface in the GPU process. None of this code is used since we have removed all NPAPI plugin support.
Differential Revision: https://phabricator.services.mozilla.com/D107152
This is the first of two patches in this series that removes a large amount of now dead code from dom/plugins as part of removing all NPAPI plugin support. This patch removes re-entrancy guards we have for Windows OnPaint messages, as the guards were only needed for windowed plugins.
Differential Revision: https://phabricator.services.mozilla.com/D107144
Removes the mac plugin_interposer (and the related NSCursorInfo behavior), as part of removing all of NPAPI plugin support, since it has no other clients.
Differential Revision: https://phabricator.services.mozilla.com/D107142
Removes the PPluginSurface actor used for windowed plugins, as part of removing all of NPAPI plugin support. SharedDIB is then unused and is also removed.
Differential Revision: https://phabricator.services.mozilla.com/D107140
This patch exposes SharedMemoryBasic::Unmap for use by callers to allow
one to unmap memory out of our address space without destroying the
SharedMemoryBasic object itself. This will be used by later patches in
this series.
Differential Revision: https://phabricator.services.mozilla.com/D109438
This patch exposes SharedMemoryBasic::Unmap for use by callers to allow
one to unmap memory out of our address space without destroying the
SharedMemoryBasic object itself. This will be used by later patches in
this series.
Differential Revision: https://phabricator.services.mozilla.com/D109438
This patch exposes SharedMemoryBasic::Unmap for use by callers to allow
one to unmap memory out of our address space without destroying the
SharedMemoryBasic object itself. This will be used by later patches in
this series.
Differential Revision: https://phabricator.services.mozilla.com/D109438
When setting up calls to `sendmsg` for IPC on Unix systems, we generate
`iovec`s for the entire message or until the `IOV_MAX` limit is reached,
whichever comes first. However, messages can be very large (up to 256
MiB currently), while the OS socket buffer is relatively small (8KiB on
macOS and FreeBSD, ~200KiB on Linux).
This patch detects the socket buffer size with the `SO_SNDBUF` socket
option and cuts off the `iovec` array after it's reached; it also adjusts
the Linux sandbox policy to allow reading that value in all processes.
On my test machines this increases throughput on large messages by about
2.5x on macOS (from ~0.3 to ~0.7 GB/s), but on Linux the improvement is
only about 5% (most of the running time is spent elsewhere).
Differential Revision: https://phabricator.services.mozilla.com/D105852
Currently we walk through the entire list of not-yet-written IPC buffers
when building the gathered I/O list for `sendmsg`, to determine the
total remaining length of the messages, even after reaching the OS's
limit on how many `iovec`s it will accept in one call.
This patch halts the iteration when we reach the `iovec` limit, because
we don't need the exact length; it's sufficient to know whether the
entire message was written, which is impossible in that case.
This increases throughput on large messages by about 7x on macOS (from
~0.04 to ~0.3 GB/s) and 1.7x on Linux (from ~0.3 to ~0.5 GB/s), on my
test machines. The effect is more significant on macOS because its
smaller socket buffer size (8kB vs. ~200kB) means we spend more time
setting up the syscall per unit data copied; see also the next patch.
Differential Revision: https://phabricator.services.mozilla.com/D105851
This patch adds a flag 'needForCheckingAntiTrackingHeuristic' in
loadInfo. This flag will be used for deciding if we need to check the
AntiTracking heuristic after the channel has been classified.
Differential Revision: https://phabricator.services.mozilla.com/D107047
Note that this patch only transforms the use of the nsDataHashtable type alias
to a directly equivalent use of nsTHashMap. It does not change the specification
of the hash key type to make use of the key class deduction that nsTHashMap
allows for in some cases. That can be done in a separate step, but requires more
attention.
Differential Revision: https://phabricator.services.mozilla.com/D106008
PGamepadEventChannelParent will need to send an async message back to the
child with info about the new shared memory region upon creation.
The IPDL channel is not ready during allocation, so we need to defer
registering the actor with GamepadPlatformService until the IPDL
"RecvConstructor" message so the channel will be available.
This is just a straight refactor to prepare for the upcoming change.
Differential Revision: https://phabricator.services.mozilla.com/D105127
Nika Layzell
Mike Hommey
Butkovits Atila
Andi-Bogdan Postelnicu
Florian Quèze
Iulian Moraru
Aaron Klotz
Alexandre Lissy
Gabriele Svelto
Brindusan Cristian
Marian-Vasile Laza
Noemi Erli
Yaron Tausky
Andreas Farre
Rob Lemley
Tim Huang
Olli Pettay
Alexandru Michis
Dorel Luca
Doug Thayer
Paul Bone
Nicolas B. Pierron
Bob Owen
Calixte Denizet
Jed Davis
David Parks
Edgar Chen
Mitchell Hentges
Sean Feng
Christoph Kerschbaumer
Brad Werth
Csoregi Natalia
Ryan VanderMeulen
Eden Chuang
Andrew Osmond
Simon Giesecke
Tom Ritter
Cosmin Sabou
Niklas Gögge
smolnar
Chris Martin
Bogdan Tara