This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
The support code for Android, which doesn't support shm_open and can't
use the memfd backend because of issues with its SELinux policy (see bug
1670277), has been reorganized to reflect that we'll always use its own
API, ashmem, in that case.
Differential Revision: https://phabricator.services.mozilla.com/D90605
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
And we remove unnecessary checks, BackgroundParent only run in the parent process and if e10s is on. Also RecvLauchRDDProcess will only ever be called if the rdd pref is on already.
By streamlining the call we also reduce the number of sync dispatch to 1.
Depends on D93317
Differential Revision: https://phabricator.services.mozilla.com/D93476
Add a synchronous Supports message to the IPDL PRemoteDecoderManager protocol so
decoder modules can query for playback support in the actual process that will
attempt to do the decoding.
Depends on D54878
Differential Revision: https://phabricator.services.mozilla.com/D54879
Aside from its use in AddProfilerMarker(), after initialization mPeerPid
is only used on the IO thread, so the write to it does not hold the monitor.
This means that the read in AddProfilerMarker() can cause a race, even
though we hold the monitor. This method is only called when we hold
the monitor and everything is set up, so I think we can just use
mListener->OtherPid() to get the PID.
Differential Revision: https://phabricator.services.mozilla.com/D93810
The `clobber` targets are superseded by `mach clobber`, so we don't need them for any reason. The `clean` target is meant to get you to a post-`configure` state, but it doesn't really work, and if it's necessary for you to be in that state for some reason you can just clobber and re-`configure`, so it doesn't seem worth it to get it working again. Instead, delete all of them. Also delete `everything` which is not useful when `clobber` doesn't exist.
Differential Revision: https://phabricator.services.mozilla.com/D93514
This patch does:
- Use LSWriteOptimizer
- Remove SessionStorageService since it's unused.
- Move IPC from PContent to PBackground
(by SessionStorageManager{Child, Parent} and SessionStorageCache{Child, Parent}).
- Extract SessionStorageManagerBase and add PBackgroundSessionStorageManager.
- Expose a getter function to get a BackgroundParentManager for top context id
on the parent.
IPC
- Before this patch:
- Copy from parent while loading a document.
- Mark cache entry on the parent process as loaded by the child id.
- Update change on checkpoint.
- Unmark cache entry on the parent process as unloaded for the child id while
the parent actor is destorying.
- After this patch:
- Sync IPC load in the first SessionStorage operation.
- Update change on checkpoint
`BackgroundSessionStorageManager`'s lifecycle on the parent process.
- Create by `SessionStorageManagerParent` and register to the `sManagers`.
- Hold by `SessionStorageManagerParent` and `sManagers`.
- Remove from the `sManagers` while the corresponding `BrowsingContext` is
destructed (on the parent process).
Depends on D89341
Differential Revision: https://phabricator.services.mozilla.com/D89342
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
Differential Revision: https://phabricator.services.mozilla.com/D90605
Each existing GamepadTestChannel needs to know when gamepad monitoring is
started or stopped so it knows whether it's safe to deliver messages.
Differential Revision: https://phabricator.services.mozilla.com/D86747
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
Differential Revision: https://phabricator.services.mozilla.com/D90605
There are at least 8 different methods for getting a range from an offset:
1. left word
2. right word
3. line
4. left line
5. right line
6. sentence
7. paragraph
8. range with same style.
Having a single wrapper and IPDL method for all of those with an enum would remove
a lot of redundancies.
Differential Revision: https://phabricator.services.mozilla.com/D90936
We use C++14's generic lambdas and its auto&& type in the generated code, in combination with a typed local variable to ensure the argument type is enforced.
The object is moved as necessary, no copies will occur.
The code generated will now be:
[this, self__, id__, seqno__](auto&& aParam) {
if ((!(self__))) {
NS_WARNING("Not resolving response because actor is dead.");
return;
}
bool resolve__ = true;
InitResultIPDL result = std::forward<decltype(aParam)>(aParam);
IPC::Message* reply__ = PRemoteDecoder::Reply_Decode(id__);
WriteIPDLParam(reply__, self__, resolve__);
// Sentinel = 'resolve__'
(reply__)->WriteSentinel(322044863);
WriteIPDLParam(reply__, self__, std::move(result));
// Sentinel = 'result'
(reply__)->WriteSentinel(153223840);
(reply__)->set_seqno(seqno__);
}
For multiple arguments return, creation of Tuple via Tie is also moved, though currently Tie method doesn't support move semantics.
Differential Revision: https://phabricator.services.mozilla.com/D90090
In most situations, JSONWriter users already know string lengths (either directly, or through `nsCString` and friends), so we should keep this information through JSONWriter and not recompute it again.
This also allows using JSONWriter with sub-strings (e.g., from a bigger buffer), without having to create null-terminated strings.
Public JSONWriter functions have overloads that accept literal strings.
Differential Revision: https://phabricator.services.mozilla.com/D86192
Some Android ARM64 devices appear to have a bug where sendmsg sometimes
returns 0xFFFFFFFF, which we're assuming is a -1 that was incorrectly
truncated to 32-bit and then zero-extended. This patch detects that
value (which should never legitimately be returned, because it's 16x
the maximum message size) and replaces it with -1, with some additional
assertions.
The workaround is also enabled on x86_64 Android on debug builds only,
so that the code has CI coverage.
Differential Revision: https://phabricator.services.mozilla.com/D89845
Doing this helps lower the chances of accidentally trying to send an
uninitialized primitive value, like a raw pointer or integer, over IPC due to a
sync method or IPDLParamTraits::Read implementation failing to initialize the
outparameter.
Differential Revision: https://phabricator.services.mozilla.com/D89963
With fission enabled, when we are starting a load, we might be saving
principals for a specific browsing context in process A, and then end up
targetting process B for the load, so during deserialization of the
LoadInfoArgs struct, we will end up using principals that were saved during a
previous load targetting that browsing context (with the same id) but in
process B.
Therefore, we cannot assert (without clearing the saved principals in the
original browsing context when a load is done, which can be done as a follow up
work) that the saved principal will be equal to the serialized one from
LoadInfoArgs.
Differential Revision: https://phabricator.services.mozilla.com/D89728
This patch enables sandboxed srcdoc loads to take place via DocumentChannel,
and adds mechanisms for enabling unsandboxed ones.
Both unsandboxed srcdoc, and in subsequent patches, about:blank, loads require
that the triggering principal and the principal to inherit point to the same
instance if the load takes place in the same process as where we are inheriting
those principals from. We save those principals on a target browsing context before
we load the URI, and later, when we are deserializing LoadInfoArgs into
LoadInfo in the content process, we retrieve the saved principals if the
current load identifier of the target BC matches the load identifier saved
along with the principals.
We also need to make sure that during a process switch for about:srcdoc load,
we don't use the original URI for about:srcdoc to determine the remote type and
instead we use channel's result principal.
Differential Revision: https://phabricator.services.mozilla.com/D85079
We need a sync IPC call for this because otherwise the number of smaller sync messages we would need to call would be variable.
Differential Revision: https://phabricator.services.mozilla.com/D88076
CanSend() is called (in ChannelSend()) by SendFoo() IPDL calls to prevent calling Send() on a dead actor but shmem creation uses a different code path to Send() -- one that does not use ChannelSend. This adds the guard to shmem allocation as well.
Differential Revision: https://phabricator.services.mozilla.com/D87357
posix_fallocate iterates over each page/block in a shmem to ensure the
OS allocates memory to back it. Large shmems will cause many read/write
calls to be made, and when profiling, it is very likely a SIGPROF signal
will interrupt us at sufficiently high sampling rates. Most attempts at
retrying will fail for the same reason, and this can cause the threads
to block for an indeterminate period of time.
To work around this we use the profiler's "thread sleep" mechanism to
indicate that the sampler thread should not interrupt this thread with
the sampling signal more than once.
Differential Revision: https://phabricator.services.mozilla.com/D87373
The goal of this patch is to reduce memory usage. On at least OSX, std::Queue
can use 4kb of memory even with nothing in it. This can be around 52kb of
memory per content process.
The segment size of 64 is fairly arbitrary, but these queues didn't have
more than a few hundred items in them, so it seemed like a reasonable
trade off between space for mostly-empty queues and segment overhead.
Differential Revision: https://phabricator.services.mozilla.com/D87325
This changes the duplicate checking/caching implemented by |parsed| to be
based on the name of the protocol etc. we're loading, not the file name.
This lets us detect when a protocol is being defined in two different
files.
This can happen if one file is included earlier in the resolve path than
a file explicitly specified on the command line. Any includes will resolve
to the former, and then we'll attempt to parse the latter. Before this
patch, this would result in weird errors, because there would be multiple
protocol types with the same name.
Differential Revision: https://phabricator.services.mozilla.com/D86113
This moves the IPC mechanism from PCompositorBridge to PLayerTransaction/
PWebRenderBridge, so that it can be used by content processes like the other
test APIs. It still only produces actual data for the layers backend; for
WR it will just return empty datasets.
Differential Revision: https://phabricator.services.mozilla.com/D86016
This patch enables sandboxed srcdoc loads to take place via DocumentChannel,
and adds mechanisms for enabling unsandboxed ones.
Both unsandboxed srcdoc, and in subsequent patches, about:blank, loads require
that the triggering principal and the principal to inherit point to the same
instance if the load takes place in the same process as where we are inheriting
those principals from. We save those principals on a target browsing context before
we load the URI, and later, when we are deserializing LoadInfoArgs into
LoadInfo in the content process, we retrieve the saved principals if the
current load identifier of the target BC matches the load identifier saved
along with the principals.
We also need to make sure that during a process switch for about:srcdoc load,
we don't use the original URI for about:srcdoc to determine the remote type and
instead we use channel's result principal.
Differential Revision: https://phabricator.services.mozilla.com/D85079
Change the GamepadEventChannel so it is fully-initialized by the IPC
constuctor and needs no separate "init" message, and so its completely
destroyed by the ActorDestroy() message so it needs no "cleanup" message.
This simplifies the object lifetime, as well as unifies the IPC error vs
clean shutdown paths.
Differential Revision: https://phabricator.services.mozilla.com/D85481
The main reason of this patch is that the source stream will be closed when `InputStreamHelper::SerializeInputStreamAsPipe` is called, but when `InputStreamHelper::SerializeInputStreamAsPipe` is not called, the stream is stayed opened. I think we should make the behavior of serialization stream consistent, which is keeping the source stream opened.
Differential Revision: https://phabricator.services.mozilla.com/D81978
Change the GamepadEventChannel so it is fully-initialized by the IPC
constuctor and needs no separate "init" message, and so its completely
destroyed by the ActorDestroy() message so it needs no "cleanup" message.
This simplifies the object lifetime, as well as unifies the IPC error vs
clean shutdown paths.
Differential Revision: https://phabricator.services.mozilla.com/D85481
CLOSED TREE
We don't need these macros anymore, for two reasons:
1. We have static analysis to provide the same sort of checks via `MOZ_RAII`
and friends.
2. clang now warns for the "temporary that should have been a declaration" case.
The extra requirements on class construction also show up during debug tests
as performance problems.
This change was automated by using the following sed script:
```
# Remove declarations in classes.
/MOZ_DECL_USE_GUARD_OBJECT_NOTIFIER/d
/MOZ_GUARD_OBJECT_NOTIFIER_INIT/d
# Remove individual macros, carefully.
{
# We don't have to worry about substrings here because the closing
# parenthesis "anchors" the match.
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_TO_PARENT)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_IN_IMPL)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_IN_IMPL)/)/g;
# Remove the longer identifier first.
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_TO_PARENT//g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM//g;
}
# Remove the actual include.
\@# *include "mozilla/GuardObjects.h"@d
```
and running:
```
find . -name \*.cpp -o -name \*.h | grep -v 'GuardObjects.h' |xargs sed -i -f script 2>/dev/null
mach clang-format
```
Differential Revision: https://phabricator.services.mozilla.com/D85168
We don't need these macros anymore, for two reasons:
1. We have static analysis to provide the same sort of checks via `MOZ_RAII`
and friends.
2. clang now warns for the "temporary that should have been a declaration" case.
The extra requirements on class construction also show up during debug tests
as performance problems.
This change was automated by using the following sed script:
```
# Remove declarations in classes.
/MOZ_DECL_USE_GUARD_OBJECT_NOTIFIER/d
/MOZ_GUARD_OBJECT_NOTIFIER_INIT/d
# Remove individual macros, carefully.
{
# We don't have to worry about substrings here because the closing
# parenthesis "anchors" the match.
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_TO_PARENT)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_IN_IMPL)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_IN_IMPL)/)/g;
# Remove the longer identifier first.
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_TO_PARENT//g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM//g;
}
# Remove the actual include.
\@# *include "mozilla/GuardObjects.h"@d
```
and running:
```
find . -name \*.cpp -o -name \*.h | grep -v 'GuardObjects.h' |xargs sed -i -f script 2>/dev/null
mach clang-format
```
Differential Revision: https://phabricator.services.mozilla.com/D85168
Previously, we only did this when IsCallerExternalProcess() returned false.
There are three reasons for changing this:
1. There seem to be cases where IsCallerExternalProcess() returns true even when marshaling for a COM query in the MTA.
2. After bug 1627084, we pre-build a11y handler payloads on the main thread for bulk fetch calls. That will marshal interceptors. However, IsCallerExternalProcess() can't work in that case because it's not running on the thread on which the COM call is being handled.
3. If MSHLFLAGS_NOPING is used, Release calls from remote clients are never sent to the server. So, as soon as we use NOPING for our parent process, we're already going to leak references, even if we don't use NOPING for external callers. Put another way, as soon as we use NOPING for one caller, we may as well use it for all callers because COM pinging will never release the object anyway.
Differential Revision: https://phabricator.services.mozilla.com/D84778
An actual use of the `BACKGROUND_X11` thread may have briefly existed in
2009 but probably never shipped; it's been unused since then, and even
the comments mentioning it have been pruned (bug 909028, bug 624422).
The other thread types besides IO have been commented out ever since
they were first committed.
This patch also removes `x11_util.h`, which has been unused since 2017
(bug 1426284); earlier in history, it had one of those comments
mentioning the nonexistent X11 thread.
Differential Revision: https://phabricator.services.mozilla.com/D85346
It could go into its own test suite, but it 1) depends on `mozbuild` code, so the `mozbuild` suite as well as this new suite would be running on any push that touches `mozbuild` code anyway, and 2) this is code that runs during the build, so it's not out of place.
Differential Revision: https://phabricator.services.mozilla.com/D84547
Having two classes in the inheritance chain inherit from SupportsWeakPtr
now won't compile, but you can use WeakPtr<Derived> when any base class
inherits from SupportsWeakPtr.
Differential Revision: https://phabricator.services.mozilla.com/D83674
The Chromium-derived IPC code was, as far as I can tell, originally
designed for Windows and assumed that channels would be named pipes,
managed and connected to via `std::wstring` paths. The port to Unix,
however, used unnamed `socketpair()` and passed them directly from
process to process, so it has no use for these channel IDs... but it
still computes and propagates them, even though they're not used, using
deprecated wide-string APIs.
This patch introduces a typedef for an abstract channel ID, which is a
`wstring` on Windows and an empty struct on Unix, to allow removing the
string code where it's not needed without needing to completely redesign
the channel abstraction.
Differential Revision: https://phabricator.services.mozilla.com/D72260
Chromium's fix for CVE-2011-3079 added an optional prefix parameter for
channel IDs, but we've never used it and have no plans to. (Chromium
itself doesn't appear to have used it except with the prefixes "gpu"
and "nacl", and the code has since been removed completely in favor of
Mojo.) So let's simplify things and remove it.
Differential Revision: https://phabricator.services.mozilla.com/D84276
This "create a pipe" operation has a mode where, on Unix, it doesn't
create a new transport but rather uses a hard-coded fd for the initial
IPC channel in a child process. (It was originally written for Windows
and the assumption of using named pipes and pathnames for everything.)
That seems like a footgun, so this patch checks for trying to "create"
that pipe twice. However, it doesn't check for accidentally calling it
in the parent process.
Differential Revision: https://phabricator.services.mozilla.com/D72259
The PipeMap class tries to simulate the Windows channel model (named
pipes that the client opens by a pathname) on Unix. However, it's
effectively dead code -- the map is empty except in some unit tests that
we never imported.
What we do is generate a "channel ID" with string formatting, then don't
pass it to the child or ever insert anything into the map, then the child
looks up an empty string and doesn't find it, so it uses the hard-coded
fixed fd for the initial channel.
Basically, it does nothing except maybe confuse unfamiliar readers, so
let's get rid of it.
Differential Revision: https://phabricator.services.mozilla.com/D72258
* Use clearer pref names.
* Default (and only support) IPDL dispatching.
* Make DispatchCommands async-only.
* Sync ipdl command per sync webgl entrypoint.
* Eat the boilerplate cost, since there's not too many.
* Run SerializedSize off same path as Serialize.
* All shmem uploads go through normal DispatchCommands.
* Defer pruning of dead code for now so we can iterate quickly.
* Use Read/Write(begin,end) instead of (begin,size).
* This would have prevented a bug where we read/wrote N*sizeof(T)*sizeof(T).
Differential Revision: https://phabricator.services.mozilla.com/D81495
WebRender makes extensive use of shared memory buffers, particularly for
images decoded in the content process. These images can be arbitrarily
large, and there being insufficient memory for an allocation must be
handled gracefully.
On Linux, we will currently crash with a SIGBUS signal during image
decoding instead of just displaying the broken image tag. This is
because the pages backing the shared memory are only allocated when we
write to them. This blocks shipping WebRender on Linux.
This patch uses posix_fallocate to force the reservation of the pages,
and allows failing gracefully if they are unavailable.
Differential Revision: https://phabricator.services.mozilla.com/D80650
CVE-2018-4435 (https://crbug.com/project-zero/1671) was fixed in macOS
10.12 and up, but when we added uses of shm_open that would be affected
by it we still supported 10.9, so we added a workaround that tests for
the bug (by trying to exploit it) and falls back to the slower
alternative of temporary files if necessary.
The minimum supported version is now 10.12, and we've already
committed changes (e.g., to sandboxing) that would break the browser
on older versions, so we can remove this code. Note that we also have
cross-platform gtests that check for this type of bug, so we'll have some
warning if it's ever reintroduced.
Differential Revision: https://phabricator.services.mozilla.com/D83197
CLOSED TREE
Backed out changeset 51d7c644a1e6 (bug 1650163)
Backed out changeset 3d2b6908447a (bug 1650163)
Backed out changeset 79141707d47b (bug 1650163)
We'll never read more than MAX_DESCRIPTORS_PER_MESSAGE file descriptors in a
single message, so size the buffer based on that value.
Differential Revision: https://phabricator.services.mozilla.com/D79162
NS_INLINE_DECL_REFCOUNTING macro doesn't properly work when the object is used on a thread that isn't backed by a single PRThread (such as TaskQueue). See bug 1648031.
The resolution of this issue is rather complex, and outside the scope of this series of change.
So for now, we create a new macro NS_INLINE_DECL_REFCOUNTING_ONEVENTTHREAD which will use a different mechanism to ensure the thread-safe usage of a class.
Differential Revision: https://phabricator.services.mozilla.com/D81269
The speedup here is not great because the Python/IPDL parser overhead is
very high, but this cuts ~10 seconds off a `-j12` build (~15s -> 4s) on
my machine.
Differential Revision: https://phabricator.services.mozilla.com/D81751
This adds 3 new profiler markers for each IPC message:
* One just before the first byte is sent over the IPC channel
* One just after the last byte is sent over the IPC channel
* One just after the last byte is received from the IPC channel
With the already-existing IPC markers (for when SendXXX and RecvXXX are
called), this allows us to calculate the following statistics:
* Send thread latency
* IPC send duration
* IPC recv latency
* Recv thread latency
For more information on how this is presented in the UI, see:
<https://github.com/firefox-devtools/profiler/pull/2535>.
Differential Revision: https://phabricator.services.mozilla.com/D70790
We want it to returning the actual nsThread if that's where the MessageLoop would dispatch its tasks; otherwise return the MessageLoop's EventTarget
Depends on D80357
Differential Revision: https://phabricator.services.mozilla.com/D80811
NS_INLINE_DECL_REFCOUNTING macro doesn't properly work when the object is used on a thread that isn't backed by a single PRThread (such as TaskQueue). See bug 1648031.
The resolution of this issue is rather complex, and outside the scope of this series of change.
So for now, we create a new macro NS_INLINE_DECL_REFCOUNTING_ONEVENTTHREAD which will use a different mechanism to ensure the thread-safe usage of a class.
Differential Revision: https://phabricator.services.mozilla.com/D81269
We want it to returning the actual nsThread if that's where the MessageLoop would dispatch its tasks; otherwise return the MessageLoop's EventTarget
Depends on D80357
Differential Revision: https://phabricator.services.mozilla.com/D80811
We move OnStartRequest from PHttpChannel to PHttpBackgroundChannel, thus adjusting
message-metadata.ini
Depends on D76970
Differential Revision: https://phabricator.services.mozilla.com/D76971
This moves it near the cross-process `PContent` actor, and makes it more clear
that this actor is only intended to be used for DOM things.
Differential Revision: https://phabricator.services.mozilla.com/D80581
This moves it near the cross-process `PContent` actor, and makes it more clear
that this actor is only intended to be used for DOM things.
Differential Revision: https://phabricator.services.mozilla.com/D80581
We move OnStartRequest from PHttpChannel to PHttpBackgroundChannel, thus adjusting
message-metadata.ini
Depends on D76970
Differential Revision: https://phabricator.services.mozilla.com/D76971
Before P1, GetCurrentThreadSerialEventTarget would have always returned the same data as NS_GetCurrentThread, making the comment incorrect Now it will properly return the running TaskQueue if any.
This change of name more clearly exposes what they are doing, as we aren't always dealing with threads directly; but a nsISerialEventTarget
Differential Revision: https://phabricator.services.mozilla.com/D80354
There's no other change than definition changes. All callers are actually passing nsISerialEventTarget.
Depends on D80423
Differential Revision: https://phabricator.services.mozilla.com/D80424
To intercept COM IPC, we provide an `IChannelHook` interface to
`CoRegisterChannelHook`, which gives us notifications about COM IPC that we can
use to insert profiler markers. Note that `IChannelHook` is not documented on
MSDN, however it is defined in the SDK header files.
When the profiler is available, once XPCOM is up:
* If the profiler is active, we immediately register the channel hook;
* Otherwise we register an observer and hold off on registering the hook until
the profiler is started, at which point we register the hook and remove the
observer.
Differential Revision: https://phabricator.services.mozilla.com/D80053
To intercept COM IPC, we provide an `IChannelHook` interface to
`CoRegisterChannelHook`, which gives us notifications about COM IPC that we can
use to insert profiler markers. Note that `IChannelHook` is not documented on
MSDN, however it is defined in the SDK header files.
When the profiler is available, once XPCOM is up:
* If the profiler is active, we immediately register the channel hook;
* Otherwise we register an observer and hold off on registering the hook until
the profiler is started, at which point we register the hook and remove the
observer.
Differential Revision: https://phabricator.services.mozilla.com/D80053
The change to MessageChannel::Clear() makes mLink get cleared before
we call ~ThreadLink. This causes a race because Clear() is not
holding the monitor. To work around this, I introduced a new method
PrepareToDestroy() that handles the ThreadLink splitting. Once the
ThreadLinks are split, MessageChannel can clear mLink without a
race.
An alternative approach would be to hold the monitor in Clear()
before mLink is cleared, but then we'd end up acquiring the lock
when we didn't need to in the case where mLink is a ProcessLink.
Differential Revision: https://phabricator.services.mozilla.com/D79185
Keeping a list of ancestor principals in a LoadInfo object, that, at times,
exists in the content process, is not secure. Since ancestor principals are
only ever needed to create a list of frameAncestors, which, in turn, are only
ever accessed from the parent process, we can assemble lists of ancestor
principals and outer windowIDs whenever we are in the parent process and are
either 1) creating a LoadInfo object or 2) deserializing a LoadInfoArgs struct,
received from content process, into a LoadInfo object.
Differential Revision: https://phabricator.services.mozilla.com/D78406
The reply argument that gets passed in is a stack reference which is move
assigned into, so it doesn't make sense as a unique pointer, although the
code could be restructured to return a freshly allocated object instead.
This mostly just eliminates a spurious round trip from UniquePtr to *
and back. The bulk of the patch is renaming uses of |msg| to |aMsg|.
Differential Revision: https://phabricator.services.mozilla.com/D77908
Gcc and Clang dumps gcda files just before an exec** or fork functions.
With ccov enabled, we can dump using a SIGUSR1 but if we're in the middle of dump (because of exec** or fork)
then a gcda file can stay locked and then another process can try to get a lock on it for ever.
So to avoid such a situation, we remove the SIGUSR1 handler just before the fork an set it back just after.
Differential Revision: https://phabricator.services.mozilla.com/D78051
This is more standard, and uses about 4kb less memory when almost empty,
which seems to be the common case in an idle content process. This should save
around 66kb per content process.
The next patch will get ride of this thin wrapper and use nsDataHashtable
directly.
Differential Revision: https://phabricator.services.mozilla.com/D76985
This method is the same as Put(), except that it asserts that the item
is not already present. It also puts the key second. Make it compatible
by hoisting out the assert and reversing the arguments. We can use the
definition of Put() defined in an earlier patch.
Differential Revision: https://phabricator.services.mozilla.com/D77167
nsTHashtable::Remove doesn't assert if the item isn't present. Match that
behavior by removing the assert and putting it at all of the call sites.
This just turns IDMap::Remove into RemoveIfPresent, so merge them.
Differential Revision: https://phabricator.services.mozilla.com/D77166
This function is similar to the Put() method in nsTHashtable, but it lists the
key second and it asserts that the key is not already in the map. This patch
swaps the arguments and hoists the assertion out, where appropriate. Note that
there are a few places that were working around this assert, so for those places
don't include the assert.
Differential Revision: https://phabricator.services.mozilla.com/D77165
This is more standard, and uses about 4kb less memory when almost empty,
which seems to be the common case in an idle content process. This should save
around 66kb per content process.
The next patch will get ride of this thin wrapper and use nsDataHashtable
directly.
Differential Revision: https://phabricator.services.mozilla.com/D76985
This method is the same as Put(), except that it asserts that the item
is not already present. It also puts the key second. Make it compatible
by hoisting out the assert and reversing the arguments. We can use the
definition of Put() defined in an earlier patch.
Differential Revision: https://phabricator.services.mozilla.com/D77167
nsTHashtable::Remove doesn't assert if the item isn't present. Match that
behavior by removing the assert and putting it at all of the call sites.
This just turns IDMap::Remove into RemoveIfPresent, so merge them.
Differential Revision: https://phabricator.services.mozilla.com/D77166
This function is similar to the Put() method in nsTHashtable, but it lists the
key second and it asserts that the key is not already in the map. This patch
swaps the arguments and hoists the assertion out, where appropriate. Note that
there are a few places that were working around this assert, so for those places
don't include the assert.
Differential Revision: https://phabricator.services.mozilla.com/D77165
GetConstructedEventTarget and GetSpecificMessageEventTarget always return
null. The idea was that subclasses could override them, but nobody does any
more.
Differential Revision: https://phabricator.services.mozilla.com/D76984
We are seeing crashes on aarch64 Fenix devices that appear to be related
to zero-sized messages. But we're seeing the crashes when we're trying
to send the messages on the IO thread, and not where we're dispatching
them from. Add some asserts so we get errors closer to the source, and
add some asserts for other things that we believe to be true and would
be useful to know aren't actually true.
Differential Revision: https://phabricator.services.mozilla.com/D76496
I don't think all this complexity is worth it for having a
marginally-more-realistic testing story. Using the pref just works and we should
do that, I think.
Differential Revision: https://phabricator.services.mozilla.com/D59980
This method always copies, and is redundant with the other getter methods on
IPDL unions. As there is only one caller, it can be removed to simplify the
code, and remove a source of complexity.
Differential Revision: https://phabricator.services.mozilla.com/D75349
Currently, the worker's COEP is saved in WorkerPrivate and not be respected when loading resources in workers.
This patch adds an attribute loadingEmbedderPolicy in nsILoadInfo, which indicates the COEP header the loading must be respected.
The default value of loadingEmbedderPolicy is nsILoadInfo::EMBEDDER_POLICY_NULL.
loadingEmbedderPolicy is initialized with the COEP of the BrowsingContext used for creating LoadInfo.
And it could be set to other value when fetch in workers.
Differential Revision: https://phabricator.services.mozilla.com/D73690
In this bug we're moving away from monolithic JNI headers to class-specific
headers so that we don't have to rebuild the world every time we make a change
to a JNI interface.
Differential Revision: https://phabricator.services.mozilla.com/D75371
We add a 'IsThirdPartyContextToTopWindow' flag in the LoadInfo. This
flag shows if the channel is considered as a third party related to the
top-level window.
This flag would be set when opening the channel in the parent process.
Differential Revision: https://phabricator.services.mozilla.com/D73199
Fix intermittent issues due to races.
We now run the MozPromise generated by the IPDL bindings to run their callbacks via a direct task dispatch.
This avoids a full trip to the back of the event queue for each additional asynchronous step when using MozPromise.
A consequence to this change is that each IPDL actor's thread must have an AbstractThread allocated if IPDL MozPromises are used.
It prevents unexpected racy behaviours when combining MozPromise with the other Resolve/Reject IPDL async declaration which was have lead to processing the events out of order.
Differential Revision: https://phabricator.services.mozilla.com/D71593
The hashes of certain strings is used indirectly in generating IDL C++ files. Before, we were using the `hash()` function to generate these hashes, which in Python 3 is non-deterministic over subsequent `python3` processes, causing bugs like bug 1635755. Instead, use a specific, deterministic hash to avoid spurious diff failures.
The `md5` hash function isn't completely cryptographically secure but the security of the hashes isn't important for us in this case since we're just using them as a per-string identifier. We could use a more robust hash function but there may be performance implications from doing so.
Differential Revision: https://phabricator.services.mozilla.com/D74446
`ply`, [by design](https://github.com/dabeaz/ply/issues/79), does not produce reproducible table files; hence bug 1633156. (Note that this was *always* true, but only became a problem once we switched to Python 3, which has more unpredictable dict iteration order than Python 2.7, at least prior to [3.7](https://docs.python.org/3/whatsnew/3.7.html#summary-release-highlights).)
In any other circumstance I would consider submitting a patch to `ply` to fix this, but as of the [in-progress version 4.0 of the library](https://github.com/dabeaz/ply/blob/master/CHANGES), it doesn't even emit this cached data any more, and indeed the [latest version of the code](1fac9fed64/ply) doesn't even call `open()` at all except to do logging or to read the text data to be parsed from `stdin`. So if we were going to pin our future on `ply` and upgrade to later versions of the library in the future, we would have to live in a world where `ply` doesn't generate cached table files for us anyway.
Emitting the cached table files so later build steps can consume them is an "optimization", but it's not clear exactly how much actual value that optimization provides overall. Quoth the `CHANGES` file from that repository:
```
PLY no longer writes cached table files. Honestly, the use of
the cached files made more sense when I was developing PLY on
my 200Mhz PC in 2001. It's not as much as an issue now. For small
to medium sized grammars, PLY should be almost instantaneous.
```
In practice, I have found this to be true; namely, `./mach build pre-export export` takes just about as long on my machine after this patch as it did before, and in a try push I performed, there's no noticeable performance regression from applying this patch. In local testing I also found that generating the LALR tables in calls to `yacc()` takes about 0.01s on my machine generally, and we generate these tables a couple dozen times total over the course of the `export` tier now. This isn't *nothing*, but in my opinion it's also not nearly long enough where it would be a concern given how long `export` already takes.
That `CHANGES` file also stresses that if caching this data is important, we have the option of doing so via `pickle`. If and when we decide that re-enabling this optimization is valuable for us, we should take control of this process and perform the generation in such a way that we can guarantee reproducibility.
Differential Revision: https://phabricator.services.mozilla.com/D73484
Currently, there is an outstanding issue where enabling the GPU sandbox breaks
scrolling using the the mouse wheel on laptops with Intel GPUs.
This will enable the GPU sandbox on Nightly for non-Intel GPUs to prevent any
sandbox regressions while we try and figure out what the scrolling issue is.
See Bug 1630860 for more info
Differential Revision: https://phabricator.services.mozilla.com/D73923
Adds IpdlQueue capability to PWebGL actors. The WebGLChild, used in content processes, implements SyncProducerActor and AsyncConsumerActor because it sends (sync and async) messages and receives responses to them that it reads as async messages. The WebGLParent, used in the compositor process, is a SyncConsumerActor and AsyncProducerActor for dual reasons.
Differential Revision: https://phabricator.services.mozilla.com/D68264
We need to separate WebGL actor construction and initialization since IpdlQueue initialization needs the actor to already exist.
Differential Revision: https://phabricator.services.mozilla.com/D68262
When the browser process starts a sandbox process, we copy the executable's IAT
for ntdll.dll into the new process to prevent DLL injection via IAT tampering as
the launcher process does. However, if IAT has been modified by a module injected
via `SetWindowHookEx`, the browser process cannot copy IAT because a modified IAT
is invalid in a different process, failing to start any sandbox processes.
The proposed fix is to cache IAT before COM initialization which may load
modules via `SetWindowHookEx` for the first time in the process.
Differential Revision: https://phabricator.services.mozilla.com/D73303
Adds IpdlQueue capability to PWebGL actors. The WebGLChild, used in content processes, implements SyncProducerActor and AsyncConsumerActor because it sends (sync and async) messages and receives responses to them that it reads as async messages. The WebGLParent, used in the compositor process, is a SyncConsumerActor and AsyncProducerActor for dual reasons.
Differential Revision: https://phabricator.services.mozilla.com/D68264
We need to separate WebGL actor construction and initialization since IpdlQueue initialization needs the actor to already exist.
Differential Revision: https://phabricator.services.mozilla.com/D68262
Both `nsAppShell::ProcessNextNativeEvent()` and `MessagePumpForUI::WaitForWork()` have a `PROFILER_AUTO_THREAD_SLEEP` surrounding the `mozilla::widget::WinUtils::WaitForMessage()` call.
However inside `WaitForMessage()` the call to `PeekMessageW()` may trigger a sequence of events (because the system delivers pending messages) that end in the initialization of a new thread, which invokes `ReentrantMonitor::Wait()` where there is a `PROFILER_AUTO_THREAD_SLEEP`.
To avoid this recursion, this patch moves `PROFILER_AUTO_THREAD_SLEEP` from both callers into `WaitForMessage()` to only enclose the actual potentially-sleeping operation `::MsgWaitForMultipleObjectsEx()`.
Differential Revision: https://phabricator.services.mozilla.com/D72850
When an Interceptor is marshaled for an external (non-chrome) process caller, we do not provide a handler and thus don't call HandlerProvider::WriteHandlerPayload.
However, GetMarshalSizeMax previously called HandlerProvider::GetPayloadSize even for external process callers.
For a11y's handlerProvider, we must build the payload to get the size.
This is wasteful in this case, since we're just going to throw it away.
Differential Revision: https://phabricator.services.mozilla.com/D72796
Because MainThreadHandoff sits between the Interceptor and the HandlerProvider, the caller must:
1. Get the event sink (the IInterceptorSink) from the Interceptor using IInterceptor::GetEventSink.
2. QI to the new IMainThreadHandoff interface. (An IInterceptorSink might not necessarily be a MainThreadHandoff.)
3. Get the HandlerProvider from the MainThreadHandoff using IMainThreadHandoff::GetHandlerProvider.
Differential Revision: https://phabricator.services.mozilla.com/D69484
This commit:
- removes sendRpcMessage, which was unused;
- removes the CPOW argument to sendAsyncMessage, broadcastAsyncMessage, and
sendSyncMessage;
- removes the aIsSync argument used internally to distinguish sendRpcMessage
and sendSyncMessage;
- removes CPOW tests;
- updates the few remaining callsites that use more than 2 arguments in
sendAsyncMessage for the removal of the cpows argument.
Differential Revision: https://phabricator.services.mozilla.com/D71514
Before 1496578, URIFixup::keywordToURI used to do a synchronous IPC call to be
able to access search engines from the content process. Consumers of URIFixup
didn't care. Bug 1496578 moved the IPC messaging to the callers, in particular
nsDocShell, but assumed nsDocShellLoadState wasn't loading from content.
It looks like in some cases it does, so this adds another sync IPC call for
GetFixupURIInfo.
The total numer of sync IPCs should not change from before Bug 1496578, URIFIxup
was just doing it internally, while now it happens at the call point.
Note the long term plan would be for these docshell objects callers to just
handle URIs, while the UI code should do fixup.
Bug 1375244 tracks the removal of these sync IPC messages.
Differential Revision: https://phabricator.services.mozilla.com/D70607
--HG--
extra : moz-landing-system : lando
It seems that all the warnings caused by the GPU sandbox have been fixed, and
the transparent window issue was resolved in D61370.
Hopefully there are no further complications and this can stay landed.
Differential Revision: https://phabricator.services.mozilla.com/D18876
--HG--
extra : moz-landing-system : lando
Dorel Luca
Jed Davis
Ricky Stewart
Jean-Yves Avenard
Dan Glastonbury
Andrew McCreight
Chris Martin
Tom Tung
Cameron McCormack
Bas Schouten
Razvan Maries
sotaro
Dan Minor
Eitan Isaacson
Kris Maglione
Simon Giesecke
Csoregi Natalia
shawnjohnjr
Gerald Squelart
Nika Layzell
Mark Banner
Brindusan Cristian
Anny Gakhokidze
Masatoshi Kimura
Luca Greco
David Parks
Andrew Osmond
Butkovits Atila
Kartikaya Gupta
Noemi Erli
Kershaw Chang
Sylvestre Ledru
Nathan Froyd
Mihai Alexandru Michis
James Teh
Sebastian Streich
Jon Bauman
Emilio Cobos Álvarez
Jeff Gilbert
David Teller
Cosmin Sabou
Narcis Beleuzu
Agi Sferro
Tom Prince
ssengupta
Kashav Madan
Jim Porter
Junior Hsu
Andrea Marchesini
Daisuke Akatsuka
Aaron Klotz
Honza Bambas
Stephen A Pohl
Ryan VanderMeulen
Nico Grunbaum
Calixte Denizet
Gian-Carlo Pascutto
Gijs Kruitbosch
Bob Owen
Chris H-C
Peter Van der Beken
Eden Chuang
nicolaspacheco
Bogdan Tara
Tim Huang
Chris Fronk
Toshihito Kikuchi
Daniel Varga
shravanrn@gmail.com
Makoto Kato
Chris Peterson
Marco Bonardo
Christoph Kerschbaumer