Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
The MOZ_MUST_USE macro is defined as clang's and gcc's nonstandard __attribute__((warn_unused_result)). Now that we compile as C++17 by default (bug 1560664), we can replace MOZ_MUST_USE with C++17's standard [[nodiscard]] attribute.
Differential Revision: https://phabricator.services.mozilla.com/D89237
We avoid going through Substring which imposes a limit on the string length of
INT32_MAX and failing a MOZ_RELEASE_ASSERT otherwise.
This adds a new `Base64EncodeAppend` function that takes 8-bit char input and
produces 16-bit char output, and appends to an existing string, which is what
we need here to avoid doing a large copy. Base64EncodeHelper is generalized
to accomodate for that.
So in addition to fixing the defect, this also optimizes performance and
memory usage.
Differential Revision: https://phabricator.services.mozilla.com/D87413
Previously similar logic existed in BlobURLProtocolHandler, which has now been removed, since such checks are now for parent process only and should be abstracted from BlobURLProtocolHandler.
Depends on D75293
Differential Revision: https://phabricator.services.mozilla.com/D81126
BlobURLProtocolHandler no longer passes blobImpl pointer to BlobURLChannel. Instead, acquisition and handling of blob data is offloaded to BlobURLInputStream.
Differential Revision: https://phabricator.services.mozilla.com/D75293
Reading from this stream type should be the preferred way to consume data from blob url. When created in the parent process, it requests blob data locally from BlobURLProtocolHandler. When created in a content process, it makes BlobURLDataRequest IPC call to the parent process. Should be wrapped in a nsBufferedInputStream, because ReadSegments() is currently not implemented for this stream type.
Differential Revision: https://phabricator.services.mozilla.com/D75292
Previously similar logic existed in BlobURLProtocolHandler, which has now been removed, since such checks are now for parent process only and should be abstracted from BlobURLProtocolHandler.
Depends on D75293
Differential Revision: https://phabricator.services.mozilla.com/D81126
BlobURLProtocolHandler no longer passes blobImpl pointer to BlobURLChannel. Instead, acquisition and handling of blob data is offloaded to BlobURLInputStream.
Differential Revision: https://phabricator.services.mozilla.com/D75293
Reading from this stream type should be the preferred way to consume data from blob url. When created in the parent process, it requests blob data locally from BlobURLProtocolHandler. When created in a content process, it makes BlobURLDataRequest IPC call to the parent process. Should be wrapped in a nsBufferedInputStream, because ReadSegments() is currently not implemented for this stream type.
Differential Revision: https://phabricator.services.mozilla.com/D75292
Before P1, GetCurrentThreadSerialEventTarget would have always returned the same data as NS_GetCurrentThread, making the comment incorrect Now it will properly return the running TaskQueue if any.
This change of name more clearly exposes what they are doing, as we aren't always dealing with threads directly; but a nsISerialEventTarget
Differential Revision: https://phabricator.services.mozilla.com/D80354
Initially, IPCInternal{Request,Response} had contained IPCStreams which would
result in unecessary copying when sending the objects over IPC. The patch
makes these streams either:
1) ParentToParentStream (just a UUID)
2) ParentToChildStream (a PIPCBlobInputStream actor, acting as a handle)
3) ChildToParentStream (a real IPCStream)
These three types are union-ed together by the BodyStreamVariant IPDL structure.
This structure replaces the IPCStream members in IPCInternal{Request,Response}
so that, depending on the particular IPDL protocol, we can avoid cloning streams
and just pass handles/IDs instead.
As a side effect, this makes file-backed Response objects cloneable. Initially,
these Responses would be backed by an nsFileInputStream, which is not cloneable
outside the parent process. They are now backed by IPCBlobInputStreams, which
are cloneable.
One thing that's not really satisfactory (IMO), is the manual management of
IPCBlobInputStreamStorage so that no streams are leaked, e.g. if we store a
stream in the IPCBlobInputStreamStorage but fail to send an IPC message and
therefore fail to remove the stream from storage on the other side of the IPC
boundary (only parent-to-parent in this case).
Differential Revision: https://phabricator.services.mozilla.com/D73173
Initially, IPCInternal{Request,Response} had contained IPCStreams which would
result in unecessary copying when sending the objects over IPC. The patch
makes these streams either:
1) ParentToParentStream (just a UUID)
2) ParentToChildStream (a PIPCBlobInputStream actor, acting as a handle)
3) ChildToParentStream (a real IPCStream)
These three types are union-ed together by the BodyStreamVariant IPDL structure.
This structure replaces the IPCStream members in IPCInternal{Request,Response}
so that, depending on the particular IPDL protocol, we can avoid cloning streams
and just pass handles/IDs instead.
As a side effect, this makes file-backed Response objects cloneable. Initially,
these Responses would be backed by an nsFileInputStream, which is not cloneable
outside the parent process. They are now backed by IPCBlobInputStreams, which
are cloneable.
One thing that's not really satisfactory (IMO), is the manual management of
IPCBlobInputStreamStorage so that no streams are leaked, e.g. if we store a
stream in the IPCBlobInputStreamStorage but fail to send an IPC message and
therefore fail to remove the stream from storage on the other side of the IPC
boundary (only parent-to-parent in this case).
Differential Revision: https://phabricator.services.mozilla.com/D73173
Initially, IPCInternal{Request,Response} had contained IPCStreams which would
result in unecessary copying when sending the objects over IPC. The patch
makes these streams either:
1) ParentToParentStream (just a UUID)
2) ParentToChildStream (a PIPCBlobInputStream actor, acting as a handle)
3) ChildToParentStream (a real IPCStream)
These three types are union-ed together by the BodyStreamVariant IPDL structure.
This structure replaces the IPCStream members in IPCInternal{Request,Response}
so that, depending on the particular IPDL protocol, we can avoid cloning streams
and just pass handles/IDs instead.
As a side effect, this makes file-backed Response objects cloneable. Initially,
these Responses would be backed by an nsFileInputStream, which is not cloneable
outside the parent process. They are now backed by IPCBlobInputStreams, which
are cloneable.
One thing that's not really satisfactory (IMO), is the manual management of
IPCBlobInputStreamStorage so that no streams are leaked, e.g. if we store a
stream in the IPCBlobInputStreamStorage but fail to send an IPC message and
therefore fail to remove the stream from storage on the other side of the IPC
boundary (only parent-to-parent in this case).
Differential Revision: https://phabricator.services.mozilla.com/D73173
These tests used <iframe mozbrowser> for convenience, mostly forcing
themselves to only run in non-e10s mode in the process, but none of them
really have any need to.
Differential Revision: https://phabricator.services.mozilla.com/D70745
These tests used <iframe mozbrowser> for convenience, mostly forcing
themselves to only run in non-e10s mode in the process, but none of them
really have any need to.
Differential Revision: https://phabricator.services.mozilla.com/D70745
These tests used <iframe mozbrowser> for convenience, mostly forcing
themselves to only run in non-e10s mode in the process, but none of them
really have any need to.
Differential Revision: https://phabricator.services.mozilla.com/D70745
For the usecases in dom/file, I'm not so sure how to get the global at
callsites, so I let aIsSystemPrincipal and aCrossOriginIsolated to be false for
now.
Differential Revision: https://phabricator.services.mozilla.com/D63905
--HG--
extra : moz-landing-system : lando
Mostly a matter of:
rg -l '\->LoadingPrincipal' | xargs sed -i 's/->LoadingPrincipal/->GetLoadingPrincipal/g'
And then clang-format. But I tweaked manually nsHttpChannelAuthProvider (move
the variable where it's used, don't take a useless strong ref),
AddonContentPolicy (move the declaration of the variable to the if condition),
and BackgroundUtils (same).
Differential Revision: https://phabricator.services.mozilla.com/D69828
--HG--
extra : moz-landing-system : lando
Mostly a matter of:
rg -l '\->LoadingPrincipal' | xargs sed -i 's/->LoadingPrincipal/->GetLoadingPrincipal/g'
And then clang-format. But I tweaked manually nsHttpChannelAuthProvider (move
the variable where it's used, don't take a useless strong ref),
AddonContentPolicy (move the declaration of the variable to the if condition),
and BackgroundUtils (same).
Differential Revision: https://phabricator.services.mozilla.com/D69828
--HG--
extra : moz-landing-system : lando
Missing includes, namespace-related issues, and missing template instantiations added to make sources compile without unifying.
Differential Revision: https://phabricator.services.mozilla.com/D68142
--HG--
extra : moz-landing-system : lando
This covers most cycle collected objects which support weak references, but
not the ones which inherit from a cycle collected class and don't do any cycle
collection on their own.
Differential Revision: https://phabricator.services.mozilla.com/D63962
--HG--
extra : moz-landing-system : lando
The default method implementations cause problems when trying to
override them with different types in a direct call class.
For the `Recv__delete__` case there's a simple solution: omit it if
there are any arguments, because it doesn't make much sense to specify
arguments and then completely ignore them, and the no-arg case isn't a
problem for overriding.
Differential Revision: https://phabricator.services.mozilla.com/D62977
--HG--
extra : moz-landing-system : lando
This was done by:
This was done by applying:
```
diff --git a/python/mozbuild/mozbuild/code-analysis/mach_commands.py b/python/mozbuild/mozbuild/code-analysis/mach_commands.py
index 789affde7bbf..fe33c4c7d4d1 100644
--- a/python/mozbuild/mozbuild/code-analysis/mach_commands.py
+++ b/python/mozbuild/mozbuild/code-analysis/mach_commands.py
@@ -2007,7 +2007,7 @@ class StaticAnalysis(MachCommandBase):
from subprocess import Popen, PIPE, check_output, CalledProcessError
diff_process = Popen(self._get_clang_format_diff_command(commit), stdout=PIPE)
- args = [sys.executable, clang_format_diff, "-p1", "-binary=%s" % clang_format]
+ args = [sys.executable, clang_format_diff, "-p1", "-binary=%s" % clang_format, '-sort-includes']
if not output_file:
args.append("-i")
```
Then running `./mach clang-format -c <commit-hash>`
Then undoing that patch.
Then running check_spidermonkey_style.py --fixup
Then running `./mach clang-format`
I had to fix four things:
* I needed to move <utility> back down in GuardObjects.h because I was hitting
obscure problems with our system include wrappers like this:
0:03.94 /usr/include/stdlib.h:550:14: error: exception specification in declaration does not match previous declaration
0:03.94 extern void *realloc (void *__ptr, size_t __size)
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/malloc_decls.h:53:1: note: previous declaration is here
0:03.94 MALLOC_DECL(realloc, void*, void*, size_t)
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/mozilla/mozalloc.h:22:32: note: expanded from macro 'MALLOC_DECL'
0:03.94 MOZ_MEMORY_API return_type name##_impl(__VA_ARGS__);
0:03.94 ^
0:03.94 <scratch space>:178:1: note: expanded from here
0:03.94 realloc_impl
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/mozmemory_wrap.h:142:41: note: expanded from macro 'realloc_impl'
0:03.94 #define realloc_impl mozmem_malloc_impl(realloc)
Which I really didn't feel like digging into.
* I had to restore the order of TrustOverrideUtils.h and related files in nss
because the .inc files depend on TrustOverrideUtils.h being included earlier.
* I had to add a missing include to RollingNumber.h
* Also had to partially restore include order in JsepSessionImpl.cpp to avoid
some -WError issues due to some static inline functions being defined in a
header but not used in the rest of the compilation unit.
Differential Revision: https://phabricator.services.mozilla.com/D60327
--HG--
extra : moz-landing-system : lando
rg -l 'mozilla/Move.h' | xargs sed -i 's/#include "mozilla\/Move.h"/#include <utility>/g'
Further manual fixups and cleanups to the include order incoming.
Differential Revision: https://phabricator.services.mozilla.com/D60323
--HG--
extra : moz-landing-system : lando
nsIMutable is used only by DOM Blob/File. But Blobs are immutable by spec.
FileBlobImpl has a couple of lazy member values, but those are called when
the object is cloned in order to be sent to a different thread/process.
Differential Revision: https://phabricator.services.mozilla.com/D58946
--HG--
extra : moz-landing-system : lando
nsIMutable is used only by DOM Blob/File. But Blobs are immutable by spec.
FileBlobImpl has a couple of lazy member values, but those are called when
the object is cloned in order to be sent to a different thread/process.
Differential Revision: https://phabricator.services.mozilla.com/D58716
--HG--
extra : moz-landing-system : lando
Using JSON.stringify in dom/tests/mochitest/chrome/test_clonewrapper.xhtml might not be the best solution, because
the testObject contains ImageData objects.
Differential Revision: https://phabricator.services.mozilla.com/D58645
--HG--
extra : moz-landing-system : lando
This is generally pretty straightforward, and rewrites nearly all calls. It
skips the ones that it can detect using frame script globals like
`sendAsyncMessage`, though.
Differential Revision: https://phabricator.services.mozilla.com/D53740
--HG--
extra : moz-landing-system : lando
This is generally pretty straightforward, and rewrites nearly all calls. It
skips the ones that it can detect using frame script globals like
`sendAsyncMessage`, though.
Differential Revision: https://phabricator.services.mozilla.com/D53740
--HG--
extra : moz-landing-system : lando
This intended to fix some windows builds, but that didn't end up working.
This removes some unused members and such, and fixes some missing includes
that they uncover (whoops).
This was needed because some windows headers used in the sandbox redefine STRICT
(which is used by `StyleContain`) and `TRANSPARENT`, which is used by some WR
stuff.
Differential Revision: https://phabricator.services.mozilla.com/D56317
--HG--
extra : moz-landing-system : lando
This is generally pretty straightforward, and rewrites nearly all calls. It
skips the ones that it can detect using frame script globals like
`sendAsyncMessage`, though.
Differential Revision: https://phabricator.services.mozilla.com/D53740
--HG--
extra : moz-landing-system : lando
This is generally pretty straightforward, and rewrites nearly all calls. It
skips the ones that it can detect using frame script globals like
`sendAsyncMessage`, though.
Differential Revision: https://phabricator.services.mozilla.com/D53740
--HG--
extra : moz-landing-system : lando
The inclusions were removed with the following very crude script and the
resulting breakage was fixed up by hand. The manual fixups did either
revert the changes done by the script, replace a generic header with a more
specific one or replace a header with a forward declaration.
find . -name "*.idl" | grep -v web-platform | grep -v third_party | while read path; do
interfaces=$(grep "^\(class\|interface\).*:.*" "$path" | cut -d' ' -f2)
if [ -n "$interfaces" ]; then
if [[ "$interfaces" == *$'\n'* ]]; then
regexp="\("
for i in $interfaces; do regexp="$regexp$i\|"; done
regexp="${regexp%%\\\|}\)"
else
regexp="$interfaces"
fi
interface=$(basename "$path")
rg -l "#include.*${interface%%.idl}.h" . | while read path2; do
hits=$(grep -v "#include.*${interface%%.idl}.h" "$path2" | grep -c "$regexp" )
if [ $hits -eq 0 ]; then
echo "Removing ${interface} from ${path2}"
grep -v "#include.*${interface%%.idl}.h" "$path2" > "$path2".tmp
mv -f "$path2".tmp "$path2"
fi
done
fi
done
Differential Revision: https://phabricator.services.mozilla.com/D55442
--HG--
extra : moz-landing-system : lando
The inclusions were removed with the following very crude script and the
resulting breakage was fixed up by hand. The manual fixups did either
revert the changes done by the script, replace a generic header with a more
specific one or replace a header with a forward declaration.
find . -name "*.idl" | grep -v web-platform | grep -v third_party | while read path; do
interfaces=$(grep "^\(class\|interface\).*:.*" "$path" | cut -d' ' -f2)
if [ -n "$interfaces" ]; then
if [[ "$interfaces" == *$'\n'* ]]; then
regexp="\("
for i in $interfaces; do regexp="$regexp$i\|"; done
regexp="${regexp%%\\\|}\)"
else
regexp="$interfaces"
fi
interface=$(basename "$path")
rg -l "#include.*${interface%%.idl}.h" . | while read path2; do
hits=$(grep -v "#include.*${interface%%.idl}.h" "$path2" | grep -c "$regexp" )
if [ $hits -eq 0 ]; then
echo "Removing ${interface} from ${path2}"
grep -v "#include.*${interface%%.idl}.h" "$path2" > "$path2".tmp
mv -f "$path2".tmp "$path2"
fi
done
fi
done
Differential Revision: https://phabricator.services.mozilla.com/D55442
--HG--
extra : moz-landing-system : lando
And clean-up surrounding code by removing useless namespace qualification.
It seems like the first call to Broadcast is useless? Or am I missing something?
Differential Revision: https://phabricator.services.mozilla.com/D55000
--HG--
extra : moz-landing-system : lando
When ReadSegments is called before the stream is ready, we return NS_BASE_STREAM_WOULD_BLOCK, not NS_ERROR_NOT_IMPLEMENTED.
If callers use AsyncWait for the IPCBlobInputStream to be ready, and then call ReadSegments again we can end up returning NS_ERROR_NOT_IMPLEMENTED, depending on what the inner type is.
Differential Revision: https://phabricator.services.mozilla.com/D51781
--HG--
extra : moz-landing-system : lando
When ReadSegments is called before the stream is ready, we return NS_BASE_STREAM_WOULD_BLOCK, not NS_ERROR_NOT_IMPLEMENTED.
If callers use AsyncWait for the IPCBlobInputStream to be ready, and then call ReadSegments again we can end up returning NS_ERROR_NOT_IMPLEMENTED, depending on what the inner type is.
Differential Revision: https://phabricator.services.mozilla.com/D51781
--HG--
extra : moz-landing-system : lando
Most of these tests have been disabled for a long time; they run well
in the current test environment.
I intend to enable still more mochitests in a future patch.
Differential Revision: https://phabricator.services.mozilla.com/D49524
--HG--
extra : moz-landing-system : lando
The other cases when ClearWindowProxy is called seem to be fine.
It is the Unlink case which was causing null .contentWindow with test_mozfiledataurl.html
Fixes bug 1580391 and backs out bug 1581004
Differential Revision: https://phabricator.services.mozilla.com/D48878
--HG--
extra : moz-landing-system : lando
We want to remove flat strings (JSFlatString). With this patch we only expose
linear strings (JSLinearString) to API consumers.
This is very mechanical for the most part, because code typically only cares
about linear strings and not the null-termination aspect.
CTypes's Library.cpp has some Windows-specific code where we relied on null-terminated
strings. This patch adds JS_CopyStringCharsZ for that use case.
Differential Revision: https://phabricator.services.mozilla.com/D48314
--HG--
extra : moz-landing-system : lando
When we have a parser-created iframe which starts out in-process, transitions
to remote, and then transitions back to in-process, we create separate
DocShells for the first and last in-process loads. Since both are
network-created, and have the same child index, they both try to add
themselves as children to their parent's SHistory at the same index. And since
the entry for the first DocShell already exists at that index when we try to
add the second, that triggers an assertion.
This isn't really ideal, but it is expected given the current state of session
history under Fission. It should hopefully be solved more gracefully when the
Fission-aware session history rewrite is done, but in the mean time, I think
we should just ignore the conflict, since it's expected.
Differential Revision: https://phabricator.services.mozilla.com/D48437
--HG--
extra : moz-landing-system : lando
This patch cleans up remote outer window proxies when we navigate back
into the process.
It adds a flag to mDanglingRemoteOuterProxies that is set in between
BrowsingContext::SetDocShell(), where we can tell that the browsing
context is going from being remote to being local, to
nsGlobalWindowOuter::SetNewDocument(), where the local outer window
proxy is actually created. Once the outer window is created, the
remote window proxies can be cleaned up in
CleanUpDanglingRemoteOuterWindowProxies().
The clean up is done by a process that is similar to object
transplanting, except that instead of looking in the cross-compartment
wrapper table for each compartment to find objects to be turned into
CCWs to the new object, it looks in the remote proxy map for each
compartment. SpiderMonkey doesn't know about the proxy maps, so this
has to be done by a new callback object CompartmentTransplantCallback.
Now that this cleanup is being done, it shouldn't be possible to wrap
a remote outer window proxy when the browsing context is local, so
MaybeWrapWindowProxy() can be simplified. I had to drop the assert
here that the browsing context has a window proxy because during clean
up we call wrap on a local outer window proxy before the BC gets the
window proxy set on it. I had the assert because my original plan was
to implicitly fix remote proxies during wrapping, but that is no
longer necessary.
Differential Revision: https://phabricator.services.mozilla.com/D38343
--HG--
extra : moz-landing-system : lando
I needed empty blobs in MediaRecorder and seeing that there was no
Blob::CreateEmptyBlob, nor an export of EmptyBlobImpl, I thought
Blob::CreateEmptyBlob would be the cleaner solution, so here it is.
Differential Revision: https://phabricator.services.mozilla.com/D35310
--HG--
extra : moz-landing-system : lando
I needed empty blobs in MediaRecorder and seeing that there was no
Blob::CreateEmptyBlob, nor an export of EmptyBlobImpl, I thought
Blob::CreateEmptyBlob would be the cleaner solution, so here it is.
Differential Revision: https://phabricator.services.mozilla.com/D35310
--HG--
extra : moz-landing-system : lando
Simon Giesecke
Karl Tomlinson
Sylvestre Ledru
Butkovits Atila
Ricky Stewart
Bogdan Tara
Andi-Bogdan Postelnicu
Dorel Luca
Valentin Gosu
Christoph Kerschbaumer
Chris Peterson
Tom Schuster
Andrea Marchesini
Mihai Alexandru Michis
ssengupta
Noemi Erli
Csoregi Natalia
Razvan Maries
Jean-Yves Avenard
Nicholas Nethercote
Perry Jiang
Brindusan Cristian
owlishDeveloper
Matt Woodrow
Kris Maglione
Ciure Andrei
Tom Tung
Andreas Farre
Emilio Cobos Álvarez
Cosmin Sabou
Sebastian Streich
Stefan Hindli
Ed Lee
Mark Banner
Subhamoy Sengupta
Boris Zbarsky
Jed Davis
Edgar Chen
Ehsan Akhgari
Daniel Varga
Gabriele Svelto
Kershaw Chang
Coroiu Cristina
Geoff Brown
Olli Pettay
Jan de Mooij
Tom Tung
Andreea Pavel
Henri Sivonen
Andrew McCreight
Andreas Pehrson
Nika Layzell
Gurzau Raul