If we don't find the block, just print out the original argument,
instead of the version converted to an integer. Python gets mad when
you try to concatenate an int and a string.
Differential Revision: https://phabricator.services.mozilla.com/D67404
--HG--
extra : moz-landing-system : lando
Apps targeting SDK 29 are not allowed to open /dev/ashmem directly, and
instead must use NDK functions. Those functions are only available in
SDK 26 and higher, so we need this shim to use the functions if they
are available, else fallback to opening /dev/ashmem directly.
Differential Revision: https://phabricator.services.mozilla.com/D61012
--HG--
extra : moz-landing-system : lando
On Linux and Mac, this makes `dmd.py` *much* faster when it is first run on a
DMD data file.
On Windows, this makes DMD actually usable locally. Previously the stacks
weren't fixed and so were rubbish.
Differential Revision: https://phabricator.services.mozilla.com/D57271
--HG--
extra : moz-landing-system : lando
Apps targeting SDK 29 are not allowed to open /dev/ashmem directly, and
instead must use NDK functions. Those functions are only available in
SDK 26 and higher, so we need this shim to use the functions if they
are available, else fallback to opening /dev/ashmem directly.
Differential Revision: https://phabricator.services.mozilla.com/D61012
--HG--
extra : moz-landing-system : lando
With the old constants the page alloc slots would fill up quickly and the hit
rate would quickly drop below 20%.
With the new constants the alloc slots don't fill up so quickly and the hit
rate remains at or near 100% for a lot longer. Also, page allocs are recycled
more slowly, which should increase the likelihood of UAFs being detected
correctly.
Differential Revision: https://phabricator.services.mozilla.com/D62539
--HG--
extra : moz-landing-system : lando
Specifically, the number of page allocs in use, and the page alloc hit rate.
Differential Revision: https://phabricator.services.mozilla.com/D62538
--HG--
extra : moz-landing-system : lando
The assert that was added in bug 1610720 assumed the node links were
reset when a node is removed from a RedBlackTree, but that wasn't the
case. We can either remove the assert, or clear node links. We pick the
latter.
Differential Revision: https://phabricator.services.mozilla.com/D61515
--HG--
extra : moz-landing-system : lando
Currently, it's expected to "lazily" dispose of the arena. That is, if
the arena still has allocations, the arena is kept around. Using it for
new allocations is an error, but otherwise, it's possible to free
allocations from there. Once it's empty, the arena is freed too. Well,
that's the idea, but that last part is not actually implemented, cf.
bug 1364359.
While that is still an (optionally) desirable disposal mode, other modes
would be useful as well. So this bug is here to switch the current mode
to something that can more easily be used for e.g. bug 1377999.
The idea is to make moz_dispose_arena() free the arena immediately, and
MOZ_CRASH if it's not already empty (on a besst effort basis).
Differential Revision: https://phabricator.services.mozilla.com/D60635
--HG--
extra : moz-landing-system : lando
The base_node_alloc and base_node_dealloc are wrappers for the base
allocator that allow to free memory (the base allocator doesn't support
freeing memory), using a free list. They are dedicated to extent_node_t
objects (the only ones we currently ever need to free).
We're going to need to free other types of objects, so make this more
generic.
Differential Revision: https://phabricator.services.mozilla.com/D60634
--HG--
extra : moz-landing-system : lando
This was done by:
This was done by applying:
```
diff --git a/python/mozbuild/mozbuild/code-analysis/mach_commands.py b/python/mozbuild/mozbuild/code-analysis/mach_commands.py
index 789affde7bbf..fe33c4c7d4d1 100644
--- a/python/mozbuild/mozbuild/code-analysis/mach_commands.py
+++ b/python/mozbuild/mozbuild/code-analysis/mach_commands.py
@@ -2007,7 +2007,7 @@ class StaticAnalysis(MachCommandBase):
from subprocess import Popen, PIPE, check_output, CalledProcessError
diff_process = Popen(self._get_clang_format_diff_command(commit), stdout=PIPE)
- args = [sys.executable, clang_format_diff, "-p1", "-binary=%s" % clang_format]
+ args = [sys.executable, clang_format_diff, "-p1", "-binary=%s" % clang_format, '-sort-includes']
if not output_file:
args.append("-i")
```
Then running `./mach clang-format -c <commit-hash>`
Then undoing that patch.
Then running check_spidermonkey_style.py --fixup
Then running `./mach clang-format`
I had to fix four things:
* I needed to move <utility> back down in GuardObjects.h because I was hitting
obscure problems with our system include wrappers like this:
0:03.94 /usr/include/stdlib.h:550:14: error: exception specification in declaration does not match previous declaration
0:03.94 extern void *realloc (void *__ptr, size_t __size)
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/malloc_decls.h:53:1: note: previous declaration is here
0:03.94 MALLOC_DECL(realloc, void*, void*, size_t)
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/mozilla/mozalloc.h:22:32: note: expanded from macro 'MALLOC_DECL'
0:03.94 MOZ_MEMORY_API return_type name##_impl(__VA_ARGS__);
0:03.94 ^
0:03.94 <scratch space>:178:1: note: expanded from here
0:03.94 realloc_impl
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/mozmemory_wrap.h:142:41: note: expanded from macro 'realloc_impl'
0:03.94 #define realloc_impl mozmem_malloc_impl(realloc)
Which I really didn't feel like digging into.
* I had to restore the order of TrustOverrideUtils.h and related files in nss
because the .inc files depend on TrustOverrideUtils.h being included earlier.
* I had to add a missing include to RollingNumber.h
* Also had to partially restore include order in JsepSessionImpl.cpp to avoid
some -WError issues due to some static inline functions being defined in a
header but not used in the rest of the compilation unit.
Differential Revision: https://phabricator.services.mozilla.com/D60327
--HG--
extra : moz-landing-system : lando
rg -l 'mozilla/Move.h' | xargs sed -i 's/#include "mozilla\/Move.h"/#include <utility>/g'
Further manual fixups and cleanups to the include order incoming.
Differential Revision: https://phabricator.services.mozilla.com/D60323
--HG--
extra : moz-landing-system : lando
This is required on some systems such as Fedora to allow
building with -O0 together with --warnings-as-errors due to
a check in /usr/include/features.h
Differential Revision: https://phabricator.services.mozilla.com/D56696
--HG--
extra : moz-landing-system : lando
Currently the PHC code uses char* and uintptr_t in various address
computations. This patch changes it to use uint8_t* instead, which is clearer
than char* and avoids the need for various casts.
Differential Revision: https://phabricator.services.mozilla.com/D43840
--HG--
extra : moz-landing-system : lando
glibc marks various allocation functions as `throw()`. This addition
hasn't been a problem until we tried to enable C++17, where clang
started complaining that we were redeclaring functions with mismatched
exception specifications. Peculiarly, glibc declares virtually
everything we redeclare as `throw()`, but clang only complains about the
mismatches for a particular subset of functions.
The approach taken in this patch is to add another potentially defined
macro to malloc_decls.h, `NOTHROW_MALLOC_DECL`. This macro works
exactly like `MALLOC_DECL`, except that clients can define
`NOTHROW_MALLOC_DECL` to add appropriate `throw()` specifiers when
declaring functions at global scope, and thereby avoid mismatched
exception specifications.
Differential Revision: https://phabricator.services.mozilla.com/D44045
--HG--
extra : moz-landing-system : lando
In the MinGW build, calls to malloc inside mozglue were not being
redirected as defined in the .def file. We create aliases for the
redirected functions to correctly redirect them inside mozglue.
An alternate solution for this exists. Rather than creating the
importlib during the linking step for mozglue, we could have used
dlltool to create it, and then provided it during linking. This
would allow mozglue to know that it should redirect calls to malloc
to je_malloc as specified in the .def file.
Differential Revision: https://phabricator.services.mozilla.com/D38407
--HG--
extra : moz-landing-system : lando
While here, properly export StackWalk.h when building with
--enable-project=memory.
Differential Revision: https://phabricator.services.mozilla.com/D40731
--HG--
extra : moz-landing-system : lando
None of the StringTable implementations were freeing their strdup'd
strings, either.
Differential Revision: https://phabricator.services.mozilla.com/D40757
--HG--
extra : moz-landing-system : lando
The most important part is the `GetPHCAddrInfo()` function in
`exception_handler.cc`, which extracts a crash address from an
`EXCEPTION_POINTERS` object.
The commit also changes the code so that it works if MozStackWalk() returns an
empty stack trace. This happens in practice on Windows on automation sometimes,
I'm not sure why.
The rest of the commit is just plumbing and the smoothing over of minor
platform differences.
Differential Revision: https://phabricator.services.mozilla.com/D39841
--HG--
extra : moz-landing-system : lando
- On Android, we were already doing it, but using fallible allocations.
- On *nix, it probably doesn't make a difference, but can't hurt. For
most things in Gecko, operator new/delete are inlined and thus
replaced by direct calls to the underlying allocator functions
(moz_xmalloc, malloc, etc.). This may have a benefit for some third
party libraries that would otherwise go through libstdc++'s to
eventually end up back into our allocator via the zone allocator
on macOS and via the exported symbols on others.
- On Windows, because of how some CRT static libraries are, a non-inlined
operator new (thanks to some disabled STL wrapping) would end up linked
against the system malloc, causing problems.
Overall, this can only be better. This also reduces the number of places
where we define those functions.
And on Android, this means operator new within mozglue becomes infallible,
which is more consistent with everything else.
Differential Revision: https://phabricator.services.mozilla.com/D36166
--HG--
extra : moz-landing-system : lando
Bug 1147248 added the workaround for GCC 4.9, but from an attempt with
GCC 6, it seems unnecessary anymore.
Differential Revision: https://phabricator.services.mozilla.com/D36165
--HG--
extra : moz-landing-system : lando
- On Android, we were already doing it, but using fallible allocations.
- On *nix, it probably doesn't make a difference, but can't hurt. For
most things in Gecko, operator new/delete are inlined and thus
replaced by direct calls to the underlying allocator functions
(moz_xmalloc, malloc, etc.). This may have a benefit for some third
party libraries that would otherwise go through libstdc++'s to
eventually end up back into our allocator via the zone allocator
on macOS and via the exported symbols on others.
- On Windows, because of how some CRT static libraries are, a non-inlined
operator new (thanks to some disabled STL wrapping) would end up linked
against the system malloc, causing problems.
Overall, this can only be better. This also reduces the number of places
where we define those functions.
And on Android, this means operator new within mozglue becomes infallible,
which is more consistent with everything else.
Differential Revision: https://phabricator.services.mozilla.com/D36166
--HG--
extra : moz-landing-system : lando
Bug 1147248 added the workaround for GCC 4.9, but from an attempt with
GCC 6, it seems unnecessary anymore.
Differential Revision: https://phabricator.services.mozilla.com/D36165
--HG--
extra : moz-landing-system : lando
This allows freelist randomization on a per-arena basis, by supplying parameters to
arena creation.
It uses an xorshift PRNG with a 128-bit state. It is not cryptographically secure. An
attacker who can observe outputs of the RNG, or read its state, is already in a position
to bypass the randomization applied. At the same time we make its state 128 bit to prevent
a trivial bypass if one or two outputs are observed.
The way a run selects masks to check has not been modified, so the randomization is limited
to at most 32 bits in the current mask being tested. It should be noted that while allocations
from the same run may now be non deterministic (up to the maximum entropy as previously
stated), an attacker who can perform multiple allocations will still be able to allocate
a targeted free region (for example while exploiting a use after free vulnerability in the
DOM). Non deterministic allocations will only impede an attacker who has less control over
how they allocate a targeted free region, and may provide some benefit during exploitation
of a heap based buffer overflow vulnerability where the attacker wishes to construct a
precise layout of regions pre overflow.
Differential Revision: https://phabricator.services.mozilla.com/D32219
--HG--
extra : moz-landing-system : lando
`jemalloc_replace_dynamic()` is badly broken. If you install a malloc table
other than the default at startup (e.g. DMD's or PHC's), when you call
`jemalloc_replace_dynamic()` it installs a new allocator that wraps the
*default* allocator, and then when you call `jemalloc_replace_dynamic(nullptr)`
it switches back to the *default* allocator.
This commits makes numerous improvements.
- It removes the "flip-flopping" between malloc tables, which didn't really
work and isn't necessary.
- `jemalloc_replace_dynamic()` now switches between the *original* malloc table
and the new one, rather than the *default* malloc table and the new one.
- It renames various things, to make the names shorter and clearer.
- It clearly documents the dangers and limitations of
`jemalloc_replace_dynamic()`.
- It removes and inlines `profiler::Init()`, because there was only one call
site.
- It rearranges `install_memory_counter()` so the control flow is simpler.
Differential Revision: https://phabricator.services.mozilla.com/D34266
--HG--
extra : moz-landing-system : lando
This makes it less mozjemalloc-specific, which is helpful for PHC. No non-test
code uses the extra detail anyway.
Differential Revision: https://phabricator.services.mozilla.com/D34441
--HG--
extra : moz-landing-system : lando
The current situation is suboptimal, where we have the same goop
repeated in multiple files, and where things kinda sorta work out fine
thanks to the linker for files that would have been forbidden, except
when the linker doesn't do its job, which apparently happen on
mingwclang builds.
This change only really covers C++ code using operator new/delete, and
not things that would be using malloc/free, because it's easier.
malloc/free is left for a followup.
Differential Revision: https://phabricator.services.mozilla.com/D32119
--HG--
extra : moz-landing-system : lando
MALLOC_STATIC_PAGESIZE is only set on some platforms. Specifically, it's
not set on ia64 and sparc. Which means the case MALLOC_STATIC_PAGESIZE
&& (sparc || ia64) never happens, and gPageSize is never 8 KiB.
Differential Revision: https://phabricator.services.mozilla.com/D31965
--HG--
extra : moz-landing-system : lando
In D25711, I added an arenaId member to `jemalloc_ptr_info_t` when `MOZ_DEBUG`
is defined. This modifies the GTest for `jemalloc_ptr_info()` to ensure that
the new member returns the correct value.
Differential Revision: https://phabricator.services.mozilla.com/D30087
--HG--
extra : moz-landing-system : lando
To ensure that any new JSString has its char buffer allocated in the new arena,
it is useful to be able to query a pointer and assert that it is in the
correct arena (at-least in Debug Build).
This adds the required functionality to mozjemalloc, and JSString can use it
for its new assertion in a later change.
Differential Revision: https://phabricator.services.mozilla.com/D25711
--HG--
extra : moz-landing-system : lando
Changes:
- added comments for tests being disabled
- disabled two additional tests in order to green the run
Differential Revision: https://phabricator.services.mozilla.com/D28085
--HG--
extra : moz-landing-system : lando
To ensure that any new JSString has its char buffer allocated in the new arena,
it is useful to be able to query a pointer and assert that it is in the
correct arena (at-least in Debug Build).
This adds the required functionality to mozjemalloc, and JSString can use it
for its new assertion in a later change.
Differential Revision: https://phabricator.services.mozilla.com/D25711
Changes:
- most tests are skipped using `moz.build` configuration file.
- `MultiWriterQueue` had to be skipped with `define` clauses in the test file due to build bustages when its `moz.build` file was used.
Differential Revision: https://phabricator.services.mozilla.com/D27944
--HG--
extra : moz-landing-system : lando
The jemalloc tests leave behind minidumps. Disable for now, for a green run.
Differential Revision: https://phabricator.services.mozilla.com/D27014
--HG--
extra : moz-landing-system : lando
This only copies the first `mLength` elements in `mPcs`.
Differential Revision: https://phabricator.services.mozilla.com/D21488
--HG--
extra : rebase_source : 91601501056fe7ea5b011adeca59033568fbde4a
This doesn't change behaviour because `sizeof(uintptr_t)` equals
`sizeof(uintptr_t*)`, but it's good to make things more obviously correct.
Differential Revision: https://phabricator.services.mozilla.com/D21487
--HG--
extra : rebase_source : 4df36d1d1e85a671c297e6e761b44b7da5b85343
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
***
Bug 1514594: Part 3a - Change ChromeUtils.import to return an exports object; not pollute global. r=mccr8
This changes the behavior of ChromeUtils.import() to return an exports object,
rather than a module global, in all cases except when `null` is passed as a
second argument, and changes the default behavior not to pollute the global
scope with the module's exports. Thus, the following code written for the old
model:
ChromeUtils.import("resource://gre/modules/Services.jsm");
is approximately the same as the following, in the new model:
var {Services} = ChromeUtils.import("resource://gre/modules/Services.jsm");
Since the two behaviors are mutually incompatible, this patch will land with a
scripted rewrite to update all existing callers to use the new model rather
than the old.
***
Bug 1514594: Part 3b - Mass rewrite all JS code to use the new ChromeUtils.import API. rs=Gijs
This was done using the followng script:
https://bitbucket.org/kmaglione/m-c-rewrites/src/tip/processors/cu-import-exports.jsm
***
Bug 1514594: Part 3c - Update ESLint plugin for ChromeUtils.import API changes. r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D16747
***
Bug 1514594: Part 3d - Remove/fix hundreds of duplicate imports from sync tests. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16748
***
Bug 1514594: Part 3e - Remove no-op ChromeUtils.import() calls. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16749
***
Bug 1514594: Part 3f.1 - Cleanup various test corner cases after mass rewrite. r=Gijs
***
Bug 1514594: Part 3f.2 - Cleanup various non-test corner cases after mass rewrite. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16750
--HG--
extra : rebase_source : 359574ee3064c90f33bf36c2ebe3159a24cc8895
extra : histedit_source : b93c8f42808b1599f9122d7842d2c0b3e656a594%2C64a3a4e3359dc889e2ab2b49461bab9e27fc10a7
Summary: Really sorry for the size of the patch. It's mostly automatic
s/nsIDocument/Document/ but I had to fix up in a bunch of places manually to
add the right namespacing and such.
Overall it's not a very interesting patch I think.
nsDocument.cpp turns into Document.cpp, nsIDocument.h into Document.h and
nsIDocumentInlines.h into DocumentInlines.h.
I also changed a bunch of nsCOMPtr usage to RefPtr, but not all of it.
While fixing up some of the bits I also removed some unneeded OwnerDoc() null
checks and such, but I didn't do anything riskier than that.
The diagnostic assert (so fortunately, it doesn't impact release builds)
as added in bug 1405159, but is costly because it uses the modulus of
the division with a variable integer, which is a slow operation.
However, in arena_run_reg_dalloc, we end up doing the same diagnostic
assert, in a different form: after performing the division in a faster
manner, we assert that the result, multiplied by the diviser, returns
the original number.
Differential Revision: https://phabricator.services.mozilla.com/D13501
--HG--
extra : moz-landing-system : lando
Sylvestre Ledru
Andrew McCreight
Nicholas Nethercote
Mark Banner
James Willcox
Ciure Andrei
Masatoshi Kimura
Coroiu Cristina
Mike Hommey
Emilio Cobos Álvarez
Brian Hackett
Tom Ritter
Andreea Pavel
Simon Giesecke
Gian-Carlo Pascutto
Christian Holler
Nathan Froyd
Cameron McCormack
Mihai Alexandru Michis
Jed Davis
Csoregi Natalia
Victor Porof
Nick Thomas
Mike Hommey
Chris Martin
Edwin Gao
Narcis Beleuzu
Geoff Brown
Petr Sumbera
Gurzau Raul
David Major
Ryan Hunt
Ehsan Akhgari
Jeff Gilbert
Myk Melez
Kris Maglione
Tooru Fujisawa