Previously there was one field here named small_max, and it wasn't obvious
from the name alone whether this meant the end of the
quantum-spaced sizes or the subpage sizes. Instead place both these fields
into the structure with clearer names. Besides, a later page will need
subpage_max.
Differential Revision: https://phabricator.services.mozilla.com/D88402
The MOZ_MUST_USE macro is defined as clang's and gcc's nonstandard __attribute__((warn_unused_result)). Now that we compile as C++17 by default (bug 1560664), we can replace MOZ_MUST_USE with C++17's standard [[nodiscard]] attribute.
We can also stop #including mozilla/Attributes.h because it was only needed for its MOZ_MUST_USE declaration.
Differential Revision: https://phabricator.services.mozilla.com/D89310
CLOSED TREE
We don't need these macros anymore, for two reasons:
1. We have static analysis to provide the same sort of checks via `MOZ_RAII`
and friends.
2. clang now warns for the "temporary that should have been a declaration" case.
The extra requirements on class construction also show up during debug tests
as performance problems.
This change was automated by using the following sed script:
```
# Remove declarations in classes.
/MOZ_DECL_USE_GUARD_OBJECT_NOTIFIER/d
/MOZ_GUARD_OBJECT_NOTIFIER_INIT/d
# Remove individual macros, carefully.
{
# We don't have to worry about substrings here because the closing
# parenthesis "anchors" the match.
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_TO_PARENT)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_IN_IMPL)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_IN_IMPL)/)/g;
# Remove the longer identifier first.
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_TO_PARENT//g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM//g;
}
# Remove the actual include.
\@# *include "mozilla/GuardObjects.h"@d
```
and running:
```
find . -name \*.cpp -o -name \*.h | grep -v 'GuardObjects.h' |xargs sed -i -f script 2>/dev/null
mach clang-format
```
Differential Revision: https://phabricator.services.mozilla.com/D85168
We don't need these macros anymore, for two reasons:
1. We have static analysis to provide the same sort of checks via `MOZ_RAII`
and friends.
2. clang now warns for the "temporary that should have been a declaration" case.
The extra requirements on class construction also show up during debug tests
as performance problems.
This change was automated by using the following sed script:
```
# Remove declarations in classes.
/MOZ_DECL_USE_GUARD_OBJECT_NOTIFIER/d
/MOZ_GUARD_OBJECT_NOTIFIER_INIT/d
# Remove individual macros, carefully.
{
# We don't have to worry about substrings here because the closing
# parenthesis "anchors" the match.
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_TO_PARENT)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_PARAM_IN_IMPL)/)/g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_IN_IMPL)/)/g;
# Remove the longer identifier first.
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM_TO_PARENT//g;
s/MOZ_GUARD_OBJECT_NOTIFIER_ONLY_PARAM//g;
}
# Remove the actual include.
\@# *include "mozilla/GuardObjects.h"@d
```
and running:
```
find . -name \*.cpp -o -name \*.h | grep -v 'GuardObjects.h' |xargs sed -i -f script 2>/dev/null
mach clang-format
```
Differential Revision: https://phabricator.services.mozilla.com/D85168
It was setup this way in bug 719579 for legitimate reasons (because
_msize would end up using the system symbol, which would crash trying to
get the size information from jemalloc allocations), but these reasons
don't apply anymore: back then the linking situation was different, and
nowadays, mozglue.dll exports its own _msize which plugs into
mozjemalloc, and sqlite is folded into nss3.dll, which links against
mozglue.dll, such that using _msize on mozjemalloc allocations works.
For some reason, while _msize (and other similar functions) are exported
from mozglue.dll without an explicit instruction to do so on clang-cl
builds, that's not the case for mingw builds (presumably related to the
definition of these functions in system headers, or lack thereof). So
we also add MOZ_EXPORT for them.
Differential Revision: https://phabricator.services.mozilla.com/D81286
The assert that was added in bug 1610720 assumed the node links were
reset when a node is removed from a RedBlackTree, but that wasn't the
case. We can either remove the assert, or clear node links. We pick the
latter.
Differential Revision: https://phabricator.services.mozilla.com/D61515
--HG--
extra : moz-landing-system : lando
Currently, it's expected to "lazily" dispose of the arena. That is, if
the arena still has allocations, the arena is kept around. Using it for
new allocations is an error, but otherwise, it's possible to free
allocations from there. Once it's empty, the arena is freed too. Well,
that's the idea, but that last part is not actually implemented, cf.
bug 1364359.
While that is still an (optionally) desirable disposal mode, other modes
would be useful as well. So this bug is here to switch the current mode
to something that can more easily be used for e.g. bug 1377999.
The idea is to make moz_dispose_arena() free the arena immediately, and
MOZ_CRASH if it's not already empty (on a besst effort basis).
Differential Revision: https://phabricator.services.mozilla.com/D60635
--HG--
extra : moz-landing-system : lando
The base_node_alloc and base_node_dealloc are wrappers for the base
allocator that allow to free memory (the base allocator doesn't support
freeing memory), using a free list. They are dedicated to extent_node_t
objects (the only ones we currently ever need to free).
We're going to need to free other types of objects, so make this more
generic.
Differential Revision: https://phabricator.services.mozilla.com/D60634
--HG--
extra : moz-landing-system : lando
glibc marks various allocation functions as `throw()`. This addition
hasn't been a problem until we tried to enable C++17, where clang
started complaining that we were redeclaring functions with mismatched
exception specifications. Peculiarly, glibc declares virtually
everything we redeclare as `throw()`, but clang only complains about the
mismatches for a particular subset of functions.
The approach taken in this patch is to add another potentially defined
macro to malloc_decls.h, `NOTHROW_MALLOC_DECL`. This macro works
exactly like `MALLOC_DECL`, except that clients can define
`NOTHROW_MALLOC_DECL` to add appropriate `throw()` specifiers when
declaring functions at global scope, and thereby avoid mismatched
exception specifications.
Differential Revision: https://phabricator.services.mozilla.com/D44045
--HG--
extra : moz-landing-system : lando
In the MinGW build, calls to malloc inside mozglue were not being
redirected as defined in the .def file. We create aliases for the
redirected functions to correctly redirect them inside mozglue.
An alternate solution for this exists. Rather than creating the
importlib during the linking step for mozglue, we could have used
dlltool to create it, and then provided it during linking. This
would allow mozglue to know that it should redirect calls to malloc
to je_malloc as specified in the .def file.
Differential Revision: https://phabricator.services.mozilla.com/D38407
--HG--
extra : moz-landing-system : lando
- On Android, we were already doing it, but using fallible allocations.
- On *nix, it probably doesn't make a difference, but can't hurt. For
most things in Gecko, operator new/delete are inlined and thus
replaced by direct calls to the underlying allocator functions
(moz_xmalloc, malloc, etc.). This may have a benefit for some third
party libraries that would otherwise go through libstdc++'s to
eventually end up back into our allocator via the zone allocator
on macOS and via the exported symbols on others.
- On Windows, because of how some CRT static libraries are, a non-inlined
operator new (thanks to some disabled STL wrapping) would end up linked
against the system malloc, causing problems.
Overall, this can only be better. This also reduces the number of places
where we define those functions.
And on Android, this means operator new within mozglue becomes infallible,
which is more consistent with everything else.
Differential Revision: https://phabricator.services.mozilla.com/D36166
--HG--
extra : moz-landing-system : lando
- On Android, we were already doing it, but using fallible allocations.
- On *nix, it probably doesn't make a difference, but can't hurt. For
most things in Gecko, operator new/delete are inlined and thus
replaced by direct calls to the underlying allocator functions
(moz_xmalloc, malloc, etc.). This may have a benefit for some third
party libraries that would otherwise go through libstdc++'s to
eventually end up back into our allocator via the zone allocator
on macOS and via the exported symbols on others.
- On Windows, because of how some CRT static libraries are, a non-inlined
operator new (thanks to some disabled STL wrapping) would end up linked
against the system malloc, causing problems.
Overall, this can only be better. This also reduces the number of places
where we define those functions.
And on Android, this means operator new within mozglue becomes infallible,
which is more consistent with everything else.
Differential Revision: https://phabricator.services.mozilla.com/D36166
--HG--
extra : moz-landing-system : lando
This allows freelist randomization on a per-arena basis, by supplying parameters to
arena creation.
It uses an xorshift PRNG with a 128-bit state. It is not cryptographically secure. An
attacker who can observe outputs of the RNG, or read its state, is already in a position
to bypass the randomization applied. At the same time we make its state 128 bit to prevent
a trivial bypass if one or two outputs are observed.
The way a run selects masks to check has not been modified, so the randomization is limited
to at most 32 bits in the current mask being tested. It should be noted that while allocations
from the same run may now be non deterministic (up to the maximum entropy as previously
stated), an attacker who can perform multiple allocations will still be able to allocate
a targeted free region (for example while exploiting a use after free vulnerability in the
DOM). Non deterministic allocations will only impede an attacker who has less control over
how they allocate a targeted free region, and may provide some benefit during exploitation
of a heap based buffer overflow vulnerability where the attacker wishes to construct a
precise layout of regions pre overflow.
Differential Revision: https://phabricator.services.mozilla.com/D32219
--HG--
extra : moz-landing-system : lando
`jemalloc_replace_dynamic()` is badly broken. If you install a malloc table
other than the default at startup (e.g. DMD's or PHC's), when you call
`jemalloc_replace_dynamic()` it installs a new allocator that wraps the
*default* allocator, and then when you call `jemalloc_replace_dynamic(nullptr)`
it switches back to the *default* allocator.
This commits makes numerous improvements.
- It removes the "flip-flopping" between malloc tables, which didn't really
work and isn't necessary.
- `jemalloc_replace_dynamic()` now switches between the *original* malloc table
and the new one, rather than the *default* malloc table and the new one.
- It renames various things, to make the names shorter and clearer.
- It clearly documents the dangers and limitations of
`jemalloc_replace_dynamic()`.
- It removes and inlines `profiler::Init()`, because there was only one call
site.
- It rearranges `install_memory_counter()` so the control flow is simpler.
Differential Revision: https://phabricator.services.mozilla.com/D34266
--HG--
extra : moz-landing-system : lando
This makes it less mozjemalloc-specific, which is helpful for PHC. No non-test
code uses the extra detail anyway.
Differential Revision: https://phabricator.services.mozilla.com/D34441
--HG--
extra : moz-landing-system : lando
MALLOC_STATIC_PAGESIZE is only set on some platforms. Specifically, it's
not set on ia64 and sparc. Which means the case MALLOC_STATIC_PAGESIZE
&& (sparc || ia64) never happens, and gPageSize is never 8 KiB.
Differential Revision: https://phabricator.services.mozilla.com/D31965
--HG--
extra : moz-landing-system : lando
To ensure that any new JSString has its char buffer allocated in the new arena,
it is useful to be able to query a pointer and assert that it is in the
correct arena (at-least in Debug Build).
This adds the required functionality to mozjemalloc, and JSString can use it
for its new assertion in a later change.
Differential Revision: https://phabricator.services.mozilla.com/D25711
--HG--
extra : moz-landing-system : lando
To ensure that any new JSString has its char buffer allocated in the new arena,
it is useful to be able to query a pointer and assert that it is in the
correct arena (at-least in Debug Build).
This adds the required functionality to mozjemalloc, and JSString can use it
for its new assertion in a later change.
Differential Revision: https://phabricator.services.mozilla.com/D25711
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
Paul Bone
Chris Peterson
Haik Aftandilian
Nathan Froyd
Mihai Alexandru Michis
Mike Hommey
Gian-Carlo Pascutto
André Bargull
Andrew McCreight
Ciure Andrei
Brian Hackett
Tom Ritter
Andreea Pavel
Sylvestre Ledru
Nicholas Nethercote
Mike Hommey
Chris Martin
Narcis Beleuzu
Petr Sumbera
David Major
Masatoshi Kimura