Currently AWSY-with-DMD doesn't work on Windows. This is because `fix-stacks`
is initialized lazily, and by the time the initialization happens some file
descriptors for files are open, and that leads to some major Python2-on-Windows
sadness as described in the big comment in the commit.
To fix the problem, this commit adds an `init` function to `fix_stacks.py` so
that `fix-stacks` can be initialized eagerly, hopefully before any file
descriptors for files are open.
For `dmd.py`, other than fixing the AWSY problems, this has little effect,
because `fix-stacks` is always initialized.
For `utils.py`, which is used to process the output of most tests, this has a
more noticeable effect: the `fix-stacks` process is always spawned, rather than
being spawned only when needed. If no stack traces appear in the test output,
this means that `fix-stacks` is spawned unnecessarily. But it's cheap to spawn;
the expensive part only happens when stack traces start getting fixed. So I
think this change in behaviour is acceptable.
Furthermore, the commit adds a `finish` function to `fix_stacks.py`, so that
the `fix-stacks` process can be explicitly shut down. This has never been done
for processes spawned for any of the stack fixing scripts. It's never caused
problems on Linux/Mac, but it seems to be necessary on Windows to avoid
similar "this file is locked" problems with the test_dmd.js test.
The commit also renames some things to more standard Python style, e.g.
`json_mode` instead of `jsonMode`.
Finally, Android tests use `utils.py` from the repository but `fix_stacks.py`
from the Android host utils. Because the two scripts must be updated in tandem,
this commit also updates the Android host utils to a version that contains the
updated `fix_stacks.py`. Thanks to aerickson for packaging up the new Android
host utils and providing the change to the `hostutils.manifest` file.
Differential Revision: https://phabricator.services.mozilla.com/D69478
This commit removes `test_fix_stack_using_bpsyms.py`. That test can't easily be
modified to work with `fix_stacks.py` because it relies on internal
implementation details of `fix_stack_using_bpsym.py`. The unit testing done in
the `fix-stacks` repo provides test coverage that is as good or better.
Differential Revision: https://phabricator.services.mozilla.com/D66924
--HG--
extra : moz-landing-system : lando
Currently AWSY-with-DMD doesn't work on Windows. This is because `fix-stacks`
is initialized lazily, and by the time the initialization happens some file
descriptors for files are open, and that leads to some major Python2-on-Windows
sadness as described in the big comment in the commit.
To fix the problem, this commit adds an `init` function to `fix_stacks.py` so
that `fix-stacks` can be initialized eagerly, hopefully before any file
descriptors for files are open.
For `dmd.py`, other than fixing the AWSY problems, this has little effect,
because `fix-stacks` is always initialized.
For `utils.py`, which is used to process the output of most tests, this has a
more noticeable effect: the `fix-stacks` process is always spawned, rather than
being spawned only when needed. If no stack traces appear in the test output,
this means that `fix-stacks` is spawned unnecessarily. But it's cheap to spawn;
the expensive part only happens when stack traces start getting fixed. So I
think this change in behaviour is acceptable.
The commit also renames some things to more standard Python style, e.g.
`json_mode` instead of `jsonMode`.
Differential Revision: https://phabricator.services.mozilla.com/D69478
--HG--
extra : moz-landing-system : lando
Each allocation page is now bracketed by a guard page, and allocations are put
at the end of their page so that bounds violations trigger a crash.
Various operations (realloc(), free(), malloc_usable_size()) now require that
the pointer they are given points to the start of an allocation.
Differential Revision: https://phabricator.services.mozilla.com/D43842
--HG--
rename : toolkit/crashreporter/test/unit_ipc/test_content_phc2.js => toolkit/crashreporter/test/unit_ipc/test_content_phc3.js
extra : moz-landing-system : lando
This is in preparation for the introduction of "guard pages", which are
interleaved with alloc pages. The specific renamings are:
- kMaxPageAllocs --> kNumAllocPages
- PagePtr --> AllocPagePtr
- PageState --> AllocPageState
- PageInfo --> AllocPageInfo
- mPages --> mAllocPages
- AssertPageInUse --> AssertAllocPageInUse
Differential Revision: https://phabricator.services.mozilla.com/D43841
--HG--
extra : moz-landing-system : lando
If we don't find the block, just print out the original argument,
instead of the version converted to an integer. Python gets mad when
you try to concatenate an int and a string.
Differential Revision: https://phabricator.services.mozilla.com/D67404
--HG--
extra : moz-landing-system : lando
Apps targeting SDK 29 are not allowed to open /dev/ashmem directly, and
instead must use NDK functions. Those functions are only available in
SDK 26 and higher, so we need this shim to use the functions if they
are available, else fallback to opening /dev/ashmem directly.
Differential Revision: https://phabricator.services.mozilla.com/D61012
--HG--
extra : moz-landing-system : lando
On Linux and Mac, this makes `dmd.py` *much* faster when it is first run on a
DMD data file.
On Windows, this makes DMD actually usable locally. Previously the stacks
weren't fixed and so were rubbish.
Differential Revision: https://phabricator.services.mozilla.com/D57271
--HG--
extra : moz-landing-system : lando
Apps targeting SDK 29 are not allowed to open /dev/ashmem directly, and
instead must use NDK functions. Those functions are only available in
SDK 26 and higher, so we need this shim to use the functions if they
are available, else fallback to opening /dev/ashmem directly.
Differential Revision: https://phabricator.services.mozilla.com/D61012
--HG--
extra : moz-landing-system : lando
With the old constants the page alloc slots would fill up quickly and the hit
rate would quickly drop below 20%.
With the new constants the alloc slots don't fill up so quickly and the hit
rate remains at or near 100% for a lot longer. Also, page allocs are recycled
more slowly, which should increase the likelihood of UAFs being detected
correctly.
Differential Revision: https://phabricator.services.mozilla.com/D62539
--HG--
extra : moz-landing-system : lando
Specifically, the number of page allocs in use, and the page alloc hit rate.
Differential Revision: https://phabricator.services.mozilla.com/D62538
--HG--
extra : moz-landing-system : lando
The assert that was added in bug 1610720 assumed the node links were
reset when a node is removed from a RedBlackTree, but that wasn't the
case. We can either remove the assert, or clear node links. We pick the
latter.
Differential Revision: https://phabricator.services.mozilla.com/D61515
--HG--
extra : moz-landing-system : lando
Currently, it's expected to "lazily" dispose of the arena. That is, if
the arena still has allocations, the arena is kept around. Using it for
new allocations is an error, but otherwise, it's possible to free
allocations from there. Once it's empty, the arena is freed too. Well,
that's the idea, but that last part is not actually implemented, cf.
bug 1364359.
While that is still an (optionally) desirable disposal mode, other modes
would be useful as well. So this bug is here to switch the current mode
to something that can more easily be used for e.g. bug 1377999.
The idea is to make moz_dispose_arena() free the arena immediately, and
MOZ_CRASH if it's not already empty (on a besst effort basis).
Differential Revision: https://phabricator.services.mozilla.com/D60635
--HG--
extra : moz-landing-system : lando
The base_node_alloc and base_node_dealloc are wrappers for the base
allocator that allow to free memory (the base allocator doesn't support
freeing memory), using a free list. They are dedicated to extent_node_t
objects (the only ones we currently ever need to free).
We're going to need to free other types of objects, so make this more
generic.
Differential Revision: https://phabricator.services.mozilla.com/D60634
--HG--
extra : moz-landing-system : lando
This was done by:
This was done by applying:
```
diff --git a/python/mozbuild/mozbuild/code-analysis/mach_commands.py b/python/mozbuild/mozbuild/code-analysis/mach_commands.py
index 789affde7bbf..fe33c4c7d4d1 100644
--- a/python/mozbuild/mozbuild/code-analysis/mach_commands.py
+++ b/python/mozbuild/mozbuild/code-analysis/mach_commands.py
@@ -2007,7 +2007,7 @@ class StaticAnalysis(MachCommandBase):
from subprocess import Popen, PIPE, check_output, CalledProcessError
diff_process = Popen(self._get_clang_format_diff_command(commit), stdout=PIPE)
- args = [sys.executable, clang_format_diff, "-p1", "-binary=%s" % clang_format]
+ args = [sys.executable, clang_format_diff, "-p1", "-binary=%s" % clang_format, '-sort-includes']
if not output_file:
args.append("-i")
```
Then running `./mach clang-format -c <commit-hash>`
Then undoing that patch.
Then running check_spidermonkey_style.py --fixup
Then running `./mach clang-format`
I had to fix four things:
* I needed to move <utility> back down in GuardObjects.h because I was hitting
obscure problems with our system include wrappers like this:
0:03.94 /usr/include/stdlib.h:550:14: error: exception specification in declaration does not match previous declaration
0:03.94 extern void *realloc (void *__ptr, size_t __size)
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/malloc_decls.h:53:1: note: previous declaration is here
0:03.94 MALLOC_DECL(realloc, void*, void*, size_t)
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/mozilla/mozalloc.h:22:32: note: expanded from macro 'MALLOC_DECL'
0:03.94 MOZ_MEMORY_API return_type name##_impl(__VA_ARGS__);
0:03.94 ^
0:03.94 <scratch space>:178:1: note: expanded from here
0:03.94 realloc_impl
0:03.94 ^
0:03.94 /home/emilio/src/moz/gecko-2/obj-debug/dist/include/mozmemory_wrap.h:142:41: note: expanded from macro 'realloc_impl'
0:03.94 #define realloc_impl mozmem_malloc_impl(realloc)
Which I really didn't feel like digging into.
* I had to restore the order of TrustOverrideUtils.h and related files in nss
because the .inc files depend on TrustOverrideUtils.h being included earlier.
* I had to add a missing include to RollingNumber.h
* Also had to partially restore include order in JsepSessionImpl.cpp to avoid
some -WError issues due to some static inline functions being defined in a
header but not used in the rest of the compilation unit.
Differential Revision: https://phabricator.services.mozilla.com/D60327
--HG--
extra : moz-landing-system : lando
rg -l 'mozilla/Move.h' | xargs sed -i 's/#include "mozilla\/Move.h"/#include <utility>/g'
Further manual fixups and cleanups to the include order incoming.
Differential Revision: https://phabricator.services.mozilla.com/D60323
--HG--
extra : moz-landing-system : lando
This is required on some systems such as Fedora to allow
building with -O0 together with --warnings-as-errors due to
a check in /usr/include/features.h
Differential Revision: https://phabricator.services.mozilla.com/D56696
--HG--
extra : moz-landing-system : lando
Currently the PHC code uses char* and uintptr_t in various address
computations. This patch changes it to use uint8_t* instead, which is clearer
than char* and avoids the need for various casts.
Differential Revision: https://phabricator.services.mozilla.com/D43840
--HG--
extra : moz-landing-system : lando
glibc marks various allocation functions as `throw()`. This addition
hasn't been a problem until we tried to enable C++17, where clang
started complaining that we were redeclaring functions with mismatched
exception specifications. Peculiarly, glibc declares virtually
everything we redeclare as `throw()`, but clang only complains about the
mismatches for a particular subset of functions.
The approach taken in this patch is to add another potentially defined
macro to malloc_decls.h, `NOTHROW_MALLOC_DECL`. This macro works
exactly like `MALLOC_DECL`, except that clients can define
`NOTHROW_MALLOC_DECL` to add appropriate `throw()` specifiers when
declaring functions at global scope, and thereby avoid mismatched
exception specifications.
Differential Revision: https://phabricator.services.mozilla.com/D44045
--HG--
extra : moz-landing-system : lando
In the MinGW build, calls to malloc inside mozglue were not being
redirected as defined in the .def file. We create aliases for the
redirected functions to correctly redirect them inside mozglue.
An alternate solution for this exists. Rather than creating the
importlib during the linking step for mozglue, we could have used
dlltool to create it, and then provided it during linking. This
would allow mozglue to know that it should redirect calls to malloc
to je_malloc as specified in the .def file.
Differential Revision: https://phabricator.services.mozilla.com/D38407
--HG--
extra : moz-landing-system : lando
While here, properly export StackWalk.h when building with
--enable-project=memory.
Differential Revision: https://phabricator.services.mozilla.com/D40731
--HG--
extra : moz-landing-system : lando
None of the StringTable implementations were freeing their strdup'd
strings, either.
Differential Revision: https://phabricator.services.mozilla.com/D40757
--HG--
extra : moz-landing-system : lando
The most important part is the `GetPHCAddrInfo()` function in
`exception_handler.cc`, which extracts a crash address from an
`EXCEPTION_POINTERS` object.
The commit also changes the code so that it works if MozStackWalk() returns an
empty stack trace. This happens in practice on Windows on automation sometimes,
I'm not sure why.
The rest of the commit is just plumbing and the smoothing over of minor
platform differences.
Differential Revision: https://phabricator.services.mozilla.com/D39841
--HG--
extra : moz-landing-system : lando
- On Android, we were already doing it, but using fallible allocations.
- On *nix, it probably doesn't make a difference, but can't hurt. For
most things in Gecko, operator new/delete are inlined and thus
replaced by direct calls to the underlying allocator functions
(moz_xmalloc, malloc, etc.). This may have a benefit for some third
party libraries that would otherwise go through libstdc++'s to
eventually end up back into our allocator via the zone allocator
on macOS and via the exported symbols on others.
- On Windows, because of how some CRT static libraries are, a non-inlined
operator new (thanks to some disabled STL wrapping) would end up linked
against the system malloc, causing problems.
Overall, this can only be better. This also reduces the number of places
where we define those functions.
And on Android, this means operator new within mozglue becomes infallible,
which is more consistent with everything else.
Differential Revision: https://phabricator.services.mozilla.com/D36166
--HG--
extra : moz-landing-system : lando
Bug 1147248 added the workaround for GCC 4.9, but from an attempt with
GCC 6, it seems unnecessary anymore.
Differential Revision: https://phabricator.services.mozilla.com/D36165
--HG--
extra : moz-landing-system : lando
- On Android, we were already doing it, but using fallible allocations.
- On *nix, it probably doesn't make a difference, but can't hurt. For
most things in Gecko, operator new/delete are inlined and thus
replaced by direct calls to the underlying allocator functions
(moz_xmalloc, malloc, etc.). This may have a benefit for some third
party libraries that would otherwise go through libstdc++'s to
eventually end up back into our allocator via the zone allocator
on macOS and via the exported symbols on others.
- On Windows, because of how some CRT static libraries are, a non-inlined
operator new (thanks to some disabled STL wrapping) would end up linked
against the system malloc, causing problems.
Overall, this can only be better. This also reduces the number of places
where we define those functions.
And on Android, this means operator new within mozglue becomes infallible,
which is more consistent with everything else.
Differential Revision: https://phabricator.services.mozilla.com/D36166
--HG--
extra : moz-landing-system : lando
Bug 1147248 added the workaround for GCC 4.9, but from an attempt with
GCC 6, it seems unnecessary anymore.
Differential Revision: https://phabricator.services.mozilla.com/D36165
--HG--
extra : moz-landing-system : lando
This allows freelist randomization on a per-arena basis, by supplying parameters to
arena creation.
It uses an xorshift PRNG with a 128-bit state. It is not cryptographically secure. An
attacker who can observe outputs of the RNG, or read its state, is already in a position
to bypass the randomization applied. At the same time we make its state 128 bit to prevent
a trivial bypass if one or two outputs are observed.
The way a run selects masks to check has not been modified, so the randomization is limited
to at most 32 bits in the current mask being tested. It should be noted that while allocations
from the same run may now be non deterministic (up to the maximum entropy as previously
stated), an attacker who can perform multiple allocations will still be able to allocate
a targeted free region (for example while exploiting a use after free vulnerability in the
DOM). Non deterministic allocations will only impede an attacker who has less control over
how they allocate a targeted free region, and may provide some benefit during exploitation
of a heap based buffer overflow vulnerability where the attacker wishes to construct a
precise layout of regions pre overflow.
Differential Revision: https://phabricator.services.mozilla.com/D32219
--HG--
extra : moz-landing-system : lando
`jemalloc_replace_dynamic()` is badly broken. If you install a malloc table
other than the default at startup (e.g. DMD's or PHC's), when you call
`jemalloc_replace_dynamic()` it installs a new allocator that wraps the
*default* allocator, and then when you call `jemalloc_replace_dynamic(nullptr)`
it switches back to the *default* allocator.
This commits makes numerous improvements.
- It removes the "flip-flopping" between malloc tables, which didn't really
work and isn't necessary.
- `jemalloc_replace_dynamic()` now switches between the *original* malloc table
and the new one, rather than the *default* malloc table and the new one.
- It renames various things, to make the names shorter and clearer.
- It clearly documents the dangers and limitations of
`jemalloc_replace_dynamic()`.
- It removes and inlines `profiler::Init()`, because there was only one call
site.
- It rearranges `install_memory_counter()` so the control flow is simpler.
Differential Revision: https://phabricator.services.mozilla.com/D34266
--HG--
extra : moz-landing-system : lando
This makes it less mozjemalloc-specific, which is helpful for PHC. No non-test
code uses the extra detail anyway.
Differential Revision: https://phabricator.services.mozilla.com/D34441
--HG--
extra : moz-landing-system : lando
The current situation is suboptimal, where we have the same goop
repeated in multiple files, and where things kinda sorta work out fine
thanks to the linker for files that would have been forbidden, except
when the linker doesn't do its job, which apparently happen on
mingwclang builds.
This change only really covers C++ code using operator new/delete, and
not things that would be using malloc/free, because it's easier.
malloc/free is left for a followup.
Differential Revision: https://phabricator.services.mozilla.com/D32119
--HG--
extra : moz-landing-system : lando
MALLOC_STATIC_PAGESIZE is only set on some platforms. Specifically, it's
not set on ia64 and sparc. Which means the case MALLOC_STATIC_PAGESIZE
&& (sparc || ia64) never happens, and gPageSize is never 8 KiB.
Differential Revision: https://phabricator.services.mozilla.com/D31965
--HG--
extra : moz-landing-system : lando
In D25711, I added an arenaId member to `jemalloc_ptr_info_t` when `MOZ_DEBUG`
is defined. This modifies the GTest for `jemalloc_ptr_info()` to ensure that
the new member returns the correct value.
Differential Revision: https://phabricator.services.mozilla.com/D30087
--HG--
extra : moz-landing-system : lando
To ensure that any new JSString has its char buffer allocated in the new arena,
it is useful to be able to query a pointer and assert that it is in the
correct arena (at-least in Debug Build).
This adds the required functionality to mozjemalloc, and JSString can use it
for its new assertion in a later change.
Differential Revision: https://phabricator.services.mozilla.com/D25711
--HG--
extra : moz-landing-system : lando
Changes:
- added comments for tests being disabled
- disabled two additional tests in order to green the run
Differential Revision: https://phabricator.services.mozilla.com/D28085
--HG--
extra : moz-landing-system : lando
To ensure that any new JSString has its char buffer allocated in the new arena,
it is useful to be able to query a pointer and assert that it is in the
correct arena (at-least in Debug Build).
This adds the required functionality to mozjemalloc, and JSString can use it
for its new assertion in a later change.
Differential Revision: https://phabricator.services.mozilla.com/D25711
Changes:
- most tests are skipped using `moz.build` configuration file.
- `MultiWriterQueue` had to be skipped with `define` clauses in the test file due to build bustages when its `moz.build` file was used.
Differential Revision: https://phabricator.services.mozilla.com/D27944
--HG--
extra : moz-landing-system : lando
The jemalloc tests leave behind minidumps. Disable for now, for a green run.
Differential Revision: https://phabricator.services.mozilla.com/D27014
--HG--
extra : moz-landing-system : lando
This only copies the first `mLength` elements in `mPcs`.
Differential Revision: https://phabricator.services.mozilla.com/D21488
--HG--
extra : rebase_source : 91601501056fe7ea5b011adeca59033568fbde4a
This doesn't change behaviour because `sizeof(uintptr_t)` equals
`sizeof(uintptr_t*)`, but it's good to make things more obviously correct.
Differential Revision: https://phabricator.services.mozilla.com/D21487
--HG--
extra : rebase_source : 4df36d1d1e85a671c297e6e761b44b7da5b85343
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
***
Bug 1514594: Part 3a - Change ChromeUtils.import to return an exports object; not pollute global. r=mccr8
This changes the behavior of ChromeUtils.import() to return an exports object,
rather than a module global, in all cases except when `null` is passed as a
second argument, and changes the default behavior not to pollute the global
scope with the module's exports. Thus, the following code written for the old
model:
ChromeUtils.import("resource://gre/modules/Services.jsm");
is approximately the same as the following, in the new model:
var {Services} = ChromeUtils.import("resource://gre/modules/Services.jsm");
Since the two behaviors are mutually incompatible, this patch will land with a
scripted rewrite to update all existing callers to use the new model rather
than the old.
***
Bug 1514594: Part 3b - Mass rewrite all JS code to use the new ChromeUtils.import API. rs=Gijs
This was done using the followng script:
https://bitbucket.org/kmaglione/m-c-rewrites/src/tip/processors/cu-import-exports.jsm
***
Bug 1514594: Part 3c - Update ESLint plugin for ChromeUtils.import API changes. r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D16747
***
Bug 1514594: Part 3d - Remove/fix hundreds of duplicate imports from sync tests. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16748
***
Bug 1514594: Part 3e - Remove no-op ChromeUtils.import() calls. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16749
***
Bug 1514594: Part 3f.1 - Cleanup various test corner cases after mass rewrite. r=Gijs
***
Bug 1514594: Part 3f.2 - Cleanup various non-test corner cases after mass rewrite. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16750
--HG--
extra : rebase_source : 359574ee3064c90f33bf36c2ebe3159a24cc8895
extra : histedit_source : b93c8f42808b1599f9122d7842d2c0b3e656a594%2C64a3a4e3359dc889e2ab2b49461bab9e27fc10a7
Summary: Really sorry for the size of the patch. It's mostly automatic
s/nsIDocument/Document/ but I had to fix up in a bunch of places manually to
add the right namespacing and such.
Overall it's not a very interesting patch I think.
nsDocument.cpp turns into Document.cpp, nsIDocument.h into Document.h and
nsIDocumentInlines.h into DocumentInlines.h.
I also changed a bunch of nsCOMPtr usage to RefPtr, but not all of it.
While fixing up some of the bits I also removed some unneeded OwnerDoc() null
checks and such, but I didn't do anything riskier than that.
The diagnostic assert (so fortunately, it doesn't impact release builds)
as added in bug 1405159, but is costly because it uses the modulus of
the division with a variable integer, which is a slow operation.
However, in arena_run_reg_dalloc, we end up doing the same diagnostic
assert, in a different form: after performing the division in a faster
manner, we assert that the result, multiplied by the diviser, returns
the original number.
Differential Revision: https://phabricator.services.mozilla.com/D13501
--HG--
extra : moz-landing-system : lando
- modify line wrap up to 80 chars; (tw=80)
- modify size of tab to 2 chars everywhere; (sts=2, sw=2)
--HG--
extra : rebase_source : 7eedce0311b340c9a5a1265dc42d3121cc0f32a0
extra : amend_source : 9cb4ffdd5005f5c4c14172390dd00b04b2066cd7
So that dmd.py isn't broken by function names that, after stack fixing, contain
escape-worthy chars such as " and /.
--HG--
extra : rebase_source : 625cef56e4599bbb810273ef04af2b971e777b58
Previously the id for a new arena was just a counter that increased by one
every time. For hardening purposes, we want to make private arenas use a secure
random ID, so an attacker will have a more difficult time finding the memory
they are looking for.
Differential Revision: https://phabricator.services.mozilla.com/D10158
--HG--
extra : moz-landing-system : lando
Previously the id for a new arena was just a counter that increased by one
every time. For hardening purposes, we want to make the new counter a secure
random ID, so an attacker will have a more difficult time finding the memory
they are looking for.
Differential Revision: https://phabricator.services.mozilla.com/D10158
--HG--
extra : moz-landing-system : lando
JSONWriter currently calls new and delete indirectly through mozilla::MakeUnique to allocate a buffer. Becuase of this, the methods of this class cannot be invoked within Spidermonkey due to https://searchfox.org/mozilla-central/source/config/check_vanilla_allocations.py#6-14. Therefore, JSONWriter needs an AllocPolicy template parameter so that the allocation and deallocation routines can be changed to match the JS AllocPolicy when invoked within SpiderMonkey.
Differential Revision: https://phabricator.services.mozilla.com/D7279
--HG--
extra : moz-landing-system : lando
On some builds we only get 'SmokeDMD' references during testing rather than a full file path. This adds 'SmokeDMD' to the acceptable frame description list.
--HG--
extra : rebase_source : 475166fa6d1f241bf4d67cbdda1aab73000a5c3a
extra : intermediate-source : fcdc1d5eaf1a932992287074d0c91504dd7180d8
extra : source : 1192c15fc9347ccd6169dcb5bea81d7f2399e3fd
On some builds we only get 'SmokeDMD' references during testing rather than a full file path. This adds 'SmokeDMD' to the acceptable frame description list.
--HG--
extra : source : 1192c15fc9347ccd6169dcb5bea81d7f2399e3fd
extra : intermediate-source : d39c2dfcc7004a4736436c564de239ba68b20c03
extra : histedit_source : 5a75a20f5b88481c994597da81fdbc29186aba3d%2C3f60f2bfbe54ab3843d4b18d66296cd4aadf4a30
On some builds we only get 'SmokeDMD' references during testing rather than a full file path. This adds 'SmokeDMD' to the acceptable frame description list.
--HG--
extra : rebase_source : 2b1d2dbf8a3a538533554c6697f0e9ee51ec1bed
extra : source : 1192c15fc9347ccd6169dcb5bea81d7f2399e3fd
On some builds we only get 'SmokeDMD' references during testing rather than a full file path. This adds 'SmokeDMD' to the acceptable frame description list.
--HG--
extra : rebase_source : 147e62e22248971980d95fe211c86116a98cbdf4
Entry storage allocation now occurs on the first lookupForAdd()/put()/putNew().
This removes the need for init() and initialized(), and matches how
PLDHashTable/nsTHashtable work. It also removes the need for init() functions
in a lot of types that are built on top of mozilla::Hash{Map,Set}.
Pros:
- No need for init() calls and subsequent checks.
- No memory allocated for empty tables, which are not that uncommon.
Cons:
- An extra branch in lookup() and lookupForAdd(), but not in put()/putNew(),
because the existing checkOverloaded() can handle it.
Specifics:
- Construction now can take a length parameter.
- init() is removed. Explicit length-setting, when necessary, now occurs in the
constructors.
- initialized() is removed.
- capacity() now returns zero when the entry storage is absent.
- lookupForAdd() is no longer `const`, because it can instantiate the storage,
which requires modifications.
- lookupForAdd() can now return an invalid AddPtr in two cases:
- old: hashing failure (due to OOM in the hasher)
- new: OOM while instantiating entry storage
The existing failure handling paths for the old case work for the new case.
- clear(), finish(), and clearAndShrink() are replaced by clear(), compact(),
and reserve(). The old compactIfUnderloaded() is also removed.
- Capacity computation code is now in its own functions, bestCapacity() and
hashShift(). setTableSizeLog2() is removed.
- uint32_t is used throughout for capacities, instead of size_t, for
consistency with other similar values.
- changeTableSize() now takes a capacity instead of a deltaLog2, and it can now
handle !mTable.
Measurements:
- Total source code size is reduced by over 900 lines. Also, lots of existing
lines got shorter (i.e. two checks were reduced to one).
- Executable size barely changed, down by 2 KiB on Linux64. The extra branches
are compensated for by the lack of init() calls.
- Speed changed negligibly. The instruction count for Bench_Cpp_MozHash
increased from 2.84 billion to 2.89 billion but any execution time change was
well below noise.
We're currently using NDK r15c, which is rather old, and happens to come
with a buggy gold linker. Let's use a more recent NDK, with a fixed
linker.
Unfortunately, we're currently at NDK API level 9, which the newer NDK
doesn't provide for x86 anymore. But that corresponds to Gingerbread
(2.3), which we've long stopped supporting. On the SDK side, we already
dropped support of versions before Jelly Bean, so we can do the same on
the NDK side. That corresponds to API level 16. So let's just use that
as a baseline.
Another change in the newer NDK is that the target-name changed from
i386-linux-android to i686-linux-android, so adjust for that in the
android x86 mozconfigs.
In PLDHashTable the equivalent functions have a "Shallow" prefix, which makes
it clear that they don't measure things hanging off the table. This patch makes
mozilla::Hash{Set,Map} do likewise.
MozReview-Commit-ID: 3kwCJynhW7d
--HG--
extra : rebase_source : 9c03d11f376a9fd4cfd5cfcdc0c446c00633b210
Also use mozilla::HashNumber where appropriate.
MozReview-Commit-ID: BTq0XDS5UfQ
--HG--
extra : rebase_source : 28f45a9b27e831e99620a2b575f373003f1301f2
Summary: GTest is permafailing on Windows because of timeout.
Reviewers: glandium
Reviewed By: glandium
Bug #: 1474254
Differential Revision: https://phabricator.services.mozilla.com/D2043
--HG--
extra : rebase_source : cea68e50f96a1788bf15dc6ca7859e5f698a6209
Ricky Stewart
Sylvestre Ledru
Cosmin Sabou
Nicholas Nethercote
pbz
Csoregi Natalia
Bogdan Tara
André Bargull
Simon Giesecke
Andrew McCreight
Mark Banner
James Willcox
Ciure Andrei
Masatoshi Kimura
Coroiu Cristina
Mike Hommey
Emilio Cobos Álvarez
Brian Hackett
Tom Ritter
Andreea Pavel
Gian-Carlo Pascutto
Christian Holler
Nathan Froyd
Cameron McCormack
Mihai Alexandru Michis
Jed Davis
Victor Porof
Nick Thomas
Mike Hommey
Chris Martin
Edwin Gao
Narcis Beleuzu
Geoff Brown
Petr Sumbera
Gurzau Raul
David Major
Ryan Hunt
Ehsan Akhgari
Jeff Gilbert
Myk Melez
Kris Maglione
Tooru Fujisawa
Benjamin Bouvier
arthur.iakab
Noemi Erli
Denis Palmeiro
Randell Jesup
Eric Rahm
Calixte Denizet