Граф коммитов

425 Коммитов

Автор SHA1 Сообщение Дата
Mike Hommey 4b17f4882c Bug 1403444 - Apply clang-format to the rb.h macros. r=njn
--HG--
extra : rebase_source : d1a55373811fea242c5b91666ba545532e6bbdde
2017-09-25 06:59:30 +09:00
Mike Hommey f43e83a278 Bug 1403444 - Use a fixed size for the stack space used during rb_foreach. r=njn
That stack space would matter if recursion was involved, but there
isn't any, and a max of 1440 bytes temporarily allocated on the stack
is not really a problem.

--HG--
extra : rebase_source : 2968fafe9d604d9e6c03ac93c21d8a3a087043a4
2017-09-25 06:57:09 +09:00
Mike Hommey bba7d810d3 Bug 1403444 - Make the "static" part of what the rb_wrap macro expands to.. r=njn
All uses of rb_wrap have "static" as first argument to rb_wrap, move that
in the macro itself.

--HG--
extra : rebase_source : cbfe87d0539452c044b415c725cb7ce6ebb5628c
2017-09-03 06:49:39 +09:00
Mike Hommey b8a3c5fa7f Bug 1403444 - Add getters and setters on RedBlackTreeNode. r=njn
--HG--
extra : rebase_source : db31bf584071164346463cd54ca359547fcd0eb6
2017-09-02 20:26:09 +09:00
Mike Hommey 02f0da2a19 Bug 1403444 - Use templates for rb_node and rb_tree, and rename them. r=njn
--HG--
extra : rebase_source : ab470e80a786d290f0df61f4adf2c128f9ecb925
2017-09-02 09:05:13 +09:00
Mike Hommey 77ff4db6d7 Bug 1402647 - Add a memalign implementation on platforms that don't have one. r=njn
--HG--
extra : rebase_source : bf1ff9da4c4647f5e930194d0d008466f0b2b593
2017-09-26 06:59:03 +09:00
Mike Hommey d7703e388f Bug 1402647 - Add missing stdlib.h header for system allocator. r=njn
--HG--
extra : rebase_source : caa72506b79e02f953911ab859244d9f9909f389
2017-09-26 06:58:05 +09:00
Mike Hommey 633a0d02d9 Bug 1402174 - Initial actual implementation of the moz_arena_* API. r=njn
Things left for followups:
- Full cleanup of disposed arenas: bug 1364359.
- Random arena Ids: bug 1402282.
- Enforcing the arena to match on moz_arena_{realloc,free}: bug 1402283.
- Make it impossible to use arenas not created with moz_create_arena
  with moz_arena_* functions: bug 1402284.

Until it's proven to require a different data structure, arena lookup by
Id is done through the use of the same RB tree used everywhere else in
the allocator.

At this stage, the implementation of the API doesn't ride the trains,
but the API can be used safely and will fall back to normal allocations
transparently for the caller.

--HG--
extra : rebase_source : aaa9bdab5b4e0c534da0c9c7a299028fc8d66dc8
2017-09-21 14:24:37 +09:00
Mike Hommey 3efd8d6f3a Bug 1402174 - Add a helper class implementing the base allocator functions for a given arena. r=njn
--HG--
extra : rebase_source : 7ddcabf5385f2fc975cf44e4eb6ccdc890d6b7e2
2017-09-21 14:24:37 +09:00
Mike Hommey 46c03f6281 Bug 1402174 - Add an arena argument to imalloc, ipalloc and iralloc. r=njn
--HG--
extra : rebase_source : 60c1ef94e4014f7ef688263541653529e63216d8
2017-09-21 13:58:17 +09:00
Mike Hommey c6730d314b Bug 1402174 - Merge imalloc and icalloc into a single function. r=njn
--HG--
extra : rebase_source : ee06bd77753e94503ce207db8608515a27ef8ea4
2017-09-21 13:23:22 +09:00
Mike Hommey e9474c957e Bug 1402174 - Move AlignedAllocator around, so that calloc, realloc and free and grouped with malloc and memalign. r=njn
--HG--
extra : rebase_source : 139ffe643ca4c6e21f1426c06e92c5d80b32eb3d
2017-09-21 11:46:57 +09:00
Sebastian Hengst 836fbd6f31 Backed out changeset 14dac365b5f6 (bug 1402174) for frequently asserting rbp_i_cmp > 0, at mozjemalloc.cpp:2381. r=backout 2017-09-22 16:48:08 +02:00
Sebastian Hengst 55910de1c7 Backed out changeset 5bba0584a7d8 (bug 1402174) 2017-09-22 16:47:25 +02:00
Sebastian Hengst a42fb039a7 Backed out changeset e356ac32e297 (bug 1402174) 2017-09-22 16:47:19 +02:00
Sebastian Hengst 4af0b5f14d Backed out changeset c589ad71d9ca (bug 1402174) 2017-09-22 16:47:14 +02:00
Sebastian Hengst d6967d0dd4 Backed out changeset b23c870c7e45 (bug 1402174) 2017-09-22 16:47:08 +02:00
Mike Hommey 480bbbd368 Bug 1402174 - Initial actual implementation of the moz_arena_* API. r=njn
Things left for followups:
- Full cleanup of disposed arenas: bug 1364359.
- Random arena Ids: bug 1402282.
- Enforcing the arena to match on moz_arena_{realloc,free}: bug 1402283.
- Make it impossible to use arenas not created with moz_create_arena
  with moz_arena_* functions: bug 1402284.

Until it's proven to require a different data structure, arena lookup by
Id is done through the use of the same RB tree used everywhere else in
the allocator.

At this stage, the implementation of the API doesn't ride the trains,
but the API can be used safely and will fall back to normal allocations
transparently for the caller.

--HG--
extra : rebase_source : 089e4cbb62c239713f40763ab819c79e5cbe28ce
2017-09-21 14:24:37 +09:00
Mike Hommey 5a1dbfeca3 Bug 1402174 - Add a helper class implementing the base allocator functions for a given arena. r=njn
--HG--
extra : rebase_source : c4eb19c295a0f1524024b8d2e6c7a0ade92b23fa
2017-09-21 14:24:37 +09:00
Mike Hommey 9ceb2cae10 Bug 1402174 - Add an arena argument to imalloc, ipalloc and iralloc. r=njn
--HG--
extra : rebase_source : d44937b08fb61b15e729b0a3fc508921fb96d027
2017-09-21 13:58:17 +09:00
Mike Hommey 74a50e4e4e Bug 1402174 - Merge imalloc and icalloc into a single function. r=njn
--HG--
extra : rebase_source : e6a17a3506ef4dddad13ab6e211b269b46ef99a8
2017-09-21 13:23:22 +09:00
Mike Hommey 494f947a18 Bug 1402174 - Move AlignedAllocator around, so that calloc, realloc and free and grouped with malloc and memalign. r=njn
--HG--
extra : rebase_source : 1973f90f98a19b4e60f742b7f3aef307bb304e7d
2017-09-21 11:46:57 +09:00
Mike Hommey 5cd3519571 Bug 1052573 - Add an API for allocation in separate arenas. r=njn
The implementation is not doing anything just yet. This will be done in
a followup bug.

--HG--
extra : rebase_source : e301eac77c6bd8247c09d369074ecb8d7b5a1a2f
2017-09-22 07:22:38 +09:00
Mike Hommey c43db36061 Bug 1052573 - Move macro helpers to mozjemalloc.h. r=njn
--HG--
extra : rebase_source : de717a2ddb13afc29ec3d795fbf1ae72b6ed1d26
2017-09-21 14:10:06 +09:00
Mike Hommey 21a0a419d0 Bug 1052573 - Separate the base allocator functions in malloc_decls.h. r=njn
malloc, free, calloc, realloc and memalign constitute some sort of
minimal interface to the allocator. posix_memalign, aligned_alloc and
valloc are already defined in terms of memalign. The remaining functions
are not related to active allocation.

--HG--
extra : rebase_source : ee27ca70e271f3abef76c7782724d607b52f58b1
2017-09-21 10:31:09 +09:00
Mike Hommey 00433d9cf4 Bug 1401099 - Move arena_ralloc_large_grow to a method of arena_t. r=njn
--HG--
extra : rebase_source : 0a1863785b2c3a22946ee735c73afa5ac764f076
2017-09-15 20:50:42 +09:00
Mike Hommey 3c2c85e407 Bug 1401099 - Move arena_ralloc_large_shrink to a method of arena_t. r=njn
--HG--
extra : rebase_source : 3b9aede58b3c9163b8a28a4371120d471533f8eb
2017-09-15 20:44:34 +09:00
Mike Hommey 24adb9a7a8 Bug 1401099 - Move arena_dalloc_large to a method of arena_t. r=njn
--HG--
extra : rebase_source : dbc79744f42ed99709ed9857ec5d91ceaea83b0d
2017-09-15 20:40:36 +09:00
Mike Hommey eea012c8aa Bug 1401099 - Move arena_dalloc_small to a method of arena_t. r=njn
--HG--
extra : rebase_source : 9cc4efbbcd0da012be9fa4e5dd3262d89bdab161
2017-09-15 20:37:47 +09:00
Mike Hommey 05188caf59 Bug 1401099 - Move arena_palloc to a method of arena_t. r=njn
--HG--
extra : rebase_source : 0c64ea5c0681704a85e15977a8803403d06d0962
2017-09-15 20:28:23 +09:00
Mike Hommey 91f0d70f6f Bug 1401099 - Move arena_malloc to a method of arena_t. r=njn
--HG--
extra : rebase_source : c06e3c5a63a12ba48f7f4bc52390ba9c8670f722
2017-09-15 19:20:09 +09:00
Mike Hommey cd573d5abc Bug 1401099 - Move arena_malloc_large to a method of arena_t. r=njn
--HG--
extra : rebase_source : 837d0d67556097140ff6141f2876fdbbb70d4e08
2017-09-15 19:14:00 +09:00
Mike Hommey 3576cd2578 Bug 1401099 - Move arena_malloc_small to a method of arena_t. r=njn
--HG--
extra : rebase_source : 7df0043060caf4cd14fc48428296428acf1771c7
2017-09-15 19:11:52 +09:00
Mike Hommey 2c4099f29f Bug 1401099 - Move arena_bin_nonfull_run_get to a method of arena_t. r=njn
--HG--
extra : rebase_source : 808f7ba52a4fc4f99a283ae894296cafac5166da
2017-09-15 18:23:33 +09:00
Mike Hommey 003004a71a Bug 1401099 - Move arena_bin_malloc_hard to a method of arena_t. r=njn
--HG--
extra : rebase_source : 2bed3221e38714d00ab95937d20ac5faafc89e9e
2017-09-15 18:20:11 +09:00
Mike Hommey 577b65d3c1 Bug 1401099 - Move arena_bin_malloc_easy to a method of arena_t. r=njn
--HG--
extra : rebase_source : 8675de21e7c79e3c458367f3f10b1331efe55094
2017-09-15 18:18:11 +09:00
Mike Hommey 0ec33d017f Bug 1401099 - Move arena_run_trim_tail to a method of arena_t. r=njn
--HG--
extra : rebase_source : e105fa4b8aae6b41c12e2f72293cdf68489a939d
2017-09-15 18:14:33 +09:00
Mike Hommey 3cc5f23c76 Bug 1401099 - Move arena_run_trim_head to a method of arena_t. r=njn
--HG--
extra : rebase_source : 2a2d2b9d454d69d0d1924f60d4771b39a1dddfb2
2017-09-15 18:11:12 +09:00
Mike Hommey b6e66b7b02 Bug 1401099 - Move arena_run_split to a method of arena_t. r=njn
--HG--
extra : rebase_source : 919dc2b9980ae59340553bcbfa029ba9c1bf9479
2017-09-15 18:08:23 +09:00
Mike Hommey 99d28b4355 Bug 1401099 - Move arena_run_dalloc to a method of arena_t. r=njn
--HG--
extra : rebase_source : 5cf37d39b6d8bdb591352051b3f37a2e3982dc69
2017-09-15 18:01:27 +09:00
Mike Hommey 7cb9914a15 Bug 1401099 - Move arena_run_alloc to a method of arena_t. r=njn
--HG--
extra : rebase_source : 6a683f4d0cf5ad68c670dbe8ea9d3a34acf11549
2017-09-15 17:57:11 +09:00
Mike Hommey d2ba03c881 Bug 1401099 - Move arena_chunk_dealloc to a method of arena_t. r=njn
--HG--
extra : rebase_source : 973b8764903f4e9c65aaaa053bb394a8a9ff2acb
2017-09-15 17:50:48 +09:00
Mike Hommey d6d578553d Bug 1401099 - Move arena_chunk_init to a method of arena_t. r=njn
--HG--
extra : rebase_source : d30e062f7117743a0c88fdf0325b0a395bff5658
2017-09-15 17:43:36 +09:00
Mike Hommey 11cfd193d1 Bug 1401099 - Move arena_new to a method of arena_t. r=njn
--HG--
extra : rebase_source : d0a9a0cebb4e8a1a74c5a48e42fb952310798604
2017-09-15 17:38:58 +09:00
Mike Hommey beb406f1a7 Bug 1401099 - Move hard_purge_arena to a method of arena_t. r=njn
--HG--
extra : rebase_source : af20d24ba0dd8ce6c9e42703688c1bca98594b4d
2017-09-15 17:34:53 +09:00
Mike Hommey a980219f16 Bug 1401099 - Move arena_purge to a method of arena_t. r=njn
--HG--
extra : rebase_source : d48623ce7ea7be5489c89a575bc1409de34e834a
2017-09-15 17:32:21 +09:00
Mike Hommey cb3efd88ef Bug 1401099 - Use Gecko style names for arena_t members. r=njn
--HG--
extra : rebase_source : cefcbc29323145557faed6c676867500c21c6ba4
2017-09-15 17:20:01 +09:00
Mike Hommey 707bc83a01 Bug 1401875 - Replace MALLOC_DECL_VOID with a clever use of templates. r=njn
This effectively means malloc_hook_table_t is now C++ only, which is not
a big problem.

This also makes some functions use a return construct with functions
that don't return a value (such as free). While that is not allowed in
ISO C, it's allowed in C++, so the simplification is welcome (although,
retrospectively, it turns out C compilers don't complain about it
without -pedantic).

--HG--
extra : rebase_source : defd88ca3f6d478e61a4b970393dba60fb6ca81d
2017-09-21 15:27:12 +09:00
Mike Hommey 4a2740def4 Bug 1401453 - Don't keep libmemory.a separate anymore. r=gps
This was done in bug 736564 for the xulrunner SDK, which later became
the firefox SDK, which is now gone. So we don't actually need to keep it
separate anymore (except for logalloc/replay, which still needs to link
it directly, so we keep the library definition intact so it can be
referenced ; we just don't DIST_INSTALL it anymore, and always make it
linked into mozglue)

--HG--
extra : rebase_source : e4d0627ec907fe0139df5c0b2b9f7d04b43c7c78
2017-09-20 14:23:57 +09:00
Mike Hommey 7bbdb00059 No bug - Remove a comment that has no significance after bug 1395776. r=me 2017-09-20 11:53:00 +09:00
Mike Hommey 078c8d1896 Bug 1399921 - Register zone allocator independently, and delay jemalloc initialization on mac. r=njn
In bug 1361258, we unified the initialization sequence on mac, and
chose to make the zone registration happen after jemalloc
initialization.

The order between jemalloc init and zone registration shouldn't actually
matter, because jemalloc initializes the first time the allocator is
actually used.

On the other hand, in some build setups (e.g. with light optimization),
the initialization of the thread_arena thread local variable can happen
after the forced jemalloc initialization because of the order the
corresponding static initializers run. In some levels of optimization,
the thread_arena initializer resets the value the jemalloc
initialization has set, which subsequently makes choose_arena() return
a bogus value (or hit an assertion in ThreadLocal.h on debug builds).

So instead of initializing jemalloc from a static initializer, which
then registers the zone, we instead register the zone and let jemalloc
initialize itself when used, which increases the chances of the
thread_arena initializer running first.

--HG--
extra : rebase_source : 4d9a5340d097ac8528dc4aaaf0c05bbef40b59bb
2017-09-15 07:34:48 +09:00
Mike Hommey 50182c9f53 Bug 1400146 - Gracefully handle the allocator not being initialized in isalloc_validate. r=njn
isalloc_validate is the function behind malloc_usable_size. If for some
reason malloc_usable_size is called before mozjemalloc is initialized,
this can lead to an unexpected crash.

The chance of this actually happening is rather slim on Linux
and Windows (although still possible), and impossible on Mac, due to the
fact the earlier something can end up calling it is after the
mozjemalloc zone is registered, which happens after initialization.

... except with bug 1399921, which reorders that initialization, and
puts the zone registration first. There's then a slim chance for the
zone allocator to call into zone_size, which calls malloc_usable_size,
to determine whether a pointer allocated by some other zone belongs to
mozjemalloc's.

And it turns out that does happen, during the startup of the
plugin-container process on OSX 10.10 (but not more recent versions).

--HG--
extra : rebase_source : 331d093b03add7b2c2ce440593f5aeccaaf4dd1f
2017-09-15 15:13:52 +09:00
Mike Hommey f3bff41d59 Bug 1400096 - Don't define the operator new/delete functions as mangled in mozmem_wrap.cpp. r=njn
Now that this is a C++ file, and that the function names are not
mangled, we can just use the actual C++ names.

We do however need to replace MOZ_MEMORY_API, which implies extern "C",
with MFBT_API.

Also use the correct type for the size given to operator new. It
happened to work before because the generated code would just jump to
malloc without touching any register, but on aarch64, unsigned int was
the wrong type.

--HG--
extra : rebase_source : 8045f30e9c609dd7d922c77d85ac017638df6961
2017-09-15 10:28:33 +09:00
Mike Hommey 9741c8e076 Bug 1400096 - Build mozmemory_wrap as C++. r=njn
And since the build system doesn't handle transitions from foo.c to
foo.cpp properly without a clobber, we work around the issue by
switching to unified sources.

--HG--
rename : memory/build/mozmemory_wrap.c => memory/build/mozmemory_wrap.cpp
extra : rebase_source : 3f074b4ccab255bb0eb16841f79582060fafbc86
2017-09-15 10:19:37 +09:00
Mike Hommey 747992864b Bug 1400096 - Annotate mozmemory_wrap.c windows functions as MOZ_MEMORY_API. r=njn
It happens to work because of mozglue.def, but we might as well have the
right annotations (which will also make things correct when building this
file to C++)

--HG--
extra : rebase_source : 61056dc21c9c29bab62ad5d648e94dd56dc53b14
2017-09-15 10:12:24 +09:00
Mike Hommey 57ecf449b4 Bug 1400096 - Remove mozmem_malloc_impl wrapping for operator methods exposed in mozmemory_wrap.c. r=njn
This used to be necessary because those functions might be prefixed with
__wrap_, and linked against with -Wl,--wrap, but that's not been the case
since bug 1077366. Furthermore, mozmem_malloc_impl nowadays only does
something on Windows, and those operators are only exposed on Android.

--HG--
extra : rebase_source : ca34442bfbc5fc8be20ffcfacb9afa0f2f818b82
2017-09-15 09:54:01 +09:00
Phil Ringnalda eab70d0a07 Backed out changeset 0e349b74bfc6 (bug 1399921) for failure to thrive on Mac
MozReview-Commit-ID: HoBhxzIzn2d
2017-09-14 20:36:41 -07:00
Mike Hommey c53e695a6e Bug 1399921 - Register zone allocator independently, and delay jemalloc initialization on mac. r=njn
In bug 1361258, we unified the initialization sequence on mac, and
chose to make the zone registration happen after jemalloc
initialization.

The order between jemalloc init and zone registration shouldn't actually
matter, because jemalloc initializes the first time the allocator is
actually used.

On the other hand, in some build setups (e.g. with light optimization),
the initialization of the thread_arena thread local variable can happen
after the forced jemalloc initialization because of the order the
corresponding static initializers run. In some levels of optimization,
the thread_arena initializer resets the value the jemalloc
initialization has set, which subsequently makes choose_arena() return
a bogus value (or hit an assertion in ThreadLocal.h on debug builds).

So instead of initializing jemalloc from a static initializer, which
then registers the zone, we instead register the zone and let jemalloc
initialize itself when used, which increases the chances of the
thread_arena initializer running first.

--HG--
extra : rebase_source : 4d9a5340d097ac8528dc4aaaf0c05bbef40b59bb
2017-09-15 07:34:48 +09:00
Mike Hommey e4f4d5fc59 Bug 1400063 - Automatically declare jemalloc_* functions in mozmemory.h. r=njn
There is a lot of churn involved in adding new API surface to
mozjemalloc, and mozmemory.h is one. Instead of declaring
everything manually in there, "generate" the declarations through
malloc_decls.h.

--HG--
extra : rebase_source : 1416fa972319c419112c4a8b16759d90692db5b2
2017-09-14 18:28:12 +09:00
Mike Hommey 9f3c42a57e Bug 1397101 - Only use a thread local arena for small sizes. r=njn
The bin-unused count in memory reports indicates how much memory is
used by runs of small and sub-page allocations that is not actually
allocated. This is generally thought as an indicator of fragmentation.

While this is generally true, with the use of thread local arenas by
stylo, combined with how stylo allocates memory, it ends up also being
an indicator of wasted memory.

For instance, over the lifetime of an AWSY iteration, there are only a
few allocations that ends up in the bucket for 2048 allocated bytes. In
the "worst" case, there's only one. But the run size for such
allocations is 132KiB. Which means just because we're allocating one
buffer of size between 1024 and 2048 bytes, we end up wasting 130+KiB.
Per thread.

Something similar happens with size classes of 512 and 1024, where the
run size is respectively 32KiB and 64KiB, and where there's at most a
handful of allocations of each class ever happening per thread.

Overall, an allocation log from a full AWSY iteration reveals that there
are only 448 of 860700 allocations happening on the stylo arenas that
involve sizes above (and excluding) 512 bytes, so 0.05%.

While there are improvements that can be done to mozjemalloc so that it
doesn't waste more than one page per sub-page size class, they are
changes that are too low-level to land at this time of the release
cycle. However, considering the numbers above and the fact that the
stylo arenas are only really meant to avoid lock contention during the
heavy parallel work involved, a short term, low risk, strategy is to
just delegate all sub-page (> 512, < 4096) and large (>= 4096) to the
main arena. Technically speaking, only sub-page allocations are causing
this waste, but it's more consistent to just delegate everything above
512 bytes.

This should save 132KiB + 64KiB = 196KiB per stylo thread.

--HG--
extra : rebase_source : c7233d60305365e76aa124045b1c9492068d9415
2017-09-14 07:36:39 +09:00
Mike Hommey 965981b68d Bug 1397101 - Reduce the number of dirty pages we allow to be kept in thread local arenas. r=njn
Until bug 1361258, there was only ever one mozjemalloc arena, and the
number of dirty pages we allow to be kept dirty, fixed to 1MB per arena,
was, in fact, 1MB for an entire process.

With stylo using thread local arenas, we now can have multiple arenas
per process, multiplying that number of dirty pages.

While those dirty pages may be reused later on, when other allocations
end up filling them later on, the fact that a relatively large number of
them is kept around for each stylo thread (in proportion to the amount of
memory ever allocated by stylo), combined with the fact that the memory
use from stylo depends on the workload generated by the pages being
visited, those dirty pages may very well not be used for a rather long
time. This is less of a problem with the main arena, used for most
everything else.

So, for each arena except the main one, we decrease the number of dirty
pages we allow to be kept around to 1/8 of the current value. We do this
by introducing a per-arena configuration of that maximum number.

--HG--
extra : rebase_source : 75eebb175b3746d5ca1c371606cface50ec70f2f
2017-09-14 07:26:30 +09:00
Mike Hommey f98e560253 Bug 1399031 - Use mozilla/ThreadLocal.h in mozjemalloc. r=njn
--HG--
extra : rebase_source : 46e7abf6f46939cfa0862930feea5f245935b8d4
2017-09-12 16:29:11 +09:00
Mike Hommey 1e26a193e9 Bug 1399031 - Remove the setting of NO_TLS when PIC is not defined in mozjemalloc: it's always defined. r=njn
But it's also useless besides that test, so remove the PIC define.

--HG--
extra : rebase_source : 87b8934c1f8b6ed94e5ba3e2150f92dab9285769
2017-09-12 16:06:56 +09:00
Mike Hommey 05e3823899 Bug 1399350 - Remove jemalloc_*_impl macros. r=njn
Those macros are one more thing that needs to be added when the
mozjemalloc API surface is increased, but after bug 1399350, nothing
actually needs them, so remove them.

--HG--
extra : rebase_source : 2bf62cc6c179540482722a72b0d0c134d2ac2a19
2017-09-13 14:25:21 +09:00
Mike Hommey 8cdad036f1 Bug 1398965 - Fold memory/mozjemalloc into memory/build. r=njn
The files relevant to the memory allocator are currently spread between
memory/mozjemalloc and memory/build, and the distinction was
historically from sharing some Mozilla-specific things between
mozjemalloc and jemalloc3. That distinction is not useful anymore, so
we fold everything together.

As we will likely rename the allocator at some point in the future, it
is preferable to move away from the mozjemalloc directory rather than in
its direction.

--HG--
rename : memory/mozjemalloc/Makefile.in => memory/build/Makefile.in
rename : memory/mozjemalloc/mozjemalloc.cpp => memory/build/mozjemalloc.cpp
rename : memory/mozjemalloc/mozjemalloc.h => memory/build/mozjemalloc.h
rename : memory/mozjemalloc/mozjemalloc_types.h => memory/build/mozjemalloc_types.h
rename : memory/mozjemalloc/rb.h => memory/build/rb.h
2017-09-12 13:14:35 +09:00
Mike Hommey 175ddd015f Bug 1398965 - Remove MOZ_JEMALLOC_IMPL in favor of MOZ_MEMORY_IMPL. r=njn
In practice, they mean the same thing, and there's only one place where
MOZ_JEMALLOC_IMPL was used.
2017-09-12 13:06:43 +09:00
Nathan Froyd c88ec30451 Bug 1396892 - turn off -Wtautological-pointer-compare in memory/build/ for Android; r=njn
clang warns about this code in mozmemory_wrap.c in the reimplementation
of vasprintf, complaining that `fmt` cannot be null:

    if (str == NULL || fmt == NULL) {
                       ^~~    ~~~~

clang is apparently exploiting knowledge about the requirements of
vasprintf here, but defensive programming on the part of our
reimplementation seems like the wiser course.  Let's turn off the
warning.
2017-09-07 08:20:12 -04:00
Mike Hommey acb85e2b41 Bug 1396361 - Avoid crashing when some system library calls malloc_zone_free(zone, NULL). r=njn
Some system libraries call malloc_zone_free directly instead of free,
and sometimes they do that with the wrong zone. When that happens, we
circle back, trying to find the right zone, and call malloc_zone_free
with the right one, but when we can't find one, we crash, which matches
what the system free() would do. Except in one case where the pointer
we're being passed is NULL, in which case we can't trace it back to any
zone, but shouldn't crash (system free() explicitly doesn't crash in
that case).

--HG--
extra : rebase_source : 17efdcd80f1a53be7ab6b7293bfb6060a9aa4a48
2017-09-04 07:32:42 +09:00
Mike Hommey e77f51a721 Bug 1395776 - Fold replace-malloc into mozjemalloc. r=njn
Add the MPL 2.0 license header per bug 1395449.

--HG--
extra : rebase_source : c85544a178aba7e77b5b8031851b8e16b22a3848
2017-08-31 12:02:01 +09:00
Mike Hommey dfc6786a68 Bug 1395776 - Move usable_ptr_t definition to mozjemalloc_types.h. r=njn
Because malloc_decls.h is meant to be included multiple times, it
shouldn't actually declare things itself.

--HG--
extra : rebase_source : 9d6f9b2c61407265377845963a19ace2614160f4
2017-08-31 09:51:23 +09:00
Mike Hommey b0303d9e77 Bug 1395776 - Uniformize posix_memalign implementations in replace-malloc and mozjemalloc. r=njn
--HG--
extra : rebase_source : bbbee18feef2d11b71d5d5a48e98017bfae78722
2017-08-31 11:35:24 +09:00
Sebastian Hengst 43bc951ac7 merge mozilla-inbound to mozilla-central. r=merge a=merge
MozReview-Commit-ID: ES9rKhiQo10
2017-09-01 10:38:51 +02:00
Nicholas Nethercote c419d5fa48 Bug 1389305 (attempt 2) - Add jemalloc_ptr_info() and moz_malloc_enclosing_size_of(). r=glandium.
--HG--
extra : rebase_source : 93a6cfcc916fb239581c2892f24b24c6fc65ac71
2017-09-01 11:52:23 +10:00
Sebastian Hengst 4310ea4600 Backed out changeset f232b5b1a0c7 (bug 1389305) for frequently failing GTest Jemalloc.PtrInfo on Linux opt. r=backout 2017-08-31 13:52:48 +02:00
Nicholas Nethercote 365285b831 Bug 1389305 - Add jemalloc_ptr_info() and moz_malloc_enclosing_size_of(). r=glandium.
jemalloc_ptr_info() gives info about any pointer, such as whether it's within a
live or free allocation, and if so, info about that allocation. It's useful for
debugging.

moz_malloc_enclosing_size_of() uses jemalloc_ptr_info() to measure the size of
an allocation from an interior pointer. It's useful for memory reporting,
especially for Rust code.

--HG--
extra : rebase_source : caa19cccf8c2d1f79cf004fe6a408775de5a7b22
2017-08-24 19:37:27 +10:00
Mike Hommey 57c65279a0 Bug 1395088 - Remove the jemalloc_bool type. r=njn
Back when it was added (for Windows CE, in bug 488608), mozjemalloc was
C and all the supported compilers didn't support C99 bools. Now
mozjemalloc is C++, and all the supported compilers support C99 bools
for the cases where the type is used from C.

--HG--
extra : rebase_source : b9c710a0c48dc36cb473af59e3119131d13523ce
2017-08-30 18:04:47 +09:00
Mike Hommey 0eb2652011 Bug 1395063 - Use MOZ_CRASH directly in mozjemalloc. r=njn
Back when mozjemalloc was considered third-party, and before bug
1365194, mozjemalloc was calling abort() and that was redirectory to
MOZ_CRASH through a moz_abort() function. Bug 1365194 changed that so
that moz_abort is called directly instead of abort, but the indirection
is actually not necessary anymore.

So we just kill moz_abort, which is unused anywhere else, and use
MOZ_CRASH directly.

Note this will (obviously) change crash signatures involving moz_abort.

--HG--
extra : rebase_source : 67698ffd8c5e52e62b9a0b7f28efb0352c8fe8ce
2017-08-30 16:20:08 +09:00
Mike Hommey 4aaef7b18a Bug 1395032 - Remove remainder of the VS CRT allocator mismatch hack. r=gps
Bug 1186064 removed most of it when we started requiring VS 2015u2, but
the "frex" function exported through mozglue.def.in was only used
through the MSVCRT being patched by fixcrt.py, which is not done anymore.
So the "frex" export is not used anymore, and so the "dumb_free_thunk"
function is not used anymore as well.

--HG--
extra : rebase_source : 879c469c317c8b6749410a4a476d6c951c9a1d0f
2017-08-30 14:06:13 +09:00
Eric Rahm 0938982c90 Bug 1389598 - Part 4: Remove remaining gonk refs. r=froydnj
--HG--
extra : rebase_source : 063c7f95dda063eafabfa1921366bd1957b8fe73
2017-08-11 17:45:18 -07:00
Nicholas Nethercote 7d699f9fa4 Bug 1382099 - Remove MOZ_WIDGET_GONK from memory/. r=glandium.
--HG--
extra : rebase_source : 641de85945cb2f0ccdb9d6f5207bdd22b597b318
2017-07-19 18:39:00 +10:00
Mike Hommey 19448fe709 Bug 1356701 - Export unprefixed malloc and duplication functions on OSX. r=njn
Going through the system zone allocator for every call to realloc/free
on OSX is costly, because the zone allocator needs to first verify that
the allocations do belong to the allocator it invokes (which ends up
calling jemalloc's malloc_usable_size), which is unnecessary when we
expect the allocations to belong to jemalloc.

So, we export the malloc/realloc/free/etc. symbols from
libmozglue.dylib, such that libraries and programs linked against it
call directly into jemalloc instead of going through the system zone
allocator, effectively shortcutting the allocator verification.

The risk is that some things in Gecko try to realloc/free pointers it
got from system libraries, if those were allocated with a system zone
that is not jemalloc.

--HG--
extra : rebase_source : ee0b29e1275176f52e64f4648dfa7ce25d61292e
2017-07-04 15:01:50 +09:00
Ryan VanderMeulen 5e1e8d2f24 Backed out changeset 261d61f4eeec (bug 1356701) for causing bug 1378339. 2017-07-05 12:58:39 -04:00
Mike Hommey c65bedc1c8 Bug 1356701 - Export unprefixed malloc and duplication functions on OSX. r=njn
Going through the system zone allocator for every call to realloc/free
on OSX is costly, because the zone allocator needs to first verify that
the allocations do belong to the allocator it invokes (which ends up
calling jemalloc's malloc_usable_size), which is unnecessary when we
expect the allocations to belong to jemalloc.

So, we export the malloc/realloc/free/etc. symbols from
libmozglue.dylib, such that libraries and programs linked against it
calls directly into jemalloc instead of going through the system zone
allocator, effectively shortcutting the allocator verification.

The risk is that some things in Gecko try to realloc/free pointers it
got from system libraries, if those were allocated with a system zone
that is not jemalloc.

--HG--
extra : rebase_source : 45b9b98499760a7f946878d41d2fdaadb6dff4d6
2017-07-04 15:01:50 +09:00
Mike Hommey 4ae1509807 Bug 1368932 - Handle missing replace_posix_memalign at the replace-malloc level. r=njn
Replace-malloc libraries, such as DMD, don't really need to care about
the details of implementing all the variants of aligned memory
allocation functions. Currently, by defining MOZ_REPLACE_ONLY_MEMALIGN
before including replace_malloc.h, they get predefined functions.

Instead of making that an opt-in at build time, we make the
replace-malloc initialization just fill the replace-malloc
malloc_table_t with implementations that rely on the replace_memalign
the library provides.

--HG--
extra : rebase_source : 0842a67d9bc27a9a86c33d14d98b9c25f39982fb
2017-05-31 13:47:17 +09:00
Mike Hommey 7fd2c94e79 Bug 1368932 - Move the replace_malloc_init_funcs function around. r=njn
--HG--
extra : rebase_source : 4e3b3bbd18a37a3ee01d6ec30449249be5481b77
2017-05-31 13:47:11 +09:00
Mike Hommey 565f74102e Bug 1368932 - Fill the replace-malloc malloc_table_t with the real allocator as a fallback. r=njn
Until now, the malloc implementation functions would call the
replace-malloc functions if they exist, and fallback to the real
allocator in no such function exists. Instead of doing this, we now
fill the empty slots in the malloc_table_t with the real allocator
functions.

--HG--
extra : rebase_source : b54634f23188906939e4dc01fc5a3007de0f3f2c
2017-05-30 15:57:28 +09:00
Mike Hommey 1a6ac2f8fb Bug 1368932 - Use a malloc_table_t for most replace-malloc function pointers, on all platforms. r=njn
We make replace_malloc_init_funcs called on all platforms and fill out a
malloc_table_t for the replace-malloc functions with what comes from
dlsym/GetProcAddress on Android/Windows, and from the dynamically linked
weak symbols replace_* on other platforms.

replace_malloc.h contains definitions of *_impl_t types for each of the
functions in the malloc_table_t, which is redundant with the
replace_*_impl_t types we were creating, so we remove those typedefs,
except for the two functions (init and get_bridge) that don't have such
a typedef. Those functions don't appear in malloc_table_t.

--HG--
extra : rebase_source : 3705a99ee07f63dbaa66973eef19ddab224e0911
2017-05-30 15:57:28 +09:00
Mike Hommey 409495f283 Bug 1368932 - Refactor such that there is only one definition of replace_malloc_init_funcs. r=njn
We want, in a subsequent patch, to have replace_malloc_init_funcs be
called on all platforms (including those relying on the replace-malloc
library being loaded already) and perform more initialization.

To prepare for that, we move the non-platform-specific pieces out.

--HG--
extra : rebase_source : 239ed363ee168bf4f8a96e0a1ca52981cb941b71
2017-05-30 15:57:28 +09:00
Mike Hommey 37b66db19d Bug 1368932 - Generate all the _impl functions with macros in replace-malloc. r=njn
All the _impl functions in replace-malloc.c are largely identical. This
replaces all of them with macro expansions.

--HG--
extra : rebase_source : 67a1809b0b0fc4645ea5041154fa3a6dcb6cce6b
2017-05-25 16:47:57 +09:00
Mike Hommey e4fddc104f Bug 1368932 - Add argument names to malloc implementation declarations in replace-malloc. r=njn
This transforms the declarations from e.g.:
    void *realloc(void *, size_t);
into:
    void *realloc(void *arg1, size_t arg2);

--HG--
extra : rebase_source : 627eb833a98d11f044c776f9d29715ccad31174d
2017-05-25 16:04:46 +09:00
Mike Hommey 2357f45d41 Bug 1368932 - Don't rely on the default MALLOC_DECL_VOID for malloc function declarations in replace-malloc. r=njn
In practice, this induces no change in what the expanded code looks
like.

--HG--
extra : rebase_source : 0df6a5dd3ca5161c0969c814e4222eeb51f9f9a2
2017-05-25 15:58:56 +09:00
Mike Hommey 7cd6d8b45d Bug 1368932 - Factor out function declarations for malloc implementation. r=njn
--HG--
extra : rebase_source : a8df78c7ade473eb5d7c71228f0939cce4554f57
2017-05-25 15:54:05 +09:00
Mike Hommey 16215c85f1 Bug 1368932 - Remove void argument from declarations in malloc_decls.h. r=njn
This makes no significant difference in practice in the macro
expansions, but will help down the line.

--HG--
extra : rebase_source : 6d61c1f28c558321478d7e5f26390d27ae8ae3ac
2017-05-25 13:56:40 +09:00
Mike Hommey f9a4ab0e55 Bug 1367695 - Remove leftovers from jemalloc4. r=njn
MOZ_REPLACE_JEMALLOC was only defined when building jemalloc4 as a
replace-malloc library.

--HG--
extra : rebase_source : fa5c402da07fa96448c170b6db99629469691efe
2017-05-25 16:24:13 +09:00
Mike Hommey b302af7bc0 Bug 1365473 - Rename jemalloc_types.h to mozjemalloc_types.h. r=njn
--HG--
rename : memory/mozjemalloc/jemalloc_types.h => memory/mozjemalloc/mozjemalloc_types.h
extra : rebase_source : 542d0f115f54099517f3eb9a2e0f93f8987f4f63
2017-05-12 21:52:25 +09:00
Mike Hommey 0400284f88 Bug 1365194 - Make `extern "C"` part of MOZ_MEMORY_API and MOZ_JEMALLOC_API. r=njn
This avoids many additions of `extern "C"` in C++ code and will avoid
having to do the same to mozjemalloc once built as C++.

--HG--
extra : rebase_source : af55696262f40a9dd16a19c29edcb9bb307d4957
2017-05-16 18:46:02 +09:00
Carsten "Tomcat" Book 9c3f61e278 Backed out changeset b56224bf370d (bug 1365194) 2017-05-17 11:43:26 +02:00
Carsten "Tomcat" Book 921529ef63 Backed out changeset 4553b67c794f (bug 1365473) for breaking spidermonkey with illegal token on right side of..
--HG--
rename : memory/mozjemalloc/mozjemalloc_types.h => memory/mozjemalloc/jemalloc_types.h
2017-05-17 11:31:32 +02:00
Mike Hommey e7764f68e0 Bug 1365473 - Rename jemalloc_types.h to mozjemalloc_types.h. r=njn
--HG--
rename : memory/mozjemalloc/jemalloc_types.h => memory/mozjemalloc/mozjemalloc_types.h
extra : rebase_source : 542d0f115f54099517f3eb9a2e0f93f8987f4f63
2017-05-12 21:52:25 +09:00
Mike Hommey 2bd75405bc Bug 1365194 - Make `extern "C"` part of MOZ_MEMORY_API and MOZ_JEMALLOC_API. r=njn
This avoids many additions of `extern "C"` in C++ code and will avoid
having to do the same to mozjemalloc once built as C++.

--HG--
extra : rebase_source : af55696262f40a9dd16a19c29edcb9bb307d4957
2017-05-16 18:46:02 +09:00
Mike Hommey 782725e19a Bug 1363992 - Remove jemalloc 4. r=njn
--HG--
extra : rebase_source : 9794320e96279cbb8b7b66db6720c959eaa6a95b
2017-05-11 13:23:07 +09:00
Mike Hommey 02a06f7958 Bug 1363992 - Remove support for making jemalloc4 the default. r=njn
--HG--
extra : rebase_source : 77e1c13aa329637d0ec875439d572ee06e6919fa
2017-05-11 13:16:00 +09:00
Mike Hommey 1a9d4c6f33 Bug 1363992 - Remove support for system jemalloc. r=njn
--HG--
extra : rebase_source : 9141402b6f9e84c67afc14303633d328eb5b652c
2017-05-11 13:03:20 +09:00
Mike Hommey fcd9a5550a Bug 1361258 - Use Thread Local Storage in mozjemalloc on mac. r=erahm
NO_TLS used to be hardcoded on mac because up to 10.6, __thread was not
supported. Until recently, we still supported for 10.6, and it's not the
case anymore, so we could make mac builds use __thread.

Unfortunately, on OSX, __thread circles back calling malloc to allocate
storage on first access, so we have an infinite loop problem here.
Fortunately, pthread_keys don't have this property, so we can use that
instead. It doesn't appear to have significantly more overhead (and TLS
overhead is small anyways compared to the amount of work involved in
allocating memory with mozjemalloc).

At the same time, we uniformize the initialization sequence between
mozjemalloc and mozjemalloc+replace-malloc, such that we have less
occasions for surprises when riding the trains (replace-malloc being
nightly only), ensuring the zone registration happens at the end of
mozjemalloc's initialization.
2017-05-12 18:12:20 +09:00
Julian Seward 83969e88eb Bug 1361258 - Add a jemalloc_thread_local_arena API with a binding for rust. r=glandium
The intent of the API is to allow threads to opt-in to use a separate
mozjemalloc arena.

This indroduces a dummy shell with no actual implementation.
2017-05-12 18:08:08 +09:00
Alex Gaynor d304ba72b3 Bug 1353752 - Fixed building with jemalloc4 r=glandium
It was broken by 516551993d16.

r=glandium

MozReview-Commit-ID: BvejKjAPB84

--HG--
extra : rebase_source : efaf5c0afb7454ecc75a733e184c82cbd16c5e5f
2017-04-05 13:05:43 -04:00
Benjamin Smedberg ca77995f5d Bug 1333826 - Remove SDK_FILES, SDK_LIBRARY, and related is_sdk support in the build goop, r=mshal
MozReview-Commit-ID: 52vPyDXdFte

--HG--
extra : rebase_source : c3217730bb70eb7319152dd07536b12f49d6a597
2017-01-30 11:24:10 -05:00
Mike Hommey c9370d609b Bug 1332508 - Reinitialize allocator mutexes in fork() child processes. r=njn
Adapted from
4e2e3dd9cf
and
d9f7b2a430

As per the latter commit, it would seem unlocking, in fork() child
processes, mutexes that were locked in the parent process is not really
well supported on OSX 10.12. The addition of the zone_reinit_lock
function in 10.12 supports this idea.

--HG--
extra : rebase_source : b3b58558cc195d63200078085c7e9b6c9b8d83ff
2017-01-20 10:06:41 +09:00
Mike Hommey 4e8f02fe22 Bug 1286613 - Add dummy implementations for most remaining OSX zone allocator functions. r=njn
Some system libraries are using malloc_default_zone() and then using
some of the malloc_zone_* API. Under normal conditions, those functions
check the malloc_zone_t/malloc_introspection_t struct for the values
that are allowed to be NULL, so that a NULL deref doesn't happen.

As of OSX 10.12, malloc_default_zone() doesn't return the actual default
zone anymore, but returns a fake, wrapper zone. The wrapper zone defines
all the possible functions in the malloc_zone_t/malloc_introspection_t
struct (almost), and calls the function from the registered default zone
(jemalloc in our case) on its own. Without checking whether the pointers
are NULL.

This means that a system library that calls e.g.
malloc_zone_batch_malloc(malloc_default_zone(), ...) ends up trying to
call jemalloc_zone.batch_malloc, which is NULL, and crash follows.

So as of OSX 10.12, the default zone is required to have all the
functions available (really, the same as the wrapper zone), even if they
do nothing.

This is arguably a bug in libsystem_malloc in OSX 10.12, but jemalloc
still needs to work in that case.

[Adapted from
c6943acb3c]

--HG--
extra : rebase_source : 7d7a5b47fa18f56183e99c3655aee003c9be161e
2017-01-18 14:35:11 +09:00
Mike Hommey dae510ade1 Bug 1286613 - Don't rely on OSX SDK malloc/malloc.h for malloc_zone struct definitions. r=njn
The SDK jemalloc is built against might be not be the latest for various
reasons, but the resulting binary ought to work on newer versions of
OSX.

In order to ensure this, we need the fullest definitions possible, so
copy what we need from the latest version of malloc/malloc.h available
on opensource.apple.com.

[Adapted from
c68bb41793]

--HG--
extra : rebase_source : ab19c478b568ea24095a3be62c39fb81efc1920a
2017-01-18 13:50:35 +09:00
Mike Hommey e52769e834 Bug 1286613 - Use the same zone allocator implementation as replace-malloc for mozjemalloc. r=njn
We have been using a different zone allocator between mozjemalloc and
replace-malloc for a long time. Jemalloc 4 uses the same as
replace-malloc, albeit as part of the jemalloc upstream code base.

We've been bitten many times in the past with Apple changes breaking the
zone allocator, and each time we've had to make changes to the three
instances, although two of them are similar and the changes there are
straightforward.

It also turns out that the way the mozjemalloc zone allocator is set up,
when a new version of OSX appears with a new version of the system zone
allocator, Firefox ends up using the system allocator, because the zone
allocator version is not supported.

So, we use the same zone allocator for both replace-malloc and
mozjemalloc, making everything on par with jemalloc 4.

--HG--
extra : rebase_source : 9c0e245b5f82bb71294370d607e690c05cc89fbc
2017-01-18 11:45:45 +09:00
Mike Hommey d293cc01a1 Bug 1286613 - Move replace-malloc zone allocator to a separate file. r=njn
The intent here is to reuse the zone allocator for mozjemalloc, to avoid
all the shortcomings of mozjemalloc using a different one. This change
only moves the replace-malloc zone allocator out of replace-malloc.c, to
make changes for mozjemalloc integration clearer.

--HG--
rename : memory/build/replace_malloc.c => memory/build/zone.c
extra : rebase_source : 8b98efaa4a88862f2967c855b511e92beb9c4031
2017-01-18 11:39:29 +09:00
Mike Hommey 50dd84a5d8 Bug 1286613 - Properly call mozjemalloc pre/post fork hooks on OSX when replace-malloc is enabled. r=njn
Somehow, we never called those hooks when replace-malloc is enabled. I'd
expect this to cause random deadlocks when forking, and I'm surprised
this hasn't surfaced. Maybe it actually causes some intermittent oranges
on automation, who knows.

This also brings consistency with what is done for jemalloc 4, and with
the mozjemalloc implementation, too, that we're going to replace with
this one in a subsequent changeset.

--HG--
extra : rebase_source : 059567d17f928098db8367e9081b631ced351110
2017-01-19 09:37:22 +09:00
Emanuel Hoogeveen b324d2ba21 Bug 1315939 - Use FINAL_LIBRARY = 'memory' in jemalloc and mozjemalloc. r=glandium
--HG--
extra : rebase_source : 94d19e08090839537ca5174baa8b37993d108acc
2016-11-10 06:57:00 -05:00
Emanuel Hoogeveen e0a81d7503 Bug 1309573 - Part 2: Define MOZ_HAS_MOZGLUE in various places so that the crash reason gets used. r=glandium
--HG--
extra : rebase_source : d55e8dd75469759f333366bdf9662178ff0a2474
2016-11-08 03:53:00 -05:00
Emanuel Hoogeveen 8210833803 Bug 1294732 - Back out all of bug 1271165 as it has served its purpose. r=glandium 2016-08-23 08:45:00 -04:00
Emanuel Hoogeveen 581ecef4dd Bug 1271165 - Part 6: Change the new functions to MFBT_API to export them on OSX. r=jandem
--HG--
extra : rebase_source : cc3d6cd7cec9c822904364ace811d2c0dff98eff
2016-08-16 07:12:00 -04:00
Emanuel Hoogeveen 57b1d1e8f2 Bug 1271165 - Part 2: Hook the new functions up and provide dummy implementations where needed. r=ehoogeveen 2016-08-12 07:37:00 -04:00
Tom Tromey 5538d692d3 Bug 1286877 - do not set c-basic-offset for python-mode; r=gps
This removes the unnecessary setting of c-basic-offset from all
python-mode files.

This was automatically generated using

    perl -pi -e 's/; *c-basic-offset: *[0-9]+//'

... on the affected files.

The bulk of these files are moz.build files but there a few others as
well.

MozReview-Commit-ID: 2pPf3DEiZqx

--HG--
extra : rebase_source : 0a7dcac80b924174a2c429b093791148ea6ac204
2016-07-14 10:16:42 -06:00
Mike Hommey 16529670ac Bug 1284677 - Change how the default OSX malloc zone is found. r=njn
--HG--
extra : rebase_source : b5ea2bfb37047fa9f8d4d833186694407c73adda
2016-07-08 14:48:16 +09:00
Chris Peterson 8a9e2d2bd4 Bug 1272513 - Part 2: Remove redundant -Wshadow CXXFLAGS from moz.build files. r=glandium 2016-05-14 00:54:55 -07:00
Ted Mielczarek 68bd5e8c4d bug 1244743 - Replace MOZ_NATIVE_X with MOZ_SYSTEM_X. r=gps
MozReview-Commit-ID: 9ip3qeAXFEe

--HG--
extra : commitid : H6aEkHprVyX
extra : rebase_source : e20d5f4b297caf66711c72cd55dd76f7f9ef9d9c
extra : histedit_source : 47545e0ea61eb66f16545f70e6d3792a389bedb1
2016-02-01 10:49:34 -05:00
Ms2ger d1f112a81b Bug 1254948 - Remove makefiles used to transition from jemalloc_config.c to jemalloc_config.cpp; r=glandium 2016-03-10 15:53:58 +01:00
Nicholas Nethercote 57bef6b196 Bug 1232219 (part 3) - Fix remaining -Wunused warnings. r=glandium.
--HG--
extra : rebase_source : 357474b4c7729b78a3030dfefd720aad4e1b2576
2015-12-16 22:57:51 -08:00
Jan Beich 16a744dd9f Bug 1229395 - Part 2 - Rely on MALLOC_H to provide function prototypes for MOZ_NATIVE_JEMALLOC. r=glandium 2015-12-01 16:54:44 +00:00
Jan Beich 93cec6dc70 Bug 1229395 - Part 1 - Unbreak MOZ_NATIVE_JEMALLOC after bug 1141079. r=glandium 2015-12-01 16:39:33 +00:00
Mike Hommey 762aba02cd Bug 1221453 - Use ObjDirPaths for GENERATED_INCLUDES and merge with LOCAL_INCLUDES. r=gps 2015-11-06 09:59:21 +09:00
Mike Hommey 538706caba Bug 1203840 - Trigger dirty pages purge after CC. r=njn,r=smaug,r=mccr8
Jemalloc 4 purges dirty pages regularly during free() when the ratio of dirty
pages compared to active pages is higher than 1 << lg_dirty_mult.  We set
lg_dirty_mult in jemalloc_config to limit RSS usage, but it also has an impact
on performance.

So instead of enforcing a high ratio to force more pages being purged, we keep
jemalloc's default ratio of 8, and force a regular purge of all dirty pages,
after cycle collection.

Keeping jemalloc's default ratio avoids cycle-collection-triggered purge to
have to go through really all dirty pages when there are a lot, in which case
the normal jemalloc purge during free() will already have kicked in. It also
takes care of everything that doesn't run the cycle collector still having
a level of purge, like plugins in the plugin-container.

At the same time, since jemalloc_purge_freed_pages does nothing with jemalloc 4,
repurpose the MEMORY_FREE_PURGED_PAGES_MS telemetry probe to track the time
spent in this cycle-collector-triggered purge.
2015-09-20 17:43:43 +09:00
David Major cc3cd34e27 Bug 1174250: Allow the CommitHook to fail without crashing. r=glandium 2015-09-15 13:53:38 -04:00
Mike Hommey 0cf8973ede Bug 1202523 - Use the default chunk size with jemalloc 4. r=njn 2015-09-08 13:10:22 +09:00
Mike Hommey 0efccdc82c Bug 1201792 - s/MOZ_JEMALLOC3/MOZ_JEMALLOC4/. r=njn 2015-09-04 15:15:47 +09:00
Mike Hommey 748d14b77a Bug 1141079 - Setup custom jemalloc chunk hooks to keep RSS usage low. r=njn 2015-09-04 14:35:54 +09:00
Mike Hommey 4fb6c55f9d Bug 1141079 - Make jemalloc_config.c a C++ source file. r=ted
--HG--
rename : memory/build/jemalloc_config.c => memory/build/jemalloc_config.cpp
2015-09-04 14:35:54 +09:00
Jan Beich 1df901bdce Bug 1201462 - Don't count arena allocated metadata once per bin. r=glandium 2015-09-04 14:35:52 +09:00
Nicholas Nethercote f44287005f Bug 1198334 (part 1) - Replace the opt-in FAIL_ON_WARNINGS with the opt-out ALLOW_COMPILER_WARNINGS. r=glandium.
The patch removes 455 occurrences of FAIL_ON_WARNINGS from moz.build files, and
adds 78 instances of ALLOW_COMPILER_WARNINGS. About half of those 78 are in
code we control and which should be removable with a little effort.

--HG--
extra : rebase_source : 82e3387abfbd5f1471e953961d301d3d97ed2973
2015-08-27 20:44:53 -07:00
Chris Peterson 1416e566de Bug 1198124 - Enable -Wshadow in directories that have no -Wshadow warnings. r=glandium 2015-08-25 09:14:38 -07:00
Guilherme Goncalves 148b18ba80 Bug 1125514 - Use jemalloc's metadata statistics to compute bookkeeping. r=glandium 2015-08-20 10:05:44 +09:00
Birunthan Mohanathas a8939590de Bug 1182996 - Fix and add missing namespace comments. rs=ehsan
The bulk of this commit was generated by running:

  run-clang-tidy.py \
    -checks='-*,llvm-namespace-comment' \
    -header-filter=^/.../mozilla-central/.* \
    -fix
2015-07-13 08:25:42 -07:00
Mike Hommey 985ca5fd67 Bug 1168719 - Generic replace-malloc library. r=njn 2015-06-05 10:40:58 +09:00
Mike Hommey c917606f95 Bug 1043692 - Move DIST_INSTALL to moz.build. r=gps 2015-05-12 07:55:22 +09:00
Paul Rouget d60a1bd75e Bug 1156628 - Don't limit the number of unused dirty pages kept by jemalloc on B2G desktop. r=gsvelto, r=glandium 2015-04-24 03:58:00 -04:00
Mike Hommey 6dd0c8c299 Bug 762449 - Fix "result of 32-bit shift implicitly converted to 64 bits" on win64. r=njn 2015-03-02 07:35:28 +09:00
Mike Hommey 03b3291d3f Bug 1120272 - Expose _aligned_alloc and _aligned_free from mozglue. r=njn 2015-01-20 13:10:00 +09:00
Masatoshi Kimura bf312ad056 Bug 1120062 - Part 1: Remove most Nullptr.h includes. r=waldo 2015-01-11 11:34:52 +09:00
Brad Lassey 0e7333b6e7 bug 1083116 - build fails: malloc_decls.h conflicting types for 'malloc_usable_size' r=glandium 2015-01-06 19:58:36 -05:00
Guilherme Goncalves c6a10e7bb5 Bug 899126 - Part 2 - Measure bookkeeping in jemalloc3. r=glandium. 2014-12-29 13:42:14 -08:00
Jan Beich f4828d35f5 Bug 1115078 - Unbreak MOZ_JEMALLOC3 build on FreeBSD after bug 899126. r=glandium 2014-12-23 12:32:00 -05:00
Guilherme Goncalves 1fb7597a1b Bug 1110514 - Use arena.<i>.purge to free jemalloc3's dirty pages. r=glandium 2014-12-23 02:41:00 -05:00
Guilherme Goncalves 802cb55877 Bug 1108045 - Junk memory with jemalloc3. r=glandium 2014-12-23 07:44:00 -05:00
Guilherme Goncalves f9d70a5255 Bug 762448 - Bump opt.lg_dirty_mult in jemalloc3 to reduce number of dirty pages. r=glandium 2014-12-23 06:01:00 -05:00
Guilherme Goncalves 5ff0106324 Bug 762448 - Deduct bin_unused from waste in jemalloc3's stats. r=glandium 2014-11-12 13:00:00 -05:00
Ryan VanderMeulen 42a3e6c0ea Merge m-c to inbound. a=merge 2014-12-23 13:18:02 -05:00
Vladimir Vukicevic a910628209 Bug 1112566 - Use jemalloc C99 msvc compat headers when building mozjemalloc compat glue and jemalloc3 replace-malloc library. r=glandium 2014-12-23 15:27:18 +09:00
Guilherme Goncalves 22c6a4800b Bug 899126 - Part 1: Implement bin_unused for jemalloc3. r=glandium 2014-12-19 09:15:00 -05:00
Guilherme Goncalves 60eb32a82c Bug 1094275 - Update the in-tree copy of jemalloc3 to commit b4acf73. r=glandium 2014-12-17 09:14:48 +09:00
Mike Hommey c04636bda3 Bug 1104173 - Properly export jemalloc_* symbols from the jemalloc3 replace-malloc library. r=njn
Bug 818922 made MOZ_REPLACE_MALLOC a global define, and that changed the
defines set when building mozjemalloc_compat.c for the jemalloc3
replace-malloc library. In turn, this made the symbol munging wrong for
that library, making the jemalloc_* functions exported as je_jemalloc_*
instead of replace_jemalloc_*.
2014-11-26 16:04:25 +09:00
Mike Hommey 3cbc3cf69f Bug 1087245 part 1 - Allow replace-malloc libraries to register debug file handles to poison IO interposer. r=nfroyd 2014-11-18 19:21:06 +09:00
Mike Hommey 7def0f9cd3 Bug 1097507 - Make libxul independent of libdmd when DMD is enabled. r=njn
This also effectively changes how DMD is enabled from requiring both
replace-malloc initialization and the DMD environment variable to
requiring only the former. The DMD environment variable can still be
used to specify options, but not to disable entirely.

This however doesn't touch all the parts that do enable DMD by setting
the DMD environment variable to 1, so the code to handle this value
is kept.
2014-11-18 19:21:06 +09:00
Mike Hommey 6d15698158 Bug 818922 - Add bidirectional method calls with replace-malloc library. r=njn 2014-11-18 19:21:06 +09:00
Guilherme Goncalves f843e98e47 Bug 762448 - Disable tcache in jemalloc3. r=glandium 2014-10-23 02:36:00 -04:00
Mike Hommey 47c853314f Bug 1077148 part 4 - Add and use new moz.build templates for Gecko programs and libraries. r=gps
There are, sadly, many combinations of linkage in use throughout the tree.
The main differentiator, though, is between program/libraries related to
Gecko or not. Kind of. Some need mozglue, some don't. Some need dependent
linkage, some standalone.

Anyways, these new templates remove the need to manually define the
right dependencies against xpcomglue, nspr, mozalloc and mozglue
in most cases.

Places that build programs and were resetting MOZ_GLUE_PROGRAM_LDFLAGS
or that build libraries and were resetting MOZ_GLUE_LDFLAGS can now
just not use those Gecko-specific templates.
2014-10-30 13:06:12 +09:00
Mike Hommey 746f48e4a4 Bug 1083686 part 0 - Don't set MOZ_MEMORY_API to static for OSX in replace_malloc.c. r=njn
It appears to be an unnecessary optimization, since the compiler is still inlining
the functions when they're not marked static. OTOH, following patches will require
the _impl functions not to be static.
2014-10-24 13:08:01 +09:00
Guilherme Goncalves 258ec900a4 Bug 1082199 - Recompute stats in jemalloc_stats when using jemalloc3. r=glandium 2014-10-17 15:55:00 +02:00
Mike Hommey b77d0f1d19 Bug 1077366 - Remove most symbol wrapping from Android builds. r=nfroyd
Since essentially everything is linked to libmozglue and libmozglue takes
precedence in symbol resolution in our dynamic linker, there is no need
to wrap most symbols. PR_GetEnv/PR_SetEnv still needs wrapping because
there's no other way to actually wrap the calls from NSPR itself and NSS,
as well as the symbols wrapped because our dynamic linker can't find them
in system libraries on some devices because they're weak.
2014-10-14 07:17:13 +09:00
Mike Hommey ed70c5f377 Bug 1041941 - Use templates for programs, simple programs, libraries and C++ unit tests. r=gps 2014-09-03 14:10:54 +09:00
Mike Hommey 5fab42fcaf Bug 1036894 part 8 - Move most in-tree library linkage information to moz.build, as USE_LIBS. r=gps 2014-07-23 13:30:52 +09:00
Mike Hommey a33d7d67e6 Bug 1036894 part 1 - Move MAKE_FRAMEWORK, SDK_LIBRARY, SHARED_LIBRARY_NAME and STATIC_LIBRARY_NAME to moz.build. r=gps
At the same time, make the Library data more useful in the build frontend.
2014-07-23 13:29:09 +09:00
Chris Peterson 0b109cf641 Bug 1033188 - #include <stdlib.h> for getenv() function declaration. r=glandium 2014-07-01 18:49:52 -07:00
Mike Hommey d895ddcdda Bug 1019881 - Ensure the default purgeable zone is after the default zone on OS X. rs=njn 2014-06-11 11:07:27 +09:00
Jan Beich 98b8342812 Bug 1015602 - Populate chunksize/bin_unused for MOZ_JEMALLOC3 as they're now used after bug 1006769. r=erahm 2014-05-24 06:41:00 -04:00
Thomas Zimmermann 739b082bdc Bug 979887: Fix GCC warning about undeclared function |vsnprintf|, r=glandium 2014-03-06 11:20:00 +01:00
Thomas Zimmermann 724de1c4a9 Bug 979887: Fix GCC warnings about inline variable declarations, r=glandium
Some code is compiled in C90 mode, where inline declarations of
variables are not allowed.
2014-03-06 11:19:57 +01:00
Ehsan Akhgari 17f4a32d8b Bug 976896 - Port STL_FLAGS to moz.build; r=mshal 2014-03-04 19:39:06 -05:00
Ehsan Akhgari 996229d3d0 Bug 973403 - Move the LOCAL_INCLUDES in memory to moz.build; r=glandium 2014-02-18 01:05:51 -05:00
Shelly Lin 4da13e0a0a Bug 801571 - Override vasprintf and asprint for Android platform to avoid the mismatch of malloc()/free(). r=glandium 2014-01-17 16:54:06 +08:00
Ryan VanderMeulen da06e090f6 Backed out changeset 4230d328b65f (bug 801571) for memory leaks. 2014-01-17 14:25:28 -05:00
Shelly Lin 5a3b503309 Bug 801571 - Override vasprintf and asprint for Android platform to avoid the mismatch of malloc()/free(). r=glandium
DONTBUILD
2014-01-17 16:54:06 +08:00
Ryan VanderMeulen 913aa4dc43 Backed out changeset 0496d73bf88e for landing with the wrong bug # in the commit message. 2014-01-17 11:13:30 -05:00
Shelly Lin 7a3e6b8008 Bug 801517 - Override vasprintf and asprint for Android platform to avoid the mismatch of malloc()/free(). r=glandium 2014-01-17 16:54:06 +08:00
Kai-Zhen Li d8cf29acff Bug 957450 - fix compile error from jemalloc in gonk-kk. r=gal 2014-01-10 14:14:53 +08:00
Marco Chen 46177c53e2 Backout 977d6be3df40 Bug 957450 - The added "or condition" didn't prevent gonk-kk version fall into non-const definition. 2014-01-15 15:55:12 +08:00
Kai-Zhen Li 5979fa3eb3 Bug 957450 - fix compile error in gonk-kk. r=iacobcatalin 2014-01-10 14:14:53 +08:00
Mike Hommey 8fd06cf41b Bug 874266 - Move all DEFINES that can be moved to moz.build. r=mshal 2013-11-27 22:55:07 +09:00
Mike Hommey b32a4ed166 Backout changeset 3fd4b546eed4 (bug 874266) and changeset a35d2e3a872f (bug 942043) for ASAN build bustage and Windows test bustage
--HG--
extra : amend_source : f20d09aeff1c8b5cbd0f1d24c7ce04e86f3aed1d
2013-11-28 14:24:05 +09:00
Mike Hommey 682364d535 Bug 874266 - Move all DEFINES that can be moved to moz.build. r=mshal 2013-11-28 13:08:16 +09:00
Ehsan Akhgari e58130769b Backed out changeset c3d41ab0a2df (bug 940218) for causing bug 941616
--HG--
extra : rebase_source : c0d9b2840ddb6a6b88b86cfc3e3a6810ac9e37e2
2013-11-21 14:28:52 -05:00
Chris Peterson 4be77ac908 Bug 940218 - Build memory/jemalloc and friends in unified mode. r=glandium 2013-11-17 01:16:36 -08:00
Mike Hommey e80e877ab7 Bug 939044 - Remove most definitions of MODULE. r=mshal 2013-11-19 11:47:39 +09:00
Mike Hommey dda5b915fe Bug 939039 - Remove now useless FORCE_STATIC_LIB definitions. r=gps 2013-11-19 11:47:28 +09:00
Mike Hommey ffe0380912 Bug 935881 - Use FINAL_LIBRARY for all (fake) libraries that end up linked in a single other library. r=gps 2013-11-19 11:47:14 +09:00
Cykesiopka d2f6b7333c Bug 914270 - Part 1: Simple/Automated moves. r=joey 2013-10-24 18:51:00 +01:00
Mike Hommey b000a846c2 Bug 929905 - Consolidate sources in moz.build. r=gps 2013-10-25 08:23:05 +09:00
Masatoshi Kimura ede451acef Bug 928210 - Get rid of MOZ_INLINE 2013-10-22 06:34:24 +09:00
Mike Hommey f8bc7fa754 Bug 912293 - Remove now redundant boilerplate from Makefile.in. r=gps 2013-09-05 09:01:46 +09:00
Ms2ger 50634577d5 Merge m-c to inbound. 2013-08-02 11:07:57 +02:00
Ms2ger a4e67abf5e Bug 897921 - Remove some dead assignments in makefiles; r=mshal 2013-08-02 09:03:55 +02:00
Justin Lebar a36239e4ef Bug 899880 - Fix jemalloc3 so it works after bug 898558, which changed jemalloc's stats struct. r=glandium 2013-08-01 16:37:54 -07:00
Brian O'Keefe 11bcc1cd9e Bug 875934 - Move LIBRARY_NAME to moz.build (batch #1); r=mshal
--HG--
extra : rebase_source : 385d3fd65475ffc18ee44ae088753649470e214b
2013-06-17 15:21:01 -04:00
Joey Armstrong 05956ba0cd bug 870406: move CSRCS to mozbuild (config batch #2) r=mshal 2013-06-04 11:08:44 -04:00
David Zbarsky b7a4767301 Fix b2g build, no bug 2013-05-16 18:07:12 -07:00
Mike Shal df7deac25b Bug 846634 - Part 2: Move EXPORTS to moz.build; r=joey 2013-04-16 15:24:43 -04:00
Kyle Machulis 72a717a860 Bug 855465 - Add emacs python mode comments to moz.build files; r=gps 2013-04-01 11:36:59 -07:00
Kyle Machulis 43628a7867 Backout for changeset 03452b187c14 (Bug 855465) due to bustage on a CLOSED TREE; r=qdot 2013-03-29 15:12:58 -07:00
Kyle Machulis 334c0800cf Bug 855465 - Add emacs python mode comments to moz.build files; r=gps
--HG--
extra : rebase_source : 004a756492323e1a049586e85b3be5037159df20
2013-03-29 13:56:18 -07:00
Mike Shal 7ecea60097 Bug 844654 - Part 2: Move MODULE to moz.build; rs=gps 2013-03-19 11:47:00 -07:00
Gregory Szorc 282edab58a Bug 784841 - Part 18c: Convert /memory, /mfbt, /mozglue; r=ted f=Ms2ger 2013-02-25 12:47:17 -08:00
Mike Hommey e6d10e0341 Bug 826171 - Fix strndup in memory/build/mozmemory_wrap.c. r=jlebar 2013-01-03 16:05:20 +01:00
Mike Hommey f5c35d6a44 Bug 804303 part 3 - Build jemalloc3 as a replace-malloc library when building with mozjemalloc. r=khuey 2012-12-07 09:32:24 +01:00
Mike Hommey e9c6d8bfe1 Bug 804303 part 2 - Add ability to dynamically replace or supplement jemalloc implementation. r=jlebar,r=khuey 2012-12-07 09:32:24 +01:00
Mike Hommey c51f5d814d Bug 804303 part 1 - Cleanup how the mozjemalloc/jemalloc3 glue is set up, attempting to make it clearer. r=jlebar,r=khuey
--HG--
rename : memory/build/extraMallocFuncs.c => memory/build/jemalloc_config.c
rename : memory/mozjemalloc/jemalloc.h => memory/build/mozmemory.h
rename : memory/build/extraMallocFuncs.c => memory/build/mozmemory_wrap.c
2012-12-07 09:32:24 +01:00
Rafael Ávila de Espíndola 7330a180a0 Bug 807112 - change MOZ_EXPORT_API and MOZ_IMPORT_API to not take the type. r=Waldo. 2012-10-31 18:18:12 -04:00
Jan Beich b1967c0ac2 Bug 762445 - Add jemalloc3 glue for heap-committed, heap-dirty in about:memory. r=glandium,jlebar 2012-10-18 16:09:07 +02:00
Jan Beich 90b1515739 Bug 788955 - Without --enable-jemalloc detect and use jemalloc3 in libc. r=glandium 2012-10-17 16:36:49 +02:00
Chris Jones 353f38f677 Bug 799849: Wrap nothrow libstdc++ allocators. r=glandium 2012-10-13 03:16:53 -07:00
Michael Wu 388d00feee Bug 780287 - Don't wrap jemalloc on gonk, r=glandium 2012-08-06 16:22:26 -04:00
Mike Hommey 70d7c821af Bug 774032 bonus - Use @DEPTH@ and @relativesrcdir@ in Makefile.in. r=ted 2012-08-04 20:26:44 +02:00
Mike Hommey ad1fa9fabe Bug 772423 - Hook mozjemalloc's abort(). r=jlebar 2012-07-11 16:55:39 +02:00
Mike Hommey b83e644292 Bug 762446 - Hook jemalloc's abort. r=jlebar 2012-06-11 08:12:40 +02:00
Mike Hommey 1510749815 Bug 580408 - Glue for jemalloc 3.0.0. r=khuey 2012-06-07 10:41:59 +02:00
Mike Hommey caf974ce43 Bug 580408 - Move Mozilla fork of jemalloc to memory/mozjemalloc. r=khuey
--HG--
rename : memory/jemalloc/Makefile.in => memory/mozjemalloc/Makefile.in
rename : memory/jemalloc/jemalloc.c => memory/mozjemalloc/jemalloc.c
rename : memory/jemalloc/jemalloc.h => memory/mozjemalloc/jemalloc.h
rename : memory/jemalloc/jemalloc_types.h => memory/mozjemalloc/jemalloc_types.h
rename : memory/jemalloc/linkedlist.h => memory/mozjemalloc/linkedlist.h
rename : memory/jemalloc/osx_zone_types.h => memory/mozjemalloc/osx_zone_types.h
rename : memory/jemalloc/ql.h => memory/mozjemalloc/ql.h
rename : memory/jemalloc/qr.h => memory/mozjemalloc/qr.h
rename : memory/jemalloc/rb.h => memory/mozjemalloc/rb.h
2012-06-07 10:41:58 +02:00
Mike Hommey 0a562bc5d0 Bug 758010 - Wrap operator new/delete to jemalloc on Android. r=jlebar 2012-05-31 23:49:03 +02:00
Mike Hommey 11b04e4bcf Bug 758010 - Wrap operator new/delete on Android. r=khuey 2012-05-26 10:21:33 +02:00
Mike Hommey 876a7d7818 Bug 751482 - Move wcsdup and je_dumb_free_thunk out of jemalloc.c. r=jlebar 2012-05-04 09:47:00 +02:00
Mike Hommey a29da17198 Fixup for bug 736564 - Ship libmemory.a in the SDK instead of libjemalloc.a. r=ted 2012-05-04 09:46:27 +02:00
Mike Hommey 3ef23117af Bug 736963 - Move jemalloc'ed strdup/strndup definitions. r=jlebar,r=ted 2012-03-31 09:23:19 +02:00