зеркало из https://github.com/mozilla/gecko-dev.git
96 строки
4.0 KiB
Plaintext
96 строки
4.0 KiB
Plaintext
Logalloc is a replace-malloc library for Firefox (see
|
|
memory/build/replace_malloc.h) that dumps a log of memory allocations to a
|
|
given file descriptor or file name. That log can then be replayed against
|
|
Firefox's default memory allocator independently or through another
|
|
replace-malloc library, allowing the testing of other allocators under the
|
|
exact same workload.
|
|
|
|
To get an allocation log the following environment variable when starting
|
|
Firefox:
|
|
MALLOC_LOG=/path/to/log-file
|
|
or
|
|
MALLOC_LOG=number
|
|
|
|
When MALLOC_LOG is a number below 10000, it is considered as a file
|
|
descriptor number that is fed to Firefox when it is started. Otherwise,
|
|
it is considered as a file name.
|
|
|
|
As those allocation logs can grow large quite quickly, it can be useful
|
|
to pipe the output to a compression tool.
|
|
|
|
MALLOC_LOG=1 would send to Firefox's stdout, MALLOC_LOG=2 would send to
|
|
its stderr. Since in both cases that could be mixed with other output
|
|
from Firefox, it is usually better to use another file descriptor
|
|
by shell redirections, such as:
|
|
|
|
MALLOC_LOG=3 firefox 3>&1 1>&2 | gzip -c > log.gz
|
|
|
|
(3>&1 copies the `| gzip` pipe file descriptor to file descriptor #3, 1>&2
|
|
then copies stderr to stdout. This leads to: fd1 and fd2 sending to stderr
|
|
of the parent process (the shell), and fd3 sending to gzip.)
|
|
|
|
Each line of the allocations log is formatted as follows:
|
|
<pid> <tid> <function>([<args>])[=<result>]
|
|
where <args> is a comma separated list of values. The number of <args> and
|
|
the presence of <result> depend on the <function>.
|
|
|
|
Example log:
|
|
18545 18545 malloc(32)=0x7f90495120e0
|
|
18545 18545 calloc(1,148)=0x7f9049537480
|
|
18545 18545 realloc(0x7f90495120e0,64)=0x7f9049536680
|
|
18545 18545 posix_memalign(256,240)=0x7f9049583300
|
|
18545 18545 jemalloc_stats()
|
|
18545 18545 free(0x7f9049536680)
|
|
|
|
This log can be replayed with the logalloc-replay tool in
|
|
memory/replace/logalloc/replay. However, as the goal of that tool is to
|
|
reproduce the recorded memory allocations, it needs to avoid as much as
|
|
possible doing its own allocations for bookkeeping. Reading the logs as
|
|
they are would require data structures and memory allocations. As a
|
|
consequence, the logs need to be preprocessed beforehand.
|
|
|
|
The logalloc_munge.py script is responsible for that preprocessing. It simply
|
|
takes a raw log on its stdin, and outputs the preprocessed log on its stdout.
|
|
It replaces pointer addresses with indexes the logalloc-replay tool can use
|
|
in a large (almost) linear array of allocation tracking slots (prefixed with
|
|
'#'). It also replaces the pids with numbers starting from 1 (such as the
|
|
first seen pid number is 1, the second is 2, etc.).
|
|
|
|
The above example log would become the following, once preprocessed:
|
|
1 1 malloc(32)=#1
|
|
1 1 calloc(1,148)=#2
|
|
1 1 realloc(#1,64)=#1
|
|
1 1 posix_memalign(256,240)=#3
|
|
1 1 jemalloc_stats()
|
|
1 1 free(#1)
|
|
|
|
The logalloc-replay tool then takes the preprocessed log on its stdin and
|
|
replays the allocations printed there, but will only replay those with the
|
|
same process id as the first line (which normally is 1).
|
|
|
|
As the log files are simple text files, though, it is easy to separate out
|
|
the different processes log with e.g. grep, and feed the separate processes
|
|
logs to logalloc-replay.
|
|
|
|
The logalloc-replay program won't output anything unless jemalloc_stats
|
|
records appears in the log. You can expect those to be recorded when going
|
|
to about:memory in Firefox, but they can also be added after preprocessing.
|
|
|
|
Here is an example of what one can do:
|
|
|
|
gunzip -c log.gz | python logalloc_munge.py | \
|
|
awk '$1 == "2" { print $0 } !(NR % 10000) { print "2 1 jemalloc_stats()" }' | \
|
|
./logalloc-replay
|
|
|
|
The above command replays the allocations of process #2, with some stats
|
|
output every 10000 records.
|
|
|
|
The logalloc-replay tool itself being hooked with replace-malloc, it is possible
|
|
to set LD_PRELOAD/DYLD_INSERT_LIBRARIES/MOZ_REPLACE_MALLOC_LIB and replay a log
|
|
through a different allocator. For example:
|
|
|
|
LD_PRELOAD=libreplace_jemalloc.so logalloc-replay < log
|
|
|
|
Will replay the log against jemalloc4 (which is, as of writing, what
|
|
libreplace_jemalloc.so contains).
|