PseudoStack requires that startJSSampling() and stopJSSampling() calls be
interleaved. But currently the conditions guarding those calls don't match:
startJSSampling() is guarded by ShouldProfileThread(), and stopJSSampling() is
guarded by HasProfile().
It's possible for HasProfile() to be true when ShouldProfileThread() is not
true -- e.g. profile many threads, then restart and profile fewer threads, and
we end up with live threads that have a profile but aren't being profiled right
now -- which leads to assertion failures in stopJSSampling().
This patch makes the stopJSSampling() condition use ShouldProfileThread(), just
like the startJSSampling() condition, which fixes the assertion failure.
--HG--
extra : rebase_source : e9931928c8ac1301f5018f9da319bc478722b98e
LUL doesn't read CFI from the main executable on x86_64-linux, and possibly
other Linux variants, because SharedLibraryInfo::GetInfoForSelf() doesn't
produce a name for the main executable object, even though it does notice the
mapping.
This causes noticeable unwind breakage because the main executable on Linux
contains various wrapper functions pertaining to memory allocation and locking,
such as
moz_xmalloc, moz_xcalloc, moz_xrealloc
mozilla::detail::MutexImpl::lock, mozilla::detail::MutexImpl::unlock
and is generally observable on x86_64-Linux as unwinding failures out of
functions with addresses around 0x40xxxx, since that's the traditional load
address for the main executable.
This patch modifies the Linux implementation of GetInfoForSelf() so as to
harvest the main executable's name from /proc/self/maps. This is then added
into the information acquired from dl_iterate_phdr. As a result
GetInfoForSelf() does correctly report the executable name, so LUL reads Dwarf
unwind info from it, and the abovementioned unwinding failures disappear.
--HG--
extra : rebase_source : 267c6d7c3967a4d29f8ff0b4a91d339a6625085d
Pick up autofix improvements in 3.19.0. Upgrade eslint-plugin-react. Also fix issues with the package.json file and the ESLint node_modules upload script.
MozReview-Commit-ID: IDZ1n4qTTuv
--HG--
extra : rebase_source : aa97cd6f314ce10d16d12446e50a27d6f994a9f1
shared-linux-libraries.cc is a maze of ifdefs which is hard to navigate, hard to
reason about and gets in the way of making a proper fix for bug 1354546. This
bug is for cleanup only. It should not change any functionality.
The following changes are made:
* adds emacs/vi tab-width lines
* removes the ARRAY_SIZE macro as it appears to be unused
* documents the 3 different configurations, splits #includes accordingly
* comments SharedLibraryInfo::GetInfoForSelf accordingly
* wraps some long lines
* documents in which cases dl_iterate_phdr is used and in which cases
/proc/<pid>/maps is used
* Puts /proc/<pid>/maps reading in its own scope
* Makes the LOG messages on failure clearer
Currently, ThreadInfos for live and dead threads are stored in a single vector.
This patch separates them into two separate vectors.
This ensures that the two kinds of ThreadInfos can't be mixed up. It also means
ThreadInfo::mPendingDelete can be removed.
Currently, when the profiler is active we hold onto the ThreadInfo of all
threads that die. Then when capturing a profile we ignore all threads that
aren't being profiled.
This patch changes things so we only hold onto the ThreadInfos of threads that
die if they are being profiled. In effect it removes state 3 from the following
list of possible ThreadInfo states:
1. !PendingDelete + !HasProfile
2. !PendingDelete + HasProfile
3. PendingDelete + !HasProfile (no longer used)
4. PendingDelete + HasProfile
Now that ThreadResponsiveness is only used on the main thread, we can refactor
ThreadInfo a bit. This patch does the following.
- Removes ThreadInfo::mThread, which is unused.
- Changes ThreadInfo::mRespInfo to a Maybe<>, and moves the is-main-thread
checking outside of ThreadInfo and ThreadResponsiveness.
- Renames {ThreadInfo,TickSample}::mRespInfo as mResponsiveness, to better
match the class name.
The state management is better done within nsProfiler::GetProfileDataAsync()
and nsProfiler::DumpProfileToFileAsync(). (The latter function is new in this
patch.)
This fixes a deadlock.
Other notes:
- The patch moves ProfileGatherer from ProfilerState to nsProfiler. This is
nice because the former is shared between threads but the latter is main
thread only. (This is how the deadlock is avoided.)
- ProfilerStateMutex and PSLockRef are no longer required in platform.h. Those
types and variables are now only used in platform.cpp and platform-*.cpp.
- ProfilerGatherer now calls profiler_get_profile() instead of ToJSON(). Which
means that ToJSON() now has a single caller, so the patch inlines it at the
callsite and removes it.
- profiler_save_profile_to_file_async() dispatched a Runnable to the main
thread. But this wasn't necessary, because it always ran on the main thread
itself. So the new function nsProfiler::DumpProfileToFileAsync() doesn't do
that.
- profiler_will_gather_OOP_profile(), profiler_gathered_OOP_profile(), and
profiler_OOP_exit_profile() are all moved into nsProfiler as well. This
removes the need for the horrible fake lock in
profiler_will_gather_OOP_profile(), hooray!
The conversion to a JSObject is better done within
nsProfiler::GetProfileData().
--HG--
extra : rebase_source : 4a0ba97d99681fca96f2d26b609bafe188095787
This reduces the amount of places where we need to specify the mozilla/frame-script environment. It does have
the side effect of allowing those globals in the whole file, but that is what specifying the environment would
do, and this is also for mochitest test files only.
MozReview-Commit-ID: 1LLFbn6fFJR
--HG--
extra : rebase_source : 82a6934d90bbbbd25f91b7b06bf4f9354e38865a
This retains the advantage of running only once per process, while
avoiding the per-process overhead of a jsm.
MozReview-Commit-ID: 1N53MvRwUpg
--HG--
rename : browser/modules/ContentObservers.jsm => browser/modules/ContentObservers.js
extra : rebase_source : 6a502cff26fcb55526f97385274bbae871f5cc6c
SetSampleContext() sets the TickSample's register fields, and the two callers
of SetSampleContext() set the TickSample's mContext.
This patch changes SetSampleContext() so it sets all the fields in one place.
It also renames SetSampleContext() as FillInSample(), because it sets more than
just the context.
--HG--
extra : rebase_source : 9b9f749fe3de687a7fd32f5c38e2321c2abebfdc
Currently each live thread has a PseudoStack that is owned by tlsPseudoStack,
and a ThreadInfo that has a non-owning pointer to the same PseudoStack.
Then, if the profile is active when the thread dies, ownership of the
PseudoStack is transferred to the ThreadInfo.
This patch simplifies the ownership rules. Every ThreadInfo now always owns its
PseudoStack and is responsible for destroying it. tlsPseudoStack is a
non-owning pointer, and so must be cleared when a PseudoStack is destroyed.
This simplifies the code in a few places.
--HG--
extra : rebase_source : 1012b6590380091d60eff98b4e0c5b1ba946cc7e
The patch also adds a MOZ_RELEASE_ASSERT in profiler_unregister_thread() for
the case where the ThreadInfo isn't found, which is informative.
--HG--
extra : rebase_source : 11a86914db235e4a60955ff1c9b77d46109af548
This avoids the need for the fake ThreadInfo in profiler_get_backtrace(). It
requires adding a few extra fields to TickSample.
--HG--
extra : rebase_source : c28e5493edc7db96a7160e78b297ae09dc05ca7c
This patch does the following.
- Splits TickSample's constructor in two, one for the periodic sample case, and
one for the synchronous sample case, and initializes more stuff in them. (The
two constructors aren't that different right now, but they will become more
different when I remove TickSample::mThreadInfo.)
- Makes all the constructor-filled fields in TickSample |const|.
- Reorders the fields so that the constructor-filled ones are before the ones
that get filled in later.
- Omits mContext on Mac via conditional compilation, to make the omission
clearer.
--HG--
extra : rebase_source : f3e392c4cf777df5b9f39577af82615890137018
LastSample only makes sense for periodic samples, which are written to the
global ProfileBuffer. It doesn't make sense for synchronous samples which are
written to their own unshared buffer. At the moment it doesn't hurt to use
them in this nonsensical way, but the ThreadInfo profiler_get_backtrace()
will be removed soon, and we won't even have a LastSample to use nonsensically.
So this patch makes the LastSample argument to addTagThreadId() optional. Which
means we have to pass in a ThreadId, so there's no longer much point
duplicating the ThreadId in LastSample, so the patch removes that field too.
This avoids the possibility of the duplicate ThreadId failing to match, which
is nice.
--HG--
extra : rebase_source : dad76ff8b33663398e6f45f85da500b0fd7a598f
Editors generally look for configurations at the top level of a project. For ESLint, they also look for the specific binary in node_modules before defaulting to the system binary. Whilst you can override the location, generally it doesn't work well when switching between projects.
The custom in-tree libraries make setup of a system ESLint more difficult as well.
Therefore to make it simple for developers to pick up the ESLint integrations with Editors, by moving the package.json and associated node_modules to the top-level directory.
MozReview-Commit-ID: 1pQpd7hTQ61
--HG--
rename : tools/lint/eslint/npm-shrinkwrap.json => npm-shrinkwrap.json
rename : tools/lint/eslint/package.json => package.json
extra : rebase_source : 9d69d791f86b5c55b1fcd5f6449f0ab84e56b05c
At this point the only things in the ThreadInfo it uses are the thread name and
id, which are easy to store instead. This gets a step closer to avoiding the
use of ThreadInfo in profiler_get_backtrace().
--HG--
extra : rebase_source : f4feb08ec9fe7880ee43f784c6878c1c04fd3294
StreamSamplesAndMarkers() is the only ThreadInfo method called on
ProfilerBacktrace::mThreadInfo. Furthermore, it doesn't use all that much stuff
from ThreadInfo, and what stuff it does use we can instead pass in as
arguments.
This patch moves StreamSamplesAndMarkers() out of the class. It's a little
ugly, but a necessary precursor for removing ProfilerBacktrace::mThreadInfo and
all the subsequent improvements.
--HG--
extra : rebase_source : 417bda4f29a27c525f7240d3427494dd86b9a868
This required a tweak to DoNativeBacktrace() to work around an ASAN false
positive.
--HG--
extra : rebase_source : 2e21ae4c132db812150f42c26aa708aefce311be
When the profiler is running in privacy mode, we don't want to include dynamic
strings from PROFILER_LABEL_DYNAMIC to end up in the profile.
Rather than checking this every time we enter a scope marked with
PROFILER_LABEL_DYNAMIC, with this patch we will push the dynamic string into
the pseudo stack entry regardless, and then check the privacy mode during
sampling and ignore the dynamic string as necessary.
This way we can avoid taking the profiler state lock in PROFILER_LABEL_DYNAMIC
and also save a branch.
MozReview-Commit-ID: 5dXrtMuFJ5r
--HG--
extra : rebase_source : 1c2057e7ced332d9001137b5b280feab77a712e5
The checking on the return of value of fetch/locate is not working as intended.
The code should be early returning, but the return value of fetch/locate is 1 in
failure cases, which will not trigger the early return. The return value has
been altered to None, which will trigger the early return.
MozReview-Commit-ID: FDCW6rbZI6T
--HG--
extra : rebase_source : 8b087174db268d7d6a0bb62e6537f82a3b15b28d
"Metadata" regularly confuses me, because it suggests something complicated
rather than a simple enum.
This change also has the benefit of removing inconsistent capitalization
("Metadata" vs. "MetaData").
--HG--
extra : rebase_source : b651e124142c8d93139d22dae1c993c899be4d7a
This patch:
- Removes TRACING_EVENT_BACKTRACE, which is unused.
- Removes TRACING_DEFAULT and replaces all its uses with TRACING_EVENT, because
there is no difference in how those two are used.
- Removes TRACING_TIMESTAMP, which is unused and also doesn't do anything
different to TRACING_EVENT.
--HG--
extra : rebase_source : 69af1c53aa918798d8050e6b9d1a2658a0902af5
This patch simplifies and increases the consistency of how HAVE_NATIVE_UNWIND
is used.
- Its definition is moved from platform.h to platform.cpp, because the latter
is the only file that uses it.
- It's now defined in the same place as USE_{NS,EHABI,LUL}_STACKWALK, and used
in preference to those, where possible. Also, it's now defined on Linux and
Android even if MOZ_PROFILING is not.
- HAVE_NATIVE_UNWIND is now used consistently and by itself for all relevant
conditions, including when defining the presence and use of the "stackwalk"
feature.
- The patch inlines and removes is_native_unwinding_avail().
Note that MOZ_PROFILING must be defined for HAVE_NATIVE_UNWIND to be true on
Windows and Mac, but not on Linux and Android.
--HG--
extra : rebase_source : 5be3e5fe65706a15179a2cf46ba9451f68fff815
Bug 1339695 part 8 unintentionally changed behaviour in profiler_init() when
MOZ_PROFILING is undefined. This patch undoes that change.
--HG--
extra : rebase_source : 16e992382e06fbc673555c87499c236e2b39bc7f
This change renames the following Marionette preferences:
marionette.defaultPrefs.enabled marionette.enabled
marionette.defaultPrefs.port marionette.port
marionette.force-local marionette.forcelocal
marionette.logging marionette.log.level
The old preference names are still usable, but are henceforth considered
deprecated. They will be removed when Firefox 55 ships.
It also registers these preferences in Firefox so that they are
discoverable through about:config.
This patch also refactors testing/marionette/components/marionette.js.
MozReview-Commit-ID: 1dAMQS2e0og
--HG--
extra : rebase_source : e59a9e656442d8e8cebeaf5e741dc8306436bc22
No functional changes apart from class'ifying the file and harmonising
the export symbol with the rest of the Marionette code base by renaming
the MarionetteServer class to server.TCPListener.
MozReview-Commit-ID: 8WJKoprGGvr
--HG--
extra : rebase_source : 1f706a0f5e4c9fdccac90bfc14f767763ec85257
This patch does the following.
- Renames TickSample's members to mFoo style.
- Changes TickSample's constructor to set mTimeStamp.
- Moves TickSample creation from
SamplerThread::SuspendAndSampleAndResumeThread() to SamplerThread::Run(), so
it's not repeated for each platform.
- Changes TickSample::PopulateContext() so it takes a |tick_context_t*|
parameter, which avoids having to cast from |void*|.
Bug 1339695 part 8 accidentally disabled native stack walking on Android by
using GP_arm_android instead of GP_PLAT_arm_android in a #if. This patch fixes
that. It also fixes a couple of compile errors that crept into the relevant
code while it was disabled.
--HG--
extra : rebase_source : a7a94b018b8de7a7ca3c621a2b662859a65e69c1
This patch adds logging to some important functions that currently lack it.
Thread registration/unregistration is done with DEBUG_LOG because it's
more verbose than the other profiler logging, but less verbose than LUL's
logging.
The patch also scraps the BEGIN/END logging pairs because they bloat the output
for little gain. Now it just logs on function entry.
--HG--
extra : rebase_source : 3ef3d263c19cda03198e8b3a9ab89866f74ed1cd
The profiler will use level 3 (Info) and 4 (Debug) logging, though this patch
only uses level 3. LUL will use level 5 (Verbose) debugging.
The patch also tweaks parts of the the usage message, including adding
MOZ_PROFILER_{STARTUP,SHUTDOWN} to it.
--HG--
extra : rebase_source : f43a023912fbce993ed367cdd26b8f25f25381de
It's a very general mechanism for replacing the implementation of
printf_stderr().
It's primarily used by the profiler, sparingly, and not in an important way.
Worse, it prevents us from using MOZ_LOG in the profiler, which is something I
want. Because if any code that locks gPSMutex also calls MOZ_LOG, that then
calls printf_stderr(), which calls profiler_log(), which locks gPSMutex, which
deadlocks.
The only other use of set_stderr_callback() is for the ultra-hacky,
for-local-use-only copy_stderr_to_file() function, which was added for B2G
debugging and is no longer necessary.
This patch removes set_stderr_callback() altogether.
--HG--
extra : rebase_source : d31ecb482fe5899f62dc56a38e87d91f9271bab0
All three platform-*.cpp files have similar structure, most especially for
SamplerThread::Run(), with considerable duplication. This patch factors out
the common parts into a single implementation in platform.cpp.
* The top level structure of class SamplerThread has been moved to
platform.cpp.
* The class has some target-dependent fields, relating to signal handling and
thread identity.
* There's a single implementation of Run() in platform.cpp.
* AllocPlatformData() and PlatformDataDestructor::operator() have also been
commoned up and moved into platform.cpp.
* Time units in SamplerThread have been tidied up. We now use microseconds
throughout, except in the constructor. All time interval field and variable
names incorporate the unit (microseconds/milliseconds) for clarity. The
Windows uses of such values are scaled up/down by 1000 accordingly.
* The pre-existing MacOS Run() implementation contained logic that attempted
to keep "to schedule" in the presence of inaccuracy in the actual sleep
intervals. This now applies to all targets. A couple of comments on this
code have been added.
* platform-{win32,macos,linux-android}.cpp have had their Run() methods
removed, and all other methods placed in the same sequences, to the extent
that is possible.
* In the Win32 and MacOS implementations, Thread::SampleContext has been
renamed to Thread::SuspendSampleAndResumeThread as that better describes
what it does. In the Linux/Android implementation there was no such
separate method, so one has been created.
* The three Thread::SuspendSampleAndResumeThread methods have been commented
in such a way as to emphasise their identical top level structure.
* The point in platform.cpp where platform-{win32,macos,linux-android}.cpp are
#included has been moved slightly earlier in the file, into the
SamplerThread encampment, as that seems like a better place for it.
--HG--
extra : rebase_source : 0f93e15967b810c09e645fa593dbf85f94b53a9b
ProfileBuffer::FindLastSampleOfThread currently involves a linear search
backwards through the sample buffer. Profiling showed that to be the largest
profiler cost by far, at least on Linux. Bugs 1344118 and 1344258
significantly improve the situation, collectively reducing the cost by a
factor of at least 5 and often much more. But the linear search is still
present and still dominant. The worst of it is that it's unnecessary: we
could achieve the same by recording the start point of the most recent sample
for each thread in that thread's ThreadInfo record.
This patch does exactly that, adding the type ProfileBuffer::LastSample to
store the start points. LastSample also includes the ID of the thread it
pertains to as a read-only field, as that is needed in various places.
addTag doesn't check whether we're overwriting buffer entries containing start
points. Instead, FindLastSample checks whether the entry pointed to the
LastSample it is given still contains a marker.
--HG--
extra : rebase_source : 2987ec744a5c16e8b6814abe7efb507fc7280605
Replace it with profiler_is_active() in one place, and simply remove it in the
other places. These other places are:
- Around the call to profiler_OOP_exit_profile: profiler_OOP_exit_profile
itself already checks whether the profiler is running and does nothing if
it's not.
- When handling the 'profiler-subprocess-gather' notification. This
notification is sent by the profiler because it's interested in the
profile, so there's little reason to reject it.
- In RecvProfile: If the child process sent us a profile, it did so in
response to a GatherProfile request, so chances are that we're still
interested in that response.
These changes may get us a little closer to a state where you can call
getProfileDataAsync, stop the profiler before the content process profiles
have all come in, and then still receive a response with all the profiles.
At the moment, stopping the profiler will abort the profile gathering process,
but that seems more like an accident and less like the behavior you'd want.
MozReview-Commit-ID: 2tRXC70BztJ
--HG--
extra : rebase_source : 3b2f6f51d75d5f0d439e1a815d84164a5a763603
Instead of copying and concatenating strings into an mDest buffer in
SamplerStackFramePrintfRAII, require callers to keep the string buffer alive
for the duration of the current scope, and store the pointer to the annotation
string in the ProfileEntry. During stackwalking, concatenate the label and the
annotation (separated by a space) and store the resulting string in the
profile buffer.
MozReview-Commit-ID: GEjcLrhhdvb
--HG--
extra : rebase_source : 683749421ee2122805a249cf413e882ee5f33331
sutagent is no longer built or used; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and test harness options like --dm_trans are eliminated.
It's not necessary and causes hangs.
The patch also inlines setup_atfork() and moves the Linux-only code closer to
the Linux-only PlatformInit(), and tweaks the comments a bit.
--HG--
extra : rebase_source : 0db23d649d9468b9308b881c0bbf5ea25a95ea13
This patch forks the breakpad files used for GUID generation and replaces them
with copies living together with the rest of the forked sources. Since other
files in the breakpad sources rely on the original files being present they
haven't been removed but we deliberately define their inclusion definitions in
the build system to prevent them from contributing to the build. This ensures
that we always use our own version of the GUID generator while not touching
the non-forked sources.
MozReview-Commit-ID: 5kW74yAPUDW
--HG--
rename : toolkit/crashreporter/google-breakpad/src/common/linux/guid_creator.cc => toolkit/crashreporter/breakpad-client/linux/handler/guid_generator.cc
rename : toolkit/crashreporter/google-breakpad/src/common/linux/guid_creator.h => toolkit/crashreporter/breakpad-client/linux/handler/guid_generator.h
extra : rebase_source : 60eb027258833996777469d2a13142077486bee9
This doesn't allow --fix to run on the whole tree in one go, but does allow it to be run on individual directories. This
is due to an ESLint issue, where once the processor is loaded, it applies to the rest of the files.
MozReview-Commit-ID: Fqg5yK22Dhu
--HG--
rename : tools/lint/eslint/eslint-plugin-mozilla/.eslintrc.js => tools/lint/eslint/.eslintrc.js
rename : tools/lint/eslint/eslint-plugin-mozilla/lib/processors/self-hosted.js => tools/lint/eslint/eslint-plugin-spidermonkey-js/lib/processors/self-hosted.js
extra : rebase_source : 9913f929985016f4a22ff78641051dc025df14f2
Currently the mach clang-format command is formating '-r tip^', which is
everything in the tip revision, plus changes. Often this does what you want,
but if you're using changeset evolution or history rewriting this doesn't work
so well. If you `hg up $rev` to some other revision with the intention of
clang-formating that revision, you don't get that revision formated when you
run clang-format. You get the lines touched by the changeset with the "tip"
bookmark formatted. We really want to have clang-format format '-r .^' which
means "the current revision in the working dir, plus any uncommited changes".
This results in the expected behaviour, and means clang-format can be used to
re-format commits which are work-in-progress and/or being evolved.
MozReview-Commit-ID: iv82HENxiA
--HG--
extra : rebase_source : a1d05472f499f5a68bcaab9ce260d60ecbca067d
sutagent is no longer built or usedr; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and the --dmTrans and similar options have been removed
from test harnesses and mach commands.
Specifically:
- Improve the documentation on all the profiler_*() functions, esp. about what
they do when the profiler is inactive.
- Fix numerous spelling mistakes.
- Wrap all comment lines at 80 chars.
- Change /**/ comments to // comments throughout.
--HG--
extra : rebase_source : 17f766301416806764413cea7f949c42e7721ab9
Specifically:
- Renames kGeckoThreadName as kMainThreadName, and RegisterCurrentThread() as
locked_register_thread().
- Removes the aIsMainThread parameter from locked_register_thread(). It can be
computed via NS_IsMainThread().
- Moves the PseudoStack initialization within locked_register_thread(), so it's
done in a single place. Also calls init() at that place; previously the
non-main threads didn't have init() called on their tlsPseudoStack! According
to ThreadLocal.h this shouldn't have worked...
- Adds some !NS_IsMainThread() assertions to
profiler_{register,unregister}_thread().
--HG--
extra : rebase_source : c44836dddc988c1f49ec91eccd35f088e0d30ed1
It took me some time to understand this snippet of code, so I wrote a comment
about it.
--HG--
extra : rebase_source : b25fc8384a4fe00df7d8a0585e7a1944226fa212
ProfilerBacktrace.h doesn't need to be visible outside the profiler because
the ProfilerBacktrace type is only used in pointers outside the profiler and
the existing forward declaration in GeckoProfiler.h suffices for that.
--HG--
rename : tools/profiler/public/ProfilerBacktrace.h => tools/profiler/core/ProfilerBacktrace.h
extra : rebase_source : 50752626fbd6948de26e193956ed7b82b41b1830
API before this change:
- nsIProfiler::getSharedLibraryInformation() returns a string containing a
JSON array of libraries.
- The profile format is at version 3.
- Every profile has a "libs" field that contains the same JSON string as the
return value of nsIProfiler::getSharedLibraryInformation.
- The array of libraries is not sorted.
- Each library has a "name" field that contains:
- The module's debug name on Windows
- The full path to the binary on Mac + Linux
API after this change:
- nsIProfiler::getSharedLibraryInformation() is removed.
- nsIProfiler has a readonly property called sharedLibraries.
- The profile format is at version 4.
- Every profile has a "libs" field that contains the same array as
nsIProfiler.sharedLibraries, no longer as a JSON string but as a regular
array.
- The array of libraries is sorted by start address.
- Each library has a "name" field that contains the binary file's basename,
on all platforms.
- Each library has a "path" field that contains the full path to the binary,
on all platforms.
- Each library has a "debugName" field that contains the library's debug
name, on all platforms. On Windows, the debug name is the filename
(basename) of the pdb file for that binary. On other platforms, debugName
is the same as |name|.
- Each library has a "debugPath" field that contains the absolute path
library's pdb file on Windows; on non-Windows, debugPath and path are the
same.
- Each library has an "arch" field that is either an empty string (Linux +
Windows) or the library's architecture; it'll differentiate between the
architectures "x86_64" and "x86_64h". (x86_64h is used for binaries that
contain instructions that are specific to the Intel Haswell
microarchitecture.)
MozReview-Commit-ID: 8Nrs4dyHhDS
--HG--
extra : rebase_source : 4039926ae4d776bf53ea71df5fe3f8200d3e2784
extra : source : 4e282aa03422de5b8d51e1aaeb3e53ee547293dd
This is interesting information on Mac because some system libraries come in
two 64 bit versions: a regular one, and one that contains Haswell-specific
instructions. The former 'architecture' is called x86_64 and the latter is
called x86_64h (h for Haswell).
We set arch to the empty string on non-Mac platforms. It's not all that
interesting on those platforms because there will be only one architecture
in the binary at the given path, unlike on Mac where you can have fat
binaries with multiple architectures.
MozReview-Commit-ID: Dgnslv0D3Ug
--HG--
extra : rebase_source : cf1fcfc1635d373eaeea35aad2f73b235097c748
API before this change:
- nsIProfiler::getSharedLibraryInformation() returns a string containing a
JSON array of libraries.
- The profile format is at version 3.
- Every profile has a "libs" field that contains the same JSON string as the
return value of nsIProfiler::getSharedLibraryInformation.
- The array of libraries is not sorted.
- Each library has a "name" field that contains:
- The module's debug name on Windows
- The full path to the binary on Mac + Linux
API after this change:
- nsIProfiler::getSharedLibraryInformation() is removed.
- nsIProfiler has a readonly property called sharedLibraries.
- The profile format is at version 4.
- Every profile has a "libs" field that contains the same array as
nsIProfiler.sharedLibraries, no longer as a JSON string but as a regular
array.
- The array of libraries is sorted by start address.
- Each library has a "name" field that contains the binary file's basename,
on all platforms.
- Each library has a "path" field that contains the full path to the binary,
on all platforms.
- Each library has a "debugName" field that contains the library's debug
name, on all platforms. On Windows, the debug name is the filename
(basename) of the pdb file for that binary. On other platforms, debugName
is the same as |name|.
- Each library has a "debugPath" field that contains the absolute path
library's pdb file on Windows; on non-Windows, debugPath and path are the
same.
- Each library has an "arch" field that is either an empty string (Linux +
Windows) or the library's architecture; it'll differentiate between the
architectures "x86_64" and "x86_64h". (x86_64h is used for binaries that
contain instructions that are specific to the Intel Haswell
microarchitecture.)
MozReview-Commit-ID: 8Nrs4dyHhDS
--HG--
extra : rebase_source : 4039926ae4d776bf53ea71df5fe3f8200d3e2784
extra : source : 4e282aa03422de5b8d51e1aaeb3e53ee547293dd
This is interesting information on Mac because some system libraries come in
two 64 bit versions: a regular one, and one that contains Haswell-specific
instructions. The former 'architecture' is called x86_64 and the latter is
called x86_64h (h for Haswell).
We set arch to the empty string on non-Mac platforms. It's not all that
interesting on those platforms because there will be only one architecture
in the binary at the given path, unlike on Mac where you can have fat
binaries with multiple architectures.
MozReview-Commit-ID: Dgnslv0D3Ug
--HG--
extra : rebase_source : cf1fcfc1635d373eaeea35aad2f73b235097c748
I also tweaked the int type of the JSON writer to match the IntProperty
method.
MozReview-Commit-ID: rtxLDKtJQZ
--HG--
extra : rebase_source : fa924479926cad6e07755d10c049a40ad896c3cd
Running eslint with --fix didn't fix many of the issues. The majority here had to be fixed by hand but a significant majority of the issues were related to a few files that I was able to use find-and-replace with. I regret not making this in to separate commits of the hand-fixes and the fixes from --fix but I don't recall --fix fixing any of the issues.
MozReview-Commit-ID: ANyg2qfo3Qx
--HG--
extra : rebase_source : 61d2aa91bf9474af3d72a5dea41b25dca442c1b7
Currently, JS sampling has major problems.
- JS sampling is enabled for all JS threads from the thread that runs
locked_profiler_start() -- currently only the main thread -- but the JS
engine can't handle enabling from off-thread, and asserts. This makes
profiling workers impossible in a debug build.
- No JS thread will be JS sampled unless enableJSSampling() is called, but that
only happens in locked_profiler_start(). That means any worker threads
created while the profiler is active won't be JS sampled.
- Only the thread that runs locked_profiler_stop() -- currently only the main
thread -- ever calls disableJSSampling(). This means that worker threads that
start being JS sampled never stop being JS sampled.
This patch fixes these three problems in the following ways.
- locked_profiler_start() now sets a flag in PseudoStack that indicates
JS sampling is desired, but doesn't directly enable it. Instead, the JS
thread polls that flag and enables JS sampling itself when it sees the flag
is set. The polling is done by the interrupt callback. There was already a
flag of this sort (mJSSampling) but the new one is better.
This required adding a call to profiler_js_operation_callback() to the
InterruptCallback() in XPCJSContext.cpp. (In comparison, the
InterruptCallback() in dom/workers/RuntimeService.cpp already had such a
call.)
- RegisterCurrentThread() now requests JS sampling of a JS thread when the
profiler is active, the thread is being profiled, and JS sampling is enabled.
- locked_profiler_stop() now calls stopJSSampling() on all live threads.
The patch makes the following smaller changes as well.
- Renames profiler_js_operation_callback() as profiler_js_interrupt_callback(),
because "interrupt callback" is the standard name (viz.
JS_AddInterruptCallback()).
- Calls js::RegisterContextProfilingEventMarker() with nullptr when stopping
JS sampling, so that ProfilerJSEventMarker won't fire unnecessarily.
- Some minor formatting changes.
--HG--
extra : rebase_source : 372f94c963a9e5b2493389892499b1ca205ebc2f
They each have a single call site, and this code makes makes more sense in
platform.cpp than as PseudoStack methods because it's mostly dealing with gPS.
--HG--
extra : rebase_source : 84d8c7d8989f0e02b238d591ece2be110cbbcb1a
It does almost entirely different things depending on whether we are setting or
clearing the context.
--HG--
extra : rebase_source : 356445957968e1d055f4b717c65476fa765d8bd5
PseudoContext::sampleContext() is always called immediately after
profiler_get_pseudo_stack(). This patch introduces profiler_set_js_context()
and profiler_clear_js_context(), which replace the profiler_get_pseudo_stack()
+ sampleContext() pairs. This takes us a step closer to not having to export
PseudoStack outside the profiler.
--HG--
extra : rebase_source : 8558d1600eafd395cc696d31f3d21fb52a1a74b0
This is more of a workaround than a real fix, but the old code wasn't working
for non-main threads either, and we'd like to change the way this information
is computed anyway (bug 1340714) and then we won't need CheckResponsivenessTask
any more.
MozReview-Commit-ID: FGiomjwpk3z
--HG--
extra : rebase_source : 95fc57c0622fdbaa928700a971a4ce839ad92797
Bug 1341255 changed PseudoStack::mStack to a js::ProfileEntry[], so we don't
need this assertion any more.
--HG--
extra : rebase_source : 7fcb5218db56b5a9409d81ff67d653e9db37414e
The most important of these is the one in profiler_get_profile_jsobject().
Currently the JS_ParseJSON() call frequently leads to deadlock, due to
profiler_log() being called within it.
--HG--
extra : rebase_source : 7825229da61ad0d567c11bd455a7a9b8f25b5b78
This patch does the following.
- Introduces NotifyObservers() for the simple notification cases in
platform.cpp.
- Removes profiler_lock() and profiler_unlock() because they do notifications
that the profiler add-on no longer listens for.
--HG--
extra : rebase_source : 77a1868ba494dea314702bbdf9478a1da36c9efb
Calling NotifyObserver() with gPSMutex locked is a bad idea; arbitrary code can
run in observers, which can easily include code that calls other profiler
functions that themselves lock gPSMutex, causing deadlock. This has been seen
in practise with locked_profiler_stop().
This patch moves all but one of the NotifyObserver() calls in platform.cpp to
after the sections where gPSMutex is locked. The remaining call (for the
"profiler-subprocess") is harmless, because it just calls a simple callback
implemented within platform.cpp, and hard to move.
In the future we plan to allow profiler_start() and profiler_stop() to be
called from different threads. When that happens, it will be possible for the
"profiler-start" and "profiler-stop" notifications to arrive out of order.
Unfortunately this seems difficult to avoid. (Well, recursive mutexes would
make this problem much easier, but we don't have those...)
--HG--
extra : rebase_source : 78455c4b2d93a0d4110cdd401d6b542b641dd217
SamplerThread::Join() is always called just before the SamplerThread object is
deleted. Might as well combine them.
--HG--
extra : rebase_source : d7fc013b83ec77e347ffc3929533c5259df77e06
profiler_start() can call locked_profiler_stop(). In that case it must then
call SamplerThread::Join() in order to free the SamplerThread, but it currently
doesn't.
This patch adds such a call. It also marks locked_profiler_start() with
MOZ_MUST_USE to make this mistake less likely in the future.
--HG--
extra : rebase_source : 9045561aa54b54099e710c3eaea5ac533ca5485b
This patch properly synchronizes all the global state in platform*.cpp, which
gets us a long way towards implementing bug 1330184.
- Most of the global state goes in a new class, ProfilerState, with a single
instance, gPS. All accesses to gPS are protected by gPSMutex. All functions
that access ProfilerState require a token proving that gPS is locked; this
makes things much clearer.
gRegisteredThreadsMutex is removed because it is subsumed by gPSMutex.
- gVerbosity, however, does not go in ProfilerState. It stays separate, and
gains its own mutex, gVerbosityMutex.
Also, the tracking of the current profiler state is streamlined. Previously it
was tracked via:
- stack_key_initialized, gInitCount, gSampler, gIsProfiling, gIsActive, and
gIsPaused.
Now it is tracked via:
- gPS, gPS->sActivity, and gPS->mIsPaused.
This means that the Sampler class is no longer necessary, and the patch removes
it.
Other changes of note made by the patch are as follows.
- It removes ThreadInfo::{mMutex,GetMutex}. This mutex was only used in two
places, and both these are now protected by gPSMutex.
- It tweaks the LOG calls. All the main functions (init(), shutdown(), start(),
stop()) now do consistent BEGIN/END logging, and a couple of other low-value
incidental LOG calls have been removed.
- It adds a lot of release assertions requiring that gPS be initialized (e.g.
profiler_init() has been called but profiler_shutdown() has not).
- It uses alphabetical order for everything involving profiler feature names.
- It removes Platform{Start,Stop}() and SamplerThread::{Start,Stop}Sampler().
These are no longer necessary now that SamplerThread::sInstance has been
replaced with ProfilerState::mSamplerThread which allows more direct access
to the current SamplerThread instance.
- It removes PseudoStack::mPrivacyMode. This was derived from the "privacy"
feature, and we now use gPS->mFeaturePrivacy directly, which is simpler.
It also replaces profiler_in_privacy_mode() with
profiler_is_active_and_not_in_privacy_mode(), which avoids an unnecessary
lock/unlock of gPSMutex on a moderately hot path.
Finally, the new code does more locking than the old one. A number of operation
The following operations now lock a mutex when they previously didn't; the
following are ones that are significant, according to some ad hoc profiling.
- profiler_tracing()
- profiler_is_active()
- profiler_is_active_and_not_in_privacy_mode()
- profiler_add_marker()
- profiler_feature_active()
- SamplerThread::Run() [when the profiler is paused]
All up this roughly doubles the amount of mutex locking done by the profiler.
It's probably possible to avoid this increase by allowing careful unlocked
access to three of the fields in ProfilerState (mActivityGeneration,
mFeaturePrivacy, mStartTime), but this should only be done as a follow-up if
the extra locking is found to be a problem.
--HG--
extra : rebase_source : c2e41231f131b3e9ccd23ddf43626b54ccc77b7b
Instead of nulling ThreadInfo::mPseudoStack, the patch changes things so that
ownership of the PseudoStack is transferred to the ThreadInfo. This avoids
crashes in some cases.
The patch also makes ThreadInfo::mPseudoStack a NotNull<>.
--HG--
extra : rebase_source : 95ace8886092ebe17ac0f4431c8c0936946c1f44
When ProfilerBuffer::reset() is called, DuplicateLastSample() will start
failing for all sleeping threads because there will be no prior thread data in
the buffer to duplicate. But the sampling loop doesn't detect such failure.
This causes two problems:
- Missing samples.
- CPU usage goes through the roof, because each time around the sampling loop
the length of the failing search increases.
The fix is simple: detect failure in the sampling loop and do a normal sample
in that case.
The patch also removes ThreadInfo::DuplicateLastSample(), because it just calls
onto ProfileBuffer::DuplicateLastSample().
--HG--
extra : rebase_source : d51709994e701fdd63c292df5f723a2d43c4d754
This is the "repair responder" - it handles a "repairRequest" command sent
by another client and attempts to take the list of IDs that client lists as
missing and upload whatever records are necessary such that the requesting
client would then be likely to find a complete and valid tree on the server.
MozReview-Commit-ID: 4xw19nH6EfL
This patch defines the concept of a "doctor" for collections. The doctor is
responsible for running all validators and deciding whether or not to initiate
a repair request based on the validation results.
MozReview-Commit-ID: 6NLRE6L0OpA
This creates a collection_repair module, somewhat analogous to the existing
collection_validator module. This defines the public interface to request a
new repair and respond to a remote repair request, and also includes changes
to clients.js to call this public interface.
MozReview-Commit-ID: 9JPpRrLgFoR
It's now a very thin wrapper around ThreadInfo, and so can be removed.
The patch also has the bonus of setting mIsMainThread correctly for the
ThreadInfos that used to be SyncProfiles (i.e. the ones created in
profiler_get_backtrace()). As far as I can tell this has only one very minor
effect, because that field is only used for those objects to determine how
ThreadResponsiveness::Update() dispatches its runnables.
--HG--
extra : rebase_source : 39318dba51316dc473321d6e4215e5b950a4d41d
Currently ThreadInfo objects all share gBuffer, while SyncProfile objects each
get their own ProfileBuffer.
This patch removes ThreadInfo::mBuffer to reduce this difference, taking us a
step towards eliminating SyncProfile.
To support this, the patch:
- passes in a buffer as an additional argument in a bunch of places where the
buffer used to be obtained from a ThreadInfo;
- adds an mBuffer field to ProfilerBacktrace;
- changes ThreadInfo::SetProfile() to SetHasProfile();
- removes ThreadInfo::{addTag,StoredMarker,bufferGeneration}(), all of which
just redirected to ThreadInfo anyway;
- changes ProfileBuffer so it's no longer refcounted, which is unnecessary now
that gBuffer and ProfilerBacktrace::mBuffer don't have multiple references,
which makes their lifetimes obvious.
The patch also removes some ThreadInfo& args in functions in platform.cpp, in
places where that ThreadInfo is available within the accompanying TickSampler*
arg.
--HG--
extra : rebase_source : 7e6cb370866d3f3fd657c6aa66d3c3eb3d83a4b1
The MacOS and Windows profiler cores have a threading structure where one
thread ("sampler thread") collects information from a thread to be profiled
("samplee thread") by suspending it, getting its register state, unwinding its
stack, and then resuming it. This requires kernel-level primitives to perform
the suspend, get-registers and resume steps.
The Linux/Android core is different, because none of those three primitives
exist. Until now, data collection has been done by sending a SIGPROF to the
samplee, and collecting all relevant data within the signal handler. This has
a number of disadvantages:
(1) Current work to rationalise/clean up the threading structure of the
profiler is complicated by the need to reason about/verify two different
schemes.
In particular, the Tick call in the Windows and MacOS implementations will
produce its output on the sampler thread. In the Linux implementation
that is produced on the sampled threads.
(2) Dynamic verification results (primarily, absence of data races and
deadlocks) established for the Linux implementation are less likely to
carry over to the other two implementations, because the threading
structures are different.
(3) It causes a lot of duplicated code in platform-*.cpp. For example
SamplerThread::Run() in the -win32.cpp and -macos.cpp files are very
similar. Ideally all three could be merged into a single file with the
identical logic commoned up.
(4) Running lots of code -- the entire contents of Tick -- in a signal handler
isn't considered good practice. POSIX severely restricts the set of
functions we can safely call from within a signal handler.
This commit changes the Linux implementation by using semaphores to implement
the suspend and resume primitives, and moves the bulk of the data collection
work to the sampler thread. By doing this, it causes the Linux implementation
to have the same threading structure as the other two.
--HG--
extra : rebase_source : 675b6ef76915d164ed263b831dddd6ce0c0e97f3
For now, we return virtually no globals for browser-window/places-overlay/simpletest as they aren't able to load the m-c files. Later we may want to find a way of enabling this to work for outside repositories.
MozReview-Commit-ID: 8SFVuQuEqfL
--HG--
extra : rebase_source : ff773796d14ef27b47f25b0b8503ae3f093bfb9e
Because profiler_time() is going to need the global lock when I add it, and the
lock will already be held when streaming is happening, so it'll cause the
thread to deadlock itself.
Unfortunately this requires adding an |aStartTime| parameter to a lot of
functions, but this is the least worst way I can think of handling it.
This also removes the need for one of the profiler_time() functions, which the
patch removes.
This makes it more like platform-{macos,win32}.cpp, and will make it easier to
introduce locking around all the globals in platform.cpp.
The following things were moved into the new class.
- Types:
- SigHandlerCoordinator
- Functions:
- SigProfHandler()
- SigprofSender() as Run()
- Variables:
- gOldSigprofHandler as mOldSigprofHandler
- gSigprofSenderThread as mThread
- gIntervalMicro as mIntervalMicro
- gSigHandlerCoordinator as sSigHandlerCoordinator
sInstance is the singleton instance. PlatformStart() and PlatformStop() now
just delegate to StartSampler/StopSampler.
The patch also does the following tidy-ups.
Linux:
- gWasPaused is now cleared in the parent and child processes after forking.
- LUL: inlined and removed gLUL_initialization_routine().
- LUL: now only calling EnabledUnwinding() and doing the unit tests on the
first call to profiler_start(), instead of all of them.
Mac:
- Removed a useless call to pthread_self() -- mThread is already set by the
earlier call to pthread_create().
- Removed some low-value checking involving kNoThread.
Mac + Win:
- Renamed SamplerThread::mInstance as sInstance, because it's static.
- Merged SamplerThread::Start() with the constructor.
- Tweaked how mInterval/mIntervalMicro is initialized, so it can become const.
All platforms:
- Converted NULL to nullptr throughout.
- A few other very minor clean-ups, e.g. formatting.
--HG--
extra : rebase_source : 4d10be873c4a7544f450e6a3b3d86c5d5b2fb603
There's no need to lock when calling Tick() on a local TickSample that uses a
fresh SyncProfile with its own fresh ProfileBuffer -- none of that data can
be touched by another thread.
--HG--
extra : rebase_source : aaabef89e8481758b566e6dd01e4bb61a5855b1a
Both of these functions are now trivial and identical in both ThreadInfo and
SyncProfile. This patch inlines and removes them.
--HG--
extra : rebase_source : 15fb7c1d4df9fbc80d8e671761b4aa1508845cac
It appears to be a remnant of a time when SyncProfile lifetimes were more
complex. Nowadays they are simple.
- profiler_get_backtrace() constructs a SyncProfile called |profile|.
|profile|'s mOwnerState is REFERENCED.
- profiler_get_backtrace() then calls BeginUnwind() and EndUnwind() on
|profile|. After the EndUnwind(), |profile->mOwnerState| is always OWNED.
- |profile| then is put into the returned ProfilerBacktrace. That
ProfilerBacktrace will destroy |profile| in its destructor because
ShouldDestroy() always returns true because mOwnerState is always OWNED.
The OWNER_DESTROYING and ORPHANED states are never used, and the whole
OwnerState type isn't necessary. This patch removes it and ShouldDestroy().
--HG--
extra : rebase_source : f1828b4a5d6b8f73245e666f457b93a24ca7266e
Wes Kocher
Nicholas Nethercote
Julian Seward
Mark Banner
Iris Hsiao
Kan-Ru Chen
Carsten "Tomcat" Book
Kearwood Gilbert
Joel Maher
Andrew McCreight
Markus Stange
Bryce Van Dyk
Edouard Oger
Florian Queze
Sebastian Hengst
Andreas Tolfsen
Thinker K.F. Li
Luke Wagner
Geoff Brown
Gabriele Svelto
Chris Pearce
Greg Tatum
Jared Wein
Sylvestre Ledru
Andi-Bogdan Postelnicu
Christoph Diehl
Dave Townsend
Dan Minor
djmdev
Zibi Braniecki
David Major
Mark Hammond
André Bargull
Marco Castelluccio