PowerOfTwo makes for a cleaner and more expressive interface, showing that the
profiler will use a power-of-2 storage size.
Using PowerOfTwoMask in ProfilerBuffer also makes it more obvious that we want
cheap modulo operations.
And we don't need to keep the original capacity, as it's only used once and can
easily be recomputed from the mask.
Differential Revision: https://phabricator.services.mozilla.com/D36027
--HG--
extra : moz-landing-system : lando
PowerOfTwo stores a power of 2 value, i.e., 2^N.
PowerOfTwoMask stores a mask corresponding to a power of 2, i.e., 2^N-1.
These should be used in places where a power of 2 (or its mask) is stored or
expected.
`% PowerOfTwo{,Mask}` and `& PowerOfTwoMask` operations are optimal.
MakePowerOfTwo{,Mask}<T, Value>() may be used to create statically-checked
constants.
{,Make}PowerOfTwo{,Mask}{32,64} shortcuts for common 32- and 64-bit types.
Differential Revision: https://phabricator.services.mozilla.com/D36026
--HG--
extra : moz-landing-system : lando
PowerOfTwo makes for a cleaner and more expressive interface, showing that the
profiler will use a power-of-2 storage size.
Using PowerOfTwoMask in ProfilerBuffer also makes it more obvious that we want
cheap modulo operations.
And we don't need to keep the original capacity, as it's only used once and can
easily be recomputed from the mask.
Differential Revision: https://phabricator.services.mozilla.com/D36027
--HG--
extra : moz-landing-system : lando
PowerOfTwo stores a power of 2 value, i.e., 2^N.
PowerOfTwoMask stores a mask corresponding to a power of 2, i.e., 2^N-1.
These should be used in places where a power of 2 (or its mask) is stored or
expected.
`% PowerOfTwo{,Mask}` and `& PowerOfTwoMask` operations are optimal.
MakePowerOfTwo{,Mask}<T, Value>() may be used to create statically-checked
constants.
{,Make}PowerOfTwo{,Mask}{32,64} shortcuts for common 32- and 64-bit types.
Differential Revision: https://phabricator.services.mozilla.com/D36026
--HG--
extra : moz-landing-system : lando
BIONIC is only platform that actually supports gettid. Easiest
solution is to check for linux and disable for BIONIC platform. This
includes the change requested by Gerald to keep the two profilers sync'd.
Differential Revision: https://phabricator.services.mozilla.com/D34919
--HG--
extra : moz-landing-system : lando
Update the tests for ARM64 to include additional functions that are now
supported via 4 byte patching.
We also convert the TEST macros to accept the DLL names as strings, as this
works better with clang-format.
Differential Revision: https://phabricator.services.mozilla.com/D32209
--HG--
extra : moz-landing-system : lando
This patch modifies arm64 so that detours are peformed via two passes:
1. The first pass uses a null trampoline to count how many bytes are available
for patching the original function.
2. If we have >= 16 bytes to patch, we reuse existing trampoline space. If we
have less than 16 bytes to patch, we reserve trampoline space within 128MB
of the function, allowing for a 4 byte patch.
3. Then we recurse, this time using a real trampoline.
Note that we still do a single-pass on x86(-64).
Differential Revision: https://phabricator.services.mozilla.com/D32193
--HG--
extra : moz-landing-system : lando
A null trampoline is just a trampoline that is not backed by a VM reservation.
These are used for tracking the number of bytes that are needed to make a patch.
This patch also contains the changes needed to work with TrampolinePool.
Differential Revision: https://phabricator.services.mozilla.com/D32192
--HG--
extra : moz-landing-system : lando
VMSharingPolicyShared needs to become much smarter. This patch modifies that
policy to track different VM reservations and reuse them whenever possible.
We add TrampolinePools to abstract away the differences between VM policies
with respect to the caller who is making the reservation.
Differential Revision: https://phabricator.services.mozilla.com/D32191
--HG--
extra : moz-landing-system : lando
In order to support 4-byte patches on ARM64, we need to be able to reserve
trampoline space within +/- 128 MB of the beginning of a function.
These changes allow us to make such reservations using OS APIs when
available.
Differential Revision: https://phabricator.services.mozilla.com/D32190
--HG--
extra : moz-landing-system : lando
VMSharingPolicyShared needs to become much smarter. This patch modifies that
policy to track different VM reservations and reuse them whenever possible.
We add TrampolinePools to abstract away the differences between VM policies
with respect to the caller who is making the reservation.
Differential Revision: https://phabricator.services.mozilla.com/D32191
--HG--
extra : moz-landing-system : lando
In order to support 4-byte patches on ARM64, we need to be able to reserve
trampoline space within +/- 128 MB of the beginning of a function.
These changes allow us to make such reservations using OS APIs when
available.
Differential Revision: https://phabricator.services.mozilla.com/D32190
--HG--
extra : moz-landing-system : lando
We remove the debugging hooks that were added to check to see whether a DLL
was loaded, as we can just as easily check that by querying the loader itself.
Plus, we shouldn't be exporting a bunch of test-only loader hooks from mozglue
in our release builds, which is what we are currently doing.
We also remove Injector, InjectorDLL, and TestDLLEject, as these tests can
just as easily be done from within TestDllBlocklist by creating a thread with
LoadLibrary* as the entry point. The CreateRemoteThread stuff, while a more
accurate simulation, has no material effect on whether or not the thread
blocking code works.
Differential Revision: https://phabricator.services.mozilla.com/D34444
--HG--
extra : moz-landing-system : lando
We also s/mincore/version/ in OS_LIBS because the former breaks the test on
Windows 7.
Differential Revision: https://phabricator.services.mozilla.com/D34437
--HG--
extra : moz-landing-system : lando
The profiler will require non-fuzzed timers for accuracy. Making the switch early will avoid surprises when FuzzyFox is enabled.
Differential Revision: https://phabricator.services.mozilla.com/D31010
--HG--
extra : moz-landing-system : lando
Start using BaseProfiler in Firefox main(), before&after XPCOM runs.
Also added a BaseProfiler label around Gecko Profiler init/shutdown (so that
samples may be ignored if user is only interested in non-XPCOM profiling).
Main process name changed to "Main Thread (Base Profiler)", so as not to confuse
the front-end, and show where this thread comes from.
Differential Revision: https://phabricator.services.mozilla.com/D31933
--HG--
extra : moz-landing-system : lando
If MOZ_BASE_PROFILER_STARTUP and MOZ_PROFILER_STARTUP are set, this will integrate
a pre-XPCOM startup profile into the main profile.
It is stored as separate threads (in a single JSON string that is moved around),
which will appear as a new track under the main process.
Only adding threads from BaseProfiler means a better integration with Gecko
Profiler profiles, and is more efficient: Less code, and a smaller memory
footprint.
Differential Revision: https://phabricator.services.mozilla.com/D31932
--HG--
extra : moz-landing-system : lando
Running identical (but separate) InitializeWin64ProfilerHooks in both profilers
confuses the DLL interceptor and the 2nd one crashes because of unexpected
opcodes introduced by the 1st one.
If MOZ_BASE_PROFILER is defined, Gecko Profiler will use that implementation of
InitializeWin64ProfilerHooks instead of its own; and that code also has a guard
so that it effectively only run once even if called from both profilers.
Differential Revision: https://phabricator.services.mozilla.com/D31931
--HG--
extra : moz-landing-system : lando
E.g., AUTO_PROFILER_INIT -> AUTO_BASE_PROFILER_INIT.
This will allow #including BaseProfiler.h anywhere as needed, without clashing
with Gecko Profiler macros.
Differential Revision: https://phabricator.services.mozilla.com/D31929
--HG--
extra : moz-landing-system : lando
Notice the extra 'BASE' in the env-var names.
This is to control BaseProfiler separately from the Gecko Profiler.
Differential Revision: https://phabricator.services.mozilla.com/D31928
--HG--
extra : moz-landing-system : lando
Android not implemented yet.
Windows not working yet when packaged, so disabled by default, but may be
enabled locally by uncommenting `#define MOZ_BASE_PROFILER` where indicated in
BaseProfiler.h.
Differential Revision: https://phabricator.services.mozilla.com/D31927
--HG--
extra : moz-landing-system : lando
Simple test program that exercises the most important APIs of BaseProfiler.
(Including checking that macros work even when BaseProfiler is not enabled.)
Differential Revision: https://phabricator.services.mozilla.com/D31926
--HG--
extra : moz-landing-system : lando
Almost-mechanical changes include:
- Removed unneeded/incompatible #includes and functions (any JS- or XPCOM-
dependent).
- Use std::string for strings and nsIDs.
- Use thin wrappers around mozilla::detail::MutexImpl for mutexes.
- Use hand-rolled AddRef&Release's for ref-counted classes -- could not use
mfbt/RefCounted.h because of bug 1536656.
- Added some platform-specific polyfills, e.g.: MicrosecondsSince1970().
- Only record the main thread by default.
- Logging controlled by env-vars MOZ_BASE_PROFILER_{,DEBUG_,VERBOSE_}LOGGING.
This now builds (with --enable-base-profiler), but is not usable yet.
Differential Revision: https://phabricator.services.mozilla.com/D31924
--HG--
extra : moz-landing-system : lando
Added baseprofiler to mozglue/moz.build, so it will be built.
However all cpp files are dependent on `MOZ_BASE_PROFILER`, which is currently
not #defined by default (in public/BaseProfiler.h).
Added mozglue/mozprofiler to js/src/make-source-package.sh, because
mozglue/moz.build now refers to it.
Differential Revision: https://phabricator.services.mozilla.com/D33258
--HG--
extra : moz-landing-system : lando
Start using BaseProfiler in Firefox main(), before&after XPCOM runs.
Also added a BaseProfiler label around Gecko Profiler init/shutdown (so that
samples may be ignored if user is only interested in non-XPCOM profiling).
Main process name changed to "Main Thread (Base Profiler)", so as not to confuse
the front-end, and show where this thread comes from.
Differential Revision: https://phabricator.services.mozilla.com/D31933
--HG--
extra : moz-landing-system : lando
If MOZ_BASE_PROFILER_STARTUP and MOZ_PROFILER_STARTUP are set, this will integrate
a pre-XPCOM startup profile into the main profile.
It is stored as separate threads (in a single JSON string that is moved around),
which will appear as a new track under the main process.
Only adding threads from BaseProfiler means a better integration with Gecko
Profiler profiles, and is more efficient: Less code, and a smaller memory
footprint.
Differential Revision: https://phabricator.services.mozilla.com/D31932
--HG--
extra : moz-landing-system : lando
Running identical (but separate) InitializeWin64ProfilerHooks in both profilers
confuses the DLL interceptor and the 2nd one crashes because of unexpected
opcodes introduced by the 1st one.
If MOZ_BASE_PROFILER is defined, Gecko Profiler will use that implementation of
InitializeWin64ProfilerHooks instead of its own; and that code also has a guard
so that it effectively only run once even if called from both profilers.
Differential Revision: https://phabricator.services.mozilla.com/D31931
--HG--
extra : moz-landing-system : lando
E.g., AUTO_PROFILER_INIT -> AUTO_BASE_PROFILER_INIT.
This will allow #including BaseProfiler.h anywhere as needed, without clashing
with Gecko Profiler macros.
Differential Revision: https://phabricator.services.mozilla.com/D31929
--HG--
extra : moz-landing-system : lando
Notice the extra 'BASE' in the env-var names.
This is to control BaseProfiler separately from the Gecko Profiler.
Differential Revision: https://phabricator.services.mozilla.com/D31928
--HG--
extra : moz-landing-system : lando
Android not implemented yet.
Windows not working yet when packaged, so disabled by default, but may be
enabled locally by uncommenting `#define MOZ_BASE_PROFILER` where indicated in
BaseProfiler.h.
Differential Revision: https://phabricator.services.mozilla.com/D31927
--HG--
extra : moz-landing-system : lando
Simple test program that exercises the most important APIs of BaseProfiler.
(Including checking that macros work even when BaseProfiler is not enabled.)
Differential Revision: https://phabricator.services.mozilla.com/D31926
--HG--
extra : moz-landing-system : lando
Almost-mechanical changes include:
- Removed unneeded/incompatible #includes and functions (any JS- or XPCOM-
dependent).
- Use std::string for strings and nsIDs.
- Use thin wrappers around mozilla::detail::MutexImpl for mutexes.
- Use hand-rolled AddRef&Release's for ref-counted classes -- could not use
mfbt/RefCounted.h because of bug 1536656.
- Added some platform-specific polyfills, e.g.: MicrosecondsSince1970().
- Only record the main thread by default.
- Logging controlled by env-vars MOZ_BASE_PROFILER_{,DEBUG_,VERBOSE_}LOGGING.
This now builds (with --enable-base-profiler), but is not usable yet.
Differential Revision: https://phabricator.services.mozilla.com/D31924
--HG--
extra : moz-landing-system : lando
Added baseprofiler to mozglue/moz.build, so it will be built.
However all cpp files are dependent on `MOZ_BASE_PROFILER`, which is currently
not #defined by default (in public/BaseProfiler.h).
Added mozglue/mozprofiler to js/src/make-source-package.sh, because
mozglue/moz.build now refers to it.
Differential Revision: https://phabricator.services.mozilla.com/D33258
--HG--
extra : moz-landing-system : lando
Start using BaseProfiler in Firefox main(), before&after XPCOM runs.
Also added a BaseProfiler label around Gecko Profiler init/shutdown (so that
samples may be ignored if user is only interested in non-XPCOM profiling).
Main process name changed to "Main Thread (Base Profiler)", so as not to confuse
the front-end, and show where this thread comes from.
Differential Revision: https://phabricator.services.mozilla.com/D31933
--HG--
extra : moz-landing-system : lando
If MOZ_BASE_PROFILER_STARTUP and MOZ_PROFILER_STARTUP are set, this will integrate
a pre-XPCOM startup profile into the main profile.
It is stored as separate threads (in a single JSON string that is moved around),
which will appear as a new track under the main process.
Only adding threads from BaseProfiler means a better integration with Gecko
Profiler profiles, and is more efficient: Less code, and a smaller memory
footprint.
Differential Revision: https://phabricator.services.mozilla.com/D31932
--HG--
extra : moz-landing-system : lando
Running identical (but separate) InitializeWin64ProfilerHooks in both profilers
confuses the DLL interceptor and the 2nd one crashes because of unexpected
opcodes introduced by the 1st one.
If MOZ_BASE_PROFILER is defined, Gecko Profiler will use that implementation of
InitializeWin64ProfilerHooks instead of its own; and that code also has a guard
so that it effectively only run once even if called from both profilers.
Differential Revision: https://phabricator.services.mozilla.com/D31931
--HG--
extra : moz-landing-system : lando
E.g., AUTO_PROFILER_INIT -> AUTO_BASE_PROFILER_INIT.
This will allow #including BaseProfiler.h anywhere as needed, without clashing
with Gecko Profiler macros.
Differential Revision: https://phabricator.services.mozilla.com/D31929
--HG--
extra : moz-landing-system : lando
Notice the extra 'BASE' in the env-var names.
This is to control BaseProfiler separately from the Gecko Profiler.
Differential Revision: https://phabricator.services.mozilla.com/D31928
--HG--
extra : moz-landing-system : lando
Android not implemented yet.
Windows not working yet when packaged, so disabled by default, but may be
enabled locally by uncommenting `#define MOZ_BASE_PROFILER` where indicated in
BaseProfiler.h.
Differential Revision: https://phabricator.services.mozilla.com/D31927
--HG--
extra : moz-landing-system : lando
Simple test program that exercises the most important APIs of BaseProfiler.
(Including checking that macros work even when BaseProfiler is not enabled.)
Differential Revision: https://phabricator.services.mozilla.com/D31926
--HG--
extra : moz-landing-system : lando
Almost-mechanical changes include:
- Removed unneeded/incompatible #includes and functions (any JS- or XPCOM-
dependent).
- Use std::string for strings and nsIDs.
- Use thin wrappers around mozilla::detail::MutexImpl for mutexes.
- Use hand-rolled AddRef&Release's for ref-counted classes -- could not use
mfbt/RefCounted.h because of bug 1536656.
- Added some platform-specific polyfills, e.g.: MicrosecondsSince1970().
- Only record the main thread by default.
- Logging controlled by env-vars MOZ_BASE_PROFILER_{,DEBUG_,VERBOSE_}LOGGING.
This now builds (with --enable-base-profiler), but is not usable yet.
Differential Revision: https://phabricator.services.mozilla.com/D31924
--HG--
extra : moz-landing-system : lando
Added baseprofiler to mozglue/moz.build, so it will be built.
However all cpp files are dependent on `MOZ_BASE_PROFILER`, which is currently
not #defined by default (in public/BaseProfiler.h).
Differential Revision: https://phabricator.services.mozilla.com/D33258
--HG--
extra : moz-landing-system : lando
The current situation is suboptimal, where we have the same goop
repeated in multiple files, and where things kinda sorta work out fine
thanks to the linker for files that would have been forbidden, except
when the linker doesn't do its job, which apparently happen on
mingwclang builds.
This change only really covers C++ code using operator new/delete, and
not things that would be using malloc/free, because it's easier.
malloc/free is left for a followup.
Differential Revision: https://phabricator.services.mozilla.com/D32119
--HG--
extra : moz-landing-system : lando
* CreateFileW will return INVALID_HANDLE_VALUE (-1) on failure, not NULL (0).
* MapViewOfFile will map the entire section if the size is 0. No explicit size
is required.
* If SEC_IMAGE is specified, the mapped image size may be different from the
file size on the storage.
Differential Revision: https://phabricator.services.mozilla.com/D32563
--HG--
extra : moz-landing-system : lando
Some parts of mozglue used to be STL wrapped because mozalloc used to be
a separate library, but that was changed a while ago (in bug 868814, 4
years ago), and those wrappings are not necessary anymore.
Differential Revision: https://phabricator.services.mozilla.com/D32430
--HG--
extra : moz-landing-system : lando
We're moving to IR-level PGO instrumentation for clang-cl. We've also
moved to using static linker ordering files, which was the primary
application of the previous style of PGO instrumentation. We therefore
we no longer need this code.
Differential Revision: https://phabricator.services.mozilla.com/D31134
--HG--
extra : moz-landing-system : lando
In part 1, we disabled the unhooking of DLL-intercepted functions at shutdown. The TestDllInterceptor relied on unhooking -- it worked by hooking functions with a "nonsense function" (nullptr) and then immediately unhooking it. That restored the original function behavior. Some hooked functions (e.g. NtWriteFile) are used by functions later in the program (e.g. printf) so the functions need to maintain their behavior.
This patch replaces the nonsense function with an identity function that also sets a global boolean as a side-effect. The function is written in machine code. x86-32, x86-64, and aarch64 variants are included.
Differential Revision: https://phabricator.services.mozilla.com/D30244
--HG--
extra : moz-landing-system : lando
QueryCredentialsAttributesA and FreeCredentialsHandle trigger an exception when null is passed for the CredHandle pointer. This exception was ignored (when not run in the debugger) but that is no longer the case with the changes in part 3. This patch passes a real CredHandle to them.
Differential Revision: https://phabricator.services.mozilla.com/D30243
--HG--
extra : moz-landing-system : lando
This patch fixes a static destructor order dependency between WindowsDllInterceptor and VMSharingPolicyUnique by telling VMSharingPolicyShared not to access the VMSharingPolicyUnique at destruction. This means that the behavior of intercepted functions is no longer restored in the given process at policy shutdown time.
Differential Revision: https://phabricator.services.mozilla.com/D28764
--HG--
extra : moz-landing-system : lando
On Windows 7, WinVerifyTrust fails unless the tag is uppercased. This patch
also adds a missing call to CryptCATAdminReleaseCatalogContext, the need for
which was poorly documented on MSDN.
Differential Revision: https://phabricator.services.mozilla.com/D30146
--HG--
extra : moz-landing-system : lando
TrampolineCollection iterates over an array of Trampolines that it has set 'write' permissions for. If this happens in a process whose sandbox forbids dynamic code then these permissions cannot be set. This patch detects that condition and returns an empty TrampolineCollection in that case. We ASSERT if we fail to set permissions for any other reason.
Differential Revision: https://phabricator.services.mozilla.com/D28613
--HG--
extra : moz-landing-system : lando
Bug 1533808 introduced code to intercept DLL methods that the Chromium sandbox had already intercepted. That patch did not store the the pointer to the intercepted function in the trampoline data, as is done when intercepting other methods.
Differential Revision: https://phabricator.services.mozilla.com/D28612
--HG--
extra : moz-landing-system : lando
We're going to convert the test to a gtest, and it's simpler not to have
to deal with finding the path to the testcase zip files. They're small
enough anyways, and can be inserted as raw binary data via some assembly
magic. This being android-only code, we don't need extreme portability
here. This is the same trick we use in
config/external/icu/data/icudata_gas.S.
Differential Revision: https://phabricator.services.mozilla.com/D28758
--HG--
extra : moz-landing-system : lando
This patch fixes a static destructor order dependency between WindowsDllInterceptor and VMSharingPolicyUnique by telling VMSharingPolicyShared not to access the VMSharingPolicyUnique at destruction. See the bug for details of the order dependency.
Differential Revision: https://phabricator.services.mozilla.com/D28764
--HG--
extra : moz-landing-system : lando
See comment 24 in the bug for details on what can go wrong without this
change. This change ensures system libraries are not going to pick
symbols from mozglue when running processes outside dalvik.
As a side effect, this makes things kind of closer to what happens when
dalvik is involved, exposing unit tests to possible allocator mismatches
that could happen like bug 1531887.
On the flip side, libraries that link against mozglue explicitly are
going to get a reference to the versioned symbols, so everything is fine
in that regard. The custom linker, however, will ignore the versions
altogether, and its symbols resolution just ends up unchanged. So we're
fine there too.
We use something that is close to what using a SYMBOLS_FILE would
generate as a version script, but we need to do so manually because
SYMBOLS_FILE doesn't support exporting all the symbols.
Differential Revision: https://phabricator.services.mozilla.com/D28030
--HG--
extra : moz-landing-system : lando
Bug 884239 added a build-time Android version check around the
pthread_atfork function definition at the same time as for timer_create,
which was subsequently removed. But it turns out the version that
documented was wrong: per the comment added in bug 680190,
pthread_atfork might have been supported since Android 2.3 (gingerbread,
API 9 or 10). That might not be entirely accurate, though, because the
bionic repository seems to show it made it to Android 4.0 (ice cream
sandwich, API 14 or 15).
Either way, that is less than the minimum API version we currently
support, which is 16.
Differential Revision: https://phabricator.services.mozilla.com/D27848
--HG--
extra : moz-landing-system : lando
There shouldn't be any need to do this for content processes as
the DLL should already be in the system file cache.
Differential Revision: https://phabricator.services.mozilla.com/D26017
--HG--
extra : moz-landing-system : lando
There shouldn't be any need to do this for content processes as
the DLL should already be in the system file cache.
Differential Revision: https://phabricator.services.mozilla.com/D26017
--HG--
extra : moz-landing-system : lando
This way we don't have to go through a bunch of printf nonsense, and we
ought to be able to arrive at optimized routines that take advantage of
constant radices, etc.
Differential Revision: https://phabricator.services.mozilla.com/D25141
--HG--
extra : moz-landing-system : lando
Due to coming changes involving the IOInterposer, the WindowsDllInterceptor may be set up later than the sandbox. The sandbox hooks some of the same functions, so the Interceptor is running into its hooks instead of the original implementations it anticipated. This patch allows it to recognize and efficiently patch those hooks when that happens.
Differential Revision: https://phabricator.services.mozilla.com/D24654
--HG--
extra : moz-landing-system : lando
When Android shuts down the ndk process, it doesn't call the registered
atexit() handlers, which is normally where the profile data gets written
to file. Since the PGO test suite closes the browser when it is
finished, the nativeRun routine can manually call out to
__llvm_profile_dump() before returning.
This method has a downside that only the profile data from the calling
library gets written out, rather than for the whole process. Since we
are most interested in optimizing libxul, a new hook is added in
Bootstrap to make sure we get the profile data for the right library.
Differential Revision: https://phabricator.services.mozilla.com/D22817
--HG--
extra : source : 0615c775a0cf6e8f98e1c051cd574c0d602a738a
When Android shuts down the ndk process, it doesn't call the registered
atexit() handlers, which is normally where the profile data gets written
to file. Since the PGO test suite closes the browser when it is
finished, the nativeRun routine can manually call out to
__llvm_profile_dump() before returning.
This method has a downside that only the profile data from the calling
library gets written out, rather than for the whole process. Since we
are most interested in optimizing libxul, a new hook is added in
Bootstrap to make sure we get the profile data for the right library.
Differential Revision: https://phabricator.services.mozilla.com/D22817
--HG--
extra : moz-landing-system : lando
In the tree we have two copies of printf_stderr() with the comment,
"Ideally this should be shared". This moves the function to a new exported
header which can be the basis for other similar debugging utility functions.
To include it,
#include "mozilla/glue/Debug.h"
A small concern with this is that printf_stderr() is in the global namespace,
and could conflict if it's inadvertently included along with, for example,
nsDebug.h which also defines this function. The warning in the comment at the
top of the file attempts to mitigate this.
Differential Revision: https://phabricator.services.mozilla.com/D13196
--HG--
extra : moz-landing-system : lando
BasicDllServices is used to gain access to the authenticode APIs in non-Gecko
contexts. One feature that WinDllServices provides is the ability to register
a callback interface to be notified when a DLL has been loaded.
This is not particularly useful in the BasicDllServices use case, and in the
"handle a launcher process failure on a background thread" use case, would
actually be harmful.
This patch modifies the DLLServices backend to offer a "basic" option that
omits the callback stuff.
Differential Revision: https://phabricator.services.mozilla.com/D19696
--HG--
extra : moz-landing-system : lando
This patch doesn't cover all possible functions for which we currently
instantiate interceptors inside Firefox/Gecko. Rather than asserting, we just
fail in those cases (at least until we have full coverage of existing uses).
This is okay, as for the upcoming milestone 2 of aarch64 builds, we are most
concerned with successfully being able to hook the following functions:
ntdll!LdrLoadDll
ntdll!LdrUnloadDll
ntdll!LdrResolveDelayLoadedAPI
user32!GetWindowInfo
So, within that context, the aarch64 implementation is fairly simple:
Each instruction is 4-bytes wide. We iterate down each instruction, and if the
current instruction is *not* PC-relative, we just copy it verbatim. If we
encounter an instruction that *is* PC-relative, we either decode it and
rewrite it inside the trampoline, or we fail. For the purposes of milestone 2,
the only instruction that is essential to decode is ADRP.
In bug 1526016 I modify TestDllInterceptor to exclude functions that are not
yet supported by this patch.
Differential Revision: https://phabricator.services.mozilla.com/D19446
--HG--
extra : moz-landing-system : lando
FramePointerStackWalk can trip ASAN when walking the stack and reading outside
of ASAN-protected objects.
Differential Revision: https://phabricator.services.mozilla.com/D20208
--HG--
extra : moz-landing-system : lando
The actual subcategories will be added in later patches, so that there are no
unused categories.
Differential Revision: https://phabricator.services.mozilla.com/D11334
--HG--
extra : moz-landing-system : lando
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
This patch takes care of a bunch of issues and does some cleanup:
* We rename mscom::MainThreadRuntime to mscom::ProcessRuntime, as the latter
is a more accurate name going forward.
* We make ProcessRuntime aware of the Win32k Lockdown process mitigation
policy. When Win32k is disabled, we perform process-wide COM initialization
in the multi-threaded apartment (since we cannot create an STA window).
* We refactor the mscom apartment region stuff to enable the Win32k lockdown
pieces in ProcessRuntime.
* We move some Gecko-specific stuff into MOZILLA_INTERNAL_API guards so that
ProcessRuntime is usable outside of xul.dll (I will be needing it for the
launcher process).
* Another thing that might happen with the launcher process is that, under
error conditions in the launcher, we create a ProcessRuntime object on a
background thread for the purposes of telemetry logging, but we also allow
the main thread to proceed to start as the browser. This could result in a
scenario where the main thread, as the browser process, is attempting to
instantiate its ProcessRuntime and ends up racing with the launcher process's
telemetry thread which has its own ProcessRuntime. To account for this
situation, we add mutual exclusion to the process-wide initialization code.
We host this part inside mozglue since that state is shared between both
firefox.exe and xul.dll.
* We clean up ProcessRuntime::InitializeSecurity by using Vector to set up
the EXPLICIT_ACCESS entries.
* We remove mscom::MainThreadClientInfo and replace it with a direct call to
CoGetCallerTID
* We revise all references to this class to use the new name.
Differential Revision: https://phabricator.services.mozilla.com/D19551
--HG--
rename : ipc/mscom/COMApartmentRegion.h => ipc/mscom/ApartmentRegion.h
rename : ipc/mscom/MainThreadRuntime.cpp => ipc/mscom/ProcessRuntime.cpp
rename : ipc/mscom/MainThreadRuntime.h => ipc/mscom/ProcessRuntime.h
extra : moz-landing-system : lando
This patch takes care of a bunch of issues and does some cleanup:
* We rename mscom::MainThreadRuntime to mscom::ProcessRuntime, as the latter
is a more accurate name going forward.
* We make ProcessRuntime aware of the Win32k Lockdown process mitigation
policy. When Win32k is disabled, we perform process-wide COM initialization
in the multi-threaded apartment (since we cannot create an STA window).
* We refactor the mscom apartment region stuff to enable the Win32k lockdown
pieces in ProcessRuntime.
* We move some Gecko-specific stuff into MOZILLA_INTERNAL_API guards so that
ProcessRuntime is usable outside of xul.dll (I will be needing it for the
launcher process).
* Another thing that might happen with the launcher process is that, under
error conditions in the launcher, we create a ProcessRuntime object on a
background thread for the purposes of telemetry logging, but we also allow
the main thread to proceed to start as the browser. This could result in a
scenario where the main thread, as the browser process, is attempting to
instantiate its ProcessRuntime and ends up racing with the launcher process's
telemetry thread which has its own ProcessRuntime. To account for this
situation, we add mutual exclusion to the process-wide initialization code.
We host this part inside mozglue since that state is shared between both
firefox.exe and xul.dll.
* We clean up ProcessRuntime::InitializeSecurity by using Vector to set up
the EXPLICIT_ACCESS entries.
* We remove mscom::MainThreadClientInfo and replace it with a direct call to
CoGetCallerTID
* We revise all references to this class to use the new name.
Differential Revision: https://phabricator.services.mozilla.com/D19551
--HG--
rename : ipc/mscom/COMApartmentRegion.h => ipc/mscom/ApartmentRegion.h
rename : ipc/mscom/MainThreadRuntime.cpp => ipc/mscom/ProcessRuntime.cpp
rename : ipc/mscom/MainThreadRuntime.h => ipc/mscom/ProcessRuntime.h
extra : moz-landing-system : lando
This patch measures the duration of module loads and passes it up to
UntrustedModulesManager where, in later patches, it will be consumed by
telemetry.
Differential Revision: https://phabricator.services.mozilla.com/D16011
--HG--
extra : moz-landing-system : lando
WindowsDllBlocklist installs a callback function that fires whenever a DLL
is loaded. The installer function shares an SRWLock with the callback
function.
SRWLock is not re-entrant, so if the installer function accidently causes a
DLL load before releasing the lock, the callback function will deadlock.
This occured trying to solve Bug 1402282, where the installer function used
"new" to allocate memory, which called the Win32 "RtlGenRandom()" function,
which loaded bcrypt.dll, which caused the callback to fire off, which tried
to lock the mutex that was already locked by the installer function.
Hopefully this will save another developer lots of debug time in the future by
turning a difficult-to-debug deadlock into a nice, loud assertion.
Differential Revision: https://phabricator.services.mozilla.com/D15840
--HG--
extra : moz-landing-system : lando
It's a define that needs winuser.rh to be included.
MozReview-Commit-ID: LPfJOwnNm6V
Differential Revision: https://phabricator.services.mozilla.com/D15333
--HG--
extra : moz-landing-system : lando
mozilla::PrintfTarget::cvt_f release asserts that the desired printf
fit into a statically-sized buffer. However, this may not be the case
if the user requested a larger width or precision. Handle this
unusual case by allocating a temporary buffer.
MozReview-Commit-ID: 2WicecHDzDR
Differential Revision: https://phabricator.services.mozilla.com/D15989
--HG--
extra : moz-landing-system : lando
Chromium IPC locks have this and the API gets used, so we need to expose
it ourselves if we're going to use our locks in place of the Chromium
IPC locks. This patch changes the mozglue parts; tweaking the xpcom
parts is the next patch.
In Windows 7 x64, GetFileAttributesW begins with a short, backwards jump that can't safely be converted by the interceptor. Additionally, the function doesn't have enough NOP space after the JMP for the trampoline. However, the target of the short JMP is a long JMP, followed by plenty of NOP space. This patch moves the trampoline location from the first JMP to the second.
Differential Revision: https://phabricator.services.mozilla.com/D11258
--HG--
extra : moz-landing-system : lando
Nightly is reporting mozglue.dll as being untrusted in the untrusted modules
ping. Until now, xpcshell tests hard-code mozglue to appear as untrusted in
order to cover certain code paths related to startup modules.
This patch:
1. Checks explicitly for the xpcshell environment and only applies this logic
during xpcshell tests.
2. Uses a purpose-build DLL, "untrusted-startup-test-dll.dll", instead of
mozglue. This is more explicit and doesn't interfere with any "true"
processing of mozglue.
Differential Revision: https://phabricator.services.mozilla.com/D14720
--HG--
extra : moz-landing-system : lando
As far as my stepping through WalkStackMain64 goes, it seems StackWalk64
doesn't work, even with more information added to the frame data it's
given.
Switching to the same code as for x86-64, however, works, albeit
skipping too many frames, but all platforms are actually skipping too
many frames, so let's ignore that for now and leave it to bug 1515229.
Differential Revision: https://phabricator.services.mozilla.com/D14929
--HG--
extra : moz-landing-system : lando
- modify line wrap up to 80 chars; (tw=80)
- modify size of tab to 2 chars everywhere; (sts=2, sw=2)
--HG--
extra : rebase_source : 7eedce0311b340c9a5a1265dc42d3121cc0f32a0
extra : amend_source : 9cb4ffdd5005f5c4c14172390dd00b04b2066cd7
This is a best effort attempt at ensuring that the adverse impact of
reformatting the entire tree over the comments would be minimal. I've used a
combination of strategies including disabling of formatting, some manual
formatting and some changes to formatting to work around some clang-format
limitations.
Differential Revision: https://phabricator.services.mozilla.com/D13371
--HG--
extra : moz-landing-system : lando
This also uses idiomatic packaging of the native libraries, which
will allow easier downstream consumption of GeckoView.
Differential Revision: https://phabricator.services.mozilla.com/D10775
--HG--
extra : moz-landing-system : lando
Added a test to TestCrossProcessInterceptor that forcibly uses a 10-byte patch
on NtMapViewOfSection (which is a realistic case) and then ensures that
disabling the hook also works.
Differential Revision: https://phabricator.services.mozilla.com/D10286
--HG--
extra : moz-landing-system : lando
This patch adds support on x64 for 10-byte detour patches in certain cases.
In particular, the reserved region of trampoline memory must be allocated
within the bottommost 2GB of the address space.
This feature is currently only activated when detouring functions exported by
ntdll.dll.
Differential Revision: https://phabricator.services.mozilla.com/D10285
--HG--
extra : moz-landing-system : lando
This also uses idiomatic packaging of the native libraries, which
will allow easier downstream consumption of GeckoView.
Differential Revision: https://phabricator.services.mozilla.com/D10775
--HG--
extra : moz-landing-system : lando
This also uses idiomatic packaging of the native libraries, which
will allow easier downstream consumption of GeckoView.
Differential Revision: https://phabricator.services.mozilla.com/D10775
--HG--
extra : moz-landing-system : lando
Added a test to TestCrossProcessInterceptor that forcibly uses a 10-byte patch
on NtMapViewOfSection (which is a realistic case) and then ensures that
disabling the hook also works.
Differential Revision: https://phabricator.services.mozilla.com/D10286
--HG--
extra : moz-landing-system : lando
This patch adds support on x64 for 10-byte detour patches in certain cases.
In particular, the reserved region of trampoline memory must be allocated
within the bottommost 2GB of the address space.
This feature is currently only activated when detouring functions exported by
ntdll.dll.
Differential Revision: https://phabricator.services.mozilla.com/D10285
--HG--
extra : moz-landing-system : lando
Added a test to TestCrossProcessInterceptor that forcibly uses a 10-byte patch
on NtMapViewOfSection (which is a realistic case) and then ensures that
disabling the hook also works.
Differential Revision: https://phabricator.services.mozilla.com/D10286
--HG--
extra : moz-landing-system : lando
This patch adds support on x64 for 10-byte detour patches in certain cases.
In particular, the reserved region of trampoline memory must be allocated
within the bottommost 2GB of the address space.
This feature is currently only activated when detouring functions exported by
ntdll.dll.
Differential Revision: https://phabricator.services.mozilla.com/D10285
--HG--
extra : moz-landing-system : lando
They were not displayed in the UI, and the instructions to initialize the line
field of a stack frame increased code size unnecessarily.
This change reduces the binary size on Linux x64 by around 100KB.
Here's a diff of the impact on the code generated for Attr_Binding::get_specified
in the Mac build:
@@ -20,17 +20,16 @@
movq 0x8(%rbx), %rax
movq %r12, %rcx
shlq $0x5, %rcx
leaq aGetAttrspecifi, %rdx ; "get Attr.specified"
movq %rdx, (%rax,%rcx)
movq $0x0, 0x8(%rax,%rcx)
leaq -40(%rbp), %rdx
movq %rdx, 0x10(%rax,%rcx)
- movl $0x106, 0x18(%rax,%rcx)
movl $0x1c, 0x1c(%rax,%rcx)
leal 0x1(%r12), %eax
movl %eax, 0x10(%rbx)
movq %r15, %rdi
call __ZNK7mozilla3dom4Attr9SpecifiedEv ; mozilla::dom::Attr::Specified() const
movzxl %al, %eax
movabsq $0xfff9000000000000, %rcx
Depends on D9193
Differential Revision: https://phabricator.services.mozilla.com/D9195
--HG--
extra : moz-landing-system : lando
They were not displayed in the UI, and the instructions to initialize the line
field of a stack frame increased code size unnecessarily.
This change reduces the binary size on Linux x64 by around 100KB.
Here's a diff of the impact on the code generated for Attr_Binding::get_specified
in the Mac build:
@@ -20,17 +20,16 @@
movq 0x8(%rbx), %rax
movq %r12, %rcx
shlq $0x5, %rcx
leaq aGetAttrspecifi, %rdx ; "get Attr.specified"
movq %rdx, (%rax,%rcx)
movq $0x0, 0x8(%rax,%rcx)
leaq -40(%rbp), %rdx
movq %rdx, 0x10(%rax,%rcx)
- movl $0x106, 0x18(%rax,%rcx)
movl $0x1c, 0x1c(%rax,%rcx)
leal 0x1(%r12), %eax
movl %eax, 0x10(%rbx)
movq %r15, %rdi
call __ZNK7mozilla3dom4Attr9SpecifiedEv ; mozilla::dom::Attr::Specified() const
movzxl %al, %eax
movabsq $0xfff9000000000000, %rcx
Depends on D9193
Differential Revision: https://phabricator.services.mozilla.com/D9195
--HG--
extra : moz-landing-system : lando
They were not displayed in the UI, and the instructions to initialize the line
field of a stack frame increased code size unnecessarily.
This change reduces the binary size on Linux x64 by around 100KB.
Here's a diff of the impact on the code generated for Attr_Binding::get_specified
in the Mac build:
@@ -20,17 +20,16 @@
movq 0x8(%rbx), %rax
movq %r12, %rcx
shlq $0x5, %rcx
leaq aGetAttrspecifi, %rdx ; "get Attr.specified"
movq %rdx, (%rax,%rcx)
movq $0x0, 0x8(%rax,%rcx)
leaq -40(%rbp), %rdx
movq %rdx, 0x10(%rax,%rcx)
- movl $0x106, 0x18(%rax,%rcx)
movl $0x1c, 0x1c(%rax,%rcx)
leal 0x1(%r12), %eax
movl %eax, 0x10(%rbx)
movq %r15, %rdi
call __ZNK7mozilla3dom4Attr9SpecifiedEv ; mozilla::dom::Attr::Specified() const
movzxl %al, %eax
movabsq $0xfff9000000000000, %rcx
Depends on D9193
Differential Revision: https://phabricator.services.mozilla.com/D9195
--HG--
extra : moz-landing-system : lando
We now record DLL load events along with stack trace and other data so we can
later determine trustworthiness and report the DLL via telemetry.
Differential Revision: https://phabricator.services.mozilla.com/D7175
--HG--
extra : moz-landing-system : lando
This patch adds a new static member to the TimeStamp class to store the
current locked *time* (in ms since the epoch) in addition to the current locked
timestamp.
We point the JS Engine at this value if Fuzzyfox is enabled.
Creates GetFuzzyfoxEnabled() functions that check a static boolean.
Exposes SetFuzzyfoxEnabled() because we cannot depend on Pref
Observation code inside the TimeStamp class.
TimeStamp::Now will now return a Fuzzy value.
We add a NowReally function to support obtaining the real timestamp.
We also add a UsedCanonicalNow to expose whether the TimeStamp was real or fuzzy.
Creates a FuzzyFox class for implementating the core of the step/sleep
algorithm. Starts it in nsLayoutStatics::Initialize()
Adds the fuzzyfox prefs.
Moves the ms2mt macros from TimeStamp_windows.cpp to TimeStamp_windows.h
and creates a new public function GetQueryPerformanceFrequencyPerSec() to
expose a static variable in the .cpp file. This is necessary to support
the macros being usable anywhere. (And we use the macros in FuzzyFox.)
SxS assemblies do not obey the usual DLL search order. It will make it possible
to load mozglue.dll from appdir even if the PreferSystem32Images mitigation is
enabled and System32 has a random mozglue.dll.
All but one of the current uses of DEFFILE use `SRCDIR + '/file.def'` to
get a srcdir-relative path anyway, and the other one wants an
objdir-relative path, so using Path makes everything clearer.
This makes it more straightforward to translate the paths for the WSL
build.
The linker has x86-64 support but currently fails to compile. This patch
fixes these compile errors to make it build under x86-64.
Differential Revision: https://phabricator.services.mozilla.com/D4481
This code throws an error in clang on the inner MMPolicy:
error: declaration of 'MMPolicy' shadows template parameter
Notethat the template parameter is declared earlier at the
class definition of ReadOnlyTargetFunction
MozReview-Commit-ID: buLE9d22YS
Differential Revision: https://phabricator.services.mozilla.com/D4571
--HG--
extra : moz-landing-system : lando
This code is untested and has been cargo-culted a little bit from the
existing x86 code, but should work OK; all the code in Windows is
compiled with frame pointers, we're compiled with frame pointers after
the previous patch, and so the frame pointer unwinding path makes the
most sense.
This introduces the machinery needed to generate crash annotations from a YAML
file. The relevant C++ functions are updated to take a typed enum. JavaScript
calls are unaffected but they will throw if the string argument does not
correspond to one of the known entries in the C++ enum. The existing whitelists
and blacklists of annotations are also generated from the YAML file and all
duplicate code related to them has been consolidated. Once written out to the
.extra file the annotations are converted in string form and are no different
than the existing ones.
All existing annotations have been included in the list (and some obsolete ones
have been removed) and all call sites have been updated including tests where
appropriate.
--HG--
extra : source : 4f6c43f2830701ec5552e08e3f1b06fe6d045860
__wrap_dlerror uses a single pointer for all threads, which means one
thread could get the dlerror result from another thread. Normally this
wouldn't cause crashes. However, because dlerror results come from a
per-thread buffer, if a thread exits and our saved dlerror result came
from that thread, the saved pointer could then refer to invalid memory.
The proper way to fix this is to use TLS and have a per-thread pointer
for __wrap_dlerror. However, instead of using up a TLS slot, this patch
keeps the single pointer for custom messages, and fallback to per-thread
dlerror call for system messages. While the race condition still exists,
I think the risk is acceptable. Even when races occur, they should no
longer cause crashes.
MozReview-Commit-ID: 4hGksidjiVz
--HG--
extra : rebase_source : 373000686c426b81ffd7cee88264e89b7a733957
There is no meaningful equality relationship on any plausible mutex
implementation other than object identity. Having MutexImpl's users simply
compare by addresses makes it clearer in the callers that that's what's going
on.
Adding or removing an FD from this API currently requires changes in about a
half dozen places. Ignoring the Java side of things. This patch changes the
API to pass a struct, rather than additional arguments for each FD, so that
adding and removing FDs only requires changing one declaration, and the two
call sites that add and consume the FDs.
MozReview-Commit-ID: CToSEVp1oqP
--HG--
extra : intermediate-source : ff41551f5ff1b98b72ed771a6f2a3f66a8b79a57
extra : absorb_source : c9fe7423fcbb47655b05209b44fb02b69b272d07
extra : source : 4b7a8a35ed956159e2f443c6211164c0cbf3d926
extra : histedit_source : b98b792791274f00a5e649c82dc25043cc1d699a
Adding or removing an FD from this API currently requires changes in about a
half dozen places. Ignoring the Java side of things. This patch changes the
API to pass a struct, rather than additional arguments for each FD, so that
adding and removing FDs only requires changing one declaration, and the two
call sites that add and consume the FDs.
MozReview-Commit-ID: CToSEVp1oqP
--HG--
extra : source : 4b7a8a35ed956159e2f443c6211164c0cbf3d926
extra : histedit_source : 01a1160ce1107d12e8b376d4512dedb0478e447c
Adding or removing an FD from this API currently requires changes in about a
half dozen places. Ignoring the Java side of things. This patch changes the
API to pass a struct, rather than additional arguments for each FD, so that
adding and removing FDs only requires changing one declaration, and the two
call sites that add and consume the FDs.
MozReview-Commit-ID: CToSEVp1oqP
--HG--
extra : rebase_source : 28e8c6075bacf5f610058227a9731aeadb50f320
extra : absorb_source : f63602a163ed19fb65e26640319750fdd9b92ad1
For clang-cl, we want to add code to libxul that only exists during the
PGO generation phase, so we can collect data. The most expedient way to
do that is to enable certain files in SOURCES to be marked as to only be
compiled during the PGO generation step.
In addition to updating the interface, this patch also significantly alters the
structure of this test. In particular, it removes the Test* functions in favour
of using template magic.
I did this because I noticed that, in the majority of cases, the stub function
was being called with all zero arguments, and then we check for the expected
error code. I thought that maybe we could replace that repetition with some
templates that instantiate a blank tuple that may then be applied to a callable
object.
See the (MAYBE_)TEST_HOOK* and TEST_DETOUR* macro definitions for detailed
information about how to use these things.
The test successfully completes with both 32-bit and 64-bit builds.
This patch makes the interceptor's AddHook functions private, and converts
the stubs from simple function pointers into objects containing both the stub
function pointer, plus a INIT_ONCE sentinel.
Setting a hook now requires calling Set or SetDetour on the stub, which ensures
that the hook attempt happens once and only once.
The constructor for the new object is constexpr, so it should not generate
static initializers if it is declared statically.
Note that, as a corollary of the new behaviour, we no longer need to set guards
around any hook setting code. I have removed those when present.
Gerald Squelart
Andreea Pavel
Ciure Andrei
Greg Tatum
Jory A. Pratt
Mihai Alexandru Michis
Aaron Klotz
Bogdan Tara
Denis Palmeiro
Noemi Erli
Mike Hommey
Ehsan Akhgari
Masatoshi Kimura
Sylvestre Ledru
Nathan Froyd
arthur.iakab
Emilio Cobos Álvarez
David Parks
Kartikaya Gupta
Dorel Luca
Geoff Brown
Doug Thayer
Cosmin Sabou
Csoregi Natalia
Narcis Beleuzu
Cameron McCormack
John Lin
Karl Tomlinson
Mike Shal
Carl Corcoran
Ryan Hunt
Markus Stange
Eric Rahm
Tom Ritter
David Major
Gabriele Svelto
Chris Martin
Coroiu Cristina
Jacek Caban
shindli
Tom Tromey
Tooru Fujisawa
Benjamin Bouvier
Honza Bambas
James Willcox
Brindusan Cristian
Jeff Gilbert
Ted Mielczarek
philipp
Thomas P.
Jim Chen
Changqing Li
Christian Holler
Brian Hackett
Jim Blandy
Kris Maglione
Chris Manchester
Nathan Froyd