Update the tests for ARM64 to include additional functions that are now
supported via 4 byte patching.
We also convert the TEST macros to accept the DLL names as strings, as this
works better with clang-format.
Differential Revision: https://phabricator.services.mozilla.com/D32209
--HG--
extra : moz-landing-system : lando
This patch modifies arm64 so that detours are peformed via two passes:
1. The first pass uses a null trampoline to count how many bytes are available
for patching the original function.
2. If we have >= 16 bytes to patch, we reuse existing trampoline space. If we
have less than 16 bytes to patch, we reserve trampoline space within 128MB
of the function, allowing for a 4 byte patch.
3. Then we recurse, this time using a real trampoline.
Note that we still do a single-pass on x86(-64).
Differential Revision: https://phabricator.services.mozilla.com/D32193
--HG--
extra : moz-landing-system : lando
A null trampoline is just a trampoline that is not backed by a VM reservation.
These are used for tracking the number of bytes that are needed to make a patch.
This patch also contains the changes needed to work with TrampolinePool.
Differential Revision: https://phabricator.services.mozilla.com/D32192
--HG--
extra : moz-landing-system : lando
VMSharingPolicyShared needs to become much smarter. This patch modifies that
policy to track different VM reservations and reuse them whenever possible.
We add TrampolinePools to abstract away the differences between VM policies
with respect to the caller who is making the reservation.
Differential Revision: https://phabricator.services.mozilla.com/D32191
--HG--
extra : moz-landing-system : lando
In order to support 4-byte patches on ARM64, we need to be able to reserve
trampoline space within +/- 128 MB of the beginning of a function.
These changes allow us to make such reservations using OS APIs when
available.
Differential Revision: https://phabricator.services.mozilla.com/D32190
--HG--
extra : moz-landing-system : lando
VMSharingPolicyShared needs to become much smarter. This patch modifies that
policy to track different VM reservations and reuse them whenever possible.
We add TrampolinePools to abstract away the differences between VM policies
with respect to the caller who is making the reservation.
Differential Revision: https://phabricator.services.mozilla.com/D32191
--HG--
extra : moz-landing-system : lando
In order to support 4-byte patches on ARM64, we need to be able to reserve
trampoline space within +/- 128 MB of the beginning of a function.
These changes allow us to make such reservations using OS APIs when
available.
Differential Revision: https://phabricator.services.mozilla.com/D32190
--HG--
extra : moz-landing-system : lando
We remove the debugging hooks that were added to check to see whether a DLL
was loaded, as we can just as easily check that by querying the loader itself.
Plus, we shouldn't be exporting a bunch of test-only loader hooks from mozglue
in our release builds, which is what we are currently doing.
We also remove Injector, InjectorDLL, and TestDLLEject, as these tests can
just as easily be done from within TestDllBlocklist by creating a thread with
LoadLibrary* as the entry point. The CreateRemoteThread stuff, while a more
accurate simulation, has no material effect on whether or not the thread
blocking code works.
Differential Revision: https://phabricator.services.mozilla.com/D34444
--HG--
extra : moz-landing-system : lando
We also s/mincore/version/ in OS_LIBS because the former breaks the test on
Windows 7.
Differential Revision: https://phabricator.services.mozilla.com/D34437
--HG--
extra : moz-landing-system : lando
The profiler will require non-fuzzed timers for accuracy. Making the switch early will avoid surprises when FuzzyFox is enabled.
Differential Revision: https://phabricator.services.mozilla.com/D31010
--HG--
extra : moz-landing-system : lando
Start using BaseProfiler in Firefox main(), before&after XPCOM runs.
Also added a BaseProfiler label around Gecko Profiler init/shutdown (so that
samples may be ignored if user is only interested in non-XPCOM profiling).
Main process name changed to "Main Thread (Base Profiler)", so as not to confuse
the front-end, and show where this thread comes from.
Differential Revision: https://phabricator.services.mozilla.com/D31933
--HG--
extra : moz-landing-system : lando
If MOZ_BASE_PROFILER_STARTUP and MOZ_PROFILER_STARTUP are set, this will integrate
a pre-XPCOM startup profile into the main profile.
It is stored as separate threads (in a single JSON string that is moved around),
which will appear as a new track under the main process.
Only adding threads from BaseProfiler means a better integration with Gecko
Profiler profiles, and is more efficient: Less code, and a smaller memory
footprint.
Differential Revision: https://phabricator.services.mozilla.com/D31932
--HG--
extra : moz-landing-system : lando
Running identical (but separate) InitializeWin64ProfilerHooks in both profilers
confuses the DLL interceptor and the 2nd one crashes because of unexpected
opcodes introduced by the 1st one.
If MOZ_BASE_PROFILER is defined, Gecko Profiler will use that implementation of
InitializeWin64ProfilerHooks instead of its own; and that code also has a guard
so that it effectively only run once even if called from both profilers.
Differential Revision: https://phabricator.services.mozilla.com/D31931
--HG--
extra : moz-landing-system : lando
E.g., AUTO_PROFILER_INIT -> AUTO_BASE_PROFILER_INIT.
This will allow #including BaseProfiler.h anywhere as needed, without clashing
with Gecko Profiler macros.
Differential Revision: https://phabricator.services.mozilla.com/D31929
--HG--
extra : moz-landing-system : lando
Notice the extra 'BASE' in the env-var names.
This is to control BaseProfiler separately from the Gecko Profiler.
Differential Revision: https://phabricator.services.mozilla.com/D31928
--HG--
extra : moz-landing-system : lando
Android not implemented yet.
Windows not working yet when packaged, so disabled by default, but may be
enabled locally by uncommenting `#define MOZ_BASE_PROFILER` where indicated in
BaseProfiler.h.
Differential Revision: https://phabricator.services.mozilla.com/D31927
--HG--
extra : moz-landing-system : lando
Simple test program that exercises the most important APIs of BaseProfiler.
(Including checking that macros work even when BaseProfiler is not enabled.)
Differential Revision: https://phabricator.services.mozilla.com/D31926
--HG--
extra : moz-landing-system : lando
Almost-mechanical changes include:
- Removed unneeded/incompatible #includes and functions (any JS- or XPCOM-
dependent).
- Use std::string for strings and nsIDs.
- Use thin wrappers around mozilla::detail::MutexImpl for mutexes.
- Use hand-rolled AddRef&Release's for ref-counted classes -- could not use
mfbt/RefCounted.h because of bug 1536656.
- Added some platform-specific polyfills, e.g.: MicrosecondsSince1970().
- Only record the main thread by default.
- Logging controlled by env-vars MOZ_BASE_PROFILER_{,DEBUG_,VERBOSE_}LOGGING.
This now builds (with --enable-base-profiler), but is not usable yet.
Differential Revision: https://phabricator.services.mozilla.com/D31924
--HG--
extra : moz-landing-system : lando
Added baseprofiler to mozglue/moz.build, so it will be built.
However all cpp files are dependent on `MOZ_BASE_PROFILER`, which is currently
not #defined by default (in public/BaseProfiler.h).
Added mozglue/mozprofiler to js/src/make-source-package.sh, because
mozglue/moz.build now refers to it.
Differential Revision: https://phabricator.services.mozilla.com/D33258
--HG--
extra : moz-landing-system : lando
Start using BaseProfiler in Firefox main(), before&after XPCOM runs.
Also added a BaseProfiler label around Gecko Profiler init/shutdown (so that
samples may be ignored if user is only interested in non-XPCOM profiling).
Main process name changed to "Main Thread (Base Profiler)", so as not to confuse
the front-end, and show where this thread comes from.
Differential Revision: https://phabricator.services.mozilla.com/D31933
--HG--
extra : moz-landing-system : lando
If MOZ_BASE_PROFILER_STARTUP and MOZ_PROFILER_STARTUP are set, this will integrate
a pre-XPCOM startup profile into the main profile.
It is stored as separate threads (in a single JSON string that is moved around),
which will appear as a new track under the main process.
Only adding threads from BaseProfiler means a better integration with Gecko
Profiler profiles, and is more efficient: Less code, and a smaller memory
footprint.
Differential Revision: https://phabricator.services.mozilla.com/D31932
--HG--
extra : moz-landing-system : lando
Running identical (but separate) InitializeWin64ProfilerHooks in both profilers
confuses the DLL interceptor and the 2nd one crashes because of unexpected
opcodes introduced by the 1st one.
If MOZ_BASE_PROFILER is defined, Gecko Profiler will use that implementation of
InitializeWin64ProfilerHooks instead of its own; and that code also has a guard
so that it effectively only run once even if called from both profilers.
Differential Revision: https://phabricator.services.mozilla.com/D31931
--HG--
extra : moz-landing-system : lando
E.g., AUTO_PROFILER_INIT -> AUTO_BASE_PROFILER_INIT.
This will allow #including BaseProfiler.h anywhere as needed, without clashing
with Gecko Profiler macros.
Differential Revision: https://phabricator.services.mozilla.com/D31929
--HG--
extra : moz-landing-system : lando
Notice the extra 'BASE' in the env-var names.
This is to control BaseProfiler separately from the Gecko Profiler.
Differential Revision: https://phabricator.services.mozilla.com/D31928
--HG--
extra : moz-landing-system : lando
Android not implemented yet.
Windows not working yet when packaged, so disabled by default, but may be
enabled locally by uncommenting `#define MOZ_BASE_PROFILER` where indicated in
BaseProfiler.h.
Differential Revision: https://phabricator.services.mozilla.com/D31927
--HG--
extra : moz-landing-system : lando
Simple test program that exercises the most important APIs of BaseProfiler.
(Including checking that macros work even when BaseProfiler is not enabled.)
Differential Revision: https://phabricator.services.mozilla.com/D31926
--HG--
extra : moz-landing-system : lando
Almost-mechanical changes include:
- Removed unneeded/incompatible #includes and functions (any JS- or XPCOM-
dependent).
- Use std::string for strings and nsIDs.
- Use thin wrappers around mozilla::detail::MutexImpl for mutexes.
- Use hand-rolled AddRef&Release's for ref-counted classes -- could not use
mfbt/RefCounted.h because of bug 1536656.
- Added some platform-specific polyfills, e.g.: MicrosecondsSince1970().
- Only record the main thread by default.
- Logging controlled by env-vars MOZ_BASE_PROFILER_{,DEBUG_,VERBOSE_}LOGGING.
This now builds (with --enable-base-profiler), but is not usable yet.
Differential Revision: https://phabricator.services.mozilla.com/D31924
--HG--
extra : moz-landing-system : lando
Added baseprofiler to mozglue/moz.build, so it will be built.
However all cpp files are dependent on `MOZ_BASE_PROFILER`, which is currently
not #defined by default (in public/BaseProfiler.h).
Added mozglue/mozprofiler to js/src/make-source-package.sh, because
mozglue/moz.build now refers to it.
Differential Revision: https://phabricator.services.mozilla.com/D33258
--HG--
extra : moz-landing-system : lando
Start using BaseProfiler in Firefox main(), before&after XPCOM runs.
Also added a BaseProfiler label around Gecko Profiler init/shutdown (so that
samples may be ignored if user is only interested in non-XPCOM profiling).
Main process name changed to "Main Thread (Base Profiler)", so as not to confuse
the front-end, and show where this thread comes from.
Differential Revision: https://phabricator.services.mozilla.com/D31933
--HG--
extra : moz-landing-system : lando
If MOZ_BASE_PROFILER_STARTUP and MOZ_PROFILER_STARTUP are set, this will integrate
a pre-XPCOM startup profile into the main profile.
It is stored as separate threads (in a single JSON string that is moved around),
which will appear as a new track under the main process.
Only adding threads from BaseProfiler means a better integration with Gecko
Profiler profiles, and is more efficient: Less code, and a smaller memory
footprint.
Differential Revision: https://phabricator.services.mozilla.com/D31932
--HG--
extra : moz-landing-system : lando
Running identical (but separate) InitializeWin64ProfilerHooks in both profilers
confuses the DLL interceptor and the 2nd one crashes because of unexpected
opcodes introduced by the 1st one.
If MOZ_BASE_PROFILER is defined, Gecko Profiler will use that implementation of
InitializeWin64ProfilerHooks instead of its own; and that code also has a guard
so that it effectively only run once even if called from both profilers.
Differential Revision: https://phabricator.services.mozilla.com/D31931
--HG--
extra : moz-landing-system : lando
E.g., AUTO_PROFILER_INIT -> AUTO_BASE_PROFILER_INIT.
This will allow #including BaseProfiler.h anywhere as needed, without clashing
with Gecko Profiler macros.
Differential Revision: https://phabricator.services.mozilla.com/D31929
--HG--
extra : moz-landing-system : lando
Notice the extra 'BASE' in the env-var names.
This is to control BaseProfiler separately from the Gecko Profiler.
Differential Revision: https://phabricator.services.mozilla.com/D31928
--HG--
extra : moz-landing-system : lando
Android not implemented yet.
Windows not working yet when packaged, so disabled by default, but may be
enabled locally by uncommenting `#define MOZ_BASE_PROFILER` where indicated in
BaseProfiler.h.
Differential Revision: https://phabricator.services.mozilla.com/D31927
--HG--
extra : moz-landing-system : lando
Simple test program that exercises the most important APIs of BaseProfiler.
(Including checking that macros work even when BaseProfiler is not enabled.)
Differential Revision: https://phabricator.services.mozilla.com/D31926
--HG--
extra : moz-landing-system : lando
Almost-mechanical changes include:
- Removed unneeded/incompatible #includes and functions (any JS- or XPCOM-
dependent).
- Use std::string for strings and nsIDs.
- Use thin wrappers around mozilla::detail::MutexImpl for mutexes.
- Use hand-rolled AddRef&Release's for ref-counted classes -- could not use
mfbt/RefCounted.h because of bug 1536656.
- Added some platform-specific polyfills, e.g.: MicrosecondsSince1970().
- Only record the main thread by default.
- Logging controlled by env-vars MOZ_BASE_PROFILER_{,DEBUG_,VERBOSE_}LOGGING.
This now builds (with --enable-base-profiler), but is not usable yet.
Differential Revision: https://phabricator.services.mozilla.com/D31924
--HG--
extra : moz-landing-system : lando
Added baseprofiler to mozglue/moz.build, so it will be built.
However all cpp files are dependent on `MOZ_BASE_PROFILER`, which is currently
not #defined by default (in public/BaseProfiler.h).
Differential Revision: https://phabricator.services.mozilla.com/D33258
--HG--
extra : moz-landing-system : lando
The current situation is suboptimal, where we have the same goop
repeated in multiple files, and where things kinda sorta work out fine
thanks to the linker for files that would have been forbidden, except
when the linker doesn't do its job, which apparently happen on
mingwclang builds.
This change only really covers C++ code using operator new/delete, and
not things that would be using malloc/free, because it's easier.
malloc/free is left for a followup.
Differential Revision: https://phabricator.services.mozilla.com/D32119
--HG--
extra : moz-landing-system : lando
* CreateFileW will return INVALID_HANDLE_VALUE (-1) on failure, not NULL (0).
* MapViewOfFile will map the entire section if the size is 0. No explicit size
is required.
* If SEC_IMAGE is specified, the mapped image size may be different from the
file size on the storage.
Differential Revision: https://phabricator.services.mozilla.com/D32563
--HG--
extra : moz-landing-system : lando
Some parts of mozglue used to be STL wrapped because mozalloc used to be
a separate library, but that was changed a while ago (in bug 868814, 4
years ago), and those wrappings are not necessary anymore.
Differential Revision: https://phabricator.services.mozilla.com/D32430
--HG--
extra : moz-landing-system : lando
We're moving to IR-level PGO instrumentation for clang-cl. We've also
moved to using static linker ordering files, which was the primary
application of the previous style of PGO instrumentation. We therefore
we no longer need this code.
Differential Revision: https://phabricator.services.mozilla.com/D31134
--HG--
extra : moz-landing-system : lando
In part 1, we disabled the unhooking of DLL-intercepted functions at shutdown. The TestDllInterceptor relied on unhooking -- it worked by hooking functions with a "nonsense function" (nullptr) and then immediately unhooking it. That restored the original function behavior. Some hooked functions (e.g. NtWriteFile) are used by functions later in the program (e.g. printf) so the functions need to maintain their behavior.
This patch replaces the nonsense function with an identity function that also sets a global boolean as a side-effect. The function is written in machine code. x86-32, x86-64, and aarch64 variants are included.
Differential Revision: https://phabricator.services.mozilla.com/D30244
--HG--
extra : moz-landing-system : lando
QueryCredentialsAttributesA and FreeCredentialsHandle trigger an exception when null is passed for the CredHandle pointer. This exception was ignored (when not run in the debugger) but that is no longer the case with the changes in part 3. This patch passes a real CredHandle to them.
Differential Revision: https://phabricator.services.mozilla.com/D30243
--HG--
extra : moz-landing-system : lando
This patch fixes a static destructor order dependency between WindowsDllInterceptor and VMSharingPolicyUnique by telling VMSharingPolicyShared not to access the VMSharingPolicyUnique at destruction. This means that the behavior of intercepted functions is no longer restored in the given process at policy shutdown time.
Differential Revision: https://phabricator.services.mozilla.com/D28764
--HG--
extra : moz-landing-system : lando
On Windows 7, WinVerifyTrust fails unless the tag is uppercased. This patch
also adds a missing call to CryptCATAdminReleaseCatalogContext, the need for
which was poorly documented on MSDN.
Differential Revision: https://phabricator.services.mozilla.com/D30146
--HG--
extra : moz-landing-system : lando
TrampolineCollection iterates over an array of Trampolines that it has set 'write' permissions for. If this happens in a process whose sandbox forbids dynamic code then these permissions cannot be set. This patch detects that condition and returns an empty TrampolineCollection in that case. We ASSERT if we fail to set permissions for any other reason.
Differential Revision: https://phabricator.services.mozilla.com/D28613
--HG--
extra : moz-landing-system : lando
Bug 1533808 introduced code to intercept DLL methods that the Chromium sandbox had already intercepted. That patch did not store the the pointer to the intercepted function in the trampoline data, as is done when intercepting other methods.
Differential Revision: https://phabricator.services.mozilla.com/D28612
--HG--
extra : moz-landing-system : lando
We're going to convert the test to a gtest, and it's simpler not to have
to deal with finding the path to the testcase zip files. They're small
enough anyways, and can be inserted as raw binary data via some assembly
magic. This being android-only code, we don't need extreme portability
here. This is the same trick we use in
config/external/icu/data/icudata_gas.S.
Differential Revision: https://phabricator.services.mozilla.com/D28758
--HG--
extra : moz-landing-system : lando
This patch fixes a static destructor order dependency between WindowsDllInterceptor and VMSharingPolicyUnique by telling VMSharingPolicyShared not to access the VMSharingPolicyUnique at destruction. See the bug for details of the order dependency.
Differential Revision: https://phabricator.services.mozilla.com/D28764
--HG--
extra : moz-landing-system : lando
See comment 24 in the bug for details on what can go wrong without this
change. This change ensures system libraries are not going to pick
symbols from mozglue when running processes outside dalvik.
As a side effect, this makes things kind of closer to what happens when
dalvik is involved, exposing unit tests to possible allocator mismatches
that could happen like bug 1531887.
On the flip side, libraries that link against mozglue explicitly are
going to get a reference to the versioned symbols, so everything is fine
in that regard. The custom linker, however, will ignore the versions
altogether, and its symbols resolution just ends up unchanged. So we're
fine there too.
We use something that is close to what using a SYMBOLS_FILE would
generate as a version script, but we need to do so manually because
SYMBOLS_FILE doesn't support exporting all the symbols.
Differential Revision: https://phabricator.services.mozilla.com/D28030
--HG--
extra : moz-landing-system : lando
Bug 884239 added a build-time Android version check around the
pthread_atfork function definition at the same time as for timer_create,
which was subsequently removed. But it turns out the version that
documented was wrong: per the comment added in bug 680190,
pthread_atfork might have been supported since Android 2.3 (gingerbread,
API 9 or 10). That might not be entirely accurate, though, because the
bionic repository seems to show it made it to Android 4.0 (ice cream
sandwich, API 14 or 15).
Either way, that is less than the minimum API version we currently
support, which is 16.
Differential Revision: https://phabricator.services.mozilla.com/D27848
--HG--
extra : moz-landing-system : lando
There shouldn't be any need to do this for content processes as
the DLL should already be in the system file cache.
Differential Revision: https://phabricator.services.mozilla.com/D26017
--HG--
extra : moz-landing-system : lando
There shouldn't be any need to do this for content processes as
the DLL should already be in the system file cache.
Differential Revision: https://phabricator.services.mozilla.com/D26017
--HG--
extra : moz-landing-system : lando
This way we don't have to go through a bunch of printf nonsense, and we
ought to be able to arrive at optimized routines that take advantage of
constant radices, etc.
Differential Revision: https://phabricator.services.mozilla.com/D25141
--HG--
extra : moz-landing-system : lando
Due to coming changes involving the IOInterposer, the WindowsDllInterceptor may be set up later than the sandbox. The sandbox hooks some of the same functions, so the Interceptor is running into its hooks instead of the original implementations it anticipated. This patch allows it to recognize and efficiently patch those hooks when that happens.
Differential Revision: https://phabricator.services.mozilla.com/D24654
--HG--
extra : moz-landing-system : lando
When Android shuts down the ndk process, it doesn't call the registered
atexit() handlers, which is normally where the profile data gets written
to file. Since the PGO test suite closes the browser when it is
finished, the nativeRun routine can manually call out to
__llvm_profile_dump() before returning.
This method has a downside that only the profile data from the calling
library gets written out, rather than for the whole process. Since we
are most interested in optimizing libxul, a new hook is added in
Bootstrap to make sure we get the profile data for the right library.
Differential Revision: https://phabricator.services.mozilla.com/D22817
--HG--
extra : source : 0615c775a0cf6e8f98e1c051cd574c0d602a738a
Greg Tatum
Aaron Klotz
Bogdan Tara
Denis Palmeiro
Gerald Squelart
Noemi Erli
Mike Hommey
Ehsan Akhgari
Masatoshi Kimura
Mihai Alexandru Michis
Sylvestre Ledru
Nathan Froyd
arthur.iakab
Emilio Cobos Álvarez
David Parks
Kartikaya Gupta
Dorel Luca
Geoff Brown
Doug Thayer
Cosmin Sabou
Csoregi Natalia
Narcis Beleuzu
Cameron McCormack
John Lin
Karl Tomlinson
Mike Shal