Граф коммитов

1363 Коммитов

Автор SHA1 Сообщение Дата
kriswright ddd0133b2a Bug 1572238 - Dump assertion stacks using __android_log_print. r=Ehsan
Defines an android-only version of nsTraceRefcnt::WalkTheStack that takes a function callback, which outputs the stack frame buffer to `__android_log_print`. Also uses `__wrap_dladdr` in MozDescribeCodeAddress, which outputs slightly more informative data for the stack trace (instead of instances of '???/??? [???]').

Differential Revision: https://phabricator.services.mozilla.com/D46868

--HG--
extra : moz-landing-system : lando
2019-09-24 20:50:41 +00:00
Tyson Smith d9bce5f0fe Bug 1581656 - Fix default UBSAN_OPTIONS. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D46970

--HG--
extra : moz-landing-system : lando
2019-09-24 17:52:11 +00:00
Bogdan Tara e4913f5aeb Backed out changeset 88334e3a605e (bug 1572238) for causing windows 2012 bustages CLOSED TREE 2019-09-24 21:57:56 +03:00
kriswright 9ac396cf78 Bug 1572238 - Dump assertion stacks using __android_log_print. r=Ehsan
Defines an android-only version of nsTraceRefcnt::WalkTheStack that takes a function callback, which outputs the stack frame buffer to `__android_log_print`. Also uses `__wrap_dladdr` in MozDescribeCodeAddress, which outputs slightly more informative data for the stack trace (instead of instances of '???/??? [???]').

Differential Revision: https://phabricator.services.mozilla.com/D46868

--HG--
extra : moz-landing-system : lando
2019-09-24 12:14:24 +00:00
Aaron Klotz edafb91e6b Bug 1583417: Remove assertion that may fire spuriously due to racy condition; r=mhowell
This assertion could fire if the profiler state changes between beginning and
ending a DLL load. That's less than ideal, so let's just get rid of it.

Differential Revision: https://phabricator.services.mozilla.com/D46897

--HG--
extra : moz-landing-system : lando
2019-09-24 14:53:23 +00:00
Gerald Squelart 16dbd4f365 Bug 1582992 - Removed unneded BlocksRingBuffer's entry destructor, and ProfilerBuffer's JS::AutoSuppressGCAnalysis - r=gregtatum
`BlocksRingBuffer` had an "entry destructor" to make it a more generic
container, and it was useful during early prototyping of the new profiler
storage (so that we could store owning pointers).
But this entry destructor is stored in an `std::function`, which gets marked as
a potential GC caller by the js rooting hazard analyzer; and as this bug found,
it's not obvious where to place `JS::AutoSuppressGCAnalysis`, because profiler
entries (including stacks) could be added on one thread while GC happens
elsewhere, which triggers the embedded `AutoAssertNoGC` check.

Since we don't actually use the entry destructor facility in the profiler, it's
easier to just get rid of it. As a bonus, it's a small optimization.
Tests that were using an entry destructor now use the `State` instead, to verify
that entries are pushed and cleared as expected.

If needed in the future outside of the profiler, `BlocksRingBuffer` could again
include an entry destructor, but it would have to be through templating, so that
the class used in the profiler wouldn't contain an `std::function`.

Differential Revision: https://phabricator.services.mozilla.com/D46738

--HG--
extra : moz-landing-system : lando
2019-09-23 20:50:20 +00:00
Aaron Klotz fccd6eb9ba Bug 1542830: Part 4 - Modify mozglue to use new untrusted modules interfaces; r=mhowell
* At this point our DLL blocking infra is complicated enough that I decided to
  bite the bullet and move all of this code out of `mozglue/build` and into its
  own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
   receives a vector containing all saved records of loaded DLLs that happened
   until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
   processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
  any other non-`firefox` processes that do not have a launcher process
  providing a loader API.
* We add some wide to UTF8 conversion functions.

Depends on D43157

Differential Revision: https://phabricator.services.mozilla.com/D43158

--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : browser/app/winlauncher/freestanding/LoaderAPIInterfaces.h => mozglue/dllservices/LoaderAPIInterfaces.h
rename : browser/app/winlauncher/freestanding/ModuleLoadInfo.h => mozglue/dllservices/ModuleLoadInfo.h
rename : browser/app/winlauncher/NtLoaderAPI.h => mozglue/dllservices/NtLoaderAPI.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
2019-09-23 20:18:41 +00:00
Aaron Klotz c27d5345fc Bug 1542830: Part 1 - Updates to NativeNt.h; r=mhowell
This patch adds the following:

* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
  owns its buffer. The buffers are null-terminated so that they may be used as
  C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
  current thread ID directly from the `TEB` when we do not yet have access to
  kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
  use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
  MFBT `Vector` in code that might not yet have access to Win32 heap functions.

Differential Revision: https://phabricator.services.mozilla.com/D43155

--HG--
extra : moz-landing-system : lando
2019-09-23 20:16:51 +00:00
philipp e517ca80d9 Bug 1576728 - Add crashing dlls from chinese unzipping tools to the blocklist. r=aklotz
--HG--
extra : rebase_source : b8eda4974164e350fa6e1df9bcf79c5f9baa966a
2019-09-23 21:19:02 +03:00
Brindusan Cristian 125c934a04 Backed out 8 changesets (bug 1542830) for causing Nightly bustages. a=backout
Backed out changeset 6fcb417f7ff4 (bug 1542830)
Backed out changeset 1aa253e6604a (bug 1542830)
Backed out changeset 73ec288886cd (bug 1542830)
Backed out changeset 84b903e60dc9 (bug 1542830)
Backed out changeset 8e2da9ff5f5a (bug 1542830)
Backed out changeset c4e547a6a039 (bug 1542830)
Backed out changeset 919f1af7c135 (bug 1542830)
Backed out changeset da7e775c4051 (bug 1542830)

--HG--
rename : browser/app/winlauncher/freestanding/DllBlocklist.cpp => browser/app/winlauncher/DllBlocklistWin.cpp
rename : browser/app/winlauncher/freestanding/DllBlocklist.h => browser/app/winlauncher/DllBlocklistWin.h
rename : mozglue/dllservices/Authenticode.cpp => mozglue/build/Authenticode.cpp
rename : mozglue/dllservices/Authenticode.h => mozglue/build/Authenticode.h
rename : mozglue/misc/WinUtils.h => mozglue/build/MozglueUtils.h
rename : mozglue/dllservices/WindowsDllBlocklist.cpp => mozglue/build/WindowsDllBlocklist.cpp
rename : mozglue/dllservices/WindowsDllBlocklist.h => mozglue/build/WindowsDllBlocklist.h
rename : mozglue/dllservices/WindowsDllBlocklistCommon.h => mozglue/build/WindowsDllBlocklistCommon.h
rename : mozglue/dllservices/WindowsDllBlocklistDefs.in => mozglue/build/WindowsDllBlocklistDefs.in
rename : mozglue/dllservices/WindowsDllServices.h => mozglue/build/WindowsDllServices.h
rename : mozglue/dllservices/gen_dll_blocklist_defs.py => mozglue/build/gen_dll_blocklist_defs.py
rename : toolkit/components/telemetry/docs/data/third-party-modules-ping.rst => toolkit/components/telemetry/docs/data/untrusted-modules-ping.rst
rename : toolkit/components/telemetry/tests/unit/test_ThirdPartyModulesPing.js => toolkit/components/telemetry/tests/unit/test_UntrustedModulesPing.js
rename : toolkit/xre/UntrustedModulesProcessor.cpp => toolkit/xre/ModuleEvaluator_windows.cpp
rename : toolkit/xre/UntrustedModulesProcessor.h => toolkit/xre/ModuleEvaluator_windows.h
rename : toolkit/xre/ModuleVersionInfo.cpp => toolkit/xre/ModuleVersionInfo_windows.cpp
rename : toolkit/xre/ModuleVersionInfo.h => toolkit/xre/ModuleVersionInfo_windows.h
2019-09-22 00:01:01 +03:00
Aaron Klotz 8d1be668f6 Bug 1542830: Part 4 - Modify mozglue to use new untrusted modules interfaces; r=mhowell
* At this point our DLL blocking infra is complicated enough that I decided to
  bite the bullet and move all of this code out of `mozglue/build` and into its
  own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
   receives a vector containing all saved records of loaded DLLs that happened
   until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
   processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
  any other non-`firefox` processes that do not have a launcher process
  providing a loader API.
* We add some wide to UTF8 conversion functions.

Differential Revision: https://phabricator.services.mozilla.com/D43158

--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
2019-09-20 19:43:33 +00:00
Aaron Klotz f5081e7b43 Bug 1542830: Part 1 - Updates to NativeNt.h; r=mhowell
This patch adds the following:

* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
  owns its buffer. The buffers are null-terminated so that they may be used as
  C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
  current thread ID directly from the `TEB` when we do not yet have access to
  kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
  use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
  MFBT `Vector` in code that might not yet have access to Win32 heap functions.

Differential Revision: https://phabricator.services.mozilla.com/D43155

--HG--
extra : moz-landing-system : lando
2019-09-20 19:42:40 +00:00
Ciure Andrei 7db4ad5d80 Backed out 8 changesets (bug 1542830) for causing spidermonkey bustages CLOSED TREE
Backed out changeset b9f7fc8d0172 (bug 1542830)
Backed out changeset b85f58fd5bbd (bug 1542830)
Backed out changeset 0a9169ab2623 (bug 1542830)
Backed out changeset 39a7c05d54ef (bug 1542830)
Backed out changeset 7a3cbd2f59f0 (bug 1542830)
Backed out changeset 268530552281 (bug 1542830)
Backed out changeset df658ffb8599 (bug 1542830)
Backed out changeset e7bef6486a38 (bug 1542830)

--HG--
rename : browser/app/winlauncher/freestanding/DllBlocklist.cpp => browser/app/winlauncher/DllBlocklistWin.cpp
rename : browser/app/winlauncher/freestanding/DllBlocklist.h => browser/app/winlauncher/DllBlocklistWin.h
rename : mozglue/dllservices/Authenticode.cpp => mozglue/build/Authenticode.cpp
rename : mozglue/dllservices/Authenticode.h => mozglue/build/Authenticode.h
rename : mozglue/misc/WinUtils.h => mozglue/build/MozglueUtils.h
rename : mozglue/dllservices/WindowsDllBlocklist.cpp => mozglue/build/WindowsDllBlocklist.cpp
rename : mozglue/dllservices/WindowsDllBlocklist.h => mozglue/build/WindowsDllBlocklist.h
rename : mozglue/dllservices/WindowsDllBlocklistCommon.h => mozglue/build/WindowsDllBlocklistCommon.h
rename : mozglue/dllservices/WindowsDllBlocklistDefs.in => mozglue/build/WindowsDllBlocklistDefs.in
rename : mozglue/dllservices/WindowsDllServices.h => mozglue/build/WindowsDllServices.h
rename : mozglue/dllservices/gen_dll_blocklist_defs.py => mozglue/build/gen_dll_blocklist_defs.py
rename : toolkit/components/telemetry/docs/data/third-party-modules-ping.rst => toolkit/components/telemetry/docs/data/untrusted-modules-ping.rst
rename : toolkit/components/telemetry/tests/unit/test_ThirdPartyModulesPing.js => toolkit/components/telemetry/tests/unit/test_UntrustedModulesPing.js
rename : toolkit/xre/UntrustedModulesProcessor.cpp => toolkit/xre/ModuleEvaluator_windows.cpp
rename : toolkit/xre/UntrustedModulesProcessor.h => toolkit/xre/ModuleEvaluator_windows.h
rename : toolkit/xre/ModuleVersionInfo.cpp => toolkit/xre/ModuleVersionInfo_windows.cpp
rename : toolkit/xre/ModuleVersionInfo.h => toolkit/xre/ModuleVersionInfo_windows.h
2019-09-20 21:21:39 +03:00
Aaron Klotz 12843d81e9 Bug 1542830: Part 4 - Modify mozglue to use new untrusted modules interfaces; r=mhowell
* At this point our DLL blocking infra is complicated enough that I decided to
  bite the bullet and move all of this code out of `mozglue/build` and into its
  own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
   receives a vector containing all saved records of loaded DLLs that happened
   until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
   processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
  any other non-`firefox` processes that do not have a launcher process
  providing a loader API.
* We add some wide to UTF8 conversion functions.

Differential Revision: https://phabricator.services.mozilla.com/D43158

--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
2019-09-20 17:00:14 +00:00
Aaron Klotz 57df2b6a01 Bug 1542830: Part 1 - Updates to NativeNt.h; r=mhowell
This patch adds the following:

* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
  owns its buffer. The buffers are null-terminated so that they may be used as
  C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
  current thread ID directly from the `TEB` when we do not yet have access to
  kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
  use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
  MFBT `Vector` in code that might not yet have access to Win32 heap functions.

Differential Revision: https://phabricator.services.mozilla.com/D43155

--HG--
extra : moz-landing-system : lando
2019-09-20 16:59:28 +00:00
Aaron Klotz 38e11b4896 Bug 1582776: Change cross-process stub to not store remote stub pointer in local mOrigFunc; r=handyman
This is showing up with hooks that are set both by the launcher process and by
the browser process when starting early DLL blocklist init on content processes:

* The browser's copy of mOrigFunc was set by the launcher process.
* The browser is setting a hook in the new child process, which writes to the
  child's mOrigFunc.
* But FuncHookCrossProcess also writes that pointer to the browser's mOrigFunc,
  thus corrupting the browser process's copy of the pointer.

For in-process hooks, we want to immediately write the stub pointer to its final
location; this is not an issue for cross-process hooks since the child process
is suspended when we do this and the parent process can't call the stub; there
is no possibility of a race.

Differential Revision: https://phabricator.services.mozilla.com/D46615

--HG--
extra : moz-landing-system : lando
2019-09-20 16:55:27 +00:00
Andreea Pavel 31d9596a28 Backed out 8 changesets (bug 1542830) for gecko decision failure on a CLOSED TREE
DONTBUILD

Backed out changeset 4f72161be496 (bug 1542830)
Backed out changeset 7ee12138946d (bug 1542830)
Backed out changeset 9e90ee4981c6 (bug 1542830)
Backed out changeset 363039c98534 (bug 1542830)
Backed out changeset 70ffff30551a (bug 1542830)
Backed out changeset c25b7bf354d9 (bug 1542830)
Backed out changeset d156dc595b69 (bug 1542830)
Backed out changeset 9f5dbd1b2959 (bug 1542830)

--HG--
rename : browser/app/winlauncher/freestanding/DllBlocklist.cpp => browser/app/winlauncher/DllBlocklistWin.cpp
rename : browser/app/winlauncher/freestanding/DllBlocklist.h => browser/app/winlauncher/DllBlocklistWin.h
rename : mozglue/dllservices/Authenticode.cpp => mozglue/build/Authenticode.cpp
rename : mozglue/dllservices/Authenticode.h => mozglue/build/Authenticode.h
rename : mozglue/misc/WinUtils.h => mozglue/build/MozglueUtils.h
rename : mozglue/dllservices/WindowsDllBlocklist.cpp => mozglue/build/WindowsDllBlocklist.cpp
rename : mozglue/dllservices/WindowsDllBlocklist.h => mozglue/build/WindowsDllBlocklist.h
rename : mozglue/dllservices/WindowsDllBlocklistCommon.h => mozglue/build/WindowsDllBlocklistCommon.h
rename : mozglue/dllservices/WindowsDllBlocklistDefs.in => mozglue/build/WindowsDllBlocklistDefs.in
rename : mozglue/dllservices/WindowsDllServices.h => mozglue/build/WindowsDllServices.h
rename : mozglue/dllservices/gen_dll_blocklist_defs.py => mozglue/build/gen_dll_blocklist_defs.py
rename : toolkit/components/telemetry/docs/data/third-party-modules-ping.rst => toolkit/components/telemetry/docs/data/untrusted-modules-ping.rst
rename : toolkit/components/telemetry/tests/unit/test_ThirdPartyModulesPing.js => toolkit/components/telemetry/tests/unit/test_UntrustedModulesPing.js
rename : toolkit/xre/UntrustedModulesProcessor.cpp => toolkit/xre/ModuleEvaluator_windows.cpp
rename : toolkit/xre/UntrustedModulesProcessor.h => toolkit/xre/ModuleEvaluator_windows.h
rename : toolkit/xre/ModuleVersionInfo.cpp => toolkit/xre/ModuleVersionInfo_windows.cpp
rename : toolkit/xre/ModuleVersionInfo.h => toolkit/xre/ModuleVersionInfo_windows.h
2019-09-20 19:49:27 +03:00
Aaron Klotz 6c7b43b9c8 Bug 1542830: Part 4 - Modify mozglue to use new untrusted modules interfaces; r=mhowell
* At this point our DLL blocking infra is complicated enough that I decided to
  bite the bullet and move all of this code out of `mozglue/build` and into its
  own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
   receives a vector containing all saved records of loaded DLLs that happened
   until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
   processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
  any other non-`firefox` processes that do not have a launcher process
  providing a loader API.
* We add some wide to UTF8 conversion functions.

Differential Revision: https://phabricator.services.mozilla.com/D43158

--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
2019-09-20 00:09:41 +00:00
Aaron Klotz f3b16f06a4 Bug 1542830: Part 1 - Updates to NativeNt.h; r=mhowell
This patch adds the following:

* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
  owns its buffer. The buffers are null-terminated so that they may be used as
  C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
  current thread ID directly from the `TEB` when we do not yet have access to
  kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
  use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
  MFBT `Vector` in code that might not yet have access to Win32 heap functions.

Differential Revision: https://phabricator.services.mozilla.com/D43155

--HG--
extra : moz-landing-system : lando
2019-09-20 00:09:33 +00:00
Gerald Squelart 1549c7e36e Bug 1582357 - Allow profiler entries up to the size of the buffer - r=gregtatum
In some situations, entries may in fact take more than half the buffer size
(e.g., when duplicating a stack into a small temporary buffer).
So we now allow blocks to take the full buffer size -- but not more, as they
would start overwriting themselves!

Differential Revision: https://phabricator.services.mozilla.com/D46453

--HG--
extra : moz-landing-system : lando
2019-09-19 13:40:01 +00:00
Gerald Squelart 64ec6b96a0 Bug 1581049 - BlocksRingBuffer (de)serializer uses optimized inter-ModuloBuffer copier - r=gregtatum
Instead of copying `BlocksRingBuffer` data byte-by-byte (using iterators byte
dereferencers), we can now use `ModuloBuffer::Iterator::ReadInto(Iterator&)` to
copy them using a small number of `memcpy`s.

Differential Revision: https://phabricator.services.mozilla.com/D45839

--HG--
extra : moz-landing-system : lando
2019-09-18 01:22:42 +00:00
Gerald Squelart 229768ce0a Bug 1581049 - ModuloBuffer::Iterator::ReadInto(ModuloBuffer::Iterator&) - r=gregtatum
Some objects are copied byte-by-byte to/from `ModuloBuffer`s.
E.g., serialized `BlocksRingBuffer`s, or duplicate stacks. (And more to come.)

`Iterator::ReadInto(Iterator&)` optimizes these copies by using the minimum
number of `memcpy`s possible.

Differential Revision: https://phabricator.services.mozilla.com/D45838

--HG--
extra : moz-landing-system : lando
2019-09-18 01:22:30 +00:00
Gerald Squelart 738f482766 Bug 1578329 - profiler_can_accept_markers(), don't store markers when paused - r=gregtatum
profiler_can_accept_markers() is a fast and racy check that markers would
currently be stored, it should be used around potentially-expensive calls to
add markers.

And now markers are no longer stored when the profiler is paused. (Note that the
profiler is paused when a profile is being stored, this will help make this
operation faster.)

Differential Revision: https://phabricator.services.mozilla.com/D44434

--HG--
extra : moz-landing-system : lando
2019-09-18 01:21:50 +00:00
Gerald Squelart a8b053c3b1 Bug 1578327 - Discard old data just before streaming - r=gregtatum
Since all profiler data is now stored inside ProfileBuffer, there is no real
need to continuously discard old data during sampling (this was particularly
useful to reclaim memory taken by old markers&payloads).

Instead, we can now just discard the old data once, just before starting to
stream it to JSON.

Differential Revision: https://phabricator.services.mozilla.com/D44433

--HG--
extra : moz-landing-system : lando
2019-09-18 01:21:32 +00:00
Gerald Squelart 58ac06a047 Bug 1576555 - Remove unneeded ProfilerMarker - r=gregtatum
Now that what was in ProfilerMarker is stored directly in `BlocksRingBuffer`,
there is no need for this class anymore!
This also removes all the pointer management around it (when added to a TLS
list, moved during sampling, deleted when expired).

Differential Revision: https://phabricator.services.mozilla.com/D43431

--HG--
extra : moz-landing-system : lando
2019-09-18 01:21:20 +00:00
Gerald Squelart 02d2886883 Bug 1576555 - Don't store temporary ProfilerMarkerPayloads in UniquePtrs - r=gregtatum
Since payloads are not kept alive long after they have been serialized, we can
just create them on the stack and pass a reference to their base (or pointer,
`nullptr` representing "no payloads") to `profiler_add_marker()`.

Differential Revision: https://phabricator.services.mozilla.com/D43430

--HG--
extra : moz-landing-system : lando
2019-09-18 01:21:02 +00:00
Gerald Squelart 666a5caa9d Bug 1576555 - Store markers directly into BlocksRingBuffer - r=gregtatum
Markers and their payloads can now be serialized straight into the profiler's
`BlocksRingBuffer`, which is now thread-safe to allow such concurrent accesses
(even when gPSMutex is not locked).
This already saves us from having to allocate a `PayloadMarker` on the heap, and
from having to manage it in different lists.

The now-thread-safe `BlocksRingBuffer` in `CorePS` cannot be used inside the
critical section of `SamplerThread::Run`, because any mutex function could hang
because of the suspended thread (even though they functionally don't appear to
interact). So the sampler now uses a local `BlocksRingBuffer` without mutex.
As a bonus, the separate buffer helps reduce the number of concurrent locking
operations needed when capturing the stack.

Differential Revision: https://phabricator.services.mozilla.com/D43429

--HG--
extra : moz-landing-system : lando
2019-09-18 01:20:45 +00:00
Gerald Squelart 06043a38e3 Bug 1576555 - BlocksRingBuffer::AppendContents(const BlocksRingBuffer&) - r=gregtatum
Copy the full contents of BlocksRingBuffer into another one.
This is mainly useful to use a temporary buffer to store some data without
contentions, and then integrate the temporary buffer into the main one.

Differential Revision: https://phabricator.services.mozilla.com/D45306

--HG--
extra : moz-landing-system : lando
2019-09-18 01:20:28 +00:00
Gerald Squelart fc93e3d9da Bug 1575448 - De/serialize ProfilerMarkerPayload derived objects - r=gregtatum
Payloads will serialize themselves into a `BlocksRingBuffer` entry when first
captured.
Later they will be deserialized, to stream JSON for the output profile.

Differential Revision: https://phabricator.services.mozilla.com/D43428

--HG--
extra : moz-landing-system : lando
2019-09-18 01:20:10 +00:00
Gerald Squelart 0ffe75f626 Bug 1575448 - ProfilerMarkerPayload::CommonProps - r=gregtatum
The common data members stored in the ProfilerMarkerPayload base class can be
gathered into a struct, which will make it easier to pass around, especially
when a derived object is constructed with these common properties.

Differential Revision: https://phabricator.services.mozilla.com/D43427

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:53 +00:00
Gerald Squelart ac26c5d9f5 Bug 1576554 - {S,Des}erializer<ProfilerBacktrace> - r=gregtatum
Markers may contain backtraces, so we need to be able to de/serialize them.
This involves de/serializing the underyling `BlocksRingBuffer`.

Differential Revision: https://phabricator.services.mozilla.com/D43426

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:40 +00:00
Gerald Squelart 0de7cb4c2e Bug 1576554 - Test baseprofiler::profiler_get_backtrace - r=gregtatum
Add basic testing of `profiler_get_backtrace` before working on serializing it.

Differential Revision: https://phabricator.services.mozilla.com/D43424

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:23 +00:00
Gerald Squelart 9f7c402a79 Bug 1576551 - Store BlocksRingBuffer outside of ProfileBuffer - r=gregtatum
The main `BlocksRingBuffer`s will be stored in `CorePS` (outside of
`ProfileBuffer`s), as we need to be able to safely access the underlying buffers
when profilers are not enabled.

Also `ProfilerBacktrace` will own the `BlocksRingBuffer` that its captured
`ProfileBuffer` uses.

Taking this opportunity to rename the different `mBuffer`s to more useful names.

Differential Revision: https://phabricator.services.mozilla.com/D43422

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:21 +00:00
Gerald Squelart 74301e6c5c Bug 1576551 - Assume capacity is in 8-byte entries - r=gregtatum
Now that we are using a byte-oriented `BlocksRingBuffer` instead of an array of
9-byte `ProfileBufferEntry`'s, internally the profiler sets the buffer size in
bytes. However all external users (popup, tests, etc.) still assume that the
requested capacity is in entries!

To limit the amount of changes, we will keep assuming externally-visible
capacities are in entries, and convert them to bytes.
Even though entries used to be 9 bytes each, and `BlocksRingBuffer` adds 1 byte
for the entry size, some entries actually need less space (e.g., 32-bit numbers
now take 6 bytes instead of 9), so converting to less than 9 bytes per entry is
acceptable.
We are settling on 8 bytes per entry: It's close to 9, and it's a power of two;
since the effective number of entries was a power of two, and `BlocksRingBuffer`
also uses a power of two size in bytes, this convertion keeps sizes in powers of
two.

Differential Revision: https://phabricator.services.mozilla.com/D44953

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:14 +00:00
Gerald Squelart 6666138a83 Bug 1576551 - Use BlocksRingBuffer in ProfileBuffer - r=gregtatum
This just replaces `ProfileBuffer`'s self-managed circular buffer with a
`BlocksRingBuffer`.

That `BlocksRingBuffer` does not need its own mutex (yet), because all uses go
through gPSMutex-guarded code.

`ProfileBuffer` now also pre-allocates a small buffer for use in
`DuplicateLastSample()`, this avoids multiple mallocs at each sleeping thread
stack duplication.

Note: Internal "magic" sizes have been multiplied by 8 (and tweaked upwards, to
handle bigger stacks), because they originally were the number of 9-byte
entries, but now it's the buffer size in bytes. (And entries can now be smaller
than 9 bytes, so overall the capacity in entries should be similar or better.)
However, external calls still think they are giving a number of "entries", this
will be handled in the next patch.

Differential Revision: https://phabricator.services.mozilla.com/D43421

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:12 +00:00
Gerald Squelart 82b9f4f30f Bug 1576551 - AUTO_PROFILER_STATS(locked_profiler_stream_json_for_this_process) - r=gregtatum
Add some stats (off by default) around streaming JSON, as the following patches
may affect it.

Differential Revision: https://phabricator.services.mozilla.com/D44952

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:10 +00:00
Gerald Squelart c90cc7f5b7 Bug 1580091 - Use BaseProfilerMaybeMutex in BlocksRingBuffer - r=gregtatum
In some situations we will *need* to use a `BlocksRingBuffer` that absolutely
does not use a mutex -- In particular in the critical section of
`SamplerThread::Run()`, when a thread is suspended, because any action on any
mutex (even a private one that no-one else interacts with) can result in a hang.

As a bonus, `BlocksRingBuffer`s that are known not to be used in multi-thread
situations (e.g., backtraces, extracted stacks for duplication, etc.) will waste
less resources trying to lock/unlock their mutex.

Differential Revision: https://phabricator.services.mozilla.com/D45305

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:08 +00:00
Gerald Squelart cac9b1645b Bug 1580091 - BaseProfilerMaybeMutex and BaseProfilerMaybeAutoLock - r=gregtatum
`BaseProfilerMaybeMutex` wraps a `BaseProfilerMutex` inside a `Maybe`.
The decision to use a mutex or not is set at construction time.
If there is no mutex, all operations do nothing (at the small cost of checking
if the mutex is present.)

`BaseProfilerMaybeAutoLock` is the recommented RAII object to lock and
automatically unlock a `BaseProfilerMaybeMutex` until the end of a block scope.

Differential Revision: https://phabricator.services.mozilla.com/D45304

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:06 +00:00
Bogdan Tara 361c83c1a9 Backed out 21 changesets (bug 1578327, bug 1575448, bug 1580091, bug 1576554, bug 1581049, bug 1576551, bug 1578329, bug 1576555) for platform.cpp related bustage CLOSED TREE
Backed out changeset 27afea20c396 (bug 1581049)
Backed out changeset be1ec4577d37 (bug 1581049)
Backed out changeset b4b6bbb18cc1 (bug 1581049)
Backed out changeset b30942f9db54 (bug 1581049)
Backed out changeset 473c431866f7 (bug 1578329)
Backed out changeset ac492dc3df20 (bug 1578329)
Backed out changeset f51875652f6f (bug 1578327)
Backed out changeset 512b7cbd18f6 (bug 1576555)
Backed out changeset 2d63a9934c00 (bug 1576555)
Backed out changeset ff73f648ab6c (bug 1576555)
Backed out changeset 49f49079bbb5 (bug 1576555)
Backed out changeset e8323157c6f3 (bug 1575448)
Backed out changeset 3b0d726f2dd6 (bug 1575448)
Backed out changeset 5924790abc4b (bug 1576554)
Backed out changeset a14ac9bb5338 (bug 1576554)
Backed out changeset b6d73f5042a7 (bug 1576551)
Backed out changeset 366030bd2d84 (bug 1576551)
Backed out changeset d7ee4148aad9 (bug 1576551)
Backed out changeset ec72dfc7301e (bug 1576551)
Backed out changeset 79b29286f906 (bug 1580091)
Backed out changeset 6f34c2e57ccf (bug 1580091)
2019-09-17 10:16:14 +03:00
Gerald Squelart 541c78764a Bug 1581049 - BlocksRingBuffer (de)serializer uses optimized inter-ModuloBuffer copier - r=gregtatum
Instead of copying `BlocksRingBuffer` data byte-by-byte (using iterators byte
dereferencers), we can now use `ModuloBuffer::Iterator::ReadInto(Iterator&)` to
copy them using a small number of `memcpy`s.

Differential Revision: https://phabricator.services.mozilla.com/D45839

--HG--
extra : moz-landing-system : lando
2019-09-17 01:54:21 +00:00
Gerald Squelart 4c2ab062ce Bug 1581049 - ModuloBuffer::Iterator::ReadInto(ModuloBuffer::Iterator&) - r=gregtatum
Some objects are copied byte-by-byte to/from `ModuloBuffer`s.
E.g., serialized `BlocksRingBuffer`s, or duplicate stacks. (And more to come.)

`Iterator::ReadInto(Iterator&)` optimizes these copies by using the minimum
number of `memcpy`s possible.

Differential Revision: https://phabricator.services.mozilla.com/D45838

--HG--
extra : moz-landing-system : lando
2019-09-17 01:54:03 +00:00
Gerald Squelart b643004d63 Bug 1578329 - profiler_can_accept_markers(), don't store markers when paused - r=gregtatum
profiler_can_accept_markers() is a fast and racy check that markers would
currently be stored, it should be used around potentially-expensive calls to
add markers.

And now markers are no longer stored when the profiler is paused. (Note that the
profiler is paused when a profile is being stored, this will help make this
operation faster.)

Differential Revision: https://phabricator.services.mozilla.com/D44434

--HG--
extra : moz-landing-system : lando
2019-09-17 01:53:54 +00:00
Gerald Squelart 7d435fc17a Bug 1578327 - Discard old data just before streaming - r=gregtatum
Since all profiler data is now stored inside ProfileBuffer, there is no real
need to continuously discard old data during sampling (this was particularly
useful to reclaim memory taken by old markers&payloads).

Instead, we can now just discard the old data once, just before starting to
stream it to JSON.

Differential Revision: https://phabricator.services.mozilla.com/D44433

--HG--
extra : moz-landing-system : lando
2019-09-17 01:53:52 +00:00
Gerald Squelart 79e1b2e357 Bug 1576555 - Remove unneeded ProfilerMarker - r=gregtatum
Now that what was in ProfilerMarker is stored directly in `BlocksRingBuffer`,
there is no need for this class anymore!
This also removes all the pointer management around it (when added to a TLS
list, moved during sampling, deleted when expired).

Differential Revision: https://phabricator.services.mozilla.com/D43431

--HG--
extra : moz-landing-system : lando
2019-09-17 01:53:50 +00:00
Gerald Squelart 8364b4ebdb Bug 1576555 - Don't store temporary ProfilerMarkerPayloads in UniquePtrs - r=gregtatum
Since payloads are not kept alive long after they have been serialized, we can
just create them on the stack and pass a reference to their base (or pointer,
`nullptr` representing "no payloads") to `profiler_add_marker()`.

Differential Revision: https://phabricator.services.mozilla.com/D43430

--HG--
extra : moz-landing-system : lando
2019-09-17 02:32:21 +00:00
Gerald Squelart b48590ff79 Bug 1576555 - Store markers directly into BlocksRingBuffer - r=gregtatum
Markers and their payloads can now be serialized straight into the profiler's
`BlocksRingBuffer`, which is now thread-safe to allow such concurrent accesses
(even when gPSMutex is not locked).
This already saves us from having to allocate a `PayloadMarker` on the heap, and
from having to manage it in different lists.

The now-thread-safe `BlocksRingBuffer` in `CorePS` cannot be used inside the
critical section of `SamplerThread::Run`, because any mutex function could hang
because of the suspended thread (even though they functionally don't appear to
interact). So the sampler now uses a local `BlocksRingBuffer` without mutex.
As a bonus, the separate buffer helps reduce the number of concurrent locking
operations needed when capturing the stack.

Differential Revision: https://phabricator.services.mozilla.com/D43429

--HG--
extra : moz-landing-system : lando
2019-09-17 01:52:16 +00:00
Gerald Squelart 6d95d503f5 Bug 1576555 - BlocksRingBuffer::AppendContents(const BlocksRingBuffer&) - r=gregtatum
Copy the full contents of BlocksRingBuffer into another one.
This is mainly useful to use a temporary buffer to store some data without
contentions, and then integrate the temporary buffer into the main one.

Differential Revision: https://phabricator.services.mozilla.com/D45306

--HG--
extra : moz-landing-system : lando
2019-09-17 01:51:53 +00:00
Gerald Squelart 5b86107991 Bug 1575448 - De/serialize ProfilerMarkerPayload derived objects - r=gregtatum
Payloads will serialize themselves into a `BlocksRingBuffer` entry when first
captured.
Later they will be deserialized, to stream JSON for the output profile.

Differential Revision: https://phabricator.services.mozilla.com/D43428

--HG--
extra : moz-landing-system : lando
2019-09-17 01:51:41 +00:00
Gerald Squelart 64f0dcdf95 Bug 1575448 - ProfilerMarkerPayload::CommonProps - r=gregtatum
The common data members stored in the ProfilerMarkerPayload base class can be
gathered into a struct, which will make it easier to pass around, especially
when a derived object is constructed with these common properties.

Differential Revision: https://phabricator.services.mozilla.com/D43427

--HG--
extra : moz-landing-system : lando
2019-09-17 01:51:18 +00:00
Gerald Squelart a14e2c944d Bug 1576554 - {S,Des}erializer<ProfilerBacktrace> - r=gregtatum
Markers may contain backtraces, so we need to be able to de/serialize them.
This involves de/serializing the underyling `BlocksRingBuffer`.

Differential Revision: https://phabricator.services.mozilla.com/D43426

--HG--
extra : moz-landing-system : lando
2019-09-17 02:16:04 +00:00