Граф коммитов

1375 Коммитов

Автор SHA1 Сообщение Дата
Gabriele Svelto 850ac9c631 Bug 1579758 - Block a nightly build of OpenSC which crashes Firefox; r=aklotz
Differential Revision: https://phabricator.services.mozilla.com/D48296

--HG--
extra : moz-landing-system : lando
2019-10-13 04:20:45 +00:00
Geoff Brown 3dbb203a84 Bug 1558885 - Use separate directory for android gtest libxul; r=bc,nalexander
In Android gtest, instead of pushing the gtest libxul.so to the
(assumed and potentially incorrect) location of the test application's
libraries, push libxul.so to /data/local/gtest, and direct the linker
to use that instead.

Differential Revision: https://phabricator.services.mozilla.com/D34942

--HG--
extra : moz-landing-system : lando
2019-10-10 21:38:01 +00:00
Gerald Squelart d24177fe2a Bug 1587332 - Encapsulate ActivePS::mProfileBuffer as a ProfileBuffer value - r=canaltinova
Having `mProfileBuffer` be a pointer is not really helpful:
- The pointer is never null (It's allocated on ActivePS construction, and
  implicitly deleted on ActivePS destruction); it's never moved-from.
- It requires an extra `new` and an extra `delete`.

Differential Revision: https://phabricator.services.mozilla.com/D48650

--HG--
extra : moz-landing-system : lando
2019-10-10 10:52:09 +00:00
Gerald Squelart dff1750481 Bug 1587329 - MOZ_ASSERT(sInstance) in all functions that dereference sInstance - r=canaltinova
Add assertions that all `sInstance` pointers (from both `CorePS` and `ActivePS`)
are not null before being dereferenced.

This is probably more than needed, but it's only `MOZ_ASSERT`s limited to
Nightly, and it should give better feedback in case something goes wrong.

Eventually, I think it would be better to make most methods non-static, and have
a checked reference-to-instance getter.

Differential Revision: https://phabricator.services.mozilla.com/D48649

--HG--
extra : moz-landing-system : lando
2019-10-10 10:51:50 +00:00
Nazım Can Altınova 2e832289ac Bug 1512500 - Record parent page information in profiler r=gerald,nika
Differential Revision: https://phabricator.services.mozilla.com/D47426

--HG--
extra : moz-landing-system : lando
2019-10-09 21:25:19 +00:00
Nazım Can Altınova 6379b166c1 Bug 1583271 - Part 3: Remove the disabled assertion and add a mechanism to capture the first non-auto:blank frame r=nika
We are not simply excluding all about:blanks because there might be some
about:blank that user really visits. But for others we don't want to include
the first about:blank because when a BrowsingContext is loaded, and if the
principal matches, the first document loaded in it will share the inner window.

Differential Revision: https://phabricator.services.mozilla.com/D47067

--HG--
extra : moz-landing-system : lando
2019-10-09 20:58:06 +00:00
Nazım Can Altınova 20fc64e558 Bug 1583271 - Part 1: Change profiler page information IDs to BrowsingContextID and InnerWindowID r=gerald,nika
We were keeping nsDocShell::mHistoryId and nsDocShell::mOSHE as keys. They
weren't quite good because:
1. While loading an iframe, they were being registered twice with the same
ids(for about:blank and the real URL) sometimes.
2. It wasn't possible to access to the parent mHistoryId and mOSHE from a child
processes if the parent is in a different process. That may not be the case for
now, but it will be after fission.
So we had to find other IDs to:
1. Determine the Tab of the frames.
2. Determine the URLs of the frames.
For the first use case, we were using nsDocShell::mHistoryId for that purpose
but that was wrong. The closest thing that we can get to a tab ID is
BrowsingContext ID because they don't change after a navigation. But iframes
have different BrowsingContext's, so we still need to create a tree to
construct a tab content. That can be either in the front-end or capture time.
For the second use case, we were using a key pair of mHistoryId and mOSHE. We
now chose to keep inner window IDs for that purpose. Inner window IDs are
unique for each navigation loads because inner window correspond to each JS
window global objects. That's why we can use that without any problem. But one
problem is that we cannot handle `history.pushState` and `history.replaceState`
changes with that change since window global objects won't change during those.
But that was the best thing we can do after fission. So this will be a small
sacrifice for us to keep that functionality working after fission.
In that patch we also remove the registration/unregistration calls. We are
going to add those calls in the next patch.

Differential Revision: https://phabricator.services.mozilla.com/D47065

--HG--
extra : moz-landing-system : lando
2019-10-09 21:25:11 +00:00
Sylvestre Ledru f12b9fa5c3 Bug 1519636 - Reformat recent changes to the Google coding style r=Ehsan
# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D47737

--HG--
extra : moz-landing-system : lando
2019-10-06 18:29:55 +00:00
Gerald Squelart 499b5db10f Bug 1583868 - New profiler feature "nostacksampling" - r=gregtatum
Previously, the absence of "stackwalk", "leaf", and "javascript" implied that
the test/user didn't want any sampling, but this caused issues in some tests
that enabled "stackwalk" on platforms that didn't support stack-walking, which
ended up suppressing label-only stacks that the test expected.

we now have an explicit feature "nostacksampling" that disables backtraces from
the samplers in both profilers. This effectively cancels "stackwalk", "leaf",
and "javascript" if present.

Differential Revision: https://phabricator.services.mozilla.com/D47731

--HG--
extra : moz-landing-system : lando
2019-10-01 22:49:13 +00:00
Brindusan Cristian 79e9f1a034 Backed out changeset 8989d1a83de8 (bug 1583868) for xpcshell failures at test_ext_geckoProfiler_schema.js. CLOSED TREE 2019-10-01 22:27:11 +03:00
Gerald Squelart fa7dc1a809 Bug 1583868 - New profiler feature "nostacksampling" - r=gregtatum
Previously, the absence of "stackwalk", "leaf", and "javascript" implied that
the test/user didn't want any sampling, but this caused issues in some tests
that enabled "stackwalk" on platforms that didn't support stack-walking, which
ended up suppressing label-only stacks that the test expected.

we now have an explicit feature "nostacksampling" that disables backtraces from
the samplers in both profilers. This effectively cancels "stackwalk", "leaf",
and "javascript" if present.

Differential Revision: https://phabricator.services.mozilla.com/D47731

--HG--
extra : moz-landing-system : lando
2019-10-01 14:00:21 +00:00
Aaron Klotz d1c0cbeb08 Bug 1584571: Fix DLL blocklist entries for oly*.dll to use the correct condition; r=mhowell
I also fixed a couple of typos in comments in the python script that didn't
seem to warrant their own bug.

Differential Revision: https://phabricator.services.mozilla.com/D47455

--HG--
extra : moz-landing-system : lando
2019-09-27 18:07:20 +00:00
kriswright ddd0133b2a Bug 1572238 - Dump assertion stacks using __android_log_print. r=Ehsan
Defines an android-only version of nsTraceRefcnt::WalkTheStack that takes a function callback, which outputs the stack frame buffer to `__android_log_print`. Also uses `__wrap_dladdr` in MozDescribeCodeAddress, which outputs slightly more informative data for the stack trace (instead of instances of '???/??? [???]').

Differential Revision: https://phabricator.services.mozilla.com/D46868

--HG--
extra : moz-landing-system : lando
2019-09-24 20:50:41 +00:00
Tyson Smith d9bce5f0fe Bug 1581656 - Fix default UBSAN_OPTIONS. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D46970

--HG--
extra : moz-landing-system : lando
2019-09-24 17:52:11 +00:00
Bogdan Tara e4913f5aeb Backed out changeset 88334e3a605e (bug 1572238) for causing windows 2012 bustages CLOSED TREE 2019-09-24 21:57:56 +03:00
kriswright 9ac396cf78 Bug 1572238 - Dump assertion stacks using __android_log_print. r=Ehsan
Defines an android-only version of nsTraceRefcnt::WalkTheStack that takes a function callback, which outputs the stack frame buffer to `__android_log_print`. Also uses `__wrap_dladdr` in MozDescribeCodeAddress, which outputs slightly more informative data for the stack trace (instead of instances of '???/??? [???]').

Differential Revision: https://phabricator.services.mozilla.com/D46868

--HG--
extra : moz-landing-system : lando
2019-09-24 12:14:24 +00:00
Aaron Klotz edafb91e6b Bug 1583417: Remove assertion that may fire spuriously due to racy condition; r=mhowell
This assertion could fire if the profiler state changes between beginning and
ending a DLL load. That's less than ideal, so let's just get rid of it.

Differential Revision: https://phabricator.services.mozilla.com/D46897

--HG--
extra : moz-landing-system : lando
2019-09-24 14:53:23 +00:00
Gerald Squelart 16dbd4f365 Bug 1582992 - Removed unneded BlocksRingBuffer's entry destructor, and ProfilerBuffer's JS::AutoSuppressGCAnalysis - r=gregtatum
`BlocksRingBuffer` had an "entry destructor" to make it a more generic
container, and it was useful during early prototyping of the new profiler
storage (so that we could store owning pointers).
But this entry destructor is stored in an `std::function`, which gets marked as
a potential GC caller by the js rooting hazard analyzer; and as this bug found,
it's not obvious where to place `JS::AutoSuppressGCAnalysis`, because profiler
entries (including stacks) could be added on one thread while GC happens
elsewhere, which triggers the embedded `AutoAssertNoGC` check.

Since we don't actually use the entry destructor facility in the profiler, it's
easier to just get rid of it. As a bonus, it's a small optimization.
Tests that were using an entry destructor now use the `State` instead, to verify
that entries are pushed and cleared as expected.

If needed in the future outside of the profiler, `BlocksRingBuffer` could again
include an entry destructor, but it would have to be through templating, so that
the class used in the profiler wouldn't contain an `std::function`.

Differential Revision: https://phabricator.services.mozilla.com/D46738

--HG--
extra : moz-landing-system : lando
2019-09-23 20:50:20 +00:00
Aaron Klotz fccd6eb9ba Bug 1542830: Part 4 - Modify mozglue to use new untrusted modules interfaces; r=mhowell
* At this point our DLL blocking infra is complicated enough that I decided to
  bite the bullet and move all of this code out of `mozglue/build` and into its
  own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
   receives a vector containing all saved records of loaded DLLs that happened
   until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
   processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
  any other non-`firefox` processes that do not have a launcher process
  providing a loader API.
* We add some wide to UTF8 conversion functions.

Depends on D43157

Differential Revision: https://phabricator.services.mozilla.com/D43158

--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : browser/app/winlauncher/freestanding/LoaderAPIInterfaces.h => mozglue/dllservices/LoaderAPIInterfaces.h
rename : browser/app/winlauncher/freestanding/ModuleLoadInfo.h => mozglue/dllservices/ModuleLoadInfo.h
rename : browser/app/winlauncher/NtLoaderAPI.h => mozglue/dllservices/NtLoaderAPI.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
2019-09-23 20:18:41 +00:00
Aaron Klotz c27d5345fc Bug 1542830: Part 1 - Updates to NativeNt.h; r=mhowell
This patch adds the following:

* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
  owns its buffer. The buffers are null-terminated so that they may be used as
  C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
  current thread ID directly from the `TEB` when we do not yet have access to
  kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
  use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
  MFBT `Vector` in code that might not yet have access to Win32 heap functions.

Differential Revision: https://phabricator.services.mozilla.com/D43155

--HG--
extra : moz-landing-system : lando
2019-09-23 20:16:51 +00:00
philipp e517ca80d9 Bug 1576728 - Add crashing dlls from chinese unzipping tools to the blocklist. r=aklotz
--HG--
extra : rebase_source : b8eda4974164e350fa6e1df9bcf79c5f9baa966a
2019-09-23 21:19:02 +03:00
Brindusan Cristian 125c934a04 Backed out 8 changesets (bug 1542830) for causing Nightly bustages. a=backout
Backed out changeset 6fcb417f7ff4 (bug 1542830)
Backed out changeset 1aa253e6604a (bug 1542830)
Backed out changeset 73ec288886cd (bug 1542830)
Backed out changeset 84b903e60dc9 (bug 1542830)
Backed out changeset 8e2da9ff5f5a (bug 1542830)
Backed out changeset c4e547a6a039 (bug 1542830)
Backed out changeset 919f1af7c135 (bug 1542830)
Backed out changeset da7e775c4051 (bug 1542830)

--HG--
rename : browser/app/winlauncher/freestanding/DllBlocklist.cpp => browser/app/winlauncher/DllBlocklistWin.cpp
rename : browser/app/winlauncher/freestanding/DllBlocklist.h => browser/app/winlauncher/DllBlocklistWin.h
rename : mozglue/dllservices/Authenticode.cpp => mozglue/build/Authenticode.cpp
rename : mozglue/dllservices/Authenticode.h => mozglue/build/Authenticode.h
rename : mozglue/misc/WinUtils.h => mozglue/build/MozglueUtils.h
rename : mozglue/dllservices/WindowsDllBlocklist.cpp => mozglue/build/WindowsDllBlocklist.cpp
rename : mozglue/dllservices/WindowsDllBlocklist.h => mozglue/build/WindowsDllBlocklist.h
rename : mozglue/dllservices/WindowsDllBlocklistCommon.h => mozglue/build/WindowsDllBlocklistCommon.h
rename : mozglue/dllservices/WindowsDllBlocklistDefs.in => mozglue/build/WindowsDllBlocklistDefs.in
rename : mozglue/dllservices/WindowsDllServices.h => mozglue/build/WindowsDllServices.h
rename : mozglue/dllservices/gen_dll_blocklist_defs.py => mozglue/build/gen_dll_blocklist_defs.py
rename : toolkit/components/telemetry/docs/data/third-party-modules-ping.rst => toolkit/components/telemetry/docs/data/untrusted-modules-ping.rst
rename : toolkit/components/telemetry/tests/unit/test_ThirdPartyModulesPing.js => toolkit/components/telemetry/tests/unit/test_UntrustedModulesPing.js
rename : toolkit/xre/UntrustedModulesProcessor.cpp => toolkit/xre/ModuleEvaluator_windows.cpp
rename : toolkit/xre/UntrustedModulesProcessor.h => toolkit/xre/ModuleEvaluator_windows.h
rename : toolkit/xre/ModuleVersionInfo.cpp => toolkit/xre/ModuleVersionInfo_windows.cpp
rename : toolkit/xre/ModuleVersionInfo.h => toolkit/xre/ModuleVersionInfo_windows.h
2019-09-22 00:01:01 +03:00
Aaron Klotz 8d1be668f6 Bug 1542830: Part 4 - Modify mozglue to use new untrusted modules interfaces; r=mhowell
* At this point our DLL blocking infra is complicated enough that I decided to
  bite the bullet and move all of this code out of `mozglue/build` and into its
  own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
   receives a vector containing all saved records of loaded DLLs that happened
   until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
   processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
  any other non-`firefox` processes that do not have a launcher process
  providing a loader API.
* We add some wide to UTF8 conversion functions.

Differential Revision: https://phabricator.services.mozilla.com/D43158

--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
2019-09-20 19:43:33 +00:00
Aaron Klotz f5081e7b43 Bug 1542830: Part 1 - Updates to NativeNt.h; r=mhowell
This patch adds the following:

* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
  owns its buffer. The buffers are null-terminated so that they may be used as
  C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
  current thread ID directly from the `TEB` when we do not yet have access to
  kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
  use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
  MFBT `Vector` in code that might not yet have access to Win32 heap functions.

Differential Revision: https://phabricator.services.mozilla.com/D43155

--HG--
extra : moz-landing-system : lando
2019-09-20 19:42:40 +00:00
Ciure Andrei 7db4ad5d80 Backed out 8 changesets (bug 1542830) for causing spidermonkey bustages CLOSED TREE
Backed out changeset b9f7fc8d0172 (bug 1542830)
Backed out changeset b85f58fd5bbd (bug 1542830)
Backed out changeset 0a9169ab2623 (bug 1542830)
Backed out changeset 39a7c05d54ef (bug 1542830)
Backed out changeset 7a3cbd2f59f0 (bug 1542830)
Backed out changeset 268530552281 (bug 1542830)
Backed out changeset df658ffb8599 (bug 1542830)
Backed out changeset e7bef6486a38 (bug 1542830)

--HG--
rename : browser/app/winlauncher/freestanding/DllBlocklist.cpp => browser/app/winlauncher/DllBlocklistWin.cpp
rename : browser/app/winlauncher/freestanding/DllBlocklist.h => browser/app/winlauncher/DllBlocklistWin.h
rename : mozglue/dllservices/Authenticode.cpp => mozglue/build/Authenticode.cpp
rename : mozglue/dllservices/Authenticode.h => mozglue/build/Authenticode.h
rename : mozglue/misc/WinUtils.h => mozglue/build/MozglueUtils.h
rename : mozglue/dllservices/WindowsDllBlocklist.cpp => mozglue/build/WindowsDllBlocklist.cpp
rename : mozglue/dllservices/WindowsDllBlocklist.h => mozglue/build/WindowsDllBlocklist.h
rename : mozglue/dllservices/WindowsDllBlocklistCommon.h => mozglue/build/WindowsDllBlocklistCommon.h
rename : mozglue/dllservices/WindowsDllBlocklistDefs.in => mozglue/build/WindowsDllBlocklistDefs.in
rename : mozglue/dllservices/WindowsDllServices.h => mozglue/build/WindowsDllServices.h
rename : mozglue/dllservices/gen_dll_blocklist_defs.py => mozglue/build/gen_dll_blocklist_defs.py
rename : toolkit/components/telemetry/docs/data/third-party-modules-ping.rst => toolkit/components/telemetry/docs/data/untrusted-modules-ping.rst
rename : toolkit/components/telemetry/tests/unit/test_ThirdPartyModulesPing.js => toolkit/components/telemetry/tests/unit/test_UntrustedModulesPing.js
rename : toolkit/xre/UntrustedModulesProcessor.cpp => toolkit/xre/ModuleEvaluator_windows.cpp
rename : toolkit/xre/UntrustedModulesProcessor.h => toolkit/xre/ModuleEvaluator_windows.h
rename : toolkit/xre/ModuleVersionInfo.cpp => toolkit/xre/ModuleVersionInfo_windows.cpp
rename : toolkit/xre/ModuleVersionInfo.h => toolkit/xre/ModuleVersionInfo_windows.h
2019-09-20 21:21:39 +03:00
Aaron Klotz 12843d81e9 Bug 1542830: Part 4 - Modify mozglue to use new untrusted modules interfaces; r=mhowell
* At this point our DLL blocking infra is complicated enough that I decided to
  bite the bullet and move all of this code out of `mozglue/build` and into its
  own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
   receives a vector containing all saved records of loaded DLLs that happened
   until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
   processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
  any other non-`firefox` processes that do not have a launcher process
  providing a loader API.
* We add some wide to UTF8 conversion functions.

Differential Revision: https://phabricator.services.mozilla.com/D43158

--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
2019-09-20 17:00:14 +00:00
Aaron Klotz 57df2b6a01 Bug 1542830: Part 1 - Updates to NativeNt.h; r=mhowell
This patch adds the following:

* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
  owns its buffer. The buffers are null-terminated so that they may be used as
  C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
  current thread ID directly from the `TEB` when we do not yet have access to
  kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
  use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
  MFBT `Vector` in code that might not yet have access to Win32 heap functions.

Differential Revision: https://phabricator.services.mozilla.com/D43155

--HG--
extra : moz-landing-system : lando
2019-09-20 16:59:28 +00:00
Aaron Klotz 38e11b4896 Bug 1582776: Change cross-process stub to not store remote stub pointer in local mOrigFunc; r=handyman
This is showing up with hooks that are set both by the launcher process and by
the browser process when starting early DLL blocklist init on content processes:

* The browser's copy of mOrigFunc was set by the launcher process.
* The browser is setting a hook in the new child process, which writes to the
  child's mOrigFunc.
* But FuncHookCrossProcess also writes that pointer to the browser's mOrigFunc,
  thus corrupting the browser process's copy of the pointer.

For in-process hooks, we want to immediately write the stub pointer to its final
location; this is not an issue for cross-process hooks since the child process
is suspended when we do this and the parent process can't call the stub; there
is no possibility of a race.

Differential Revision: https://phabricator.services.mozilla.com/D46615

--HG--
extra : moz-landing-system : lando
2019-09-20 16:55:27 +00:00
Andreea Pavel 31d9596a28 Backed out 8 changesets (bug 1542830) for gecko decision failure on a CLOSED TREE
DONTBUILD

Backed out changeset 4f72161be496 (bug 1542830)
Backed out changeset 7ee12138946d (bug 1542830)
Backed out changeset 9e90ee4981c6 (bug 1542830)
Backed out changeset 363039c98534 (bug 1542830)
Backed out changeset 70ffff30551a (bug 1542830)
Backed out changeset c25b7bf354d9 (bug 1542830)
Backed out changeset d156dc595b69 (bug 1542830)
Backed out changeset 9f5dbd1b2959 (bug 1542830)

--HG--
rename : browser/app/winlauncher/freestanding/DllBlocklist.cpp => browser/app/winlauncher/DllBlocklistWin.cpp
rename : browser/app/winlauncher/freestanding/DllBlocklist.h => browser/app/winlauncher/DllBlocklistWin.h
rename : mozglue/dllservices/Authenticode.cpp => mozglue/build/Authenticode.cpp
rename : mozglue/dllservices/Authenticode.h => mozglue/build/Authenticode.h
rename : mozglue/misc/WinUtils.h => mozglue/build/MozglueUtils.h
rename : mozglue/dllservices/WindowsDllBlocklist.cpp => mozglue/build/WindowsDllBlocklist.cpp
rename : mozglue/dllservices/WindowsDllBlocklist.h => mozglue/build/WindowsDllBlocklist.h
rename : mozglue/dllservices/WindowsDllBlocklistCommon.h => mozglue/build/WindowsDllBlocklistCommon.h
rename : mozglue/dllservices/WindowsDllBlocklistDefs.in => mozglue/build/WindowsDllBlocklistDefs.in
rename : mozglue/dllservices/WindowsDllServices.h => mozglue/build/WindowsDllServices.h
rename : mozglue/dllservices/gen_dll_blocklist_defs.py => mozglue/build/gen_dll_blocklist_defs.py
rename : toolkit/components/telemetry/docs/data/third-party-modules-ping.rst => toolkit/components/telemetry/docs/data/untrusted-modules-ping.rst
rename : toolkit/components/telemetry/tests/unit/test_ThirdPartyModulesPing.js => toolkit/components/telemetry/tests/unit/test_UntrustedModulesPing.js
rename : toolkit/xre/UntrustedModulesProcessor.cpp => toolkit/xre/ModuleEvaluator_windows.cpp
rename : toolkit/xre/UntrustedModulesProcessor.h => toolkit/xre/ModuleEvaluator_windows.h
rename : toolkit/xre/ModuleVersionInfo.cpp => toolkit/xre/ModuleVersionInfo_windows.cpp
rename : toolkit/xre/ModuleVersionInfo.h => toolkit/xre/ModuleVersionInfo_windows.h
2019-09-20 19:49:27 +03:00
Aaron Klotz 6c7b43b9c8 Bug 1542830: Part 4 - Modify mozglue to use new untrusted modules interfaces; r=mhowell
* At this point our DLL blocking infra is complicated enough that I decided to
  bite the bullet and move all of this code out of `mozglue/build` and into its
  own subdirectory, `mozglue/dllservices`.
* We delete the original `UntrustedDllsHandler` code which is now obsolete.
* We implement mozglue's `LoaderObserver`:
** When this observer registers itself with the launcher process API, it
   receives a vector containing all saved records of loaded DLLs that happened
   until that moment.
** This code handles profiler labels and stackwalking suppression.
** Once a load has completed, we either pass the load on to XUL for further
   processing, or save it for later if XUL is not initialized yet.
* mozglue has its own `ModuleLoadFrame` implementation for the legacy blocklist.
* `DllServicesBase` is updated to support the new interfaces.
* We implement `FallbackLoaderAPI` for `plugin-container`, `xpcshell`, and
  any other non-`firefox` processes that do not have a launcher process
  providing a loader API.
* We add some wide to UTF8 conversion functions.

Differential Revision: https://phabricator.services.mozilla.com/D43158

--HG--
rename : mozglue/build/Authenticode.cpp => mozglue/dllservices/Authenticode.cpp
rename : mozglue/build/Authenticode.h => mozglue/dllservices/Authenticode.h
rename : mozglue/build/WindowsDllBlocklist.cpp => mozglue/dllservices/WindowsDllBlocklist.cpp
rename : mozglue/build/WindowsDllBlocklist.h => mozglue/dllservices/WindowsDllBlocklist.h
rename : mozglue/build/WindowsDllBlocklistCommon.h => mozglue/dllservices/WindowsDllBlocklistCommon.h
rename : mozglue/build/WindowsDllBlocklistDefs.in => mozglue/dllservices/WindowsDllBlocklistDefs.in
rename : mozglue/build/WindowsDllServices.h => mozglue/dllservices/WindowsDllServices.h
rename : mozglue/build/gen_dll_blocklist_defs.py => mozglue/dllservices/gen_dll_blocklist_defs.py
rename : mozglue/build/moz.build => mozglue/dllservices/moz.build
rename : mozglue/build/MozglueUtils.h => mozglue/misc/WinUtils.h
extra : moz-landing-system : lando
2019-09-20 00:09:41 +00:00
Aaron Klotz f3b16f06a4 Bug 1542830: Part 1 - Updates to NativeNt.h; r=mhowell
This patch adds the following:

* The `AllocatedUnicodeString` class which encapsulates a `UNICODE_STRING` and
  owns its buffer. The buffers are null-terminated so that they may be used as
  C-style strings without modification.
** We do not allow either creation or copying within XUL
* `RtlGetCurrentThreadId` and a test to validate it, so that we may obtain the
  current thread ID directly from the `TEB` when we do not yet have access to
  kernel32.
* An implementation of `SRWLock` that uses Rtl instead of Win32 so that we may
  use them before we have access to Win32 DLLs.
* A memory allocation policy that uses Rtl heap functions so that we may use
  MFBT `Vector` in code that might not yet have access to Win32 heap functions.

Differential Revision: https://phabricator.services.mozilla.com/D43155

--HG--
extra : moz-landing-system : lando
2019-09-20 00:09:33 +00:00
Gerald Squelart 1549c7e36e Bug 1582357 - Allow profiler entries up to the size of the buffer - r=gregtatum
In some situations, entries may in fact take more than half the buffer size
(e.g., when duplicating a stack into a small temporary buffer).
So we now allow blocks to take the full buffer size -- but not more, as they
would start overwriting themselves!

Differential Revision: https://phabricator.services.mozilla.com/D46453

--HG--
extra : moz-landing-system : lando
2019-09-19 13:40:01 +00:00
Gerald Squelart 64ec6b96a0 Bug 1581049 - BlocksRingBuffer (de)serializer uses optimized inter-ModuloBuffer copier - r=gregtatum
Instead of copying `BlocksRingBuffer` data byte-by-byte (using iterators byte
dereferencers), we can now use `ModuloBuffer::Iterator::ReadInto(Iterator&)` to
copy them using a small number of `memcpy`s.

Differential Revision: https://phabricator.services.mozilla.com/D45839

--HG--
extra : moz-landing-system : lando
2019-09-18 01:22:42 +00:00
Gerald Squelart 229768ce0a Bug 1581049 - ModuloBuffer::Iterator::ReadInto(ModuloBuffer::Iterator&) - r=gregtatum
Some objects are copied byte-by-byte to/from `ModuloBuffer`s.
E.g., serialized `BlocksRingBuffer`s, or duplicate stacks. (And more to come.)

`Iterator::ReadInto(Iterator&)` optimizes these copies by using the minimum
number of `memcpy`s possible.

Differential Revision: https://phabricator.services.mozilla.com/D45838

--HG--
extra : moz-landing-system : lando
2019-09-18 01:22:30 +00:00
Gerald Squelart 738f482766 Bug 1578329 - profiler_can_accept_markers(), don't store markers when paused - r=gregtatum
profiler_can_accept_markers() is a fast and racy check that markers would
currently be stored, it should be used around potentially-expensive calls to
add markers.

And now markers are no longer stored when the profiler is paused. (Note that the
profiler is paused when a profile is being stored, this will help make this
operation faster.)

Differential Revision: https://phabricator.services.mozilla.com/D44434

--HG--
extra : moz-landing-system : lando
2019-09-18 01:21:50 +00:00
Gerald Squelart a8b053c3b1 Bug 1578327 - Discard old data just before streaming - r=gregtatum
Since all profiler data is now stored inside ProfileBuffer, there is no real
need to continuously discard old data during sampling (this was particularly
useful to reclaim memory taken by old markers&payloads).

Instead, we can now just discard the old data once, just before starting to
stream it to JSON.

Differential Revision: https://phabricator.services.mozilla.com/D44433

--HG--
extra : moz-landing-system : lando
2019-09-18 01:21:32 +00:00
Gerald Squelart 58ac06a047 Bug 1576555 - Remove unneeded ProfilerMarker - r=gregtatum
Now that what was in ProfilerMarker is stored directly in `BlocksRingBuffer`,
there is no need for this class anymore!
This also removes all the pointer management around it (when added to a TLS
list, moved during sampling, deleted when expired).

Differential Revision: https://phabricator.services.mozilla.com/D43431

--HG--
extra : moz-landing-system : lando
2019-09-18 01:21:20 +00:00
Gerald Squelart 02d2886883 Bug 1576555 - Don't store temporary ProfilerMarkerPayloads in UniquePtrs - r=gregtatum
Since payloads are not kept alive long after they have been serialized, we can
just create them on the stack and pass a reference to their base (or pointer,
`nullptr` representing "no payloads") to `profiler_add_marker()`.

Differential Revision: https://phabricator.services.mozilla.com/D43430

--HG--
extra : moz-landing-system : lando
2019-09-18 01:21:02 +00:00
Gerald Squelart 666a5caa9d Bug 1576555 - Store markers directly into BlocksRingBuffer - r=gregtatum
Markers and their payloads can now be serialized straight into the profiler's
`BlocksRingBuffer`, which is now thread-safe to allow such concurrent accesses
(even when gPSMutex is not locked).
This already saves us from having to allocate a `PayloadMarker` on the heap, and
from having to manage it in different lists.

The now-thread-safe `BlocksRingBuffer` in `CorePS` cannot be used inside the
critical section of `SamplerThread::Run`, because any mutex function could hang
because of the suspended thread (even though they functionally don't appear to
interact). So the sampler now uses a local `BlocksRingBuffer` without mutex.
As a bonus, the separate buffer helps reduce the number of concurrent locking
operations needed when capturing the stack.

Differential Revision: https://phabricator.services.mozilla.com/D43429

--HG--
extra : moz-landing-system : lando
2019-09-18 01:20:45 +00:00
Gerald Squelart 06043a38e3 Bug 1576555 - BlocksRingBuffer::AppendContents(const BlocksRingBuffer&) - r=gregtatum
Copy the full contents of BlocksRingBuffer into another one.
This is mainly useful to use a temporary buffer to store some data without
contentions, and then integrate the temporary buffer into the main one.

Differential Revision: https://phabricator.services.mozilla.com/D45306

--HG--
extra : moz-landing-system : lando
2019-09-18 01:20:28 +00:00
Gerald Squelart fc93e3d9da Bug 1575448 - De/serialize ProfilerMarkerPayload derived objects - r=gregtatum
Payloads will serialize themselves into a `BlocksRingBuffer` entry when first
captured.
Later they will be deserialized, to stream JSON for the output profile.

Differential Revision: https://phabricator.services.mozilla.com/D43428

--HG--
extra : moz-landing-system : lando
2019-09-18 01:20:10 +00:00
Gerald Squelart 0ffe75f626 Bug 1575448 - ProfilerMarkerPayload::CommonProps - r=gregtatum
The common data members stored in the ProfilerMarkerPayload base class can be
gathered into a struct, which will make it easier to pass around, especially
when a derived object is constructed with these common properties.

Differential Revision: https://phabricator.services.mozilla.com/D43427

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:53 +00:00
Gerald Squelart ac26c5d9f5 Bug 1576554 - {S,Des}erializer<ProfilerBacktrace> - r=gregtatum
Markers may contain backtraces, so we need to be able to de/serialize them.
This involves de/serializing the underyling `BlocksRingBuffer`.

Differential Revision: https://phabricator.services.mozilla.com/D43426

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:40 +00:00
Gerald Squelart 0de7cb4c2e Bug 1576554 - Test baseprofiler::profiler_get_backtrace - r=gregtatum
Add basic testing of `profiler_get_backtrace` before working on serializing it.

Differential Revision: https://phabricator.services.mozilla.com/D43424

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:23 +00:00
Gerald Squelart 9f7c402a79 Bug 1576551 - Store BlocksRingBuffer outside of ProfileBuffer - r=gregtatum
The main `BlocksRingBuffer`s will be stored in `CorePS` (outside of
`ProfileBuffer`s), as we need to be able to safely access the underlying buffers
when profilers are not enabled.

Also `ProfilerBacktrace` will own the `BlocksRingBuffer` that its captured
`ProfileBuffer` uses.

Taking this opportunity to rename the different `mBuffer`s to more useful names.

Differential Revision: https://phabricator.services.mozilla.com/D43422

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:21 +00:00
Gerald Squelart 74301e6c5c Bug 1576551 - Assume capacity is in 8-byte entries - r=gregtatum
Now that we are using a byte-oriented `BlocksRingBuffer` instead of an array of
9-byte `ProfileBufferEntry`'s, internally the profiler sets the buffer size in
bytes. However all external users (popup, tests, etc.) still assume that the
requested capacity is in entries!

To limit the amount of changes, we will keep assuming externally-visible
capacities are in entries, and convert them to bytes.
Even though entries used to be 9 bytes each, and `BlocksRingBuffer` adds 1 byte
for the entry size, some entries actually need less space (e.g., 32-bit numbers
now take 6 bytes instead of 9), so converting to less than 9 bytes per entry is
acceptable.
We are settling on 8 bytes per entry: It's close to 9, and it's a power of two;
since the effective number of entries was a power of two, and `BlocksRingBuffer`
also uses a power of two size in bytes, this convertion keeps sizes in powers of
two.

Differential Revision: https://phabricator.services.mozilla.com/D44953

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:14 +00:00
Gerald Squelart 6666138a83 Bug 1576551 - Use BlocksRingBuffer in ProfileBuffer - r=gregtatum
This just replaces `ProfileBuffer`'s self-managed circular buffer with a
`BlocksRingBuffer`.

That `BlocksRingBuffer` does not need its own mutex (yet), because all uses go
through gPSMutex-guarded code.

`ProfileBuffer` now also pre-allocates a small buffer for use in
`DuplicateLastSample()`, this avoids multiple mallocs at each sleeping thread
stack duplication.

Note: Internal "magic" sizes have been multiplied by 8 (and tweaked upwards, to
handle bigger stacks), because they originally were the number of 9-byte
entries, but now it's the buffer size in bytes. (And entries can now be smaller
than 9 bytes, so overall the capacity in entries should be similar or better.)
However, external calls still think they are giving a number of "entries", this
will be handled in the next patch.

Differential Revision: https://phabricator.services.mozilla.com/D43421

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:12 +00:00
Gerald Squelart 82b9f4f30f Bug 1576551 - AUTO_PROFILER_STATS(locked_profiler_stream_json_for_this_process) - r=gregtatum
Add some stats (off by default) around streaming JSON, as the following patches
may affect it.

Differential Revision: https://phabricator.services.mozilla.com/D44952

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:10 +00:00
Gerald Squelart c90cc7f5b7 Bug 1580091 - Use BaseProfilerMaybeMutex in BlocksRingBuffer - r=gregtatum
In some situations we will *need* to use a `BlocksRingBuffer` that absolutely
does not use a mutex -- In particular in the critical section of
`SamplerThread::Run()`, when a thread is suspended, because any action on any
mutex (even a private one that no-one else interacts with) can result in a hang.

As a bonus, `BlocksRingBuffer`s that are known not to be used in multi-thread
situations (e.g., backtraces, extracted stacks for duplication, etc.) will waste
less resources trying to lock/unlock their mutex.

Differential Revision: https://phabricator.services.mozilla.com/D45305

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:08 +00:00
Gerald Squelart cac9b1645b Bug 1580091 - BaseProfilerMaybeMutex and BaseProfilerMaybeAutoLock - r=gregtatum
`BaseProfilerMaybeMutex` wraps a `BaseProfilerMutex` inside a `Maybe`.
The decision to use a mutex or not is set at construction time.
If there is no mutex, all operations do nothing (at the small cost of checking
if the mutex is present.)

`BaseProfilerMaybeAutoLock` is the recommented RAII object to lock and
automatically unlock a `BaseProfilerMaybeMutex` until the end of a block scope.

Differential Revision: https://phabricator.services.mozilla.com/D45304

--HG--
extra : moz-landing-system : lando
2019-09-18 01:19:06 +00:00