While all toolkit and js-based projects make use of mfbt, some others,
like tools/crashreporter and tools/update-packaging, don't.
So instead of including mfbt from the top-level directory, include it
from the relevant project top-level mozbuilds.
This allows to remove the dependency on mfbt files in the hash for the
minidump-stackwalk and mar-tools toolchains.
Differential Revision: https://phabricator.services.mozilla.com/D98378
TimeStamps in markers must now be streamed through `SpliceableJSONWriter::TimeProperty(name, timestamp)`.
This is consistent with all other JSON-writing functions being in `SpliceableJSONWriter` (and base class `JSONWriter`).
Depends on D97556
Differential Revision: https://phabricator.services.mozilla.com/D97557
If a changeset has multiple parents, and those parents get pruned away
such that the changeset ends up with identical parents, we now collapse
those parents into a single parent. This avoids unnecessary recursion
and repetition of work. Generally this is only a problem when processing
a large number of changesets, as the recursion will be exponential with
the depth of the tree, and small numbers of changesets generally have
shallow trees.
Differential Revision: https://phabricator.services.mozilla.com/D97683
Expand `profiler_is_locked_on_current_thread()` to also return true if the ProfileBufferGlobalController mutex is locked on the current thread.
This will prevent some profiler-re-entrant operations (like native allocation markers) from running.
In particular, this removes potential deadlocks similar to the one found in bug 1671403:
- SamplerThread: In the sampling loop, lock the main profiler mutex, run a local update, which attempts to lock the ProfileBufferGlobalController mutex.
- ProfilerChild thread: While processing an IPC message with an update, lock the ProfileBufferGlobalController mutex, then resolve the update, which records a native allocation with a backtrace that attempts to lock the main profiler mutex.
With this patch, the native allocation won't record a marker while the ProfileBufferGlobalController mutex is locked.
Differential Revision: https://phabricator.services.mozilla.com/D96970
Expand `profiler_is_locked_on_current_thread()` to also return true if the ProfilerChild mutex is locked on the current thread.
This will prevent some profiler-re-entrant operations (like native allocation markers) from running.
In particular, this removes the potential deadlock found in bug 1671403:
- SamplerThread: In the sampling loop, lock the main profiler mutex, run a ProfilerChild update, which attempts to lock the ProfilerChild mutex.
- ProfilerChild thread: While processing an IPC message with an update, lock the ProfilerChild mutex, then resolve the update, which records a native allocation with a backtrace that attempts to lock the main profiler mutex.
With this patch, the native allocation won't record a marker while the ProfilerChild mutex is locked.
Differential Revision: https://phabricator.services.mozilla.com/D96969
Because it's easy to send markers to the main thread, we don't need to store the main thread id in memory_hooks functions and objects.
Differential Revision: https://phabricator.services.mozilla.com/D96049
In this case, the same marker type "CONTENT_FULL_PAINT_TIME" is used in separate places, so it makes sense to put the marker type definition in a common location.
Differential Revision: https://phabricator.services.mozilla.com/D96042
Because it's easy to send markers to the main thread, we don't need to store the main thread id in memory_hooks functions and objects.
Differential Revision: https://phabricator.services.mozilla.com/D96049
In this case, the same marker type "CONTENT_FULL_PAINT_TIME" is used in separate places, so it makes sense to put the marker type definition in a common location.
Differential Revision: https://phabricator.services.mozilla.com/D96042
Because it's easy to send markers to the main thread, we don't need to store the main thread id in memory_hooks functions and objects.
Differential Revision: https://phabricator.services.mozilla.com/D96049
In this case, the same marker type "CONTENT_FULL_PAINT_TIME" is used in separate places, so it makes sense to put the marker type definition in a common location.
Differential Revision: https://phabricator.services.mozilla.com/D96042
The DevTools mochitests on Fission platforms have been promoted to tier 1.
This changeset updates our try presets to increase our coverage of fission platforms.
Differential Revision: https://phabricator.services.mozilla.com/D96407
The fzf-bin repository is now archived and no longer receives new releases.
Releases are published directly to the fzf repository instead.
Differential Revision: https://phabricator.services.mozilla.com/D96238
This adds back part of the code that was removed in bug 1339182,
reformats it with black, adjusts it to make flake8 happy, and converts
it to python 3.
This also adjusts the script after bug 1534003, which changed the
about:buildconfig page title.
Differential Revision: https://phabricator.services.mozilla.com/D95977
Before, on Windows, this resulted in installing the package in the parent environment (not the `virtualenv`). We fix this by passing down the `virtualenv_manager` so linters can install packages they need using that object's helper methods.
Differential Revision: https://phabricator.services.mozilla.com/D95738
This patch sets up unit tests for the perfdocs linter and adds a few simple tests for it which will be expanded on in future patches.
Differential Revision: https://phabricator.services.mozilla.com/D75480
Unique strings are used to encode all markers' 'name' field, SpliceableJSONWriter::UniqueStringElement can be used for that (instead of a caller-provided callback, which was necessary before UniqueJSONStrings was de-duplicated).
Differential Revision: https://phabricator.services.mozilla.com/D95513
Some markers (e.g., Screenshot) use unique strings in their data.
The UniqueJSONStrings used during streaming is attached to the SpliceableJSONWriter, and StreamJSONMarkerData can use pass-through functions UniqueStringProperty() and UniqueStringElement().
Differential Revision: https://phabricator.services.mozilla.com/D95512
Some markers (e.g., GC major/minor/slice) need to splice JSON strings in their data.
So now, instead of JSONWriter, StreamJSONMarkerData functions will be given a mozilla::baseprofiler::SpliceableJSONWriter.
Differential Revision: https://phabricator.services.mozilla.com/D95511
This adds symbol servers from Intel, AMD and NVidia. Unfortunately not all
symbols are available there, we still have to manually download those that
aren't.
Differential Revision: https://phabricator.services.mozilla.com/D95854
This patch changes a few different things:
- The Docker image used for both scrapers are now based off of our standard
Debian 10 image instead of an old Ubuntu one
- Both images were changed to make it easy to work with them when an
interactive task is used
- The python packages used by the scripts had their versions pinned and
cryptographic hashes have been recorded so that they can be verified before
running the task
- The dump_syms version used on Windows was bumped to include fixes that allow
us to scrape public symbols from DLLs and EXEs that have no PDB file available
- The maximum open file limit of the Window scraper was bumped up to cope with
runs where we gather plenty of symbols in one go
- Similarly the maximum number of symultaneous connections to a single symbol
server has been limited to 4 to avoid being throttled
- The macOS Python tools were modified to work with Python 3
Differential Revision: https://phabricator.services.mozilla.com/D95853
PathUtils is a path manipulation component to IOUtils, which is based on
simplified file I/O. This work is part of the larger goal of removing
osfile.jsm et al., ospath.jsm et al., and the entire OS.* namespace, especially
from the startup path.
No equivalent was provided for OS.Path.fromFileURI because it is unused.
Differential Revision: https://phabricator.services.mozilla.com/D95105
This patch adds the basic docs for Talos using the PerfDocs system. It
includes a high-level overview of what Talos is, and how to interact with
it.
Differential Revision: https://phabricator.services.mozilla.com/D95604
This patch upgrade browsertime to v10, enables webdriver-based navigation in browsertime tests, and also disables the Firefox WindowRecorder for a fairer comparison between Chrome and Firefox.
Differential Revision: https://phabricator.services.mozilla.com/D95425
For consistency with `JSONWriter` (which UniqueJSONStrings' functions use), and for added safety and some efficiency, UniqueJSONStrings now takes `Span<const char`> arguments instead of raw pointers to null-terminated strings.
Differential Revision: https://phabricator.services.mozilla.com/D95114
Document the class and methods.
`GetOrAddIndex` is only used internally, so it can be private.
`SpliceStringTableElements` can now only work on rvalue UniqueJSONStrings, this emphasizes that it shouldn't be used anymore after this call.
Differential Revision: https://phabricator.services.mozilla.com/D95113
There is an unfortunate-but-necessary ordering issue that needs to be fixed in
the api.txt file which is causing a lot of one time updates. The ordering of
annotations was not stable (and now it is).
Differential Revision: https://phabricator.services.mozilla.com/D95382
No significant changes here, mostly clarifying the wording in some places and
making sure it's accurate. Also added a section on the access group since
people looking at this tool may not know about those pieces.
Depends on D95394
Differential Revision: https://phabricator.services.mozilla.com/D95395
We can just use the author of the commit as the committer. I don't know why
I used anything different in the first place, but I'd like to remove references
to the graphics-team email address so this seems like a good time to fix it.
Also remove another TODO that I'm probably never going to do.
Differential Revision: https://phabricator.services.mozilla.com/D95394
The logic the `black` and `flake8` linters were using to find the location of the appropriate binaries for linting was wrong in certain cases given how `mach lint` uses subprocesses to batch work. Instead, we allow the option to override the old janky behavior with a known-good path.
Differential Revision: https://phabricator.services.mozilla.com/D95396
The definition of `patch_main()` has behavior that kicks in only on Windows and for Python 2. Unfortunately, not all of our `mach` commands have been migrated to Python 3, so this still matters.
Bug 1654103 replaced the single-quoted strings in this function with double-quoted strings. This should be fine, except that we call into `multiprocessing.forking.main()` with some monkey-patching that is meant to fix a Windows-specific bug (see bug 1316140). We don't do any clever serialization or anything here and we end up just passing that source to `multiprocessing.forking.main()` which aggregates that source code [dumbly](https://github.com/python/cpython/blob/2.7/Lib/multiprocessing/forking.py#L259), wrapping everything in double-quotes again and passing it to `_subprocess.CreateProcess()`, which ends up failing if the source contains strings formatted with double quotes.
We could revert bug 1654103 and exempt this file from linting, but that is overkill given that this file otherwise contains useful stuff. Instead we move everything to another file, exempt that file from linting, and update `util.py` accordingly.
Differential Revision: https://phabricator.services.mozilla.com/D94909
Instead of a repeating timeout of only twice the parent's serialization time + 1s, use that double parent time and multiply it by the number of children, and add the number of seconds from the about:config preference "devtools.performance.recording.child.timeout_s" (still 1s by default).
Differential Revision: https://phabricator.services.mozilla.com/D94955
Simon Giesecke
Gregory Mierzwinski
Mike Hommey
Christian Holler
Andrew Osmond
Gerald Squelart
Zac Svoboda
Sylvestre Ledru
Renovate Bot
championshuttler
smolnar
Greg Tatum
Florian Quèze
Kartikaya Gupta
Johan Lorenzo
Tim Nguyen
Markus Stange
Razvan Maries
Mihai Alexandru Michis
Andi-Bogdan Postelnicu
Csoregi Natalia
Ben Hearsum
alwu
Brindusan Cristian
Kris Maglione
Julian Descottes
Barret Rennie
Andrew Halberstadt
Ricky Stewart
alexandru.ionescu
Gabriele Svelto
Tom Ritter
Greg Tatum
Dorel Luca
Agi Sferro
Kartikaya Gupta