Profile buffer updates could be triggered from a number of locations, including scopes where profiler and/or system locks are held, making deadlocks possible if profiler and/or system function are called.
So instead of dispatching updates to the main thread (which may use OS task queue functions), we fold updates into a static storage. The profiler sampler loop regularly triggers processing of these pending updates.
Differential Revision: https://phabricator.services.mozilla.com/D83747
Profile buffer updates could be triggered from a number of locations, including scopes where profiler and/or system locks are held, making deadlocks possible if profiler and/or system function are called.
So instead of dispatching updates to the main thread (which may use OS task queue functions), we fold updates into a static storage. When child updates arrive, we can safely handle parent updates as well.
Child updates are assumed to arrive regularly enough to properly handle parent updates.
But in the worst case, if no updates came from children, it should mean nothing is happening, and in any case we wouldn't know how the memory is used elsewhere. Note that the chunk manager still enforces local limits automatically, so memory usage would still get limited.
Differential Revision: https://phabricator.services.mozilla.com/D83746
Knowing the time it takes for the parent process to gather its profile, we expect sub-processes not to take longer than twice that time (plus a bit more).
Each time we receive a profile, the timer is restarted* to allow more time for pending processes.
If the most current timer fires, we assume that pending processes may be frozen and unable to ever respond, so we use all profiles gathered so far.
(There is currently no indication to the user that processes are missing, this should be added in the future.)
* Tech detail: We just cancel the current timer and forget about it, then start a new one and keep track of it. When a timer fires, we can compare its address to the address of the last (most current) timer, and only that one can force the end of profile-gathering.
Differential Revision: https://phabricator.services.mozilla.com/D83626
Try syntax users wishing to preserve their default selector can modify
~/.mozbuild/machrc (create it if it doesn't exist), and add:
[try]
default = syntax
Depends on D82983
Differential Revision: https://phabricator.services.mozilla.com/D82985
Since partials have started verifying signatures, the partial task has been failing in
`mach try scriptworker`. Since we are not concerned with the partial task itself,
split the tasks into two groups, so that it does not need to run.
Differential Revision: https://phabricator.services.mozilla.com/D83370
It is possible that some threads fail/forget to unregister themselves, in which case a registered thread id could get recycled by a later thread, which was not allowed before this patch.
Note: The thread name cannot currently be changed. We record a special marker with the new name, so the frontend could process it to split the thread track at that point.
We also record a marker when profiler_unregister_thread is called from an already-unregistered thread, this could help find reg/unreg mismatches or nesting in Firefox threads.
Differential Revision: https://phabricator.services.mozilla.com/D83293
On Linux (including Android), it was assumed that a registered thread could always be suspended through `tgkill`.
However in some cases a thread may not be correctly unregistered, in which case this would trigger `MOZ_ASSERT` or wait forever in the following loop.
This will especially be needed when `profiler_{,un}register_thread()` are made less strict in the following patch.
Windows and Mac already handle suspension failures.
Differential Revision: https://phabricator.services.mozilla.com/D83292
Add an action that will trigger a task that runs
`mach release push-scriptworker-canary`
to test a new scriptworker deployment.
Differential Revision: https://phabricator.services.mozilla.com/D82821
Adds some additional assertions against what would be ran with |mach try auto|.
This also fixes a legit bug that the 'test_important_manifests_only' test
caught.
Differential Revision: https://phabricator.services.mozilla.com/D81404
Here "update programs" refers to the various standalone binaries we
produce in order to update Firefox. There's not strong conceptual
coherence between them; rather, it's the pieces that the
Install/Update team generally work on.
To use this build project, prepare a minimal mozconfig with
`--enable-project=tools/update-programs`. Depending on the mozconfig
options and host and target OS, some of the following will be built:
1. the maintenance service;
2. the updater binary;
3. the Windows Default Browser Agent;
4. the Background Update Agent.
Differential Revision: https://phabricator.services.mozilla.com/D82644
CLOSED TREE
Backed out changeset 51d7c644a1e6 (bug 1650163)
Backed out changeset 3d2b6908447a (bug 1650163)
Backed out changeset 79141707d47b (bug 1650163)
This patches does the following:
1. Test fetch in service worker should be classified.
2. Test requests should not be classified when it is whitelisted.
3. Test requests should not be classified when it is a first-party.
Depends on D80184
Differential Revision: https://phabricator.services.mozilla.com/D80185
This patch adds all desktop pageload tests to browsertime. It adds taskcluster tasks for these but doesn't enable them in production. They can only be run when --full is used.
Differential Revision: https://phabricator.services.mozilla.com/D82432
Platform-specific observations will be able to specify a file type.
They will appear in distinct rows in the profiler.firefox.com Marker Chart.
The default type is "File", which is shown in markers as "FileIO" like before.
Differential Revision: https://phabricator.services.mozilla.com/D80399
test_extension_storage_actor_upgrade.js was originally created from test_extension_storage_actor.js
Both tests use the PromiseTestUtils.whitelistRejectionsGlobally API which fails the rejected-words linter.
While we wait for the API to be renamed, I am adding the new test to the exclude list.
The profiler can be "paused", which stops sampling, and since bug 1578329 stops markers as well.
Some test suites use pausing between tests (to better differentiate the tests, to keep the profiler ready to run, and to lower the amount of recorded data). But this causes problems with some tracing markers, as their matching ends have not been recorded (e.g., an end marker is missing), which show up as very loooong markers.
To solve this, we need to be able to pause sampling only, but keep recording markers.
But we still need to be able to pause the whole profiler, in particular before capturing, to avoid recording anything around that time.
This big patch is mostly mechanical changes: Wherever there are "Pause" and "Unpause/Resume" profiler functions, we add matching "PauseSampling" and "UnpauseSampling/ResumeSampling" functions that only impact the periodic sampling loop; And existing "Pause/Unpause/Resume" imply pausing sampling as well.
Exceptions and extra work:
- nsIProfiler (the JS API) already had `Pause/ResumeSampling()`, which misleadingly paused everything! Now they do the right thing, and we have `Pause/Resume()` as well.
- All tests using `Pause/ResumeSampling()` now use `Pause/Resume()`, except for Talos tests that only pause sampling between tests; Added some extra `Pause()` calls to pause everything before capturing profiles.
- GeckoJavaSampler doesn't handle pausing/resuming everything, this should be done in a follow-up bug.
- Sampling-only pauses are not streamed into JSON. If needed, we should follow-up, with potential work on the front-end to deal with these.
Differential Revision: https://phabricator.services.mozilla.com/D81492
Use universal_newlines as a convenient way of treating the process I/O as text,
so string read/writes continue to succeed under python 3.
Differential Revision: https://phabricator.services.mozilla.com/D81774
This adds 3 new profiler markers for each IPC message:
* One just before the first byte is sent over the IPC channel
* One just after the last byte is sent over the IPC channel
* One just after the last byte is received from the IPC channel
With the already-existing IPC markers (for when SendXXX and RecvXXX are
called), this allows us to calculate the following statistics:
* Send thread latency
* IPC send duration
* IPC recv latency
* Recv thread latency
For more information on how this is presented in the UI, see:
<https://github.com/firefox-devtools/profiler/pull/2535>.
Differential Revision: https://phabricator.services.mozilla.com/D70790
Some profile meta information can be gathered before the profiler mutex must be locked.
This reduces the main-thread locking when capturing.
Most importantly, it prevents deadlocking in case any of the data-gathering operation would itself rely on profiler-locking functions (e.g., starting a thread, which requires the lock to register the new thread with the profiler).
Differential Revision: https://phabricator.services.mozilla.com/D81491
Some profile meta information can be gathered before the profiler mutex must be locked.
This reduces the main-thread locking when capturing.
Most importantly, it prevents deadlocking in case any of the data-gathering operation would itself rely on profiler-locking functions (e.g., starting a thread, which requires the lock to register the new thread with the profiler).
Differential Revision: https://phabricator.services.mozilla.com/D81491
Gerald Squelart
Tomislav Jovanovic
Randell Jesup
Myeongjun Go
Emil Farisan
Csoregi Natalia
Andrew Halberstadt
Andrew Halberstadt
championshuttler
Andreea Pavel
Dan Mosedale
Tom Prince
Mark Banner
Narcis Beleuzu
Harry Twyford
Cosmin Sabou
Kris Maglione
Axel Hecht
Sylvestre Ledru
Chris AtLee
Dorel Luca
Nick Alexander
Mihai Alexandru Michis
Nika Layzell
Dimi Lee
Gregory Mierzwinski
Manish Giri
Julian Descottes
Dave Townsend
Razvan Maries
Geoff Brown
Kershaw Chang
Jim Porter
Jean-Yves Avenard
chocos
Simon Giesecke
Matthew Noorenberghe
Paul Adenot