The mozjs_sys crate is actually responsible for all the problems
encountered with the js crate we just removed, but the sm-mozjs-crate
task that builds and runs tests for it doesn't fail because... there is
actually no test to build and run, so all it builds is a static library,
which doesn't expose all the problems that exist.
The content of this crate in mozilla-central is also outdated compared
to servo upstream.
Differential Revision: https://phabricator.services.mozilla.com/D106390
The sm-rust-bindings task that builds and runs tests for it actually only
doesn't fail by three(!) distinct happenstances of chance:
- It somehow doesn't hit a race condition that can happen, but that
attempting to change the build process for jsglue.cpp did,
- The second call to `cargo test` without `--feature debugmozjs` only
works because nothing was cleaned up from the first and the second
build uses artifacts from the first that are otherwise not there if
only running the second.
- The addition of rust code to spidermonkey (the jsrust crate in
js/src/rust) somehow didsn't cause problems in the current
configuration, but in other configurations (which fixing the second
thing above unveils) the build fails because of the multiple
definition of rust_eh_personality between the jsrust staticlib and the
rust stdlib that rustc links to test programs.
On top of all the above, the content of the crate in mozilla-central
has not received updates from upstream servo in years, and doesn't
reflect what servo actually uses.
Differential Revision: https://phabricator.services.mozilla.com/D106389
Use python3 in more places so we don't have to explicitly install
the python 2.x package.
The newer version of curl in 20.04 will hopefully fix the intermittent
http2 framing errors we've been seeing recently.
Differential Revision: https://phabricator.services.mozilla.com/D106259
This patch wants to solve several quirks around the shutdown terminator.
- Use the same shutdown phase definitions in AppShutdown and nsTerminator. This touches quite a few files.
- Ensure that the terminator phase shift is handled before any shutdown observer notifications are sent and reduce its heartbeat duration.
- Add missing phases to the shutdown telemetry.
Please note that this changes the unit of "tick" to 100ms rather than 1s.
As a side effect, we also remove the obsolete "shutdown-persist" context.
While the existing test coverage continues to prove the most important functions, we acknowledge the wish for better test coverage with [[ https://bugzilla.mozilla.org/show_bug.cgi?id=1693966 | bug 1693966 ]].
Differential Revision: https://phabricator.services.mozilla.com/D103626
This patch wants to solve several quirks around the shutdown terminator.
- Use the same shutdown phase definitions in AppShutdown and nsTerminator. This touches quite a few files.
- Ensure that the terminator phase shift is handled before any shutdown observer notifications are sent and reduce its heartbeat duration.
- Add missing phases to the shutdown telemetry.
Please note that this changes the unit of "tick" to 100ms rather than 1s.
As a side effect, we also remove the obsolete "shutdown-persist" context.
While the existing test coverage continues to prove the most important functions, we acknowledge the wish for better test coverage with [[ https://bugzilla.mozilla.org/show_bug.cgi?id=1693966 | bug 1693966 ]].
Differential Revision: https://phabricator.services.mozilla.com/D103626
This makes the naming more consistent with other functions called
Insert and/or Update. Also, it removes the ambiguity whether
Put expects that an entry already exists or not, in particular because
it differed from nsTHashtable::PutEntry in that regard.
Differential Revision: https://phabricator.services.mozilla.com/D105473
Conflicting pylint requirements were causing the new pip dependency resolver
to report an error. This updates the dependencies for pylint (using the hashin package)
so that the conflict is resolved.
Differential Revision: https://phabricator.services.mozilla.com/D106154
In XREMain(), if something went wrong between `PROFILER_INIT` and `MakeUnique<ScopedXPCOMStartup>()`,
- XPCOM would not be initialized -- Meaning `ExtensionPolicyService::GetSingleton()` would assert when not finding the Observer Service,
- But `gXPCOMShuttingDown` would still be false.
So `StreamMetaJSCustomObject` (invoked by `profiler_shutdown`, called by the `PROFILER_INIT` object destructor at the end of `XREMain()`) would still attempt to use `ExtensionPolicyService::GetSingleton()`.
In this patch, `StreamMetaJSCustomObject` now checks that the observer service exists, instead of only testing `!gXPCOMShuttingDown`. This should handle all situations where `ExtensionPolicyService::GetSingleton()` could assert, not just during normal shutdown.
Differential Revision: https://phabricator.services.mozilla.com/D106061
This will reduce the false-positive rate that sometimes causes devs to
ignore other important warnings because they expect to see these
warnings because of module/legacy code styles.
```
if (foo) return; // no longer warns
if (bar)
return; // still warns
```
clang-format will sometimes turn long-line versions of the former into
the latter, and this warning will pick those as readability warnings.
NB: This is Chromium style for same-line conditional statements.
Another test result json has its EOL-newline removed too, since that's
what the script does and I'm just committing its results.
Differential Revision: https://phabricator.services.mozilla.com/D105497
No code changes.
Build issues were found by renaming `MOZ_GECKO_PROFILER` to something else in toolkit/moz.configure, in both unified and non-unified builds, on all supported platforms.
Also updated some profiler-related comments.
Differential Revision: https://phabricator.services.mozilla.com/D105375
New headers BaseProfilerLabels.h and ProfilerLabels.h now contain all label-related APIs.
These files were hg-copied from the main headers, to preserve history, and then non-label content was removed from the main headers.
The "RAII" macros were moved to these *ProfilerLabels.h headers, because that's the most-common header in which they're needed. Meta-bug 1681416 will probably move these again as needed.
Differential Revision: https://phabricator.services.mozilla.com/D104587
{Base,}ProfilerMarkers.h can now rely on {Base,}ProfilerState.h (and just one function forward declaration), so they don't need to include {Base,Gecko}Profiler.h anymore.
Thanks to that, {Base,Gecko}Profiler.h can now include {Base,}ProfilerMarkers.h at the top.
Also, BaseProfilingStack.h should not include BaseProfiler.h (include loop, can cause failures) not algorithm (not used).
***
roll up into 2nd patch
Differential Revision: https://phabricator.services.mozilla.com/D104969
New headers BaseProfilerState.h and ProfilerState.h now contain most state-reading APIs.
These files were hg-copied from the main headers, to preserve history, and then chosen declarations were kept only in the relevant header.
This is needed in a following patch, where new headers *ProfilerLabels.h use `profiler_is_active()`.
Differential Revision: https://phabricator.services.mozilla.com/D104968
Because of conflicts between gcov_flush from gcc and the one from llvm, we renamed llvm one into ___custom_llvm_gcov_flush.
Since we switched to clang for linux ccov builds, this workaround is now useless.
Differential Revision: https://phabricator.services.mozilla.com/D104990
Because of conflicts between gcov_flush from gcc and the one from llvm, we renamed llvm one into ___custom_llvm_gcov_flush.
Since we switched to clang for linux ccov builds, this workaround is now useless.
Differential Revision: https://phabricator.services.mozilla.com/D104990
This adds a linter for Fluent files based upon the existing test for bad
strings in browser_misused_characters_in_strings.js. It also adds a check
for identifiers that only permits lowercase letters, numbers and the
hyphen character (in ascii). Since a large number of existing identifiers
use uppercase letters, an exclusions file is used to disable the identifier
check on a file by file basis.
Differential Revision: https://phabricator.services.mozilla.com/D104414
ProfileBufferCollector::SamplePositionInBuffer() and BufferRangeStart() need to provide indices in the same main buffer, because they will be used to discard old data (at some previous `SamplePositionInBuffer`) once the `BufferRangeStart` indicates that it is not referenced by the profiler anymore.
Because the periodic sampler uses a local buffer (to avoid allocations and locks), we need to record the special location from the main profiler buffer in ProfileBufferCollector.
Differential Revision: https://phabricator.services.mozilla.com/D104497
On Windows, when `moz-phab` is installed, two "executable" files are
created:
* moz-phab.exe
* moz-phab-script.py
Our `moz-phab.exe` detection was choking on `moz-phab-script.py` and was
unsure which of the two to use.
Differential Revision: https://phabricator.services.mozilla.com/D103834
These instructions were imported from the MDN page which was written at a time when clang-cl was an unstable non-default. Nowadays it should Just Work so we can cut out some unnecessary verbosity.
Differential Revision: https://phabricator.services.mozilla.com/D103636
This should make errors easier to diagnose, since what we seem to be
getting now is just the exit code, which in the recent instance was 16,
aka:
HTTP/2 error. A problem was detected in the HTTP2 framing layer. This
is somewhat generic and can be one out of several problems, see the
error message for details.
So hopefully future errors will include the error message.
Differential Revision: https://phabricator.services.mozilla.com/D103490
This change remove the files used for building and testing vrhost.dll, which was used for an early prototype of Firefox Reality PC.
This binary is no longer needed for FxR PC.
Differential Revision: https://phabricator.services.mozilla.com/D103237
Right now we don't store missing symbols when binary is missing for 64 bit platforms.
But some people don't need CFI to unwind stack since they've their own stuff but need to get symbol from address.
Differential Revision: https://phabricator.services.mozilla.com/D103152
This diff looks busy, but the only actual change is the addition of colorama to
the list; everything else is just formatting changes that the pip-compile tool
generated for me.
Differential Revision: https://phabricator.services.mozilla.com/D102869
More exactly: Instead of trying to compensate for only the previous sleep over/undershoot, we now try to keep each sampling loop to a schedule based on the very beginning of sampling, by adding the requested interval to the scheduled sampling time.
In addition, the sleep time is always kept to a minimum, to avoid making the system busier by having one loop right after the other -- also, this very-close data may be less useful.
And in presumably very busy times, one sleep and the following sampling work may take much more time than the requested sampling interval, trying to keep to schedule is now futile (it would require trying to effectively multiply the sampling rate, which seems unlikely to succeed, and would impact Firefox even more), in which case we revert to the full sampling interval.
Differential Revision: https://phabricator.services.mozilla.com/D102253
This patch rewrites some parts of the GeckoProfiler code to make it clearer and easier to maintain. It also changes how the profiles get organized into separate folders for each type. Furthermore, the archives no longer have the full directory path in them. To do this, we also have to update browsertime.
Differential Revision: https://phabricator.services.mozilla.com/D102043
mPostMeasurementTimeStamp records the time right after CPU measurements (point-based or interval) ended.
It is then used as the main sample timestamp, to both avoid another TimeStamp::Now() call, and to keep measurements and timestamp as close together as possible (and even closer in the next patch).
Differential Revision: https://phabricator.services.mozilla.com/D101545
This handles the conversion (from TimeStamp to number of milliseconds since process start) once and gives it to subroutines.
It will also help in a following patch where this value will be more closely tied with the CPU usage value, so we need to make sure the sample timestamp is taken at a single point and then forwarded wherever it's needed, be it a duplicate or a real sample.
While here, the nested `delta` variables in the Sampler have been disambiguated for better clarity:
- `sampleStartDeltaMs` is at the start of each sampling loop,
- `threadSampleDeltaMs` is associated to one thread being sampled during that loop.
Differential Revision: https://phabricator.services.mozilla.com/D101544
Now that we use an external dump_syms, we don't need to build
breakpad's.
This means we also don't need the dump_syms_rust_demangle crate anymore.
Differential Revision: https://phabricator.services.mozilla.com/D101865
This patch converts `GeckoChildProcessServices.java` into a jinja template.
We also add an overlay generated from a jinja template for `AndroidManifest.xml`
that provides the definitions for content process services.
Note that even though Gradle supports simple substitution of variables in
manifests, I opted not to use that functionality. Since we need the more
powerful template functionality that jinja provides, I felt that having multiple
ways to substitute information into the manifest would be confusing, so we're
using jinja exclusively.
Differential Revision: https://phabricator.services.mozilla.com/D82578
This patch converts `GeckoChildProcessServices.java` into a jinja template.
We also add an overlay generated from a jinja template for `AndroidManifest.xml`
that provides the definitions for content process services.
Note that even though Gradle supports simple substitution of variables in
manifests, I opted not to use that functionality. Since we need the more
powerful template functionality that jinja provides, I felt that having multiple
ways to substitute information into the manifest would be confusing, so we're
using jinja exclusively.
Differential Revision: https://phabricator.services.mozilla.com/D82578
pylint_requirements.txt fail to install with the new pip resolver due
to a conflict between astroid and lazy-object-proxy.
Rather than bumping those packages and handling the potential fallout,
the package-upgrade has been deferred and we will use the legacy
resolver in the interrim.
Differential Revision: https://phabricator.services.mozilla.com/D99940
In addition to the usual dot-release type of fixes, this also lets us drop a good amount of code that we had patched into our clang 11.
Differential Revision: https://phabricator.services.mozilla.com/D100959
The `RunningTimes` class stores CPU measurements. It may seem overkill for only one value, but in the future more measurements will be added.
During sampling, CPU measurements are collected by platform-specific code. This patch doesn't produce anything yet, see later patches.
These are stored with the samples.
Note that for duplicated samples (when a thread is known to be "asleep"), we still need to collect new measurements, because there could potentially be some activity happening, e.g. in system calls.
Finally the measurements are output as extra "samples" values.
Units for these values may platform-specific, so they are stored in the top-level JSON "meta" object.
We don't collect running times in the Base Profiler (yet), but we still need to add the appropriate field names in the samples' "schema", as expected by profiler.firefox.com.
Differential Revision: https://phabricator.services.mozilla.com/D99413
This patch adds "CPU Utilization" ("cpu" for short) as a new feature that will control the upcoming still-experimental CPU measurements.
Differential Revision: https://phabricator.services.mozilla.com/D99054
Instead of only capturing one feature (NoStackSampling), the sampler thread now stores all features so that any feature can be quickly looked at during sampling.
Currently this is still limited to NoStackSampling, a later patch will start using another feature.
Differential Revision: https://phabricator.services.mozilla.com/D99053
pylint_requirements.txt fail to install with the new pip resolver due
to a conflict between astroid and lazy-object-proxy.
Rather than bumping those packages and handling the potential fallout,
the package-upgrade has been deferred and we will use the legacy
resolver in the interrim.
Differential Revision: https://phabricator.services.mozilla.com/D99940
We need it from both FormAutofillHeuristics and CreditCardRuleset, and it would make a circular import otherwise: FormAutofillHeuristics -> CreditCardRuleset -> FormAutofillHeuristics.
Differential Revision: https://phabricator.services.mozilla.com/D100140
The "expected maximum stack size" (currently 64KiB) value was present in multiple places.
Now it's accessible from everywhere as ProfileBufferChunkManager::scExpectedMaximumStackSize, so it's easier to modify as needed.
Differential Revision: https://phabricator.services.mozilla.com/D100222
The profiler sampler was using the "cleared block count" as indication that some data could not be stored.
But this was incorrect, because cleared blocks only account for blocks in chunks that have been destroyed or recycled, but this missed data that could not be stored due to lack of space.
In particular, `ProfileBufferChunkManagerSingle` only provides one chunk, so when it gets full, a new chunk cannot be allocated, and the data is lost; but the "cleared block count" is left unchanged (the one chunk is only released, but not destroyed nor recycled).
Now the sampler correctly looks at the "failed puts bytes", and can report precisely how many bytes could not be stored.
Differential Revision: https://phabricator.services.mozilla.com/D99978
This gtest failure was useful during early development of markers 2.0 schema, to catch all known (at the time) marker types, so they could be tested.
But now that most marker types live locally where they're used, they cannot easily be unit-tested here, so it's easy to miss some of those types. And more of them will be added in the future, potentially causing more failures here.
At the same time, these Profiler tests run along with lots of other tests, which may "naturally" produce some of those marker types, in which case the corresponding schema will be present in output profiles when running Profiler tests.
So we shouldn't make our tests fail anymore when encountering unknown marker types. They may still be verified in other tests related to the code they live in (e.g., an XPCShell test could exercise some Firefox functionality that generates markers when profiling.)
There is still a benign `printf` message, which may be useful during development, but shouldn't appear as a failure (to be fixed) in CI.
Differential Revision: https://phabricator.services.mozilla.com/D100221
This is to match the trivial change in bug 1682349.
The failure was intermittent because we only test the BHR marker schema if that marker was actually used, and there's no easy way to force it during our tests; however while running the full gtest suite, it's possible that a previous test triggered that marker and failed in the non-updated test.
Differential Revision: https://phabricator.services.mozilla.com/D99973
Currently the MOZ_WEBRENDER environment variable isn't being for browsertime tests, this patch fixes this isse. This patch also updates the browsertime version to pick up a fix in how the Firefox environment variables are set.
Differential Revision: https://phabricator.services.mozilla.com/D99925
Currently the MOZ_WEBRENDER environment variable isn't being for browsertime tests, this patch fixes this isse. This patch also updates the browsertime version to pick up a fix in how the Firefox environment variables are set.
Differential Revision: https://phabricator.services.mozilla.com/D99925
The draft spec now officially lives at https://drafts.csswg.org/css-grid-3/
and we don't need this in-tree version anymore.
This commit entirely removes the directory layout/docs/css-grid-3/ and also
removes the line for this directory from tools/rewriting/ThirdPartyPaths.txt.
Differential Revision: https://phabricator.services.mozilla.com/D99850
I'm keeping the --enable-update-agent config option and the corresponding
MOZ_UPDATE_AGENT config flag and define, as these should still be useful.
As we never shipped this there is no need to keep anything around to
clean up the scheduled tasks.
Differential Revision: https://phabricator.services.mozilla.com/D99574
Under Python 3 on Windows we sometimes get an OSError which may be a
race with the process exiting. There's not much we can do about
exceptions during shutdown so just ignore them.
Differential Revision: https://phabricator.services.mozilla.com/D99434
JSONOutputCheck used to only check the profile output for the presence of some strings.
Now it parses the output as JSON, and navigates the JSON data to check expected properties, including their types, and values as needed.
Differential Revision: https://phabricator.services.mozilla.com/D98890
JSONOutputCheck used to only check the profile output for the presence of some strings.
Now it parses the output as JSON, and navigates the JSON data to check expected properties, including their types, and values as needed.
Differential Revision: https://phabricator.services.mozilla.com/D98890
While all toolkit and js-based projects make use of mfbt, some others,
like tools/crashreporter and tools/update-packaging, don't.
So instead of including mfbt from the top-level directory, include it
from the relevant project top-level mozbuilds.
This allows to remove the dependency on mfbt files in the hash for the
minidump-stackwalk and mar-tools toolchains.
Differential Revision: https://phabricator.services.mozilla.com/D98378
Jens Stutte
Mike Hommey
Dan Minor
Julien Cristau
Butkovits Atila
Simon Giesecke
Alex Lopez
Kershaw Chang
Daisuke Akatsuka
smolnar
Henrik Skupin
Gerald Squelart
Alexandre Poirot
championshuttler
Neil Deakin
Jeff Gilbert
Simon Fraser
Greg Tatum
Matthew Gregan
Florian Quèze
Renovate Bot
Julien Wajsberg
Bogdan Tara
Gregory Mierzwinski
Csoregi Natalia
Calixte Denizet
Sylvestre Ledru
Marco Castelluccio
Mark Banner
Markus Stange
Akshat Dixit
Alex Ionescu
Mitchell Hentges
James Graham
Myeongjun Go
David Major
Paul Bone
Xidorn Quan
thomasmo
Molly Howell
Yura Zenevich
Tyson Smith
Noemi Erli
Chris H-C
Aaron Klotz
Kartik Gautam
Kartikaya Gupta
Cosmin Sabou
Razvan Maries
Erik Rose
Tarek Ziadé
Daniel Holbert
Adam Gashlin
Sonia Singla
Dorel Luca
Gabriele Svelto
Christian Holler
Barret Rennie
Andrew Osmond