The following python-test paths are being moved out of 'make check' and into their own task:
- python/mozlint
- testing/mozbase
- tools/lint
The following python-test paths previously did not run on Windows:
- python/mozterm
- testing/marionette
- testing/raptor
- tools/tryselect
MozReview-Commit-ID: C07FANaYzf7
Depends on D10758
Differential Revision: https://phabricator.services.mozilla.com/D10759
--HG--
extra : moz-landing-system : lando
This assumes that a lineno of 0 denotes a "file-level" issue, e.g an issue
associated with the filename. In the future it might be better to treat these
"file-level" issues as first class citizens. This would involve updating things
like formatters, editor integrations and review bot to not assume a lineno.
Differential Revision: https://phabricator.services.mozilla.com/D10383
--HG--
extra : moz-landing-system : lando
This field is in addition to the existing process type fields we already have:
- profile.threads[i].processType contains the string for the GeckoProcessType.
- profile.threads[i].name contains the ThreadInfo name.
Differential Revision: https://phabricator.services.mozilla.com/D10549
--HG--
extra : moz-landing-system : lando
Added a mechanism to register and unregister the DocShells from the CorePS depending
on the state of the profiler. Registering mechanism is straightforward. During
unregistration, if profiler is not active, we remove the DocShell information
immediately. If profiler is active, we don't remove and we keep the profiler buffer
position at that moment. During another DocShell registration we Discard the
unregistered DocShells. If the profiler buffer position is greater than the position
when we captured during unregistration, we delete the DocShell since that means there
can't be any markers associated to this DocShell anymore.
MozReview-Commit-ID: IVuKQ6drvkR
Differential Revision: https://phabricator.services.mozilla.com/D4914
--HG--
extra : moz-landing-system : lando
These flags will be used by WebIDL APIs in an upcoming patch.
Depends on D9199
Differential Revision: https://phabricator.services.mozilla.com/D9203
--HG--
extra : moz-landing-system : lando
This code is run during JSON serialization so performance is not a big concern.
Depends on D9195
Differential Revision: https://phabricator.services.mozilla.com/D9197
--HG--
extra : moz-landing-system : lando
They were not displayed in the UI, and the instructions to initialize the line
field of a stack frame increased code size unnecessarily.
This change reduces the binary size on Linux x64 by around 100KB.
Here's a diff of the impact on the code generated for Attr_Binding::get_specified
in the Mac build:
@@ -20,17 +20,16 @@
movq 0x8(%rbx), %rax
movq %r12, %rcx
shlq $0x5, %rcx
leaq aGetAttrspecifi, %rdx ; "get Attr.specified"
movq %rdx, (%rax,%rcx)
movq $0x0, 0x8(%rax,%rcx)
leaq -40(%rbp), %rdx
movq %rdx, 0x10(%rax,%rcx)
- movl $0x106, 0x18(%rax,%rcx)
movl $0x1c, 0x1c(%rax,%rcx)
leal 0x1(%r12), %eax
movl %eax, 0x10(%rbx)
movq %r15, %rdi
call __ZNK7mozilla3dom4Attr9SpecifiedEv ; mozilla::dom::Attr::Specified() const
movzxl %al, %eax
movabsq $0xfff9000000000000, %rcx
Depends on D9193
Differential Revision: https://phabricator.services.mozilla.com/D9195
--HG--
extra : moz-landing-system : lando
This eliminates a few instructions from every profiler label and saves code size.
We have around 9000 WebIDL constructors + methods + getters + setters which all
have an inlined instance of this code.
This change reduces the binary size on Linux x64 by around 160KB.
Here's a diff of the impact on the code generated for Attr_Binding::get_specified
in the Mac build:
movq %rsp, %rbp
pushq %r15
pushq %r14
pushq %r12
pushq %rbx
subq $0x10, %rsp
movq %rcx, %r14
movq %rdx, %r15
- movq __ZN7mozilla8profiler6detail12RacyFeatures18sActiveAndFeaturesE@GOT, %rax ; __ZN7mozilla8profiler6detail12RacyFeatures18sActiveAndFeaturesE@GOT
- movl (%rax), %eax
- testl %eax, %eax
- js loc_xxxxx
-
- movq $0x0, -40(%rbp)
- jmp loc_xxxxx
-
- movq 0x78(%rdi), %rbx
+ movq 0x80(%rdi), %rbx
movq %rbx, -40(%rbp)
testq %rbx, %rbx
je loc_xxxxx
movl 0x10(%rbx), %r12d
cmpl %r12d, (%rbx)
jbe loc_xxxxx
Differential Revision: https://phabricator.services.mozilla.com/D9192
--HG--
extra : moz-landing-system : lando
This prevents a race condition where the JS Engine would get the Fuzzed Time
before the dispatched job was run, and the time was zero and triggered an assertion.
This patch also includes more verbose logging for additional debugging.
This implementation is considerably more memory efficient than the existing JS
implementation. Migrating to it fully saves us about 28K per base content
process.
Differential Revision: https://phabricator.services.mozilla.com/D9887
--HG--
extra : source : a19c6b3e0402d16a77185f82d9fedab83a7ca52e
extra : intermediate-source : 28a19b7290ab288a1cb2cbf6d49f905cecc9682b
This implementation is considerably more memory efficient than the existing JS
implementation. Migrating to it fully saves us about 28K per base content
process.
Differential Revision: https://phabricator.services.mozilla.com/D9887
--HG--
extra : source : a19c6b3e0402d16a77185f82d9fedab83a7ca52e
This implementation is considerably more memory efficient than the existing JS
implementation. Migrating to it fully saves us about 28K per base content
process.
Differential Revision: https://phabricator.services.mozilla.com/D9887
--HG--
extra : rebase_source : f358f1f7097eaa8cd62ae916d048a58489e9f5ff
extra : histedit_source : c44c42de02750d71e23e70fdf22d3c3a40b91d3d
This prevents a race condition where the JS Engine would get the Fuzzed Time
before the dispatched job was run, and the time was zero and triggered an assertion.
This patch also includes more verbose logging for additional debugging.
These flags will be used by WebIDL APIs in an upcoming patch.
Depends on D9199
Differential Revision: https://phabricator.services.mozilla.com/D9203
--HG--
extra : moz-landing-system : lando
This code is run during JSON serialization so performance is not a big concern.
Depends on D9195
Differential Revision: https://phabricator.services.mozilla.com/D9197
--HG--
extra : moz-landing-system : lando
They were not displayed in the UI, and the instructions to initialize the line
field of a stack frame increased code size unnecessarily.
This change reduces the binary size on Linux x64 by around 100KB.
Here's a diff of the impact on the code generated for Attr_Binding::get_specified
in the Mac build:
@@ -20,17 +20,16 @@
movq 0x8(%rbx), %rax
movq %r12, %rcx
shlq $0x5, %rcx
leaq aGetAttrspecifi, %rdx ; "get Attr.specified"
movq %rdx, (%rax,%rcx)
movq $0x0, 0x8(%rax,%rcx)
leaq -40(%rbp), %rdx
movq %rdx, 0x10(%rax,%rcx)
- movl $0x106, 0x18(%rax,%rcx)
movl $0x1c, 0x1c(%rax,%rcx)
leal 0x1(%r12), %eax
movl %eax, 0x10(%rbx)
movq %r15, %rdi
call __ZNK7mozilla3dom4Attr9SpecifiedEv ; mozilla::dom::Attr::Specified() const
movzxl %al, %eax
movabsq $0xfff9000000000000, %rcx
Depends on D9193
Differential Revision: https://phabricator.services.mozilla.com/D9195
--HG--
extra : moz-landing-system : lando
This eliminates a few instructions from every profiler label and saves code size.
We have around 9000 WebIDL constructors + methods + getters + setters which all
have an inlined instance of this code.
This change reduces the binary size on Linux x64 by around 160KB.
Here's a diff of the impact on the code generated for Attr_Binding::get_specified
in the Mac build:
movq %rsp, %rbp
pushq %r15
pushq %r14
pushq %r12
pushq %rbx
subq $0x10, %rsp
movq %rcx, %r14
movq %rdx, %r15
- movq __ZN7mozilla8profiler6detail12RacyFeatures18sActiveAndFeaturesE@GOT, %rax ; __ZN7mozilla8profiler6detail12RacyFeatures18sActiveAndFeaturesE@GOT
- movl (%rax), %eax
- testl %eax, %eax
- js loc_xxxxx
-
- movq $0x0, -40(%rbp)
- jmp loc_xxxxx
-
- movq 0x78(%rdi), %rbx
+ movq 0x80(%rdi), %rbx
movq %rbx, -40(%rbp)
testq %rbx, %rbx
je loc_xxxxx
movl 0x10(%rbx), %r12d
cmpl %r12d, (%rbx)
jbe loc_xxxxx
Differential Revision: https://phabricator.services.mozilla.com/D9192
--HG--
extra : moz-landing-system : lando
These flags will be used by WebIDL APIs in an upcoming patch.
Depends on D9199
Differential Revision: https://phabricator.services.mozilla.com/D9203
--HG--
extra : moz-landing-system : lando
This code is run during JSON serialization so performance is not a big concern.
Depends on D9195
Differential Revision: https://phabricator.services.mozilla.com/D9197
--HG--
extra : moz-landing-system : lando
They were not displayed in the UI, and the instructions to initialize the line
field of a stack frame increased code size unnecessarily.
This change reduces the binary size on Linux x64 by around 100KB.
Here's a diff of the impact on the code generated for Attr_Binding::get_specified
in the Mac build:
@@ -20,17 +20,16 @@
movq 0x8(%rbx), %rax
movq %r12, %rcx
shlq $0x5, %rcx
leaq aGetAttrspecifi, %rdx ; "get Attr.specified"
movq %rdx, (%rax,%rcx)
movq $0x0, 0x8(%rax,%rcx)
leaq -40(%rbp), %rdx
movq %rdx, 0x10(%rax,%rcx)
- movl $0x106, 0x18(%rax,%rcx)
movl $0x1c, 0x1c(%rax,%rcx)
leal 0x1(%r12), %eax
movl %eax, 0x10(%rbx)
movq %r15, %rdi
call __ZNK7mozilla3dom4Attr9SpecifiedEv ; mozilla::dom::Attr::Specified() const
movzxl %al, %eax
movabsq $0xfff9000000000000, %rcx
Depends on D9193
Differential Revision: https://phabricator.services.mozilla.com/D9195
--HG--
extra : moz-landing-system : lando
This eliminates a few instructions from every profiler label and saves code size.
We have around 9000 WebIDL constructors + methods + getters + setters which all
have an inlined instance of this code.
This change reduces the binary size on Linux x64 by around 160KB.
Here's a diff of the impact on the code generated for Attr_Binding::get_specified
in the Mac build:
movq %rsp, %rbp
pushq %r15
pushq %r14
pushq %r12
pushq %rbx
subq $0x10, %rsp
movq %rcx, %r14
movq %rdx, %r15
- movq __ZN7mozilla8profiler6detail12RacyFeatures18sActiveAndFeaturesE@GOT, %rax ; __ZN7mozilla8profiler6detail12RacyFeatures18sActiveAndFeaturesE@GOT
- movl (%rax), %eax
- testl %eax, %eax
- js loc_xxxxx
-
- movq $0x0, -40(%rbp)
- jmp loc_xxxxx
-
- movq 0x78(%rdi), %rbx
+ movq 0x80(%rdi), %rbx
movq %rbx, -40(%rbp)
testq %rbx, %rbx
je loc_xxxxx
movl 0x10(%rbx), %r12d
cmpl %r12d, (%rbx)
jbe loc_xxxxx
Differential Revision: https://phabricator.services.mozilla.com/D9192
--HG--
extra : moz-landing-system : lando
Removing this field since we already have start/end time for each marker payloads.
Also we are assigning the same timestamp to them. We don't need to unnecessarily
duplicate the information here.
Differential Revision: https://phabricator.services.mozilla.com/D8551
--HG--
extra : moz-landing-system : lando
Now autotest does not require java to be installed, but
it will let the user know that infer is not being tested if java
is missing.
Differential Revision: https://phabricator.services.mozilla.com/D7326
--HG--
extra : moz-landing-system : lando
We only use `include_nigthly` where we are also using
`filter_beta_release_tasks`, so just change the later to include nightly.
Differential Revision: https://phabricator.services.mozilla.com/D9687
--HG--
extra : moz-landing-system : lando
JSONWriter currently calls new and delete indirectly through mozilla::MakeUnique to allocate a buffer. Becuase of this, the methods of this class cannot be invoked within Spidermonkey due to https://searchfox.org/mozilla-central/source/config/check_vanilla_allocations.py#6-14. Therefore, JSONWriter needs an AllocPolicy template parameter so that the allocation and deallocation routines can be changed to match the JS AllocPolicy when invoked within SpiderMonkey.
Differential Revision: https://phabricator.services.mozilla.com/D7279
--HG--
extra : moz-landing-system : lando
This lets ProfilerMarkerPayload implementations live outside of the profiler.
Differential Revision: https://phabricator.services.mozilla.com/D9325
--HG--
rename : tools/profiler/core/ProfileJSONWriter.h => tools/profiler/public/ProfileJSONWriter.h
extra : moz-landing-system : lando
This is no longer needed because the manifest update now happens out of tree.
The taskcluster task was already removed, though the actual linter
implementation was forgotten.
Differential Revision: https://phabricator.services.mozilla.com/D9118
--HG--
extra : moz-landing-system : lando
This adds `mach try release` which adds temporary changes to enable staging
release to run.
Differential Revision: https://phabricator.services.mozilla.com/D8625
--HG--
extra : moz-landing-system : lando
Right now there are excludes defined in the linter definition (via the .yml
files), as well as excludes defined in lintargs (via the mach_commands.py).
This is a minor simplification that extends each linter definition's local
excludes with the global ones right off the bat. This just makes it a bit
easier to keep track of.
Depends on D5863
Differential Revision: https://phabricator.services.mozilla.com/D8844
--HG--
extra : moz-landing-system : lando
Bug 1438688 made it so that XPT information is compiled directly into
the binary instead of being shipped separately in interface
files. This means that manifests are no longer necessary for JS
components, which means the manifest check in emitter.py can be
removed.
That check is the only use of NO_JS_MANIFEST, so that can in turn be
removed entirely.
Differential Revision: https://phabricator.services.mozilla.com/D8885
--HG--
extra : moz-landing-system : lando
When using globs in exclude directorives, FileFinder will return every *file*
that gets matched. This is can be thousands of files in the case of an objdir.
While we now collapse these files down to highest possible directories, this
collapse operation can still take a noticeable amount of time (0.6s). This
simply scans topsrcdir for files that start with 'obj' to avoid the glob.
This also moves the '_activate_virtualenv' call to the top of the function
because in CI, this will cause an objdir to be created (to store the
virtualenv). If this happens *after* calculating the global excludes, we won't
catch it since it doesn't exist yet. This will result in the objdir's
virtualenv being linted and erroneous failures.
Depends on D7739
Differential Revision: https://phabricator.services.mozilla.com/D7740
--HG--
extra : moz-landing-system : lando
Often we specify globs in our exclude patterns, e.g:
exclude:
- **/node_modules
- obj*
However, these globs get expanded out to *every* file that matches them. This
can sometimes be thousands or even tens of thousands of files.
We then pass these paths on to the underlying linters and tell them to
exclude them all. This causes a lot of overhead and slows down performance.
This commit implements a "collapse" function. Given a set of paths, it'll
collapse them into the smallest set of parent directories that contain the
original set, and that don't contain any extra files.
For example, given a directory structure like this:
a
-- foo.txt
-- b
-- bar.txt
-- baz.txt
-- c
-- ham.txt
-- d
-- spam.txt
Then the following will happen:
>>> collapse(['a/foo.txt', 'a/b/bar.txt', 'a/c/ham.txt', 'a/c/d/spam.txt'])
['a/foo.txt', 'b/bar.txt', 'c']
Since all files under directory 'c' are specified by the original set (both
'c/ham.txt' and 'c/d/spam.txt'), we can collapse it down to just 'c'. However
not all files under 'b' are specified (we're missing 'a/b/baz.txt'), so we
can't collapse 'b' (and by extension also can't collapse 'a').
If we had included 'a/b/baz.txt':
>>> collapse(['a/foo.txt', 'a/b/bar.txt', 'a/b/baz.txt', 'a/c/ham.txt', 'a/c/d/spam.txt'])
['a']
In both cases, the smallest set of paths that contains the original set (and
only the original set) is computed.
The collapse function has a little bit of overhead but it's not too bad.
For example collapsing all files matched by '**/node_modules' takes ~0.015s.
Collapsing two full objdirs, takes ~0.6s. But a follow up commit is planned to
make sure we stop using 'obj*' to reduce that overhead.
Depends on D7738
Differential Revision: https://phabricator.services.mozilla.com/D7739
--HG--
extra : moz-landing-system : lando
If scandir is already present on the system the attempt to import the
c helper library will currently find the c helper from the system
install which may well be an outdated verion, so causing mach to
break. To solve this this patch does two things:
* Stops importing scandir in files that are run unconditionally when
invoking mach. This is generally considered good for performance
reasons.
* Installs the vendored scandir into the virtualenv for `mach lint`
rather than trying to import it directly from the source tree and so
not getting the c helper library.
Differential Revision: https://phabricator.services.mozilla.com/D8379
--HG--
extra : moz-landing-system : lando
When using globs in exclude directorives, FileFinder will return every *file*
that gets matched. This is can be thousands of files in the case of an objdir.
While we now collapse these files down to highest possible directories, this
collapse operation can still take a noticeable amount of time (0.6s). This
simply scans topsrcdir for files that start with 'obj' to avoid the glob.
This also moves the '_activate_virtualenv' call to the top of the function
because in CI, this will cause an objdir to be created (to store the
virtualenv). If this happens *after* calculating the global excludes, we won't
catch it since it doesn't exist yet. This will result in the objdir's
virtualenv being linted and erroneous failures.
Depends on D7739
Differential Revision: https://phabricator.services.mozilla.com/D7740
--HG--
extra : moz-landing-system : lando
This is follow-up to Bug 1291335, which inadvertently prevented Fennec
single-locale repacks from running |mach compare-locales|, since the
command was incorrectly considered part of the `testing` category.
Differential Revision: https://phabricator.services.mozilla.com/D8182
--HG--
extra : moz-landing-system : lando
This patch removes linux64-jsdcov from the available builds on taskcluster along with any hacks used to run it. It also removes any 'coverage' entries that were added to skip tests.
Differential Revision: https://phabricator.services.mozilla.com/D7919
--HG--
extra : moz-landing-system : lando
I don't understand what's happening in this code and would appreciate some
guidance. It's possible that there are two different meanings of the word
"offset" that are clashing here.
SharedLibrary::GetOffset() means firstMappingStart - baseAddress.
I get a feeling that the "zero offset" check here cares more about the mapping
offset phdr.p_offset, but I'm not sure.
Anyway, the purpose of this patch is the following:
Now that the previous patch sets SharedLibrary::mOffset to something non-zero,
this check breaks unwinding for the affected libraries. With the check removed,
unwinding seems to work mostly fine, as evidenced by this profile:
https://perfht.ml/2MBKzON
But removing the check might break something else that I'm not seeing.
Depends on D3835
Differential Revision: https://phabricator.services.mozilla.com/D3836
--HG--
extra : moz-landing-system : lando
r?njn for the profiler parts
r?jrmuizel for the ELF parsing parts
Depends on D7020
Differential Revision: https://phabricator.services.mozilla.com/D7021
--HG--
extra : moz-landing-system : lando
r?njn for the profiler parts
r?jrmuizel for the ELF parsing parts
Depends on D7020
Differential Revision: https://phabricator.services.mozilla.com/D7021
--HG--
extra : moz-landing-system : lando
packge-lock.json and tools/lint/eslint/manifest.tt were generated automatically by running tools/lint/eslint/setup.sh, as per previous revisions to package.json.
I've intentionally kept the package.json file simple for now, we can extend it as we get the requirements defined for vendoring new node modules.
Differential Revision: https://phabricator.services.mozilla.com/D7272
--HG--
extra : moz-landing-system : lando
There is only a single linter (test-disable.yml) that uses a glob in any
include path, and that usage is easily replaced by using the 'extensions' key
instead.
Since globs in include directives aren't very useful, let's disallow them. This
will allow us to simplify the 'filterpaths' logic quite substantially and make
future refactorings in this area easier.
Differential Revision: https://phabricator.services.mozilla.com/D6798
--HG--
extra : moz-landing-system : lando
This extracts the current logic for finding nodejs into its own module in mozbuild. Configure and ESLint then use it.
For ESLint, this will change the first location it looks for nodejs to be the .mozbuild directory.
Differential Revision: https://phabricator.services.mozilla.com/D6430
--HG--
extra : moz-landing-system : lando
shindli
Cosmin Sabou
Marco Castelluccio
arthur.iakab
Andreea Pavel
Andrew Halberstadt
Markus Stange
Randell Jesup
Ciure Andrei
Andrea Marchesini
Tudor-Gabriel Vîjială
Sylvestre Ledru
Junior Hsu
Tom Prince
Gabriele Svelto
Mark Striemer
Gerald Squelart
Nazım Can Altınova
Gurzau Raul
Tom Ritter
Kris Maglione
Brindusan Cristian
Narcis Beleuzu
Julian Descottes
Csoregi Natalia
Andi-Bogdan Postelnicu
Rob Wood
Mark Banner
Nika Layzell
Robert Bartlensky
Kearwood "Kip" Gilbert
Noemi Erli
Denis Palmeiro
Jeff Muizelaar
Andrew McCreight
Gijs Kruitbosch
ScottDowne
James Graham
Ryan VanderMeulen
Nick Alexander
Ursula Sarracini
Doug Thayer
Gregory Mierzwinski
Matt Brubeck
Nathan Froyd
Marco Bonardo
Brian Grinstead
Margareta Eliza Balazs
