Explicitly specify that the linter should run on all files (`*`), to
circumvent bug 1753701's behaviour of defaulting to VCS-changed files.
Differential Revision: https://phabricator.services.mozilla.com/D138173
Tweaks `./mach lint` behaviour to always fall back to only linting files
that have changed according to VCS - previously, this only happened if
no linter was provided.
Adjusts "am I at $topsrcdir" check to use `pathlib` to avoid mismatches
due to inconsistent capitalization or slash direction.
Updates CI references to explicitly provide `*` as the path to avoid
the only-lint-files-changed restriction.
Differential Revision: https://phabricator.services.mozilla.com/D137870
The platform(s) were not specified, and it seems when that is the case that
it will only run on linux by default. The platforms are now explicitly
set to include Windows, Linux, and Mac.
Differential Revision: https://phabricator.services.mozilla.com/D136214
This will ensure we don't accidentally cause bustage to graph generation with
any of the parameters checked into `taskcluster/test/params`.
This was previously being (sort of) tested by the `tgdiff` task. Now that this
test exists, we no longer need to rely on it.
I removed 'always-target' from the task since this now takes ~25 min to run and
which is the new bottleneck for reviewbot turn around times.
Differential Revision: https://phabricator.services.mozilla.com/D136514
The test itself takes around 25 minutes. If the task hits the clone step, it will take an additional 10minutes.
Note that there is also a slow clone issue where clone can take between 15 and 30 mn.
Differential Revision: https://phabricator.services.mozilla.com/D135476
This change adds a new lint `android-format` which enforces formatting of Java
code using google-java-format.
To run the lint simply run:
./mach lint -l android-format
This command also support automatically fixing all errors running by adding
--fix:
./mach lint -l android-format --fix
This change also removes all the formatting-related checkstyle checks which are
now implicitly enforced by the formatter.
Differential Revision: https://phabricator.services.mozilla.com/D127734
For a long time two copies of the 'taskgraph' module have existed in parallel.
We've attempted to keep them in sync, but over time they have diverged and the
maintenance burden has increased.
In order to reduce this burden, we'd like to re-join the two code bases. The
canonical repo will be the one that lives outside of mozilla-central, and this
module will depend on it. Since they both have the same module name (taskgraph)
we need to rename the version in mozilla-central to avoid collisions.
Other consumers of 'taskgraph' (like mobile repos) have standardized on
'<project>_taskgraph' as their module names. So replicating that here as well.
Differential Revision: https://phabricator.services.mozilla.com/D127118
Since all commits on non-try branches are public, vcs.base_ref was returning
the current revision each time and the diff task was always diffing against the
same revision twice.
By using 'json-automationrelevance' to determine the parent push, we can be
sure we're always diffing against the proper revision.
Depends on D125974
Differential Revision: https://phabricator.services.mozilla.com/D125975
For now this will only upload differences in tasks added / removed, until we
can figure out a way to remove ambient diffs from the bodies of the tasks (e.g
timestamps, revisions, etc).
Depends on D125866
Differential Revision: https://phabricator.services.mozilla.com/D125867
We set always-target: true for Python unittest tasks. This means they show up on every push (when appropriate files are modified). The reasoning behind this is that they run so fast anyway, and if you modify the relevant code then you almost always want to see the unittests for said code on your try pushes.
However on MacOS, the pool is limited. Given the differences between Linux and Mac for most Python unittests are likely extremely small, I don't think the cost of bogging down the Mac pool outweighs the benefits here.
Differential Revision: https://phabricator.services.mozilla.com/D125281
This is the last version that doesn't require Java 11, we will upgrade to
Gradle 7 once all components are ready (namely, apilint).
Co-authored-by: Jan-Erik Rediger <janerik@fnordig.de>
Differential Revision: https://phabricator.services.mozilla.com/D123569
Looks like 6G is not enough for an ASAN build when updating the gradle version.
I tried 8G and 16G on try but that's not enough either.
This also:
* Moves the asan job to `b-linux-large` as the `b-linux` builder does not have
enough memory to run this build.
* Stops running a full build during lints, which is not necessary (and
sometimes uses more memory than the build runner has, failing the lint).
Differential Revision: https://phabricator.services.mozilla.com/D123970
This is the last version that doesn't require Java 11, we will upgrade to
Gradle 7 once all components are ready (namely, apilint).
Co-authored-by: Jan-Erik Rediger <janerik@fnordig.de>
Differential Revision: https://phabricator.services.mozilla.com/D123569
This is the last version that doesn't require Java 11, we will upgrade to
Gradle 7 once all components are ready (namely, apilint).
Co-authored-by: Jan-Erik Rediger <janerik@fnordig.de>
Differential Revision: https://phabricator.services.mozilla.com/D123569
The wasi-sysroot toolchain contains both a sysroot for wasi and a
compiler-rt for clang. That makes it impractical to use as a
bootstrapped sysroot for wasm32-wasi builds of Spidermonkey.
We thus split the toolchain in two, one for the compiler-rt and one
for the sysroot. Ideally, the compiler-rt one would avoid building
clang/llvm the same way the sysroot one does, but that leads to
a case of chicken-and-egg, because the compiler-rt is needed to build
the clang toolchain. Eventually, the clang build would be split from
the addition of the compiler-rt, but we're not there yet.
Differential Revision: https://phabricator.services.mozilla.com/D122402
Followup for bug 1719229 ; some builds have not been tweaked to use the
x86_64-linux-gnu sysroot.
In a few cases, this also use a sysroot for the target part when that
wasn't already the case.
Differential Revision: https://phabricator.services.mozilla.com/D119957
In cross-compilation setups (x86_64 host, i686 or aarch64 target), we're
going to need two sysroots. Obviously, we need the sysroot paths to be
different in that case, so the sysroot path themselves need to contain
some distinctive name, and we'll use the `target.toolchain` name for
that (the target triplet with the vendor/machine stripped out).
Because the path name needs to be reflected in the artifact name as well
as the toolchain name, we also change them.
And because the current prefix in the toolchain name is now redundant
with the suffix, we remove the prefix, and allow the bootstrapping
mechanism to try toolchains without the prefix.
Differential Revision: https://phabricator.services.mozilla.com/D119846
This patch adds a new linter that will error when new code mentions any one of
the following strings:
* `CoInitialize`;
* `CoInitializeEx`;
* `OleInitialize`;
* `RoInitialize`;
* `CoUninitialize`;
* `OleUninitialize`; and
* `RoUninitialize`.
Since I don't care about context, and just want to flag code containing these
names, I opted for a `regex` linter.
Yes, the regex does match a few strings beyond the above list (in particular, it
also matches additional strings that end with an `Ex` suffix), but since
functions with those names don't exist anyway (and would be errors in their own
right), I am not concerned about it.
All existing occurrences have been added to the exclusion list, with the
intent of removing most of them over time.
Differential Revision: https://phabricator.services.mozilla.com/D119129
The `mozbase` modules were being unconditionally added to the
`sys.path` regardless of the Mach command being run, so there isn't
much value keeping them in a separate file. Besides, all other
source module paths are described in `common_virtualenv_packages`,
why is `mozbase` special?
In the future, we're going to want to make improvements here (such as:
there's a difference between informing mach of first-party code
versus defining which third_party vendored packages should be in scope,
and that workflow difference should be represented in-code).
It's useful to peel out the existing, less useful abstraction before
we can build a stronger one.
Differential Revision: https://phabricator.services.mozilla.com/D117711
We don't offer API splits any more, and with the separation of GeckoView with
the rest of the front-end it's increasingly unlikely that we will in the
future.
This change makes it so that the build name doesn't contain the API version so
that we can update it without breaking all the automation that relies on the
build name.
Differential Revision: https://phabricator.services.mozilla.com/D114369
We don't offer API splits any more, and with the separation of GeckoView with
the rest of the front-end it's increasingly unlikely that we will in the
future.
This change makes it so that the build name doesn't contain the API version so
that we can update it without breaking all the automation that relies on the
build name.
Differential Revision: https://phabricator.services.mozilla.com/D114369
CLOSED TREE
Backed out changeset 7a4401a358e8 (bug 1639164)
Backed out changeset f1377ee7e2d2 (bug 1639164)
Backed out changeset f9c73976484d (bug 1639164)
The parallel run of sequences of "clean; clippy; clean" creates race
conditions that alter the results of clippy and can mask errors.
Moreover, chances are clippy is not allowing any parallelization
anyways, because with two concurrent clippy running, one will be waiting
for the other because of cargo locking.
This turns the current intermittent failure into a permanent one.
Differential Revision: https://phabricator.services.mozilla.com/D114209
This will tie the version used for CI lints to the version of rust used
for builds on CI.
Bonus point: we can now have rustfmt and clippy on Windows and mac,
which allows to run the corresponding mozlint unit tests on those
platforms.
Differential Revision: https://phabricator.services.mozilla.com/D113905
This will warn if someone includes something like:
skip-if = <condition A> || <condition B> # reason A is skipped; reason B is skipped
Instead they should use:
skip-if =
<condition A> # reason A is skipped
<condition B> # reason B is skipped
Differential Revision: https://phabricator.services.mozilla.com/D113707
This adds a `mach wpt-fission-regressions` command that uses the wpt
expectation data to look for tests which have a worse result in
fission. With the `--all-json=<path>` argument it will output a JSON
file containing details of all the regressions. With the
`--untriaged=<path>` argument it will output a file containing a list
of failures that have not yet been triaged.
It also adds a try job to produce those files as artifacts whenever
wpt metadata is changed.
The actual implementation is based on reading the wpt expectation data
with sample run_info values corresponding to the configurations in
which we have fission enabled, but with the "fission" property set to
False (to get a baseline result) and True (to get a with-fission
result) and then comparing the resulting expectations.
The implemenation is pretty suboptimal performance wise since we end
up reading the metadata once per configuration i.e. 6 times, and this
is slow. It could be optimised by using the conditional metadata
backend, reading it once, and then evaluating per
configuration. However that would require a little more work and the
presumption is that this will be shortlived until fission becomes the
default configuration.
Differential Revision: https://phabricator.services.mozilla.com/D106954
Because we use sysroots, this makes no difference to the shipped builds.
This would normally allow to remove the debian8-amd64-build docker image,
but we need to keep it for the valgrind jobs for now.
Differential Revision: https://phabricator.services.mozilla.com/D106404
This adds a linter for Fluent files based upon the existing test for bad
strings in browser_misused_characters_in_strings.js. It also adds a check
for identifiers that only permits lowercase letters, numbers and the
hyphen character (in ascii). Since a large number of existing identifiers
use uppercase letters, an exclusions file is used to disable the identifier
check on a file by file basis.
Differential Revision: https://phabricator.services.mozilla.com/D104414
The 'Bugzilla' task which e.g. creates the mapping of files to Bugzilla
components only runs on mozilla-central since bug 1608421. This means its
failures don't justify a backout and the task should be tier 2 for which
developers get needinfoed and fix the issue in a follow-up. If there is no
action, the changes can still be backed out.
Differential Revision: https://phabricator.services.mozilla.com/D104425
This patch adds a new "--ci" argument for the
"mach puppeteer-test" command. As such it can
also be used locally to simulate a test job in CI.
Differential Revision: https://phabricator.services.mozilla.com/D101780
For simplicity, this implements just on in `NO_TASKS` (the default) or
on in `ALL_TASKS` (opt-in). This disables all category registrations
when in background task mode; we'll selectively re-enable things as
appropriate.
The flag constants were chosen to smoothly extend to a (16-)bit set in
the future, should we want to add a `JUST_TASKS("task", "other-task")`
option in the future.
This also adds ython tests for gen_static_components.py exercising
categories, simply 'cuz it's easiest to see what this adds in such
tests. Functional tests will follow in patches that actually
implement the new background tasks functionality.
Differential Revision: https://phabricator.services.mozilla.com/D96654
Due to font cache issues with the ub18-test docker image in
TaskCluster the first startup of Firefox sometimes takes more
than 15s.
As workaround for now just start Firefox once before running
Puppeteer unit tests.
Differential Revision: https://phabricator.services.mozilla.com/D93110
Due to font cache issues with the ub18-test docker image in
TaskCluster the first startup of Firefox sometimes takes more
than 15s.
As workaround for now just start Firefox once before running
Puppeteer unit tests.
Differential Revision: https://phabricator.services.mozilla.com/D93110
This removes the last uses of the 'push-interval-10' and 'push-interval-20' strategies.
They are being removed because they are dangerous in that its easy to accidentally not run
tasks when they should.
Instead, task authors should decide whether they want their tasks to run on
"backstop" pushes (run everything) or "expanded" pushes (run more than usual,
but still not as much as a backstop). Note that using "expanded" means the task
will *also* run on backstop pushes. It'll just additionally run on "expanded"
pushes.
In practice 'backstop' pushes will be every 20th push and 'expanded' pushes
will be every 10th push. Though this may vary due to the time component in
backstops.
Differential Revision: https://phabricator.services.mozilla.com/D89503
Pipenv is heavy weight and overkill for the purposes it is being used. We'd like to remove it from the tree and |mach python-test| was one of the last remanining use cases.
Remove the `--python` command-line argument as a result. Users who wish to run unit tests with Python 2 can do `MACH_PY2=1 ./mach python-test ...` or `python2 ./mach python-test ...`.
Also update a few unit tests that would break otherwise in the presence of this change.
There were a couple lines in the `setup.py` for `mozlog` that were problematic for tests and was resulting in errors due to the `mozlog` plugin being loaded by `pytest` more than once. We just delete those lines and bump up the major version number of the package to fix it.
Differential Revision: https://phabricator.services.mozilla.com/D88296
It could go into its own test suite, but it 1) depends on `mozbuild` code, so the `mozbuild` suite as well as this new suite would be running on any push that touches `mozbuild` code anyway, and 2) this is code that runs during the build, so it's not out of place.
Differential Revision: https://phabricator.services.mozilla.com/D84547
It could go into its own test suite, but it 1) depends on `mozbuild` code, so the `mozbuild` suite as well as this new suite would be running on any push that touches `mozbuild` code anyway, and 2) this is code that runs during the build, so it's not out of place.
Differential Revision: https://phabricator.services.mozilla.com/D84547
It could go into its own test suite, but it 1) depends on `mozbuild` code, so the `mozbuild` suite as well as this new suite would be running on any push that touches `mozbuild` code anyway, and 2) this is code that runs during the build, so it's not out of place.
Differential Revision: https://phabricator.services.mozilla.com/D84547
We could make a new task for this, but `mozwebidlcodegen` depends on code in `mozbuild`, and vice-versa, so there doesn't really seem to be any meaningful advantage to that.
Differential Revision: https://phabricator.services.mozilla.com/D83187
The unit test is broken under Python 3 on macOS, so I haven't included any macOS version of this task; one should probably be added if/when that's fixed.
Differential Revision: https://phabricator.services.mozilla.com/D83169
Butkovits Atila
Thomas Wisniewski
Mitchell Hentges
Dan Mosedale
ahochheiden
Andrew Halberstadt
Julian Descottes
Joel Maher
Geoff Brown
Sylvestre Ledru
Agi Sferro
Henrik Skupin
Andi-Bogdan Postelnicu
Michelle Goossens
Cosmin Sabou
Andrew Halberstadt
Agi Sferro
Mike Hommey
Iulian Moraru
Alexandru Michis
Mark Banner
Tom Ritter
Aaron Klotz
Sandor Molnar
Myeongjun Go
Marco Castelluccio
Sebastian Hengst
championshuttler
James Graham
Andi-Bogdan Postelnicu
Csoregi Natalia
Dorel Luca
Dan Minor
Nick Alexander
Frederik Braun
Chris H-C
Andrew Erickson
Razvan Maries
Andi-Bogdan Postelnicu
Raphael Pierzina
june wilde
Andrew Halberstadt
Ricky Stewart
Aki Sasaki
Tom Prince
Narcis Beleuzu
Ben Hearsum
Chris AtLee