Граф коммитов

6168 Коммитов

Автор SHA1 Сообщение Дата
Rail Aliiev e10ddefd8a Bug 1642868 - Update verify tests check ja-JP-mac on all platforms now r=nthomas
Differential Revision: https://phabricator.services.mozilla.com/D78844
2020-06-09 03:31:56 +00:00
Gregory Mierzwinski de3c9ae058 Bug 1643685 - Use name instead of test_name when finding the metric test name. r=mozperftest-reviewers,tarek
This patch fixes the test name being used for the App-link test in CI (regression). The patch also adds a new feature to ScriptInfo which checks to make sure that the test doesn't have any extra unknown keys in it. If it does, a failure is raised. This change is to attempt to help prevent these kinds of regressions from happening again. A new test is added for this functinality as well.

Differential Revision: https://phabricator.services.mozilla.com/D78516
2020-06-08 21:02:53 +00:00
Mitchell Hentges ed90a04ad1 Bug 1632874: |mach bootstrap| should write mozconfig file if it doesn't exist r=rstewart
Rather than always printing instructions at the end of the bootstrap phase, we will now create a mozconfig
file if one doesn't exist and there's configuration to be written.

Differential Revision: https://phabricator.services.mozilla.com/D78417
2020-06-08 19:36:56 +00:00
Mitchell Hentges 300422099b Bug 1632429: Improves update-verify compatibility with python 3 r=nthomas
Depends on D77962

Differential Revision: https://phabricator.services.mozilla.com/D77964
2020-06-08 21:07:07 +00:00
Gijs Kruitbosch 6f1da44727 Bug 1643321 - fix missing/differently-identified fluent localization of browser profile migration data types, r=MattN,fluent-reviewers,flod
Differential Revision: https://phabricator.services.mozilla.com/D78412
2020-06-08 12:00:38 +00:00
Tarek Ziadé 7489c5233a Bug 1642779 - implement the 'androidlog' test r=acreskey,sparky
Add a layer that scans the android logcat

Differential Revision: https://phabricator.services.mozilla.com/D78268
2020-06-08 10:47:52 +00:00
Yue Zhang fcaa3e19c7 Bug 1639692 - Re-enable custom transform discovery. r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D76653
2020-06-05 21:09:32 +00:00
Francesco Lodolo (:flod) bd81f0ef27 Bug 1642557 - Remove Fluent migration recipes for Firefox 76 r=Pike
Differential Revision: https://phabricator.services.mozilla.com/D77767
2020-06-05 15:57:48 +00:00
Gregory Mierzwinski 7a9ed37810 Bug 1643103 - Fix how system layers work with the test layers. r=mozperftest-reviewers,tarek
This patch fixes how the system and browser/test layers are run. With this fix, the system layer no longer fully runs its setup and teardown stages before the browser layer has started and finished. Now the setup/teardown happens before/after the full test layer run.

Depends on D78016

Differential Revision: https://phabricator.services.mozilla.com/D78128
2020-06-05 13:42:14 +00:00
Tarek Ziadé 18f9ccd8f2 Bug 1642615 - Renamed the browser layer to test r=sparky
This patch renames the browser layer to test, since
it could be something else than a browser

Differential Revision: https://phabricator.services.mozilla.com/D78016
2020-06-04 18:31:54 +00:00
Tarek Ziadé 9bcf786abb Bug 1643621 - add moz.build r=Bebe
forgot to add this file, so mozperftest docs are added in the sphinx sparse profile

Differential Revision: https://phabricator.services.mozilla.com/D78465
2020-06-05 09:46:15 +00:00
Mitchell Hentges a8304147a8 Bug 1643158: remove mozboot/util.py dependency on six r=rstewart
bootstrap.py depends on mozboot/util.py, but bootstrap needs to run in a bare environment without `six`.

Differential Revision: https://phabricator.services.mozilla.com/D78352
2020-06-04 18:59:51 +00:00
Valentin Gosu 9e9b45920a Bug 1643370 - Add POP!_OS to supported debian distros r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D78316
2020-06-04 15:45:15 +00:00
Andi-Bogdan Postelnicu affacf5510 Bug 1639406 - for `code-review` generate the propper patch when dealing with `clang-format` issues. r=marco
Differential Revision: https://phabricator.services.mozilla.com/D78060
2020-06-04 08:53:02 +00:00
emcminn de56c3d325 Bug 1637617 - Adjusted message margins to prevent panel closing; added panel header to main view panel & updated fluent strings r=andreio,fluent-reviewers,flod
Differential Revision: https://phabricator.services.mozilla.com/D76162
2020-06-03 18:25:02 +00:00
John Paul Adrian Glaubitz e1b5790f4d Bug 1325771 - build: Add m68k as target architecture to mozbuild r=glandium
Adds the basic definitions for m68k to mozbuild, allowing to build Spidermonkey.

Differential Revision: https://phabricator.services.mozilla.com/D77285
2020-06-03 18:30:50 +00:00
Ricky Stewart 1914a51fc8 Bug 1643026 - Squash stray reference to deleted global variable in virtualenv.py r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D78069
2020-06-03 17:23:30 +00:00
Mitchell Hentges 651dec64b7 Bug 1632429: Makes mozrelease compatible with python 3 r=aki
Differential Revision: https://phabricator.services.mozilla.com/D77962
2020-06-03 18:16:09 +00:00
Ed Lee 96ed60b4bd Bug 1641863 - Upgrade Node 10.x (in preparation for ESR 78) r=Standard8. CLOSED TREE
Node v10.21.0 (LTS) just released https://nodejs.org/en/blog/release/v10.21.0/

Differential Revision: https://phabricator.services.mozilla.com/D77904
2020-06-03 18:00:52 +00:00
Tom Prince 9eac413f54 Bug 1642446: Add an attribute to annotate toolchain tasks used by `mach bootstrap`; r=firefox-build-system-reviewers,rstewart
Toolchains that are used for local development need to be built on a level-3
branch to installable via `mach bootstrap`. Add an attribute to track the fact
that a toolchain is used that way, and:
- ensure that everything installed via `mach boostrap` has that attribute set
- ensure that everything with that attribute set is built on trunk projects

We could additionally verify that attribute is only set on things used by
bootstrap, but bootstrap doesn't currently have an exhaustive list of things
that it might install, making that difficult.

Differential Revision: https://phabricator.services.mozilla.com/D77706
2020-06-03 15:55:26 +00:00
Dorel Luca 0154691eae Backed out 5 changesets (bug 1325771) for Spidermonkey failures and build bustage in build/src/mfbt/tests/TestCompactPair.cpp. CLOSED TREE
Backed out changeset ace40545b46c (bug 1325771)
Backed out changeset b3e0fb410a1c (bug 1325771)
Backed out changeset d69ac62c063f (bug 1325771)
Backed out changeset 24e6299e112d (bug 1325771)
Backed out changeset 393a6ff847ce (bug 1325771)
2020-06-03 20:23:08 +03:00
John Paul Adrian Glaubitz 25481ca65b Bug 1325771 - build: Add m68k as target architecture to mozbuild r=glandium
Adds the basic definitions for m68k to mozbuild, allowing to build Spidermonkey.

Differential Revision: https://phabricator.services.mozilla.com/D77285
2020-06-02 07:00:48 +00:00
xuanqi xu 33b538ca48 Bug 1636534 - Added test for notebook layer and perftestnotebook. r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D77386
2020-06-02 21:35:03 +00:00
xuanqi xu 2b9bf62319 Bug 1636534 - Separate perftestetl from perftestnotebook r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D75684
2020-06-02 21:34:49 +00:00
Tarek Ziadé 7c486e2ce0 Bug 1641834 - started a Sphinx doc r=sparky,acreskey
Initial Sphinx structure with minimal content.
This is mostly to get started in writing doc

Differential Revision: https://phabricator.services.mozilla.com/D77440
2020-06-03 08:17:06 +00:00
Bogdan Tara 979c77d94e Backed out changeset 83ee79caab92 (bug 1622963) for python related bustages CLOSED TREE 2020-06-03 10:23:57 +03:00
Mike Hommey 323ea73e5a Bug 1622963 - Require python 3.6 for the build system. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D77754
2020-06-03 02:16:48 +00:00
Tom Prince 581eba974b Bug 1641971: Pass `mach perftest` options explicitly, rather than inpsect parameters; r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-03 02:35:09 +00:00
Narcis Beleuzu 479c45bbea Backed out 7 changesets (bug 1641971) for breaking Gecko Decision Task opt . CLOSED TREE
Backed out changeset a039e60a26d2 (bug 1641971)
Backed out changeset e7d4c490c31a (bug 1641971)
Backed out changeset f2322981b294 (bug 1641971)
Backed out changeset d123b08d2c20 (bug 1641971)
Backed out changeset 57772c0b9b3e (bug 1641971)
Backed out changeset 11a887f73dbb (bug 1641971)
Backed out changeset e9c7a59d4bc5 (bug 1641971)
2020-06-03 05:30:51 +03:00
Tom Prince 11aedca9b2 Bug 1641971: Pass `mach perftest` options explicitly, rather than inpsect parameters; r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-02 18:23:07 +00:00
Andy Grover 95b90fb23e Bug 1633006 - Update to Neqo 0.4.0 r=dragana,necko-reviewers
Document qlog uses 2 clause bsd license.

Adapt to API changes in glue and http3server

Differential Revision: https://phabricator.services.mozilla.com/D77903
2020-06-02 21:28:31 +00:00
Riley Byrd ad82a96dad Bug 1518234 - Migrate migration wizard to Fluent r=flod,Gijs,fluent-reviewers
Based on a patch originally written by Ian Kirkpatrick <kirkpa47@msu.edu>

Differential Revision: https://phabricator.services.mozilla.com/D17393
2020-06-02 06:46:09 +00:00
Ricky Stewart d2861db731 Bug 1642439 - Fix test_error_output.py r=froydnj
These variables were renamed in bug 1641992, and furthermore they were changed to "templates" (with inline `%s` for string formatting), so fix the tests with that in mind.

Differential Revision: https://phabricator.services.mozilla.com/D77697
2020-06-01 19:33:29 +00:00
Emily McDonough c6e4f99f13 Bug 1642040 - Add VoidLinux support to mach bootstrap r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D77571
2020-06-01 19:37:06 +00:00
Ricky Stewart 7ed5e2165a Bug 1642418 - Fix mach completion unit test by updating bad conditional r=froydnj
I misinterpreted one of the conditionals when working on bug 1641991 (the "not X or Y" formulation confused me :( ). The correct fix here is to remove this conditional which isn't relevant any more after we removed all the old pre-`mochitest` mach commands.

Differential Revision: https://phabricator.services.mozilla.com/D77692
2020-06-01 18:59:46 +00:00
Ricky Stewart 7bee84802f Bug 1641992 - Add argument to `mach busted file` to specify which component the bug should be filed against r=froydnj
With the addition of this change, a lone `mach busted file` will throw an error (since this should only ever be used by actual humans, and not in automation or anything, the backwards-compatibility breakage isn't a huge deal). Now it's expected to pass in `mach busted file $COMMAND`, where $COMMAND is the mach command that you were running when the error occurred, and we'll figure out which bugzilla component to file the bug against for you instead of directing it to `Firefox Build System :: General` regardless of whether the issue has *anything* to do with the build system. We preserve `mach busted file general` as a backup command that doesn't do any of this heavy logic.

Differential Revision: https://phabricator.services.mozilla.com/D77538
2020-06-01 16:23:48 +00:00
Ricky Stewart 7f3fa0551b Bug 1641991 - Remove deprecated mochitest mach commands r=ahal
These commands were removed 5 years ago (with the exception of robocop, which was removed about a year ago). We're way past the point where anyone would glean useful info from keeping the stubs here with an error message.

Differential Revision: https://phabricator.services.mozilla.com/D77536
2020-05-29 20:43:05 +00:00
Ricky Stewart 0bbaac721b Bug 1641693 - Replace a bunch of uses of `GENERATED_FILES` with the `GeneratedFile` template r=necko-reviewers,geckoview-reviewers,aklotz,dragana,froydnj
Also update documentation to suggest using the `GeneratedFile` template rather than directly referencing `GENERATED_FILES` where possible.

Differential Revision: https://phabricator.services.mozilla.com/D77496
2020-06-01 16:00:28 +00:00
Mike Hommey 9088528947 Bug 1635764 - Move --with-system-icu and --with-intl-api to python configure. r=firefox-build-system-reviewers,rstewart
At the same time, because it's now simpler to do so, set the right data
file name for big-endians, even though we don't have or produce it
(bug #1264836). Also remove USE_ICU, which is redundant with
JS_HAS_INTL_API, and actively break the build at configure time when
using --without-intl-api with Firefox because this hasn't actually
worked for close to 3 years (since bug 1402048).

Differential Revision: https://phabricator.services.mozilla.com/D77416
2020-05-29 21:29:52 +00:00
Mitchell Hentges 811fbf178f Bug 1640064: Don't attempt to install python-pip (py2) on Debian Sid r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D77517
2020-05-29 20:17:40 +00:00
Mitchell Hentges 1b525e3c0c Bug 1640815: Raise helpful error if no non-public commits on bootstrap r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D77505
2020-05-29 18:35:35 +00:00
Ricky Stewart 7a15c311ac Bug 1639313 - Print usage if you call a bare `mach vendor`, `mach cargo`, or `mach remote` r=remote-protocol-reviewers,froydnj,whimboo
The architecture of `mach` does not make this easy to do from within one process, not least of all due to persistent global, mutable state. :( There may be another way to do this, I'm not saying it's completely impossible (and obviously with a significant enough refactor there's a way to make this happen), but subprocessing is a foolproof way to accomplish the same.

Differential Revision: https://phabricator.services.mozilla.com/D77374
2020-05-29 16:16:31 +00:00
Gregory Mierzwinski 55c95e297a Bug 1637994 - Add a transformer for extracting times/durations from logcat logs. r=acreskey,mozperftest-reviewers,tarek
This patch adds a transformer that is capable of extracting times/entries or durations for logcat logs. Here, we also implement the ability for users to pass options to the transformers so that they can be more generic (through transformer-options). The logcat transformer makes use of this, and also shows how a custom function can be used within transformers. A new test file is added for the transformer, as well as a test in test_perfherder.py so we can test the full pipeline.

Differential Revision: https://phabricator.services.mozilla.com/D77094
2020-05-29 12:40:40 +00:00
Gregory Mierzwinski 115d1b4294 Bug 1641566 - Update in-tree browsertime to latest. r=tarek,perftest-reviewers,Bebe
Differential Revision: https://phabricator.services.mozilla.com/D77257
2020-05-29 12:24:33 +00:00
Tarek Ziadé 3c6bbe84ba Bug 1641549 - Run the metrics layer separately r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D77292
2020-05-29 12:21:52 +00:00
Tarek Ziadé 77da205d14 Bug 1640627 - added android logs options r=sparky
--android-capture-logcat  - captures the logcat
--android-clear-logcat - clears the logcat on start
--android-capture-adb - captures the adb commands

Differential Revision: https://phabricator.services.mozilla.com/D77038
2020-05-28 16:42:37 +00:00
Nils Ohlmeier [:drno] 18ef0cb66a Bug 1641289: Bye, bye Steeplechase. r=dminor,froydnj
Differential Revision: https://phabricator.services.mozilla.com/D77114
2020-05-29 03:19:27 +00:00
Ricky Stewart d620b9e6a6 Bug 1641242 - Handle difference in API of ImportError between py2 and py3 in artifact_commands.py r=catlee
Differential Revision: https://phabricator.services.mozilla.com/D77337
2020-05-28 20:33:03 +00:00
Axel Hecht 1343d506c0 Bug 1641184, use only the merge dir for repacks, r=nalexander
l10n-merge creates a full merge dir for a while now, let's
simplify the build logic to only read from that directory
during repacks and langpacks.

Differential Revision: https://phabricator.services.mozilla.com/D77023
2020-05-27 17:04:06 +00:00
Tarek Ziadé c0c06d6f1f Bug 1641351 - raise on node error in BT r=acreskey
We forgot to check that the node command succeeded.
This patch fixes it.

Differential Revision: https://phabricator.services.mozilla.com/D77148
2020-05-27 21:43:38 +00:00
Geoff Brown 21053c0624 Bug 1638965 - Run 'mach gtest' in python 3; r=bc
Differential Revision: https://phabricator.services.mozilla.com/D77092
2020-05-27 20:53:22 +00:00
Artem 30dba11bdd Bug 1609556 - Migrate toolbar-context-menu to Fluent r=Gijs,fluent-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D76016
2020-05-27 18:09:40 +00:00
Mitchell Hentges 0482fe58ed Bug 1641073: improve BaseFile.copy() performance r=rstewart
Arrays are mutable, so appending each substring at a time and joining at the endis much faster than re-allocating
a new string on each loop.

Differential Revision: https://phabricator.services.mozilla.com/D76945
2020-05-27 16:52:51 +00:00
Ricky Stewart 1d10fc6783 Bug 1639624 - Don't import from `distutils` unless absolutely necessary in virtualenv.py r=froydnj
Similar reasoning to bug 1638780.

Differential Revision: https://phabricator.services.mozilla.com/D76221
2020-05-27 15:07:47 +00:00
Mike Hommey a97c7ebfe7 Bug 1640578 - Remove --disable-install-strip from mac mozconfigs. r=froydnj
The need for --disable-install-strip in the mac mozconfigs comes from a
discrepancy in how stripping is handled between platforms. On Windows,
there is no stripping. On non-Mac unix, `strip` removes local symbols as
well as debug info and debug symbols. On Mac, it actually removes too
much, and one has to pass flags to remove both local symbols (`-x`) and
debug symbols (`-S`). Debug info is already in a separate file
(`.dSYM`).

For profiling reasons, we do ship e.g. nightlies with local symbols but
not debug info or symbols (or at least that's the intent). On Windows,
again, nothing to do. On non-Mac unix, we pass `--strip-debug` to
`strip` so that it keeps local symbols. That's where the discrepancy
comes in for Mac: the build system doesn't handle this at all, so the
mozconfigs contain --disable-install-strip to avoid stripping.

The build system should be doing what it's expected to be doing from the
start, without mozconfigs opting into anything.

AFAIK, we only really need the local symbols, so we can `strip -S` on
Mac when profiling is enabled, rather than `strip -x -S`. This also
significantly reduces the size of the installer for nightlies.

And while we're here, move the logic out of old-configure and into
python configure.

Differential Revision: https://phabricator.services.mozilla.com/D76789
2020-05-27 01:42:07 +00:00
Andi-Bogdan Postelnicu 622cf2a71a Bug 1640878 - for `clang-tidy` static-analysis keep a deep copy of the checks. r=froydnj
`StaticAnalysisMonitor` uses `clang-tidy` config dictionary in order to
match different options for the checkers that are being used. For this `StaticAnalysisMonitor`
modifies this dictionary. The dictionary not being a deep copy of the original one, the
modifications are echoed in the initial dictionary thus having the potential to lead to
unexpected behavior from `clang-tidy` when generating the checkers list.

Differential Revision: https://phabricator.services.mozilla.com/D76834
2020-05-26 17:23:38 +00:00
egao c37762fb9f Bug 1640716 - correct string substitution in mozbuild/frontend/reader.py r=jmaher
Changes:

Current in-tree code will simply print `{path}` as many times as it reads a path.

This change will ensure that paths that have been read are output to the screen, which is much more useful.

Differential Revision: https://phabricator.services.mozilla.com/D76729
2020-05-26 09:57:17 +00:00
Tarek Ziadé bddb8551ed Bug 1640747 - Add test-level support for handling exceptions while running the tests r=acreskey
This patch adds the `on_exception` hook, that will let developers
drive the runner behavior when a layer fails.

Differential Revision: https://phabricator.services.mozilla.com/D76794
2020-05-26 13:35:00 +00:00
Tarek Ziadé 92a2c142f7 Bug 1640649 - add --android-timeout r=sparky
This patch surfaces the timeout value for ADBDevice.
It also adds the ability to run a single test with
mach perftest-test and a new --skip-lint option
to skip black/flake8

Differential Revision: https://phabricator.services.mozilla.com/D76791
2020-05-26 14:42:58 +00:00
Tarek Ziadé af3ea56177 Bug 1639321 - increase the coverage to 85 r=sparky
Increase the coverage to 85%

Differential Revision: https://phabricator.services.mozilla.com/D76048
2020-05-26 14:44:14 +00:00
Gregory Mierzwinski f94f60a9c2 Bug 1640171 - Make the test file a str before using it. r=mozperftest-reviewers,tarek
This patch fixes a bug where a PosixPath is attempted to be used rather than a string path. There's also a regression in the coverage test run in `./mach perftest-test` on some platforms that is fixed by this patch (the attrs module being used by pytest is too old, so we update during setup). Lastly, a regression test is added to the browsertime tests to ensure that the command only contains knowingly compatible types.

Depends on D76713

Differential Revision: https://phabricator.services.mozilla.com/D76599
2020-05-25 20:07:58 +00:00
Geoff Brown 9e26d9583b Bug 1636797 - Purge __pycache__ in 'mach clobber python'; r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D76560
2020-05-23 19:37:09 +00:00
Gijs Kruitbosch cb82792e02 Bug 1626842 - migrate the browser window title to fluent, r=zbraniecki,fluent-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D75854
2020-05-25 10:01:42 +00:00
Tarek Ziadé d634c1741b Bug 1639493 - improved README r=acreskey
Added some minimal doc

Differential Revision: https://phabricator.services.mozilla.com/D76107
2020-05-20 13:12:15 +00:00
Tilden Windsor b7355730e6 Bug 1640142 - Added spaces to "mach clang-format" help message. r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D76467
2020-05-22 11:53:48 +00:00
Ricky Stewart 6cda2c27f5 Bug 1640261 - Expand set of possible Python 3's you can use in bootstrap/debian.py r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D76529
2020-05-22 19:34:08 +00:00
Ricky Stewart e2fe57521b Bug 1638060 - Standardize interface of `mozfile` classes as `bytes`-based streams r=glandium
At the beginning of the Python 3 migration (circa bug 1602540), we made an update to the interface of `mozpack/files.py` in the direction of aligning with Python 3's built-in `file` support; namely, that opening a file in text mode returns a stream of `str` (text), and that opening a file in binary mode returns a stream of `bytes`. This was deemed to be more trouble than it was worth. This patch undoes all of those changes to the interface in favor of moving back to the Python 2 style, where all files are bytestreams.

Differential Revision: https://phabricator.services.mozilla.com/D75424
2020-05-22 01:11:29 +00:00
Andrew Halberstadt 0591e31c1f Bug 1639349 - [ci] Set up a SCHEDULES rule to exclude non-webrender tests on webrender-only pushes, r=gbrown
Depends on D76433

Differential Revision: https://phabricator.services.mozilla.com/D76434
2020-05-22 14:03:46 +00:00
adiabtaic28394 43294f5124 Bug 1632345 - mach doctor does not complain about noatime/reltime on tmpfs anymore r=Callek,glandium
Differential Revision: https://phabricator.services.mozilla.com/D75917
2020-05-22 13:36:42 +00:00
Mike Hommey 6ba1065508 Bug 1639815 - Move --enable-strip and --enable-install-strip to python configure. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D76291
2020-05-21 23:51:58 +00:00
Ricky Stewart 0a8ac76fef Bug 1639986 - Add hack to bootstrap to install Python 2 on macOS r=firefox-build-system-reviewers,nalexander
This is ugly and complicates the code some but it's manageable and allows us to keep things afloat on macOS while the testing team plugs along with the `mach` migration.

Differential Revision: https://phabricator.services.mozilla.com/D76386
2020-05-21 20:25:34 +00:00
Chris AtLee cd35328624 Bug 1637381: Add support for extracting zst files to mozbuild tooltool r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D75203
2020-05-21 13:28:33 +00:00
Chris AtLee ef685f3549 Bug 1637379: Optimize android toolchains r=gbrown
Create an emulator toolchain
Stop including mozboot/sdk-*.zip in our tarball

Differential Revision: https://phabricator.services.mozilla.com/D74931
2020-05-21 13:28:10 +00:00
Cameron McCormack 252e4e5c55 Bug 1639788 - Update IRC reference in mach bootstrap output. r=glob
Differential Revision: https://phabricator.services.mozilla.com/D76271
2020-05-21 05:39:24 +00:00
Csoregi Natalia 160736caed Backed out changeset 4376d47f325b (bug 1639624) for bustage. CLOSED TREE 2020-05-21 03:58:54 +03:00
Ricky Stewart 5aefcd726b Bug 1639624 - Don't unconditionally import `VirtualenvManager` in `python/mach_commands.py`. r=froydnj
Similar reasoning to bug 1638780.

Differential Revision: https://phabricator.services.mozilla.com/D76221
2020-05-20 23:32:43 +00:00
Dan Minor fdcf957cca Bug 1634675 - Remove webrtc-gtests r=drno,froydnj
Differential Revision: https://phabricator.services.mozilla.com/D73451
2020-05-20 19:59:45 +00:00
Ricky Stewart e8d734c549 Bug 1639521 - Fix Py3 incompatibility in `mach artifact` r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D76215
2020-05-20 20:34:51 +00:00
Yue Zhang d8d83da843 Bug 1636535 - Remove subclass requirement for custom data transformers. r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D75394
2020-05-20 19:48:58 +00:00
Alex Vincent 449dbae8b1 Bug 1588166 - Remove python-dbus build dependency. (notify-send replaced it and it's no longer needed.) r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D74798
2020-05-20 15:19:13 +00:00
Ricky Stewart 1a44b2a1d9 Bug 1638983 - Run `mach repackage` with Python 3 r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D76058
2020-05-20 15:35:58 +00:00
Tarek Ziadé f8d5627e71 Bug 1639212 - allow URLs for hooks and scripts r=sparky
This patch allows using URLS for --hooks and scripts

Differential Revision: https://phabricator.services.mozilla.com/D75967
2020-05-19 20:27:27 +00:00
Gregory Mierzwinski 7788d4b576 Bug 1639211 - Implement --perfherder-app in mozperftest Perfherder layer. r=mozperftest-reviewers,tarek
This patch adds a new --perfherder-app that can be used to specify the shorthand name of the application being tested. The choices here are limited to what the perfherder schema can accept. We also add these settings to the existing taskcluster tasks to split the perfherder data. A couple tests are also added for this new flag.

Differential Revision: https://phabricator.services.mozilla.com/D76010
2020-05-19 19:07:27 +00:00
Ricky Stewart aefac46494 Bug 1638951 - Run `mach cargo` with Python 3 r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D75999
2020-05-19 19:35:52 +00:00
Gregory Mierzwinski 99953d32a0 Bug 1638001 - Add test name from metadata to perfherder data. r=acreskey,mozperftest-reviewers,tarek
This patch adds a `test_name` field to the test metadata that can be used to change the name of the test when output in the perfherder format. It also implements this for the app-link tests.

Differential Revision: https://phabricator.services.mozilla.com/D75544
2020-05-19 19:52:52 +00:00
Tarek Ziadé 7d6014b5a8 Bug 1635573 - vendor coverage r=ahal
Vendor coverage, and make sure we can run out of third_party as much as possible

Differential Revision: https://phabricator.services.mozilla.com/D74531
2020-05-19 15:04:28 +00:00
Ricky Stewart 57d020b2e6 Bug 1638780 - Don't import `virtualenv` unless necessary in `mach` commands r=froydnj
The module `distutils.sysconfig` isn't installed by default on Ubuntu -- the package `python3-distutils` exposes it. That's fine, except we unconditionally import it in `virtualenv.py`, which gets up indirectly being imported whenever you run any `mach` command, which will cause `bootstrap` to break before it even has the chance to install it.

`python3-pip` seems to rely on `python3-distutils` being installed so `bootstrap` will install it, so all we need to do is make sure it doesn't import `virtualenv` in any circumstance unless it's necessary (when surfacing an error in the module would be appropriate).

Differential Revision: https://phabricator.services.mozilla.com/D75833
2020-05-19 12:07:47 +00:00
Nathan Froyd 9f876bab7a Bug 1638012 - commonize taskcluster-related bootstrappers for Linux; r=nalexander
This change doesn't fix all of the boilerplate involved in declaring
that certain packages should be fetched from taskcluster, but it's a
start, at least.

Differential Revision: https://phabricator.services.mozilla.com/D75330
2020-05-18 16:27:09 +00:00
Andi-Bogdan Postelnicu 86f93ce272 Bug 1635112 - Mark `clang-diagnostic-error` as `publishable` in the `clang-tidy` artifact. r=marco
For `clang-tidy` static-analysis mark all diagnostic-messages that are `clang-diagnostic-error` as publishable.

Differential Revision: https://phabricator.services.mozilla.com/D74229
2020-05-08 07:19:51 +00:00
Andi-Bogdan Postelnicu 0b18f7ab13 Bug 1635112 - Disable the build error reporting at review-phase using `Coverity`. r=sylvestre,marco
Differential Revision: https://phabricator.services.mozilla.com/D74003
2020-05-08 09:30:02 +00:00
Andi-Bogdan Postelnicu ac1e721802 Bug 1635112 - Use `clang-tidy` as warning build error detector for our `static-analysis` targets. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D73692
2020-05-08 07:19:23 +00:00
Andi-Bogdan Postelnicu 7a23d681c5 Bug 1635112 - remove MSVC support from `WarningsCollector`. r=glandium
MSVC is not supported by the build system anymore.

Differential Revision: https://phabricator.services.mozilla.com/D73809
2020-05-08 07:19:02 +00:00
Rob Lemley bd812fce54 Bug 1631197 - Make mach valgrind-test work for Thunderbird builds. r=froydnj
Create a new MachCommandCondition, "is_firefox_or_thunderbird" which is then used
to allow mach valgrind-test work for Thunderbird builds.

Differential Revision: https://phabricator.services.mozilla.com/D73153
2020-05-19 00:10:06 +00:00
Ricky Stewart ab8348bc51 Bug 1638783 - Install Mercurial with pip3 rather than pip, which no longer exists on new Ubuntu/Debian versions r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D75816
2020-05-18 20:10:19 +00:00
Corentin Arnould e7a750541a Bug 1638799 - Added `python-pip` to archlinux's bootstrap. r=froydnj
pip3 is an optional dependency of python on archlinux. It needs to be installed explicitly.

Differential Revision: https://phabricator.services.mozilla.com/D75785
2020-05-18 12:50:54 +00:00
Dão Gottwald 379b7cff73 Bug 1638362 - Rename urlbar-go-end-cap l10n id to urlbar-go-button. r=fluent-reviewers,flod
Differential Revision: https://phabricator.services.mozilla.com/D75558
2020-05-15 16:08:22 +00:00
Tarek Ziadé ffc7bb2c00 Bug 1637959 - Added more test coverage r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D75309
2020-05-15 07:19:39 +00:00
Mike Hommey 28de21d935 Bug 1638193 - Remove build system support for iOS. r=nalexander
iOS support for Gecko has not been tested in years and is most probably
out of date. The build system part of it, specifically the checks in
build/autoconf/ios.m4, are not trivial to port to python configure, and
they prevent other things from moving to python configure (because some
of them change value when MOZ_IOS is set).

The code is left alone, although it could probably be stripped off as
well, but I'll leave that as an exercise for someone else.

Differential Revision: https://phabricator.services.mozilla.com/D75463
2020-05-15 03:56:16 +00:00
Ricky Stewart 5af71702d3 Bug 1637709 - Configure purge extension when calling `hg purge` r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D75328
2020-05-14 18:14:11 +00:00
Ricky Stewart 69ecad92c6 Bug 1579329 - Run `mach file-info` with Python 3 r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D75343
2020-05-14 18:20:22 +00:00
Marco Castelluccio d845cf5481 Bug 1637911 - Skip modules with a None __file__ attribute. r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D75267
2020-05-14 18:28:55 +00:00
Nathan Froyd 142b960302 Bug 1637737 - cleanup test failures from `mach run` JS shell changes; r=firefox-build-system-reviewers,rstewart
In bug 1637665, we changed the OS X codepath for `mach run` to consult
`MOZ_WIDGET_TOOLKIT` as a substitute for "browserness".  Unfortunately,
this caused tests that only set `MOZ_MACBUNDLE_NAME` to fail.  Rather
than set `MOZ_WIDGET_TOOLKIT` in those tests, we probably should have
been checking `MOZ_MACBUNDLE_NAME` in the first place, since
`MOZ_MACBUNDLE_NAME` is also set only for the browser and not for the JS
shell.  We can also remove the xulrunner codepath, as xulrunner was
removed from the tree several years ago.

Differential Revision: https://phabricator.services.mozilla.com/D75315
2020-05-14 14:58:47 +00:00
Tarek Ziadé e5c6044cc3 Bug 1636179 - switch mpu to perftest-test r=sparky
This patch will use perftest-test on any code change in python/mozperftest

Differential Revision: https://phabricator.services.mozilla.com/D75080
2020-05-14 13:01:59 +00:00
Tarek Ziadé c84c8756b8 Bug 1637231 - add applink tests r=acreskey
Adds tests for applink for gve/fenix on p2/g5

Differential Revision: https://phabricator.services.mozilla.com/D75097
2020-05-14 13:01:59 +00:00
Nicklas Boman 7c411a044b Bug 1637721 - openSUSEs bootstrap does not download the stackwalk toolchain r=firefox-build-system-reviewers,rstewart
In Bug1127565 bootstrap was given support for the openSUSE Linux distribution
Fetching the Stackwalk toolchain was not added unfortunately

his bug1636797 was hit during testing...
(also fixed some formatting "issues")

Differential Revision: https://phabricator.services.mozilla.com/D75189
2020-05-13 21:38:07 +00:00
Nathan Froyd 35647659d4 Bug 1637665 - make `mach run` work with the JS shell on OS X; r=firefox-build-system-reviewers,rstewart
We need to skip the xulrunner/browser code (do we even support xulrunner
anymore?) for `--enable-application=js` builds.  `MOZ_WIDGET_TOOLKIT` is
used elsewhere to check for "browserness".

Differential Revision: https://phabricator.services.mozilla.com/D75140
2020-05-13 16:55:55 +00:00
Julian 92345feaaa Bug 1631122 - Convert URL bar dtd strings in browser.xhtml to fluent. r=fluent-reviewers,flod,Gijs
Differential Revision: https://phabricator.services.mozilla.com/D72467
2020-05-13 15:13:08 +00:00
Ricky Stewart 8bfe989824 Bug 1634391 - Include more specific instructions than "consult a build peer" in Mozbuild comments/error messages r=froydnj
The official decision appears to be that we want people to ask questions in the build channel on chat.mozilla.org for queries that require build peer approval, as opposed to emailing specific people directly, filing bugs, etc. Rather than the vague "consult a build peer" suggestion currently in the code, specify exactly what we expect people to do in mozbuild.

Differential Revision: https://phabricator.services.mozilla.com/D74963
2020-05-13 13:40:53 +00:00
Yue Zhang 740c6c3ab7 Bug 1630610 - Add tests for transformers/single_json.py. r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D74554
2020-05-13 00:29:36 +00:00
Yue Zhang 44c0a36b80 Bug 1630610 - Add tests for perftestnotebook.py and transformer.py. r=sparky
Depends on D73926

Differential Revision: https://phabricator.services.mozilla.com/D73927
2020-05-13 12:08:40 +00:00
Yue Zhang a8fc57d316 Bug 1630610 - Add setup function for pytest. r=sparky
Depends on D73154

Differential Revision: https://phabricator.services.mozilla.com/D73926
2020-05-13 00:29:16 +00:00
Yue Zhang 118aa2368b Bug 1630610 - Fix broken code. r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D73154
2020-05-13 00:29:08 +00:00
Razvan Maries 123741c4e8 Backed out changeset 5694a728839e (bug 1636251) for build bustages. CLOSED TREE 2020-05-13 02:19:46 +03:00
Mitchell Hentges 2646aa4245 Bug 1636251: report |./mach| errors with Sentry r=rstewart
These errors are reported to the "mach" project here:
https://sentry.prod.mozaws.net/operations/mach/

Should only report exceptions caused by a failure in `mach` or its subcommands. Build/test/etc failures
should not be sent to Sentry.

Differential Revision: https://phabricator.services.mozilla.com/D74738
2020-05-12 21:11:41 +00:00
Tarek Ziadé 734633e6c5 Bug 1637315 - Plug flake8 r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D74886
2020-05-12 16:57:27 +00:00
Tarek Ziadé 080870795e Bug 1635136 - add a --push-to-try option r=aerickson
This patch is adding an option to push a perftest run in the CI.

It's based on :
- sparse profiles
- push_to_try
- options passed through try_task_config.json

Differential Revision: https://phabricator.services.mozilla.com/D74115
2020-05-12 21:19:48 +00:00
Tarek Ziadé c1ea70b2d6 Bug 1636871 - allow URLs and aliases for --android-install-apk r=sparky
This patch let us use urls for that option

Differential Revision: https://phabricator.services.mozilla.com/D74629
2020-05-12 17:01:17 +00:00
Gregory Mierzwinski b563884d06 Bug 1637303 - Add perfherder data validation to mozperftest. r=mozperftest-reviewers,tarek
This patch enables perfherder data validation with jsonschema and also adds some extra tests for the Perfherder layer.

Differential Revision: https://phabricator.services.mozilla.com/D74877
2020-05-12 18:06:15 +00:00
Jon Bauman 2bec40dcfe Bug 1637144 - Syntax for different moz.build paths is undocumented. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D74799
2020-05-12 17:33:47 +00:00
Ricky Stewart 492cfe7265 Bug 1637333 - Ensure we don't pass bad environment variables to `subprocess.check_call()` in `mozversioncontrol` r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D74888
2020-05-12 16:58:21 +00:00
Andrew Halberstadt c44fed4839 Bug 1637305 - [mozversioncontrol] Don't use bytes in HGRepository.working_directory_clean, r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D74880
2020-05-12 16:49:24 +00:00
Ricky Stewart 853de17575 Bug 1633494 - Augment `hg` command used to get commit time r=glandium
`hg` gives us some additional levers to tweak when determining which commits are public and non-public. All the better, because `hg`'s default behavior under a rebase is to preserve the original commit time, so if you rebase a month-old commit on the latest version of central, you might still get the "revision too old" error despite your tree actually being completely up-to-date. Here we replace the original logic with something a little bit more robust to that scenario.

As before, this is a best-effort check that might have false positives or false negatives; this commit doesn't change that, but hopefully it decreases the amount of false positives under `hg`.

Differential Revision: https://phabricator.services.mozilla.com/D74745
2020-05-12 15:53:14 +00:00
Ricky Stewart f46e54c22a Bug 1636797 - Improve error message when artifacts cannot be downloaded r=froydnj
The current error message leaves you with basically no recourse besides filing a bug if you're already at the latest HEAD. Meanwhile, `mach clobber` will fix it but in doing so you're taking a very blunt sledgehammer to the problem. Instead, I've updated this error message to tell you you can `mach clobber python`. I also removed the explicit reference to "artifact builds" because you can encounter this error outside of artifact builds as well. Finally, I added another reminder that `mach bootstrap` and `mach artifact` don't work for old revisions of central because I keep getting bugs about it and more screaming about how it's unsupported can't hurt.

Differential Revision: https://phabricator.services.mozilla.com/D74732
2020-05-11 20:44:30 +00:00
Mitchell Hentges b8108d6064 Bug 1632429: Migrate |./mach try| to python 3; r=rstewart,ahal
|./mach try| subcommands are now compatible with both python 2 and 3.

Hand-tested with many combinations of subcommand and subcommand flags.

Updates tryselect unit tests to use Python 3.

Differential Revision: https://phabricator.services.mozilla.com/D73398
2020-05-06 22:17:14 +00:00
Andrew Creskey 5276336dcf Bug 1634516 Extend the android applink tests to cover all variations of fenix r=tarek,sparky
Adds support for all variations of fenix (app name and activity passed by argument to mach perfest)
Fixes bug in mach perftest argument passing (android-activity was not being set)

Differential Revision: https://phabricator.services.mozilla.com/D73334
2020-05-11 14:30:01 +00:00
Nicklas Boman a9633edb83 Bug 1127565 - bootstrap.py not implemented for openSUSE r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D74544
2020-05-10 00:51:03 +00:00
Geoff Brown 1c32970a45 Bug 1635834 - Install minidump_stackwalk in 'mach bootstrap'; r=nalexander
Install minidump_stackwalk as part of 'mach bootstrap' so that it is readily available
for generating crash reports, if desired.

Differential Revision: https://phabricator.services.mozilla.com/D74442
2020-05-08 19:09:43 +00:00
Jonathan Watt e9737ef887 Bug 1636265. Fix the Eclipse CDT build backend (broken by the switch to Python 3). r=botond
Python 3 doesn't allow strings to be written to files opened in binary mode
(it requires a byte array in that case).  As it happens, we should really be
opening these Eclipse config files in text mode since it seems on Windows the
files use Windows line ending characters.  So rather than change the strings
to byte arrays, this patch simply changes the code to open the files in text
mode.

Differential Revision: https://phabricator.services.mozilla.com/D74318
2020-05-08 16:26:35 +00:00
Mitchell Hentges 2e71833471 Bug 1636209: resolves tests that were incorrectly marked as xfail r=tomprince
When handling bug 1632429, I found some tests that worked on Python 2, but not Python 3.
They were marked accordingly as "expected failures". However, my system version of Python
is 3.8, while CI (and a non-trivial number of devs, probably) use 3.6.

Some of these tests marked as xfail were actually still working on versions of Python until 3.8.

The failure of this test was due to a change in default tarfile format. Explicitly setting this
format makes the tests pass in all relevant python versions.

Differential Revision: https://phabricator.services.mozilla.com/D74337
2020-05-07 23:41:35 +00:00
Axel Hecht 509e84e17b Bug 1635481, workaround python and virtualenv, r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D74207
2020-05-07 15:38:14 +00:00
Ricky Stewart 933b3522b8 Bug 1633156 - Don't emit cached table files from ply r=glandium
`ply`, [by design](https://github.com/dabeaz/ply/issues/79), does not produce reproducible table files; hence bug 1633156. (Note that this was *always* true, but only became a problem once we switched to Python 3, which has more unpredictable dict iteration order than Python 2.7, at least prior to [3.7](https://docs.python.org/3/whatsnew/3.7.html#summary-release-highlights).)

In any other circumstance I would consider submitting a patch to `ply` to fix this, but as of the [in-progress version 4.0 of the library](https://github.com/dabeaz/ply/blob/master/CHANGES), it doesn't even emit this cached data any more, and indeed the [latest version of the code](1fac9fed64/ply) doesn't even call `open()` at all except to do logging or to read the text data to be parsed from `stdin`. So if we were going to pin our future on `ply` and upgrade to later versions of the library in the future, we would have to live in a world where `ply` doesn't generate cached table files for us anyway.

Emitting the cached table files so later build steps can consume them is an "optimization", but it's not clear exactly how much actual value that optimization provides overall. Quoth the `CHANGES` file from that repository:

```
PLY no longer writes cached table files.  Honestly, the use of
the cached files made more sense when I was developing PLY on
my 200Mhz PC in 2001. It's not as much as an issue now. For small
to medium sized grammars, PLY should be almost instantaneous.
```

In practice, I have found this to be true; namely, `./mach build pre-export export` takes just about as long on my machine after this patch as it did before, and in a try push I performed, there's no noticeable performance regression from applying this patch. In local testing I also found that generating the LALR tables in calls to `yacc()` takes about 0.01s on my machine generally, and we generate these tables a couple dozen times total over the course of the `export` tier now. This isn't *nothing*, but in my opinion it's also not nearly long enough where it would be a concern given how long `export` already takes.

That `CHANGES` file also stresses that if caching this data is important, we have the option of doing so via `pickle`. If and when we decide that re-enabling this optimization is valuable for us, we should take control of this process and perform the generation in such a way that we can guarantee reproducibility.

Differential Revision: https://phabricator.services.mozilla.com/D73484
2020-05-07 00:39:28 +00:00
Mike Hommey ece7e458a5 Bug 1635664 - Don't capture the output from sdkmanager in ensure_android_packages. r=rstewart
We always print the output when there is no error. In case of error, we
stick the output in the thrown exception, but nothing actually prints
that out. It's simpler to just let the subprocess print its own output
instead of capturing it, so that important error messages are not hidden
in the case of failure.

Differential Revision: https://phabricator.services.mozilla.com/D74004
2020-05-07 00:08:16 +00:00
Ricky Stewart 7fa6493c9c Bug 1635585 - Run compilation/test_warnings.py under Python 3 r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D73970
2020-05-06 19:36:57 +00:00
Andi-Bogdan Postelnicu 94b15f6444 Bug 1635795 - Activate virtualenv for `mach static-analysis`. r=rstewart,marco
Differential Revision: https://phabricator.services.mozilla.com/D74058
2020-05-06 16:47:15 +00:00
Nathan Froyd 68154b269c Bug 1635229 - output relative paths in XPIDL-generated source files; r=asuth,glandium,mccr8
We currently generate absolute paths in all of our XPIDL-generated
source files, which is not so great for several reasons (deterministic
generation of files across machines, Searchfox analysis logic, shared
compilation caches, etc.).  Let's generate paths that still indicate
where you should be looking, but are identical across compilations,
objdirs, etc.

Differential Revision: https://phabricator.services.mozilla.com/D73747
2020-05-06 10:56:58 +00:00
Csoregi Natalia d68addcbf2 Backed out changeset 44a166a4efdb (bug 1555255) for causing bustage of android-sdk-linux because the docs package doesn't exist anymore. CLOSED TREE 2020-05-06 07:12:30 +03:00
Mike Hommey 435b446654 Bug 1634204 - Bump glibc and libstdc++ runtime requirement to resp. 2.17 and 3.4.19 (4.8.1). r=froydnj
Combined with the upcoming upgrade of the build environment to Debian 8,
which is also going to upgrade the Gtk+3 requirement to 3.14, of the
major Linux distros Firefox currently supports running on, this removes
runtime support for:
- Fedora 20 and earlier (EOLed in 2015),
- OpenSUSE 13.1 and earlier (EOLed in 2016),
- Debian 7 (EOLed in 2018),
- Ubuntu 14.04 (EOLed in 2019).

Some of them might actually be supported in practice because the Gtk+3
requirement might be more relaxed than 3.14, but figuring that out is not
worth the effort.

Differential Revision: https://phabricator.services.mozilla.com/D73783
2020-05-06 02:41:20 +00:00
Mike Hommey 5cef88b2f1 Bug 1634204 - Allow newer versions of symbols if they are weakly linked. r=froydnj
We currently check that the binaries we ship are not using symbol
versions of system libraries that would not be available on some older
systems. In some cases, however, we may get dependencies on newer symbol
versions in the form of weak symbols, that are checked for at runtime.
This happens with __cxa_thread_atexit_impl when building against a glibc
newer than 2.18, and the supporting code in Rust libstd actually checks
at runtime whether the weak symbol is resolved before using it.

Differential Revision: https://phabricator.services.mozilla.com/D73782
2020-05-06 02:00:13 +00:00
Csoregi Natalia 5da0fac6d9 Backed out changeset eb1b773902c3 (bug 1635229) for bustages on xpidl/runtests.py. CLOSED TREE 2020-05-06 05:05:41 +03:00
Nathan Froyd c2d1969d66 Bug 1635229 - output relative paths in XPIDL-generated source files; r=asuth,glandium,mccr8
We currently generate absolute paths in all of our XPIDL-generated
source files, which is not so great for several reasons (deterministic
generation of files across machines, Searchfox analysis logic, shared
compilation caches, etc.).  Let's generate paths that still indicate
where you should be looking, but are identical across compilations,
objdirs, etc.

Differential Revision: https://phabricator.services.mozilla.com/D73747
2020-05-06 01:35:30 +00:00
Ricky Stewart 0015091b18 Bug 1633039 - Don't check for Python 2 in configure r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D72895
2020-05-05 16:02:02 +00:00
Ricky Stewart 56c10c5aa2 Bug 1633037 - Remove all remaining references to $(PYTHON) across Makefiles r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D72894
2020-05-05 20:32:23 +00:00
Ricky Stewart 4d4b22b3de Bug 1599658 - Delete previous definition of py_action in Makefiles. Now py_action calls into Python 3 and py3_action doesn't exist. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D72487
2020-05-05 20:04:30 +00:00
Ricky Stewart fd72a5d35e Bug 1633016 - Remove a bunch of references to PYTHON(2) in Makefiles r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D72479
2020-05-05 19:53:22 +00:00
Ricky Stewart 3749c34fb4 Bug 1632916 - Run JS/web-platform/ipdl build machinery in Python 3 r=jgraham,nika,glandium
Differential Revision: https://phabricator.services.mozilla.com/D72478
2020-05-05 20:32:12 +00:00
Tarek Ziadé fa240018c6 Bug 1635546 - deactivate test under macOS r=sparky
until we have Bug 1635573

Differential Revision: https://phabricator.services.mozilla.com/D73943
2020-05-05 21:29:00 +00:00
Tarek Ziadé 9a003b79ba Bug 1635546 - fixed dependencies r=sparky
We need to set up the notebook deps.
The train of patch re-activated some
tests that required the dependencies used
by the notebook, which were not added in the
path.

Differential Revision: https://phabricator.services.mozilla.com/D73936
2020-05-05 20:35:28 +00:00
Ricky Stewart ea0981892a Bug 1635526 - Activate virtualenv in `mach static-analysis autotest` r=andi
After bug 1633039, we won't be guaranteed to have the `init` virtualenv already created after running a build, so `static-analysis autotest` needs to make sure it's created all by itself. This results in failures like [this](https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedTaskRun=WDRF1EzDRA6jPqPb60X5Pw-0&revision=332ce0963b4e92d68b0de25860debb7694dc38d5).

Differential Revision: https://phabricator.services.mozilla.com/D73924
2020-05-05 18:31:34 +00:00
Ricky Stewart 9855279a32 Bug 1635519 - Delete test_graph.py r=froydnj
This isn't useful now that the Tup backend is deleted.

Differential Revision: https://phabricator.services.mozilla.com/D73919
2020-05-05 18:07:19 +00:00
Nazım Can Altınova 418de925f0 Bug 1635381 - Change the -q flag of brew cask outdated into --quiet. r=rstewart,mhentges
Differential Revision: https://phabricator.services.mozilla.com/D73837
2020-05-05 17:31:58 +00:00
Ricky Stewart 8ffbec29d9 Bug 1635491 - Delete tup backend and `mach analyze` r=froydnj
It's not maintained and doesn't work as-is. If we want to revive it we can grab it from source control later, but in the meantime it's just confusing and it comes up in search results even though we never look for it.

Also delete `mach analyze` which depends on the existence of a Tup backend.

Differential Revision: https://phabricator.services.mozilla.com/D73911
2020-05-05 17:28:07 +00:00
Razvan Maries 3b592b6add Backed out changeset 5cf7e2bc5640 (bug 1635491) as per Ricky's request for Lint failure. CLOSED TREE 2020-05-05 19:57:33 +03:00
Ricky Stewart cdac3a2e15 Bug 1635491 - Delete tup backend and `mach analyze` r=froydnj
It's not maintained and doesn't work as-is. If we want to revive it we can grab it from source control later, but in the meantime it's just confusing and it comes up in search results even though we never look for it.

Also delete `mach analyze` which depends on the existence of a Tup backend.

Differential Revision: https://phabricator.services.mozilla.com/D73911
2020-05-05 16:32:42 +00:00
Tarek Ziadé a44d97ff35 Bug 1635389 - clean the code r=sparky
We missed a pass on black, and python.ini was incomplete

Differential Revision: https://phabricator.services.mozilla.com/D73822
2020-05-05 14:47:58 +00:00
Tarek Ziadé 7517317c35 Bug 1635076 - add a custom test runner r=sparky
We're running black+coverage

Differential Revision: https://phabricator.services.mozilla.com/D73693
2020-05-05 15:00:27 +00:00
Tarek Ziadé bacdbf01c7 Bug 1635075 - add --browsertime-geckodriver r=sparky
Adds an option to point geckodriver

Differential Revision: https://phabricator.services.mozilla.com/D73617
2020-05-04 18:12:59 +00:00
Gregory Mierzwinski 2a62f49c9a Bug 1630665 - Implement new browser intermediate results standard. r=tarek
This patch implements the new intermediate results standard and adds the mechanisms required to handle it. Results validation is done with jsonschema and some manual validation (because of some unfortunate issues with jsonschema) and some tests were implemented to ensure that we fail/pass where expected. The metrics modules were modified to handle multiple suites.

One thing that is disabled in this patch is the subtest/single-metric specifications through the "results" field. We'll do one thing at a time here and we also have no use for subtests yet (although we definitely will).

Differential Revision: https://phabricator.services.mozilla.com/D72067
2020-05-04 16:28:27 +00:00
Andi-Bogdan Postelnicu 54f965e51e Bug 1634050 - [clang-format] When analyzing files outside of the tree take along the appropriate `.clang-format` file as well. r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D73053
2020-04-30 16:46:26 +00:00
Bogdan Tara f137fa0613 Backed out 6 changesets (bug 1632916, bug 1599658, bug 1633037, bug 1633039, bug 1633016, bug 1632920) for SA bustages CLOSED TREE
Backed out changeset 332ce0963b4e (bug 1633039)
Backed out changeset a9904cbc40d9 (bug 1633037)
Backed out changeset d06b0ec349f8 (bug 1599658)
Backed out changeset 8fd300cad80f (bug 1633016)
Backed out changeset f8820941c703 (bug 1632916)
Backed out changeset ac9c2c8746ed (bug 1632920)
2020-05-02 01:49:29 +03:00
Ricky Stewart 035981e445 Bug 1633039 - Don't check for Python 2 in configure r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D72895
2020-04-30 15:23:51 +00:00
Ricky Stewart 69685d8ab5 Bug 1633037 - Remove all remaining references to $(PYTHON) across Makefiles r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D72894
2020-05-01 16:19:21 +00:00
Ricky Stewart 0daacc12c3 Bug 1599658 - Delete previous definition of py_action in Makefiles. Now py_action calls into Python 3 and py3_action doesn't exist. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D72487
2020-04-30 15:27:13 +00:00
Ricky Stewart bb4e86d85a Bug 1633016 - Remove a bunch of references to PYTHON(2) in Makefiles r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D72479
2020-04-30 15:25:22 +00:00
Ricky Stewart d990224458 Bug 1632916 - Run JS/web-platform/ipdl build machinery in Python 3 r=jgraham,nika,glandium
Differential Revision: https://phabricator.services.mozilla.com/D72478
2020-05-01 16:31:21 +00:00
Tarek Ziadé 0e14df854f Bug 1634359 - improve coverage r=sparky
Adds more coverage

Differential Revision: https://phabricator.services.mozilla.com/D73253
2020-04-30 16:13:22 +00:00
Tarek Ziadé f738edb6cd Bug 1634349 - follow up patch r=sparky
Fix a regression on macOs@CI

Differential Revision: https://phabricator.services.mozilla.com/D73317
2020-04-30 18:07:58 +00:00
sjustus f26aefb772 Bug 1619517 - Add Close tooltip to Mobile Call Card in about:protections r=prathiksha,fluent-reviewers,Gijs,flod
Differential Revision: https://phabricator.services.mozilla.com/D69308
2020-04-30 16:03:03 +00:00
Tarek Ziadé 00444bef79 Bug 1634349 - Split a pure Python runner r=sparky
In order to be able to run outside mach, we want to split the runner so we can
call it directly from Python.

Differential Revision: https://phabricator.services.mozilla.com/D73252
2020-04-30 14:32:56 +00:00
Mitchell Hentges 1b7be4fbeb Bug 1624380: upgrades brew packages if they're out-of-date r=rstewart
`./mach bootstrap` prepares your environment for a build and installs missing "brew" packages".
However, if a package is installed but out-of-date, it was being ignored by the bootstrapping logic.

This change ensures that `brew update` is run, and updates any out-of-date packages that are requested.

Differential Revision: https://phabricator.services.mozilla.com/D72956
2020-04-29 21:17:25 +00:00
Ricky Stewart 233ffddc4d Bug 1632974 - Handle case where mach Command conditions don't have a `__name__` attribute r=glandium
There are `conditions` in tree that are callables but which don't have a `__name__` attribute; for example, `functools.partial` instances don't have a `__name__` since they're effectively anonymous functions. If you get to this branch and one of your `conditions` are that kind of object then you'll get a confusing error message instead of the understandable one we're trying to produce here, so account for that possibility.

Differential Revision: https://phabricator.services.mozilla.com/D72957
2020-04-29 19:32:14 +00:00
Tarek Ziadé 56bca0b81a Bug 1633875 - pick the depencendy in third party r=Bebe
This patch will use the dependency that is in the third_party/python/ dir

Differential Revision: https://phabricator.services.mozilla.com/D73016
2020-04-29 09:26:34 +00:00
Ricky Stewart 7b699ef488 Bug 1632461 - Delete "hacking environment to allow binary Python" message and corresponding hack. r=dmajor
This warning dates from bug 910487, which was 7 years ago. Since joining Mozilla I have *always* gotten this warning, and as far as I can tell since I never had a pre-2019 version of Visual Studio on my dev machine, the VS90COMNTOOLS variable was *never* set. Moreover, the "hack" is written in such a way that it does nothing *unless* you have `VS{100,110,120}COMNTOOLS` set, which I never have on my machine either, as you might expect since I only have the one version of Visual Studio installed.

The [latest public build documentation](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Windows_Prerequisites) recommends that you install the Community edition of Visual Studio 2019, and as of 2019 the variable that's being used is `VS160COMNTOOLS`, so the only way someone would get value out of this hack is if they're using a substantially older version of Visual Studio than we recommend anyway.

Since 1) I *suspect* the hack is not doing anything for the large majority, if not all, of the people currently running builds on Windows on a day-to-day basis and 2) even if the hack continues to do something useful under some hypothetical scenarios, the content of the hack as well as the corresponding warning is so outdated that it should be updated anyway, I propose deleting it entirely.

Differential Revision: https://phabricator.services.mozilla.com/D72925
2020-04-28 19:24:45 +00:00
Tarek Ziadé 33f1eee4d5 Bug 1633437 - Support for test metadata r=acreskey
This patch adds support for tests metadata. A test script parser is added as
well as a new "doc" flavor that can be used to display the script info in the
command line. This parser will be the basis for building automated docs and
scripts verifications if we want to do this.

Differential Revision: https://phabricator.services.mozilla.com/D72800
2020-04-28 17:07:14 +00:00
Francesco Lodolo (:flod) 5bd35a8174 Bug 1633654 - Remove Fluent migration recipes for Firefox 75 r=Pike
Differential Revision: https://phabricator.services.mozilla.com/D72797
2020-04-28 12:52:26 +00:00
Artem 069a4488a1 Bug 1609559 - Migrate protectionsPanel.inc.xhtml to Fluent r=Gijs,fluent-reviewers,flod
Differential Revision: https://phabricator.services.mozilla.com/D71234
2020-04-28 06:23:10 +00:00
Kagami Sascha Rosylight be0aac133d Bug 1543872 - Use `git log --not --remotes` r=glandium
The current way to get outgoing files in Git passes every branch to merge-base, which generates a way too long command that fails on Windows. This patch tries using `git log --not --remotes` instead of passing merge-base value.

Differential Revision: https://phabricator.services.mozilla.com/D71802
2020-04-28 07:27:51 +00:00
Tarek Ziadé 89f357d5e0 Bug 1632780 - improve test coverage r=sparky
Added test coverage

Differential Revision: https://phabricator.services.mozilla.com/D72329
2020-04-27 15:40:58 +00:00
Jeff Muizelaar ee1ffa4dfc Bug 1631063. Use utf-8 when reading TOML files. r=glandium
Without this we get encoding errors on 'Álvarez' on Windows because
the default encoding there is not 'utf-8'.

Differential Revision: https://phabricator.services.mozilla.com/D71382
2020-04-23 08:08:51 +00:00
Mike Hommey 9c03a7015d Bug 1632348 - Convert mach artifact to python 3. r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D72114
2020-04-26 21:53:57 +00:00
Noemi Erli 7f6a7f8cf2 Backed out changeset 90bd5342c58f (bug 1632348) for causing toolchains bustage 2020-04-26 02:37:07 +03:00
Mike Hommey 568944c117 Bug 1632348 - Convert mach artifact to python 3. r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D72114
2020-04-24 20:51:24 +00:00
Mitchell Hentges b6eb2ea22e Bug 1627072 - Runs pip-compile in same directory as temporary requirements file r=rstewart
pip-compile can annotate each requirement with a reason why it is included (e.g.: transitive dependency, or depended-on directly).

When annotating direct dependencies, it notes it as "via -r <direct path to requirements.in file>".
Since we were using a temporary directory, the path of the directory was being included, making the output non-deterministic.

This change ensures that we run pip-compile in the same working directory as the temporary requirements file, enabling
the annotations to be deterministic: "via -r requirements-mach-vendor-python.in".

Differential Revision: https://phabricator.services.mozilla.com/D72181
2020-04-24 20:54:06 +00:00
Kagami Sascha Rosylight 2f8b998e96 Bug 1628726 - Use os.path.realpath consistently r=rstewart
`os.path.realpath` in Python 3.8 now always uppercases Windows drive letter, while it was just an alias of `os.path.abspath` in Windows. This patch uses `.realpath()` consistently to get `topobjdir` to fix the incompatibility from the behavior change.

Differential Revision: https://phabricator.services.mozilla.com/D72188
2020-04-24 20:59:03 +00:00
Chris AtLee b246a1493c Bug 1632601: Create test archives for jsreftest and jittest r=ahal
Split out jsreftest and jittest files into their own packages, removing them
from the common package.

This speeds up extracting files from the common test archive for
non-jsreftest/jittest suites.

Also, remove some files from the web-platform test archive that are already
present in the common archive.

Differential Revision: https://phabricator.services.mozilla.com/D72192
2020-04-24 20:18:58 +00:00
Joel Maher f8875234b3 Bug 1621095 - add reftest-qr to run focused webrender reftests on android phones. r=bc
add reftest-qr to run focused webrender reftests on android phones.

Differential Revision: https://phabricator.services.mozilla.com/D72312
2020-04-24 18:54:51 +00:00
Coroiu Cristina 04697a7080 Merge mozilla-central to autoland a=merge on a CLOSED TREE 2020-04-24 21:32:19 +03:00
Tom Prince 07fa71e0fe Bug 1632688: [mozversioncontrol] Don't try to add files if none are provided; r=ahal,rstewart a=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D72414
2020-04-24 12:18:26 -06:00
Mike Hommey 5e8b3e9559 Bug 1632353 - Convert mach resource-usage to python3. r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D72117
2020-04-23 22:56:37 +00:00
Ricky Stewart 725728fc20 Bug 1621451 - Remove remaining in-build references to Python 2 GENERATED_FILES as well as underlying build system support r=dmajor
We still need to resolve bug 1621448, at which point we can delete the `py_action` macro entirely.

Differential Revision: https://phabricator.services.mozilla.com/D71795
2020-04-22 17:34:23 +00:00
Mitchell Hentges 38e0a4b340 Bug 1550832 - synchronize makefile blacklist with mozbuild variables r=rstewart
As we migrate from makefiles to moz.build, we want to warn developers if
variables only used by moz.build are accidentally defined in a makefile.

This was the purpose of `MOZBUILD_VARIABLES` in recursivemake.py, though it became out-of-date.

This patch defines `MOZBUILD_VARIABLES` off the official list from `mozbuild.frontend.context.VARIABLES`, and
removes unused (?) code from makefiles accordingly

Note that the following variables use to be in `MOZBUILD_VARIABLES`, but aren't there any more because
they aren't in `mozbuild.frontend.context.VARIABLES`:
* CMSRCS
* CMMSRCS
* EXTRA_JS_MODULES
* EXTRA_PP_COMPONENTS
* EXTRA_PP_JS_MODULES
* HOST_CSRCS
* HOST_CMMSRCS
* HOST_EXTRA_LIBS
* JAVA_JAR_TARGETS
* LIBS
* MAKE_FRAMEWORK
* MODULE
* NO_DIST_INSTALL
* NO_INTERFACES_MANIFEST
* PARALLEL_DIRS
* PREF_JS_EXPORTS
* RESOURCE_FILES
* SHARED_LIBRARY_LIBS
* TEST_DIRS
* TOOL_DIRS
* XPCSHELL_TESTS

Differential Revision: https://phabricator.services.mozilla.com/D72076
2020-04-24 14:47:04 +00:00
Cosmin Sabou f56aee4c5c Backed out changeset 022f4f67d388 (bug 1550832) for causing SM bustages. CLOSED TREE 2020-04-24 19:22:27 +03:00
Mitchell Hentges 373ffb5d65 Bug 1550832 - synchronize makefile blacklist with mozbuild variables r=rstewart
As we migrate from makefiles to moz.build, we want to warn developers if
variables only used by moz.build are accidentally defined in a makefile.

This was the purpose of `MOZBUILD_VARIABLES` in recursivemake.py, though it became out-of-date.

This patch defines `MOZBUILD_VARIABLES` off the official list from `mozbuild.frontend.context.VARIABLES`, and
removes unused (?) code from makefiles accordingly

Note that the following variables use to be in `MOZBUILD_VARIABLES`, but aren't there any more because
they aren't in `mozbuild.frontend.context.VARIABLES`:
* CMSRCS
* CMMSRCS
* EXTRA_JS_MODULES
* EXTRA_PP_COMPONENTS
* EXTRA_PP_JS_MODULES
* HOST_CSRCS
* HOST_CMMSRCS
* HOST_EXTRA_LIBS
* JAVA_JAR_TARGETS
* LIBS
* MAKE_FRAMEWORK
* MODULE
* NO_DIST_INSTALL
* NO_INTERFACES_MANIFEST
* PARALLEL_DIRS
* PREF_JS_EXPORTS
* RESOURCE_FILES
* SHARED_LIBRARY_LIBS
* TEST_DIRS
* TOOL_DIRS
* XPCSHELL_TESTS

Differential Revision: https://phabricator.services.mozilla.com/D72076
2020-04-24 14:47:04 +00:00
Bogdan Tara a5112d03db Backed out changeset 97280448a042 (bug 1621095) for test_ext_test.html failures CLOSED TREE 2020-04-24 17:09:37 +03:00
Joel Maher b9a6e9d31c Bug 1621095 - add reftest-qr to run focused webrender reftests on android phones. r=bc
add reftest-qr to run focused webrender reftests on android phones.

Differential Revision: https://phabricator.services.mozilla.com/D72312
2020-04-24 12:35:12 +00:00
Mike Hommey 56ca1ef0e7 Bug 1631211 - In configure, pass extra compiler flags after source path. r=dmajor,rstewart
When running e.g. check_symbols with extra flags like when checking
for vpx_codec_dec_init_ver when building against system libvpx, in some
configurations, the test can fail when the library flags (-l) appear
before the source file path.

The reason is that in some configurations, the compiler passes
--as-needed to the linker before both the flags and the object file
path, and the object file path is in the same position as the source
file path was. With --as-needed, -l flags are dropped if the library
wasn't needed for any of the linked code that appears *before* the flag.
So linking with `--as-needed -lfoo foo.o`, is equivalent to linking with
`foo.o` only in practice, while `--as-needed foo.o -lfoo` is equivalent
to `foo.o -lfoo`.

Differential Revision: https://phabricator.services.mozilla.com/D71456
2020-04-23 21:46:01 +00:00
Tarek Ziadé b1ad9498a5 Bug 1632404 - added missing error logger r=sparky
The android layer calls layer.error(), let's make sure it's
really there :)

Differential Revision: https://phabricator.services.mozilla.com/D72102
2020-04-23 21:46:49 +00:00
Tarek Ziadé 92eb19eb28 Bug 1632402 - added --browsertime-iterations r=sparky
Added the --browsertime-iterations option

Differential Revision: https://phabricator.services.mozilla.com/D72101
2020-04-23 19:40:56 +00:00
Tarek Ziadé b6d9b10181 Bug 1632267 - Clean up the android layer r=acreskey
We're back to an explicit --android option and --android-app-name, the layer won't use the --browser-binary option as an implicit
fallback anymore. This is cleaner.

Differential Revision: https://phabricator.services.mozilla.com/D72031
2020-04-23 14:33:32 +00:00
Gabriele Svelto 96edbb04b7 Bug 1631382 - Update the Gentoo |mach bootstrap| implementation r=glandium
This fixes multiple issues:
* It switches mobile builds from the Oracle JDK to OpenJDK and removes all the
  logic needed to download the former
* It only installs the build dependencies required for building Firefox and
  stores them in the world file

Differential Revision: https://phabricator.services.mozilla.com/D71539
2020-04-23 08:09:37 +00:00
shravanrn@gmail.com e0273c024b Bug 1626174 - Enable use of wasm sandboxed libOgg in the OggDemuxer in linux, mac, try servers r=padenot,erahm,dmajor,firefox-build-system-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D70652
2020-04-22 11:16:10 +00:00