Граф коммитов

6272 Коммитов

Автор SHA1 Сообщение Дата
Sylvestre Ledru d61fa8949f Bug 1656764 - Relax the version check for clang-format r=andi DONTBUILD
So that it works when CLANG_VENDOR is set

Differential Revision: https://phabricator.services.mozilla.com/D85687
2020-08-03 12:41:15 +00:00
Tarek Ziadé 03cbd4d536 Bug 1650363 - Performance testing of HTTP3 in Firefox using only xpcshell tests r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D84021
2020-08-02 17:28:57 +00:00
Gregory Mierzwinski a4e1ebe204 Bug 1656540 - Allow specifying perfherder settings on a per-metric basis. r=tarek
This patch fixes an issue where the metric settings were not being used because they don't use the test name. It also handles some changes (from a bad copy-paste) that didn't make it into the last live-site patch series.

Differential Revision: https://phabricator.services.mozilla.com/D85609
2020-08-01 16:13:28 +00:00
Mike Hommey 38cc2e0609 Bug 1654994 - Switch full symbols archive to tar.zst. r=firefox-build-system-reviewers,rstewart,tomprince
Differential Revision: https://phabricator.services.mozilla.com/D85252
2020-07-31 20:27:54 +00:00
Mike Hommey 36a8aaf1b4 Bug 1654994 - Support using .tar.zst symbol archives in mach artifact. r=nalexander,firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D85249
2020-07-31 10:01:52 +00:00
Mike Hommey 557b6e2811 Bug 1654994 - Support creating .tar.zst archives instead of .zip for symbol archives. r=firefox-build-system-reviewers,rstewart,mhentges
Differential Revision: https://phabricator.services.mozilla.com/D85248
2020-08-01 05:33:05 +00:00
Mitchell Hentges d0ec90c4d5 Bug 1654074: Make @CommandProvider validation more specific r=firefox-build-system-reviewers,rstewart
@CommandProvider does parameter validation and collects information (such
as "pass_context") that will be needed by Registrar.

However, rather than just checking parameter length, we can make it more
specific and assert that the specific expected parameter ("context") exists.

Differential Revision: https://phabricator.services.mozilla.com/D85482
2020-07-30 22:25:05 +00:00
Yue Zhang 210c0b2307 Bug 1635930 - Activate a layer when its options are used. r=sparky,tarek
Differential Revision: https://phabricator.services.mozilla.com/D83551
2020-07-31 17:23:51 +00:00
Gregory Mierzwinski ee2aa57e5c Bug 1655031 - Fix how mozperftest metrics are parsed. r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D85318
2020-07-31 15:33:12 +00:00
Gregory Mierzwinski d93acb6b75 Bug 1655032 - Add splitting functionality to metrics module. r=tarek
This patch adds the `--<LAYER>-split-by` option to the metric layers. It allows users to split the data they obtain using a given data field name. For instance, if `browserScripts.pageinfo.url` is provided, then the data will be split based on the unique URLs that are found.

Differential Revision: https://phabricator.services.mozilla.com/D84822
2020-07-30 23:12:28 +00:00
Dorel Luca 0226b8feef Backed out 6 changesets (bug 1654994) for build bustage on fuzzy builds. DONTBUILD
Backed out changeset cd3e1074b785 (bug 1654994)
Backed out changeset 45ee50e76d61 (bug 1654994)
Backed out changeset 77f1b667baec (bug 1654994)
Backed out changeset 9fca8b0bf8c1 (bug 1654994)
Backed out changeset 7131943d8db9 (bug 1654994)
Backed out changeset 435b90f6ca70 (bug 1654994)
2020-07-31 12:36:14 +03:00
Mike Hommey 311e167747 Bug 1654994 - Switch full symbols archive to tar.zst. r=firefox-build-system-reviewers,rstewart,tomprince
Differential Revision: https://phabricator.services.mozilla.com/D85252
2020-07-30 22:21:28 +00:00
Mike Hommey 1bf0ac0aca Bug 1654994 - Support using .tar.zst symbol archives in mach artifact. r=nalexander,firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D85249
2020-07-30 15:53:26 +00:00
Mike Hommey ddea6bd3ce Bug 1654994 - Support creating .tar.zst archives instead of .zip for symbol archives. r=firefox-build-system-reviewers,rstewart,mhentges
Differential Revision: https://phabricator.services.mozilla.com/D85248
2020-07-30 15:52:51 +00:00
Mitchell Hentges d3dfb78af4 Bug 1656303: Only print $A_SPECIFIC_TARGET warning if one of the targets is not in the allowlist r=firefox-build-system-reviewers,rstewart
The "what" value contains a list of build targets, the warning should be printed if any one of the targets is unexpected.

Differential Revision: https://phabricator.services.mozilla.com/D85462
2020-07-30 19:38:42 +00:00
Mike Hommey f70f000a4f Bug 1614626 - Move xpcshell selftests to before running actual xpcshell tests. r=gbrown
This allows to test them in the exact same environment as the tests are
going to run, which turns out to have revealed a few issues that would
only appear once xpcshell tests fail, impeding on debugging those
failures.

Differential Revision: https://phabricator.services.mozilla.com/D84781
2020-07-30 20:27:14 +00:00
Ricky Stewart 63f010c613 Bug 1646406 - Distinguish between user errors and actual reportable exceptions in `mach` r=mhentges,froydnj
Add a new `UserError` class which when thrown doesn't get reported to Sentry.

Differential Revision: https://phabricator.services.mozilla.com/D85026
2020-07-29 21:34:17 +00:00
Tarek Ziadé cb00bce720 Bug 1655956 - removed unused perf tuning code r=sparky
Removes unused API and add missing coverage

Differential Revision: https://phabricator.services.mozilla.com/D85285
2020-07-29 17:35:35 +00:00
Csoregi Natalia b0e102995f Backed out 2 changesets (bug 1646406, bug 1655529) for geckoview failures. CLOSED TREE
Backed out changeset 0630f4154c62 (bug 1655529)
Backed out changeset 18d42084d135 (bug 1646406)
2020-07-30 00:28:59 +03:00
Ricky Stewart ffd5064859 Bug 1646406 - Distinguish between user errors and actual reportable exceptions in `mach` r=mhentges,froydnj
Add a new `UserError` class which when thrown doesn't get reported to Sentry.

Differential Revision: https://phabricator.services.mozilla.com/D85026
2020-07-29 19:34:24 +00:00
Dorel Luca c24650c12a Backed out changeset 929e1acc42af (bug 1614626) for XPCshell failures in extensions/test/xpcshell/test_ext_dns.js. CLOSED TREE 2020-07-29 07:37:28 +03:00
Mike Hommey efebeaa2c6 Bug 1614626 - Move xpcshell selftests to before running actual xpcshell tests. r=gbrown
This allows to test them in the exact same environment as the tests are
going to run, which turns out to have revealed a few issues that would
only appear once xpcshell tests fail, impeding on debugging those
failures.

Differential Revision: https://phabricator.services.mozilla.com/D84781
2020-07-29 00:23:03 +00:00
Andrew Halberstadt cc0b1f8e5f Bug 985141 - [mozperftest] Fix missed instances of _activate_virtualenv in mozperftest framework, r=perftest-reviewers,sparky
Differential Revision: https://phabricator.services.mozilla.com/D85203
2020-07-28 20:54:58 +00:00
Andrew Halberstadt bef34ebc4b Bug 1408051 - Backed out changeset 7864e0d0295f
This change had a few regressions that will be easier to fix without the pressure of time.

Differential Revision: https://phabricator.services.mozilla.com/D85193
2020-07-28 19:04:53 +00:00
Ricky Stewart da3fe81bea Bug 1655781- Allow specifying that certain packages should only be included in a `virtualenv` when parsing `virtualenv_packages.txt` r=chutten,nalexander
This solves the same problem we attempted to solve in bug 1654663. That was a low-cost, sensible solution when there was only one in-build reference to `glean_parser`, but with project FOG we're about to drastically increase the in-build reliance on the library, so the ad-hoc `sys.path` manipulation is an increasingly insensible solution. Here we address this in a first-class way by specifying that `glean_parser` should be imported in `virtualenv`s, but NOT by top-level `mach` commands that run outside of an in-`objdir` `virtualenv`.

Differential Revision: https://phabricator.services.mozilla.com/D85182
2020-07-28 18:29:55 +00:00
Andrew Halberstadt d0fbcca0b6 Bug 985141 - [mozbuild] Remove leading underscore from MozbuildObject._activate_virtualenv, r=firefox-build-system-reviewers,perftest-reviewers,andi,AlexandruIonescu,rstewart
This function is used all across the tree and should be considered a public API.

Differential Revision: https://phabricator.services.mozilla.com/D85045
2020-07-28 16:06:10 +00:00
Philip Chimento c936b44447 Bug 1653560 - Fix distutils.spawn import. r=andi
Differential Revision: https://phabricator.services.mozilla.com/D84023
2020-07-28 14:54:32 +00:00
Andi-Bogdan Postelnicu f2a6f56b99 Bug 1655701 - for Coverity static-analysis do not relay on ensure exit code. r=marco
Differential Revision: https://phabricator.services.mozilla.com/D85133
2020-07-28 12:52:17 +00:00
Mike Hommey 2a4eb30ed4 Bug 1654465 - Set -Cembed-bitcode=yes instead of CARGO_PROFILE_RELEASE_LTO. r=firefox-build-system-reviewers,rstewart
It turns out setting CARGO_PROFILE_RELEASE_LTO has unwanted side
effects.

First it's not actually strictly equivalent to using `cargo rustc --
-Clto`. For instance, it apparently also enables cross-language LTO in
newer versions of cargo.

Second, it changes the rust computed hash for all the dependencies of
the crate being built with the variable set, which makes them diverge
from when the same dependencies are built through another crate in the
tree that is not LTOed. This effectively makes us build a _lot_ of
crates twice, many of which are not cacheable.

Since the original problem is that cargo >= 1.45 passes extra flags (`-C
embed-bitcode=no`) to rustc that are incompatible with `-Clto`, and while
it knows to adjust based on the `lto` setting in the build profile
(which CARGO_PROFILE_RELEASE_LTO overrides the default of), cargo
ignores flags passed via `cargo rustc -- ...` when making those
adjustments.

So, we need to override with `-C embed-bitcode=yes` on our own at the
same time we pass `-Clto`. But doing that through `cargo rustc -- ...`
is not enough because all the dependencies of the crate built with
`-Clto` need to be built with `-C embed-bitcode=yes`. So we need to
override with `RUSTFLAGS`, which will affect all the dependencies.
But we also need to do this consistently across all crates, not only the
dependencies of crates built with `-Clto`, otherwise we'd still end up
building crates twice (once with and once without the override).

Unfortunately, the `-C embed-bitcode=*` flag is also not supported in
versions older than 1.45, so we have to avoid adding it on older
versions.

We unfortunately support a large range of versions of rustc (albeit only
for tools/crashreporter), but we actually need to upgrade the smaller
supported version because rustc < 1.38 doesn't support our top-level
Cargo.lock. This makes the version check slightly less awful.

Differential Revision: https://phabricator.services.mozilla.com/D84652
2020-07-27 21:23:57 +00:00
Ricky Stewart 3e84200ad6 Bug 1655340 - Install `self.packages` in `mozboot/debian.py` r=froydnj
This got trampled in an overzealous code deletion in bug 1647814.

Differential Revision: https://phabricator.services.mozilla.com/D85004
2020-07-27 15:57:16 +00:00
Mike Hommey f8b53d8e93 Bug 1654961 - Re-enable some python tests on py2 on mac. r=dmajor
Bug 1648506 disabled tests that failed on py2 on mac. Some now pass
properly, presumably from more recent changes to
build/moz.configure/init.configure.

Differential Revision: https://phabricator.services.mozilla.com/D84776
2020-07-24 05:40:41 +00:00
Coroiu Cristina 828f2319c0 Backed out changeset 2439a32e1a06 (bug 1654961) for mbu failures 2020-07-24 08:19:33 +03:00
Mike Hommey 29e887fea7 Bug 1654961 - Re-enable some python tests on py2 on mac. r=dmajor
Bug 1648506 disabled tests that failed on py2 on mac. They now pass
properly, presumably from more recent changes to
build/moz.configure/init.configure.

Differential Revision: https://phabricator.services.mozilla.com/D84776
2020-07-24 04:19:51 +00:00
Tom Prince 97a298b135 Bug 1652503: [mozlint] Use `attrs` for `mozlint.result.Issue`; r=ahal
This gives sorting on `Issue` for free, which makes it easier to write tests
for linters.

Differential Revision: https://phabricator.services.mozilla.com/D84643
2020-07-23 14:55:45 +00:00
Andrew Creskey c6c9bc4f6f Bug 1648857 - Add applink perftest for the reference browser r=tarek,sparky
The Reference Browser represents our tech stack up to and including the Android-Components layer.
It would be useful to have an applink startup test for this browser to detect improvements and regressions.

Differential Revision: https://phabricator.services.mozilla.com/D81407
2020-07-23 14:11:38 +00:00
Emilio Cobos Álvarez a8f027f4ff Bug 1654795 - Fix mach clang-format python3 error. r=andi
Differential Revision: https://phabricator.services.mozilla.com/D84691
2020-07-23 14:06:14 +00:00
Francesco Lodolo (:flod) 3be1f9c7e0 Bug 1654177 - Remove Fluent migration recipes for Firefox 78, r=Pike
Differential Revision: https://phabricator.services.mozilla.com/D84294
2020-07-23 08:03:48 +00:00
Mitchell Hentges 81ec118ee7 Bug 1654607: Upgrades glean in bootstrap r=firefox-build-system-reviewers,rstewart
A newer version of Glean is needed to take advantage of the recent "optional version" change.

Pins glean to 31.5.0 to avoid breaking changes, such as to the YAML schema.

Differential Revision: https://phabricator.services.mozilla.com/D84566
2020-07-22 19:48:23 +00:00
Nathan Froyd a8ddf893b6 Bug 1654633 - install new dump_syms during bootstrap on OS X; r=nalexander,mhentges
This change brings Mac into parity with our other development platforms.

Differential Revision: https://phabricator.services.mozilla.com/D84580
2020-07-22 19:11:45 +00:00
Hamzah 848d6992f5 Bug 1408051 - Remove mozharness' copy of virtualenv and use the one under /third_party/python instead r=ahal,rail
Differential Revision: https://phabricator.services.mozilla.com/D83181
2020-07-17 19:21:18 +00:00
Hamzah 570e9a50fc Bug 1606475 - Move mach clang-format to Python 3 r=andi
Manually tested various combination of commands for `./mach clang-format`. It works fine on both py2 as well as py3

Differential Revision: https://phabricator.services.mozilla.com/D79330
2020-07-20 12:12:28 +00:00
Mike Hommey 268a226874 Bug 1654182 - Only look at the dynamic symbols table for the version checks. r=froydnj
The main reason we look at the complete symbols table is that before bug
1541792, we needed to look at that table for _NSModule symbols.

In bug 1516228, we also made everything llvm-objdump to limit the
differences cross-platform, but that's not necessary anymore per the
previous change.

llvm-objdump doesn't support getting only the dynamic symbols table, so
we go back to what we were using before bug 1516228, namely readelf,
while preserving a code path to use the complete symbols table for the
networking test on libgkrust.a, which doesn't have a dynamic symbols
table.

With this change, check_binary goes from 45s to 0.2s on my machine.

Differential Revision: https://phabricator.services.mozilla.com/D84305
2020-07-22 04:09:39 +00:00
Mike Hommey 7a304e4f58 Bug 1654182 - Remove support for iterating symbols on non-Linux platforms. r=froydnj
The way check_binary currently works is the result of many years of
changes, but some of the more recent ones actually make some of the
earliest ones unnecessary.

For instance, bug 1541792 removed the need to use iter_symbols on
non-Linux platforms. So we remove support for non-Linux (non-ELF,
really) platforms in iter_symbols (and rename the function to reflect
that).

Differential Revision: https://phabricator.services.mozilla.com/D84304
2020-07-22 04:09:21 +00:00
Mike Hommey 42410ac80f Bug 1654182 - Reduce the amount of work done for binary compatibility check. r=froydnj
Iterating symbols is actually expensive because of all the line
splitting. So iterating over all symbols 4 times is actually a huge
waste of time.

Instead, iterate over them once, doing all the version checks at once
for each of them.

This brings down the time spent in check_binary from 2:25 to 0:45 on my
machine.

Differential Revision: https://phabricator.services.mozilla.com/D84303
2020-07-22 00:27:08 +00:00
Dorel Luca 3f7ad38666 Backed out 4 changesets (bug 1654182) for Spidermonkey failure and Android build bustage. CLOSED TREE
Backed out changeset 8ecbb1da91cf (bug 1654182)
Backed out changeset 205329cc5f30 (bug 1654182)
Backed out changeset 13ab8f6d24bf (bug 1654182)
Backed out changeset 6100f0d3c0ad (bug 1654182)
2020-07-22 03:04:57 +03:00
Cosmin Sabou 25f4621719 Bug 1654182 - Fix spidermonkey bustages. r=glandium 2020-07-22 01:54:38 +03:00
Ricky Stewart ffb40eb3c2 Bug 1636797 - Tweak `git clean` call in `mach clobber python` r=mhentges,jgilbert,froydnj
Differential Revision: https://phabricator.services.mozilla.com/D84440
2020-07-21 22:27:21 +00:00
Mike Hommey 06ab4ddb50 Bug 1654182 - Only look at the dynamic symbols table for the version checks. r=froydnj
The main reason we look at the complete symbols table is that before bug
1541792, we needed to look at that table for _NSModule symbols.

In bug 1516228, we also made everything llvm-objdump to limit the
differences cross-platform, but that's not necessary anymore per the
previous change.

llvm-objdump doesn't support getting only the dynamic symbols table, so
we go back to what we were using before bug 1516228, namely readelf,
while preserving a code path to use the complete symbols table for the
networking test on libgkrust.a, which doesn't have a dynamic symbols
table.

With this change, check_binary goes from 45s to 0.2s on my machine.

Differential Revision: https://phabricator.services.mozilla.com/D84305
2020-07-21 13:28:51 +00:00
Mike Hommey a59a02755b Bug 1654182 - Remove support for iterating symbols on non-Linux platforms. r=froydnj
The way check_binary currently works is the result of many years of
changes, but some of the more recent ones actually make some of the
earliest ones unnecessary.

For instance, bug 1541792 removed the need to use iter_symbols on
non-Linux platforms. So we remove support for non-Linux (non-ELF,
really) platforms in iter_symbols (and rename the function to reflect
that).

Differential Revision: https://phabricator.services.mozilla.com/D84304
2020-07-21 21:00:50 +00:00
Mike Hommey 9af7ceef05 Bug 1654182 - Reduce the amount of work done for binary compatibility check. r=froydnj
Iterating symbols is actually expensive because of all the line
splitting. So iterating over all symbols 4 times is actually a huge
waste of time.

Instead, iterate over them once, doing all the version checks at once
for each of them.

This brings down the time spent in check_binary from 2:25 to 0:45 on my
machine.

Differential Revision: https://phabricator.services.mozilla.com/D84303
2020-07-21 22:15:51 +00:00