Граф коммитов

718 Коммитов

Автор SHA1 Сообщение Дата
Mike Hommey 0fb4a38faf Bug 1621436 - Run process_install_manifest with python3. r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D70156

--HG--
extra : moz-landing-system : lando
2020-04-09 00:21:12 +00:00
Andrea Marchesini d7badaf4fb Bug 1623593 - Get rid of NECKO_COOKIES compilation flag, r=mayhemer
Differential Revision: https://phabricator.services.mozilla.com/D67451

--HG--
extra : moz-landing-system : lando
2020-03-24 16:30:03 +00:00
Mike Shal 419f244300 Bug 1620744 - Convert generated_sources.py to py3; r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D65853

--HG--
extra : moz-landing-system : lando
2020-03-10 20:19:32 +00:00
Mike Shal 2723002de4 Bug 1620744 - Convert fat_aar.py to py3; r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D65849

--HG--
extra : moz-landing-system : lando
2020-03-10 20:19:22 +00:00
Mike Shal 4017368b2f Bug 1620744 - Use py3_action for cases that already support it; r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D65846

--HG--
extra : moz-landing-system : lando
2020-03-10 20:19:13 +00:00
Daniel Varga 09acd57d19 Backed out 13 changesets (bug 1620744) for causing diffoscope failures firefox/browser/chrome/browser/content/browser/built_in_addons.json
CLOSED TREE

Backed out changeset 6beda54bcb9b (bug 1620744)
Backed out changeset a1e97f0b91ef (bug 1620744)
Backed out changeset b8faa0184d4f (bug 1620744)
Backed out changeset 3bc8fda68107 (bug 1620744)
Backed out changeset 8e95b21b2ae3 (bug 1620744)
Backed out changeset 1de09de1a802 (bug 1620744)
Backed out changeset 622a2f7414fa (bug 1620744)
Backed out changeset 3372c9ab721c (bug 1620744)
Backed out changeset 0997313a9f99 (bug 1620744)
Backed out changeset 2fa34749bbfa (bug 1620744)
Backed out changeset 6d597d2eb792 (bug 1620744)
Backed out changeset 78e78f7c7b26 (bug 1620744)
Backed out changeset 6e4d85b19f88 (bug 1620744)
2020-03-10 21:13:18 +02:00
Mike Shal d58365f0ce Bug 1620744 - Convert generated_sources.py to py3; r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D65853

--HG--
extra : moz-landing-system : lando
2020-03-09 22:02:39 +00:00
Mike Shal fd97dcb49c Bug 1620744 - Convert fat_aar.py to py3; r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D65849

--HG--
extra : moz-landing-system : lando
2020-03-09 22:02:29 +00:00
Mike Shal a1ea58eb94 Bug 1620744 - Use py3_action for cases that already support it; r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D65846

--HG--
extra : moz-landing-system : lando
2020-03-09 22:02:22 +00:00
Nathan Froyd 179cc1aa78 Bug 1618280 - make `make distclean` work again; r=dmajor
We were using `config.status*` as a catchall in `DIST_GARBAGE` for
`config.status` and `config.statusd`.  Unfortunately, the latter was a
directory, so `rm config.status*` error'd out.  We can fix this easily
by putting `config.statusd` in `GARBAGE_DIRS`, which will use `rm -r`,
as intended.

Differential Revision: https://phabricator.services.mozilla.com/D64422

--HG--
extra : moz-landing-system : lando
2020-02-26 18:14:36 +00:00
Mike Shal a33fa339c3 Bug 1557788 - Remove spurious MOZ_PROFILE_USE ifndefs; r=firefox-build-system-reviewers,chmanchester
One of these appears to be mistakenly leftover from bug 861178
which was intended to avoid purging dist/, but the code inside the ifndef
no longer does that.

The other is from bug 1246881 to avoid re-writing the buildid.h file
during the profile-use build, but now that there are two separate
builds, they will each have their own buildid.h / source-repo.h files.

Differential Revision: https://phabricator.services.mozilla.com/D56116

--HG--
extra : moz-landing-system : lando
2019-12-09 18:03:53 +00:00
Mike Shal 32e90b04d9 Bug 1557788 - Remove MOZ_1TIER_PGO; r=firefox-build-system-reviewers,chmanchester
MOZ_1TIER_PGO was a temporary hack to support 1-tier PGO builds while
they were being ported to 3-tier. Now that all builds are 3-tier, it can
be removed.

Differential Revision: https://phabricator.services.mozilla.com/D56112

--HG--
extra : moz-landing-system : lando
2019-12-09 18:03:36 +00:00
Mike Shal 25308d30a2 Bug 1563403 - Use 3-tier PGO for local MOZ_PGO=1 builds; r=firefox-build-system-reviewers,dmajor,chmanchester
Local PGO builds now use 3-tier machinery under the hood. Instead of a
single object directory that gets cleaned in between the instrumented
and profile-use builds, now the instrumented build uses a separate
'${objdir}/instrumented' directory. This makes it easier to handle
within mach since we can drive the two builds with environment variables
and keep all build artifacts separate, without needing to do manual
cleanup in between.

Differential Revision: https://phabricator.services.mozilla.com/D50098

--HG--
extra : moz-landing-system : lando
2019-12-06 01:07:32 +00:00
Chris AtLee 4a7ce514be Bug 1540276: Migrate authenticode signing to autograph r=Callek,mshal
Differential Revision: https://phabricator.services.mozilla.com/D47114

--HG--
extra : moz-landing-system : lando
2019-09-30 13:57:32 +00:00
Nick Alexander 6790ecb3a2 Bug 1550146 - Part 2: Make mach "build" run "stage-package && android assemble-app" for mobile/android. r=glandium
Android artifacts (GeckoView AARs, GeckoViewExample (and Fennec) APKs)
require native libraries (`libxul.so`) and an omnijar (`omni.ja`).
These are produced by `mach package` (really, the `stage-package`
target).  Engineers essentially never want a build without a package
for mobile/android.  This adds mobile/android-only tiers that run
`mach package` and then `mach android assemble-app`.  The latter
consumes `libxul.so` and `omni.ja` to produce _all the things_
relevant to GeckoView engineers.

Differential Revision: https://phabricator.services.mozilla.com/D41450

--HG--
extra : moz-landing-system : lando
2019-08-21 03:07:16 +00:00
Mike Hommey d427c37271 Bug 1570798 - Restore relative path for MOZ_FETCHES_DIR. r=nalexander
Bug 1563711 changed MOZ_FETCHES_DIR to make the MOZ_ANDROID_FAT_AAR_*
environment variables absolute paths. Unfortunately, the replacement
relies on non-osx/windows tasks to run in docker-worker, which is not
necessarily true, and that makes the MOZ_FETCHES_DIR wrong in
non-osx/windows generic-worker tasks.  Apparently, that currently works,
but that's not guaranteed to stay this way.

The MOZ_ANDROID_FAT_AAR_* environment variables don't need to be
absolute paths, though. MOZ_FETCHES_DIR is normalized by run-task, and
MOZ_ANDROID_FAT_AAR_* can be set relative to that, which we do here.

Differential Revision: https://phabricator.services.mozilla.com/D40334
2019-08-03 07:08:39 +09:00
Johan Lorenzo 13648fa0df Bug 1563711 - Remove target.maven.zip r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D38175

--HG--
extra : moz-landing-system : lando
2019-07-30 15:35:12 +00:00
Mike Hommey 6098f22649 Bug 1565706 - Allow to run `mach build rust` without setting MOZ_RUST_TIER. r=froydnj
At the same time, make things a little more tangible in the tiers setup
in baseconfig.mk.

Differential Revision: https://phabricator.services.mozilla.com/D37942

--HG--
extra : moz-landing-system : lando
2019-07-15 23:50:31 +00:00
Mike Shal 26b533deae Bug 1557785 - Process install manifests in 3-tier Windows builds; r=firefox-build-system-reviewers,chmanchester
Windows 1-tier PGO builds only partially clobber between the
profile-generate and profile-use stages, so that exports/installed files
don't have to be reprocessed. Unfortunately we can't skip the install
manifests in 3-tier PGO because the profile-generate build happens on a
different machine, so we have to differentiate between 1-tier and 3-tier
PGO builds. A new variable, MOZ_1TIER_PGO, is used for this purpose.

Eventually this logic can be cleaned up in bug 1557788 once all PGO
builds use the 3-tier model.

Differential Revision: https://phabricator.services.mozilla.com/D34802

--HG--
extra : moz-landing-system : lando
2019-06-19 22:06:14 +00:00
Mike Hommey 6f6e8cbd72 Bug 1551639 - Always update buildid but avoid rebuilding libxul. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D33772

--HG--
extra : moz-landing-system : lando
2019-06-06 04:35:02 +00:00
Nick Alexander 5893476bdb Bug 1508976 - Produce a multi-architecture GeckoView "fat AAR". r=glandium
This follows the model set down for EME artifacts:

- a new tier is added that uses a new Python build action to fetch and
  artifacts
- the action unpacks the fetched artifacts and moves specific inputs
  into places expected by the build and packager
- in automation, MOZ_ARTIFACT_TASK* is used to ensure the artifacts
  come from the correct tasks

In this case, the artifact fetching is done entirely in a new Python
build action that internally uses `mach artifact install --job ...`.
The action also verifies that the fetched artifacts are compatible and
that we're not assembling a fat AAR that is nonsensical.  The specific
inputs are not used in the Fennec APK that is produced; they're only
used in the GeckoView AAR that is produced.

The artifact fetching itself required tweaking to fetch only
`target.maven.zip` artifacts and to not unpack them.

The specific inputs used are the native libraries (libs/$ARCH/*.so)
and the architecture-specific preference files ($ARCH/greprefs.js and
defaults/pref/$ARCH/geckoview-prefs.js).  None of these inputs are
impacted by l10n.

Differential Revision: https://phabricator.services.mozilla.com/D31572

--HG--
extra : moz-landing-system : lando
2019-05-30 15:22:09 +00:00
Nathan Froyd 2b3a15afe9 Bug 1542746 - use a static MOZ_PROFILE_ORDER_FILE; r=nalexander,chmanchester
We're planning on switching to IR-based profiling, so we can't use the
frontend-based instrumentation to collect the order in which functions
are executed...at least not during the build itself.  Performance tests
indicate that not having the order information decreases performance
significantly.  So we're going to check in static files for Win32 and
Win64 and use those to perform the ordering.  It's OK if these files are
slightly out of date; as of this writing, builds that generate and then
use these files complain that ~1/3 of the functions can't be found (!).
We're just trying to do something slightly smarter than whatever the
linker default is.

Differential Revision: https://phabricator.services.mozilla.com/D31132

--HG--
extra : moz-landing-system : lando
2019-05-24 01:54:59 +00:00
Mike Shal e28f6db31f Bug 1537574 - Use mozbuild's backend-out-of-date logic for RecursiveMake; r=firefox-build-system-reviewers,chmanchester
If mozbuild parsing fails due to a missing file (eg: a file not existing
in UNIFIED_SOURCES), then no Makefiles are written out, but
config.status exists. This would cause mozbuild to think that configure
doesn't need to run, and rely on make to perform the backend-out-of-date
check in rebuild-backend.mk. Unfortunately since no Makefiles were
written, the make command fails immediately and no attempt is made to
re-create the backend. Note that this is only a problem if the first
mozbuild parsing from a clobber build fails, otherwise there is
typically a top-level Makefile from a previous build to call into (at
which point make can determine it is out-of-date, and re-invoke itself).

The fix is to have the RecursiveMake backend re-use the same logic that
was introduced into mozbuild for alternate backends, and remove
rebuild-backend.mk. This way, mozbuild can always determine if the
backend needs to be regenerated, even if the initial parsing failed.

Test code was also relying on rebuild-backend.mk to generate the
TestBackend, but moving backend_out_of_date() into MozbuildObject allows
this code to be shared.

Differential Revision: https://phabricator.services.mozilla.com/D26262

--HG--
rename : build/gen_test_backend.py => python/mozbuild/mozbuild/gen_test_backend.py
extra : moz-landing-system : lando
2019-04-10 22:14:27 +00:00
Nathan Froyd a780e0f5a1 Bug 1541068 - add Rust compilation directories to GARBAGE_DIRS; r=nalexander
We add to `GARBAGE_DIRS` in the toplevel `Makefile.in` because all of
our Rust libraries share a single `CARGO_TARGET_DIR`, located in
topobjdir.

We add to `GARBAGE_DIRS` for Rust programs because Rust programs
currently do not share compilation artifacts with Rust libraries (as our
libraries are built with `panic=abort` and our programs are not, sharing
compilation artifacts between the two is a non-starter).

Differential Revision: https://phabricator.services.mozilla.com/D26762

--HG--
extra : moz-landing-system : lando
2019-04-09 18:59:19 +00:00
Andreea Pavel a7f868ac25 Backed out changeset 95b3298fd2d4 (bug 1537574) for windows task timeouts a=backout
--HG--
rename : python/mozbuild/mozbuild/gen_test_backend.py => build/gen_test_backend.py
2019-04-08 22:23:03 +03:00
Mike Shal 96425af79a Bug 1537574 - Use mozbuild's backend-out-of-date logic for RecursiveMake; r=firefox-build-system-reviewers,chmanchester
If mozbuild parsing fails due to a missing file (eg: a file not existing
in UNIFIED_SOURCES), then no Makefiles are written out, but
config.status exists. This would cause mozbuild to think that configure
doesn't need to run, and rely on make to perform the backend-out-of-date
check in rebuild-backend.mk. Unfortunately since no Makefiles were
written, the make command fails immediately and no attempt is made to
re-create the backend. Note that this is only a problem if the first
mozbuild parsing from a clobber build fails, otherwise there is
typically a top-level Makefile from a previous build to call into (at
which point make can determine it is out-of-date, and re-invoke itself).

The fix is to have the RecursiveMake backend re-use the same logic that
was introduced into mozbuild for alternate backends, and remove
rebuild-backend.mk. This way, mozbuild can always determine if the
backend needs to be regenerated, even if the initial parsing failed.

Test code was also relying on rebuild-backend.mk to generate the
TestBackend, but moving backend_out_of_date() into MozbuildObject allows
this code to be shared.

Differential Revision: https://phabricator.services.mozilla.com/D26262

--HG--
rename : build/gen_test_backend.py => python/mozbuild/mozbuild/gen_test_backend.py
extra : moz-landing-system : lando
2019-04-08 16:37:56 +00:00
Mike Hommey c7e738d69d Bug 1535501 - Allow to get full crashreporter symbols on artifact builds. r=mshal
Differential Revision: https://phabricator.services.mozilla.com/D23605

--HG--
extra : moz-landing-system : lando
2019-03-15 21:56:14 +00:00
Mike Hommey e8b6f15845 Bug 1529194 - Allow to opt-in to widevine on aarch64 Windows. r=firefox-build-system-reviewers,chmanchester
This uses artifacts from a win32 build to get the necessary DLLs.

Differential Revision: https://phabricator.services.mozilla.com/D20597

--HG--
extra : moz-landing-system : lando
2019-03-05 08:41:04 +00:00
Mike Hommey fa7300a70b Bug 1531680 - Don't disable LTO on standalone profile-generate builds. r=chmanchester
On one-go MOZ_PGO builds, it's generally not wanted to do LTO during the
profile-generate phase. And the build system doesn't really support
different build options between both phases in this case, so we relied
on MOZ_PROFILE_GENERATE to disable the LTO flags.

However, in standalone profile-generate builds, if --enable-lto is
passed explicitly, the build should respect that choice.

So instead of checking MOZ_PROFILE_GENERATE to disable the LTO flags,
we disable them when MOZ_LTO is not set, and we force it to be disabled
during the profile-generate phase of one-go MOZ_PGO builds.

Differential Revision: https://phabricator.services.mozilla.com/D21659

--HG--
extra : moz-landing-system : lando
2019-03-05 22:09:46 +00:00
Mike Hommey d3af9d50b9 Bug 1529195 - Allow to opt-in to symbols/host-bin artifacts from the CLI. r=chmanchester
and to opt-out of test artifacts.

Depends on D20445

Differential Revision: https://phabricator.services.mozilla.com/D20446

--HG--
extra : moz-landing-system : lando
2019-02-21 22:53:43 +00:00
Mike Hommey ef3ad686ee Bug 1512504 - Remove support for MSVC. r=froydnj
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
  worked in non-ASCII cases.

This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.

Depends on D19614

Differential Revision: https://phabricator.services.mozilla.com/D19615

--HG--
extra : moz-landing-system : lando
2019-02-14 21:45:27 +00:00
Mike Shal 49fc6484f8 Bug 1507334 - Remove '10' argument to profileserver.py; r=firefox-build-system-reviewers,chmanchester#firefox-build-system-reviewers
Support for running profileserver.py multiple times was originally added
in bug 663251 and subsequently removed in bug 672563. Since then we've
been copying around the argument to profileserver.py from mozconfigs in
buildbot-configs, to mozconfigs in mozilla-central, to client.mk, and
ultimately to Makefile.in. It no longer does anything and can be
removed.

Differential Revision: https://phabricator.services.mozilla.com/D15748

--HG--
extra : moz-landing-system : lando
2019-01-10 01:34:39 +00:00
Mike Hommey e489736f60 Bug 1518703 - Remove R_386_PC32 check. r=froydnj,ted
The test is actually redundant with the TEXTREL check in
check_binary.py (R_386_PC32 relocations will only happen as text
relocations)

Differential Revision: https://phabricator.services.mozilla.com/D16002

--HG--
extra : moz-landing-system : lando
2019-01-10 00:07:53 +00:00
Chris Manchester b101c5a8f3 Bug 1517532 - Extend artifact builds to find and download artifacts from pgo builds when MOZ_PGO is set. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D15988

--HG--
extra : moz-landing-system : lando
2019-01-08 23:03:27 +00:00
Chris Manchester 25e14c4f13 Bug 1507338 - Add a configure option for MOZ_PROFILE_GENERATE. r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D12999

--HG--
extra : moz-landing-system : lando
2019-01-03 16:16:38 +00:00
Chris Manchester d69003537d Bug 1498031 - Merge code paths for running configure between Tup and Make based backends. r=firefox-build-system-reviewers,mshal
This addresses a related issue along the way: a build that results in running
configure would not update the value of self.config_environment (and therefore
self.substs) as seen from the build driver, so out of date values would have
been used.

The changes to Makefile.in and client.mk made exploit the assumption that by
he time anything in the Make build is running, config.status is up to date.
Users running "make" without the benefit of "mach" will need to manually run
configure when necessary in order to take this into account.

Differential Revision: https://phabricator.services.mozilla.com/D8473

--HG--
extra : moz-landing-system : lando
2018-10-16 22:21:36 +00:00
Ted Mielczarek e11506c714 bug 1490325 - write sccache log directly to artifact directory, make logs slightly more verbose. r=mshal
This patch makes it so we write the sccache log directly to the artifact
directory, so that it will be uploaded even if the build fails. It also makes
the log slightly more verbose. Both of these should help with diagnosing
sccache failures in CI.

The sccache log will no longer be explicitly gzip compressed, but some
Taskcluster client implementations will store it as gzip compressed.

Differential Revision: https://phabricator.services.mozilla.com/D6187

--HG--
extra : moz-landing-system : lando
2018-09-18 20:02:17 +00:00
Gabriele Svelto b48b3e4db0 Bug 1473821 - The MOZ_PGO_INSTRUMENTED environment variable is not used anymore so remove all remaining mentions of it r=gps
Differential Revision: https://phabricator.services.mozilla.com/D3219

--HG--
extra : moz-landing-system : lando
2018-08-17 16:40:10 +00:00
Mike Hommey f50e1c662a Bug 1483779 - Enable PGO on clang-cl 32-bits builds. r=dmajor
--HG--
extra : source : 5fcbe08fb321a5076f56c380b554c63068efbb00
2018-08-16 16:44:36 +09:00
arthur.iakab 07aa46fd13 Backed out 2 changesets (bug 1479800, bug 1483779)for frequent cgx and arm64 failures
Backed out changeset 5fcbe08fb321 (bug 1483779)
Backed out changeset dccd5299c5ad (bug 1479800)
2018-08-17 09:50:30 +03:00
Mike Hommey 152366b300 Bug 1483779 - Enable PGO on clang-cl 32-bits builds. r=dmajor 2018-08-17 10:19:52 +09:00
Mike Hommey 6d4eb0a92e Bug 1464170 - Support PGO in clang builds on Linux. r=dmajor
This uses the same code path as for clang-cl builds.
2018-08-10 05:54:11 +09:00
David Major 45083c4934 Bug 1341525: Enable PGO in 64-bit clang-cl builds. r=froydnj 2018-08-01 17:59:23 -04:00
Andreea Pavel 11f1c09ffe Backed out changeset 7a3882f27aa3 (bug 1341525) for failing win Sa builds on a CLOSED TREE
--HG--
extra : amend_source : dbf7c208749c67b6bb8674eb5ba55c6514f89a2d
2018-08-02 02:26:13 +03:00
David Major ebe3264f11 Bug 1341525: Enable PGO in 64-bit clang-cl builds. r=froydnj 2018-08-01 17:59:23 -04:00
Andreea Pavel 250fd9a027 Backed out changeset 2011dae40293 (bug 1341525) for failing win Sa builds 2018-08-02 01:33:30 +03:00
David Major 622dccd28f Bug 1341525: Enable PGO in 64-bit clang-cl builds. r=froydnj
--HG--
extra : rebase_source : 06038cb081f7b2b2789a8b867b03340bf103b05e
2018-08-01 17:59:23 -04:00
Chris AtLee 38cced543a Bug 1466537: Disable full crashsymbols by branch r=ted
Differential Revision: https://phabricator.services.mozilla.com/D1558

--HG--
extra : moz-landing-system : lando
2018-07-30 20:23:14 +00:00
Aaron Klotz 9e27dd8849 Bug 1473423: Allow test symbols to be included in full symbol archive in try builds; r=ted
This patch moves the logic for include/exclude/compress from the makefile into
symbols_archive.py, and adds an additional check to the full-archive case to
allow the inclusion of test symbols in try builds.

--HG--
extra : amend_source : 306419aae90a1935ced49fec89112df05142501c
2018-07-11 17:38:37 -06:00
Nathan Froyd 5c5a455e97 Bug 1444171 - use the order file during re-link with clang-cl; r=glandium
Now that we've generated an order file of the first N functions invoked
during startup, let's tell the linker about said functions so it can
cluster them appropriately.
2018-07-09 18:36:14 -04:00