mozilla
/
gecko-dev
зеркало из https://github.com/mozilla/gecko-dev.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Bug 1725125 - Make a hybrid build system combining non-unified with unified. CLOSED TREE 

By hybrid unified system we understand a system that encapsulates modules that are built in the unified mode
but also other modules, like `dom/Animation`, as an example, in the non unified environment.

This approach is desirable since we already have most of the modules transitioned to the non unified system but there are
still some that are not yet compatible, but in the long term this will be done by each module owner and can be also tested
locally using the build system.

If a module can't be built outside the unified method it's `moz.build` config file needs to have `REQUIRES_UNIFIED_BUILD = False`

To also enable this we need to have a flag from `mozconfig`, like:

```
ac_add_options --disable-unified-build
```

Differential Revision: https://phabricator.services.mozilla.com/D122328
This commit is contained in:
Andi-Bogdan Postelnicu 2021-08-25 11:21:42 +00:00
Родитель 50dd0540cd
Коммит 0ac52aa9d7
9 изменённых файлов: 171 добавлений и 1 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

4
browser/config/mozconfigs/linux64/hybrid Normal file
Просмотреть файл

@ -0,0 +1,4 @@
. $topsrcdir/browser/config/mozconfigs/linux64/nightly
ac_add_options --disable-unified-build
ac_add_options --disable-warnings-as-errors

4
browser/config/mozconfigs/macosx64/hybrid Normal file
Просмотреть файл

@ -0,0 +1,4 @@
. $topsrcdir/browser/config/mozconfigs/macosx64/nightly
ac_add_options --disable-unified-build
ac_add_options --disable-warnings-as-errors

4
browser/config/mozconfigs/win64/hybrid Normal file
Просмотреть файл

@ -0,0 +1,4 @@
. $topsrcdir/browser/config/mozconfigs/win64/nightly
ac_add_options --disable-unified-build
ac_add_options --disable-warnings-as-errors

16
build/docs/unified-builds.rst
Просмотреть файл

@ -31,6 +31,22 @@ together for a single compilation, so sometimes the addition of a new file will
to be bumped into a different chunk. If that other chunk doesn't meet the implicit requirements
of the bumped file, there will be a tough-to-debug compilation failure.
Building outside of the unified environment
===========================================
As described above, unified builds can cause source files to implicitly depend on each other, which
not only causes unexpected build failures but also can cause issues when using source-analysis tools.
To combat this, we'll use a "hybrid" build that attempts to perform a build with as many files compiled
individually as possible.
Due to the implicit dependency problem, not all modules are able to be compiled in a non-unified
environment yet. To designate these for the hybrid build, the ``REQUIRES_UNIFIED_BUILD`` option can be
set in their corresponding ``moz.build`` file.
To build in the hybrid mode, set the following flag in your ``mozconfig``:
``ac_add_options --disable-unified-build``
Other notes:
============

11
moz.configure
Просмотреть файл

@ -179,6 +179,17 @@ add_old_configure_assignment("DEVELOPER_OPTIONS", developer_options)
set_config("DEVELOPER_OPTIONS", developer_options)
# hybrid build handling
# ==============================================================
option(
"--disable-unified-build",
help="Enable building modules that are not marked with `REQUIRES_UNIFIED_BUILD` in non unified context",
)
set_config("ENABLE_UNIFIED_BUILD", True, when="--disable-unified-build")
option(
env="MOZ_FETCHES_DIR",
nargs=1,

10
python/mozbuild/mozbuild/frontend/emitter.py
Просмотреть файл

@ -649,6 +649,8 @@ class TreeMetadataEmitter(LoggingMixin):
host_linkables = []
wasm_linkables = []
unified_build = context.config.substs.get("ENABLE_UNIFIED_BUILD", False)
def add_program(prog, var):
if var.startswith("HOST_"):
host_linkables.append(prog)
@ -1129,7 +1131,13 @@ class TreeMetadataEmitter(LoggingMixin):
self._asm_compile_dirs.add(context.objdir)
arglist = [context, list(files), canonical_suffix]
if variable.startswith("UNIFIED_"):
arglist.append(context.get("FILES_PER_UNIFIED_FILE", 16))
if (
unified_build is False
and context.get("REQUIRES_UNIFIED_BUILD", False) is False
):
arglist.append(1)
else:
arglist.append(context.get("FILES_PER_UNIFIED_FILE", 16))
obj = cls(*arglist)
srcs = list(obj.files)
if isinstance(obj, UnifiedSources) and obj.have_unified_mapping:

42
taskcluster/ci/build/linux.yml
Просмотреть файл

@ -127,6 +127,48 @@ linux64-plain/opt:
optimization:
skip-unless-expanded: null
linux64-hybrid/plain:
description: "Linux64 Plain Hybrid"
index:
product: firefox
job-name: linux64-hybrid-plain
treeherder:
platform: linux64/plain
symbol: Bp-hybrid
tier: 1
worker-type: b-linux
worker:
docker-image: {in-tree: debian11-amd64-build}
max-run-time: 5400
env:
PERFHERDER_EXTRA_OPTIONS: hybrid
run:
using: mozharness
actions: [get-secrets, build]
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
script: mozharness/scripts/fx_desktop_build.py
secrets: true
tooltool-downloads: public
extra-config:
disable_package_metrics: true
mozconfig-variant: hybrid
run-on-projects: ['integration']
use-sccache: true
fetches:
toolchain:
- linux64-binutils
- linux64-clang
- linux64-rust
- linux64-nasm
- linux64-node
- linux64-cbindgen
- linux64-sccache
- linux64-dump_syms
- sysroot-x86_64-linux-gnu
- sysroot-wasm32-wasi
linux64-gcc/opt:
description: "Linux64 GCC Opt"
index:

29
taskcluster/ci/build/macosx.yml
Просмотреть файл

@ -57,6 +57,35 @@ macosx64/debug:
- macosx64-sdk-10.12
- sysroot-wasm32-wasi
macosx64-hybrid/plain:
description: "MacOS X x64 Hybrid Cross-compile"
index:
job-name: macosx64-hybrid
treeherder:
platform: osx-cross/plain
symbol: Bp-hybrid
tier: 1
worker:
max-run-time: 5400
env:
PERFHERDER_EXTRA_OPTIONS: hybrid
run:
config:
- builds/releng_base_firefox.py
- builds/releng_base_mac_64_cross_builds.py
mozconfig-variant: hybrid
extra-config:
disable_package_metrics: true
use-sccache: true
run-on-projects: ['integration']
fetches:
toolchain:
- linux64-cctools-port
- linux64-clang-macosx-cross
- linux64-sccache
- macosx64-sdk-10.12
- sysroot-wasm32-wasi
macosx64/opt:
description: "MacOS X x64 Cross-compile"
index:

52
taskcluster/ci/build/windows.yml
Просмотреть файл

@ -174,6 +174,58 @@ win64/debug:
- nsis-3.01-win
- upx-3.95-win
win64-hybrid/plain:
description: "Win64 Hybrid Plain"
index:
product: firefox
job-name: win64-hybrid-plain
treeherder:
platform: windows2012-64/debug
symbol: Bp-hybrid
tier: 1
worker-type: b-linux
worker:
docker-image: {in-tree: debian11-amd64-build}
max-run-time: 7200
env:
PERFHERDER_EXTRA_OPTIONS: hybrid
TOOLTOOL_MANIFEST: "browser/config/tooltool-manifests/vs2017-15.8.manifest"
run:
options: [append-env-variables-from-configs]
actions: [get-secrets, build]
script: mozharness/scripts/fx_desktop_build.py
secrets: true
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
mozconfig-variant: hybrid
extra-config:
mozconfig_platform: win64
env:
LD_PRELOAD: "/builds/worker/fetches/liblowercase/liblowercase.so"
LOWERCASE_DIRS: "/builds/worker/checkouts/gecko/vs2017_15.8.4"
use-sccache: true
run-on-projects: ['integration']
fetches:
toolchain:
- linux64-binutils
- linux64-clang-win-cross
- linux64-rust-cross
- linux64-rust-size
- linux64-nasm
- linux64-node
- linux64-cbindgen
- linux64-sccache
- linux64-dump_syms
- linux64-wine
- linux64-liblowercase
- linux64-winchecksec
- sysroot-x86_64-linux-gnu
- sysroot-wasm32-wasi
fetch:
- nsis-3.01-win
- upx-3.95-win
win64-fuzzing/debug:
description: "Win64 Fuzzing Debug"
index: