Bug 1652098 - move linux base toolchain builds to their own file; r=dmajor

The explanatory comment at the top of the new file provides some context.
The larger motivating context here is wanting to use `job-defaults` to clean
up the toolchain fetches for the Linux build jobs.  But if we did that, we
run into the problem that such fetches are merged with whatever the
individual job(s) specify for their toolchain fetches, not replaced.  So
we'd wind up with multiple clang toolchains being downloaded, or similar,
and things would go downhill from there, as workers attempt to unpack those
things to the same location (which is bad) or perhaps error out (which is
also bad).

This change therefore paves the way for providing `job-defaults` in linux.yml.

Differential Revision: https://phabricator.services.mozilla.com/D83158
This commit is contained in:
Nathan Froyd 2020-07-14 13:55:31 +00:00
Родитель 3d8845fae5
Коммит aa851e0e30
3 изменённых файлов: 168 добавлений и 160 удалений

Просмотреть файл

@ -20,6 +20,7 @@ jobs-from:
- android.yml
- android-stuff.yml
- linux.yml
- linux-base-toolchains.yml
- macosx.yml
- windows.yml
- windows-mingw.yml

Просмотреть файл

@ -0,0 +1,167 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
# We have these build jobs in a separate file because their definitions are
# different enough from the main Linux build jobs that their presence in
# linux.yml would make using job-defaults there significantly less useful.
---
linux64-base-toolchains/opt:
description: "Linux64 base toolchains Opt"
index:
product: firefox
job-name: linux64-base-toolchains-opt
treeherder:
platform: linux64/opt
symbol: Bb
run-on-projects: ['mozilla-central']
worker-type: b-linux
worker:
docker-image: {in-tree: debian8-amd64-build-base}
max-run-time: 7200
env:
MOZ_LOW_PARALLELISM_BUILD: '1'
PERFHERDER_EXTRA_OPTIONS: base-toolchains
FORCE_GCC: '1'
run:
using: mozharness
actions: [get-secrets, build]
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
script: "mozharness/scripts/fx_desktop_build.py"
secrets: true
tooltool-downloads: public
need-xvfb: true
use-sccache: true
fetches:
toolchain:
- linux64-clang-5.0
- linux64-gcc-7
- linux64-rust-1.43
- linux64-cbindgen
- linux64-sccache
- linux64-nasm-2.14.02
- linux64-node
linux64-base-toolchains/debug:
description: "Linux64 base toolchains Debug"
index:
product: firefox
job-name: linux64-base-toolchains-debug
treeherder:
platform: linux64/debug
symbol: Bb
optimization:
push-interval-10: null
worker-type: b-linux
worker:
docker-image: {in-tree: debian8-amd64-build-base}
max-run-time: 7200
env:
MOZ_LOW_PARALLELISM_BUILD: '1'
PERFHERDER_EXTRA_OPTIONS: base-toolchains
FORCE_GCC: '1'
run:
using: mozharness
actions: [get-secrets, build]
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
script: "mozharness/scripts/fx_desktop_build.py"
secrets: true
custom-build-variant-cfg: debug
mozconfig-variant: debug
tooltool-downloads: public
need-xvfb: true
use-sccache: true
fetches:
toolchain:
- linux64-clang-5.0
- linux64-gcc-7
- linux64-rust-1.43
- linux64-cbindgen
- linux64-sccache
- linux64-nasm-2.14.02
- linux64-node
# `fix-stacks` is needed because xpcshell self tests run as part of
# the build step. Once bug 1614626 moves them to the xpcshell task,
# this dependency can be removed.
- linux64-fix-stacks
linux64-base-toolchains-clang/opt:
description: "Linux64 base toolchains clang Opt"
index:
product: firefox
job-name: linux64-base-toolchains-clang-opt
treeherder:
platform: linux64/opt
symbol: Bbc
run-on-projects: ['mozilla-central']
worker-type: b-linux
worker:
docker-image: {in-tree: debian8-amd64-build-base}
max-run-time: 7200
env:
PERFHERDER_EXTRA_OPTIONS: base-toolchains-clang
DISABLE_CLANG_PLUGIN: '1'
run:
using: mozharness
actions: [get-secrets, build]
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
script: "mozharness/scripts/fx_desktop_build.py"
secrets: true
tooltool-downloads: public
need-xvfb: true
fetches:
toolchain:
- linux64-binutils
- linux64-clang-5.0
- linux64-rust-1.43
- linux64-cbindgen
- linux64-nasm-2.14.02
- linux64-node
linux64-base-toolchains-clang/debug:
description: "Linux64 base toolchains clang Debug"
index:
product: firefox
job-name: linux64-base-toolchains-clang-debug
treeherder:
platform: linux64/debug
symbol: Bbc
optimization:
push-interval-10: null
worker-type: b-linux
worker:
docker-image: {in-tree: debian8-amd64-build-base}
max-run-time: 5400
env:
PERFHERDER_EXTRA_OPTIONS: base-toolchains-clang
DISABLE_CLANG_PLUGIN: '1'
run:
using: mozharness
actions: [get-secrets, build]
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
script: "mozharness/scripts/fx_desktop_build.py"
secrets: true
custom-build-variant-cfg: debug
mozconfig-variant: debug
tooltool-downloads: public
need-xvfb: true
fetches:
toolchain:
- linux64-binutils
- linux64-clang-5.0
- linux64-rust-1.43
- linux64-cbindgen
- linux64-nasm-2.14.02
- linux64-node
# `fix-stacks` is needed because xpcshell self tests run as part of
# the build step. Once bug 1614626 moves them to the xpcshell task,
# this dependency can be removed.
- linux64-fix-stacks

Просмотреть файл

@ -307,166 +307,6 @@ linux64-devedition/opt:
- linux64-lucetc
- wasi-sysroot
linux64-base-toolchains/opt:
description: "Linux64 base toolchains Opt"
index:
product: firefox
job-name: linux64-base-toolchains-opt
treeherder:
platform: linux64/opt
symbol: Bb
run-on-projects: ['mozilla-central']
worker-type: b-linux
worker:
docker-image: {in-tree: debian8-amd64-build-base}
max-run-time: 7200
env:
MOZ_LOW_PARALLELISM_BUILD: '1'
PERFHERDER_EXTRA_OPTIONS: base-toolchains
FORCE_GCC: '1'
run:
using: mozharness
actions: [get-secrets, build]
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
script: "mozharness/scripts/fx_desktop_build.py"
secrets: true
tooltool-downloads: public
need-xvfb: true
use-sccache: true
fetches:
toolchain:
- linux64-clang-5.0
- linux64-gcc-7
- linux64-rust-1.43
- linux64-cbindgen
- linux64-sccache
- linux64-nasm-2.14.02
- linux64-node
linux64-base-toolchains/debug:
description: "Linux64 base toolchains Debug"
index:
product: firefox
job-name: linux64-base-toolchains-debug
treeherder:
platform: linux64/debug
symbol: Bb
optimization:
push-interval-10: null
worker-type: b-linux
worker:
docker-image: {in-tree: debian8-amd64-build-base}
max-run-time: 7200
env:
MOZ_LOW_PARALLELISM_BUILD: '1'
PERFHERDER_EXTRA_OPTIONS: base-toolchains
FORCE_GCC: '1'
run:
using: mozharness
actions: [get-secrets, build]
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
script: "mozharness/scripts/fx_desktop_build.py"
secrets: true
custom-build-variant-cfg: debug
mozconfig-variant: debug
tooltool-downloads: public
need-xvfb: true
use-sccache: true
fetches:
toolchain:
- linux64-clang-5.0
- linux64-gcc-7
- linux64-rust-1.43
- linux64-cbindgen
- linux64-sccache
- linux64-nasm-2.14.02
- linux64-node
# `fix-stacks` is needed because xpcshell self tests run as part of
# the build step. Once bug 1614626 moves them to the xpcshell task,
# this dependency can be removed.
- linux64-fix-stacks
linux64-base-toolchains-clang/opt:
description: "Linux64 base toolchains clang Opt"
index:
product: firefox
job-name: linux64-base-toolchains-clang-opt
treeherder:
platform: linux64/opt
symbol: Bbc
run-on-projects: ['mozilla-central']
worker-type: b-linux
worker:
docker-image: {in-tree: debian8-amd64-build-base}
max-run-time: 7200
env:
PERFHERDER_EXTRA_OPTIONS: base-toolchains-clang
DISABLE_CLANG_PLUGIN: '1'
run:
using: mozharness
actions: [get-secrets, build]
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
script: "mozharness/scripts/fx_desktop_build.py"
secrets: true
tooltool-downloads: public
need-xvfb: true
fetches:
toolchain:
- linux64-binutils
- linux64-clang-5.0
- linux64-rust-1.43
- linux64-cbindgen
- linux64-nasm-2.14.02
- linux64-node
linux64-base-toolchains-clang/debug:
description: "Linux64 base toolchains clang Debug"
index:
product: firefox
job-name: linux64-base-toolchains-clang-debug
treeherder:
platform: linux64/debug
symbol: Bbc
optimization:
push-interval-10: null
worker-type: b-linux
worker:
docker-image: {in-tree: debian8-amd64-build-base}
max-run-time: 5400
env:
PERFHERDER_EXTRA_OPTIONS: base-toolchains-clang
DISABLE_CLANG_PLUGIN: '1'
run:
using: mozharness
actions: [get-secrets, build]
config:
- builds/releng_base_firefox.py
- builds/releng_base_linux_64_builds.py
script: "mozharness/scripts/fx_desktop_build.py"
secrets: true
custom-build-variant-cfg: debug
mozconfig-variant: debug
tooltool-downloads: public
need-xvfb: true
fetches:
toolchain:
- linux64-binutils
- linux64-clang-5.0
- linux64-rust-1.43
- linux64-cbindgen
- linux64-nasm-2.14.02
- linux64-node
# `fix-stacks` is needed because xpcshell self tests run as part of
# the build step. Once bug 1614626 moves them to the xpcshell task,
# this dependency can be removed.
- linux64-fix-stacks
linux/opt:
description: "Linux32 Opt"
index: