There are now only a handful of buildbot jobs remaining and the concern over
outdated treeherder exclusion profiles has largely been resolved.
This does remove the tc() group from a substantial number of tasks which will
now show up as top level tasks, potentially adding clutter. In some cases, we
might want to re-add a new group (e.g group builds or compiled tests together).
However rather than try to predict the best group names for tasks I'm unfamiliar
with, I think it's best to land this as is. Then if things are looking too
cluttered at the root namespace, file follow-up bugs as needed.
MozReview-Commit-ID: 8SMwjDwAOzV
--HG--
extra : rebase_source : 2f6d89d11c139bdcd404e7537db799d0e36ee4c3
The mozharness transform is supposed to set the docker image to
desktop-build when not already set, but was not doing it properly.
I guess this is why some jobs were setting the image themselves, despite
using the mozharness transform.
Consequently, don't manually set the image to desktop-build when it's
the default.
--HG--
extra : rebase_source : 024bd10960bedaee3416785348a5c12498c5286f
also add devedition and the missing linux{,64} balrog_props.json (by
defining taskcluster_nightly configs for devedition).
MozReview-Commit-ID: 3MAYjSL20FV
--HG--
extra : rebase_source : 45c7d6e63c18f77d9434cebeb65d05608f7d2508
extra : histedit_source : 13b4d908a02571f6c8506fe27de2edfe7342f424
Add a toolchain job description which calls the
repack_rust.py script to package the requested
upstream build of Rust and its standard libraries
for use in gecko builds.
Links are added to these new toolchains for various build
and analysis tasks as appropriate. The base-toolchain
tasks use an explicitly-versioned toolchain since those
can be different from the current release used for most builds.
The corresponding tooltool manifest entries are removed
now that taskcluster artifact versions are available.
This simplifies the update process since new toolchains
can be packaged and used automatically by just updating
the versions in the task descriptions.
A 'linux64-rust' toolchain can be added to other tasks
as a dependency and artifact. It supports linux64-
hosted builds of Rust code targeting linux64 or linux32.
A 'linux64-rust-macos' toolchain targets linux64-hosted
builds of Rust code targeting macOS on x86_64.
A 'linux64-rust-android' toolchain targets linux64-hosted
builds of Rust code targeting various Android architectures.
Two 'win64-rust' and 'win32-rust' toolchain tasks create
similar entries for Windows-hosted builds. All our automation
builds are hosted on win64, so we could use one artifact
with support for both targets, but currently this doesn't
work because of cross-compilation issues in some crates.
This patch maintains the previous separation between
win32 and win64 rust toolchains until that can be addressed.
MozReview-Commit-ID: GRiJml8CtzO
--HG--
extra : rebase_source : 09a3698ce7f9a8b5f2b5d9b5a1fde9c05dc6b540
This leaves out fuzzing and linux static analysis builds, which are
using, respectively, clang 4.0.1 and clang 3.8, while linux64-clang
produces a 3.9 and win*-clang a 5.0
--HG--
extra : rebase_source : 45128ac74bf4fe7e6a2ace57043c34ecdf0fe929
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: 4HgW0D2BR8B
--HG--
extra : rebase_source : b87b1159307205cebd10e1d8598386aac3976706
This adds the mozconfigs, mozharness configs and taskcluster changes required
to create optimized DMD builds for linux64, win32, win64 and macosx64.
These builds will happen nightly on mozilla-central
We also add support for custom build variants on Windows (or other generic
worker environments).
MozReview-Commit-ID: HrVT9PLSWVx
--HG--
extra : rebase_source : 39ac752a312afe04187728da82a4a7f722634811
This patch enables OSX cross compiled jobs as tier 1, and additionally adds testing for OSX Nightlies via Taskcluster. Initially tested on the 'date' project branch primarily with Bug 1373384
MozReview-Commit-ID: 7nWfiSEDbJd
--HG--
extra : rebase_source : 2a1a83e94ecead90763a7ce0a614e5daed44275d
extra : source : 68ff1aa745ff36119abde734ed313209b7730ea4
{central, autoland, inbound} are logically treated as a single unit for
many tasks and policies. Let's formalize that collection via a "trunk"
alias.
MozReview-Commit-ID: H4JPTyu2J2F
--HG--
extra : rebase_source : f4cabfc48e24b9f55b833bd95bdbf81b036cee6d
This refers to the incubator/stylo repo, which has been dead for a
few months ever since Stylo was merged into mozilla-central. Let's
remove references to it.
MozReview-Commit-ID: 35vWUdlqqus
--HG--
extra : rebase_source : 2f5e8eb1bab1c3d538136bf033e6447ad0429362
Since bug 1321847, mozharness tooltool manifests can be overriden from
the environment. We use that possibility to now define tooltool
manifests from taskcluster job definitions. Ideally, we'd also remove
the definitions from the mozharness configs, but with things still
running on buildbot, it's not clear what things might break because of
that. We'll do it in a separately back-out-able followup.
--HG--
extra : rebase_source : 860b8f1d4fdc4a557770a3749055f19b1ec45e93
Previously, mozharness defined a separate action to collect build
metrics. This required the script and/or config to define that
action.
Metrics collection for CI is important. So it should be enabled by
default.
This commit changes the "build" action/method to always call the
metrics collection function after successful build. References to
the "generate-build-stats" action have been removed because it is
redundant.
A side-effect of this change is we may generate build metrics where
we weren't before. This could lead to e.g. duplicate entries in some
Perfherder series. Let's see what breaks ;)
MozReview-Commit-ID: 42UQI5YQTMC
--HG--
extra : rebase_source : c57dc9ec6ac46003384edff098a0ad81c75539b7
extra : source : c9812dd7d27a174c0ee46d44ec595fbe29c9e1db
To date we have variously specified both worker-type and worker-implementation,
often manually coordinated. We also embedded a few awkward assumptions such as
that the native engine only runs on OS X.
But a worker type has one and only one implementation, and that implementation
is stable over time (as changing it would require simultaneous landings on all
trees).
Instead, this change makes worker-type the primary configuration, and derives
both a worker implementation (defining the payload format) and worker OS
(determining what to include in the payload) from that value. The derivation
occurs when deciding how to implement a particular job, where the run_using
functions are distinguished by worker implementation.
The two-part logic to determine how and where to run a test task based on its
platform is combined into a single transform, `set_worker_type`.
This contains some other related changes:
- MOZ_AUTOMATION is set in specific jobs, rather than everywhere docker-worker
is used
- the URL to test packages is factored out into a shared function
- docker-worker test defaults are applied in `mozharness_test.py`
- the WORKER_TYPE array in `task.py`, formerly mixing two types of keys, is
split
- the 'invalid' workerType is assigned an 'invalid' implementation
- all tasks that do not use job descriptions but use docker-worker, etc. have
`worker.os` added
Tested to not produce a substantially different taskgraph for a regular push, a
try push, or a nightly cron.
MozReview-Commit-ID: LDHrmrpBo7I
--HG--
extra : rebase_source : 4cdfe6b8d9874b0c156671515b213d820b48482f
As of bug 1342503 being fixed, all of our desktop firefox builds have
webrender compiled in by default. Webrender can therefore be enabled at
runtime either by a pref or environment variable on any desktop firefox
build. The old builds that we originally used to stand up webrender are
no longer needed, as the *only* difference between them and the regular
builds are that they build with the pref turned on instead of turned
off. This doesn't warrant keeping around these extra builds, and this
patch removes them along with all the associated goop that was needed to
configure them.
MozReview-Commit-ID: 5wlOWo11fEk
--HG--
extra : rebase_source : 696afdd2d9fb5f7932d0737a7d71c3aa6af0bd64
Because of bug 1338651, we need to stick to BBB macosx64 builds for now.
MozReview-Commit-ID: AwQc5r6ikUN
--HG--
extra : transplant_source : %97%1B%95H%D8L1%87%84h%F4%89%0D%283w%98%7E%2C%F4
Adds l10n and nightly indexing, matching (better) what Buildbot is currently doing with these types of tasks
(This patch is against `date`, will be grafted on review for real landing, using autoland)
MozReview-Commit-ID: K0BYwaCm6xL
--HG--
extra : rebase_source : b3254f6cc3f6f526ec3877add890d12a8947080c
Tom Prince
Andrew Halberstadt
Mike Hommey
Aki Sasaki
Steve Armand
Sebastian Hengst
Chris AtLee
Ralph Giles
Rail Aliiev
Justin Wood
Gregory Arndt
Wander Lairson Costa
Gregory Szorc
Nathan Froyd
Ryan VanderMeulen
Dustin J. Mitchell
Kartikaya Gupta
Kim Moir
Wes Kocher
Mike Shal
Haard Panchal