The mozharness transform is supposed to set the docker image to
desktop-build when not already set, but was not doing it properly.
I guess this is why some jobs were setting the image themselves, despite
using the mozharness transform.
Consequently, don't manually set the image to desktop-build when it's
the default.
--HG--
extra : rebase_source : 024bd10960bedaee3416785348a5c12498c5286f
It turns out that in all cases it was the last tooltool manifest entry,
so we can remove the tooltool manifests entirely, and remove all
references to them.
--HG--
extra : rebase_source : d8447b5422e63e88444008fddb76d658829694de
It turns out that in all cases it was the last tooltool manifest entry,
so we can remove the tooltool manifests entirely, and remove all
references to them.
--HG--
extra : rebase_source : 0aa9ef8151c2fccf62507dfecc0bc57b157772e1
They just inherited the dependencies because they were using the same
tooltool manifests as compiling builds.
--HG--
extra : rebase_source : 03ee8180e455e01b72016b3132001745b87f78e7
There are essentially four categories of jobs that have dependencies on
sccache currently and that shouldn't:
- jobs that don't compile anything. They just inherited the dependency
because they were using the same tooltool manifests as compiling
builds.
- jobs that don't use sccache. Ideally, we'd make them use sccache, but
things are not currently setup to make that easy, so we'll keep that for
later.
- jobs that explicitly disable sccache through needs-sccache: false.
Like above, ideally, they would use sccache.
- jobs that can't use sccache. Those are hazard jobs, that rely on a GCC
plugin and AIUI on a global knowledge of the code, which the plugin needs
to see. Caching would break that.
--HG--
extra : rebase_source : 77455b9f0a58919838c8c64c36aa1db99baf8c7e
This leaves out fuzzing and linux static analysis builds, which are
using, respectively, clang 4.0.1 and clang 3.8, while linux64-clang
produces a 3.9 and win*-clang a 5.0
--HG--
extra : rebase_source : 45128ac74bf4fe7e6a2ace57043c34ecdf0fe929
Such a definition automatically sets up the corresponding dependencies
in the taskgraph, and adds the necessary artifact definitions for use in
the corresponding jobs. The jobs end up with a MOZ_TOOLCHAINS
environment variable with a list of path@task-id strings, where task-id
is corresponding to the (possibly optimized) toolchain job, and path
corresponding to the toolchain-artifact defined for that toolchain job.
--HG--
extra : rebase_source : b2d297bd75d9c416b30d2a6c6d61efcb64681727
Since bug 1321847, mozharness tooltool manifests can be overriden from
the environment. We use that possibility to now define tooltool
manifests from taskcluster job definitions. Ideally, we'd also remove
the definitions from the mozharness configs, but with things still
running on buildbot, it's not clear what things might break because of
that. We'll do it in a separately back-out-able followup.
--HG--
extra : rebase_source : 860b8f1d4fdc4a557770a3749055f19b1ec45e93
Previously, mozharness defined a separate action to collect build
metrics. This required the script and/or config to define that
action.
Metrics collection for CI is important. So it should be enabled by
default.
This commit changes the "build" action/method to always call the
metrics collection function after successful build. References to
the "generate-build-stats" action have been removed because it is
redundant.
A side-effect of this change is we may generate build metrics where
we weren't before. This could lead to e.g. duplicate entries in some
Perfherder series. Let's see what breaks ;)
MozReview-Commit-ID: 42UQI5YQTMC
--HG--
extra : rebase_source : c57dc9ec6ac46003384edff098a0ad81c75539b7
extra : source : c9812dd7d27a174c0ee46d44ec595fbe29c9e1db
To date we have variously specified both worker-type and worker-implementation,
often manually coordinated. We also embedded a few awkward assumptions such as
that the native engine only runs on OS X.
But a worker type has one and only one implementation, and that implementation
is stable over time (as changing it would require simultaneous landings on all
trees).
Instead, this change makes worker-type the primary configuration, and derives
both a worker implementation (defining the payload format) and worker OS
(determining what to include in the payload) from that value. The derivation
occurs when deciding how to implement a particular job, where the run_using
functions are distinguished by worker implementation.
The two-part logic to determine how and where to run a test task based on its
platform is combined into a single transform, `set_worker_type`.
This contains some other related changes:
- MOZ_AUTOMATION is set in specific jobs, rather than everywhere docker-worker
is used
- the URL to test packages is factored out into a shared function
- docker-worker test defaults are applied in `mozharness_test.py`
- the WORKER_TYPE array in `task.py`, formerly mixing two types of keys, is
split
- the 'invalid' workerType is assigned an 'invalid' implementation
- all tasks that do not use job descriptions but use docker-worker, etc. have
`worker.os` added
Tested to not produce a substantially different taskgraph for a regular push, a
try push, or a nightly cron.
MozReview-Commit-ID: LDHrmrpBo7I
--HG--
extra : rebase_source : 4cdfe6b8d9874b0c156671515b213d820b48482f
Instead of using a class's static method, use a simple function, specified by
the `loader` key.
MozReview-Commit-ID: IeOl9qiSCXf
--HG--
extra : rebase_source : 72e0a9dd8385b250a46c9f4adf8a8a0e5b01c156
Tom Prince
Mike Hommey
Coroiu Cristina
Steve Armand
Dustin J. Mitchell
Ryan VanderMeulen
Gregory Szorc