This creates a new "job-from" field that contains the relative filename the job was defined
in. The filename is relative to 'config.path'. If the task came from the 'jobs' key defined
in kind.yml, this field will be set to 'kind.yml'.
MozReview-Commit-ID: 9e1tEb6XuZT
--HG--
rename : taskcluster/ci/source-test/python-tests.yml => taskcluster/ci/source-test/python.yml
extra : rebase_source : 89010f00a96f88a4eecc8b37028f26b483b820ec
The default value for run-on-projects is "all." Many of the tasks
changed in this commit were defining "run-on-projects" to
integration+release, which is essentially all (notably missing "try"). I
can't think of a good reason to not schedule these lint tasks on all
repos by default. So just do that.
MozReview-Commit-ID: EowUGNzC7K3
--HG--
extra : rebase_source : 563abb0059e833881c8d484113aa9ee773b9af27
To date we have variously specified both worker-type and worker-implementation,
often manually coordinated. We also embedded a few awkward assumptions such as
that the native engine only runs on OS X.
But a worker type has one and only one implementation, and that implementation
is stable over time (as changing it would require simultaneous landings on all
trees).
Instead, this change makes worker-type the primary configuration, and derives
both a worker implementation (defining the payload format) and worker OS
(determining what to include in the payload) from that value. The derivation
occurs when deciding how to implement a particular job, where the run_using
functions are distinguished by worker implementation.
The two-part logic to determine how and where to run a test task based on its
platform is combined into a single transform, `set_worker_type`.
This contains some other related changes:
- MOZ_AUTOMATION is set in specific jobs, rather than everywhere docker-worker
is used
- the URL to test packages is factored out into a shared function
- docker-worker test defaults are applied in `mozharness_test.py`
- the WORKER_TYPE array in `task.py`, formerly mixing two types of keys, is
split
- the 'invalid' workerType is assigned an 'invalid' implementation
- all tasks that do not use job descriptions but use docker-worker, etc. have
`worker.os` added
Tested to not produce a substantially different taskgraph for a regular push, a
try push, or a nightly cron.
MozReview-Commit-ID: LDHrmrpBo7I
--HG--
extra : rebase_source : 4cdfe6b8d9874b0c156671515b213d820b48482f
b2gtest uses *.xlarge instances, so it seems a reasonable substitution.
MozReview-Commit-ID: Ev6tnNpx13c
--HG--
extra : rebase_source : 87184b182d1428f041a5ae36e465d72dee6f7621
This clears up some confusion and undocumented behavior around platforms in
source-tests (and available to any job).
With this change, the attributes come out like this:
"source-test-mozbase-linux64/opt": {
"attributes": {
"build_platform": "linux64",
"build_type": "opt",
"kind": "source-test",
"run_on_projects": [
"integration",
"release"
]
},
MozReview-Commit-ID: HN1Zi8YUf0
--HG--
extra : rebase_source : 552bffc4646a3eec46e7edb508d8eb4d2a8e2e03
The name 'source-check' is a bit of a misnomer, because it already includes a bunch
of tasks that are proper unittests, not lints. Some of these unittests will soon
depend on a build task, which makes 'source-check' feel even more wrong.
They still have a lot in common with the lint tasks though, so it's probably not
worth splitting this into two separate kinds. Instead, let's just rename this kind
to 'source-test', which means, any task that tests stuff and is run from the source
directory (instead of a tests.zip). I think both lints and python-tests fall squarely
under this definition.
MozReview-Commit-ID: K0gZ5rVLyeD
--HG--
rename : taskcluster/ci/source-check/doc.yml => taskcluster/ci/source-test/doc.yml
rename : taskcluster/ci/source-check/kind.yml => taskcluster/ci/source-test/kind.yml
rename : taskcluster/ci/source-check/mozlint.yml => taskcluster/ci/source-test/mozlint.yml
rename : taskcluster/ci/source-check/python-tests.yml => taskcluster/ci/source-test/python-tests.yml
rename : taskcluster/ci/source-check/webidl.yml => taskcluster/ci/source-test/webidl.yml
extra : rebase_source : a683b5b6e243849de57f2681993046c776d6a8f2
Tom Prince
Andrew Halberstadt
Gregory Szorc
Dustin J. Mitchell