Before, test.sh (duplicated between the desktop-test and
desktop1604-test images) was dropping permissions, creating a workspace,
and executing test-linux.sh. This is functionality now provided by
run-task.
So, convert the test tasks to use run-task.
It's worth noting that creating the workspace directory is no longer
necessary because it is defined as a VOLUME in the Dockerfile or a
cache in some task configurations. However, it is default owned by
root:root, so we do need to instruct run-task to chown it.
Since the test.sh files are no longer used after this change, they have
been deleted. The desktop-test image no longer has any files in the
bin/ directory, so the Dockerfile entry to copy those files has been
removed.
MozReview-Commit-ID: 1BiskrMs6xW
--HG--
extra : rebase_source : f4b8f320fafa2da1aa4b06c9cbef76c69acdae8f
extra : intermediate-source : 905fb4a53386c76d931e4a04a31d03739a00a1ff
extra : source : 8335aa40265b1d17421d06d9e9a180eb8419fe47
It is common for tasks to chown paths before permissions are dropped
from root:root. This commonly occurs when Docker volumes/caches are
involved since they are default owned by root:root and not writable
by any other user.
Since this will likely be a common request, add the functionality to
run-task.
MozReview-Commit-ID: AHmSfY5Ce0S
--HG--
extra : rebase_source : 705891869413495b015b490ee75efd40b26d24d8
extra : source : 213a4986ccfd71f04f822e56b1a9ad9505ed5e98
Set up tooltool to actually use its cache, and remove the unused tc-vcs cache
scope
MozReview-Commit-ID: H1VLxryQPEM
--HG--
extra : rebase_source : b52de7f5a10e5fb4576872bf3169b2faf87ded32
Secrets are accessed via http://taskcluster/secrets, and the hazard scripts
show no evidence of such calls.
MozReview-Commit-ID: 6xp6D2ybwap
--HG--
extra : rebase_source : 5bfe02773f26768c7e79621a43515d6bf8b921ca
This leaves a lot of room for improvement, but gets the job done for now.
MozReview-Commit-ID: HmeTxZjOfOb
--HG--
extra : rebase_source : b6f55134c25cb670409e906da1acc7c0bd8d0dc0
This has some notes for future work on the task, but will work for now.
MozReview-Commit-ID: 7J4tQeKj3KJ
--HG--
extra : rebase_source : d171b5393fa340a47927e8b89ece93392ff23e56
This uses the run_on_projects attribute introduced earlier for most branches,
adjusts the `ash` method to handle that branch as the legacy implementation
did, and updates try syntax to match builds as well as tests.
In the process, this enables optimizing target tasks, meaning that tasks
specifically requested in the try syntax might be optimized. While this is
probably not ideal, it matches the existing behavior of try (where `-j all` is
the default but all jobs are set to run only when certain files have been
modified). This change can be reverted later, in a more advanced version of
try.
MozReview-Commit-ID: 5FYeUTAsafr
--HG--
extra : rebase_source : b358e0e7cd8a401c50009e63dd55c59489c9b75b
The task description now includes
* flexible specification of index routes (this will get simpler once buildbot
and gecko.v1 routes are removed)
* "run-on-projects", indicating the projects on which this task should run
* "{level}" is allowed in workerTypes
* For the docker-worker/docker-engine worker implementations, "docker-image"
can have the form {in-tree: in-tree-name} to use an in-tree image. This was
previously implemented in the test transforms, but it is useful for other
tasks too!
* Optimizations, currently limited to "only-if-files-changed", can be specified
for each task.
* TreeHerder groupSymbol is optional
* expires-after and and deadline-after have default values (with the former
differing for try and non-try)
* coalesce-name triggers creation of both a coalesce route and a superseder URL
MozReview-Commit-ID: 70vtYs5lz5P
--HG--
extra : rebase_source : 9c557d68239f42466d9724d48ed5bf77648f9aa0
Rename to taskgraph.transforms.task.
This also adds some Required and Optional declarations to the schema to be explicit,
and adjusts the transform to handle treeherder being optional.
MozReview-Commit-ID: FuKYayvlwB9
--HG--
rename : taskcluster/taskgraph/transforms/make_task.py => taskcluster/taskgraph/transforms/task.py
extra : rebase_source : 0913aa8cdf153cd086a7786de957535e9b3a4ee8
This is cosmetic, and makes the cache names easier to generate automatically.
MozReview-Commit-ID: SPwDDnUhx9
--HG--
extra : rebase_source : 2666c8e9073f9598db4146773059065b0d9ba76e
The automatically-generated gecko.v2 routes will remain until a later commit in
this patchset.
MozReview-Commit-ID: A194OvRVWJi
--HG--
extra : rebase_source : f93569efdd6c4978cc42d27d4bbe4f56129ac440
For the most part, this means including the full build_platform (which is not
always the same as build_name) and build_type in the cache name. This makes
the names consistent so they can be generated automatically rather than
specified explicitly.
MozReview-Commit-ID: 7VWm48F1Fzy
--HG--
extra : rebase_source : 3a346b7fd3ed24df75b21a9f1db7e75f8ddfcbbf
MikeLing initially did this in bug 1287018. The intent of this conditional was
to make optimization faster by not even checking most tasks, based on the
assumption that if the prerequisite to a task has changed (for example, a
docker image or a build), then naturally we will want to execute that task.
However, as we have developed actual optimization methods, this has proven not
to be the case: we might want to optimize a test out if its inputs have not
changed, even if a new installer has been built. Similarly, SETA may optimize
tasks out even if their inputs have changed.
MozReview-Commit-ID: LgHET3Z84GB
--HG--
extra : rebase_source : efd297d37bd49dbe655266380641abc258dda725
Version control tends to leave kind directories around containing .orig files
or other such detritus. It's best to just ignore such directories, rather than
failing to generate the taskgraph.
MozReview-Commit-ID: B9GIZndbfi1
--HG--
extra : rebase_source : 58f1debe5219fc52ead2971718da932b63bb06d7
David Anderson
Gregory Szorc
Olli Pettay
Dustin J. Mitchell