This adds a new morph template for modifying a task's env and the corresponding glue
to specify it from |mach try fuzzy|. It can be used like:
./mach try fuzzy --env FOO=1 --env BAR=baz
This will simply set those environment variables in *all* tasks. We could add the
ability to only specify it for a subset of tasks in the future, but that seems like
a hard problem that probably isn't worth it.
MozReview-Commit-ID: C4sokv886PU
--HG--
extra : rebase_source : 64c9105848e6becd33fae63aeabc3423cfda3bdf
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: 554IPMRWgzK
--HG--
extra : rebase_source : 00827d3f6bd705419bc801eb05b543af1ddc274f
Previously, we uploaded the main Firefox tree docs to /.
In reality, there are multiple Sphinx projects in the repo. In
addition, it is sometimes desirable to access docs for an older
version of Firefox.
In this commit, we add support for specifying the S3 key prefix
for uploads. Then we change the upload code to upload to multiple
locations:
* <project>/latest (always)
* <project>/<version> (if a version is defined in the Sphinx config)
* / (for the main Sphinx docs project)
For the Firefox docs, ``version`` corresponds to a sanitized value from
``milestone.txt``. Currently, it resolves to ``57.0``.
While we're here, we add support for declaring an alternate project
name in the Sphinx conf.py file. If ``moz_project_name`` is defined,
we use that as the project name. For Firefox, we set it to ``main``.
This means our paths (local and uploaded) are now ``main`` instead of
``Mozilla_Source_Tree_Docs``. That's much more pleasant.
MozReview-Commit-ID: 8Gl6l2m6uU4
--HG--
extra : rebase_source : e56885092c12eb8cc76e5e7300f938be566e3e5a
extra : intermediate-source : 8509af1e135177a93460270b27f263c10a62d996
extra : source : 71b4f32caf209fe9dffc340c0b8ccb51ac79c7de
We now have an --upload flag to control whether upload is performed.
We don't inline it because we want to maintain a "firewall" between
regular docs and all the extra packages and imports needed for S3.
MozReview-Commit-ID: DVKhsS545gp
--HG--
extra : rebase_source : 7fee832145189be882db0f0ca057eda6158e0492
By using mozpack, we get deterministic archives. This also makes the
task command simpler.
MozReview-Commit-ID: EPI7tuGQuso
--HG--
extra : rebase_source : babcbf3d5607480321106b40eaa4598c0faee188
We want to verify that Sphinx docs work on all repos. It is only
uploading that should be limited.
MozReview-Commit-ID: An6EZ7jpaWh
--HG--
extra : rebase_source : 86400968630b3ab1c09c43d5fa42d456cec159c4
This fixes the issue where "-u web-platform-tests-wdspec" scheduled
nothing. Now it will schedule a e10s-enabled run of wdspec tests.
MozReview-Commit-ID: AdHZqtk1hLy
--HG--
extra : rebase_source : 8d5926277b37952616c4dffdb20133868913bab2
The updated Docker image contains robustcheckout and run-task support
for sparse checkouts, which are obvious prerequisites.
We change the cache name so sparse and non-sparse checkouts don't
use the same working directory. If we didn't do this, tasks running
from images with old Mercurial clients or without a sparse aware
robustcheckout would fail.
The effect of using a sparse checkout is that we reduce the number
of files in the checkout from ~234,000 to ~3,600. This reduces time
for a fresh checkout from several dozen seconds to under 2s.
MozReview-Commit-ID: IJz794g8ZKH
--HG--
extra : source : 9923fffd4f64a1aa9d762e6027e0e2424a19c49c
The updated Docker image contains robustcheckout and run-task support
for sparse checkouts, which are obvious prerequisites.
We change the cache name so sparse and non-sparse checkouts don't
use the same working directory. If we didn't do this, tasks running
from images with old Mercurial clients or without a sparse aware
robustcheckout would fail.
The effect of using a sparse checkout is that we reduce the number
of files in the checkout from ~234,000 to ~3,600. This reduces time
for a fresh checkout from several dozen seconds to under 2s.
MozReview-Commit-ID: IJz794g8ZKH
--HG--
extra : rebase_source : d262c8314381a136cf5cdc5c33669c8c61818d1d
`run-task` is taught a --sparse-profile argument to be passed down
to `hg robustcheckout` for the main source checkout. It does what
you expect: performs a sparse checkout using the named profile.
The Taskgraph YAML for run-task is taught a "sparse-profile"
property to define the sparse profile. When defined, --sparse-profile
will be passed down to `run-task` and the cache name will be updated
to reflect the use of sparse checkout.
Our cache checking transform is updated to audit for the use of
--sparse-profile without the corresponding "-sparse" cache name
variation.
The reason we need a distinct cache name for sparse is because
clients that aren't sparse aware will be unable to read checkouts
that are sparse. By forcing sparse and non-sparse into different
cache pools, we avoid compatibility issues.
In the ideal world, we probably support sparse profiles on all the
VCS checkouts that `run-task` supports (e.g. --tools-checkout).
Perfect is the enemy of done. All of this is defined in-tree and
it is easy enough to change atomically.
MozReview-Commit-ID: 79k7Vul0hHO
--HG--
extra : rebase_source : babe9b42e2796c2341bffc6ecfe829f4daff9e0f
Include "error:" so Treeherder log parser picks things up (hopefully).
Also, actually include cache path in string.
MozReview-Commit-ID: 8b8ou7TJYZs
--HG--
extra : rebase_source : 52b43dd18419a6a8cfb096a6bf4262852c7c0ccc
Phil Ringnalda
Wes Kocher
Geoff Brown
Sebastian Hengst
Johan Lorenzo
Dustin J. Mitchell
Joel Maher
Kartikaya Gupta
J. Ryan Stinnett
Brian Stack
Mike Hommey
Xidorn Quan
Andrew Halberstadt
Chris Manchester
Rob Wood
Wander Lairson Costa
Aki Sasaki
Gregory Szorc
Maja Frydrychowicz