While we don't have physical machines running taskcluster-worker, we
want to schedule talos tests on Taskcluster and run them on buildbot.
buildbot-bridge acts as a worker that runs its tasks in buildbot
machines. Any talos task is redirected to buildbot-bridge.
MozReview-Commit-ID: Im9IhhT0Mr1
--HG--
extra : rebase_source : ede38a683abdf411612e885f6985c79b0532cb46
Stylo automation doesn't work unless Servo is present in the source
directory. This commit introduces a "check_servo" filter that prunes
tasks requiring Servo. Currently, this is implemented as a test against
platforms that are unique to Servo.
The use of relative path checking to find the topsrcdir is a bit
unfortunate. But we use this pattern elsewhere in this code.
MozReview-Commit-ID: IRtd53tudJW
--HG--
extra : rebase_source : 8c4742c13878d762fe7970eedfa5937fdaebe8c4
Previously, we ran a single "target task" function to mutate the full
task graph into a subset based on input parameters (try syntax,
repository being built for, etc). This concept is useful. But
the implementation was limiting because we could only have a single
"target tasks" function.
This commit introduces the concept of "filters." They conceptually
do the same thing as "target tasks methods" but you can run more than
1 of them.
Filters are simply functions that examine an input graph+parameters
and emit nodes that should be retained. Filters, like target tasks
methods, are defined via decorated functions in a module.
TaskGraphGenerator has been converted to use filters. The list of
defined filters can be defined in the parameters dict passed into
TaskGraphGenerator. A default filter list is provided in decision.py.
The intent is to eventually convert target tasks to filters. Until
that happens, we always run the registered target tasks method via
a filter proxy function.
No new tests have been added because we don't yet have any
functionality relying explicitly on filters. Tests will be added in
a subsequent commit once we add a new filter.
While I was here, I also snuck in some logging on the size of the
graphs.
MozReview-Commit-ID: ERn2hIYbMRp
--HG--
extra : rebase_source : 36b8e86aa64b2f52b03b31b5497759b0009fb921
Previously, all callers outside of tests that passed
"target_tasks_method" to TaskGraphGenerator all used the same pattern
of looking for a key in the parameters and calling a function in
the target_tasks module.
Future commits will refactor how target tasks graph work. To
make the transition easier, we move the logic for obtaining the
target tasks method into TaskGraphGenerator.
MozReview-Commit-ID: 3QU09iGhoXh
--HG--
extra : rebase_source : fbcc31d705c4b0e148aa3709ddcb18ad99953231
This commit introduces support for performing build and test variations
with Stylo enabled.
From a high level, this is similar to the e10s/non-e10s variations that
we currently run. However, it is different in that Stylo is a build
time (not run time) flag. So, we need to perform separate builds for
Stylo.
In the current implementation of the task graph, we can either
introduce a new platform or a new build type/configuration. I've
chosen to introduce a new platform if nothing else so the Treeherder
results clearly separate Stylo from non-Stylo. This decision can be
revisited in the future once Stylo becomes more stable and visible
outside the "stylo" repository.
Stylo functionality is limited to layout functionality and Stylo
developers aren't very interested in many tests not related to layout
at the moment. So we introduce a new test set running a limited
functionality of tests in Stylo builds.
In the current state of this commit, Stylo builds will permafail.
This is because Stylo builds require a vendored copy of Servo. Logic
for pruning the task graph of Stylo/Servo related tasks will be
implemented in subsequent commits.
MozReview-Commit-ID: JivhkpXDjIw
--HG--
extra : rebase_source : fdc9a5300c0bfa27133478fc456bc618f26fb9b2
Job tasks are tasks that are not tied to a specific build or test. As such, they cannot be scheduled
with the regular -p/-u try syntax options. There exists a -j try syntax option, to schedule them, which
defaults to running "all" of them if not specified.
However, there is a bug here where they will only default to "all" if a try syntax exists in the commit
message. They will not be considered if a developer pushes to try without a try syntax. This happens
because self.jobs is initially initialized with '[]' and we use None to determine when to schedule "all"
later on.
I want to move towards a world without try syntax, so we should start improving the UX of the no try
syntax use case.
Note: When I say "schedule" here, I mean added to the target set. They may still be optimized away.
MozReview-Commit-ID: 4TrC84RGiaL
--HG--
extra : rebase_source : 8fd50113c37745bc10181e1311cc62d75485723a
We use a Maven repository and the (misleadingly named!) uploadArchives
task because this is the best way to make Android Studio download and
recognize the Javadoc and sources. With this, it's automatic; with a
single AAR file, it's a nightmare of point-and-click configuration.
This patch does a bunch of Gradle hacking to make -javadoc and
-sources JARs; there's nothing special or particularly likely to break
here.
This patch also adds Proguard declarations to the :geckoview library
project. That involves moving a good part of the Proguard
configuration into mobile/android/geckoview. (I also expand upon the
existing configuration.) This should be only a re-arrangement, and
the resulting file is included in the original, so nothing should be
changed.
MozReview-Commit-ID: BGNW1v92J0k
--HG--
extra : rebase_source : 94633d27e8ae6bafa3d6823996355c22d2e2e6eb
* Compress docker images with zstd
* Removed need for context.tar from decision task
* Index images by level rather than project
MozReview-Commit-ID: 4RL4QXNWmpd
--HG--
extra : rebase_source : 677d8030a15af3288866a70fc648a10b22c396a3
We add support for Mac OSX tests in Taskcluster by support
taskcluster-worker macosx engine. We also add a new configuration
blob-upload-branch needed by mochitest-other.
MozReview-Commit-ID: 5ZaYYRueRVr
--HG--
extra : rebase_source : f81b3d97d44797a323431a640b1f8e29bf192606
Often we need to setup a test configuration for different platforms, but
not for different platform builds (opt/debug). This leads to cumbersome
configuration duplicates. This patch adds support for platform regular
expression matching. For example, if you want to configure the chunk
size for linux64 platform both for opt and debug, originally you would
do this:
chunks:
by-test-platform:
linux64/opt: 4
linux64/debug: 4
default: 8
With regular expression matching, you only need:
chunks:
by-test-platform:
linux64/.*: 4
default: 8
This patch was originally written by Geoffrey Brown for Windows support.
MozReview-Commit-ID: KbMHV7UkTLe
--HG--
extra : rebase_source : 79a4344c7e3e978bb6b93713c6e6e4114ba5d5b8
The `from_parameters` method was never used, and let do confusion over the role
of these parameters. Now there are only two, and they are always required.
MozReview-Commit-ID: AbPqijXucu5
--HG--
extra : rebase_source : 85affd063a543c549afaaa36ce7ee31ed1f943d5
We add support for Mac OSX tests in Taskcluster by support
taskcluster-worker macosx engine. We also add a new configuration
blob-upload-branch needed by mochitest-other.
MozReview-Commit-ID: CxT3BP3sydG
--HG--
extra : rebase_source : 25e16bf5b5d5a8121fd431788185d318612d7ea5
Often we need to setup a test configuration for different platforms, but
not for different platform builds (opt/debug). This leads to cumbersome
configuration duplicates. This patch adds support for platform regular
expression matching. For example, if you want to configure the chunk
size for linux64 platform both for opt and debug, originally you would
do this:
chunks:
by-test-platform:
linux64/opt: 4
linux64/debug: 4
default: 8
With regular expression matching, you only need:
chunks:
by-test-platform:
linux64/.*: 4
default: 8
This patch was originally written by Geoffrey Brown for Windows support.
MozReview-Commit-ID: HFP52N9Ef0k
--HG--
extra : rebase_source : d2a5129b7459fc7f71f59da76d45526cef028e44
Hammad Akhtar
Gregory Szorc
Ted Mielczarek
Geoff Brown
elluls
Carsten "Tomcat" Book
Wander Lairson Costa
Julian Seward
Phil Ringnalda
Dustin J. Mitchell
Mike Shal
Brian Stack
Ting-Yu Chou
Jonas Finnemann Jensen
Andrew Halberstadt
Rob Thijssen
Sebastian Hengst
Kartikaya Gupta
Nick Alexander
Henrik Skupin
Wes Kocher
Kim Moir
Joel Maher