As |./mach| commands are migrated to python3, "taskgraph" needs to be compatible while still working with python2.
This patch migrates several iter*() calls and python2-specific imports to work with "six" instead.
Note that there's still python2-specific parts of taskgraph, I'm just modifying the pieces that are affecting
the code paths that I'm currently migrating (in this case, |./mach try|
Differential Revision: https://phabricator.services.mozilla.com/D73397
The single-locale group_by function works for tasks with a single locale each, but doesn't work when we want to maintain the same l10n chunks as our parent task. Because we want to make nightly-l10n-signing multi_dep, we'll need to group_by chunk-locales.
Differential Revision: https://phabricator.services.mozilla.com/D57730
--HG--
extra : moz-landing-system : lando
First, let's add {only,not}-for-attributes to multi_dep so we can switch single_dep kinds over more easily. Second, let's put it into its own helper function, so we don't have to copy the same set of if blocks in each grouping function.
Differential Revision: https://phabricator.services.mozilla.com/D57729
--HG--
extra : moz-landing-system : lando
The single-locale group_by function works for tasks with a single locale each, but doesn't work when we want to maintain the same l10n chunks as our parent task. Because we want to make nightly-l10n-signing multi_dep, we'll need to group_by chunk-locales.
Differential Revision: https://phabricator.services.mozilla.com/D57730
--HG--
extra : moz-landing-system : lando
First, let's add {only,not}-for-attributes to multi_dep so we can switch single_dep kinds over more easily. Second, let's put it into its own helper function, so we don't have to copy the same set of if blocks in each grouping function.
Differential Revision: https://phabricator.services.mozilla.com/D57729
--HG--
extra : moz-landing-system : lando
Makes most kinds that reference nightly attribute reference the shippable attribute.
Also makes most transforms that use nightly use shippable
Transfers dependencies/ownership for some things to shippable from nightly where it was harder to support both.
In no particular order, full list:
Send shippable attribute down to dep tasks.
Set tests as shippable attribute
Add new signing routes
Add shippable routes to repackage_routes transform
Adjust target tasks
Add shippable nightly-l10n
Add nightly-signing and as a side affect add repackage and repackage-signing
Add support for proper balrog platforms for shippable
Add shippable to the nightly sccache guard
Fix TC_PLATFORM_PER_FTP in partners.py to use shippable
Add shippable to mozharness_test variants
Only actually used for android which doesn't have shippable at this time.
Add shippable variant to beetmover transforms
Do nightly signing for mars on shippable
Support shippable in partner-repack transform
Support shippable in amo langpacks transform
Use proper signing for shippable tasks in repackage transforms
Set upload symbols to use shippable too
Use shippable as deps for geckodriver extraction
Use shippable as dep for autograph-stage signing
Do not run beetmover-l10n for shippable
Run shippable style jobs for repackage signing
Set build_platform for update verify and uvc to be shippable
Run repackage-msi for shippable
Add shippable to osx partner repack signing
add shippable to emefree repackage
add shippable to emefree repackage signing
add shippable to beetmover checksums
Add shippable to partner repack repackage signing
add partner repack beetmover
Add shippable to mar signing
Add shippable to mar-signing-l10n
add shippable to eme free beetmover checksums
Add shippable to upload-generated-sources
Add beetmover langpacks to shippable
Add repackage-l10n to shippable
Add shippable to partner repack chunk-dummy
Do eme free builds with shippable
Add signing of language packs to shippable
Add snap repackage for shippable
Add shippable for release-eme-free repack signing
Add partials for shippable
Add partner repack repackage for shippable
Add emefree beetmover for shippable
Add checksums-signing to shippable
Switch partner repacks to shippable
Add shippable to beetmover-repackage
Add secondary update verify configs for shippable
secondary update verify for shippable
Differential Revision: https://phabricator.services.mozilla.com/D24699
--HG--
extra : moz-landing-system : lando
Makes most kinds that reference nightly attribute reference the shippable attribute.
Also makes most transforms that use nightly use shippable
Transfers dependencies/ownership for some things to shippable from nightly where it was harder to support both.
In no particular order, full list:
Send shippable attribute down to dep tasks.
Set tests as shippable attribute
Add new signing routes
Add shippable routes to repackage_routes transform
Adjust target tasks
Add shippable nightly-l10n
Add nightly-signing and as a side affect add repackage and repackage-signing
Add support for proper balrog platforms for shippable
Add shippable to the nightly sccache guard
Fix TC_PLATFORM_PER_FTP in partners.py to use shippable
Add shippable to mozharness_test variants
Only actually used for android which doesn't have shippable at this time.
Add shippable variant to beetmover transforms
Do nightly signing for mars on shippable
Support shippable in partner-repack transform
Support shippable in amo langpacks transform
Use proper signing for shippable tasks in repackage transforms
Set upload symbols to use shippable too
Use shippable as deps for geckodriver extraction
Use shippable as dep for autograph-stage signing
Do not run beetmover-l10n for shippable
Run shippable style jobs for repackage signing
Set build_platform for update verify and uvc to be shippable
Run repackage-msi for shippable
Add shippable to osx partner repack signing
add shippable to emefree repackage
add shippable to emefree repackage signing
add shippable to beetmover checksums
Add shippable to partner repack repackage signing
add partner repack beetmover
Add shippable to mar signing
Add shippable to mar-signing-l10n
add shippable to eme free beetmover checksums
Add shippable to upload-generated-sources
Add beetmover langpacks to shippable
Add repackage-l10n to shippable
Add shippable to partner repack chunk-dummy
Do eme free builds with shippable
Add signing of language packs to shippable
Add snap repackage for shippable
Add shippable for release-eme-free repack signing
Add partials for shippable
Add partner repack repackage for shippable
Add emefree beetmover for shippable
Add checksums-signing to shippable
Switch partner repacks to shippable
Add shippable to beetmover-repackage
Add secondary update verify configs for shippable
secondary update verify for shippable
Differential Revision: https://phabricator.services.mozilla.com/D24699
--HG--
extra : moz-landing-system : lando
When we set the nightly attribute the tasks don't run on-push, so we use a new attribute.
Differential Revision: https://phabricator.services.mozilla.com/D22709
--HG--
extra : moz-landing-system : lando
If a task has `required_signoffs` that match a release's `required_signoffs`, we should defer running that task until we have a matching `signoff_url`.
- add filter_out_missing_signoffs
- add transforms changes to inherit upstream `required_signoffs`
- add `mar-signing` `required_signoffs` to the `partials-signing`, `mar-signing`, and `mar-signing-l10n` kinds
Differential Revision: https://phabricator.services.mozilla.com/D11734
--HG--
extra : moz-landing-system : lando
If a task has `required_signoffs` that match a release's `required_signoffs`, we should defer running that task until we have a matching `signoff_url`.
- add filter_out_missing_signoffs
- add transforms changes to inherit upstream `required_signoffs`
- add `mar-signing` `required_signoffs` to the `partials-signing`, `mar-signing`, and `mar-signing-l10n` kinds
Differential Revision: https://phabricator.services.mozilla.com/D11734
--HG--
extra : moz-landing-system : lando
To make switching between `single_dep` and `multi_dep` easier, change
the `dependent-task` to `primary-dependency`.
Differential Revision: https://phabricator.services.mozilla.com/D9722
--HG--
extra : moz-landing-system : lando
Justin Wood
Narcis Beleuzu
Mitchell Hentges
Aki Sasaki
Dorel Luca
Ricky Stewart
Razvan Maries
Johan Lorenzo
Nick Alexander
Cosmin Sabou
Brindusan Cristian
Tom Prince
Noemi Erli
Aki Sasaki
Andreea Pavel
Ryan VanderMeulen
Jim Chen