Set owners for Contextual Services tasks (#3048)
* Set owners for Contextual Services tasks I made some guesses here about who would make the most sense to own individual pieces; treat this as a starting point and we can discuss in PR comments to get this to final state. * Generate DAGs after rebase * Respond to review comments Co-authored-by: kik-kik <42538694+kik-kik@users.noreply.github.com> Co-authored-by: whd <whd@users.noreply.github.com>
This commit is contained in:
Родитель
1a23ebf480
Коммит
ad1d9cb4e4
14
dags.yaml
14
dags.yaml
|
@ -553,12 +553,13 @@ bqetl_ctxsvc_derived:
|
|||
default_args:
|
||||
depends_on_past: false
|
||||
email:
|
||||
- jklukas@mozilla.com
|
||||
- ctroy@mozilla.com
|
||||
- wstuckey@mozilla.com
|
||||
- telemetry-alerts@mozilla.com
|
||||
email_on_failure: true
|
||||
email_on_retry: true
|
||||
end_date: null
|
||||
owner: jklukas@mozilla.com
|
||||
owner: ctroy@mozilla.com
|
||||
retries: 2
|
||||
retry_delay: 30m
|
||||
start_date: '2021-05-01'
|
||||
|
@ -571,25 +572,26 @@ bqetl_search_terms_daily:
|
|||
default_args:
|
||||
depends_on_past: false
|
||||
email:
|
||||
- jklukas@mozilla.com
|
||||
- ctroy@mozilla.com
|
||||
- wstuckey@mozilla.com
|
||||
- rburwei@mozilla.com
|
||||
- telemetry-alerts@mozilla.com
|
||||
email_on_failure: true
|
||||
email_on_retry: true
|
||||
end_date: null
|
||||
owner: jklukas@mozilla.com
|
||||
owner: ctroy@mozilla.com
|
||||
retries: 2
|
||||
retry_delay: 30m
|
||||
start_date: '2021-09-20'
|
||||
description: |
|
||||
Derived tables on top of search terms data.
|
||||
|
||||
Note that the task for populating `suggest_impression_sanitized_v2` is
|
||||
Note that the tasks for populating `suggest_impression_sanitized_v*` are
|
||||
particularly important because the source unsanitized dataset has only
|
||||
a 2-day retention period, so errors fairly quickly become unrecoverable
|
||||
and can impact reporting to partners. If this task errors out, it could
|
||||
indicate trouble with an upstream task that runs in a restricted project
|
||||
outside of Airflow. Contact `jklukas` and/or `jbuck`.
|
||||
outside of Airflow. Contact `ctroy`, `wstuckey`, `whd`, and `jbuck`.
|
||||
schedule_interval: 0 3 * * *
|
||||
tags:
|
||||
- impact/tier_1
|
||||
|
|
|
@ -18,15 +18,19 @@ Built from bigquery-etl repo, [`dags/bqetl_ctxsvc_derived.py`](https://github.co
|
|||
Contextual services derived tables
|
||||
#### Owner
|
||||
|
||||
jklukas@mozilla.com
|
||||
ctroy@mozilla.com
|
||||
"""
|
||||
|
||||
|
||||
default_args = {
|
||||
"owner": "jklukas@mozilla.com",
|
||||
"owner": "ctroy@mozilla.com",
|
||||
"start_date": datetime.datetime(2021, 5, 1, 0, 0),
|
||||
"end_date": None,
|
||||
"email": ["jklukas@mozilla.com", "telemetry-alerts@mozilla.com"],
|
||||
"email": [
|
||||
"ctroy@mozilla.com",
|
||||
"wstuckey@mozilla.com",
|
||||
"telemetry-alerts@mozilla.com",
|
||||
],
|
||||
"depends_on_past": False,
|
||||
"retry_delay": datetime.timedelta(seconds=1800),
|
||||
"email_on_failure": True,
|
||||
|
@ -49,8 +53,13 @@ with DAG(
|
|||
destination_table="event_aggregates_v1",
|
||||
dataset_id="contextual_services_derived",
|
||||
project_id="moz-fx-data-shared-prod",
|
||||
owner="jklukas@mozilla.com",
|
||||
email=["jklukas@mozilla.com", "telemetry-alerts@mozilla.com"],
|
||||
owner="rburwei@mozilla.com",
|
||||
email=[
|
||||
"ctroy@mozilla.com",
|
||||
"rburwei@mozilla.com",
|
||||
"telemetry-alerts@mozilla.com",
|
||||
"wstuckey@mozilla.com",
|
||||
],
|
||||
date_partition_parameter="submission_date",
|
||||
depends_on_past=False,
|
||||
arguments=["--schema_update_option=ALLOW_FIELD_ADDITION"],
|
||||
|
|
|
@ -17,25 +17,26 @@ Built from bigquery-etl repo, [`dags/bqetl_search_terms_daily.py`](https://githu
|
|||
|
||||
Derived tables on top of search terms data.
|
||||
|
||||
Note that the task for populating `suggest_impression_sanitized_v2` is
|
||||
Note that the tasks for populating `suggest_impression_sanitized_v*` are
|
||||
particularly important because the source unsanitized dataset has only
|
||||
a 2-day retention period, so errors fairly quickly become unrecoverable
|
||||
and can impact reporting to partners. If this task errors out, it could
|
||||
indicate trouble with an upstream task that runs in a restricted project
|
||||
outside of Airflow. Contact `jklukas` and/or `jbuck`.
|
||||
outside of Airflow. Contact `ctroy`, `wstuckey`, `whd`, and `jbuck`.
|
||||
|
||||
#### Owner
|
||||
|
||||
jklukas@mozilla.com
|
||||
ctroy@mozilla.com
|
||||
"""
|
||||
|
||||
|
||||
default_args = {
|
||||
"owner": "jklukas@mozilla.com",
|
||||
"owner": "ctroy@mozilla.com",
|
||||
"start_date": datetime.datetime(2021, 9, 20, 0, 0),
|
||||
"end_date": None,
|
||||
"email": [
|
||||
"jklukas@mozilla.com",
|
||||
"ctroy@mozilla.com",
|
||||
"wstuckey@mozilla.com",
|
||||
"rburwei@mozilla.com",
|
||||
"telemetry-alerts@mozilla.com",
|
||||
],
|
||||
|
@ -61,11 +62,12 @@ with DAG(
|
|||
destination_table="adm_daily_aggregates_v1",
|
||||
dataset_id="search_terms_derived",
|
||||
project_id="moz-fx-data-shared-prod",
|
||||
owner="jklukas@mozilla.com",
|
||||
owner="ctroy@mozilla.com",
|
||||
email=[
|
||||
"jklukas@mozilla.com",
|
||||
"ctroy@mozilla.com",
|
||||
"rburwei@mozilla.com",
|
||||
"telemetry-alerts@mozilla.com",
|
||||
"wstuckey@mozilla.com",
|
||||
],
|
||||
date_partition_parameter="submission_date",
|
||||
depends_on_past=False,
|
||||
|
@ -79,9 +81,10 @@ with DAG(
|
|||
project_id="moz-fx-data-shared-prod",
|
||||
owner="rburwei@mozilla.com",
|
||||
email=[
|
||||
"jklukas@mozilla.com",
|
||||
"ctroy@mozilla.com",
|
||||
"rburwei@mozilla.com",
|
||||
"telemetry-alerts@mozilla.com",
|
||||
"wstuckey@mozilla.com",
|
||||
],
|
||||
date_partition_parameter="submission_date",
|
||||
depends_on_past=False,
|
||||
|
@ -93,11 +96,12 @@ with DAG(
|
|||
destination_table="suggest_impression_sanitized_v2",
|
||||
dataset_id="search_terms_derived",
|
||||
project_id="moz-fx-data-shared-prod",
|
||||
owner="jklukas@mozilla.com",
|
||||
owner="ctroy@mozilla.com",
|
||||
email=[
|
||||
"jklukas@mozilla.com",
|
||||
"ctroy@mozilla.com",
|
||||
"rburwei@mozilla.com",
|
||||
"telemetry-alerts@mozilla.com",
|
||||
"wstuckey@mozilla.com",
|
||||
],
|
||||
date_partition_parameter="submission_date",
|
||||
depends_on_past=False,
|
||||
|
@ -112,9 +116,9 @@ with DAG(
|
|||
owner="ctroy@mozilla.com",
|
||||
email=[
|
||||
"ctroy@mozilla.com",
|
||||
"jklukas@mozilla.com",
|
||||
"rburwei@mozilla.com",
|
||||
"telemetry-alerts@mozilla.com",
|
||||
"wstuckey@mozilla.com",
|
||||
],
|
||||
date_partition_parameter="submission_date",
|
||||
depends_on_past=False,
|
||||
|
|
|
@ -2,7 +2,7 @@ friendly_name: Contextual Services Event Aggregates
|
|||
description: |-
|
||||
Aggregated event and user counts for topsites and quicksuggest
|
||||
owners:
|
||||
- jklukas@mozilla.com
|
||||
- rburwei@mozilla.com
|
||||
workgroup_access:
|
||||
- role: roles/bigquery.dataViewer
|
||||
members:
|
||||
|
|
|
@ -2,7 +2,7 @@ friendly_name: Contextual Services Event Aggregates
|
|||
description: |-
|
||||
Aggregated event and user counts for topsites and quicksuggest
|
||||
owners:
|
||||
- jklukas@mozilla.com
|
||||
- rburwei@mozilla.com
|
||||
labels:
|
||||
incremental: true
|
||||
schedule: daily
|
||||
|
|
|
@ -5,7 +5,6 @@ description: |-
|
|||
Contextual Services Suggest Impression Pings
|
||||
owners:
|
||||
- rburwei@mozilla.com
|
||||
- jklukas@mozilla.com
|
||||
workgroup_access:
|
||||
- role: roles/bigquery.dataViewer
|
||||
members:
|
||||
|
|
|
@ -7,7 +7,7 @@ description: |-
|
|||
The sending to adMarketplace takes place in a separate DAG defined in
|
||||
the telemetry-airflow repository directly.
|
||||
owners:
|
||||
- jklukas@mozilla.com
|
||||
- ctroy@mozilla.com
|
||||
workgroup_access:
|
||||
- role: roles/bigquery.dataViewer
|
||||
members:
|
||||
|
|
|
@ -8,7 +8,6 @@ workgroup_access:
|
|||
- workgroup:search-terms/aggregated
|
||||
owners:
|
||||
- rburwei@mozilla.com
|
||||
- jklukas@mozilla.com
|
||||
bigquery:
|
||||
time_partitioning:
|
||||
field: submission_date
|
||||
|
|
|
@ -9,7 +9,7 @@ description: |-
|
|||
|
||||
Fulfills https://mozilla-hub.atlassian.net/browse/ROAD-85
|
||||
owners:
|
||||
- jklukas@mozilla.com
|
||||
- ctroy@mozilla.com
|
||||
labels:
|
||||
incremental: true
|
||||
scheduling:
|
||||
|
|
Загрузка…
Ссылка в новой задаче