Set owners for Contextual Services tasks (#3048)

* Set owners for Contextual Services tasks

I made some guesses here about who would make the most sense to own individual
pieces; treat this as a starting point and we can discuss in PR comments to
get this to final state.

* Generate DAGs after rebase

* Respond to review comments

Co-authored-by: kik-kik <42538694+kik-kik@users.noreply.github.com>
Co-authored-by: whd <whd@users.noreply.github.com>
This commit is contained in:
Jeff Klukas 2022-07-21 16:18:23 -04:00 коммит произвёл GitHub
Родитель 1a23ebf480
Коммит ad1d9cb4e4
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
9 изменённых файлов: 41 добавлений и 28 удалений

Просмотреть файл

@ -553,12 +553,13 @@ bqetl_ctxsvc_derived:
default_args:
depends_on_past: false
email:
- jklukas@mozilla.com
- ctroy@mozilla.com
- wstuckey@mozilla.com
- telemetry-alerts@mozilla.com
email_on_failure: true
email_on_retry: true
end_date: null
owner: jklukas@mozilla.com
owner: ctroy@mozilla.com
retries: 2
retry_delay: 30m
start_date: '2021-05-01'
@ -571,25 +572,26 @@ bqetl_search_terms_daily:
default_args:
depends_on_past: false
email:
- jklukas@mozilla.com
- ctroy@mozilla.com
- wstuckey@mozilla.com
- rburwei@mozilla.com
- telemetry-alerts@mozilla.com
email_on_failure: true
email_on_retry: true
end_date: null
owner: jklukas@mozilla.com
owner: ctroy@mozilla.com
retries: 2
retry_delay: 30m
start_date: '2021-09-20'
description: |
Derived tables on top of search terms data.
Note that the task for populating `suggest_impression_sanitized_v2` is
Note that the tasks for populating `suggest_impression_sanitized_v*` are
particularly important because the source unsanitized dataset has only
a 2-day retention period, so errors fairly quickly become unrecoverable
and can impact reporting to partners. If this task errors out, it could
indicate trouble with an upstream task that runs in a restricted project
outside of Airflow. Contact `jklukas` and/or `jbuck`.
outside of Airflow. Contact `ctroy`, `wstuckey`, `whd`, and `jbuck`.
schedule_interval: 0 3 * * *
tags:
- impact/tier_1

Просмотреть файл

@ -18,15 +18,19 @@ Built from bigquery-etl repo, [`dags/bqetl_ctxsvc_derived.py`](https://github.co
Contextual services derived tables
#### Owner
jklukas@mozilla.com
ctroy@mozilla.com
"""
default_args = {
"owner": "jklukas@mozilla.com",
"owner": "ctroy@mozilla.com",
"start_date": datetime.datetime(2021, 5, 1, 0, 0),
"end_date": None,
"email": ["jklukas@mozilla.com", "telemetry-alerts@mozilla.com"],
"email": [
"ctroy@mozilla.com",
"wstuckey@mozilla.com",
"telemetry-alerts@mozilla.com",
],
"depends_on_past": False,
"retry_delay": datetime.timedelta(seconds=1800),
"email_on_failure": True,
@ -49,8 +53,13 @@ with DAG(
destination_table="event_aggregates_v1",
dataset_id="contextual_services_derived",
project_id="moz-fx-data-shared-prod",
owner="jklukas@mozilla.com",
email=["jklukas@mozilla.com", "telemetry-alerts@mozilla.com"],
owner="rburwei@mozilla.com",
email=[
"ctroy@mozilla.com",
"rburwei@mozilla.com",
"telemetry-alerts@mozilla.com",
"wstuckey@mozilla.com",
],
date_partition_parameter="submission_date",
depends_on_past=False,
arguments=["--schema_update_option=ALLOW_FIELD_ADDITION"],

Просмотреть файл

@ -17,25 +17,26 @@ Built from bigquery-etl repo, [`dags/bqetl_search_terms_daily.py`](https://githu
Derived tables on top of search terms data.
Note that the task for populating `suggest_impression_sanitized_v2` is
Note that the tasks for populating `suggest_impression_sanitized_v*` are
particularly important because the source unsanitized dataset has only
a 2-day retention period, so errors fairly quickly become unrecoverable
and can impact reporting to partners. If this task errors out, it could
indicate trouble with an upstream task that runs in a restricted project
outside of Airflow. Contact `jklukas` and/or `jbuck`.
outside of Airflow. Contact `ctroy`, `wstuckey`, `whd`, and `jbuck`.
#### Owner
jklukas@mozilla.com
ctroy@mozilla.com
"""
default_args = {
"owner": "jklukas@mozilla.com",
"owner": "ctroy@mozilla.com",
"start_date": datetime.datetime(2021, 9, 20, 0, 0),
"end_date": None,
"email": [
"jklukas@mozilla.com",
"ctroy@mozilla.com",
"wstuckey@mozilla.com",
"rburwei@mozilla.com",
"telemetry-alerts@mozilla.com",
],
@ -61,11 +62,12 @@ with DAG(
destination_table="adm_daily_aggregates_v1",
dataset_id="search_terms_derived",
project_id="moz-fx-data-shared-prod",
owner="jklukas@mozilla.com",
owner="ctroy@mozilla.com",
email=[
"jklukas@mozilla.com",
"ctroy@mozilla.com",
"rburwei@mozilla.com",
"telemetry-alerts@mozilla.com",
"wstuckey@mozilla.com",
],
date_partition_parameter="submission_date",
depends_on_past=False,
@ -79,9 +81,10 @@ with DAG(
project_id="moz-fx-data-shared-prod",
owner="rburwei@mozilla.com",
email=[
"jklukas@mozilla.com",
"ctroy@mozilla.com",
"rburwei@mozilla.com",
"telemetry-alerts@mozilla.com",
"wstuckey@mozilla.com",
],
date_partition_parameter="submission_date",
depends_on_past=False,
@ -93,11 +96,12 @@ with DAG(
destination_table="suggest_impression_sanitized_v2",
dataset_id="search_terms_derived",
project_id="moz-fx-data-shared-prod",
owner="jklukas@mozilla.com",
owner="ctroy@mozilla.com",
email=[
"jklukas@mozilla.com",
"ctroy@mozilla.com",
"rburwei@mozilla.com",
"telemetry-alerts@mozilla.com",
"wstuckey@mozilla.com",
],
date_partition_parameter="submission_date",
depends_on_past=False,
@ -112,9 +116,9 @@ with DAG(
owner="ctroy@mozilla.com",
email=[
"ctroy@mozilla.com",
"jklukas@mozilla.com",
"rburwei@mozilla.com",
"telemetry-alerts@mozilla.com",
"wstuckey@mozilla.com",
],
date_partition_parameter="submission_date",
depends_on_past=False,

Просмотреть файл

@ -2,7 +2,7 @@ friendly_name: Contextual Services Event Aggregates
description: |-
Aggregated event and user counts for topsites and quicksuggest
owners:
- jklukas@mozilla.com
- rburwei@mozilla.com
workgroup_access:
- role: roles/bigquery.dataViewer
members:

Просмотреть файл

@ -2,7 +2,7 @@ friendly_name: Contextual Services Event Aggregates
description: |-
Aggregated event and user counts for topsites and quicksuggest
owners:
- jklukas@mozilla.com
- rburwei@mozilla.com
labels:
incremental: true
schedule: daily

Просмотреть файл

@ -5,7 +5,6 @@ description: |-
Contextual Services Suggest Impression Pings
owners:
- rburwei@mozilla.com
- jklukas@mozilla.com
workgroup_access:
- role: roles/bigquery.dataViewer
members:

Просмотреть файл

@ -7,7 +7,7 @@ description: |-
The sending to adMarketplace takes place in a separate DAG defined in
the telemetry-airflow repository directly.
owners:
- jklukas@mozilla.com
- ctroy@mozilla.com
workgroup_access:
- role: roles/bigquery.dataViewer
members:

Просмотреть файл

@ -8,7 +8,6 @@ workgroup_access:
- workgroup:search-terms/aggregated
owners:
- rburwei@mozilla.com
- jklukas@mozilla.com
bigquery:
time_partitioning:
field: submission_date

Просмотреть файл

@ -9,7 +9,7 @@ description: |-
Fulfills https://mozilla-hub.atlassian.net/browse/ROAD-85
owners:
- jklukas@mozilla.com
- ctroy@mozilla.com
labels:
incremental: true
scheduling: