* DENG-2784 - add wait task for blogs partitions to 4 blogs tables

* DENG-2784 - add sensors for events table for production property

* DENG-2784 - add sensor for production property for the 3 tables in mozilla org derived

* DENG-2784 fix backticks

* DENG-2784 - add spaces to jinja, also update gclid conv to ga4 dag

* DENG-2784 - leave Marlene's DAG where it is

* DENG-2784 - switch to depends_on

* DENG-2784 add dag names to depends_on

* DENG-2784 - remove some of the unneeded dependencies and change name from partition to table

* DENG-2784 switch from prod to wmo to be more clear

* DENG-2784 switch from prod to wmo to be more clear
This commit is contained in:
Katie Windau 2024-02-23 13:45:28 -06:00 коммит произвёл GitHub
Родитель 5f9512fa01
Коммит a78e30c1f2
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: B5690EEEBB952194
7 изменённых файлов: 29 добавлений и 0 удалений

Просмотреть файл

@ -8,6 +8,9 @@ labels:
owner1: kwindau@mozilla.com
scheduling:
dag_name: bqetl_google_analytics_derived_ga4
depends_on:
- task_id: wait_for_blogs_events_table
dag_name: bqetl_google_analytics_derived_ga4
bigquery:
time_partitioning:
type: day

Просмотреть файл

@ -8,6 +8,9 @@ labels:
owner1: kwindau@mozilla.com
scheduling:
dag_name: bqetl_google_analytics_derived_ga4
depends_on:
- task_id: wait_for_blogs_events_table
dag_name: bqetl_google_analytics_derived_ga4
bigquery:
time_partitioning:
type: day

Просмотреть файл

@ -8,6 +8,13 @@ labels:
owner1: kwindau@mozilla.com
scheduling:
dag_name: bqetl_google_analytics_derived_ga4
depends_on_tables_existing:
- task_id: wait_for_blogs_events_table
table_id: 'moz-fx-data-marketing-prod.analytics_314399816.events_{{ ds_nodash }}'
poke_interval: 30m
timeout: 10h
retries: 1
retry_delay: 30m
bigquery:
time_partitioning:
type: day

Просмотреть файл

@ -8,6 +8,9 @@ labels:
owner1: kwindau@mozilla.com
scheduling:
dag_name: bqetl_google_analytics_derived_ga4
depends_on:
- task_id: wait_for_wmo_events_table
dag_name: bqetl_google_analytics_derived_ga4
bigquery:
time_partitioning:
type: day

Просмотреть файл

@ -8,6 +8,9 @@ labels:
owner1: kwindau@mozilla.com
scheduling:
dag_name: bqetl_google_analytics_derived_ga4
depends_on:
- task_id: wait_for_wmo_events_table
dag_name: bqetl_google_analytics_derived_ga4
bigquery:
time_partitioning:
type: day

Просмотреть файл

@ -8,6 +8,13 @@ labels:
owner1: kwindau@mozilla.com
scheduling:
dag_name: bqetl_google_analytics_derived_ga4
depends_on_tables_existing:
- task_id: wait_for_wmo_events_table
table_id: 'moz-fx-data-marketing-prod.analytics_313696158.events_{{ ds_nodash }}'
poke_interval: 30m
timeout: 10h
retries: 1
retry_delay: 30m
bigquery:
time_partitioning:
type: day

Просмотреть файл

@ -10,6 +10,9 @@ scheduling:
dag_name: bqetl_google_analytics_derived_ga4
date_partition_parameter: null
parameters: ["submission_date:DATE:{{ds}}"]
depends_on:
- task_id: wait_for_wmo_events_table
dag_name: bqetl_google_analytics_derived_ga4
bigquery:
time_partitioning:
type: day