Add retention on KPI table and some other tables

This commit is contained in:
alekhyamoz 2025-01-29 15:56:37 -05:00
Родитель d9bbcbf34d
Коммит 95cee01df8
14 изменённых файлов: 29 добавлений и 15 удалений

Просмотреть файл

@ -6,11 +6,12 @@ description: |
Exposed to users as view `fenix.clients_yearly` and used
as the basis for LTV calculations.
owners:
- frank@mozilla.com
- kwindau@mozilla.com
labels:
schedule: daily
dag: bqetl_org_mozilla_firefox_derived
owner1: frank
owner1: kwindau
table_type: aggregate
scheduling:
dag_name: bqetl_org_mozilla_firefox_derived
depends_on_past: true
@ -19,7 +20,7 @@ bigquery:
type: day
field: submission_date
require_partition_filter: true
expiration_days: null
expiration_days: 775
clustering:
fields:
- sample_id

Просмотреть файл

@ -16,6 +16,7 @@ bigquery:
type: day
field: submission_date
require_partition_filter: false
expiration_days: 775
clustering:
fields:
- sample_id

Просмотреть файл

@ -16,6 +16,7 @@ bigquery:
type: day
field: submission_date
require_partition_filter: false
expiration_days: 775
clustering:
fields:
- sample_id

Просмотреть файл

@ -11,7 +11,7 @@ bigquery:
type: day
field: submission_date
require_partition_filter: true
expiration_days: null
expiration_days: 775
clustering:
fields: [sample_id, country, adjust_network]
scheduling:

Просмотреть файл

@ -6,12 +6,15 @@ owners:
labels:
incremental: true
owner1: kwindau@mozilla.com
table_type: client_level
scheduling:
dag_name: bqetl_firefox_desktop_ad_click_history
depends_on_past: true
date_partition_parameter: null
parameters: ["submission_date:DATE:{{ds}}"]
bigquery:
time_partitioning:
expiration_days: 775
clustering:
fields:
- client_id

Просмотреть файл

@ -2,10 +2,11 @@ friendly_name: Onboarding
description: |-
Contains CFR specific data extracted from messaging_system_v1
owners:
- gleonard@mozilla.com
- kwindau@mozilla.com
labels:
incremental: true
owner1: gleonard
owner1: kwindau
table_type: client_level
scheduling:
dag_name: bqetl_messaging_system
bigquery:
@ -13,7 +14,7 @@ bigquery:
type: day
field: submission_timestamp
require_partition_filter: true
expiration_days: null
expiration_days: 775
clustering:
fields:
- normalized_channel

Просмотреть файл

@ -2,10 +2,10 @@ friendly_name: Snippets
description: |-
Contains Snippets specific data extracted from messaging_system_v1
owners:
- gleonard@mozilla.com
- kwindau@mozilla.com
labels:
incremental: true
owner1: gleonard
owner1: kwindau
scheduling:
dag_name: bqetl_messaging_system
bigquery:
@ -13,7 +13,7 @@ bigquery:
type: day
field: submission_timestamp
require_partition_filter: true
expiration_days: null
expiration_days: 775
clustering:
fields:
- normalized_channel

Просмотреть файл

@ -12,15 +12,18 @@ description: |-
are available as a MAP, keyed by date,
where the value is the number of ad clicks.
owners:
- frank@mozilla.com
- kwindau@mozilla.com
labels:
incremental: true
owner1: frank@mozilla.com
owner1: kwindau
table_type: client_level
scheduling:
dag_name: bqetl_firefox_ios
date_partition_parameter: null
parameters: ["submission_date:DATE:{{ds}}"]
bigquery:
time_partitioning:
expiration_days: 775
clustering:
fields: [sample_id]
references: {}

Просмотреть файл

@ -17,7 +17,7 @@ bigquery:
type: day
field: submission_date
require_partition_filter: false
expiration_days: null
expiration_days: 775
clustering:
fields:
- sample_id

Просмотреть файл

@ -16,6 +16,7 @@ bigquery:
type: day
field: submission_date
require_partition_filter: false
expiration_days: 775
clustering:
fields:
- sample_id

Просмотреть файл

@ -16,6 +16,7 @@ bigquery:
type: day
field: submission_date
require_partition_filter: false
expiration_days: 775
clustering:
fields:
- sample_id

Просмотреть файл

@ -11,7 +11,7 @@ bigquery:
type: day
field: submission_date
require_partition_filter: true
expiration_days: null
expiration_days: 775
clustering:
fields: [sample_id, country]
scheduling:

Просмотреть файл

@ -13,7 +13,7 @@ bigquery:
type: day
field: date
require_partition_filter: false
expiration_days: null
expiration_days: 775
clustering:
fields:
- sample_id

Просмотреть файл

@ -13,6 +13,7 @@ owners:
- mhirose@mozilla.com
labels:
incremental: true
table_type: client_level
scheduling:
dag_name: bqetl_unified
date_partition_parameter: cohort_date
@ -21,6 +22,7 @@ bigquery:
type: day
field: cohort_date
require_partition_filter: true
expiration_days: 775
clustering:
fields:
- attribution_medium