65402717f2
* Create iOS_onboarding.toml Added iOS_onboarding.toml - query for generating the underlying table in the iOS onboarding dashboard. * Update iOS_onboarding.toml * feat: remove locale from dimension list, it does not exist in the source table * Delete sql_generators/funnels/configs/iOS_onboarding.toml * update ios onboarding funnel * break ios onboarding funnel into two smaller funnels * break ios onboarding funnel into two smaller funnels (#6537) * Temporarily add active_users v3 to codeowners until v4 backfill is completed. (#6534) * chore(DO-1886): Add shredder mitigation to more tables (#6532) * Add shredder mitigation to desktop eng & ret tables * Add shredder mitigation to desktop new profile agg v1 * Add fenix_derived.funnel_retention_week_4_v1 to shredder mitigation * Add urlbar_events_daily_eng_by_pos_v1 to shredder mitigation * Add firefox_ios_derived.funnel_retention_week_4_v1 shredder mitigation * Add desktop funnel installs v1 to shredder mitigation * Add firefox_nondesktop_exact_mau28_v1 * Add shredder mitigation to urlbar_events_daily_v1 * chore(DO-1886): Add shredder mitigation to more tables (#6535) * Add shredder mitigation to searchreport v1 * Add mobile search agg for search report v1 to shredder mitigation * Add smoot usage fxa compressed v2 to shredder mitigation * Add smoot usage new profiles comp v2 to shredder mitigation * change types ofr created_at and job_created_at from STRING to DATE (#6536) --------- Co-authored-by: Lucia <30448600+lucia-vargas-a@users.noreply.github.com> Co-authored-by: Katie Windau <153020235+kwindau@users.noreply.github.com> Co-authored-by: Marlene Hirose <92952117+Marlene-M-Hirose@users.noreply.github.com> * Use events_stream table --------- Co-authored-by: kik-kik <kignasiak@mozilla.com> Co-authored-by: Lucia <30448600+lucia-vargas-a@users.noreply.github.com> Co-authored-by: Katie Windau <153020235+kwindau@users.noreply.github.com> Co-authored-by: Marlene Hirose <92952117+Marlene-M-Hirose@users.noreply.github.com> Co-authored-by: Anna Scholtz <anna@scholtzan.net> |
||
|---|---|---|
| .. | ||
| active_users_aggregates_v3 | ||
| active_users_aggregates_v4 | ||
| active_users_deletion_requests | ||
| country_code_lookup | ||
| derived_view_schemas | ||
| desktop_crashes | ||
| events_daily | ||
| experiment_monitoring | ||
| feature_usage | ||
| funnels | ||
| glean_usage | ||
| mobile_kpi_support_metrics | ||
| search_v2 | ||
| serp_events_v2 | ||
| stable_views | ||
| table_partition_expirations | ||
| urlbar_events | ||
| use_counters | ||
| README.md | ||
| __init__.py | ||
README.md
SQL generators
sql_generators/ contains scripts for generating SQL queries. Generated SQL query code should not be checked in to main. The scripts for generating SQL queries are executed by CI only and will are followed by generating Airflow DAGs.
Expected structure
The directories in sql_generators/ represent the generated queries and will contain all of the scripts and templates necessary to generate these queries. Each query-specific directory will contain a __init__.py file that contains the query generation logic. Optionally, a templates/ directory can be added which contains the Jinja templates queries are generated from.
Each __init__.py file needs to implement a generate() method that is configured as a click command. The bqetl CLI will automatically add these commands to the ./bqetl query generate command group.
After changes to a schema or adding new tables, the schema is automatically derived from the query and deployed the next day in DAG bqetl_artifact_deployment. Alternatively, it can be manually generated and deployed using ./bqetl generate all and ./bqetl query schema deploy.