bigquery-etl/sql
Alessio Placitelli 76bac7a98e
Create an ETL job for the Internet Outages (#1058)
* Add aggregation by country

* Copy the initial Italy focus query

This initial commit provides a baseline for the
next commits to ease review, since this initial
code was already reviewed.

* Cleanup the country list and replace FULL OUTER with LEFT joins

* Aggregate by city for cities with more than 15k inhabitants

The actual 15k limit is enforced at ingestion time.
This further limits the resulting cities to ones with at
least 1000 active daily users.

* Produce hourly aggregates

* Move the query to the `internet_outage` dataset

* Provide automatic daily scheduling through AirFlow

* Tweak the SQL addressing review comments

This additionally changes the `CAST` to
`SAFE_CAST` to account for weirdnesses in
the data.

* Add ssl_error_prop

* Add missing_dns_success

* Add missing_dns_failure

* Lower the minimum reported bucket size to 50

This allows us to match the EDA by Saptarshi and
to have a better comparable baseline.

* Document the oddities around `submission_timestamp_min`
2020-07-01 06:44:40 +02:00
..
activity_stream Bug 1616895 Move impression_stats_flat into bi dataset 2020-03-12 14:23:56 -04:00
activity_stream_bi/impression_stats_flat_v1 Update impression_stats_flat description 2020-06-12 08:32:42 -07:00
amo_dev Remove deprecated amo_stats_dau_v1 queries 2020-06-12 13:22:47 -04:00
amo_prod DAGs for client queries 2020-06-24 12:18:04 -07:00
default_browser_agent/default_browser Add AS 2020-06-04 16:28:49 -04:00
firefox_accounts Better regex for Amplitude email export (#1100) 2020-06-26 15:28:28 -04:00
firefox_accounts_derived Typo in fxa_log_content query 2020-06-13 08:57:34 -04:00
glam_etl [glam-etl] Use transpose logic for fenix extracts (#1011) 2020-05-28 11:57:55 -07:00
internet_outages/global_outages_v1 Create an ETL job for the Internet Outages (#1058) 2020-07-01 06:44:40 +02:00
messaging_system Fix onboarding events user properties (#1099) 2020-06-25 17:09:56 -04:00
messaging_system_derived DAG for messaging system queries 2020-06-10 13:45:22 -07:00
monitoring Use stable tables rather than live 2020-03-16 08:39:04 -04:00
org_mozilla_fenix_nightly Update fenix_nightly generated queries 2020-04-08 11:11:46 -04:00
org_mozilla_fenix_nightly_derived Update fenix_nightly generated queries 2020-04-08 11:11:46 -04:00
org_mozilla_firefox Fenix baseline_daily and clients_last_seen tables 2020-04-22 13:30:00 -04:00
org_mozilla_firefox_derived Fix SQL format validation step in CI 2020-05-26 15:06:01 -07:00
org_mozilla_vrbrowser/clients_last_seen Undo formatting for old SQL files 2020-02-07 09:48:23 -08:00
org_mozilla_vrbrowser_derived depends_on_past=true for vrbrowser clients_last_seen 2020-06-05 10:52:12 -07:00
pocket/pocket_reach_mau Fix dateset reference for pocket_reach_mau view (#906) 2020-04-14 11:09:54 -07:00
revenue_derived/client_ltv_v1 Add submission date to ltv revenue join (#1103) 2020-06-30 09:18:58 -04:00
search Remove search aggregate union with v6 (#1079) 2020-06-17 10:38:06 -04:00
search_derived Version search contribution and add task (#1091) 2020-06-24 13:00:44 -04:00
shredder_state/progress Add a view that shows shredder progress (#1047) 2020-06-15 10:19:06 -07:00
static Add static normal_distribution table 2020-02-20 15:20:07 -05:00
telemetry Fix view regressions (#1097) 2020-06-25 13:01:59 -04:00
telemetry_derived Rename bqetl_clients to bqetl_clients_daily 2020-06-24 12:18:04 -07:00