557346bd33
Github actions for pushing forks to upstream
2021-06-22 15:41:42 -07:00
06b4f39d10
[DS-1613] Remove search_clients_daily ( #2146 )
...
* [DS-1613] Remove search_clients_daily
Final step to finish https://mozilla-hub.atlassian.net/browse/DS-1613
After this is merged, I will run the following to make sure
the views no longer exist:
```
bq rm -f moz-fx-data-shared-prod:search.search_clients_daily
bq rm -f moz-fx-data-shared-prod:search.mobile_search_clients_daily
bq rm -f mozdata:search.search_clients_daily
bq rm -f mozdata:search.mobile_search_clients_daily
```
* Update references to search_clients_daily
* yamllint
2021-06-22 14:04:43 -04:00
656faac37c
Bump google-cloud-storage from 1.38.0 to 1.39.0 ( #2145 )
2021-06-22 16:18:41 +00:00
f2cce13983
Add metadata for retention UDF and fix type error
2021-06-21 19:58:39 -07:00
11a42c6030
Fix vpn retention table to correctly handle periods other than 1 month ( #2141 )
2021-06-21 16:19:21 -07:00
014012c4cb
Expose full start and end timestamp for IAP subscriptions ( #2142 )
2021-06-21 22:03:08 +00:00
cf16ae06fc
Shredder/Rally: fix table discovery for Glean studies
...
This makes Shredder properly discover derived and analysis tables from Glean studies that submit rally_id as a nested field.
2021-06-21 23:03:17 +02:00
a9cf2f250c
Shredder/Rally: fix special handling of rally_zero_one study
2021-06-21 23:03:17 +02:00
83f2bab236
DS-1650 - Update available geos ( #2140 )
2021-06-21 13:27:35 -07:00
008a34a684
Bump pytest-xdist from 2.2.1 to 2.3.0 ( #2135 )
2021-06-21 19:11:57 +00:00
ca9714031b
Bump gitpython from 3.1.17 to 3.1.18 ( #2138 )
2021-06-21 18:52:27 +00:00
6006eaa8c6
Revert "Airflow SLA in public data dag"
...
This reverts commit 6532fee02c
2021-06-17 12:27:37 -07:00
4886f8fe66
Revert "Add sla and execution_timeout parameters to Airflow tasks"
...
This reverts commit 011dbd6367
2021-06-17 12:27:37 -07:00
151f799713
Revert "Test sla and execution_timeout on public data dag"
...
This reverts commit e1f530d168
2021-06-17 12:27:37 -07:00
ce361b74e6
Add ios to experiments_daily_active_clients
2021-06-17 09:28:39 -07:00
e1f530d168
Test sla and execution_timeout on public data dag
2021-06-17 08:30:29 -07:00
011dbd6367
Add sla and execution_timeout parameters to Airflow tasks
2021-06-17 08:30:29 -07:00
6532fee02c
Airflow SLA in public data dag
2021-06-17 08:30:29 -07:00
a69af66ae4
Add experiment monitoring for ios fennec
2021-06-16 15:00:14 -07:00
a647d05d21
Add ios support to experiment aggregates queries
2021-06-16 15:00:14 -07:00
ba9a5ac2d9
Ignore null client_ids in events_daily
2021-06-16 13:43:39 -04:00
be113b5988
Add field to stripe subscription and invoice schemas ( #2131 )
2021-06-15 20:32:34 -04:00
e61d65e2de
DS-1647 - Add pricing_plan to VPN retention table ( #2130 )
2021-06-15 16:02:18 -07:00
ba335f8049
Bug 1698934 - support Rally datasets in Shredder ( #2033 )
2021-06-15 15:00:26 -04:00
2c57582354
Add report date as time partitioning field for apple reports ( #2129 )
2021-06-15 11:03:13 -07:00
65c6e8be6f
Remove incorrect arguments from apple subscriber report ( #2128 )
2021-06-15 12:45:37 -04:00
23b27075f9
Update the interval and start date for bqetl_app_store_connect ( #2123 )
2021-06-15 18:22:03 +02:00
9a763efe09
Update DAG with new dependency
2021-06-15 12:06:19 -04:00
84659af454
Use materialized events for rollups
2021-06-15 12:06:19 -04:00
df3b851d42
Generate DAG
2021-06-15 12:06:19 -04:00
a80dcb62a2
Use timestamp instead of sub_date
2021-06-15 12:06:19 -04:00
8f652758f3
Add labels dict
2021-06-15 12:06:19 -04:00
e56cdfef97
Materialize funnel events source
2021-06-15 12:06:19 -04:00
dfb2020eda
Prefix dataset with project id
2021-06-15 08:54:39 -04:00
143c27a353
Don't validate funnel view
2021-06-14 21:42:37 -04:00
7b20aeff6c
Ignore more complex queries
2021-06-14 21:42:37 -04:00
3696ba2fae
Remove space
2021-06-14 21:42:37 -04:00
42ddea0203
Dont dryrun complex query
2021-06-14 21:42:37 -04:00
d58c325436
Generate DAG
2021-06-14 21:42:37 -04:00
752b8dc106
Fix view creation SQL statement
2021-06-14 21:42:37 -04:00
d53a1e7dbd
Add events_daily view for Firefox Accounts
...
- Add funnel_events_source as base table for funnel analysis
- Don't include metadata in events_daily (clashes with other col names)
2021-06-14 21:42:37 -04:00
aa50fcd21f
Add command to import subscriber report from apple ( #2116 )
2021-06-14 18:16:38 -04:00
c2b1c895ae
Fix pricing_plan for apple subs in Mozilla VPN ( #2120 )
...
also:
- expose user_id for joining devices table in Looker
- use stricter definition of vpn subscriptions in retention
- update schemas
2021-06-14 16:11:16 -04:00
37003b09f3
Add VPN funnel query from redash ( #2119 )
2021-06-14 16:05:44 -04:00
233f0c24ab
Add dryrun options to more bqetl commands ( #2121 )
2021-06-14 12:52:21 -07:00
b273a08dd0
Use metadata to rewrite date params for dry run ( #2122 )
2021-06-14 15:45:46 -04:00
03d4b1d39f
Handle exceptions and float type when adding descriptions to stable views
2021-06-14 10:38:58 -07:00
8809da61d7
Add field descriptions to stable views
2021-06-14 10:38:58 -07:00
860b7e7367
Bug 1712828 Add urlbar_picked keyed scalars to clients_daily ( #2115 )
...
* Bug 1712828 Add urlbar_picked keyed scalars to clients_daily
Addresses https://bugzilla.mozilla.org/show_bug.cgi?id=1712828
and also adds the preferences_paneHome keyed scalar from
https://bugzilla.mozilla.org/show_bug.cgi?id=1715377
* Update schemas
* Update workflow docs
2021-06-14 09:11:41 -04:00
cc288f5dc0
Remove test_aggregation case for clients_daily_v6 ( #2117 )
...
Motivated in particular by https://github.com/mozilla/bigquery-etl/pull/2115
where new changes ended up making the test case too complex to run.
The difficulty of updating this test case is outweighing the safety benefit
at this point, so we are removing, but leaving a pointer in case we want to
reestablish the test in the future.
2021-06-11 14:02:46 -04:00
Anna Scholtz
Jeff Klukas
dependabot[bot]
Daniel Thorn
Arkadiusz Komarzewski
Frank Bertsch
akkomar