340259e54c
Migrate mode UDFs
2020-07-15 11:24:17 -07:00
4ddb5d4b58
Fix metadata migration
2020-07-15 11:24:17 -07:00
312e0ed21a
Refactor migrate_to_mozfun script
2020-07-15 11:24:17 -07:00
e24c7bdf41
Add script for migrating UDFs to mozfun
2020-07-15 11:24:17 -07:00
2156023e16
Add new search types, experiments, and uri count to mobile search ( #1167 )
2020-07-14 19:00:37 -04:00
f267e38f14
Fix asn_aggregates and fxa_events
2020-07-14 09:54:04 -07:00
83f3b1c2bd
Reformating
2020-07-14 09:07:18 -07:00
33aa4ed63d
Improve determining of experiment analysis tables for shredder
...
Co-authored-by: Daniel Thorn <dthorn@mozilla.com>
2020-07-14 09:07:18 -07:00
40f2918402
Remove experiment_analysis_target
...
Co-authored-by: Daniel Thorn <dthorn@mozilla.com>
2020-07-14 09:07:18 -07:00
227a471d6a
Use client_id_target for experiment analysis tables
...
Co-authored-by: Daniel Thorn <dthorn@mozilla.com>
2020-07-14 09:07:18 -07:00
c755e4da1d
Re-use thread pool for experiment analysis targets
...
Co-authored-by: Daniel Thorn <dthorn@mozilla.com>
2020-07-14 09:07:18 -07:00
ee381a96ed
Add experiment analysis tables to shredder
2020-07-14 09:07:18 -07:00
f1e0634f86
Fix regressions in legacy parquet compatibility views
...
:dmu reported a query using one of these views that was not working due to
a schema incompatibility. Looks like we made a mistake in pointing the view
directly at a table rather than a view that unions with more recent data.
2020-07-14 08:31:08 -04:00
b0efb898c5
Add bqetl_fxa_events DAG ( #1163 )
...
* Add bqetl_fxa_events DAG
Replaces the fxa_events DAG in telemetry-airflow
Co-authored-by: Anna Scholtz <anna@scholtzan.net>
2020-07-14 07:58:11 -04:00
d5bbfe297c
Fix main_summary copy_dedupe dependency
2020-07-13 11:33:23 -07:00
502965f282
Bug 1649074 Extract a table for FxA device-command events ( #1150 )
...
* Bug 1649074 Extract a table for FxA device-command events
Based on example query from :rfkelly in
https://bugzilla.mozilla.org/show_bug.cgi?id=1649074#c1
* Address review comments
* Exempt query from dry run due to permissions on hmac key
* Add metadata.yaml
2020-07-13 14:20:43 -04:00
289b473922
Use referenced_table option for asn_aggregates
2020-07-13 11:15:30 -07:00
2fa01c6ddb
bqetl_asn_aggregates
2020-07-13 11:15:30 -07:00
2d8e08294b
Add query for monitoring deletion request volume ( #1106 )
2020-07-13 11:11:06 -07:00
9a9397bfce
Update last-seen views to use mozfun udfs ( #1154 )
2020-07-13 10:32:37 -07:00
2500b1b3e3
Fix public data DAG
2020-07-13 09:24:25 -07:00
3cefa96e15
Add more data to `amo_dev.amo_stats_dau`
2020-07-13 11:11:40 -04:00
6487847292
Reference default_args for email when generating dag tasks ( #1159 )
2020-07-10 16:17:07 -07:00
89d297f69a
Add tasks to dags all at once instead of one at a time ( #1158 )
2020-07-10 15:46:26 -07:00
6de612ba67
Optionally provide referenced_tables in metadata.yaml ( #1156 )
...
because queries that reference main_v4 are slow to dry run
2020-07-10 15:38:54 -07:00
41b05f58d3
Rescheduling
2020-07-10 13:30:24 -07:00
8d773e5fff
Update tests
2020-07-10 13:30:24 -07:00
ee4a3ee0ce
More task re-scheduling
2020-07-10 13:30:24 -07:00
eacd688f38
Rename week_start column to submission_date in user_activity query ( #1149 )
...
This makes this query consistent with others.
2020-07-10 13:46:20 +02:00
4c3345581f
Add FXA support to shredder ( #1140 )
2020-07-09 15:14:04 -07:00
cb68926361
Fix 'Namespace' object has no attribute 'table' in shredder_amplitude ( #1152 )
2020-07-09 15:01:45 -07:00
67d93b1677
Parallelize Airflow DAG generation
2020-07-09 14:03:52 -07:00
eef4adc913
Remove sensor pool for now
2020-07-09 13:08:17 -07:00
4d43055bfb
Update DAG tests
2020-07-09 13:08:17 -07:00
8e27b71369
Set priority for clients_daily tasks
2020-07-09 13:08:17 -07:00
beb83e49b5
Set sensor pool
2020-07-09 13:08:17 -07:00
c4054ccdbf
Reschedule and set main_summary priority
2020-07-09 13:08:17 -07:00
64f2bd7c56
Use mozfun.hist.extract in generated desktop GLAM queries
...
This will make GLAM queries tolerate the new compact histogram encodings
2020-07-09 12:59:01 -04:00
bbb75da158
Bug 1639196 - add user_activity query for Public Data Report ( #1076 )
2020-07-09 13:50:30 +02:00
efd3395874
Search for UDFs in mozfun directory by default
2020-07-08 14:53:44 -07:00
7dec33911c
Fix getting mozfun UDF dependencies
2020-07-08 14:53:44 -07:00
fdfcfe8dc6
Prevent running pytest on UDF example files
2020-07-08 14:53:44 -07:00
5112d9dca0
Run pytest on mozfun UDFs
2020-07-08 14:53:44 -07:00
e13ee14db7
Set build_date to null instead of defaulting to glob ( #1145 )
2020-07-08 14:07:03 -07:00
9928da015c
Update execution delta
2020-07-08 13:46:03 -07:00
804ea6919a
Fix schedule interval delta
2020-07-08 13:46:03 -07:00
ba77360f8f
Update integration tests
2020-07-08 13:46:03 -07:00
c2d1430ede
Reschedule DAGs
2020-07-08 13:46:03 -07:00
75ad93c9cd
Calculate execution delta for tasks
2020-07-08 13:46:03 -07:00
a65d6788a4
Don't delete from org in script/shredder_amplitude ( #1142 )
2020-07-08 12:03:24 -07:00
Anna Scholtz
Ben Wu
Jeff Klukas
Jeff Klukas
Daniel Thorn
William Durand
Arkadiusz Komarzewski
Anthony Miyaguchi