d73b21ff6e
Update DAGs
2020-08-14 11:08:15 -07:00
b1a9b4a781
Add dry run tests
2020-08-14 11:08:15 -07:00
0a5adf91ed
Refactor Airflow DAG generation to use dry run
2020-08-14 11:08:15 -07:00
e5c8946171
Dry run get referenced tables
2020-08-14 11:08:15 -07:00
53c856b781
Fix GCS integration test issues
2020-08-13 12:17:23 -07:00
1af07bdb1e
Use qualified table names and udf functions ( #1227 )
...
This makes backfilling inside a sandbox project possible
2020-08-12 16:13:22 -04:00
4bb8eff997
Add distinct docid count queries ( #1225 )
2020-08-11 18:38:42 -04:00
1c71e70838
Bug 1654330 AMO download stats by UTM params ( #1224 )
2020-08-11 13:32:10 -04:00
285efeb991
Add burnham test report table ( #1220 )
...
* Add burnham test report table
For https://github.com/mozilla/burnham/issues/39
* Add test_log_url and test_duration_millis
* Apply suggestions from code review
Co-authored-by: Raphael Pierzina <raphael@hackebrot.de>
Co-authored-by: Raphael Pierzina <raphael@hackebrot.de>
2020-08-10 09:50:27 -04:00
3c6d63739c
Fix formatting.
2020-08-10 09:39:50 -04:00
7298d43c32
Add missing array length call.
2020-08-10 09:39:50 -04:00
b57f24ff1b
Add addon and cert error features
2020-08-10 09:39:50 -04:00
6a83b6223e
Fix mozfun format in generated docs ( #1212 )
2020-08-07 16:35:44 -04:00
5702d96dae
Bug 1646362 - Accessibility inspector derived dataset ( #1171 )
...
Co-authored-by: Jeff Klukas <jklukas@mozilla.com>
2020-08-07 16:27:06 -04:00
4633b5fda4
Provide extra context when dag generation fails due to a malformed query ( #1222 )
2020-08-07 14:40:14 -04:00
5bab4e9d26
Bug 1655146 Add impression_stats_by_experiment
...
This table will be a bit larger than impression_stats_flat, about 1 TB/day
uncompressed. But it will be much more efficient for the class of queries
that are concerned with evaluating performance of a single experiment.
2020-08-07 12:54:10 -04:00
36b7c184e6
Add script to backfill glam tables for a glean product ( #1108 )
...
* Add backfill script for glean products
* Specify product correctly and add target dataset
* Add product to example
* Use datetime.fromisoformat
2020-08-06 15:48:40 -07:00
a58821eaae
Add build date udf mapping for fenix_nightly ( #1218 )
2020-08-06 15:32:45 -04:00
ac182ca56b
Add parsed_x_source_tags to metadata in user-facing views
...
As discussed in https://bugzilla.mozilla.org/show_bug.cgi?id=1657360#c1
2020-08-06 15:25:48 -04:00
c7eae14b25
Add submission_date to clients_scalar_aggregates ( #1216 )
2020-08-06 14:28:53 -04:00
d5d64359f6
Bug 1657360 Exclude pings with "automation" tag from stable
...
We will also need to update monitoring queries to account for this when
counting unique document_ids in decoded and live tables.
2020-08-06 12:56:15 -04:00
2acb30c9b0
Rewrite duplicated map udfs to mozfun shims ( #1211 )
...
* Rewrite duplicated map udfs to mozfun shims
* Format get_key_with_null.sql
2020-08-04 13:26:13 -07:00
8192ee323f
Fix URI comparison ( #1205 )
...
* Fix URI comparison
No need to backfill for this now. Just fixing for future use.
* Fix formatting.
Co-authored-by: Jeff Klukas <jklukas@mozilla.com>
2020-08-03 09:49:54 -04:00
0969a51673
Include sample_id in feature_usage ( #1201 )
...
Used as clustering key and allows sampling
2020-08-03 09:38:02 -04:00
cc06758a9a
Add tagged_search current to LTV ( #1210 )
2020-07-31 15:48:34 -04:00
8d6c2f47c1
Add sqrt to LTV calc ( #1202 )
...
* Add sqrt to LTV calc
* Format query
Co-authored-by: Frank Bertsch <fbertsch@mozilla.com>
2020-07-31 10:48:20 -04:00
019666b51b
Add queries for exported app store data ( #1207 )
2020-07-29 18:02:16 -04:00
a39e168a2d
Update query scheduling section
2020-07-29 14:11:00 -07:00
111382b37a
Rewrite duplicated bits28 udfs to mozfun shims ( #1208 )
2020-07-29 11:17:26 -07:00
6dedaca002
Improve query CLI based on feedback
2020-07-29 08:26:24 -07:00
8171e30010
Refactor CLI
2020-07-29 08:26:24 -07:00
bb47afedd5
Docs for query CLI command
2020-07-29 08:26:24 -07:00
fee4f44e9a
Fix CLI version
2020-07-29 08:26:24 -07:00
3a85538e7f
Add CLI command for creating queries
2020-07-29 08:26:24 -07:00
2df0b038ff
Add CLI versioning
2020-07-29 08:26:24 -07:00
ea95b0a1c4
CLI setup
2020-07-29 08:26:24 -07:00
306c667b2d
Add unit tests and documentations for udfs ( #1197 )
...
* Add unit tests and documentations for udfs
* Auto format SQL files
* fix and format
Co-authored-by: Frank Bertsch <fbertsch@mozilla.com>
2020-07-28 11:54:44 -07:00
2306a92242
Add clarifications for lower-friction scheduled queries
2020-07-24 14:43:53 -07:00
e7e5e774c8
Lower-friction scheduled queries wording and link fixes
2020-07-24 14:43:53 -07:00
1efc742cfb
Refactor lower-friction schedule query docs
2020-07-24 14:43:53 -07:00
a58d1c61d8
Add some lower-friction scheduled query docs
2020-07-24 14:43:53 -07:00
4a9223a742
Bug 1648241 - Create mobile search clients last seen ( #1196 )
2020-07-24 11:31:52 -04:00
a813031c52
Initial feature_usage table. ( #1193 )
...
Co-authored-by: Daniel Thorn <dthorn@mozilla.com>
Co-authored-by: Jeff Klukas <jklukas@mozilla.com>
2020-07-24 10:49:12 -04:00
8c6818327a
Reduce main_summary complexity in popup_notification_stats
2020-07-24 09:40:59 -04:00
c2f2887478
Scale up Threadpool
2020-07-23 12:54:35 -07:00
56f79c08e3
Parallelize stable table sizes monitoring script
2020-07-23 12:54:35 -07:00
1ea0bf9b64
Script for monitoring partition sizes of stable tables
2020-07-23 12:54:35 -07:00
36048c950d
Update integration tests
2020-07-23 07:54:10 -07:00
35df38308a
ExternalTaskSensor pool
2020-07-23 07:54:10 -07:00
350339b60d
Reduce main_summary complexity by bundling threshold counts ( #1194 )
...
It appears that #1191 which changes mozfun.hist.extract caused complexity
to tip over the threshold for main_summary again.
The `threshold_count` UDF uses mozfun.hist.extract under the hood and also
itself involves a subquery, so bundling multiple thresholds together in
a single function call reduces complexity.
2020-07-22 11:56:47 -07:00
Anna Scholtz
William Lachance
Ben Wu
Jeff Klukas
Jesse McCrosky
asiOvOtus
Jeff Klukas
Anthony Miyaguchi
jmccrosky
Ben Miroglio