f0c3ad8fda
Fix writing to temporary table when publishing JSON ( #1273 )
...
* Fix writing to temporary table when publishing JSON
* Update test_script_incremental_query test
* Ensure temporary artifacts are deleted
* Wrap deletion of artifacts into try finally block
2020-09-01 08:51:15 -07:00
3da15a3001
Prepare search_clients_last_seen for 100% backfill ( #1261 )
...
* Move null search engines to Other
* Remove sample_id limit
2020-09-01 09:46:35 -04:00
5ed271b8b4
Bump pytest-black from 0.3.8 to 0.3.10 ( #1265 )
2020-08-31 21:10:47 +00:00
34e2c21405
Add CLI command to initialize destination tables for queries
2020-08-31 12:47:58 -07:00
4cd100e064
Bump pyyaml from 5.3 to 5.3.1 ( #1263 )
...
Bumps [pyyaml](https://github.com/yaml/pyyaml ) from 5.3 to 5.3.1.
- [Release notes](https://github.com/yaml/pyyaml/releases )
- [Changelog](https://github.com/yaml/pyyaml/blob/master/CHANGES )
- [Commits](https://github.com/yaml/pyyaml/compare/5.3...5.3.1 )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-08-31 12:22:56 -07:00
dd29a862c2
Migrate to new dependabot ( #1262 )
2020-08-31 12:00:48 -07:00
3725a64e31
Check error output for entrypoint tests
2020-08-31 09:25:45 -07:00
99b11cbf93
Install bq for integration tests in CircleCI
2020-08-31 09:25:45 -07:00
6970b73a21
Add integration tests for entrypoint run query
2020-08-31 09:25:45 -07:00
debd57c662
Fix entrypoint run_query call
2020-08-27 21:08:14 -07:00
ffaaa2ab26
Call bigquery_etl.run_query from script/run_query
2020-08-27 14:48:32 -07:00
bebe352335
Add tests for run_query
2020-08-27 14:48:32 -07:00
e2007fe9fc
Add CLI tests
2020-08-27 14:48:32 -07:00
043c28dd4a
Add cost and last_updated information to query info CLI
2020-08-27 14:48:32 -07:00
9b587bf374
Add backfill command
2020-08-27 14:48:32 -07:00
04cbf80eab
Add format command to CLI
2020-08-27 14:48:32 -07:00
1ffbd574ea
CLI backfill command
2020-08-27 14:48:32 -07:00
256b3cb648
CLI query info
2020-08-27 14:48:32 -07:00
5f9f4e9027
CLI dryrun command
2020-08-27 14:48:32 -07:00
d8f782dc62
Add scripts for backfilling and exporting all fenix aggregates ( #1255 )
...
* Add scripts for backfilling and exporting all fenix aggregates
* Update script/glam/export_glean_all_fenix
Co-authored-by: Ben Wu <benjaminwu124@gmail.com>
Co-authored-by: Ben Wu <benjaminwu124@gmail.com>
2020-08-26 11:08:32 -07:00
d518b35575
Fix table name extraction and set write disposition for bigquery_etl_scheduled_queries_cost_v1
2020-08-26 10:20:07 -07:00
bd73aebb6f
Add submission_date to bigquery_etl_scheduled_queries_cost_v1 table
2020-08-26 10:20:07 -07:00
f3b0ab9350
Add dataset for monitoring bigquery-etl query cost
2020-08-26 10:20:07 -07:00
6f30c7feef
Add sample_id to desktop retention view ( #1252 )
...
Co-authored-by: Jeff Klukas <jklukas@mozilla.com>
2020-08-26 09:53:25 -04:00
9eeca4ad5c
Add Fenix, Fx Lite to Day 2-7 Activation ( #1254 )
...
* Add Fenix, Fx Lite to Day 2-7 Activation
Fenix and Firefox Lite were previously filtered from the Day 2-7
Activation metric pending validation in https://jira.mozilla.com/browse/DS-696
* Remove excessive comments
2020-08-25 17:03:49 -04:00
7a069cf408
Allow baseline pings with 'foreground' reason in KPIs ( #1253 )
...
After merging, I will need to backfill these tables back to early April.
DS asked for this change in https://jira.mozilla.com/browse/DS-1018
2020-08-25 10:01:31 -04:00
a5c5cb84ae
CLI secho fix
...
Co-authored-by: Anthony Miyaguchi <amiyaguchi@mozilla.com>
2020-08-24 12:19:18 -07:00
48164652af
CLI allow info for specific DAG
2020-08-24 12:19:18 -07:00
8998d88d50
Add tests for CLI utils
2020-08-24 12:19:18 -07:00
c0fc5b8db2
Add tests for CLI dag command
2020-08-24 12:19:18 -07:00
a81c4b06a3
Refactor CLI utility functions
2020-08-24 12:19:18 -07:00
01585616fa
CLI command for removing DAGs
2020-08-24 12:19:18 -07:00
0289e120e1
CLI command for generating DAGs
2020-08-24 12:19:18 -07:00
6aa1f7aa8d
Add CLI command for creating new DAGs
2020-08-24 12:19:18 -07:00
00d31b73a7
List DAGs
2020-08-24 12:19:18 -07:00
8c1c699771
Fix CLI tests
2020-08-21 11:10:58 -07:00
1d5483afb5
Update DAG generation tests
2020-08-21 11:10:58 -07:00
e1dab7de0f
Add tests for CLI query schedule
2020-08-21 11:10:58 -07:00
cbf560a1fa
Add CLI tests for creating queries
2020-08-21 11:10:58 -07:00
e9d1f16e2c
CLI query schedule allow re-generation
2020-08-21 11:10:58 -07:00
78f3e8f012
CLI query schedule re-generate DAG
2020-08-21 11:10:58 -07:00
e49618fdf5
Add query schedule command to CLI
2020-08-21 11:10:58 -07:00
146a5aebe3
Bug 1660066 - Create view for onboarding user retention analysis in Amplitude ( #1241 )
...
Co-authored-by: Frank Bertsch <fbertsch@mozilla.com>
2020-08-21 13:02:38 -04:00
3715ba87cd
Revert "Use mozfun.norm.fenix_app_info for nondesktop_clients_last_seen ( #1245 )" ( #1249 )
...
This reverts commit 4fe432337e
2020-08-21 10:56:50 -04:00
4fe432337e
Use mozfun.norm.fenix_app_info for nondesktop_clients_last_seen ( #1245 )
...
* Use mozfun.norm.fenix_app_info for nondesktop_clients_last_seen
After this is merged, we can backfill of smoot tables to have GUD
reflect the current and historical app naming schemes.
2020-08-21 09:29:03 -04:00
aa3f827d6f
More complete docs for fenix_app_info ( #1244 )
...
Fixes #1243
2020-08-20 15:42:47 -04:00
6a2d48f037
fenix_app_info documentation fixups ( #1237 )
2020-08-19 08:55:37 -04:00
96b85854d2
Update Glam ETL for Fenix ( #1240 )
...
* Use UNION ALL instead of UNION
* Move tests into separate directory and add test for all fenix products
* Replace channel with *
2020-08-18 16:15:56 -07:00
e72b2777d2
Move experiments_search_aggregates logic to a view ( #1239 )
2020-08-18 15:17:22 -07:00
654401a36d
Fix intermittent verify-dags-up-to-date failures
2020-08-18 15:07:35 -07:00
Anna Scholtz
Frank Bertsch
dependabot[bot]
Daniel Thorn
Anthony Miyaguchi
Ryan Harter
Jeff Klukas
Nan Jiang