f3e11b4255
Use pip-compile instead of constraints.txt ( #736 )
...
* Fix docstyle check
* Use pip-compile instead of constraints.txt
2020-02-12 20:26:24 +01:00
cf511d8cc2
Add template for clients_scalar_aggregates ( #727 )
...
* Add initial moustache files for scalar_aggregates
* Add Jinja2 dependency
* Update templates with more parameters
* Add format clauses and add query to be formatted
* Add formatted sql
* Add generated sql
python -m bigquery_etl.glam.scalar_aggregates_incremental > sql/telemetry_derived/clients_scalar_aggregates_v1/query.sql
* Generalize clients_scalar_aggregates
* Refactor into attributes and attributes_list
* Add generated sql for generalized query
* Add glam templates to format_sql SKIP
* Fix dryrun by using AS and USING properly in sql
* Add generated sql
* Add instructions on adding new Python library
* Fix linting issues
* Use r""" for backslash in docstring
* Add Jinja2 dependencies to constraints.txt
* Document process for adding new Python dependencies
2020-02-12 09:43:37 -08:00
dce350d0cf
Add UDF manual publishing to docs
2020-02-07 10:42:08 -08:00
ea2ddf7b8e
Update README with persistent UDF changes
2020-02-07 10:42:08 -08:00
d21c39725e
Document intent to have new user-facing views without version indirection
...
Closes #343
2020-01-21 16:36:17 -05:00
2f7de8683d
Enforce script/format_sql for all new sql files ( #656 )
2020-01-09 13:55:46 -08:00
8ca73c2b60
Rewrite script/format_sql in python ( #640 )
2020-01-06 16:17:41 -08:00
6d5ce0d980
docs: fix typo
2019-10-18 19:12:32 -04:00
8c3d73647e
Add link to query scheduling cookbook ( #424 )
2019-10-15 16:01:01 -04:00
0d5eab8fe4
Fix documentation for script/format-sql ( #389 )
2019-09-27 17:33:05 -07:00
469c03ec10
Add script to format sql ( #173 )
2019-09-18 17:48:53 -07:00
d1a6b0739a
Implement udf_sample_id in SQL ( #356 )
2019-09-18 09:36:12 -07:00
525b9f05af
Add initial legacy Presto/Athena compatibility udfs ( #316 )
...
* Add initial legacy Presto/Athena compatibility udfs
* fixup! Add initial legacy Presto/Athena compatibility udfs
2019-08-28 17:16:59 -05:00
99fe0dfd9e
Move queries into destination-table directories ( #286 )
...
* Move queries into destination-table directories
* Apply suggestions from code review
Co-Authored-By: Jeff Klukas <jeff@klukas.net>
2019-08-26 12:52:49 -07:00
01cb6e1074
Refactor naming of UDFs
2019-07-24 09:01:13 -04:00
ee96864f78
Refactor JavaScript UDFs to a udf_js folder
...
This allows us to easily deploy these as persistent UDFs in the udf_js dataset
now that we have the tooling from #242 .
2019-07-24 09:01:13 -04:00
61c012539d
Accept null submission_date to reprocess all history
...
See discussion in #211
This may not be exhaustive, but converts most of the incremental queries
where days are independent of one another.
2019-07-16 16:04:32 -04:00
06dd4a120f
Link to app name documentation
2019-06-26 10:22:04 -04:00
81ebd2fd7b
Small improvements in prep for initial GUD demo
2019-06-26 10:22:04 -04:00
e5cb4c0336
Consistently use _js suffix for JS udfs and document
2019-06-25 14:20:35 -04:00
62443b1270
Update README.md ( #172 )
2019-06-08 18:05:17 -07:00
8ce5714562
Add ci token to status badge ( #153 )
2019-05-23 10:17:31 -07:00
789cabe1b3
Use bit arrays in clients_last_seen
...
This methodology was developed in draft #80
The result here is that `clients_last_seen_v1` becomes a view that should
be compatible with existing usage (it has all the same columns as the existing
table), but also has underlying bit patterns for each usage criterion,
which can be used for calculating engagement, retention, and new profile
metrics (those will be in a later PR).
In the next day or two, there will be similar PRs for nondesktop and FxA.
2019-05-21 20:22:01 -04:00
d43741c14f
Escape all identifiers matching keywords ( #142 )
2019-05-16 10:25:25 -07:00
952d6d2e76
fix circleci build badge
2019-05-15 14:22:44 -04:00
de6eb0a914
Use earliest row with document_id by timestamp ( #117 )
...
* Use earliest instance of document_id by timestamp
* Fix mode_last for fxa amplitude export
* Update README.md
2019-05-06 13:07:54 -07:00
6a8e37ceeb
Update documentation ( #113 )
...
* Update documentation
* address review
2019-05-02 13:51:26 -07:00
d15bcadf21
Recommend against DATETIME ( #84 )
...
* Recommend against DATETIME
* Update README.md
* fix link
2019-04-16 10:39:04 -07:00
e8cc2863ea
Add backfill section to README ( #67 )
...
* Add backfill section to README
* Update README.md
Co-Authored-By: relud <dthorn@mozilla.com>
* Fix backfill copy operation information
2019-04-05 13:20:58 -07:00
51378e8049
Use days_since_* and new table name ( #41 )
2019-03-28 10:39:14 -07:00
0c7e3a9a60
fix readme link
2019-03-19 12:23:31 -04:00
124b201018
Reorganise README for slightly better readability ( #24 )
2019-03-19 08:57:15 -07:00
9bef1b7fe3
Update README.md ( #17 )
...
* Add CircleCI badge
* Fix link to tests/
2019-03-12 11:24:56 -07:00
79070068ad
Add first test ( #9 )
2019-03-07 12:43:21 -08:00
123552c320
version queries and add clients_last_seen and mau28_by_dimensions ( #1 )
2019-02-15 13:37:24 -08:00
b91dcdd15e
Initial commit
2019-02-08 11:11:59 -08:00
Daniel Thorn
Anthony Miyaguchi
Anna Scholtz
Jeff Klukas
Jason Cooke
Ben Wu
Sunah Suh
William Lachance