Граф коммитов

17 Коммитов

Автор SHA1 Сообщение Дата
Anna Scholtz 3637559a1c
Add bigquery-etl-integration-test to testpath (#5976) 2024-07-30 10:16:18 -07:00
Daniel Thorn a0d810275b
Remove java dependency in favor of sqlglot (#3755) 2023-05-17 14:56:42 -07:00
Daniel Thorn f0ec77e27d
Use zetasql to find sql dependencies (#1802) 2021-02-17 11:48:40 -08:00
Anthony Miyaguchi 4234c40040
Add minimal set of tests for GLAM Fenix queries (#1488)
* Add script to determine query dependencies

* Add schemas and folders for minimal test

* Add schema for geckoview_versions

* Add query params to each query

* Update schema for new queries

* Remove main from bootstrap file

* Add dataset prefix to schemas

* Add failing test for clients_histogram_aggregates

It turns out that the dependency resolution I'm using for autogenerate
the schemas is ignoring the views. I actually want to keep the views
around. The tables also all need to be prefixed with the dataset name or
they won't be inserted into the sql query correctly.

* Add successful test for clients histogram aggregates

* Add minimal tests for clients_scalar_aggregates

* Remove skeleton files for views (no test support for views)

* Add tests for latest versions

* Add tests for scalar bucket counts that passes

* Add scalar bucket counts

* Add test for scalar percentiles

* Add test for histogram bucket counts

* Add passing test for probe counts

* Add test for histogram percentiles

* Add tests for extract counts

* Update readme

* Add data for scalar percentiles test

* Fix linting errors

* Fix mypy issues with tests module

* Name it data instead of tests.*.data

* Ignore mypy on tests directory

* Remove mypy section

* Remove extra line in pytest

* Try pytest invocation of mypy-scripts-are-modules

* Run mypy outside of pytest

* Use exec on pytest instead of mypy

* Update tests/sql/glam-fenix-dev/glam_etl/bootstrap.py

Co-authored-by: Ben Wu <benjaminwu124@gmail.com>

* Update tests/sql/glam-fenix-dev/glam_etl/README.md

Co-authored-by: Ben Wu <benjaminwu124@gmail.com>

* Document bootstrap in documentation

* Use artificial range for histogram_percentiles

* Simplify parameters for scalar probe counts

* Simplify tests for histogram probe counts

* Add test for incremental histogram aggregates

* Update scalar percentile counts to count distinct client ids

* Update readme for creating a new test

* Use unorded list for sublist

* Use --ignore-glob for pytest to avoid data files

Co-authored-by: Ben Wu <benjaminwu124@gmail.com>
2020-12-01 17:11:45 -08:00
Anna Scholtz fc95e32a31 Update DAGs 2020-10-05 12:59:58 -07:00
Anna Scholtz d1c67dab53 Move projects into high-level sql/ folder 2020-10-05 12:59:58 -07:00
Anna Scholtz 714f832898 Update tests 2020-10-05 12:59:58 -07:00
Daniel Thorn 26c67c7ee8
Upgrade to pytest 6.0.1 (#1281)
Also upgrade and fix pytest plugins
2020-09-02 11:30:14 -07:00
Anna Scholtz 5112d9dca0 Run pytest on mozfun UDFs 2020-07-08 14:53:44 -07:00
Daniel Thorn a08fa06b69
update dependencies (#994)
also fix unregistered marker warnings
also fix new lint errors
2020-05-20 15:55:17 -07:00
Sunah Suh ddfaf63f83
Add athena query migration script for posterity (#592)
* Add athena query migration script for posterity

* Add legacy athena migration script to pytest exclusions
2019-12-12 17:55:02 -05:00
Daniel Thorn e872a76860
Add pytest plugins to lint python scripts (#410)
* Add pytest plugins to lint python scripts

* Fix lint errors
2019-10-08 14:00:11 -07:00
Daniel Thorn ce69d308dd
Improve pytest failure output for udf and sql tests (#391)
* Improve pytest failure output for udf and sql tests

* address review
2019-10-01 12:48:22 -07:00
Daniel Thorn 41d718ebc0
Generate query tests with pytest plugin (#284)
so that we can do things like `pytest tests/telemetry_derived`
2019-08-06 10:02:54 -07:00
Daniel Thorn e1bf990b9a
Add support for testing queries with persistent UDFs (#285) 2019-08-05 14:14:19 -07:00
Daniel Thorn 52d10541d3
Create and publish docker image (#95)
* Create and publish docker image

* Update config.yml
2019-04-19 09:19:36 -07:00
Daniel Thorn 79070068ad
Add first test (#9) 2019-03-07 12:43:21 -08:00