* Fix logging in backfill validation
* validate backfill should fail
* validate backfill should still fail
* revert entry, move validate-backfills to after generate-sql
* feat: add BigEye RunMetricOperator to DAG generation
* Fix Bigeye DAG generation and move configuration to bqetl_project.yaml
* Reformatting; test fixing
* Install Airflow override requirements
---------
Co-authored-by: Anna Scholtz <anna@scholtzan.net>
* Generate Bigeye monitoring configs in CI
* Ensure Bigconfig files are loaded exactly once
* Avoid duplicate validation of Bigconfig files
* Authentication using API key to Bigeye
* Remove api-key option for Bigeye and rely on env var instead
* Add fully-qualified identifiers when formatting queries
* Fully-qualified identifiers for queries in sql/
* Check in only formatted SQL to generated-sql branch
* Add comment
* Fully qualify more tables
* Fully qualify test files
* Formatting improvements around CTEs and unit tests
* Option to skip auto qualifying queries
* Fix PyPI package publishing in CI
* Update .circleci/workflows.yml
Co-authored-by: Mike Williams <102263964+mikewilli@users.noreply.github.com>
---------
Co-authored-by: Mike Williams <102263964+mikewilli@users.noreply.github.com>
* Run `post-diff.js` in the same CI job that uploads `sql.diff` as a CI artifact.
This is necessary for `post-diff.js` to get the correct CI job ID to use in the link to download the full `sql.diff` file.
* Only show link to full `sql.diff` file if it has content.
* Change the path `sql.diff` is stored at so the same `post-diff.js` script can used for both `bigquery-etl` and `private-bigquery-etl`.
* Require authentication for dry run function and run gcloud auth when not logged in
* authenticate step in CI, remove interactive gcloud auth
* Skip dryrun for ltv_state_values_v2
* Refactor skip_fork in CI, clarify login requirements
* Revert "Revert "Path Filtering in CI (#4939)""
This reverts commit d6189a7641.
* Revert "Revert "Run SQL checks when requirements change (#5030)""
This reverts commit 3991a2574b.
* Revert "Revert "Add parameter to skip stage depoys in CI (#5035)""
This reverts commit c4c651d0c5.
* run-all CI workflow
* Use conditionals in jobs
* Add deploy parameter and use yaml references for conditions
* Add backfill validation
* Push tags for commits on generated-sql branch
* Update .circleci/config.yml
Co-authored-by: Alexander <anicholson@mozilla.com>
---------
Co-authored-by: Alexander <anicholson@mozilla.com>
Ben Wu
Curtis Morales
kik-kik
Anna Scholtz
Corban Cloud
Sean Rose
whd
Mikaël Ducharme
Winnie Chan
Arkadiusz Komarzewski