Граф коммитов

96 Коммитов

Автор SHA1 Сообщение Дата
Daniel Thorn a0d810275b
Remove java dependency in favor of sqlglot (#3755) 2023-05-17 14:56:42 -07:00
Sean Rose b7c3e3c183
Add Visual Studio Code debug configuration for bqetl. (#3395) 2022-12-01 16:42:52 -08:00
Curtis Morales 1897ec9c05
Add M1 Mac setup guide (#3302) 2022-11-14 11:03:32 -05:00
kik-kik 323fb96ecb
Tweaking Apple Silicon comment to red to make it more prominent (#3346) 2022-11-09 16:45:18 +01:00
kik-kik cd224645f2
updated badge status url (#3344) 2022-11-08 18:07:04 +01:00
Frank Bertsch 00a04b699c
Move to python 3.10 (#3236)
* WIP: Move to python 3.10

* Remove netlify
2022-09-22 15:39:40 -04:00
Sean Rose 3fb6ebc160
Specify pyjnius in `requirements.in` (#2924) 2022-04-29 12:23:47 -07:00
Daniel Thorn 7e1833b37b
Support java 8 for Netlify compat (#2923) 2022-04-29 11:00:46 -07:00
Lucia f6a71623d3
DENG-72 add required data for GUD to unified_metrics query (#2912)
* Add attribution to unified_metrics

* Add fields to unified_metrics query (app_version, distribution_id, city, os_version_major,os_version_minor and attribution content, source, medium, campaign, experiment, variation). Jira issue DENG-72.

* Update version major and minor to numeric values and remove unneeded field aliases

Co-authored-by: Lucia Vargas <lvargas@mozilla.com>
2022-04-29 10:36:57 +02:00
Daniel Thorn 0235d6087c
Reapply "Update to zetasql 2022.02.1 (#2862)" (#2881) 2022-04-07 14:02:20 -07:00
Daniel Thorn 22835e5706
Revert "Update to zetasql 2022.02.1 (#2862)" (#2879)
This reverts commit 9c9761feac.
2022-04-07 12:03:09 -07:00
Daniel Thorn 9c9761feac
Update to zetasql 2022.02.1 (#2862) 2022-04-05 18:55:52 +02:00
Daniel Thorn 4406437be2
Use --no-deps when installing compiled requirements files (#2752) 2022-02-24 21:36:47 +00:00
Jeff Klukas ffb3d226e3
Add links to advice for running with and without Rosetta (#2622) 2022-01-05 18:56:09 +00:00
Linh Nguyen 4876f84475
Require downloading Java dependencies as part of setup (#2589)
Downloading JAVA is now required in many places in our repo, such as generating docs, running dry-run, etc. I think it's time that we remove the "optionally" part from this step instruction.
2021-12-21 15:13:46 +00:00
Will Lachance 1d6fea8f2d
Big docs/README refactoring (#2527)
* Move most reference and cookbook documentation in README.md into
  the "reference" and "cookbook" sections of the generated documentation,
  respectively.
* Try to steer people to the generated docs inside the README.md (since
  it is now basically just a set of quickstart instructions)
* Provide a bit of guidance that this repository isn't great for 3rd
  party contributors in a new CONTRIBUTING.md.

Co-authored-by: Jeff Klukas <jklukas@mozilla.com>
2021-11-29 21:28:07 +00:00
= 4d253f8322 Tweaked Quick Start section to emphasise extra step for M1 users 2021-11-05 17:51:59 +01:00
Daniel Thorn d268bc4cdd
Remove virtualenv install from readme (#2449) 2021-10-21 19:02:08 +00:00
Daniel Thorn 2d534af8a2
Update minimum JDK to 11 and update install recommendation (#2450) 2021-10-21 10:39:43 -07:00
Alexander Nicholson 4909841b98
Added notes about Rosetta2 and a JDK version for Apple Silicon users (#2447) 2021-10-21 12:22:19 -04:00
Will Lachance 411af312e4
Replace `format_sql` with "bqetl format" (#2348)
* Make bqetl work equivalently to format_sql (which had some extra
options to format standard in)
* Remove `format_sql` and update everything that uses it to use
  `bqetl format` instead.
2021-09-20 17:18:25 +00:00
Daniel Thorn 1ce8be1343
Add --update-adc to quick-start instructions (#2324) 2021-09-13 23:01:22 +00:00
Anna Scholtz f3203ddd8c Update docs and add pull request template 2021-06-24 13:56:53 -07:00
Jeff Klukas 9c7dadc6ec
Add documentation for dataset_metadata.yaml (#2107)
* Add documentation for dataset_metadata.yaml

* Mention operator approval for ACLs

* Update details about automated deploys
2021-06-09 16:03:26 -04:00
Anna Scholtz bb94b7e360 Remove allow_field_addition_on_date usage 2021-05-07 10:30:02 -07:00
Daniel Thorn ca38326e91
Fix dependabot updates by separating pyjnius requirement (#1990) 2021-05-04 13:02:56 -04:00
whd 7c1b03934b
Default branch (#1939)
* Rename default branch

* Rename branch

* Update circleci for default branch name
2021-04-06 21:15:21 +00:00
William Lachance 9f21097a49
Remove .vscode settings and add to gitignore (#1929)
.vscode's settings.json can get cluttered with installation-specific
settings, which can lead to confusing pull requests. Instead, let's use
the approach outlined here:

https://stackoverflow.com/a/48387809

This provides a set of defaults that people can use, which we can
extend over time.
2021-03-30 12:25:05 -04:00
Anna Scholtz bcfadec33b Link bqetl docs in README 2021-03-25 14:16:06 -07:00
Daniel Thorn 744600069d
Configure shared pre-commit hooks to enforce linting (#1839) 2021-02-24 14:03:37 -08:00
Daniel Thorn 5d07beaca7
Use zetasql to get dependencies for dag generation (#1817) 2021-02-18 17:49:46 -05:00
William Lachance 5b04b0f33b
Add (mandatory) description parameter when creating a DAG using cli (#1769) 2021-02-08 14:21:15 -05:00
Anna Scholtz 8e90f0b063 Update DAGs with Markdown description 2021-02-01 13:16:17 -08:00
William Lachance a0516dd656
More minor README tweaks (#1704) 2021-01-22 10:40:19 -05:00
William Lachance e4c823d90b
Add some data engineering specific setup instructions (#1702) 2021-01-21 13:45:31 -08:00
Sunah Suh 6a96fbebe2
Add quickstart section to README (#1669) 2021-01-13 16:45:21 -06:00
William Lachance 6657d92153
Fix link to GCP access request (#1655)
Co-authored-by: Sunah Suh <ssuh@mozilla.com>
2021-01-12 14:51:38 -06:00
William Lachance dcedaee919
Streamline bqetl initialization and utilization (#1645)
This simplifies the multiple choice installation process into
a simplified version which uses a local virtualenv, which will
hopefully be easier to support and debug.
2021-01-06 11:11:20 -05:00
jailang 0952c4140a Update README 2020-10-28 09:01:43 -07:00
Anna Scholtz bd2fc3005c Add clarifications about `project` to README
Co-authored-by: Jeff Klukas <jklukas@mozilla.com>
2020-10-05 12:59:58 -07:00
Anna Scholtz a16bc2824f Fix validating docs 2020-10-05 12:59:58 -07:00
Anna Scholtz d1c67dab53 Move projects into high-level sql/ folder 2020-10-05 12:59:58 -07:00
Anna Scholtz 06233819ab Remove sql/ directory 2020-10-05 12:59:58 -07:00
Anna Scholtz 291222a926 Update README paths to specify project 2020-10-05 12:59:58 -07:00
Anna Scholtz ad896eed82 Remove anomdtct 2020-10-02 10:31:42 -07:00
Frank Bertsch 5257299104
Add submission date parameter (#1312) 2020-09-14 16:53:28 -04:00
Anna Scholtz 682c3bad6d Add docs for scheduling queries using bqetl CLI 2020-09-11 14:26:59 -07:00
Anna Scholtz ce23af6d0b Add bqetl docs 2020-09-11 14:26:59 -07:00
Frank Bertsch ceb7379858
Create view for event_types from recent day's data (#1306)
* Add option to specify (or not) dest table

* Add option to DAG generation for single dag

* Add query for recent day's event_types

* Use explicit None for no destination table

* Reformat SQL

* Use event_types view in events_daily

* Update tests

* Run black

* Add view for events_daily

* Format SQL

* Ignore event properties in query

* Shorten events_name_v1

* Fix naming for test

* Update README.md

Co-authored-by: Anna Scholtz <anna@scholtzan.net>

Co-authored-by: Anna Scholtz <anna@scholtzan.net>
2020-09-11 16:39:07 -04:00
Anna Scholtz a39e168a2d Update query scheduling section 2020-07-29 14:11:00 -07:00