bigquery-etl/requirements.in

20 строки
360 B
Plaintext
Исходник Обычный вид История

gcloud==0.18.3
google-cloud-bigquery==1.24.0
2020-05-26 21:56:18 +03:00
google-cloud-storage==1.27.0
2020-05-23 02:28:49 +03:00
Jinja2==2.11.2
pytest-black==0.3.8
pytest-docstyle==1.5.0
pytest-flake8==1.0.6
pytest-mypy==0.4.2
pytest-xdist==1.31.0
pytest==5.4.1
2020-05-15 01:21:12 +03:00
PyYAML==5.3
2020-05-26 21:56:18 +03:00
smart_open==1.10.0
sqlparse==0.3.1
Add histograms to fenix glam etl (#766) * Add initial template for histogram aggregates * Factor out common functions and get all distributions * Add viable query for histogram aggregates * Add more efficient aggregation * Update header and update comment * Add code to generate clients daily histograms * Add queries for generated sql * Return non-zero exit code when histograms not found * Delete empty queries to reduce data scanned * Add non-zero exit code for scalars if probes are not found * Sort histograms for stable output * Add view for histogram aggregates * Add initial sql for histogram aggregates * Format template scripts * Add mostly reformatted sql for aggregates * Update histogram aggregates before adding statements * Fix up details for daily aggregation * Add completed histograms template * Add code to generate clients histogram aggregates * Add init for clients histogram aggregates * Remove sample_id from set of attributes * Add sections to run generated sql * Add generated sql * Remove extra latest_version columns * Fix many small issues during first draft of sql * Fix clients histogram aggregates * Add initial modification to probe counts * Add histogram bucket counts * Add option to generate histogram probe counts * Update generated_fenix_sql for histograms * Add generated sql * Update run_fenix_sql * Fix bucket counts * Update source table for probe counts * Add missing ping_type to histograms * Add first,last,num buckets * Update probe counts so it succeeds * Add mozilla_schema_generator to dependencies * Add metadata from probe-info for custom distributions * Update probe counts with metadata for custom distributions * Add UDF for generating functional buckets * Add proper bucketing by including range_max of measures * Format histogram udfs * Add updated templates to skip * Add new queries to dryrun ignore * Add view to the publish ignore list * Fix python linting * Remove old comments from probe counts * Do not count metadata from custom distributions twice * Remove sum from histogram aggregates * Add generated SQL * Add sample_id to histograms earlier in pipeline * Add generated SQL * Add comments to functional bucketing for metrics
2020-03-18 23:53:28 +03:00
mozilla_schema_generator==0.1.4
2020-05-13 22:01:35 +03:00
GitPython==3.1.2
2020-05-15 01:21:12 +03:00
cattrs==1.0.0
attrs==19.3.0
typing==3.7.4.1
2020-07-24 00:40:28 +03:00
click==7.1.2