2d407f7e93
GROWTH-101 - Update gclid_conversions view to 1-row per conversion ( #4612 )
...
* Update gclid_conversions view to 1-row per conversion
* Fully qualify table
2024-02-12 14:16:59 -08:00
b605cd9e26
DENG-2492 blogs sessions v2 ( #5019 )
...
* DENG-2492 filter to blog.mozilla.org only since new ID contains other domains also
2024-02-12 13:32:09 -08:00
96b3fc379b
DS-3103. Update view to query clients_first_seen_v2. ( #4237 )
2024-02-12 14:55:12 -05:00
8ec7516157
Issue 4135: Added publish metadata cli command ( #5011 )
...
* Added publish metadata cli command
* Removed publish metadata script
2024-02-12 11:12:14 -08:00
ee8de94705
DENG-2492 Create new GA4 derived table: blogs_sessions_v2 ( #5018 )
...
* DENG-2492 initial commit for new table blogs_sessions_v2
* DENG-2492 wrap keywords with backticks
2024-02-12 10:54:41 -06:00
63a4d72197
feat(): added scheduling settings to all fxa_users_*_v2 queries ( #4893 )
...
* added scheduling settings to all fxa_users_* queries
* updated start date
2024-02-12 15:46:48 +01:00
3a5b5f0132
Bump grpcio from 1.54.2 to 1.54.3 ( #5012 )
...
Bumps [grpcio](https://github.com/grpc/grpc ) from 1.54.2 to 1.54.3.
- [Release notes](https://github.com/grpc/grpc/releases )
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md )
- [Commits](https://github.com/grpc/grpc/compare/v1.54.2...v1.54.3 )
---
updated-dependencies:
- dependency-name: grpcio
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-12 11:12:23 +01:00
5db055303e
Bump jsonschema from 4.19.2 to 4.21.1 ( #5013 )
...
Bumps [jsonschema](https://github.com/python-jsonschema/jsonschema ) from 4.19.2 to 4.21.1.
- [Release notes](https://github.com/python-jsonschema/jsonschema/releases )
- [Changelog](https://github.com/python-jsonschema/jsonschema/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/python-jsonschema/jsonschema/compare/v4.19.2...v4.21.1 )
---
updated-dependencies:
- dependency-name: jsonschema
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-12 09:45:12 +01:00
cfbf296f32
DENG-2492 create new GA4 derived table firefox_whatsnew_summary_v2 ( #5006 )
...
* DENG-2492 create new GA4 derived table firefox_whatsnew_summary_v2
* DENG-2492 shorten column descriptions
* DENG-2492 clear up visits column desc
2024-02-09 14:21:58 -06:00
55eb0c8299
DENG-2492 fix offsets for page levels, they were all off by 1 ( #5005 )
2024-02-09 12:25:21 -06:00
27f15163a1
Fix `bqetl query schema deploy` to find script ETLs specified as `{dataset}.{table}`. ( #5004 )
2024-02-09 10:17:06 -08:00
ea192745ef
Bump mozilla-metric-config-parser from 2023.10.2 to 2023.11.1 ( #4999 )
...
Bumps [mozilla-metric-config-parser](https://github.com/mozilla/metric-config-parser ) from 2023.10.2 to 2023.11.1.
- [Release notes](https://github.com/mozilla/metric-config-parser/releases )
- [Commits](https://github.com/mozilla/metric-config-parser/compare/2023.10.2...2023.11.1 )
---
updated-dependencies:
- dependency-name: mozilla-metric-config-parser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-09 08:22:58 -08:00
7c5270dab5
Bump types-requests from 2.31.0.10 to 2.31.0.20240125 ( #5000 )
...
Bumps [types-requests](https://github.com/python/typeshed ) from 2.31.0.10 to 2.31.0.20240125.
- [Commits](https://github.com/python/typeshed/commits )
---
updated-dependencies:
- dependency-name: types-requests
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-09 08:22:27 -08:00
82fc3d0743
Adds absolute value to check for unreasonable increase ( #4998 )
2024-02-09 08:02:49 -08:00
9e794dee20
update_ios_feature_usage_events_w_ping_parse_time_logic ( #4968 )
...
* update_ios_feature_usage_events_w_ping_parse_time_logic
* Update query.sql
* Update schema.yaml
* Update query.sql
* Update query.sql
2024-02-09 14:56:00 +01:00
9049398c36
adding firefox_ios_derived.retention_clients to shredder config ( #4993 )
2024-02-09 12:14:00 +01:00
133adfd472
Don't overwrite existing `gcp.json` files unless that's actually necessary ( #4997 )
2024-02-08 15:37:28 -08:00
eeb78dc7e4
DENG-2492 update the product download types and the clustering ( #4996 )
2024-02-08 15:47:03 -06:00
c05ac1d661
Stop using data-eng-circleci-tests context in CI ( #4884 )
2024-02-08 21:35:55 +00:00
638e4e2065
DENG-2492 - fix time_on_site formula to convert to seconds ( #4994 )
...
* DENG-2492 - fix time_on_site formula to convert to seconds
* DENG-2492 update SQL format with new formula change
2024-02-08 14:51:04 -06:00
e741b7a967
Add clustering and partitioning to meta_attribution_country_counts_v1 init.sql ( #4995 )
2024-02-08 12:50:07 -08:00
1364b2af17
Fixed array concat if null ( #4975 )
2024-02-08 18:28:24 +00:00
8beb092a33
DENG-2492 ga_sessions_v2 logic updates ( #4988 )
...
* DENG-2492 initial commit
* DENG-2492 reformat script.sql
* DENG-2492 fix typos
* switch out DL types and fix a typo
* DENG-2492
* DENG-2492 update query logic
* DENG 2492 add meaningful aliases
* DENG-2492 rename rnk to rownum to be more clear
* DENG-2492 add AS for aliasing
* DENG-2492 switch to using
* DENG-2492 move row number out of select and keep only in qualify
* DENG-2492 remove unnecessary a aliases
2024-02-08 12:20:09 -06:00
e056243587
Bump google-cloud-bigquery-storage[fastavro] from 2.23.0 to 2.24.0 ( #4761 )
...
Bumps [google-cloud-bigquery-storage[fastavro]](https://github.com/googleapis/python-bigquery-storage ) from 2.23.0 to 2.24.0.
- [Release notes](https://github.com/googleapis/python-bigquery-storage/releases )
- [Changelog](https://github.com/googleapis/python-bigquery-storage/blob/main/CHANGELOG.md )
- [Commits](https://github.com/googleapis/python-bigquery-storage/compare/v2.23.0...v2.24.0 )
---
updated-dependencies:
- dependency-name: google-cloud-bigquery-storage[fastavro]
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-08 09:41:53 -06:00
37af26efdf
Bump gitpython from 3.1.40 to 3.1.41 ( #4802 )
...
Bumps [gitpython](https://github.com/gitpython-developers/GitPython ) from 3.1.40 to 3.1.41.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases )
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES )
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.40...3.1.41 )
---
updated-dependencies:
- dependency-name: gitpython
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anna Scholtz <anna@scholtzan.net>
2024-02-08 09:41:29 -06:00
dab99e9a4b
Bump symbolic from 12.4.1 to 12.8.0 ( #4820 )
...
Bumps [symbolic]() from 12.4.1 to 12.8.0.
---
updated-dependencies:
- dependency-name: symbolic
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anna Scholtz <anna@scholtzan.net>
2024-02-08 09:40:43 -06:00
180db53c86
Bump rich-click from 1.7.2 to 1.7.3 ( #4992 )
...
Bumps [rich-click](https://github.com/ewels/rich-click ) from 1.7.2 to 1.7.3.
- [Release notes](https://github.com/ewels/rich-click/releases )
- [Changelog](https://github.com/ewels/rich-click/blob/main/CHANGELOG.md )
- [Commits](https://github.com/ewels/rich-click/compare/v1.7.2...v1.7.3 )
---
updated-dependencies:
- dependency-name: rich-click
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-08 09:39:37 -06:00
dd96eea480
Adding daily country and channel counts for fenix meta attribution data ( #4806 )
...
* Adding daily country and channel counts for fenix meta attribution data
Co-authored-by: Anna Scholtz <anna@scholtzan.net>
* Removing extraneous field
---------
Co-authored-by: Rowan Vasquez <rvasquez@XD91WLTP7Q.lan>
Co-authored-by: Frank Bertsch <frank.bertsch@gmail.com>
Co-authored-by: Anna Scholtz <anna@scholtzan.net>
2024-02-07 18:15:28 -05:00
373fa54c61
Deng 2579 gclid conversions v2 creation ( #4972 )
...
* initial commit of code for gclid_conversions_v2
* change owners to Marlene and Katie
* add gclid_conversions_v2 to Access Denied bqetl_project.yaml
2024-02-07 14:03:32 -08:00
378ce60be3
Added user type to view ( #4986 )
2024-02-07 13:36:33 -08:00
4d7c2cf0bf
DS-3102 Add checks to clients_first_seen_v2 ( #4982 )
...
* DS-3102 Add checks to clients_first_seen_v2
* Formatting
* Fully qualify and template source tables
* Update to countif instead of assert not exists
* Add AS table aliases
2024-02-07 15:55:30 -05:00
d980b3b211
DENG-2492 - update logic of new ga_sessions_v2 table ( #4985 )
...
* DENG-2492 - update logic of new ga_sessions_v2 table
* DENG-2492 - update logic of new ga_sessions_v2 table
2024-02-07 14:54:18 -06:00
802da71a2c
Add ETLs and views for Google Search Console data (DENG-1733) ( #4892 )
...
* Add ETLs for historical Google Search Console data synced by Fivetran.
* Fix formatting of `CASE` subclauses like `WHEN` inside Jinja blocks.
* Add ETLs for current Google Search Console data exported directly to BigQuery.
* Add views for Google Search Console data.
2024-02-07 12:53:32 -08:00
0b8e8f14a4
Push tags for commits on generated-sql branch ( #4984 )
...
* Push tags for commits on generated-sql branch
* Update .circleci/config.yml
Co-authored-by: Alexander <anicholson@mozilla.com>
---------
Co-authored-by: Alexander <anicholson@mozilla.com>
2024-02-07 11:07:32 -08:00
e10b93c06c
fix typo in CI ( #4983 )
...
Typo in circleci
2024-02-07 10:08:56 -08:00
df98c38e6c
Replace the `mozilla_vpn_derived.vat_rates_v1` ETL with a CSV file (bug 1878898) ( #4976 )
...
* Replace the `mozilla_vpn_derived.vat_rates_v1` ETL with a CSV file containing the same data.
The source Google Sheet for the `mozilla_vpn_derived.vat_rates_v1` ETL has apparently been deleted (bug 1878898).
* Add missing VAT rates for one VPN wave 3 country.
* Add missing VAT rates for seven VPN wave 6 countries.
2024-02-07 09:48:31 -08:00
07b748849d
Update `country_codes_v1` regions from latest UN Statistics Division data. ( #4978 )
2024-02-07 09:30:22 -08:00
fa1c2492bd
Use caching in generate-sql ( #4953 )
...
* Use caching in generate-sql
* Make change in sql generator
* Remove file
* Add CI comments
2024-02-07 08:57:05 -08:00
e2f33ed29b
Add debug messages for `gcp.json` file. ( #4979 )
2024-02-07 08:12:25 -08:00
ef7bd6f34f
Bump pathos from 0.3.1 to 0.3.2 ( #4980 )
...
Bumps [pathos](https://github.com/uqfoundation/pathos ) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/uqfoundation/pathos/releases )
- [Commits](https://github.com/uqfoundation/pathos/compare/pathos-0.3.1...0.3.2 )
---
updated-dependencies:
- dependency-name: pathos
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-07 10:29:50 +01:00
3bc79e6745
DENG-2492 - update time on site to convert to seconds, add comments ( #4970 )
...
* DENG-2492 - update time on site to convert to seconds, add additional comments to code
* DENG-2492 update formatting of updated SQL file
* DENG-2492 update time_on_site to be int64
* DENG-2492 update query formatting
2024-02-06 15:21:38 -06:00
0a9ee2434a
AD-178 Update event_aggregates tables to pull query_type from the Ads team's table ( #4967 )
...
* Pull query_type from adm.blocks table
* Do the same in event_aggregates_v1
* Update tests
2024-02-06 13:37:45 -05:00
ab68b6fa04
events stream: Convert nested maps in metrics ( #4964 )
2024-02-06 14:21:00 +01:00
d192f9a627
Bump cryptography from 41.0.4 to 42.0.0 ( #4963 )
...
Bumps [cryptography](https://github.com/pyca/cryptography ) from 41.0.4 to 42.0.0.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/pyca/cryptography/compare/41.0.4...42.0.0 )
---
updated-dependencies:
- dependency-name: cryptography
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-06 10:47:33 +01:00
5d0c30c84a
Fb deng 2492 ga sessions v2 ( #4962 )
...
* DENG-2492 - update logic for new ga_sessions_v2 load process
* DENG-2492 change update process
2024-02-05 16:23:17 -06:00
f7648d32fe
DENG-2492 - Create new GA4 derived table ga_sessions_v2 ( #4961 )
...
* DENG-2492 - initial commit
* DENG-2492 rename query.sql to script.sql
2024-02-05 15:29:26 -06:00
d43d6ebc23
removed view ( #4932 )
2024-02-05 12:04:08 -08:00
013c63a600
Add moso datasets with DBT workgroup access ( #4960 )
...
Co-authored-by: Wesley Dawson <whd@mozilla.com>
2024-02-05 18:50:17 +00:00
138841d351
Package bqetl and publish to PyPI ( #4917 )
...
* pyproject.toml for bqetl
* Correctly resolve SQL generators from package
* CircleCI config to publish tagged versions to PyPI
* Get version from git tags
2024-02-05 09:04:04 -08:00
a4c7b0ab40
Remove gke_command usages ( #4900 )
2024-02-05 08:40:34 -08:00
Frank Bertsch
Katie Windau
Lucia
Winnie Chan
kik-kik
dependabot[bot]
Sean Rose
m-d-bowerman
rzhao
whd
Anna Scholtz
Rowan
Marlene Hirose
Alexander
Curtis Morales
Jan-Erik Rediger
Chelsey Beck