* updates to mobile_search_aggregates and search_revenue_levers_table
adding search_with_ads_organic to mobile_search_aggregates and search_revenue_levers_table
* Update query.sql
* Fix schema.yaml and test files
* Fix CI issue
Fix CI issue
---------
Co-authored-by: Alekhya Kommasani <akommasani@mozilla.com>
Co-authored-by: Alekhya <88394696+alekhyamoz@users.noreply.github.com>
* feat: update firefox_android_clients_v1 to pull distribution_id only from the baseline ping
* feat: update firefox_android_clients_v1 baseline test schema to include distribution_id
* fix: resolve distribution_id not in baseline error
---------
Co-authored-by: Katie Windau <153020235+kwindau@users.noreply.github.com>
* removing acoustic last engaged timestamp
* removing suppression list model and tests
* removing dependency until frequency is increased
* formatting
* formatting
* adding organic searches with ads to this table
* updating mobile_search_aggregates table with search_with_ads_organic column
* updating the search revenue lever table
- include search_with_ads_organic columns for Bing, Google and DDG
* Fix CI issues
* Fix tests CI failure
* fix tests
* Fix test sql failure
* Update query.sql
reverting back to original code for search_revenue_levers table
---------
Co-authored-by: Alekhya Kommasani <akommasani@mozilla.com>
Co-authored-by: Alekhya <88394696+alekhyamoz@users.noreply.github.com>
* Add mobile suggest to event_aggregates and event_aggregates_suggest
* Rename test files
* Update expected values
* Fix tests
* Omit null values from expect.yaml
* updating source table and adding column
* updating tests to add new field
* changing to use updated_at
* formatting
---------
Co-authored-by: Leli <33942105+lelilia@users.noreply.github.com>
* changing structure of payload and structuring timestamps
* updating time filter and standardizing
* adding changed subscriptions model and updating time filter
* moving udf to separate pr and changing source table
* adding comments and moving max to timestamps
* filtering out mofo users and fxa_account_deleted true"
* updating filter on tables to pull most recent and adding acoustic event data
* formatting and refactoring
* formatting
* updating format and adding comments
* removing time filter
* fix alias
* reverting waitlists code back to original
* add lower() to waitlists platform and source
* ading inner join to users
* fix test_braze_subscriptions
* fix test_braze_waitlists
* DENG-3658 braze add unit test for users_v1
* add more tests
* only show users with active braze subscriptions
* reformat query
---------
Co-authored-by: Chelsey Beck <cbeck@mozilla.com>
Co-authored-by: Chelsey Beck <64881557+chelseybeck@users.noreply.github.com>
* Update description, partitioning and clustering setting for fenix_derived.firefox_android_clients_v1
* making sure _previous only includes entries older than the submission_date
* adding back parameter settings
* Update sql/moz-fx-data-shared-prod/fenix_derived/firefox_android_clients_v1/metadata.yaml
Co-authored-by: Lucia <30448600+lucia-vargas-a@users.noreply.github.com>
* feat: remove sample_id from clustering
* feat: use first_seen_date as filter when retrieving previous records. This has no impact as first_seen_date is the same as submission_date
---------
Co-authored-by: Lucia <30448600+lucia-vargas-a@users.noreply.github.com>
* changing structure of payload and structuring timestamps
* updating time filter and standardizing
* adding changed subscriptions model and updating time filter
* moving udf to separate pr and changing source table
* adding comments and moving max to timestamps
* filtering out mofo users and fxa_account_deleted true"
* updating filter on tables to pull most recent and adding acoustic event data
* formatting and refactoring
* formatting
* updating format and adding comments
* removing time filter
* fix alias
* reverting waitlists code back to original
* add lower() to waitlists platform and source
* ading inner join to users
* fix test_braze_subscriptions
* fix test_braze_waitlists
---------
Co-authored-by: Leli Schiestl <lschiestl@mozilla.com>
* status quo
* fix tests
* reformat query
* renaming to add v:
* add lower() to braze countries
* adding type
* updating field name
* updating user attributes payload within array to be _v1
* adding newsletter check
* updating snapshot to correct name
* formatting sql
* adding product checks
* adding subscriptions checks
* fixing schema
* removing checks for now and updating snapshot metadata
* updating interval
---------
Co-authored-by: Leli Schiestl <lschiestl@mozilla.com>
* updating products and adding changed products model
* updating user profiles to join on external id
* updating tests to match new schema
* reverting naming
* updating test
* updating products model to include additional attributes and updating tests
* updating order and tests
* updating format
* upddating source table
* updating schema
* updating user profiles table schema to match new products array
* adding models for changed and deleted users
* adding filter to changed waitlists model
* updating tests to include has_fxa field
* updating tests to include an unchanged user
* adding new line
* removing unnecessary cross join
* updating create statement
* joining on users table to filter for active and ensuring there is at least one subscription
* joining on users to filter for active
* adding dev subscription group
* removing fxa_id in favor of has_fxa
* bringing in update timestamp for downstream use
* updating formatting and adding filter for active users
* adding filter for one active newsletter
* updating tests
* adding fxa id back to users table to join to products
* updating query
* updating values
* updating tests
* fix test for subscriptions
* changing schema to array
* updating format
* updating to pull in all subscriptions with statuses
* removing create statement
* updating subscriptions query to make it an array and updating associated tests
* updating formatting and comment
---------
Co-authored-by: Leli Schiestl <lschiestl@mozilla.com>
* Remove init.sql files for fenix and use is_init() instead
* Remove init.sql files for search_derived datasets
* Simplify is_init() for acer_cohort_v1
* DENG-2979 - add logic to use GA4 client ID after it starts coming through instead of old GA3 one, & start updating tests
* DENG-2979 fix SQL formatting
* DENG-2979 - add temp table to do a final distinct
* DENG-2979 add a data check to make sure the code errors out if someone tries to run prior to 8-25-2023
* DENG-2979 update the date of the tests to be more recent and not fall in the data check fail period
* DENG-2979 update test cases
* Bump black from 23.10.1 to 24.1.1
Bumps [black](https://github.com/psf/black) from 23.10.1 to 24.1.1.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/23.10.1...24.1.1)
---
updated-dependencies:
- dependency-name: black
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* Reformat files with black to fix dependabot update.
* Reformat with black 24.1.1. Update test dag with required space.
* Update test dags.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Tweaking firefox_android_clients_v1 to also include play_store attribution fields
* removed additional logic used for testing found within _previous CTE
* removed firefox_android_clients_v1 init.sql in favour of templating via is_init() inside the query
* Made changes as suggested by fbertsch in PR#4940
* Fixing sql tests
* Avoid using `Path.glob()` or `Path.rglob()` for recursive file searches.
Because they don't currently support following symlinks (they will in Python 3.13).
* Specify `followlinks=True` as necessary when calling `os.walk()`.
* Adding topsite dismissals to newtab_* jobs.
* Update test to include new dismissal fields
---------
Co-authored-by: Curtis Morales <cmorales@mozilla.com>
Curtis Morales
Preethi Issac
Eduardo Filho
Anna Scholtz
kik-kik
Chelsey Beck
Alekhya
Leli
Lucia
Winnie Chan
m-d-bowerman
Katie Windau
Alexander
Sean Rose
Sergio E. Betancourt
Daniel Thorn