c5aecd6df4
fix(bqetl_artifact_deployment): Fix Bigeye monitoring deploy ( #2094 )
2024-09-17 14:44:08 -07:00
e153811500
fix(managed-backfills): Add clarity to slack notifications ( #2091 )
...
* fix(managed-backfills): Add clarity to slack notifications
* Update dags/bqetl_backfill_complete.py
Co-authored-by: Alexander <anicholson@mozilla.com>
---------
Co-authored-by: Alexander <anicholson@mozilla.com>
2024-09-17 13:31:30 -07:00
3f5e4fd541
feat(bqetl_artifact_deployment): Add task to deploy Bigeye monitors ( #2090 )
2024-09-17 12:42:47 -07:00
902d611372
chore(shredder): Update links and descriptions in shredder doc ( #2092 )
2024-09-13 22:10:03 +01:00
b814d4708d
feat: Remove spectacles pip install ( #2089 )
2024-09-11 17:01:59 -07:00
1bf8b8eeaa
fix(DENG-4142): Update from marketing-prod to shared-prod path ( #2088 )
2024-09-09 08:12:03 -05:00
f1b4305757
fix(GLAM): wrap bash params within quotes ( #2086 )
...
* fix(GLAM): escape param quotes
* no need to escape quotes
2024-09-06 15:20:49 -04:00
f877b39e5b
fix(GLAM): Add --append_table when needed to FoG bucket counts ( #2085 )
2024-09-06 12:48:00 -04:00
d1f2e74318
fix(socorro_import): try manually built image ( #2084 )
2024-09-04 23:25:43 +00:00
b3eb16eff5
fix(GLAM): Proper TaskGroup dependency handling ( #2083 )
2024-09-04 13:30:25 -04:00
4fb9f703b3
fix(GLAM): Fix TaskGroup upstream deps ( #2082 )
2024-09-04 12:56:38 -04:00
c5c8b873b6
fix(GLAM): Fix glam gen query ( #2081 )
...
* chore(GLAM): Turin FoG histogram_bucket_counts into TaskGroup
* Add comment to use_sample_id logic
* fix(GLAM): Fix bool param passing to generate_glean_sql
2024-09-04 11:23:37 -04:00
ebfaad7261
chore(GLAM): Turn FoG histogram_bucket_counts into TaskGroup ( #2080 )
...
* chore(GLAM): Turin FoG histogram_bucket_counts into TaskGroup
* Add comment to use_sample_id logic
2024-09-03 14:02:52 -04:00
70dee34346
feat(code): Workday Netsuite DAG ( #2079 )
...
Co-authored-by: Katie Windau <153020235+kwindau@users.noreply.github.com>
2024-09-03 09:42:31 -05:00
606b1e2900
feat: Publish tables and views in the `moz-fx-data-bq-people` project. ( #2075 )
2024-08-29 10:00:16 -07:00
9356ac473d
fix(fxci): change fxci metric export schedule to run at 00:30 ( #2062 )
...
There's a check in the code that prevents the export from running too close to midnight. This is because there's a slight lag in the Google Cloud Monitoring data becoming available.
Running 30 minutes past the hour should be plenty of buffer time.
2024-08-29 06:08:47 -07:00
5c2474039a
fix: update fxci_metric_export dag to pass in --date ( #2074 )
...
The underlying ETL is switching to explicitly passing in the date, so
adjust accordingly.
2024-08-28 22:58:22 +02:00
7df57eaa16
refactor: Change `lookml_validate_spectacles` task to use environment variables that Spectacles expects ( #2078 )
...
* refactor: Change `lookml_validate_spectacles` task to use environment variables that Spectacles expects.
* refactor: Change `lookml_validate_spectacles` task to specify the branch directly rather than using an environment variable.
2024-08-28 08:29:36 -07:00
b75268c661
feat: Add task in probe_scraper for ping_expiry_alerts ( #2077 )
2024-08-27 21:53:00 +01:00
b4a473ffdf
fix: Use Airflow templated variables for LookML validation. ( #2076 )
...
So the variable values aren't shown in the task instance details.
2024-08-27 12:41:05 -07:00
86860253c0
chore: Skip artifact deploys when there are other queued runs ( #2072 )
2024-08-23 16:25:45 +01:00
6db42cd4e1
fix: Using Airflow variables for LookML validation ( #2071 )
2024-08-22 12:06:13 -07:00
ffccc6f078
feat: Fix typos in spectacles ( #2070 )
2024-08-22 09:42:58 -07:00
169d9a07fa
feat: Add Looker content validation for KPI related dashboards and explores ( #2068 )
...
* feat(looker): Add Looker content validation for KPI related dashboards and explores
* Update dags/looker.py
Co-authored-by: Sean Rose <1994030+sean-rose@users.noreply.github.com>
* Update dags/looker.py
Co-authored-by: Sean Rose <1994030+sean-rose@users.noreply.github.com>
* Update dags/looker.py
Co-authored-by: Sean Rose <1994030+sean-rose@users.noreply.github.com>
* Update dags/looker.py
Co-authored-by: Sean Rose <1994030+sean-rose@users.noreply.github.com>
* Update dags/looker.py
---------
Co-authored-by: Sean Rose <1994030+sean-rose@users.noreply.github.com>
2024-08-22 09:25:10 -07:00
d65839c57f
feat: remove data-observability-dev from artifact deployment ( #2069 )
...
Follow up to https://github.com/mozilla/bigquery-etl/pull/6081 , will fix one of the issues currently affecting artifact deployment
2024-08-22 15:06:19 +02:00
65a83cbdc5
feat(code): New Slack Channels DAG ( #2067 )
...
Co-authored-by: Julio Cezar Moscon <jcmoscon@gmail.com>
2024-08-21 15:05:31 -05:00
2d4eac6287
fix(code):[Workday/XMatters] - Change retries and retry_delay ( #2066 )
...
* fix(code):Change retries and retry_delay
* Update dags/eam_workday_xmatters_integration.py
Co-authored-by: Alexander <anicholson@mozilla.com>
* fix(code): Added import timedelta
---------
Co-authored-by: Julio Cezar Moscon <jcmoscon@gmail.com>
Co-authored-by: Alexander <anicholson@mozilla.com>
2024-08-15 16:14:57 -04:00
fe3de9b5df
fix: change owner for probe scraper dag glean dictonary task ( #2065 )
2024-08-14 21:44:47 +02:00
76a322f035
fix: remove linh from dags ( #2064 )
...
* cleanup: remove linh from dags
* empty commit
2024-08-14 16:57:29 +02:00
3354a5a736
docs: add tags to eam_workday_xmatters_integration_dag ( #2063 )
...
* add tags to eam_workday_xmatters_integration_dag
* empty commit
2024-08-14 16:33:38 +02:00
36d653dbe4
feat: Bug 1910995 - Enable the detection of TB metrics ( #2059 )
...
* Bug 1910995 - Enable the detection of TB metrics
As called out in mozilla/probe-scraper#790 this PR is necessary to address the lack of a way to run the GH workflow for Thunderbird ([docs](https://mozilla.github.io/glean/book/user/adding-glean-to-your-project/enable-data-ingestion.html#validate-and-publish-metrics )). Given that TB uses Mercurial/Git Mirror, I'm replicating the same mechanism used for Firefox Desktop.
* feat: Bug 1910995 - Enable the detection of TB metrics
---------
Co-authored-by: akkomar <akkomar@users.noreply.github.com>
2024-08-13 15:38:12 +02:00
13374139d8
feat(code):Change the Jira domain to Prod and remove no_triage tag ( #2061 )
...
Co-authored-by: Julio Cezar Moscon <jcmoscon@gmail.com>
2024-08-12 09:38:14 -04:00
ddcba610f5
chore: Add bugzilla token for webcompat_kb DAG and update docs ( #2060 )
2024-08-09 10:02:36 -07:00
cebb534a13
fix(dbt_daily): Set dbt account ID ( #2057 )
...
* fix(dbt_daily): Set dbt account ID
* Update dags/dbt_daily.py
Co-authored-by: Ben Wu <12437227+BenWu@users.noreply.github.com>
---------
Co-authored-by: Ben Wu <12437227+BenWu@users.noreply.github.com>
2024-08-06 08:35:18 -07:00
d0d5205b84
feat(url):change Jira link to prod. Remove tag no_triage. ( #2058 )
...
Co-authored-by: Julio Cezar Moscon <jcmoscon@gmail.com>
2024-08-05 07:35:03 -05:00
9dd21df2c7
feat(dag): Schedule daily dbt runs ( #2056 )
2024-07-31 15:08:31 -07:00
fe42a97bb7
chore(GLAM): extracts no longer parametrized ( #2055 )
...
* chore(GLAM): extracts no longer parametrized
* Fix query paths
Co-authored-by: Ben Wu <12437227+BenWu@users.noreply.github.com>
---------
Co-authored-by: Ben Wu <12437227+BenWu@users.noreply.github.com>
2024-07-30 16:30:17 -04:00
2e7e1c2fd7
chore: remove unused aws_socorro_readonly_s3 connection ( #2041 )
...
Co-authored-by: Mikaël Ducharme <mducharme@mozilla.com>
2024-07-30 18:48:33 +00:00
e2b99e85cc
refactor(GLAM): run sample_counts on shared-prod ( #2054 )
2024-07-30 12:32:31 -04:00
2efe003555
fix(plugins): Use tuple for timetable plugin registration ( #2053 )
2024-07-29 13:45:46 -04:00
33370d8f4b
chore: Replace Black with Ruff and remove diff-based CI. ( #2052 )
2024-07-29 11:57:46 -04:00
acc73decb5
feat(code): Workday-Everfi [ASP-5043] - changes in the DAG configuration ( #2051 )
...
* feat(): changes in retries, retry_delay and schedule_interval of the DAG
* fix(format): fix string format
* fix(code): make fixes
* fix(code): fix import datetime
* fixed datetime call
---------
Co-authored-by: Julio Cezar Moscon <jcmoscon@gmail.com>
2024-07-25 08:36:06 -07:00
d22f9005b6
refactor(GLAM): Remove percentile calculation ( #2050 )
...
* refactor(GLAM): Remove percentile calculation
* fix ruff
2024-07-24 09:40:01 -04:00
3ec47fb306
feat(dags): adjust dap collector run schedules ( #2044 )
...
* adjust dap collector run schedules
* feat(dags) adjust dap collector run schedules
* pass in the end date of the data interval to the etl job
---------
Co-authored-by: Anna Scholtz <anna@scholtzan.net>
2024-07-22 13:41:58 -07:00
79b3199fd4
chore: Remove data observability POC DAGs ( #2049 )
2024-07-22 12:21:23 -07:00
23970b61a5
chore(glam): remove non-normalized bucket counts ( #2048 )
2024-07-18 11:34:18 -04:00
de9037522c
chore: remove search engine usage study ( #2047 )
2024-07-16 08:01:33 +02:00
37be229e90
change offset of day from 1 to 2 ( #2046 )
2024-07-15 10:06:38 -07:00
b0ec26b650
feat: create two new dags for the Firefox-CI ETL ( #2039 )
...
This adds two new DAGs:
* fxci_metric_export - Retrieves metrics from Google Cloud Monitoring
and inserts them into BigQuery
* fxci_pulse_export - Connects to Taskcluster pulse queues, drains the
events and inserts them into BigQuery
Bug: 1904928
Co-authored-by: akkomar <akkomar@users.noreply.github.com>
2024-07-12 18:15:26 +02:00
e2fc9a799c
docs: update docs for broken_site_report_ml DAG ( #2045 )
2024-07-12 14:52:20 +02:00
Anna Scholtz
Winnie Chan
Ben Wu
Katie Windau
Eduardo Filho
whd
JCMOSCON1976
Sean Rose
Andrew Halberstadt
akkomar
Leli
Alessio Placitelli
Ksenia
Mikaël Ducharme
Daniel Mueller
Marlene Hirose
Andrew Halberstadt