Граф коммитов

2463 Коммитов

Автор SHA1 Сообщение Дата
Anna Scholtz f2b920d085 Add info comment for \b markers 2021-03-25 14:16:06 -07:00
Anna Scholtz b1596a6b35 Consistend indentation of CLI help text 2021-03-25 14:16:06 -07:00
Anna Scholtz e6478806a8 Use jinja macros for recursive bqetl docs generation 2021-03-25 14:16:06 -07:00
Anna Scholtz 1554d045cb Add docs for bqetl routine and mozfun commands 2021-03-25 14:16:06 -07:00
Anna Scholtz 5fd43bcd42 Add examples for bqetl queries, format and dryrun 2021-03-25 14:16:06 -07:00
Anna Scholtz d0dd7e0dce Add options and arguments to bqetl docs 2021-03-25 14:16:06 -07:00
Anna Scholtz de2c7c9945 Generate bqetl command docs from click command help text 2021-03-25 14:16:06 -07:00
Anna Scholtz 0dd748a8ed Add some docs for bqetl commands 2021-03-25 14:16:06 -07:00
Frank Bertsch e3561eb306 Move Forecast tables to looker_derived 2021-03-25 16:35:23 -04:00
Arkadiusz Komarzewski df6db36aad
Bug 1697454 - Shredder: use derived deletion request views as sources when deleting Rally data (#1912)
Co-authored-by: Anthony Miyaguchi <amiyaguchi@mozilla.com>
2021-03-25 18:56:04 +00:00
Jeff Klukas 6a7b0402d0
Cluster desktop usage on attribution and country (#1915)
* Cluster desktop usage on attribution and country

* Materialize country_name
2021-03-25 10:20:14 -04:00
Arkadiusz Komarzewski c50381eccc
adM engagements DAG: adjust schedule to minimize waiting for deduplication (#1911) 2021-03-24 10:15:23 -04:00
Daniel Thorn d1f996503e
Handle job.num_dml_affected_rows is None in shredder delete summary (#1909) 2021-03-23 09:53:33 -07:00
Anna Scholtz 16b17f6891 Extract search counts from histogram in telemetry search metrics view 2021-03-22 12:55:05 -07:00
Anna Scholtz 18ca51fcef Add UDF for testing the experiment search metrics string transformations 2021-03-22 12:55:05 -07:00
Anna Scholtz df80a62715 Use raw strings for search metrics materialized view string magic 2021-03-22 12:55:05 -07:00
Anna Scholtz 66a93c8aad nested JSON regex workaround for experiment search metrics 2021-03-22 12:55:05 -07:00
Anna Scholtz 34dce7a8b7 WIP another attempt with counts in arrays 2021-03-22 12:55:05 -07:00
Anna Scholtz 2f522c1e00 Add clarifications to experiment search aggregate materialized views 2021-03-22 12:55:05 -07:00
Anna Scholtz f87692d3e8 Fix inflated search metrics numbers in experiment_search_aggregates_v1 2021-03-22 12:55:05 -07:00
Anna Scholtz 90afd82edc Fix inflated search metrics numbers in desktop materialized view 2021-03-22 12:55:05 -07:00
Anna Scholtz 81c1b6868f Fix count inflation in materialized search metrics views for Fenix 2021-03-22 12:55:05 -07:00
Daniel Thorn 3495984e41
Handle job.total_bytes_processed is None in shredder delete summary (#1908) 2021-03-22 14:23:57 -04:00
Frank Bertsch bb23a9d041 Rename view 2021-03-22 13:56:47 -04:00
Frank Bertsch a99a97b74b Fix formatting 2021-03-22 13:56:47 -04:00
Frank Bertsch a6d7769525 Rename to looker_forecast_cache
- Add doc strings for certain lines
2021-03-22 13:56:47 -04:00
Frank Bertsch 29b668be09 Add tables for storing and querying forecasts 2021-03-22 13:56:47 -04:00
Anna Scholtz 5ac3f5eee3 Add bigquery metadata to desktop funnel queries 2021-03-19 13:48:43 -07:00
Anna Scholtz fd9c71301a Add view for desktop funnel activation 2021-03-19 13:48:43 -07:00
Anna Scholtz 9db4f79029 Distinct new profiles in desktop funnel 2021-03-19 13:48:43 -07:00
Anna Scholtz 80f67aef0c Simplify desktop funnel queries 2021-03-19 13:48:43 -07:00
Anna Scholtz 5945a44218 Create desktop funnel query schemas 2021-03-19 13:48:43 -07:00
Anna Scholtz cb1d82be5d Schedule queries 2021-03-19 13:48:43 -07:00
Anna Scholtz 597822a792 Fix inflated numbers in desktop funnel queries 2021-03-19 13:48:43 -07:00
William Lachance 5692df54a5 WIP 2021-03-19 13:48:43 -07:00
Jeff Klukas 9b93c3486a
Add country_name to KPI views (#1903)
As discussed in https://github.com/mozilla/looker-spoke-default/pull/7#issuecomment-801891077
2021-03-18 10:14:31 -04:00
Jeff Klukas 59a559b0d2
Add XK for Kosovo to country_codes (#1902)
* Add XK for Kosovo to country_codes

This addresses the missing code discovered in https://github.com/mozilla/looker-spoke-default/pull/7

* Run generation step for good measure
2021-03-18 09:22:05 -04:00
Anthony Miyaguchi 21fdb481e3
Add option to limit max rows during backfill (#1901)
* Add option to limit max rows during backfill

* Run black
2021-03-17 17:36:38 -07:00
Ben Wu 5049ca6551
Add check for ungenerated and untracked dags (#1900) 2021-03-17 16:28:06 -04:00
Arkadiusz Komarzewski ed67471153
Add daily adMarketplace engagements DAG (#1899) 2021-03-17 20:41:18 +01:00
Arkadiusz Komarzewski 3c3b6fe443
Create adMarketplace Daily Engagements table (#1891) 2021-03-17 19:03:54 +01:00
Anthony Miyaguchi 7a30f84e46
Add fixes for emulating ios glean pings (#1888)
* Use _PARTITIONTIME when applicable

* Use view instead of stable table

* Use metadata from the events ping for filling in

* Use count(*) since one document may have many events

* Add copy of v1 for v2

* Make v2 schema a subset of the metrics ping

* Make events counts v2 a subset of metrics

* Make unified view use v2 tables for event count and core

* !fixup format query
2021-03-16 14:36:27 -04:00
Jeff Klukas c79d6d5c9d
Bug 1697602 Remove derived tables for AET (#1894)
* Bug 1697602 Remove derived tables for AET

* Update GRAVEYARD doc
2021-03-16 06:22:43 -07:00
Anthony Miyaguchi fd03e1d96b
Allow schema updates on backfill command (#1896) 2021-03-15 15:38:11 -07:00
Frank Bertsch 75aabead71 Add attributed to fx_usage 2021-03-15 17:03:49 -04:00
Frank Bertsch 4240f31bf4 Ignore failing dryrun 2021-03-15 14:57:01 -04:00
Jeff Klukas 2cab2338e9
Remove fxa_amplitude_email_clicks (#1892)
* Remove fxa_amplitude_email_clicks

Per @cvalaas, the SFMC email system is now retired, so the underlying table
here is no longer receiving updates.

Depends on https://github.com/mozilla/telemetry-airflow/pull/1268

* Add diff-filter

* yamllint hint

* More yamllint fixing
2021-03-15 14:02:20 -04:00
Frank Bertsch 59d8bbc1f2
Materialize cumulative_new_profiles (#1890)
* Materialize cumulative new_profiles

* Share window defn
2021-03-15 09:52:23 -04:00
Anna Scholtz 48ef90f75c Remove old experiment search aggregates views 2021-03-12 14:10:26 -08:00
Anthony Miyaguchi fb09b53b89
LOOK-1: Emulating glean counters using legacy iOS events (#1815)
* Add initial query for legacy mobile event counts

* Optimize query so it runs in < 60 seconds

* Add initial query to convert events to counts

* Add labeled counters for bookmarks

* Add reading-list-item

* Add qr-scanned

* Add gleanified structure

* Update queries for a longer time span

* Add final query that is schema evolution compatible

* Add filter on the events counts

* Move views into org_mozilla_ios_firefox_derived

* Convert event_counts to a query and add parameters

* Add metadata for legacy_mobile_event_counts

* Add metadata to legacy_mobile_events_counts_v1

* Add incremental query for legacy metrics

* Update table location

* Add search counts from the core ping

* Add final few metrics

* Add first pass of a script to generate a view from mps

* Update legacy table through a schema patch

* Add some hacks to get around missing fields in the schema

* Use stable tables instead of views

* Remove workarounds by using stable tables in queries

* Update views on runs

* Use a full join instead of inner join

* Modify generated query to retain nested structure

* Add tests and add documentation

* Get commit hash from dataset label

* Fix issue when two columns differ at ancestor

* Add black formatting

* Add yet another test case for generating view

* Remove extra subquery for labeled counters

* Add submission_timestamp and document_id as valid fields

* Add test for uneven column selection with ancestor

* Ensure there is document id and submission timestamp

* Trim trailing space
2021-03-12 16:41:49 -05:00