64a9f4c76d
Parallelize lookml generation ( #1021 )
...
* use dryrun
* Add dryrun
* Update tests to mock dry run
* Have use_cloud_function as parameter
* Pass dryrun instance
* raise DryRunErrors
* Download files from looker-hub
* Review feedback
* Parallelize lookml generation
* Update datagroup tests
* Update tests for multiprocessing usage
* Use FileSystemLoader for loading templates
* Address review feedback
* Review feedback
* Fix repo init
2024-08-06 09:45:55 -07:00
28b4dccaf1
Use dry run function for generation ( #1003 )
...
* Add --ignore option when generating namespaces
* use dryrun
* Add dryrun
* Update tests to mock dry run
* Have use_cloud_function as parameter
* Pass dryrun instance
* raise DryRunErrors
* Download files from looker-hub
* Review feedback
* Update generator/lookml.py
Co-authored-by: Sean Rose <1994030+sean-rose@users.noreply.github.com>
---------
Co-authored-by: Sean Rose <1994030+sean-rose@users.noreply.github.com>
2024-07-29 09:24:50 -07:00
1357973c9d
Revert "Parallel lookml generation ( #994 )" ( #1000 )
...
This reverts commit 09c4aa9f8e
2024-06-24 09:34:19 -07:00
09c4aa9f8e
Parallel lookml generation ( #994 )
...
* Parallel generation
* Run lookml geneation in parallel
* Parallelism parameter
* Use starmap
2024-06-24 08:10:33 -07:00
b398a842f0
Add --only parameter to generate lookml only for select namespaces ( #992 )
2024-06-11 05:38:49 -07:00
693cf80851
Fix date filtering in metric definition views ( #921 )
2024-03-19 15:43:59 -07:00
244ec963e5
Generate per-datasource explores for metric definitions ( #865 )
...
* Generate per-datasource explores for metric definitions
* Combine clients daily views with metrics data sources
---------
Co-authored-by: Leli <33942105+lelilia@users.noreply.github.com>
2024-02-12 14:55:11 -08:00
10d8577c55
Metric hub integration ( #740 )
...
* Update to Python 3.10
* Generate metric-hub namespaces
* Generate namespaces for metric hub metrics data sources
* Generate metric hub views
* Generate metric hub explores
* Add command line option to set metric hub repo
* Use client_counts views as basis for metric definition explores
2023-09-08 12:09:06 -07:00
f2cfe9e7e5
Generate datagroups.lkml ( #615 )
...
* Moved bigquery view references map to utils for use in multiple places
* Create datagroup generator script
* Added tests
* Formatting: update type reference to include dataset dict
2023-02-21 13:15:50 -05:00
c3db0820ec
Bump lkml from 1.2.0 to 1.3.1 ( #548 )
2022-11-03 11:09:16 -07:00
db7ccdaf4b
Merge pull request #391 from mozilla/opmon-dashboards
...
Opmon dashboards
2022-03-25 11:40:45 -07:00
acc59e256d
File clarification to extend views
...
Co-authored-by: Will Lachance <wlachance@mozilla.com>
2021-12-03 14:41:17 -08:00
4e97a8c5ca
Add comment to generated contents that files are generated
2021-12-03 14:41:17 -08:00
8816f01cff
Support submission_date on x-axis in operational monitoring.
2021-11-29 13:11:13 -05:00
a55120e6c6
Add aggregate tables to speed up operational monitoring dashboards.
2021-11-16 11:29:54 -05:00
98ef40718a
Add auto-generated opmon dashboards.
2021-10-19 13:36:02 -04:00
5a29dde6b9
Remove workaround for joshtemple/lkml#53 ( #237 )
2021-09-20 11:45:22 -07:00
aec929b8c5
Generate descriptions for Glean Ping explores ( fixes #106 ) ( #143 )
...
Format should be `Explore for <x> ping. <x ping description>`. For
example, for the metrics ping, you would see:
```
Explore for the metrics ping. The metrics ping is intended for all of
the metrics that are explicitly set by the application or are included
in the application's metrics.yaml file (except events). The reported
data is tied to the ping's measurement window, which is the time
between the collection of two metrics ping. Ideally, this window is
expected to be about 24 hours, given that the collection is scheduled
daily at 4AM. Data in the ping_info section of the ping can be used to
infer the length of this window.
```
2021-06-02 14:30:49 -07:00
5c69aca3e1
Add view and explore for event names
...
- Have explores return many explores
- Only add filters to first explore
2021-06-01 15:35:11 -04:00
c6fa7f447f
Bug 1707727 - Publish namespaces.yaml to GitHub alongside generated lookml ( #122 )
2021-05-20 14:28:29 -07:00
5ea466ff12
Add clarifying comments; import updated lkml file
2021-05-07 10:35:57 -04:00
224489f5b4
Remove print statement
...
Co-authored-by: William Lachance <wrlach@gmail.com>
2021-05-07 10:35:57 -04:00
1764f41f2d
Updated lkml parser to handle explore query fields
...
Explore query fields have pluralized fields that lkml
expects to unnest into non-plural fields. For example,
"dimensions" is a plural field that, in a view, gets
unnested to each constituent dimension.
However, queries don't work that way; we retain the plural,
and keep it nested. We handle that here.
2021-05-07 10:35:57 -04:00
4d4c5fcdf7
Add Client Count View from clients_daily
2021-05-07 10:35:57 -04:00
224f895cf9
Remove print statement
2021-05-06 21:03:49 -04:00
db8ee74499
Make Glean data reliable for Fenix
...
- Show metrics without categories under Glean
- Add Fenix to allowlist
- Ignore duplicate metrics
2021-05-06 21:03:49 -04:00
2edcc420a1
WIP: Put glean metrics in categories
2021-05-06 21:03:49 -04:00
e23d805ea1
Fix links with namespace filled-in
2021-05-05 13:33:50 -04:00
717ff58f5d
Use consistent method to gather views
...
- Remove runner from lookml tests
- Rebase changes; apply to TableView
2021-05-04 21:39:49 -04:00
b6a99d0278
Fix
2021-05-04 21:39:49 -04:00
bd62567a78
Ignore all the things
2021-05-04 21:39:49 -04:00
21068ff32a
Hopefully fix type issue
2021-05-04 21:39:49 -04:00
ef8f9172b4
More CI fixes
2021-05-04 21:39:49 -04:00
bc01cd93db
updates
2021-05-04 21:39:49 -04:00
4a54afecd7
Generate glean dictionary links for metric columns in pings ( fixes #64 )
...
This also adds Glean Ping Views and Explores, which fixes #67 .
2021-05-04 21:39:49 -04:00
9b1ee49196
Use uppercase for view and explore types constants ( #73 )
2021-04-30 12:31:29 -04:00
659ae2d155
Require date and channel filters in ping explores
2021-04-29 15:02:14 -04:00
3154d13dc9
Generate LookML For growth_accounting
2021-04-19 17:04:54 -04:00
8bc8be6b32
Generate LookML inside View classes
2021-04-19 17:04:54 -04:00
be0d398121
WIP: Use classes in lookml generation
2021-04-19 17:04:54 -04:00
daba1d38e4
Change view defns for tests and code
2021-04-15 21:43:32 -04:00
55608b4297
Dont try and create explores for ga
2021-04-15 21:43:32 -04:00
5a702a7d38
Generate views via lkml ( #26 )
2021-03-23 09:27:28 -07:00
25ed1c724c
Generate explores for Ping Tables
...
- Use lkml for lookml generation / comparison
- Add some logging about generation
2021-03-22 17:19:15 -04:00
8d62e03705
Parameterize sql_table_name with channel ( #16 )
2021-03-16 13:38:06 -07:00
199a8640c2
Fail lookml generation for duplicate dimensions and measures ( #17 )
2021-03-16 13:03:46 -07:00
d5664ac40e
Add cli command to generate view lookml from namespaces.yaml ( #12 )
2021-03-15 14:49:32 -07:00
Anna Scholtz
Alexander
dependabot[bot]
Marina Samuel
Daniel Thorn
William Lachance
Frank Bertsch
William Lachance