mozilla
/
opmon
Π·Π΅Ρ€ΠΊΠ°Π»ΠΎ ΠΈΠ· https://github.com/mozilla/opmon.git
1
0
Π€ΠΎΡ€ΠΊΠ½ΡƒΡ‚ΡŒ 0
Код Π—Π°Π΄Π°Ρ‡ΠΈ ΠŸΠ°ΠΊΠ΅Ρ‚Ρ‹ ΠŸΡ€ΠΎΠ΅ΠΊΡ‚Ρ‹ Π Π΅Π»ΠΈΠ·Ρ‹ Π’ΠΈΠΊΠΈ ΠΠΊΡ‚ΠΈΠ²Π½ΠΎΡΡ‚ΡŒ
153 ΠΊΠΎΠΌΠΌΠΈΡ‚ΠΎΠ² 18 Π’Π΅Ρ‚ΠΊΠΈ 0 Π’Π΅Π³ΠΈ 2,6 MiB
Π“Ρ€Π°Ρ„ ΠΊΠΎΠΌΠΌΠΈΡ‚ΠΎΠ²

153 ΠšΠΎΠΌΠΌΠΈΡ‚ΠΎΠ²

Автор SHA1 Π‘ΠΎΠΎΠ±Ρ‰Π΅Π½ΠΈΠ΅ Π”Π°Ρ‚Π°
Nick Alexander 54a8809ef7 Add submission_timestamp column sibling to submission_date. (#126) 
This makes it easier to visualize Opmon aggregate metrics in Grafana:
Grafana's Bigquery integration is fundamentally `TIMESTAMP` shaped,
not `DATE` shaped, and this avoids writing nested queries.
 2023-03-10 10:55:02 -08:00
Anna Scholtz 895e0d9033 Fix test_spec_for_experiment 2023-03-10 10:50:25 -08:00
Anna Scholtz b7908ddeed SQL introspection for multipart queries 2023-02-28 11:42:27 -08:00
Nick Alexander 049274eae3 Optionally write generated SQL for inspection during development. (#115) 
There are many ways to skin this cat; here's one.  We add a somewhat
general "before execution" callback that the BigQuery client itself
invokes before queuing jobs for processing.  This callback is a member
of the `Monitoring` instance and is threaded through in turn to its
BigQuery client.  In addition, we manually invoke the callback around
the dry run service.

The callback takes arbitrary annotations (in a dictionary) and takes
action as appropriate.  For now, we annotate with the slug, submission
date, and a "type" describing the query.

A new optional `--sql-output-dir` command line option determines
whether output is written to disk and, when present, to what
directory.  Each SQL query is written to a separate file whose name is
based on the provided annotations.

This callback mechanism could be generalized to, for example, be
invoked "after exection" as well; this would allow collecting BigQuery
timing and job details (data processed or estimates there-of, for
example).

This commit will require an additional commit to the `opmon-config`
repository to actually make use of the new feature.
 2023-02-28 11:42:27 -08:00
Nick Alexander ae7001e382 [preview] Fix date timestamp to midnight, like "YYYY-MM-DD 00:00:00+00:00". 
This is simply a quality of life improvement: it makes it easier to
reason about various timestamps, most of which are treated as date
fields anyway.
 2023-02-28 11:42:27 -08:00
Nick Alexander 12a697ee44 [preview] Fish slug from `--config-file` by default. 
The slug does not appear explicitly in (some) configuration files;
this eases invocation.
 2023-02-28 11:42:27 -08:00
Nick Alexander 2360578bd9 [preview] Add `preview --num-days=N` to extend preview time window. 
In the future, we might teach the preview window computation to
consider `window_size` for relevant alerts such as those with `type =
"avg_diff"`.
 2023-02-28 11:42:27 -08:00
Daniel Thorn 45f1fde118 Remove pytest-flake8 
it hasn't been updated to work with flake8>=5 and this project uses flake8 6
 2023-02-27 08:29:22 -08:00
Anna Scholtz 84e49040a3 Remove unused UDFs 2023-02-13 15:15:31 -08:00
Anna Scholtz 7785257019 Use new percentile UDF 2023-02-13 15:13:06 -08:00
Anna Scholtz 73148e3d6e Update metric parser 2023-01-24 15:57:40 -08:00
Anna Scholtz 7857ebabd6 Update metrics parser 2023-01-24 15:34:18 -08:00
Anna Scholtz 64eaf95515 Load configs from metric-hub 2023-01-23 15:44:34 -08:00
Daniel Thorn 3291261a49
Automatically split up metrics queries based on probe quantity (#104) 2023-01-17 09:36:23 -08:00
Anna Scholtz 9d1308b927 Only create alerts view if alerts table exists 2023-01-11 09:04:13 -08:00
Anna Scholtz 31d608edc0 destination_table parameter name 2023-01-11 09:04:13 -08:00
Daniel Thorn 6a6a5abf25
Refactor table generation to repeat less logic (#117) 2023-01-10 14:17:44 -08:00
Anna Scholtz 2987b4cb34 Pass derived_dataset into Metadata 2022-12-21 09:33:59 -08:00
Anna Scholtz dc21e3b1c4 Fix derived_dataset_id option 2022-12-21 09:03:54 -08:00
Anna Scholtz ce585a46f6 Preview tweaks 2022-12-20 10:26:57 -08:00
Anna Scholtz fa3f033703 Remove previously created preview tables 2022-12-20 09:57:49 -08:00
Anna Scholtz 8bee4ae301 opmon preview CLI command 2022-12-20 09:57:49 -08:00
Daniel Thorn 84865269ee
Fix population join to prevent duplication of input rows (#102) 2022-11-21 14:40:18 -08:00
Anna Scholtz df6dcb370b ETL docs 2022-11-10 13:47:25 -08:00
Anna Scholtz 0851a16347 Support multiple Glean channels 2022-11-09 13:40:14 -08:00
Anna Scholtz 3d16e5d2c1 Correct client_id_column referenced in query 2022-11-03 13:29:42 -07:00
Anna Scholtz 3138195825 Add option to percentiles to remove nulls 2022-11-02 19:09:34 -07:00
Anna Scholtz 288c807bd4 Add total ratio statistic 2022-11-02 11:49:33 -07:00
Anna Scholtz 7b58c7fcf1 Update metric parser version 2022-11-01 09:55:22 -07:00
Anna Scholtz 268d3d0034 Support metric groups 2022-10-31 10:42:30 -07:00
Anna Scholtz af48baddc5 Enable Glean apps 2022-10-25 15:08:21 -07:00
Anna Scholtz 3bf6a710b3 Fix type check for non-rollout run configs 2022-10-25 12:50:41 -07:00
Anna Scholtz 6e9e189483 Validation of definitions files 2022-10-19 13:57:53 -07:00
Anna Scholtz 2919589b0f Fix validation for definition configs 2022-10-19 11:55:13 -07:00
Anna Scholtz a45df5b20b Integrate metric hub 2022-10-19 11:47:45 -07:00
Anna Scholtz e943188df8 Fix alerts dummy value 2022-10-13 13:22:31 -07:00
Anna Scholtz f4b9cdc974 Fix build-id type in alerts 2022-10-13 11:37:54 -07:00
dependabot[bot] 584ff1869f Bump protobuf from 3.19.4 to 3.19.5 
Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 3.19.4 to 3.19.5.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v3.19.4...v3.19.5)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-10-13 09:51:47 -07:00
Anna Scholtz a5aca2b9fa Use metric_config_parser 2022-10-13 09:46:11 -07:00
Anna Scholtz f9ca0b13a4 Use new parser lib 2022-10-13 09:40:52 -07:00
Anna Scholtz 3aff027b70 Bump version 2022-10-13 09:01:34 -07:00
Anna Scholtz dca959be6c Remove python-version 2022-10-12 14:37:47 -07:00
Anna Scholtz e9966bc719 Support parser in CLI 2022-10-12 14:31:04 -07:00
Anna Scholtz 1643e7acf4 Update tests to use config parser library 2022-10-12 14:31:04 -07:00
Anna Scholtz 14745d4ab3 Use jetstream-config-parser 2022-10-12 14:31:04 -07:00
Anna Scholtz 4e7f3c6d6d Add support for jetstream parser library 2022-10-12 14:31:04 -07:00
Anna Scholtz 4bfe0c3d1f Fix schema version in validation 2022-10-07 13:21:32 -07:00
Anna Scholtz 3a08a27e57 Add support for Glean 2022-10-04 10:27:37 -07:00
Anna Scholtz 915f59d060 Fix historical alerts for buildid aggs 2022-09-23 11:31:43 -07:00
Anna Scholtz 9187947012 Fix statistic view when looking over build ID period 2022-09-22 13:12:58 -07:00