diff --git a/.dictionary b/.dictionary index d809731cc..305943a68 100644 --- a/.dictionary +++ b/.dictionary @@ -1,7 +1,6 @@ -personal_ws-1.1 en 178 utf-8 +personal_ws-1.1 en 184 utf-8 AAR AARs -anonymized APIs APK BUGFIX @@ -14,6 +13,7 @@ Cartfile CircleCI Ciufo DNS +DOM Datetime Datetimes Fenix @@ -52,6 +52,8 @@ Protobufs PyPI Redash SDK +SDK's +TLDs TODO TWiG Taskcluster @@ -66,6 +68,7 @@ Xcode YAML alphanumerics analytics +anonymized aspell async autodetected @@ -137,6 +140,7 @@ mypy namespace ns ol +ons pdoc pidcat pipenv @@ -175,6 +179,7 @@ und unhandled uploader vendored +webpages xcpretty yaml zsh diff --git a/docs/README.md b/docs/README.md index a37ff0ec0..a45b77a35 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,7 +14,9 @@ The source code is available [on GitHub](https://github.com/mozilla/glean/). ## Using this book -This book is divided into 4 main chapters: +This book is specifically about the `Glean SDK` (the client side code for collecting telemetry). Documentation about the broader end-to-end [Glean project](https://docs.telemetry.mozilla.org/concepts/glean/glean.html). + +This book is divided into 5 main chapters: ### [Using the Glean SDK](user/index.html) @@ -32,6 +34,10 @@ This chapter lists all metrics collected by the Glean SDK itself. This chapter describes how to develop the Glean SDK and its various implementations. This is relevant if you plan to contribute to the Glean SDK code base. +### [API Reference Documentation](api/index.md) + +Reference documentation for the API in its various language bindings. + ### [This Week in Glean](appendix/twig.md) “This Week in Glean” is a series of blog posts that the Glean Team at Mozilla is using to try to communicate better about our work. diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 2eac95c72..05c06e2f8 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -41,7 +41,7 @@ - [Testing custom pings](user/pings/testing-custom-pings.md) - [Android build configuration options](user/android-build-configuration-options.md) - [Focused Use Cases](user/focused-use-cases.md) - - [Instrumenting Android Crashes With Glean](user/instrument-android-crashes-example.md) + - [Instrumenting Android crashes with the Glean SDK](user/instrument-android-crashes-example.md) - [Metrics collected by the Glean SDK](user/collected-metrics/metrics.md) - [Developing the Glean SDK](dev/index.md) - [Testing](dev/testing.md) diff --git a/docs/user/adding-glean-to-your-project.md b/docs/user/adding-glean-to-your-project.md index a458cb662..59af07596 100644 --- a/docs/user/adding-glean-to-your-project.md +++ b/docs/user/adding-glean-to-your-project.md @@ -2,9 +2,9 @@ ## Before using Glean -Products using the Glean SDK to collect telemetry **must**: +Products (applications or libraries) using the Glean SDK to collect telemetry **must**: -- add documentation for any new metric collected with the library in its repository (see [an example](pings/index.md)); +- add documentation for any new metrics collected to its repository (see [an example](collected-metrics/metrics.md)); - include the markdown-formatted documentation generated from the `metrics.yaml` and `pings.yaml` files in the project's documentation; - go through data review for the newly collected data by following [this process](https://wiki.mozilla.org/Firefox/Data_Collection); - provide a way for users to turn data collection off (e.g. providing settings to control @@ -20,9 +20,9 @@ Products using the Glean SDK to collect telemetry **must**: #### Setting up the dependency -Glean is published on [maven.mozilla.org](https://maven.mozilla.org/). +The Glean SDK is published on [maven.mozilla.org](https://maven.mozilla.org/). To use it, you need to add the following to your project's top-level build file, -in the `allprojects` block (see e.g. [Glean's own `build.gradle`](https://github.com/mozilla/glean/blob/main/build.gradle)): +in the `allprojects` block (see e.g. [Glean SDK's own `build.gradle`](https://github.com/mozilla/glean/blob/main/build.gradle)): ```Groovy repositories { @@ -32,7 +32,7 @@ repositories { } ``` -Each module that uses Glean needs to specify it in its build file, in the `dependencies` block. +Each module that uses Glean SDK needs to specify it in its build file, in the `dependencies` block. Add this to your Gradle configuration: ```Groovy @@ -57,12 +57,12 @@ implementation "org.mozilla.components:service-glean:33.0.0" #### Requirements -* Python >= 3.6 +* Python >= 3.6. #### Setting up the dependency -Glean can be consumed through [Carthage](https://github.com/Carthage/Carthage), a dependency manager for macOS and iOS. -For consuming the latest version of Glean, add the following line to your `Cartfile`: +The Glean SDK can be consumed through [Carthage](https://github.com/Carthage/Carthage), a dependency manager for macOS and iOS. +For consuming the latest version of the Glean SDK, add the following line to your `Cartfile`: ``` github "mozilla/glean" "{latest-version}" @@ -108,17 +108,17 @@ For integration with the build system you can follow the [Carthage Quick Start s We recommend using a virtual environment for your work to isolate the dependencies for your project. There are many popular abstractions on top of virtual environments in the Python ecosystem which can help manage your project dependencies. -The Python Glean bindings currently have [prebuilt wheels on PyPI for x86_64 Windows, Linux and macOS](https://pypi.org/project/glean-sdk/#files). +The Glean SDK Python bindings currently have [prebuilt wheels on PyPI for Windows (i686 and x86_64), Linux (x86_64) and macOS (x86_64)](https://pypi.org/project/glean-sdk/#files). -If you're running one of those platforms and have your virtual environment set up and activated, you can install Glean into it using: +If you're running one of those platforms and have your virtual environment set up and activated, you can install the Glean SDK into it using: ```bash $ python -m pip install glean_sdk ``` -If you are not on one of these platforms, you will need to build the Glean Python bindings from source using [these instructions](../dev/python/setting-up-python-build-environment.html). +If you are not on one of these platforms, you will need to build the Glean SDK Python bindings from source using [these instructions](../dev/python/setting-up-python-build-environment.html). -The Python Glean bindings make extensive use of type annotations to catch type related errors at build time. We highly recommend adding [mypy](https://mypy-lang.org) to your continuous integration workflow to catch errors related to type mismatches early. +The Glean SDK Python bindings make extensive use of type annotations to catch type related errors at build time. We highly recommend adding [mypy](https://mypy-lang.org) to your continuous integration workflow to catch errors related to type mismatches early. @@ -134,8 +134,8 @@ TODO. To be implemented in [bug 1643568](https://bugzilla.mozilla.org/show_bug.c All metrics that your project collects must be defined in a `metrics.yaml` file. -The format of that file is documented [with `glean_parser`](https://mozilla.github.io/glean_parser/metrics-yaml.html). To learn more, see [adding new metrics](adding-new-metrics.md). +See the [metric parameters](metric-parameters.md) documentation which provides reference information about the contents of that file. > **Important**: as stated [before](adding-glean-to-your-project.md#before-using-glean), any new data collection requires documentation and data-review. > This is also required for any new metric automatically collected by the Glean SDK. @@ -271,9 +271,9 @@ metrics = load_metrics(resource_filename(__name__, "metrics.yaml")) metrics.your_category.your_metric.set("value") ``` -The documentation for your application or library's metrics and pings are written in `metrics.yaml` and `pings.yaml`. However, you should also provide human-readable markdown files based on this information, and this is a requirement for Mozilla projects using Glean. For other languages and platforms, this transformation is done automatically as part of the build. However, for Python the integration to automatically generate docs is an additional step. +The documentation for your application or library's metrics and pings are written in `metrics.yaml` and `pings.yaml`. However, you should also provide human-readable markdown files based on this information, and this is a requirement for Mozilla projects using the Glean SDK. For other languages and platforms, this transformation is done automatically as part of the build. However, for Python the integration to automatically generate docs is an additional step. -Glean provides a commandline tool for automatically generating markdown documentation from your `metrics.yaml` and `pings.yaml` files. To perform that translation, run `glean_parser`'s `translate` command: +The Glean SDK provides a commandline tool for automatically generating markdown documentation from your `metrics.yaml` and `pings.yaml` files. To perform that translation, run `glean_parser`'s `translate` command: ```sh python3 -m glean_parser translate -f markdown -o docs metrics.yaml pings.yaml @@ -306,23 +306,23 @@ Please refer to the [custom pings documentation](pings/custom.md). ### Parallelism -All of Glean's target languages use a separate worker thread to do most of Glean's work, including any I/O. This thread is fully managed by Glean as an implementation detail. Therefore, users should be free to use the Glean API wherever it is most convenient, without worrying about the performance impact of updating metrics and sending pings. +All of the Glean SDK's target languages use a separate worker thread to do most of its work, including any I/O. This thread is fully managed by the Glean SDK as an implementation detail. Therefore, users should feel free to use the Glean SDK wherever it is most convenient, without worrying about the performance impact of updating metrics and sending pings. {{#include ../tab_header.md}}
-Since Glean performs disk and networking I/O, it tries to do as much of its work as possible on separate threads and processes. +Since the Glean SDK performs disk and networking I/O, it tries to do as much of its work as possible on separate threads and processes. Since there are complex trade-offs and corner cases to support Python parallelism, it is hard to design a one-size-fits-all approach. #### Default behavior -When using the Python bindings, most of Glean's work is done on a separate thread, managed by Glean itself. -Glean releases the Global Interpreter Lock (GIL), therefore your application's threads should not be in contention with Glean's thread. +When using the Python bindings, most of the Glean SDK's work is done on a separate thread, managed by the Glean SDK itself. +The Glean SDK releases the Global Interpreter Lock (GIL) for most of its operations, therefore your application's threads should not be in contention with the Glean SDK's worker thread. -Glean installs an [`atexit` handler](https://docs.python.org/3/library/atexit.html) so the Glean thread can cleanly finish when your application exits. +The Glean SDK installs an [`atexit` handler](https://docs.python.org/3/library/atexit.html) so that its worker thread can cleanly finish when your application exits. This handler will wait up to 30 seconds for any pending work to complete. -In addition, by default ping uploading is performed in a separate child process. This process will continue to upload any pending pings even after the main process shuts down. This is important for commandline tools where you want to return control to the shell as soon as possible and not be delayed by network connectivity. +By default, ping uploading is performed in a separate child process. This process will continue to upload any pending pings even after the main process shuts down. This is important for commandline tools where you want to return control to the shell as soon as possible and not be delayed by network connectivity. #### Cases where subprocesses aren't possible @@ -331,8 +331,8 @@ The default approach may not work with applications built using [`PyInstaller`]( #### Using the `multiprocessing` module Additionally, the default approach does not work if your application uses the `multiprocessing` module for parallelism. -Glean can not wait to finish its work in a `multiprocessing` subprocess, since `atexit` handlers are not supported in that context. -Therefore, if Glean detects that it is running in a `multiprocessing` subprocess, all of its work that would normally run on a worker thread will run on the main thread. +The Glean SDK can not wait to finish its work in a `multiprocessing` subprocess, since `atexit` handlers are not supported in that context. +Therefore, if the Glean SDK detects that it is running in a `multiprocessing` subprocess, all of its work that would normally run on a worker thread will run on the main thread. In practice, this should not be a performance issue: since the work is already in a subprocess, it will not block the main process of your application.
diff --git a/docs/user/adding-new-metrics.md b/docs/user/adding-new-metrics.md index 83b9f33c5..81ce4519c 100644 --- a/docs/user/adding-new-metrics.md +++ b/docs/user/adding-new-metrics.md @@ -6,8 +6,7 @@ When adding a new metric, the workflow is: * Add a new entry to [`metrics.yaml`](#The-metricsyaml-file). * Add code to your project to record into the metric by calling the Glean SDK. -> **Important**: Any new data collection requires documentation and data-review. -This is also required for any new metric automatically collected by the Glean SDK. +> **Important**: Any new data collection requires documentation and [data-review](https://wiki.mozilla.org/Firefox/Data_Collection). This is also required for any new metric automatically collected by the Glean SDK. ## Choosing a metric type @@ -17,7 +16,9 @@ The following is a set of questions to ask about the data being collected to hel If the value is true or false, use a [boolean metric](metrics/boolean.html). -If the value is a string, use a [string metric](metrics/string.html). For example, to record the name of the default search engine. Beware: string metrics are exceedingly general, and you are probably best served by selecting the most specific metric for the job, since you'll get better error checking and richer analysis tools for free. For example, avoid storing a number in a string metric --- you probably want a [counter metric](metrics/counter.html) instead. +If the value is a string, use a [string metric](metrics/string.html). For example, to record the name of the default search engine. + +> **Beware:** string metrics are exceedingly general, and you are probably best served by selecting the most specific metric for the job, since you'll get better error checking and richer analysis tools for free. For example, avoid storing a number in a string metric --- you probably want a [counter metric](metrics/counter.html) instead. If you need to store multiple string values in a metric, use a [string list metric](metrics/string_list.html). For example, you may want to record the list of other Mozilla products installed on the device. @@ -25,7 +26,7 @@ If you need to store multiple string values in a metric, use a [string list metr -For all of the simple metric types in this section that measure single values, it is especially important to consider how the lifetime of the value relates to the ping it is being sent in. Since these metrics don't perform any aggregation on the client side, when a ping containing the metric is submitted, it will contain only the "last known" value for the metric, potentially resulting in **data loss**. There is further discussion of [metric lifetimes](#When-should-Glean-automatically-reset-the-measurement) below. +For all of the metric types in this section that measure single values, it is especially important to consider how the lifetime of the value relates to the ping it is being sent in. Since these metrics don't perform any aggregation on the client side, when a ping containing the metric is submitted, it will contain only the "last known" value for the metric, potentially resulting in **data loss**. There is further discussion of [metric lifetimes](#When-should-Glean-automatically-reset-the-measurement) below. ### Are you counting things? @@ -37,7 +38,7 @@ If you need to know when the things being counted happened relative to other thi If you need to record an absolute time, use a [datetime metric](metrics/datetime.html). Datetimes are recorded in the user's local time, according to their device's real time clock, along with a timezone offset from UTC. Datetime metrics allow specifying the resolution they are collected at, and to stay [lean][lean-data], they should only be collected at the minimum resolution required to answer your question. -If you need to record how long something takes you have a few options. +If you need to record how long something takes you have a few options. If you need to measure the total time spent doing a particular task, look to the [timespan metric](metrics/timespan.html). Timespan metrics allow specifying the resolution they are collected at, and to stay [lean][lean-data], they should only be collected at the minimum resolution required to answer your question. Note that this metric should only be used to measure time on a single thread. If multiple overlapping timespans are measured for the same metric, an invalid state error is recorded. @@ -50,7 +51,7 @@ If you need to know the time between multiple distinct actions that aren't a sim If you need to know the order of actions relative to other actions, such as, the user performed tasks A, B, and then C, and this is meaningfully different from the user performing tasks A, C and then B, (in other words, the order is meaningful beyond just the *fact* that a set of tasks were performed), use an [event metric](metrics/event.html). -Importantly, events are the most expensive metric type to record, transmit, store and analyze, so they should be used sparingly, and only when none of the other metric types are sufficient for answering your question. +> **Important:** events are the most expensive metric type to record, transmit, store and analyze, so they should be used sparingly, and only when none of the other metric types are sufficient for answering your question. ### For how long do you need to collect this data? @@ -65,7 +66,7 @@ Removing a metric does not affect the availability of data already collected by If the metric is still needed after its expiration date, it should go back for [another round of data review](https://wiki.mozilla.org/Firefox/Data_Collection) to have its expiration date extended. -### When should Glean automatically clear the measurement? +### When should the Glean SDK automatically clear the measurement? The `lifetime` parameter of a metric defines when its value will be cleared. There are three lifetime options available: @@ -75,7 +76,7 @@ While lifetimes are important to understand for all metric types, they are parti #### A lifetime example -Let's work through an example to see how these lifetimes play out in practice. Let's suppose we have a user preference, "turbo mode", which defaults to `false`, but the user can turn it to `true` at any time. We want to know when this flag is `true` so we can measure its affect on other metrics in the same ping. In the following diagram, we look at a time period that sends 4 pings across two separate runs of the application. We assume here, that like Glean's built-in [metrics ping](pings/metrics.html), the developer writing the metric isn't in control of when the ping is submitted. +Let's work through an example to see how these lifetimes play out in practice. Let's suppose we have a user preference, "turbo mode", which defaults to `false`, but the user can turn it to `true` at any time. We want to know when this flag is `true` so we can measure its affect on other metrics in the same ping. In the following diagram, we look at a time period that sends 4 pings across two separate runs of the application. We assume here, that like the Glean SDK's built-in [metrics ping](pings/metrics.html), the developer writing the metric isn't in control of when the ping is submitted. In this diagram, the ping measurement windows are represented as rectangles, but the moment the ping is "submitted" is represented by its right edge. The user changes the "turbo mode" setting from `false` to `true` in the first run, and then toggles it again twice in the second run. @@ -105,12 +106,12 @@ This is especially useful when metrics need to be tightly related to one another #### Reuse names from other applications -There's a lot of value using the same name for analogous metrics collected across different products. For example, BigQuery makes it simple to join columns with the same name across multiple tables. Therefore, we encourage you to investigate if a similar metric is already being collected by another product. If it is, there may be an opportunity for code reuse across these products, and if all the projects are using Glean, it's easy for libraries to send their own metrics. If sharing the code doesn't make sense, at a minimum we recommend using the same metric name for similar actions and concepts whenever possible. +There's a lot of value using the same name for analogous metrics collected across different products. For example, BigQuery makes it simple to join columns with the same name across multiple tables. Therefore, we encourage you to investigate if a similar metric is already being collected by another product. If it is, there may be an opportunity for code reuse across these products, and if all the projects are using the Glean SDK, it's easy for libraries to send their own metrics. If sharing the code doesn't make sense, at a minimum we recommend using the same metric name for similar actions and concepts whenever possible. #### Make names unique within an application Metric identifiers (the combination of a metric's category and name) must be unique across all metrics that are sent by a single application. -This includes not only the metrics defined in the app's `metrics.yaml`, but the `metrics.yaml` of any Glean-using library that the application uses, including Glean itself. +This includes not only the metrics defined in the app's `metrics.yaml`, but the `metrics.yaml` of any Glean SDK-using library that the application uses, including the Glean SDK itself. Therefore, care should be taken to name things specifically enough so as to avoid namespace collisions. In practice, this generally involves thinking carefully about the `category` of the metric, more than the `name`. @@ -126,13 +127,13 @@ For example, if defining a set of events related to search, put them in a catego ### What if none of these metric types is the right fit? -The current set of metrics Glean supports is based on known common use cases, but new use cases are discovered all the time. +The current set of metrics the Glean SDK supports is based on known common use cases, but new use cases are discovered all the time. Please reach out to us on [#glean:mozilla.org](https://chat.mozilla.org/#/room/#glean:mozilla.org). If you think you need a new metric type, we [have a process for that](metrics/index.html#adding-or-changing-metric-types). ### How do I make sure my metric is working? -Glean has rich support for writing unit tests involving metrics. Writing a good unit test is a large topic, but in general, you should write unit tests for all new telemetry that does the following: +The Glean SDK has rich support for writing unit tests involving metrics. Writing a good unit test is a large topic, but in general, you should write unit tests for all new telemetry that does the following: - Performs the operation being measured. diff --git a/docs/user/android-build-configuration-options.md b/docs/user/android-build-configuration-options.md index 586a9f6ec..d19b1502c 100644 --- a/docs/user/android-build-configuration-options.md +++ b/docs/user/android-build-configuration-options.md @@ -1,13 +1,13 @@ # Android build script configuration options -This chapter describes build configuration options that control the behavior of Glean's Gradle plugin. +This chapter describes build configuration options that control the behavior of the Glean SDK's Gradle plugin. These options are not usually required for normal use. Options can be turned on by setting a variable on the Gradle [`ext`](https://docs.gradle.org/current/dsl/org.gradle.api.plugins.ExtraPropertiesExtension.html) object *before* applying the Glean Gradle plugin. ## `allowMetricsFromAAR` -Normally, Glean looks for `metrics.yaml` and `pings.yaml` files in the root directory of the Glean-using project. +Normally, the Glean SDK looks for `metrics.yaml` and `pings.yaml` files in the root directory of the Glean-using project. However, in some cases, these files may need to ship inside the dependencies of the project. For example, this is used in the `engine-gecko` component to grab the `metrics.yaml` from the `geckoview` AAR. diff --git a/docs/user/debugging/android.md b/docs/user/debugging/android.md index 8196af805..22c9027f6 100644 --- a/docs/user/debugging/android.md +++ b/docs/user/debugging/android.md @@ -59,9 +59,9 @@ persist until the application is closed or manually reset. > --es debugViewTag test-metrics-ping > ``` -### Glean Log messages +### Glean SDK Log messages -When running a Glean-powered app in the Android emulator or on a device connected to your computer via cable, there are several ways to read the log output. +When running a Glean SDK-powered app in the Android emulator or on a device connected to your computer via cable, there are several ways to read the log output. #### Android Studio diff --git a/docs/user/debugging/index.md b/docs/user/debugging/index.md index 295ff1e1a..60c8e8e74 100644 --- a/docs/user/debugging/index.md +++ b/docs/user/debugging/index.md @@ -10,7 +10,7 @@ ### Available commands and query format -There are 4 available commands that you can use with the Glean SDK debug tools +There are 4 available commands that you can use with the Glean SDK debug tools: - `logPings`: This is either true or false and will cause pings that are submitted to also be echoed to the device's log. - `debugViewTag`: This command will tag outgoing pings with the provided value, in order to identify them in the Glean Debug View. Tags need to be string with upper and lower case letters, numbers and dashes, with a max length of 20 characters. @@ -45,9 +45,9 @@ Enabling debugging features using environment variables is available for all sup 2. You can tag _and_ send pings using the debug tools. This has the side effect of potentially sending a ping which does not include all fields because `sendPing` triggers pings to be sent before certain application behaviors can occur which would record that information. For example, `duration` is not calculated or included in a baseline ping sent with `sendPing` because it forces the ping to be sent before the `duration` metric has been recorded. Keep in mind that there may be nothing to send, in which case no ping is generated. 3. You can trigger a command while the instrumented application is still running. This is useful for toggling commands or for triggering pings that have schedules that are difficult to trigger manually. This is especially useful if you need to trigger a ping submission after some activity within the application, such as with the metrics ping. -### Glean Log messages +### Glean SDK Log messages -Glean logs warnings and errors through the platform-specific logging frameworks. See the platform-specific instructions for information on how to view the logs. +The Glean SDK logs warnings and errors through the platform-specific logging frameworks. See the platform-specific instructions for information on how to view the logs. ### Implementation details diff --git a/docs/user/debugging/ios.md b/docs/user/debugging/ios.md index 84a6a2ef4..640f54799 100644 --- a/docs/user/debugging/ios.md +++ b/docs/user/debugging/ios.md @@ -10,7 +10,7 @@ These environment variables must be set on the device that is running the applic # Debugging iOS applications using the Glean SDK -For debugging and validation purposes on iOS, Glean makes use of a custom URL scheme which is implemented _within the application_ that is consuming Glean. Glean provides some convenience functions to facilitate this, but it's up to the consuming application to enable this functionality. Applications that enable this Glean SDK feature will be able to launch the application from a URL with the Glean debug commands embedded in the URL itself. +For debugging and validation purposes on iOS, the Glean SDK makes use of a custom URL scheme which is implemented _within the application_ that is consuming the Glean SDK. The Glean SDK provides some convenience functions to facilitate this, but it's up to the consuming application to enable this functionality. Applications that enable this Glean SDK feature will be able to launch the application from a URL with the Glean debug commands embedded in the URL itself. ### Available commands and query format @@ -40,7 +40,7 @@ There are a few things to consider when creating the custom URL: - The `logPings` command doesn't trigger ping submission and you won't see any output until a ping has been submitted. You can use the `sendPings` command to force a ping to be sent, but it could be more desirable to trigger the pings submission on their normal schedule. For instance, the `baseline` and `events` pings can be triggered by moving the app out of the foreground and the `metrics` ping can be triggered normally if it is overdue for the current calendar day. See the [ping documentation](../pings/index.md) for more information on ping scheduling to learn when pings are sent. - Enabling debugging features through custom URLs overrides any debugging features set through environment variables. -### Instrumenting the application for Glean debug functionality +### Instrumenting the application for Glean SDK debug functionality In order to enable the debugging features in a Glean SDK consuming iOS application, it is necessary to add some information to the application's `Info.plist`, and add a line and possibly an override for a function in the `AppDelegate.swift`. diff --git a/docs/user/experiments-api.md b/docs/user/experiments-api.md index 184b27401..00afaec30 100644 --- a/docs/user/experiments-api.md +++ b/docs/user/experiments-api.md @@ -25,8 +25,8 @@ Glean.setExperimentActive( Glean.setExperimentInactive("blue-button-effective") ``` -> **Important**: Experiment IDs and branch don't need to be pre-defined in the Glean SDK registry files. -Please also note that the `extra` map is non-nested arbitrary `String` to `String` map. +> **Important**: Experiment IDs and branches don't need to be pre-defined in the Glean SDK registry files. +Please also note that the `extra` map is a non-nested arbitrary `String` to `String` map. It also has limits on the size of the keys and values defined below. There are test APIs available too: diff --git a/docs/user/general-api.md b/docs/user/general-api.md index 690a50811..fd1fa772d 100644 --- a/docs/user/general-api.md +++ b/docs/user/general-api.md @@ -1,6 +1,6 @@ # The General API -Glean has a minimal API available on its top-level `Glean` object. +The Glean SDK has a minimal API available on its top-level `Glean` object. This API allows one to enable and disable upload, register [custom pings][custom-pings] and set [experiment data][experiments-api]. [custom-pings]: pings/custom.md @@ -8,7 +8,7 @@ This API allows one to enable and disable upload, register [custom pings][custom > **Important:** The Glean SDK should only be initialized from the main application, not individual libraries. -If you are adding Glean support to a library, you can safely skip this section. +If you are adding Glean SDK support to a library, you can safely skip this section. ## The API @@ -143,7 +143,7 @@ Glean.shared.initialize(Configuration(channel: "beta"))
-The main control for Glean is on the `glean.Glean` singleton. +The main control for the Glean SDK is on the `glean.Glean` singleton. The Glean SDK should be initialized as soon as possible, and importantly, before any other libraries in the application start using Glean. Library code should never call `Glean.initialize`, since it should be called exactly once per application. @@ -159,7 +159,7 @@ Glean.initialize( ) ``` -Once initialized, if `upload_enabled` is true, the Glean SDK will automatically start collecting [baseline metrics](pings/metrics.md) and sending its [pings](pings/index.md), according to their respective schedules. +Once initialized, if `upload_enabled` is true, the Glean SDK will automatically start collecting [baseline metrics](pings/metrics.md). If `upload_enabled` is false, any persisted metrics, events and pings (other than `first_run_date`) are cleared, and subsequent calls to record metrics will be no-ops. Additional configuration is available on the `glean.Configuration` object, which can be passed into `Glean.initialize()`. @@ -171,7 +171,7 @@ See the [custom pings documentation](pings/custom.md) about adding custom pings
-The main control for Glean is on the `GleanInstance` singleton. +The main control for the Glean SDK is on the `GleanInstance` singleton. The Glean SDK should be initialized as soon as possible, and importantly, before any other libraries in the application start using Glean. Library code should never call `Glean.initialize`, since it should be called exactly once per application. @@ -195,13 +195,12 @@ GleanInstance.Initialize( ## Behavior when uninitialized -Metric recording that happens before Glean is initialized is queued and applied when Glean is initialized. -To avoid unbounded memory growth the queue is bounded (currently to a maximum of 100 tasks). -Further recordings are dropped. +Metric recording that happens before the Glean SDK is initialized is queued and applied at initialization. +To avoid unbounded memory growth the queue is bounded (currently to a maximum of 100 tasks), and further recordings are dropped. The number of recordings dropped, if any, is recorded in the `glean.error.preinit_tasks_overflow` metric. Custom ping submission will not fail before initialization. -Collection and upload of the custom ping is delayed until Glean is initialized. +Collection and upload of the custom ping is delayed until the Glean SDK is initialized. Built-in pings are only available after initialization. ## Enabling and disabling metrics diff --git a/docs/user/instrument-android-crashes-example.md b/docs/user/instrument-android-crashes-example.md index b1d0a0b9c..d57eb3d2d 100644 --- a/docs/user/instrument-android-crashes-example.md +++ b/docs/user/instrument-android-crashes-example.md @@ -1,4 +1,4 @@ -# Instrumenting Android Crashes With The Glean SDK +# Instrumenting Android crashes with the Glean SDK One of the things that might be useful to collect data on in an Android application is crashes. This guide will walk through a basic strategy for instrumenting an Android application with crash telemetry using a custom ping. diff --git a/docs/user/metric-parameters.md b/docs/user/metric-parameters.md index 595d51c61..f52917d49 100644 --- a/docs/user/metric-parameters.md +++ b/docs/user/metric-parameters.md @@ -39,4 +39,14 @@ - `version`: (default: 0) The version of the metric. A monotonically increasing integer value. This should be bumped if the metric changes in a backward-incompatible way. +- `data_sensitivity`: (default: []) A list of data sensitivity categories that the metric falls under. There are four data collection categories related to data sensitivity [defined in Mozilla's data collection review process](https://wiki.mozilla.org/Firefox/Data_Collection): + + - **Category 1: Technical Data:** (`technical`) Information about the machine or Firefox itself. Examples include OS, available memory, crashes and errors, outcome of automated processes like updates, safe browsing, activation, version \#s, and build id. This also includes compatibility information about features and APIs used by websites, add-ons, and other 3rd-party software that interact with Firefox during usage. + + - **Category 2: Interaction Data:** (`interaction`) Information about the user’s direct engagement with Firefox. Examples include how many tabs, add-ons, or windows a user has open; uses of specific Firefox features; session length, scrolls and clicks; and the status of discrete user preferences. + + - **Category 3: Web activity data:** (`web_activity`) Information about user web browsing that could be considered sensitive. Examples include users’ specific web browsing history; general information about their web browsing history (such as TLDs or categories of webpages visited over time); and potentially certain types of interaction data about specific webpages visited. + + - **Category 4: Highly sensitive data:** (`highly_sensitive`) Information that directly identifies a person, or if combined with other data could identify a person. Examples include e-mail, usernames, identifiers such as google ad id, apple id, Firefox account, city or country (unless small ones are explicitly filtered out), or certain cookies. It may be embedded within specific website content, such as memory contents, dumps, captures of screen data, or DOM data. + diff --git a/docs/user/pings/baseline.md b/docs/user/pings/baseline.md index cd5d98ae5..bdaf7af1a 100644 --- a/docs/user/pings/baseline.md +++ b/docs/user/pings/baseline.md @@ -2,7 +2,7 @@ ## Description -This ping is intended to provide metrics that are managed by the library itself, and not explicitly set by the application or included in the application's `metrics.yaml` file. +This ping is intended to provide metrics that are managed by the Glean SDK itself, and not explicitly set by the application or included in the application's `metrics.yaml` file. > **Note:** As the `baseline` ping was specifically designed for mobile operating systems, it is not sent when using the Glean Python bindings. diff --git a/docs/user/pings/custom.md b/docs/user/pings/custom.md index 645810c62..3fe096a89 100644 --- a/docs/user/pings/custom.md +++ b/docs/user/pings/custom.md @@ -51,7 +51,7 @@ The Glean SDK build generates code from `pings.yaml` in a `Pings` object, which
-In Kotlin, this object must be registered with Glean from your startup code (such as in your application's `onCreate` method or a function called from that method). +In Kotlin, this object must be registered with the Glean SDK from your startup code (such as in your application's `onCreate` method or a function called from that method). ```Kotlin import org.mozilla.yourApplication.GleanMetrics.Pings @@ -69,7 +69,7 @@ override fun onCreate() {
-In Swift, this object must be registered with Glean from your startup code +In Swift, this object must be registered with the Glean SDK from your startup code (such as in your application's `UIApplicationDelegate` [`application(_:didFinishLaunchingWithOptions:)`](https://developer.apple.com/documentation/uikit/uiapplicationdelegate/1622921-application) method or a function called from that method). ```swift @@ -99,7 +99,7 @@ from glean import load_pings pings = load_pings("pings.yaml") ``` -If your project is a distributable Python package, you need to include the `metrics.yaml` file using [one of the myriad ways to include data in a Python package](https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files) and then use [`pkg_resources.resource_filename()`](https://setuptools.readthedocs.io/en/latest/pkg_resources.html#resource-extraction) to get the filename at runtime. +If your project is a distributable Python package, you need to include the `pings.yaml` file using [one of the myriad ways to include data in a Python package](https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files) and then use [`pkg_resources.resource_filename()`](https://setuptools.readthedocs.io/en/latest/pkg_resources.html#resource-extraction) to get the filename at runtime. ```Python from glean import load_pings @@ -112,7 +112,7 @@ pings = load_pings(resource_filename(__name__, "pings.yaml"))
-In C#, this object must be registered with Glean from your startup code (such as in your application's `Main` method or a function called from that method). +In C#, this object must be registered with the Glean SDK from your startup code (such as in your application's `Main` method or a function called from that method). ```C# using static Mozilla.YourApplication.GleanMetrics.Pings; diff --git a/docs/user/pings/index.md b/docs/user/pings/index.md index ef9734794..5efabd70f 100644 --- a/docs/user/pings/index.md +++ b/docs/user/pings/index.md @@ -137,7 +137,7 @@ The system may also call `onStop()` when the activity has finished running, and ### Foreground -For iOS, Glean attaches to the [`willEnterForegroundNotification`](https://developer.apple.com/documentation/uikit/uiapplication/1622944-willenterforegroundnotification). +For iOS, the Glean SDK attaches to the [`willEnterForegroundNotification`](https://developer.apple.com/documentation/uikit/uiapplication/1622944-willenterforegroundnotification). This notification is posted by the OS shortly before an app leaves the background state on its way to becoming the active app. ### Background diff --git a/docs/user/pings/testing-custom-pings.md b/docs/user/pings/testing-custom-pings.md index 5a98cfc78..d939424a3 100644 --- a/docs/user/pings/testing-custom-pings.md +++ b/docs/user/pings/testing-custom-pings.md @@ -297,7 +297,7 @@ def test_verify_custom_ping_metrics():
-
+
TODO. To be implemented in [bug 1648446](https://bugzilla.mozilla.org/show_bug.cgi?id=1648446). diff --git a/docs/user/testing-metrics.md b/docs/user/testing-metrics.md index 14c9b5548..27bb8e93c 100644 --- a/docs/user/testing-metrics.md +++ b/docs/user/testing-metrics.md @@ -11,7 +11,7 @@ These functions expose a way to inspect and validate recorded metric values with
-In order to prevent issues with async calls when unit testing Glean, +In order to prevent issues with async calls when unit testing the Glean SDK, it is important to put the Glean SDK into testing mode by applying the JUnit `GleanTestRule` to your test class. When the Glean SDK is in testing mode, it enables uploading and clears the recorded metrics at the beginning of each test run. The rule can be used as shown below: @@ -56,7 +56,7 @@ so the IDE should complain if you attempt to use them inside of client code. > **NOTE**: There's no automatic test rule for Glean tests implemented. -In order to prevent issues with async calls when unit testing Glean, it is important to put the Glean SDK into testing mode. +In order to prevent issues with async calls when unit testing the Glean SDK, it is important to put the Glean SDK into testing mode. When the Glean SDK is in testing mode, it enables uploading and clears the recorded metrics at the beginning of each test run. Activate it by resetting Glean in your test's setup: @@ -100,11 +100,11 @@ Note that each of these functions is marked as `internal`, you need to import `G
-It is generally a good practice to "reset" Glean prior to every unit test that uses Glean, to prevent side effects of one unit test impacting others. -Glean contains a helper function `glean.testing.reset_glean()` for this purpose. +It is generally a good practice to "reset" the Glean SDK prior to every unit test that uses the Glean SDK, to prevent side effects of one unit test impacting others. +The Glean SDK contains a helper function `glean.testing.reset_glean()` for this purpose. It has two required arguments: the application ID, and the application version. -Each reset of Glean will create a new temporary directory for Glean to store its data in. -This temporary directory is automatically cleaned up the next time Glean is reset or when the testing framework finishes. +Each reset of the Glean SDK will create a new temporary directory for Glean to store its data in. +This temporary directory is automatically cleaned up the next time the Glean SDK is reset or when the testing framework finishes. The instructions below assume you are using [pytest](https://pypi.org/project/pytest/) as the test runner. Other test-running libraries have similar features, but are different in the details. @@ -144,7 +144,7 @@ assert (
-
+
TODO. To be implemented in [bug 1648448](https://bugzilla.mozilla.org/show_bug.cgi?id=1648448). @@ -250,7 +250,7 @@ assert 1 == metrics.url.visit.test_get_value()
-
+
TODO. To be implemented in [bug 1648448](https://bugzilla.mozilla.org/show_bug.cgi?id=1648448).