Fix #134: Add METRICS.md and initial telemetry setup
* Adds METRICS.md document in a new 'docs' subfolder. * Adds 'telemetry.js' background script which registers all events and provides a wrapper method to record events. * The event 'send_system_notice' was originally 'send_system_notification', however, that exceeded the [string length limit](https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/events.html#limits) imposed by Event Telemetry. * There is no need to unregister the events, as the events are only registered for the duration of the [current Firefox session](https://dxr.mozilla.org/mozilla-central/source/toolkit/components/telemetry/core/nsITelemetry.idl#484). Note: This PR does not actually record any events. That will happen in subsequent PRs. For the setup here, I did confirm that I can successfully records events in 'about:telemetry#events-tab' once 'browser.telemetry.recordEvent' is called. This can be confirmed by adding `telemetry.recordEvent('badge_toolbar_button', 'toolbar_button', null, {badge_type: 'add'});` after `telemetry.init()` in './src/background/index.js'.
This commit is contained in:
Bianca Danforth
Родитель
7074d3fb48
Коммит
8d3a7e8030
|
|
@ -0,0 +1,280 @@
|
|||
[👈 Back to README](../README.md)
|
||||
|
||||
|
||||
# Metrics
|
||||
|
||||
A summary of the metrics the Price Alerts extension will record.
|
||||
|
||||
|
||||
## Definitions
|
||||
|
||||
* **Fathom**: A [JavaScript framework](https://github.com/erikrose/fathom) used to extract product information from a Product Page.
|
||||
* **Product Page**: A webpage ([example](https://www.amazon.com/LEGO-Fantastic-Beasts-Grindelwald-Grindelwalds/dp/B07BKQXCZR/ref=sr_1_3_sspa?s=toys-and-games&ie=UTF8&qid=1538418041&sr=1-3-spons&keywords=legos&psc=1)) displaying a single product that the user could purchase online.
|
||||
* **Extraction**: The process of extracting the necessary product information from a Product Page using one of two methods: Fathom or fallback (CSS selectors or Open Graph attributes).
|
||||
* **Product Card**: A list item in the list of tracked products for which the user has opted to receive Price Alerts displayed on the browserAction popup. Each Product Card displays the product title, image and price.
|
||||
* **Survey**: a short survey collecting user feedback.
|
||||
* **Onboarding Popup**: The popup displayed when the user has zero products tracked, including the first time the popup is opened.
|
||||
|
||||
|
||||
## Analysis
|
||||
|
||||
Data collected by the Price Alerts extension will be used to answer the following questions:
|
||||
|
||||
### User Engagement
|
||||
|
||||
- Is the Price Alerts experience compelling?
|
||||
- How often do users set price alerts?
|
||||
- Do alerts cause users to re-engage with our shopping feature?
|
||||
- How often do they respond to price alerts?
|
||||
- At what threshold do users respond to price changes?
|
||||
- How do users respond when they receive a price alert?
|
||||
- Do they click through to the product page?
|
||||
- In what intervals?
|
||||
- In what circumstances?
|
||||
- How often are UI elements used?
|
||||
- What are users' tolerance for inaccuracy?
|
||||
- Are alerts annoying to users?
|
||||
- How often do users keep alerts?
|
||||
- How often do users delete alerts?
|
||||
- How much time passes before the "set alert" button is available in the drop down?
|
||||
- Where do users use Price Alerts?
|
||||
- For what kinds of products?
|
||||
- Which sites do users track products on most?
|
||||
- What's the price distribution of products users track?
|
||||
|
||||
|
||||
### Experiment Health
|
||||
|
||||
- Can users be compelled to report positive experiences as well as negative ones?
|
||||
- How often are products detected on supported product pages? (#125)
|
||||
- How often is Fathom recognizing products on a page? (#125)
|
||||
- On which sites do users report the most problems?
|
||||
- How often do product prices change, irrespective of user action?
|
||||
- How many price alerts are received?
|
||||
- In what intervals?
|
||||
|
||||
|
||||
### Performance
|
||||
|
||||
There are open questions about whether and how much Fathom may affect page performance. Time permitting, we may ask:
|
||||
|
||||
- What is the frame rate for a page while Fathom is running? (#124)
|
||||
|
||||
|
||||
### Questions to answer in future experiments
|
||||
|
||||
Answering these questions requires controlled A/B testing or more extensive data collection.
|
||||
|
||||
- Does our feature increase shopping browsing?
|
||||
- Does our feature increase shopping purchase?
|
||||
- Which sites do users browse on the most?
|
||||
- On what sites do users shop?
|
||||
|
||||
## Sample Pings
|
||||
|
||||
We will be sending pings using [Event Telemetry](https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/events.html) via the [WebExtensions Telemetry API](https://bugzilla.mozilla.org/show_bug.cgi?id=1280234).
|
||||
|
||||
Each event will exist as part of the `main` ping, under `payload.processes.dynamic.events` as an array. Below is a sample ping for the `badge_toolbar_button` event:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "main",
|
||||
// ...
|
||||
"payload": {
|
||||
// ...
|
||||
"processes": {
|
||||
// ...
|
||||
"dynamic": {
|
||||
// ...
|
||||
"events": [
|
||||
[
|
||||
618573,
|
||||
"extension.price_alerts",
|
||||
"badge_toolbar_button",
|
||||
"toolbar_button",
|
||||
null,
|
||||
{
|
||||
"badge_type": "add"
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Collection (User Events)
|
||||
|
||||
Each event here is the value of the `methods` key used in Event Telemetry.
|
||||
|
||||
|
||||
### `open_popup`
|
||||
|
||||
Fired when the user clicks the Price Alerts browserAction toolbar button to open the popup.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
- `objects`:
|
||||
- `'toolbar_button'`
|
||||
- `extra_keys`:
|
||||
- `'badge-type'`: String indicating what, if any, badge was present on the toolbar. One of 'add', 'price_alert', or 'none'.
|
||||
|
||||
|
||||
### `open_external_page`
|
||||
|
||||
Fired when the user clicks on a UI element that opens a page in a new tab.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
- `objects`: (All links are in the Onboarding Popup only.)
|
||||
- `'help_button'`: Sends the user to a Price Alerts support.mozilla.org page.
|
||||
- `'feedback_button'`: Sends the user to a feedback Survey.
|
||||
- `'system_notification'`: Sends the user to the product page for the price alert displayed in the notification.
|
||||
- `'product_card'`: Sends the user to the product page for the given Product Card.
|
||||
- `'amazon_link'`: Sends the user to Amazon.
|
||||
- `'best_buy_link'`: Sends the user to Best Buy.
|
||||
- `'ebay_link'`: Sends the user to Ebay.
|
||||
- `'home_depot_link'`: Sends the user to Home Depot.
|
||||
- `'walmart_link'`: Sends the user to Walmart.
|
||||
- `'learn_more_link'`: Sends the user to a Price Alerts support.mozilla.org page.
|
||||
- `extra_keys`: For objects of type `'system_notification'` and `'product'` only
|
||||
- `'product_id'`: String. The unique product identifier.
|
||||
- `'price'`: Number. The price of the product in subunits (e.g. a $10.00 product would have a value of `1000`).
|
||||
- `'price_original'`: Number. The original price of the product in subunits (e.g. a $10.00 product would have a value of `1000`).
|
||||
|
||||
|
||||
### `add_product`
|
||||
|
||||
Fired when the user clicks the add product button in the browserAction popup.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
|
||||
- `objects`:
|
||||
- `'add_button'`
|
||||
- `extra_keys`:
|
||||
- `'product_id'`: String. The unique product identifier.
|
||||
|
||||
|
||||
### `delete_product`
|
||||
|
||||
Fired when the user clicks a delete product button in a Product Card in the browserAction popup.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
|
||||
- `objects`:
|
||||
- `'delete_button'`
|
||||
- `extra_keys`:
|
||||
- `'product_id'`: String. The unique product identifier.
|
||||
|
||||
|
||||
### `undo_delete_product`
|
||||
|
||||
Fired when the user clicks an undo button in a Product Card in the browserAction popup.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
|
||||
- `objects`:
|
||||
- `'undo_button'`
|
||||
- `extra_keys`:
|
||||
- `'product_id'`: String. The unique product identifier.
|
||||
|
||||
|
||||
### `uninstall`
|
||||
|
||||
Fired when the user uninstalls the extension.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
|
||||
- `objects`:
|
||||
- `'uninstall'`
|
||||
|
||||
|
||||
### `hide_toolbar_button`
|
||||
|
||||
Fired when the user hides the extension's browserAction toolbar button from the browser chrome.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
|
||||
- `objects`:
|
||||
- `'toolbar_button'`
|
||||
|
||||
|
||||
## Collection (Non-User Events)
|
||||
|
||||
|
||||
### `badge_toolbar_button`
|
||||
|
||||
Fired whenever the toolbar is badged in response to:
|
||||
1. A successful product extraction or
|
||||
2. A price alert
|
||||
|
||||
|
||||
#### Payload properties
|
||||
|
||||
- `objects`:
|
||||
- `'toolbar_button'`
|
||||
- `extra_keys`:
|
||||
- `'badge-type'`: String indicating what, if any, badge was present on the toolbar. One of 'add' or 'price_alert'.
|
||||
|
||||
|
||||
### `send_system_notice`
|
||||
|
||||
Fired whenever a system notification is sent to the user notifying them of a price alert.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
|
||||
- `objects`:
|
||||
- `'system_notification'`
|
||||
- `extra_keys`:
|
||||
- `'product_id'`: String. The unique product identifier.
|
||||
- `'price'`: Number. The price of the product in subunits (e.g. a $10.00 product would have a value of `1000`).
|
||||
- `'price_original'`: Number. The original price of the product in subunits (e.g. a $10.00 product would have a value of `1000`).
|
||||
|
||||
|
||||
### `attempt_extraction`
|
||||
|
||||
Fired whenever a supported page loads and the add-on attempts to extract product information from the page.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
|
||||
- `objects`:
|
||||
- `'page'`
|
||||
- `extra_keys`:
|
||||
- `url`: String. The url of the page the extraction script is running in
|
||||
|
||||
|
||||
### `complete-extraction`
|
||||
|
||||
Fired whenever extraction on a supported page completes, whether or not the extraction was successful.
|
||||
|
||||
|
||||
#### Payload properties
|
||||
|
||||
- `objects`:
|
||||
- `'page'`
|
||||
- `extra_keys`:
|
||||
- `url`: String. The url of the page the extraction script is running in
|
||||
- `method`: String. The extraction method that was successful, if any. One of: 'fathom', 'fallback' or 'neither'. A value of 'neither' means that extraction failed.
|
||||
|
||||
|
||||
## Opt-out
|
||||
|
||||
All data collection occurs through Firefox telemetry, and standard telemetry opt-out methods apply.
|
||||
|
||||
No telemetry will be sent from the extension in the following additional cases:
|
||||
* [Do Not Track](https://support.mozilla.org/en-US/kb/how-do-i-turn-do-not-track-feature) is enabled
|
||||
* Preference: `privacy.donottrackheader.enabled`
|
||||
* [Tracking Protection](https://support.mozilla.org/en-US/kb/tracking-protection) is enabled
|
||||
* Preference: `privacy.trackingprotection.enabled`
|
||||
* The user is in a [Private Browsing](https://support.mozilla.org/en-US/kb/private-browsing-use-firefox-without-history?redirectlocale=en-US&redirectslug=Private+Browsing) window
|
||||
* Preference: `browser.privatebrowsing.autostart`
|
||||
* [`windows.Window`](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/windows/Window) property: `window.incognito`
|
||||
|
|
@ -18,8 +18,11 @@ import {handleWebRequest, updatePrices} from 'commerce/background/price_updates'
|
|||
import store from 'commerce/state';
|
||||
import {checkMigrations} from 'commerce/state/migrations';
|
||||
import {loadStateFromStorage} from 'commerce/state/sync';
|
||||
import telemetry from 'commerce/background/telemetry';
|
||||
|
||||
(async function main() {
|
||||
telemetry.init();
|
||||
|
||||
// Set browser action default badge color, which can't be set via manifest
|
||||
browser.browserAction.setBadgeBackgroundColor({
|
||||
color: await config.get('badgeAlertBackground'),
|
||||
|
|
|
|||
|
|
@ -0,0 +1,130 @@
|
|||
/* This Source Code Form is subject to the terms of the Mozilla Public
|
||||
* License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
|
||||
|
||||
/**
|
||||
* TODO: add desc.
|
||||
* @module
|
||||
*/
|
||||
|
||||
const telemetry = {
|
||||
EVENT_CATEGORY: 'extension.price_alerts',
|
||||
events: {
|
||||
/**
|
||||
* User Events
|
||||
*/
|
||||
// User clicks toolbar button to open the popup
|
||||
open_popup: {
|
||||
methods: ['open_popup'],
|
||||
objects: ['toolbar_button'],
|
||||
extra_keys: ['badge_type'],
|
||||
},
|
||||
// User clicks on a UI element opening a page in a new tab
|
||||
open_external_page: {
|
||||
methods: ['open_external_page'],
|
||||
objects: [
|
||||
'help_button',
|
||||
'feedback_button',
|
||||
'system_notification',
|
||||
'product_card',
|
||||
'amazon_link',
|
||||
'best_buy_link',
|
||||
'ebay_link',
|
||||
'home_depot_link',
|
||||
'walmart_link',
|
||||
'learn_more_link',
|
||||
],
|
||||
// For objects of type 'system_notification' and 'product' only
|
||||
extra_keys: [
|
||||
'product_id',
|
||||
'price',
|
||||
'price_original',
|
||||
],
|
||||
},
|
||||
// User adds a product to the product listing
|
||||
add_product: {
|
||||
methods: ['add_product'],
|
||||
objects: ['add_button'],
|
||||
extra_keys: ['product_id'],
|
||||
},
|
||||
// User deletes a product from the product listing
|
||||
delete_product: {
|
||||
methods: ['delete_product'],
|
||||
objects: ['delete_button'],
|
||||
extra_keys: ['product_id'],
|
||||
},
|
||||
// User undeletes a product from the product listing
|
||||
undo_delete_product: {
|
||||
methods: ['undo_delete_product'],
|
||||
objects: ['undo_button'],
|
||||
extra_keys: ['product_id'],
|
||||
},
|
||||
// User uninstalls the extension
|
||||
uninstall: {
|
||||
methods: ['uninstall'],
|
||||
objects: ['uninstall'],
|
||||
extra_keys: [],
|
||||
},
|
||||
// User hides the toolbar button for the extension
|
||||
hide_toolbar_button: {
|
||||
methods: ['hide_toolbar_button'],
|
||||
objects: ['toolbar_button'],
|
||||
extra_keys: [],
|
||||
},
|
||||
/**
|
||||
* Non-user Events
|
||||
*/
|
||||
// Toolbar button is badged either due to a price alert or an extracted product
|
||||
badge_toolbar_button: {
|
||||
methods: ['badge_toolbar_button'],
|
||||
objects: ['toolbar_button'],
|
||||
extra_keys: ['badge_type'],
|
||||
},
|
||||
// System notification is sent notifying user of a price alert
|
||||
send_system_notice: {
|
||||
methods: ['send_system_notice'],
|
||||
objects: ['system_notification'],
|
||||
extra_keys: [
|
||||
'product_id',
|
||||
'price_original',
|
||||
'price',
|
||||
],
|
||||
},
|
||||
// Product extraction is attempted on the content page
|
||||
attempt_extraction: {
|
||||
methods: ['attempt_extraction'],
|
||||
objects: ['page'],
|
||||
extra_keys: ['url'],
|
||||
},
|
||||
// Product extraction is completed on the content page
|
||||
complete_extraction: {
|
||||
methods: ['complete_extraction'],
|
||||
objects: ['page'],
|
||||
extra_keys: [
|
||||
'url',
|
||||
'method',
|
||||
],
|
||||
},
|
||||
},
|
||||
async init() {
|
||||
await browser.telemetry.registerEvents(this.EVENT_CATEGORY, this.events);
|
||||
},
|
||||
async recordEvent(method, object, value, extra) {
|
||||
if (!browser.telemetry.canUpload()) {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await browser.telemetry.recordEvent(
|
||||
this.EVENT_CATEGORY,
|
||||
method,
|
||||
object,
|
||||
value,
|
||||
extra,
|
||||
);
|
||||
} catch (error) {
|
||||
throw new Error(`recordEvent failed with error: ${error}`);
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
export default telemetry;
|
||||
|
|
@ -30,7 +30,8 @@
|
|||
"unlimitedStorage",
|
||||
"notifications",
|
||||
"webRequest",
|
||||
"webRequestBlocking"
|
||||
"webRequestBlocking",
|
||||
"telemetry"
|
||||
],
|
||||
"experiment_apis": {
|
||||
"shoppingPrefs": {
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче