88 строки
5.1 KiB
Plaintext
88 строки
5.1 KiB
Plaintext
What questions will you answer with this data?
|
|
|
|
* Does blocking tracking cookies cause significant negative effect on user engagement?
|
|
* Does blocking tracking cookies cause significant breakage?
|
|
* When users experience such breakage, are they able to recover?
|
|
|
|
Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?
|
|
|
|
* Determine whether tracking-cookie-blocking has an effect on user or browser behavior
|
|
* Provide data essential to advancing Firefox users' privacy
|
|
|
|
What alternative methods did you consider to answer these questions? Why were they not sufficient?
|
|
|
|
* This functionality is brand new in Firefox 63 - we have not been able to measure this until now.
|
|
|
|
Can current instrumentation answer these questions?
|
|
|
|
* No, this functionality is brand new.
|
|
|
|
List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the Mozilla wiki.
|
|
|
|
Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.
|
|
Measurement Description Data Collection Category Tracking Bug #
|
|
Cohort ID 1 See PHD below
|
|
Timestamp 1
|
|
hash(top level page etld+1, user seed) (scalar string) 2.5
|
|
# of trackers trying to read/write cookies (integer) 1
|
|
# of trackers loaded (integer; could be 0?) 1
|
|
Exception status: was protection disabled on this page in the past? (boolean) 2
|
|
Has protection been re-enabled on this page (boolean) 2
|
|
Was the page reloaded? (boolean) 2
|
|
Page reloaded survey answer (integer) 2
|
|
User reports page broken through Control Center (boolean) 2
|
|
User hit Disable Protection Button in Control Center (causing page to unload and reload) (boolean) 2
|
|
User interacts with Control Center (boolean) 2
|
|
Was there a password field present? (boolean) 1
|
|
Was it autofilled by the password manager? (boolean) 2
|
|
Did the user interact and populate the field? (boolean) 2
|
|
Does this page embed a social login button? 1
|
|
|
|
PHD: https://docs.google.com/document/d/1JeCS71Kf0ZJivxgNrrIvpViHCwU-QOhKvH92Av0b7Yc/
|
|
|
|
|
|
How long will this data be collected? Choose one of the following:
|
|
|
|
* This is scoped to a time-limited experiment/project until 10-01-2018.
|
|
|
|
What populations will you measure?
|
|
|
|
* Windows
|
|
|
|
Which release channels?
|
|
|
|
* Beta 63
|
|
|
|
Which countries?
|
|
|
|
* all countries
|
|
|
|
Which locales?
|
|
|
|
* en-US
|
|
|
|
Any other filters? Please describe in detail below.
|
|
|
|
* See https://docs.google.com/document/d/1JeCS71Kf0ZJivxgNrrIvpViHCwU-QOhKvH92Av0b7Yc/edit#heading=h.a21jqmxt0qte
|
|
|
|
If this data collection is default on, what is the opt-out mechanism for users?
|
|
|
|
* Only users in Beta with shield enabled will receive the study. Users can disable shield, and users are prompted to disable the study during the course of the study.
|
|
|
|
Please provide a general description of how you will analyze this data.
|
|
|
|
* All Test pings will be filtered by those where Protection is enabled. This implies that the following holds for the boolean:
|
|
* Exception status = False
|
|
* Join with main ping client data to calculate search volume/user, uri count/user and active hours per user for each cohort. Then, determine if the two cohort distributions are significantly different using a t-test.
|
|
* Calculate the proportion of positive breakage responses versus total responses for the reload survey for each cohort. Then, determine if the two cohort proportions are are different.
|
|
* Using the two user reported mediums for page breakage (reload survey and through the control center) calculate the proportion of positive breakage responses for the test group, grouping them on whether the page had a social login button. Then, determine if the two cohort proportions are different.
|
|
*
|
|
The same as above, but instead grouping on whether there was a password field present. Then, determine if the two cohort proportions are different. For the group where the password field is present, perform the same analysis but further grouping by whether the password field was auto populated or if the user interacted with it.
|
|
* Calculate the proportion of users leaving the study for both cohorts. Then determine if the two cohort proportions are statistically different.
|
|
* Using the two user reported mediums for page breakage, calculate the # of trackers loaded distribution per page for the Test cohort. The determine if the # of trackers loaded distributions are different for positive and negative page breakage reports using a t-test.
|
|
* Using only the reload survey perform the above for both the Test and Control cohorts. Then determine if the # of trackers loaded distribution is different between the Test and Control cohorts for positive breakage reports using a t-test. Repeat for negative breakage reports.
|
|
|
|
Where do you intend to share the results of your analysis?
|
|
|
|
* We will provide internal data dashboards for ad strategy, anti-tracking, and product data science teams.
|