mozilla
/
gecko-dev
зеркало из https://github.com/mozilla/gecko-dev.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Bug 1880450 - Do not use visiblity to filter out elements for autofilling, but use visibility check when classify sections r=credential-management-reviewers,jneuberger,issammani 

We introduced visibility/focusability check to filter out invisible or unfocusable elements
to address the following issues:

1. Bug 1688209: The website's credit card form contains hidden credit card fields beneath the visible ones.

Example:
  <input id="cc-number" autocomplete="cc-number">
  <input id="hidden-cc-number" autocomplete="cc-number" hidden>.
  <input id="cc-name" autocomplete="cc-name">

The issue occurs because when our heuristic encounters consecutive fields that should not appear multiple times in a row,
we divide them and treat them as separate sections. In this example, the
visible cc-number and visible cc-name are put in different sections so we don't autofill both of the fields at the same time.

2. Bug 1822494: There is one hidden cc-exp-month field and one visible cc-exp field.

Example:
  <input id="cc-exp" autocomplete="cc-exp">
  <input id="hidden-cc-exp" autocomplete="cc-exp" hidden>.

When two cc-exp-* fields appear consecutively, our heuristic adjusts the first one to cc-exp-month and the second one to cc-exp-year.
However, in this bug, we should just honor the autocomplete attribute and do not change the field name

Bug 1753669: An invisible country field is located between tel-* fields.

Example:
  <input id="country" autocomplete="country">
  <input id="tel-area-code" autocomplete="tel-area-code">
  <input id="hidden-country" autocomplete="country" hidden>
  <input id="tel-local" autocomplete="tel-local">

When the heuristic sees the hidden country field, since it has already identified another country field previously,
our heuristic creates a new section upon encountering the invisible country field. This results that
we don't put tel-local field in the same section as the rest of the address fields.

---

However, introducing visibility and focusability checks also brings issues.

Some websites implement their own dropdowns for certain fields, like province, and include an invisible or unfocusable
field to store the value, as seen in Bug 1873202 and Bug 1836036.

We also see, in some cases, websites prefill certain address fields for users, and those fields are unfocusable.
For example, websites can use known-address data to determine the "state/province" field so users don't have to fill it.
But in these cases, we still want to identify this type of field so we can capture the data after users submit the form.

So, given the information collected so far, I think we should not filter out unfocusable or invisible elements before
running heuristics. Instead, we should adjust our heuristic to consider invisible elements in some cases. For example,
we should not create a new section upon encountering an invisible field, recognizing that it's common for websites to
place an invisible field near a visible field of the same type for various reasons.

Differential Revision: https://phabricator.services.mozilla.com/D202297
This commit is contained in:
Dimi 2024-02-26 11:43:58 +00:00
Родитель 05a17ea470
Коммит 6cabc3f465
13 изменённых файлов: 257 добавлений и 242 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

4
browser/extensions/formautofill/test/browser/heuristics/browser.toml
Просмотреть файл

@ -20,8 +20,6 @@ skip-if = ["apple_silicon && !debug"]
["browser_fr_fields.js"]
["browser_ignore_unfocusable_fields.js"]
["browser_label_rules.js"]
["browser_multiple_section.js"]
@ -37,3 +35,5 @@ skip-if = ["apple_silicon && !debug"]
["browser_section_validation_address.js"]
["browser_sections_by_name.js"]
["browser_sections_with_invisible_fields.js"]

159
browser/extensions/formautofill/test/browser/heuristics/browser_ignore_unfocusable_fields.js
Просмотреть файл

@ -1,159 +0,0 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
/* global add_heuristic_tests */
"use strict";
add_heuristic_tests([
{
description: "All visual fields are considered focusable.",
fixtureData: `
<html>
<body>
<form>
<input type="text" id="name" autocomplete="name" />
<input type="text" id="tel" autocomplete="tel" />
<input type="text" id="email" autocomplete="email"/>
<select id="country" autocomplete="country">
<option value="United States">United States</option>
</select>
<input type="text" id="postal-code" autocomplete="postal-code"/>
<input type="text" id="address-line1" autocomplete="address-line1" />
<div>
<input type="text" id="address-line2" autocomplete="address-line2" />
</div>
</form>
</form>
</body>
</html>
`,
expectedResult: [
{
default: {
reason: "autocomplete",
},
fields: [
{ fieldName: "name" },
{ fieldName: "tel" },
{ fieldName: "email" },
{ fieldName: "country" },
{ fieldName: "postal-code" },
{ fieldName: "address-line1" },
{ fieldName: "address-line2" },
],
},
],
},
{
// ignore opacity (see Bug 1835852),
description:
"Invisible fields with style.opacity=0 set are considered focusable.",
fixtureData: `
<html>
<body>
<form>
<input type="text" id="name" autocomplete="name" style="opacity:0" />
<input type="text" id="tel" autocomplete="tel" />
<input type="text" id="email" autocomplete="email" style="opacity:0"/>
<select id="country" autocomplete="country">
<option value="United States">United States</option>
</select>
<input type="text" id="postal-code" autocomplete="postal-code" />
<input type="text" id="address-line1" autocomplete="address-line1" />
<div>
<input type="text" id="address-line2" autocomplete="address-line2" />
</div>
</form>
</form>
</body>
</html>
`,
expectedResult: [
{
default: {
reason: "autocomplete",
},
fields: [
{ fieldName: "name" },
{ fieldName: "tel" },
{ fieldName: "email" },
{ fieldName: "country" },
{ fieldName: "postal-code" },
{ fieldName: "address-line1" },
{ fieldName: "address-line2" },
],
},
],
},
{
description:
"Some fields are considered unfocusable due to their invisibility.",
fixtureData: `
<html>
<body>
<form>
<input type="text" id="name" autocomplete="name" />
<input type="text" id="tel" autocomplete="tel" />
<input type="text" id="email" autocomplete="email" />
<input type="text" id="country" autocomplete="country" />
<input type="text" id="postal-code" autocomplete="postal-code" hidden />
<input type="text" id="address-line1" autocomplete="address-line1" style="display:none" />
<div style="visibility: hidden">
<input type="text" id="address-line2" autocomplete="address-line2" />
</div>
</form>
</body>
</html>
`,
expectedResult: [
{
default: {
reason: "autocomplete",
},
fields: [
{ fieldName: "name" },
{ fieldName: "tel" },
{ fieldName: "email" },
{ fieldName: "country" },
],
},
],
},
{
description: `Disabled field and field with tabindex="-1" is considered unfocusable`,
fixtureData: `
<html>
<body>
<form>
<input type="text" id="name" autocomplete="name" />
<input type="text" id="tel" autocomplete="tel" />
<input type="text" id="email" autocomplete="email" />
<input type="text" id="country" autocomplete="country" disabled/>
<input type="text" id="postal-code" autocomplete="postal-code" tabindex="-1"/>
<input type="text" id="address-line1" autocomplete="address-line1" />
<input type="text" id="address-line2" autocomplete="address-line2" />
</form>
</body>
</html>
`,
expectedResult: [
{
default: {
reason: "autocomplete",
},
fields: [
{ fieldName: "name" },
{ fieldName: "tel" },
{ fieldName: "email" },
{ fieldName: "address-line1" },
{ fieldName: "address-line2" },
],
},
],
},
]);

131
browser/extensions/formautofill/test/browser/heuristics/browser_sections_with_invisible_fields.js Normal file
Просмотреть файл

@ -0,0 +1,131 @@
/* Any copyright is dedicated to the Public Domain.
http://creativecommons.org/publicdomain/zero/1.0/ */
/* global add_heuristic_tests */
"use strict";
add_heuristic_tests([
{
description: `Create a new section when the section already has a field with the same field name`,
fixtureData: `
<html><body>
<input type="text" autocomplete="cc-number"/>
<input type="text" autocomplete="cc-name"/>
<input type="text" autocomplete="cc-exp"/>
<input type="text" autocomplete="cc-exp"/>
</body></html>
`,
expectedResult: [
{
default: {
reason: "autocomplete",
},
fields: [
{ fieldName: "cc-number" },
{ fieldName: "cc-name" },
{ fieldName: "cc-exp" },
],
},
{
fields: [{ fieldName: "cc-exp", reason: "autocomplete" }],
},
],
},
{
description: `Do not create a new section for an invisible field`,
fixtureData: `
<html><body>
<input type="text" autocomplete="cc-number"/>
<input type="text" autocomplete="cc-name"/>
<input type="text" autocomplete="cc-exp"/>
<input type="text" autocomplete="cc-exp" style="display:none"/>
</body></html>
`,
expectedResult: [
{
default: {
reason: "autocomplete",
},
fields: [
{ fieldName: "cc-number" },
{ fieldName: "cc-name" },
{ fieldName: "cc-exp" },
{ fieldName: "cc-exp" },
],
},
],
},
{
description: `Do not create a new section when the field with the same field name is an invisible field`,
fixtureData: `
<html><body>
<input type="text" autocomplete="cc-number""/>
<input type="text" autocomplete="cc-name"/>
<input type="text" autocomplete="cc-exp" style="display:none"/>
<input type="text" autocomplete="cc-exp"/>
</body></html>
`,
expectedResult: [
{
default: {
reason: "autocomplete",
},
fields: [
{ fieldName: "cc-number" },
{ fieldName: "cc-name" },
{ fieldName: "cc-exp" },
{ fieldName: "cc-exp" },
],
},
],
},
{
description: `Do not create a new section for an invisible field (match field is not adjacent)`,
fixtureData: `
<html><body>
<input type="text" autocomplete="cc-number"/>
<input type="text" autocomplete="cc-name"/>
<input type="text" autocomplete="cc-exp"/>
<input type="text" autocomplete="cc-number" style="display:none"/>
</body></html>
`,
expectedResult: [
{
default: {
reason: "autocomplete",
},
fields: [
{ fieldName: "cc-number" },
{ fieldName: "cc-name" },
{ fieldName: "cc-exp" },
{ fieldName: "cc-number" },
],
},
],
},
{
description: `Do not create a new section when the field with the same field name is an invisible field (match field is not adjacent)`,
fixtureData: `
<html><body>
<input type="text" autocomplete="cc-number" style="display:none"/>
<input type="text" autocomplete="cc-name"/>
<input type="text" autocomplete="cc-exp"/>
<input type="text" autocomplete="cc-number"/>
</body></html>
`,
expectedResult: [
{
default: {
reason: "autocomplete",
},
fields: [
{ fieldName: "cc-number" },
{ fieldName: "cc-name" },
{ fieldName: "cc-exp" },
{ fieldName: "cc-number" },
],
},
],
},
]);

2
browser/extensions/formautofill/test/browser/heuristics/third_party/browser_Euronics.js поставляемый
Просмотреть файл

@ -16,6 +16,8 @@ add_heuristic_tests(
//{ fieldName: "cc-cvc" },
{ fieldName: "cc-exp-month" },
{ fieldName: "cc-exp-year" },
{ fieldName: "cc-number", reason: "regex-heuristic" }, // invisible
{ fieldName: "cc-number", reason: "regex-heuristic" }, // invisible
],
},
{

11
browser/extensions/formautofill/test/browser/heuristics/third_party/browser_HomeDepot.js поставляемый
Просмотреть файл

@ -35,9 +35,16 @@ add_heuristic_tests(
reason: "autocomplete",
},
fields: [
{ fieldName: "cc-exp-month" },
{ fieldName: "cc-exp-year" },
{ fieldName: "cc-number", reason: "fathom" },
{ fieldName: "cc-exp-month", reason: "regex-heuristic" },
{ fieldName: "cc-exp-year", reason: "regex-heuristic" },
],
},
{
invalid: true,
fields: [
{ fieldName: "cc-exp-month", reason: "regex-heuristic" }, // invisible
{ fieldName: "cc-exp-year", reason: "regex-heuristic" }, // invisible
],
},
],

18
browser/extensions/formautofill/test/unit/head.js
Просмотреть файл

@ -71,24 +71,6 @@ region-name-tw = Taiwan
L10nRegistry.getInstance().registerSources([mockSource]);
}
/**
* Mock the return value of Services.focus.elementIsFocusable
* since a field's focusability can't be tested in a unit test.
*/
(function ignoreAFieldsFocusability() {
let stub = sinon.stub(Services, "focus").get(() => {
return {
elementIsFocusable() {
return true;
},
};
});
registerCleanupFunction(() => {
stub.restore();
});
})();
do_get_profile();
const EXTENSION_ID = "formautofill@mozilla.org";

33
browser/extensions/formautofill/test/unit/test_autofillFormFields.js
Просмотреть файл

@ -421,6 +421,39 @@ const TESTCASES = [
"cc-exp-year": "25",
},
},
{
description:
"Form with hidden input and visible input that share the same autocomplete attribute",
document: `<form>
<input id="hidden-cc" autocomplete="cc-number" hidden>
<input id="hidden-cc-2" autocomplete="cc-number" style="display:none">
<input id="visible-cc" autocomplete="cc-number">
<input id="hidden-name" autocomplete="cc-name" hidden>
<input id="hidden-name-2" autocomplete="cc-name" style="display:none">
<input id="visible-name" autocomplete="cc-name">
<input id="cc-exp-month" autocomplete="cc-exp-month">
<input id="cc-exp-year" autocomplete="cc-exp-year">
</form>`,
focusedInputId: "visible-cc",
profileData: {
guid: "123",
"cc-number": "4111111111111111",
"cc-name": "test name",
"cc-exp-month": 6,
"cc-exp-year": 25,
},
expectedResult: {
guid: "123",
"visible-cc": "4111111111111111",
"visible-name": "test name",
"cc-exp-month": "06",
"cc-exp-year": "25",
"hidden-cc": "4111111111111111",
"hidden-cc-2": "4111111111111111",
"hidden-name": "test name",
"hidden-name-2": "test name",
},
},
{
description:
"Fill credit card fields in a form where the value property is being used as a placeholder for cardholder name",

3
browser/extensions/formautofill/test/unit/xpcshell.toml
Просмотреть файл

@ -6,6 +6,9 @@ skip-if = [
firefox-appdir = "browser"
head = "head.js"
support-files = ["../fixtures/**"]
prefs = [
"extensions.formautofill.test.ignoreVisibilityCheck=true",
]
["test_activeStatus.js"]

1
toolkit/components/formautofill/Constants.ios.mjs
Просмотреть файл

@ -31,6 +31,7 @@ const IOS_DEFAULT_PREFERENCES = {
"extensions.formautofill.heuristics.captureOnFormRemoval": false,
"extensions.formautofill.heuristics.captureOnPageNavigation": false,
"extensions.formautofill.focusOnAutofill": false,
"extensions.formautofill.test.ignoreVisibilityCheck": false,
};
// Used Mimic the behavior of .getAutocompleteInfo()

19
toolkit/components/formautofill/Helpers.ios.mjs
Просмотреть файл

@ -45,6 +45,12 @@ HTMLElement.prototype.getAutocompleteInfo = function () {
};
};
// Bug 1835024. Webkit doesn't support `checkVisibility` API
// https://drafts.csswg.org/cssom-view-1/#dom-element-checkvisibility
HTMLElement.prototype.checkVisibility = function (options) {
throw new Error(`Not implemented: WebKit doesn't support checkVisibility `);
};
// This function helps us debug better when an error occurs because a certain mock is missing
const withNotImplementedError = obj =>
new Proxy(obj, {
@ -137,23 +143,10 @@ export const OSKeyStore = withNotImplementedError({
ensureLoggedIn: () => true,
});
// Checks an element's focusability and accessibility via keyboard navigation
const checkFocusability = element => {
return (
!element.disabled &&
!element.hidden &&
element.style.display != "none" &&
element.tabIndex != "-1"
);
};
// Define mock for Services
// NOTE: Services is a global so we need to attach it to the window
// eslint-disable-next-line no-shadow
export const Services = withNotImplementedError({
focus: withNotImplementedError({
elementIsFocusable: checkFocusability,
}),
locale: withNotImplementedError({ isAppLocaleRTL: false }),
prefs: withNotImplementedError({ prefIsLocked: () => false }),
strings: withNotImplementedError({

13
toolkit/components/formautofill/shared/FieldScanner.sys.mjs
Просмотреть файл

@ -2,6 +2,11 @@
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
const lazy = {};
ChromeUtils.defineESModuleGetters(lazy, {
FormAutofillUtils: "resource://gre/modules/shared/FormAutofillUtils.sys.mjs",
});
/**
* Represents the detailed information about a form field, including
* the inferred field name, the approach used for inferring, and additional metadata.
@ -73,6 +78,14 @@ export class FieldDetail {
get sectionName() {
return this.section || this.addressType;
}
#isVisible = null;
get isVisible() {
if (this.#isVisible == null) {
this.#isVisible = lazy.FormAutofillUtils.isFieldVisible(this.element);
}
return this.#isVisible;
}
}
/**

68
toolkit/components/formautofill/shared/FormAutofillHeuristics.sys.mjs
Просмотреть файл

@ -547,7 +547,9 @@ export const FormAutofillHeuristics = {
* all sections within its field details in the form.
*/
getFormInfo(form) {
let elements = this.getFormElements(form);
const elements = Array.from(form.elements).filter(element =>
lazy.FormAutofillUtils.isCreditCardOrAddressFieldType(element)
);
const scanner = new lazy.FieldScanner(elements, element =>
this.inferFieldInfo(element, elements)
@ -596,22 +598,6 @@ export const FormAutofillHeuristics = {
);
},
/**
* Get focusable form elements that are of credit card or address type
*
* @param {HTMLElement} form
* @returns {Array<HTMLElement>} focusable elements
*/
getFormElements(form) {
let elements = Array.from(form.elements).filter(
element =>
lazy.FormAutofillUtils.isCreditCardOrAddressFieldType(element) &&
lazy.FormAutofillUtils.isFieldFocusable(element)
);
return elements;
},
/**
* The result is an array contains the sections with its belonging field details.
*
@ -621,46 +607,54 @@ export const FormAutofillHeuristics = {
_classifySections(fieldDetails) {
let sections = [];
for (let i = 0; i < fieldDetails.length; i++) {
const fieldName = fieldDetails[i].fieldName;
const sectionName = fieldDetails[i].sectionName;
const cur = fieldDetails[i];
const [currentSection] = sections.slice(-1);
// The section this field might belong to
// The section this field might be placed into.
let candidateSection = null;
// If the field doesn't have a section name, MAYBE put it to the previous
// section if exists. If the field has a section name, maybe put it to the
// nearest section that either has the same name or it doesn't has a name.
// Otherwise, create a new section.
if (!currentSection || !sectionName) {
// Use name group from autocomplete attribute (ex, section-xxx) to look for the section
// we might place this field into.
// If the field doesn't have a section name, the candidate section is the previous section.
if (!currentSection || !cur.sectionName) {
candidateSection = currentSection;
} else if (sectionName) {
} else if (cur.sectionName) {
// If the field has a section name, the candidate section is the nearest section that
// either shares the same name or lacks a name.
for (let idx = sections.length - 1; idx >= 0; idx--) {
if (!sections[idx].name || sections[idx].name == sectionName) {
if (!sections[idx].name || sections[idx].name == cur.sectionName) {
candidateSection = sections[idx];
break;
}
}
}
// We got an candidate section to put the field to, check whether the section
// already has a field with the same field name. If yes, only add the field to when
// the type of the field might appear multiple times in a row.
if (candidateSection) {
let createNewSection = true;
if (candidateSection.fieldDetails.find(f => f.fieldName == fieldName)) {
// We might create a new section instead of placing the field in the candiate section if
// the section already has a field with the same field name.
// We also check visibility for both the fields with the same field name because we don't
// wanht to create a new section for an invisible field.
if (
candidateSection.fieldDetails.find(
f => f.fieldName == cur.fieldName && f.isVisible && cur.isVisible
)
) {
// For some field type, it is common to have multiple fields in one section, for example,
// email. In that case, we will not create a new section even when the candidate section
// already has a field with the same field name.
const [lastFieldDetail] = candidateSection.fieldDetails.slice(-1);
if (lastFieldDetail.fieldName == fieldName) {
if (MULTI_FIELD_NAMES.includes(fieldName)) {
if (lastFieldDetail.fieldName == cur.fieldName) {
if (MULTI_FIELD_NAMES.includes(cur.fieldName)) {
createNewSection = false;
} else if (fieldName in MULTI_N_FIELD_NAMES) {
} else if (cur.fieldName in MULTI_N_FIELD_NAMES) {
// This is the heuristic to handle special cases where we can have multiple
// fields in one section, but only if the field has appeared N times in a row.
// For example, websites can use 4 consecutive 4-digit `cc-number` fields
// instead of one 16-digit `cc-number` field.
const N = MULTI_N_FIELD_NAMES[fieldName];
const N = MULTI_N_FIELD_NAMES[cur.fieldName];
if (lastFieldDetail.part) {
// If `part` is set, we have already identified this field can be
// merged previously
@ -673,7 +667,7 @@ export const FormAutofillHeuristics = {
N == 2 ||
fieldDetails
.slice(i + 1, i + N - 1)
.every(f => f.fieldName == fieldName)
.every(f => f.fieldName == cur.fieldName)
) {
lastFieldDetail.part = 1;
fieldDetails[i].part = 2;

37
toolkit/components/formautofill/shared/FormAutofillUtils.sys.mjs
Просмотреть файл

@ -302,20 +302,27 @@ FormAutofillUtils = {
},
/**
* Determines if an element is focusable
* and accessible via keyboard navigation or not.
* Determines if an element is visually hidden or not.
*
* @param {HTMLElement} element
*
* @returns {bool} true if the element is focusable and accessible
* @param {boolean} visibilityCheck true to run visiblity check against
* element.checkVisibility API. Otherwise, test by only checking
* `hidden` and `display` attributes
* @returns {boolean} true if the element is visible
*/
isFieldFocusable(element) {
return (
// The Services.focus.elementIsFocusable API considers elements with
// tabIndex="-1" set as focusable. But since they are not accessible
// via keyboard navigation we treat them as non-interactive
Services.focus.elementIsFocusable(element, 0) && element.tabIndex != "-1"
);
isFieldVisible(element, visibilityCheck = true) {
if (
visibilityCheck &&
element.checkVisibility &&
!FormAutofillUtils.ignoreVisibilityCheck
) {
return element.checkVisibility({
checkOpacity: true,
checkVisibilityCSS: true,
});
}
return !element.hidden && element.style.display != "none";
},
/**
@ -1127,3 +1134,11 @@ XPCOMUtils.defineLazyPreferenceGetter(
"extensions.formautofill.focusOnAutofill",
true
);
// This is only used for testing
XPCOMUtils.defineLazyPreferenceGetter(
FormAutofillUtils,
"ignoreVisibilityCheck",
"extensions.formautofill.test.ignoreVisibilityCheck",
false
);