Before this commit, we used event telemetry to record timing and counts
for the different steps of a bookmark merge (fetching the local tree,
new local contents, remote tree, and new remote contents; merging;
applying; fetching outgoing records; and notifying observers).
This has several limitations. We need to store all numbers as strings,
include a "flow ID" to tag events from the same merge, and collect
failure reasons twice. We also can't correlate these events to the
existing engine telemetry, meaning we can't see other stats for that
engine, or for the entire sync. Finally, we need to run separate
queries on these events for analysis, instead of extending our
existing engine dashboards.
This approach also feels like an abuse of event telemetry, so this
commit adds a "steps" field for each engine in the Sync ping. Each step
has a name, time taken, and additional named counts, like the number of
items in the tree, or merged structure stats.
Currently, only the `buffered-bookmarks` engine records these steps.
Differential Revision: https://phabricator.services.mozilla.com/D33411
--HG--
extra : moz-landing-system : lando
This commit introduces a `mozISyncedBookmarksMirrorProgressListener`
interface for capturing telemetry and updating shutdown blocker state
after each step of the merge, instead of waiting until the end. This
also means we can also record events for interrupted and failed merges,
and pass validation data through to the Sync ping.
Shutdown hang crash reports now have a `steps` field, indicating the
sequence of completed steps and when they were recorded. If the last
step in the hang report is `fetchLocalTree`, we know the merger is
blocked on `fetchNewLocalContents`. If the last step is
`fetchNewLocalContents`, the merger is stuck at `fetchRemoteTree`,
since that's the next step after `fetchNewLocalContents`.
This commit also implements `Driver::record_telemetry_event` to
dispatch progress callback runnables to the main thread.
Differential Revision: https://phabricator.services.mozilla.com/D31950
--HG--
extra : moz-landing-system : lando
The number of Sync users on these channels is low compared to Release,
so we can do this without a gradual rollout. This also lets more users
test the new bookmark sync engine without manually flipping the pref.
Differential Revision: https://phabricator.services.mozilla.com/D29859
--HG--
extra : moz-landing-system : lando
This commit exports livemarks before syncing for the first time, to
avoid losing livemarks that Sync may have resurrected. As of v0.2.4,
Dogear treats livemarks as non-syncable, and deletes them on both
sides.
This also bumps the mirror schema version, to trigger a first sync.
Differential Revision: https://phabricator.services.mozilla.com/D28543
--HG--
extra : moz-landing-system : lando
- `Array.map` becomes `Array.from`
- Array copying via `Array.slice` becomes `Array.from`.
- `Array.forEach` that did not rely on closures becomes `for`-`of` loops.
- Anything else: `Array.X` becomes `Array.prototype.X`.
Complex cases:
dom/bindings/test/TestInterfaceJS.js and
dom/bindings/test/test_exception_options_from_jsimplemented.html
use `Array.indexOf` to generate an error with a specific error message.
Switched to `Array.prototype.forEach` to generate the same error.
js/src/jit-test/tests/basic/exception-column-number.js
In this test `Array.indexOf()` is used to generate an error. Since the
exact message doesn't matter, I switched to `Array.from()`.
Intentionally not changed:
editor/libeditor/tests/browserscope/lib/richtext/richtext/js/range.js
Did not modify because this is 3rd-party code and the code uses
feature detection as a fall back when Array generics are not used.
testing/talos/talos/tests/dromaeo/lib/mootools.js
Did not modify because mootools adds the `Array.slice` method to the
`Array` object.
Not changed because they check the implementation of Array generics:
js/src/jit-test/tests/basic/arrayNatives.js
js/src/jit-test/tests/basic/bug563243.js
js/src/jit-test/tests/basic/bug618853.js
js/src/jit-test/tests/basic/bug830967.js
js/src/jit-test/tests/jaeger/recompile/bug656753.js
js/src/jit-test/tests/self-hosting/alternate-static-and-instance-array-extras.js
js/src/tests/non262/Array/generics.js
js/src/tests/non262/Array/regress-415540.js
js/src/tests/non262/extensions/regress-355497.js
js/src/tests/non262/extensions/typedarray-set-neutering.js
Depends on D27802
Differential Revision: https://phabricator.services.mozilla.com/D27803
--HG--
extra : moz-landing-system : lando
`Async.jankYielder` is known to, unfortunately, cause jank by creating a lot of
immediately resolved promises that must be then GCed. For a collection of 50
items, it will create 50 promises and 49 of them will immediately resolve.
Instead of `Async.jankYielder`, we now have `Async.yieldState`, which simply
keeps track of whether or not the caller should yield to the event loop. Two
higher level looping constructs are built on top of it:
* `Async.yieldingIterator`, which has been rewritten to not create extraneous
promises; and
* `Async.yieldingForEach`, which is a replacement for awaiting
`Async.jankYielder` in a loop. Instead, it accepts the loop body as a
function.
Each of these can share an instance of an `Async.yieldState`, which allows an
object with multiple loops to yield every N iterations overall, instead of
every N iterations of each loop, which keeps the behaviour of using one
`Async.jankYielders` in multiple places.
Differential Revision: https://phabricator.services.mozilla.com/D26229
--HG--
extra : moz-landing-system : lando
`Async.jankYielder` is known to, unfortunately, cause jank by creating a lot of
immediately resolved promises that must be then GCed. For a collection of 50
items, it will create 50 promises and 49 of them will immediately resolve.
Instead of `Async.jankYielder`, we now have `Async.yieldState`, which simply
keeps track of whether or not the caller should yield to the event loop. Two
higher level looping constructs are built on top of it:
* `Async.yieldingIterator`, which has been rewritten to not create extraneous
promises; and
* `Async.yieldingForEach`, which is a replacement for awaiting
`Async.jankYielder` in a loop. Instead, it accepts the loop body as a
function.
Each of these can share an instance of an `Async.yieldState`, which allows an
object with multiple loops to yield every N iterations overall, instead of
every N iterations of each loop, which keeps the behaviour of using one
`Async.jankYielders` in multiple places.
Differential Revision: https://phabricator.services.mozilla.com/D26229
--HG--
extra : moz-landing-system : lando
`Async.jankYielder` is known to, unfortunately, cause jank by creating a lot of
immediately resolved promises that must be then GCed. For a collection of 50
items, it will create 50 promises and 49 of them will immediately resolve.
Instead of `Async.jankYielder`, we now have `Async.yieldState`, which simply
keeps track of whether or not the caller should yield to the event loop. Two
higher level looping constructs are built on top of it:
* `Async.yieldingIterator`, which has been rewritten to not create extraneous
promises; and
* `Async.yieldingForEach`, which is a replacement for awaiting
`Async.jankYielder` in a loop. Instead, it accepts the loop body as a
function.
Each of these can share an instance of an `Async.yieldState`, which allows an
object with multiple loops to yield every N iterations overall, instead of
every N iterations of each loop, which keeps the behaviour of using one
`Async.jankYielders` in multiple places.
Differential Revision: https://phabricator.services.mozilla.com/D26229
--HG--
extra : moz-landing-system : lando
This commit:
* Uses chunking to insert child GUIDs in `storeRemoteFolder`.
* Changes the mirror schema to store diverging structure, so that it
can be passed to Dogear.
* Sorts remote items by GUID, so that diverging ones can be reparented
in a deterministic order.
* Measures and logs the time taken to run the Rust merger.
* Adds tests for multiple parents and replacing invalid remote items.
* Fixes two tests in `test_bookmark_structure_changes` to ensure the
remote structure is consistent.
Differential Revision: https://phabricator.services.mozilla.com/D26275
--HG--
extra : moz-landing-system : lando
AddonManagergetInstallForURL() has a number of optional arguments, most
of which are passed infrequently. Convert them from positional arguments
to a single options object.
Differential Revision: https://phabricator.services.mozilla.com/D18475
--HG--
extra : rebase_source : 503c09b54fab90cefe69286b05def43ef70074df
***
Bug 1514594: Part 3a - Change ChromeUtils.import to return an exports object; not pollute global. r=mccr8
This changes the behavior of ChromeUtils.import() to return an exports object,
rather than a module global, in all cases except when `null` is passed as a
second argument, and changes the default behavior not to pollute the global
scope with the module's exports. Thus, the following code written for the old
model:
ChromeUtils.import("resource://gre/modules/Services.jsm");
is approximately the same as the following, in the new model:
var {Services} = ChromeUtils.import("resource://gre/modules/Services.jsm");
Since the two behaviors are mutually incompatible, this patch will land with a
scripted rewrite to update all existing callers to use the new model rather
than the old.
***
Bug 1514594: Part 3b - Mass rewrite all JS code to use the new ChromeUtils.import API. rs=Gijs
This was done using the followng script:
https://bitbucket.org/kmaglione/m-c-rewrites/src/tip/processors/cu-import-exports.jsm
***
Bug 1514594: Part 3c - Update ESLint plugin for ChromeUtils.import API changes. r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D16747
***
Bug 1514594: Part 3d - Remove/fix hundreds of duplicate imports from sync tests. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16748
***
Bug 1514594: Part 3e - Remove no-op ChromeUtils.import() calls. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16749
***
Bug 1514594: Part 3f.1 - Cleanup various test corner cases after mass rewrite. r=Gijs
***
Bug 1514594: Part 3f.2 - Cleanup various non-test corner cases after mass rewrite. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16750
--HG--
extra : rebase_source : 359574ee3064c90f33bf36c2ebe3159a24cc8895
extra : histedit_source : b93c8f42808b1599f9122d7842d2c0b3e656a594%2C64a3a4e3359dc889e2ab2b49461bab9e27fc10a7
SyncIntegration firefox-ios tests are failing if the server is wiped in first phase
Differential Revision: https://phabricator.services.mozilla.com/D10219
--HG--
extra : moz-landing-system : lando
This isn't strictly part of removing support for non-restartless
extensions, but it ensures that sync is testing with the types of
extensions that people are really using these days.
Differential Revision: https://phabricator.services.mozilla.com/D10111
--HG--
extra : rebase_source : 9737bdce09464d7efb6150c15b1936bac99278f9
extra : source : 05f4f82cd81ced4c8cd441e7af0a31b728287bf1
This patch morphs MasterPassword.jsm to OSKeyStore.jsm while keeping the same
API, as an adaptor between the API and the native API exposed as nsIOSKeyStore.idl.
Noted that OS Key Store has the concept of "recovery phrase" that we won't
be adopting here. The recovery phrase, together with our label, allow
the user to re-create the same key in OS key store.
Test case changes are needed because we have started asking for login in
places where we'll only do previously when "master password is enabled".
This also made some "when master password is enabled" tests invalid because
it is always considered enabled.
Some more test changes are needed simply because they previously rely on the
stable order of microtask resolutions (and the stable # of promises for a
specific operation). That has certainly changed with OSKeyStore.
The credit card form autofill is only enabled on Nightly.
Differential Revision: https://phabricator.services.mozilla.com/D4498
--HG--
rename : browser/extensions/formautofill/MasterPassword.jsm => browser/extensions/formautofill/OSKeyStore.jsm
rename : browser/extensions/formautofill/test/browser/browser_creditCard_fill_master_password.js => browser/extensions/formautofill/test/browser/browser_creditCard_fill_cancel_login.js
extra : rebase_source : cabbd8cdec86e5b3965cf1c8b6e635b73b6c2095
extra : histedit_source : 65e71057104465553fefa1d0b293580efed53075
This patch morphs MasterPassword.jsm to OSKeyStore.jsm while keeping the same
API, as an adaptor between the API and the native API exposed as nsIOSKeyStore.idl.
Noted that OS Key Store has the concept of "recovery phrase" that we won't
be adopting here. The recovery phrase, together with our label, allow
the user to re-create the same key in OS key store.
Test case changes are needed because we have started asking for login in
places where we'll only do previously when "master password is enabled".
This also made some "when master password is enabled" tests invalid because
it is always considered enabled.
Some more test changes are needed simply because they previously rely on the
stable order of microtask resolutions (and the stable # of promises for a
specific operation). That has certainly changed with OSKeyStore.
The credit card form autofill is only enabled on Nightly.
Differential Revision: https://phabricator.services.mozilla.com/D4498
--HG--
rename : browser/extensions/formautofill/MasterPassword.jsm => browser/extensions/formautofill/OSKeyStore.jsm
rename : browser/extensions/formautofill/test/browser/browser_creditCard_fill_master_password.js => browser/extensions/formautofill/test/browser/browser_creditCard_fill_cancel_login.js
extra : moz-landing-system : lando
This patch disables a couple of tests, removes old async support from TPS,
and puts more effort into preventing Syncs while TPS is running, and
removed an unused import.
Differential Revision: https://phabricator.services.mozilla.com/D8692
--HG--
extra : moz-landing-system : lando
This commit replaces incoming synced livemarks with tombstones, and
reuploads the tombstone and updated parent to the server. Existing
livemarks are left untouched to minimize data loss; we'll either
delete them during migration, or when another client runs a full sync.
Differential Revision: https://phabricator.services.mozilla.com/D7084
--HG--
extra : rebase_source : db78efc696cce13cf596d2ab0906f049a2cbe12f
D5831 refactors validatePageInfo to use validateItemProperties. The record generated by _recordToPlaceInfo makes use of getters. validateItemProperties does a shallow copy of the properties and the visits properties is being left out because of the use of getters. Therefore as discussed we are returning a new object from _recordToPlaceInfo.
Differential Revision: https://phabricator.services.mozilla.com/D7467
--HG--
extra : moz-landing-system : lando
Some global imports, like `fetch`, expose multiple symbols. This patch
ensures we can define lazy getters for those symbols, by mapping the
extra symbol name to the main import name.
Differential Revision: https://phabricator.services.mozilla.com/D7522
--HG--
extra : moz-landing-system : lando
This also makes various AutofillRecords methods async, with the exception of
remove() and removeAll().
Noted that I didn't implement any kind of "lock" for FormAutofillStorage --
please do not call these methods concurrently -- if you must please |await|
for the last call to resolve. This most likely would happen in tests, and
shouldn't happen in the real world, given that all user actions happen on
macrotasks, and probably not at the next tick, unless Quicksilver is a
Firefox user.
FormAutofillStorage can be improved if there are complex use cases for it.
Differential Revision: https://phabricator.services.mozilla.com/D4420
--HG--
extra : moz-landing-system : lando
This also makes various AutofillRecords methods async, with the exception of
remove() and removeAll().
Noted that I didn't implement any kind of "lock" for FormAutofillStorage --
please do not call these methods concurrently -- if you must please |await|
for the last call to resolve. This most likely would happen in tests, and
shouldn't happen in the real world, given that all user actions happen on
macrotasks, and probably not at the next tick, unless Quicksilver is a
Firefox user.
FormAutofillStorage can be improved if there are complex use cases for it.
Differential Revision: https://phabricator.services.mozilla.com/D4420
--HG--
extra : moz-landing-system : lando
Automatic changes by ESLint, except for manual corrections for .xml files.
Differential Revision: https://phabricator.services.mozilla.com/D4439
--HG--
extra : moz-landing-system : lando
Two cases were hiding permanently failing tests. I've commented those out and
filed bug 1487431.
Differential Revision: https://phabricator.services.mozilla.com/D4680
--HG--
extra : rebase_source : 232fa6173de8844a9c47d59926ec8e39d0640ecd
This makes it much easier to update existing consumers of
XPCOMUtils.enumerateCategoryEntries to use the category manager directly.
It also, unfortunately, requires updating existing category manager consumers
to use the Services getter in order to avoid ESLint errors.
Differential Revision: https://phabricator.services.mozilla.com/D4278
--HG--
extra : rebase_source : fb9fd9b21db80af472ff6250a2e9a35e8d538147
Creating non-shared scopes for frame scripts is fairly expensive. After these
changes it's even more expensive. However, many frame scripts have no use for
the shared scopes at all. Run-once scripts which execute in closures, for
instance, make no use of them. And after bug 1472491, neither do most of our
default frame scripts.
MozReview-Commit-ID: 9PK7bYdQ0yh
--HG--
extra : rebase_source : db2516d2f00a75e233e1957649f2b62a9299b7cd
The tagging API is being merged into the bookmarking API. This is part of it.
Differential Revision: https://phabricator.services.mozilla.com/D2450
--HG--
extra : moz-landing-system : lando
This patch:
* Exposes a global Sync change counter on `nsINavBookmarksService`.
This is similar to SQLite's `total_changes()`, but just for changes
to bookmarks that affect Sync, and accounts for changes from multiple
threads and connections.
* Adds a SQL function to bump the counter, and extends the
`moz_bookmarks` triggers to call it.
* Moves merging outside the transaction in the bookmarks mirror, and
checks that the counters match before applying.
Differential Revision: https://phabricator.services.mozilla.com/D2004
--HG--
extra : moz-landing-system : lando
This patch unifies `updateBookmark` and `moveBookmark`.
`update` already handles the Sync change counter, and `move` specially
optimizes for moves. We can consolidate the two by reusing the queries
from `move` in `update`, moving its extra `setAncestorsLastModified`
call to `PlacesUtils.bookmarks.update`, and removing its transaction
and wrapper.
Differential Revision: https://phabricator.services.mozilla.com/D2016
--HG--
extra : moz-landing-system : lando
Consuming the new 'page-visited' notification was fairly trivial,
since it was already brought over to onVisits. There's not much to
say about this other than that I'm a little bit uncertain about
all the hoops we have to jump through to get a JSContext and
GlobalObject from History.cpp (which is discussed in the earlier
commit in the series).
MozReview-Commit-ID: LHaBWSylyLI
--HG--
extra : rebase_source : ef8fa174f902e6abd397fb80294e83843120aa81
Consuming the new 'page-visited' notification was fairly trivial,
since it was already brought over to onVisits. There's not much to
say about this other than that I'm a little bit uncertain about
all the hoops we have to jump through to get a JSContext and
GlobalObject from History.cpp (which is discussed in the earlier
commit in the series).
MozReview-Commit-ID: LHaBWSylyLI
--HG--
extra : rebase_source : fd064e53e8645c3914188c05e71c8800d6f2174b
Consuming the new 'page-visited' notification was fairly trivial,
since it was already brought over to onVisits. There's not much to
say about this other than that I'm a little bit uncertain about
all the hoops we have to jump through to get a JSContext and
GlobalObject from History.cpp (which is discussed in the earlier
commit in the series).
MozReview-Commit-ID: LHaBWSylyLI
--HG--
extra : rebase_source : 1190d4f127453cdcb692deb5982e92a93e236b9e
Consuming the new 'page-visited' notification was fairly trivial,
since it was already brought over to onVisits. There's not much to
say about this other than that I'm a little bit uncertain about
all the hoops we have to jump through to get a JSContext and
GlobalObject from History.cpp (which is discussed in the earlier
commit in the series).
MozReview-Commit-ID: LHaBWSylyLI
--HG--
extra : rebase_source : 1190d4f127453cdcb692deb5982e92a93e236b9e
Consuming the new 'page-visited' notification was fairly trivial,
since it was already brought over to onVisits. There's not much to
say about this other than that I'm a little bit uncertain about
all the hoops we have to jump through to get a JSContext and
GlobalObject from History.cpp (which is discussed in the earlier
commit in the series).
MozReview-Commit-ID: LHaBWSylyLI
--HG--
extra : rebase_source : 148555189f73fb29a48296215e367c406f3f0286
Boris Zbarsky
Sebastian Hengst
Edouard Oger
Matthew Noorenberghe
Lina Cambridge
Coroiu Cristina
shindli
Cosmin Sabou
Michael Kaply
Gurzau Raul
ffxbld
Jonas Allmann
Honza Bambas
Arpit
Rob Wu
Barret Rennie
Csoregi Natalia
Noemi Erli
Mark Hammond
Oana Pop Rus
Mark Banner
Kris Maglione
Dorel Luca
Marco Bonardo
Andrew Swan
Myk Melez
Jan-Erik Rediger
Andreea Pavel
irios
vinoth
Ryan Hunt
Eliza Balazs
Timothy Guan-tin Chien
Doug Thayer
Siddhant085
Luca Greco
Bogdan Tara
Thom Chiovoloni
Dave Townsend
irrationalagent
Tiberius Oros
Devika Sugathan
Narcis Beleuzu
Mihir
Pavan Veginati
arthur.iakab