Legacy extensions are no longer loaded, so we can drop the build config for it. We
still need flags for handling experimental APIs since what we require differs between builds
and distributions.
Differential Revision: https://phabricator.services.mozilla.com/D57413
--HG--
extra : moz-landing-system : lando
This is a narrow patch that simply adds a `maxMatches` param to `PlacesRemoteTabsAutocompleteProvider.getMatches`.
Differential Revision: https://phabricator.services.mozilla.com/D50019
--HG--
extra : moz-landing-system : lando
Instead of using the list of FxA devices from the Sync clients engine,
we now fetch the list of Send Tab devices from FxA. This works like
this:
* `FxAccountsDevice#getDeviceList` has been split up into
`recentDeviceList` and `refreshDeviceList`.
* `recentDeviceList` synchronously returns the last fetched list, so
that consumers like Send Tab can use it right away.
* `refreshDeviceList` is asynchronous, and refreshes the last fetched
list. Refreshes are limited to once every minute by default, matching
the minimum sync interval (Send Tab passes the `ignoreCached` option
to override the limit if the user clicks the "refresh" button).
Concurrent calls to `refreshDeviceList` are also serialized, to
ensure the list is only fetched once.
* The list is flagged as stale when a device is connected or
disconnected. It's still kept around, but the next call to
`refreshDeviceList` will fetch a new list from the server.
* The Send Tab UI refreshes FxA devices in the background. Matching FxA
devices to Sync client records is best effort; we don't do it if Sync
isn't configured or hasn't run yet. This only impacts the fallback
case if the target doesn't support FxA commands.
Differential Revision: https://phabricator.services.mozilla.com/D47521
--HG--
extra : moz-landing-system : lando
Most of these tests have been disabled for a long time; they run well
in the current test environment.
Differential Revision: https://phabricator.services.mozilla.com/D46642
--HG--
extra : moz-landing-system : lando
When a local or remote item changed, we'd potentially scan three tables
(with an expensive `LEFT JOIN`!) to check if anything changed...then
scan the same tables again to build the local and remote trees. This
check was originally meant to avoid unnecessary merges. However, the
bottleneck isn't merging now; it's reading from the database.
Since the merger has been rewritten in Rust, is synchronous, doesn't
keep a transaction open for the entire merge (see the
`total_sync_changes` check), and only emits ops for items that actually
changed, it's more efficient to build and merge optimistically, and
bail before applying if nothing changed.
This commit also moves `validateLocalRoots` into Rust.
Differential Revision: https://phabricator.services.mozilla.com/D41690
--HG--
extra : moz-landing-system : lando
This commit introduces a new `Watchdog` class that signals an abort,
either after a delay or at shutdown, and wires up the buffered engine
to use it.
Differential Revision: https://phabricator.services.mozilla.com/D41311
--HG--
extra : moz-landing-system : lando
This commit reduces the number of database writes and table scans
needed to merge synced bookmarks.
* Remove `fetchNew{Local, Remote}Contents`. Fetching the tree already
scans the table, so we can piggyback on it to fetch content info for
deduping.
* Store completion ops in temp tables to only update changed parts of
the local tree, and remove the `mergeStates` table and views.
* Replace the `itemsToMerge` view with an indexed `itemsToApply` temp
table.
* Replace the `updateGuidsAndSyncFlags` trigger with a `changeGuidOps`
table and a `changeGuids` trigger.
* Replace the `updateLocalItems` trigger with an `apply_remote_items`
function to bulk upsert new and updated items.
* Replace the `structureToMerge` view with an
`applyNewLocalStructureOps` table that holds parents and positions
for moved items, and an `applyNewLocalStructure` trigger to update
them.
* Remove tombstones for revived items, update change counters, and flag
mirror items as merged directly in `update_local_items_in_places`,
instead of indirecting through temp tables.
* Don't mark items flagged for reupload as merged, since we'll write
them back to the mirror after upload.
* Use a scalar subquery instead of a join in the `localTags` view to
look up the tags root ID.
* Replace `relatedIdsToReupload` with a `Store::prepare` method that
flags all bookmarks with keyword-URL mismatches for reupload.
* Trigger frecency updates for origins once, not for every item.
* Remove two extra scans on `itemsAdded` and `itemsChanged` when
recording observer notifications for changed keywords.
* Notify all `bookmark-added` listeners in a single batch.
This also fixes some edge cases:
* Update root positions correctly after deleting a non-syncable root
or item.
* Keyword-URL mismatches may reupload more items than before, but now
ensure that all bookmarks with the same URL have the same keyword.
* Only set items with deduped GUIDs to `SYNC_STATUS_NORMAL` after
merging.
* Bump the last modified time for modified items only.
Differential Revision: https://phabricator.services.mozilla.com/D39889
--HG--
extra : moz-landing-system : lando
Specifically, a "control pref" for a pref must already exist locally, or
a new preference, `services.sync.prefs.dangerously_allow_arbitrary` must
be set to true.
This also removes a few preferences from the set we sync by default based
due to potential harm which can be caused syncing inappropriate values.
Differential Revision: https://phabricator.services.mozilla.com/D29775
--HG--
extra : moz-landing-system : lando
This is a test-only change to make the existing synced prefs tests more
explicit about values being checked and moves a couple of prefs around to make
things a bit clearer, which should make the test changes in part 2 clearer.
Differential Revision: https://phabricator.services.mozilla.com/D29774
--HG--
extra : moz-landing-system : lando
Before this commit, we used event telemetry to record timing and counts
for the different steps of a bookmark merge (fetching the local tree,
new local contents, remote tree, and new remote contents; merging;
applying; fetching outgoing records; and notifying observers).
This has several limitations. We need to store all numbers as strings,
include a "flow ID" to tag events from the same merge, and collect
failure reasons twice. We also can't correlate these events to the
existing engine telemetry, meaning we can't see other stats for that
engine, or for the entire sync. Finally, we need to run separate
queries on these events for analysis, instead of extending our
existing engine dashboards.
This approach also feels like an abuse of event telemetry, so this
commit adds a "steps" field for each engine in the Sync ping. Each step
has a name, time taken, and additional named counts, like the number of
items in the tree, or merged structure stats.
Currently, only the `buffered-bookmarks` engine records these steps.
Differential Revision: https://phabricator.services.mozilla.com/D33411
--HG--
extra : moz-landing-system : lando
This commit introduces a `mozISyncedBookmarksMirrorProgressListener`
interface for capturing telemetry and updating shutdown blocker state
after each step of the merge, instead of waiting until the end. This
also means we can also record events for interrupted and failed merges,
and pass validation data through to the Sync ping.
Shutdown hang crash reports now have a `steps` field, indicating the
sequence of completed steps and when they were recorded. If the last
step in the hang report is `fetchLocalTree`, we know the merger is
blocked on `fetchNewLocalContents`. If the last step is
`fetchNewLocalContents`, the merger is stuck at `fetchRemoteTree`,
since that's the next step after `fetchNewLocalContents`.
This commit also implements `Driver::record_telemetry_event` to
dispatch progress callback runnables to the main thread.
Differential Revision: https://phabricator.services.mozilla.com/D31950
--HG--
extra : moz-landing-system : lando
The number of Sync users on these channels is low compared to Release,
so we can do this without a gradual rollout. This also lets more users
test the new bookmark sync engine without manually flipping the pref.
Differential Revision: https://phabricator.services.mozilla.com/D29859
--HG--
extra : moz-landing-system : lando
This commit exports livemarks before syncing for the first time, to
avoid losing livemarks that Sync may have resurrected. As of v0.2.4,
Dogear treats livemarks as non-syncable, and deletes them on both
sides.
This also bumps the mirror schema version, to trigger a first sync.
Differential Revision: https://phabricator.services.mozilla.com/D28543
--HG--
extra : moz-landing-system : lando
- `Array.map` becomes `Array.from`
- Array copying via `Array.slice` becomes `Array.from`.
- `Array.forEach` that did not rely on closures becomes `for`-`of` loops.
- Anything else: `Array.X` becomes `Array.prototype.X`.
Complex cases:
dom/bindings/test/TestInterfaceJS.js and
dom/bindings/test/test_exception_options_from_jsimplemented.html
use `Array.indexOf` to generate an error with a specific error message.
Switched to `Array.prototype.forEach` to generate the same error.
js/src/jit-test/tests/basic/exception-column-number.js
In this test `Array.indexOf()` is used to generate an error. Since the
exact message doesn't matter, I switched to `Array.from()`.
Intentionally not changed:
editor/libeditor/tests/browserscope/lib/richtext/richtext/js/range.js
Did not modify because this is 3rd-party code and the code uses
feature detection as a fall back when Array generics are not used.
testing/talos/talos/tests/dromaeo/lib/mootools.js
Did not modify because mootools adds the `Array.slice` method to the
`Array` object.
Not changed because they check the implementation of Array generics:
js/src/jit-test/tests/basic/arrayNatives.js
js/src/jit-test/tests/basic/bug563243.js
js/src/jit-test/tests/basic/bug618853.js
js/src/jit-test/tests/basic/bug830967.js
js/src/jit-test/tests/jaeger/recompile/bug656753.js
js/src/jit-test/tests/self-hosting/alternate-static-and-instance-array-extras.js
js/src/tests/non262/Array/generics.js
js/src/tests/non262/Array/regress-415540.js
js/src/tests/non262/extensions/regress-355497.js
js/src/tests/non262/extensions/typedarray-set-neutering.js
Depends on D27802
Differential Revision: https://phabricator.services.mozilla.com/D27803
--HG--
extra : moz-landing-system : lando
`Async.jankYielder` is known to, unfortunately, cause jank by creating a lot of
immediately resolved promises that must be then GCed. For a collection of 50
items, it will create 50 promises and 49 of them will immediately resolve.
Instead of `Async.jankYielder`, we now have `Async.yieldState`, which simply
keeps track of whether or not the caller should yield to the event loop. Two
higher level looping constructs are built on top of it:
* `Async.yieldingIterator`, which has been rewritten to not create extraneous
promises; and
* `Async.yieldingForEach`, which is a replacement for awaiting
`Async.jankYielder` in a loop. Instead, it accepts the loop body as a
function.
Each of these can share an instance of an `Async.yieldState`, which allows an
object with multiple loops to yield every N iterations overall, instead of
every N iterations of each loop, which keeps the behaviour of using one
`Async.jankYielders` in multiple places.
Differential Revision: https://phabricator.services.mozilla.com/D26229
--HG--
extra : moz-landing-system : lando
`Async.jankYielder` is known to, unfortunately, cause jank by creating a lot of
immediately resolved promises that must be then GCed. For a collection of 50
items, it will create 50 promises and 49 of them will immediately resolve.
Instead of `Async.jankYielder`, we now have `Async.yieldState`, which simply
keeps track of whether or not the caller should yield to the event loop. Two
higher level looping constructs are built on top of it:
* `Async.yieldingIterator`, which has been rewritten to not create extraneous
promises; and
* `Async.yieldingForEach`, which is a replacement for awaiting
`Async.jankYielder` in a loop. Instead, it accepts the loop body as a
function.
Each of these can share an instance of an `Async.yieldState`, which allows an
object with multiple loops to yield every N iterations overall, instead of
every N iterations of each loop, which keeps the behaviour of using one
`Async.jankYielders` in multiple places.
Differential Revision: https://phabricator.services.mozilla.com/D26229
--HG--
extra : moz-landing-system : lando
`Async.jankYielder` is known to, unfortunately, cause jank by creating a lot of
immediately resolved promises that must be then GCed. For a collection of 50
items, it will create 50 promises and 49 of them will immediately resolve.
Instead of `Async.jankYielder`, we now have `Async.yieldState`, which simply
keeps track of whether or not the caller should yield to the event loop. Two
higher level looping constructs are built on top of it:
* `Async.yieldingIterator`, which has been rewritten to not create extraneous
promises; and
* `Async.yieldingForEach`, which is a replacement for awaiting
`Async.jankYielder` in a loop. Instead, it accepts the loop body as a
function.
Each of these can share an instance of an `Async.yieldState`, which allows an
object with multiple loops to yield every N iterations overall, instead of
every N iterations of each loop, which keeps the behaviour of using one
`Async.jankYielders` in multiple places.
Differential Revision: https://phabricator.services.mozilla.com/D26229
--HG--
extra : moz-landing-system : lando
This commit:
* Uses chunking to insert child GUIDs in `storeRemoteFolder`.
* Changes the mirror schema to store diverging structure, so that it
can be passed to Dogear.
* Sorts remote items by GUID, so that diverging ones can be reparented
in a deterministic order.
* Measures and logs the time taken to run the Rust merger.
* Adds tests for multiple parents and replacing invalid remote items.
* Fixes two tests in `test_bookmark_structure_changes` to ensure the
remote structure is consistent.
Differential Revision: https://phabricator.services.mozilla.com/D26275
--HG--
extra : moz-landing-system : lando
AddonManagergetInstallForURL() has a number of optional arguments, most
of which are passed infrequently. Convert them from positional arguments
to a single options object.
Differential Revision: https://phabricator.services.mozilla.com/D18475
--HG--
extra : rebase_source : 503c09b54fab90cefe69286b05def43ef70074df
***
Bug 1514594: Part 3a - Change ChromeUtils.import to return an exports object; not pollute global. r=mccr8
This changes the behavior of ChromeUtils.import() to return an exports object,
rather than a module global, in all cases except when `null` is passed as a
second argument, and changes the default behavior not to pollute the global
scope with the module's exports. Thus, the following code written for the old
model:
ChromeUtils.import("resource://gre/modules/Services.jsm");
is approximately the same as the following, in the new model:
var {Services} = ChromeUtils.import("resource://gre/modules/Services.jsm");
Since the two behaviors are mutually incompatible, this patch will land with a
scripted rewrite to update all existing callers to use the new model rather
than the old.
***
Bug 1514594: Part 3b - Mass rewrite all JS code to use the new ChromeUtils.import API. rs=Gijs
This was done using the followng script:
https://bitbucket.org/kmaglione/m-c-rewrites/src/tip/processors/cu-import-exports.jsm
***
Bug 1514594: Part 3c - Update ESLint plugin for ChromeUtils.import API changes. r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D16747
***
Bug 1514594: Part 3d - Remove/fix hundreds of duplicate imports from sync tests. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16748
***
Bug 1514594: Part 3e - Remove no-op ChromeUtils.import() calls. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16749
***
Bug 1514594: Part 3f.1 - Cleanup various test corner cases after mass rewrite. r=Gijs
***
Bug 1514594: Part 3f.2 - Cleanup various non-test corner cases after mass rewrite. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16750
--HG--
extra : rebase_source : 359574ee3064c90f33bf36c2ebe3159a24cc8895
extra : histedit_source : b93c8f42808b1599f9122d7842d2c0b3e656a594%2C64a3a4e3359dc889e2ab2b49461bab9e27fc10a7
SyncIntegration firefox-ios tests are failing if the server is wiped in first phase
Differential Revision: https://phabricator.services.mozilla.com/D10219
--HG--
extra : moz-landing-system : lando
ffxbld
Christian Holler
Mark Banner
Siddhant085
Shane Caraveo
shindli
Mark Hammond
Moritz Birghan
rfkelly
Coroiu Cristina
Thom Chiovoloni
lougenia
Cosmin Sabou
Oana Pop Rus
Drew Willcoxon
Lina Cambridge
Sam Foster
Edouard Oger
Andreea Pavel
Geoff Brown
Bogdan Tara
Sylvestre Ledru
Ciure Andrei
Nicholas Nethercote
Jared Wein
Luca Greco
Narcis Beleuzu
Jonathan Kingston
Victor Porof
Marco Bonardo
Boris Zbarsky
Sebastian Hengst
Matthew Noorenberghe
Michael Kaply
Gurzau Raul
Jonas Allmann
Honza Bambas
Arpit
Rob Wu
Barret Rennie
Csoregi Natalia
Noemi Erli
Kris Maglione
Dorel Luca
Andrew Swan
Myk Melez
Jan-Erik Rediger
irios
vinoth
Ryan Hunt
Eliza Balazs