Also ensures only one `ensureEcosystemAnonId()` is in flight at once,
`getEcosystemAnonId()` no longer calls `ensureEcosystemAnonId()` and a minor
fix or 2.
Differential Revision: https://phabricator.services.mozilla.com/D85366
This is done so that if the login manager loses all data sync is "reset",
meaning will do a full reconcile with the server and pull all sync records
down and apply them locally - ie, sync will be ble to recover passwords for
many users.
The sync GUID is stored encrypted, so that even if the login manager data
isn't lost, but the encryption key is, it will still reset and recover.
This will also make restoring an older logins.json a little more reliable for
sync - if an old version is restored then sync should "rewind" itself to the
state it was in the backup.
Differential Revision: https://phabricator.services.mozilla.com/D82663
This extends the current try/catch to cover writing of the dump to the database and reading it back again. It does not cover the verify signature section, as it is expected that it will need to be known if the signature fails.
Differential Revision: https://phabricator.services.mozilla.com/D84234
Building the sync event data has an extra cost on large collections. This bypasses iterations or records in case the client has no listener.
Differential Revision: https://phabricator.services.mozilla.com/D81046
To get this fix: https://github.com/mozilla/application-services/pull/3235
Updated as follows:
```
sed -i 's/e8d7530319fa6c20d9de78d031c9398630eca3cd/61dcc364ac0d6d0816ab88a494bbf20d824b009b/g' services/fxaccounts/rust-bridge/firefox-accounts-bridge/Cargo.toml services/sync/golden_gate/Cargo.toml toolkit/components/extensions/storage/webext_storage_bridge/Cargo.toml toolkit/components/glean/Cargo.toml toolkit/library/rust/shared/Cargo.toml
./mach vendor rust
```
Verified by running the new regression test that I added in the bug:
```
./mach test toolkit/components/extensions/test/xpcshell/test_ext_storage_{local,sync,sync_kinto}.js
```
Differential Revision: https://phabricator.services.mozilla.com/D79628
Specifically:
> Log warning: The log 'Services.Common.RESTRequest' is configured to use the preference 'services.common.log.logger.rest.request' - you must adjust the level by setting this preference, not by using the level setter
This is because the hawkrequest module attempts to explicitly set the level
from the preference value, but these days Log.jsm manages that itself.
Also skips obtaining the log itself until it's needed, as it almost never
actually is.
Differential Revision: https://phabricator.services.mozilla.com/D79633
This new method fetches pending synced changes from the extension
storage store, and passes them to `storage.onChanged` listeners.
This allows extensions that listen for these events to know when a
sync happened, which Kinto supported as well.
To guard against misuse, this method is implemented on a separate
`mozISyncedExtensionStorageArea` interface. To avoid multiple
inheritance (if `mozI{Synced, Configurable}ExtensionStorageArea` both
inherited from `mozIExtensionStorageArea`, which base method is called
when?), both of these internal-ish interfaces now inherit from
`nsISupports` instead.
Finally, because `fetchPendingSyncChanges` can return changes for
multiple extensions, `mozIExtensionStorageListener.onChanged` now takes
the affected extension ID as its first argument.
Differential Revision: https://phabricator.services.mozilla.com/D76976
This new method fetches pending synced changes from the extension
storage store, and passes them to `storage.onChanged` listeners.
This allows extensions that listen for these events to know when a
sync happened, which Kinto supported as well.
To guard against misuse, this method is implemented on a separate
`mozISyncedExtensionStorageArea` interface. To avoid multiple
inheritance (if `mozI{Synced, Configurable}ExtensionStorageArea` both
inherited from `mozIExtensionStorageArea`, which base method is called
when?), both of these internal-ish interfaces now inherit from
`nsISupports` instead.
Finally, because `fetchPendingSyncChanges` can return changes for
multiple extensions, `mozIExtensionStorageListener.onChanged` now takes
the affected extension ID as its first argument.
Differential Revision: https://phabricator.services.mozilla.com/D76976
This commit does the following:
- Feature / Optimization: Check the dump before the cache, instead of
the reverse. The dump is expected to match the requested attachment in
the common case, and checking it first helps with ensuring that the
expected (packaged dump) is used when available.
- Optimization: Defer reading the cached attachment until it's needed.
- Refactor / Feature: Treat a missing `.meta.json` file as a sign that
the attachment dump does not exist, rather than an error.
Previously, if an attachment cannot be downloaded from the network,
that error would be replaced with a generic `DownloadError` (from the
missing `.meta.json` file). This is mostly relevant for telemetry.
- Refactor / Maintainability: Create helper to manage lazy access to the
record and attachment, to ensure that the record and attachment is
only read on demand, and at most once.
- Refactor / Readability: Move the common return value generation logic
to the helper as `getResult`, to avoid the verbose duplication of the
logic. Now the return value fits in one line instead of 5-6 lines.
- Fix test: Rename filename-of-dump.meta.json and fix test expectation
to ensure that the test checks the absence of the file content,
rather than the absence of the meta data file.
Differential Revision: https://phabricator.services.mozilla.com/D76962
In practice, the cache of the attachment downloader can become corrupt
and unusable when IndexedDB breaks. The implementation correctly handled
this case, but there were no tests that verified that it did.
This patch adds test coverage for the scenario of a broken cache,
to ensure that the implementation continues to behave in a sane way.
Differential Revision: https://phabricator.services.mozilla.com/D76961
The crypto API does most of its work on the background thread. There is
no benefit in posting the buffer to a worker thread.
Differential Revision: https://phabricator.services.mozilla.com/D76960
This implements reading the list from remote settings. We only read it at startup if necessary, or on add-on installation.
We do not check for updates - if something is removed, we'll wait until next startup before processing it.
Also adds lots of tests for canOverride as this seems a critical part to get right.
Differential Revision: https://phabricator.services.mozilla.com/D76473
By default, toggling the add-ons engine also toggles extension storage.
However, it's also possible to enable extension storage without add-ons
by setting an override pref. If the override pref is set, we'll treat
extension storage as an engine that can be toggled independently. If
it's not set, we'll ignore attempts to toggle the engine separately.
Differential Revision: https://phabricator.services.mozilla.com/D76635
This implements reading the list from remote settings. We only read it at startup if necessary, or on add-on installation.
We do not check for updates - if something is removed, we'll wait until next startup before processing it.
Also adds lots of tests for canOverride as this seems a critical part to get right.
Differential Revision: https://phabricator.services.mozilla.com/D76473
Alternative engines are registered using the lowercased version of
the keys in the modules object. But `extension-storage` is hyphenated,
so we need to use `Extension-Storage` (not `ExtensionStorage`) as the
key name, to match the name of the engine and its collection.
Without the hyphen, we'll register the alternative as
`extensionstorage`, so it'll never be used because everything else
expects the engine to be called `extension-storage`.
Differential Revision: https://phabricator.services.mozilla.com/D76355
This commit fixes two issues: moves `trackRemainingChanges` to the
engine, since that's where it defined (not on the store), and skips
setting `modified` if all records have been successfully uploaded.
Differential Revision: https://phabricator.services.mozilla.com/D76270
This matches how the `Dispatch(already_AddRefed<nsIRunnable>)`
overloads work in C++: `Dispatch` takes ownership of the runnable, and
leaks it if dispatch fails—because the thread manager is shutting down,
for instance. This avoids a race where a runnable can be released on
either the owning or target thread.
Rust doesn't allow arbitrary `Self` types yet (see
rust-lang/rust#44874), so we need to change `dispatch` and
`dispatch_with_options` to be associated methods.
Differential Revision: https://phabricator.services.mozilla.com/D75858
The tracker base class currently does two things: bump the score in
response to observer notifications, and store a list of changed IDs.
The bookmarks, form autofill, and now bridged trackers need to hack
around this to opt out of persistence, since they handle change
tracking in the storage layer.
This commit keeps the score logic in `Tracker`, but moves all the
persistence code into an intermediate `LegacyTracker` class, and
changes all engines that need persistence to inherit from it.
`ignoreAll` is more interesting. We want new-style stores to emit
observer notifications with change sources, so that the tracker knows
to ignore changes made by Sync. Ignoring all observer notifications
during a sync is a blunter version of this. But, not every new store
supports change sources, so we reimplement `ignoreAll` manually for
ones that don't.
Differential Revision: https://phabricator.services.mozilla.com/D74374
To use a single transaction for `importJSONDump`, this commit:
- changes IDBHelpers' `executeIDB` method to take either a string or array
pointing to `objectStore`s that the caller wants to use.
- uses that from RemoteSettingsWorker to start a single transaction using both
the `records` and the `timestamps` store
- updates `bulkOperationHelper` to take an optional `completion` callback, in
addition to the rejection callback, to be called when all the bulk
operations are complete
- uses that optional argument from RemoteSettingsWorker's `importDumpIDB`
(the actual implementation of IDB access from `importJSONDump`) to first
bulk-import the actual records, and then update the timestamp stored for
that remote settings collection.
Then to abort that single transaction, this commit:
- stores pending transactions in a set, similar to what Database.jsm already
does, and removes items from that set when the `promise` from `executeIDB`
either resolves or rejects.
- adds a `prepareShutdown` action on the RemoteSettingsWorker's `Agent` class,
to be called by the jsm side of the worker manager when shutdown happens.
When called, it iterates over the pending transactions and aborts all of
them.
This also sets a `gShutdown` flag.
- ensures that where code `await`s in the middle of an operation, it stops
(throws) immediately if `gShutdown` has been set.
- adds a test to test_shutdown_handling.js to verify that this mechanism now
stops pending import tasks in the worker.
Finally, as a driveby, fixes an oversight in test_remote_settings_worker.js
where the second `.get()` call wasn't actually testing whether the
`importJSONDump` call in the worker had succeeded, because if the collection
was empty it would do the import itself, which I realized when I used similar
code in the shutdown test...
Differential Revision: https://phabricator.services.mozilla.com/D74315
Mihai Alexandru Michis
Dão Gottwald
ffxbld
lougeniac64
Luca Greco
Mark Hammond
Ryan Kelly
Brindusan Cristian
Mozilla Releng Treescript
Mark Banner
Doug Thayer
Vlad Filippov
Kris Maglione
R. Martinho Fernandes
Mathieu Leplatre
Simon Giesecke
Csoregi Natalia
Razvan Maries
Edouard Oger
Cosmin Sabou
Rob Wu
Thom Chiovoloni
Lina Cambridge
Dorel Luca
Bogdan Tara
Sid Stamm
Florian Quèze
Noemi Erli
Michael Kaply
Vijay Budhram
Gijs Kruitbosch