Version 4.0.0 removes the default five-second timeout for all HTTP
requests. This should hopefully fix bug 1329069, bug 1335519, and bug
1333677. It should also make us more robust "in the wild" against
laggy connections or slow servers. We don't need to worry too much
about instituting our own timeout because Necko will impose a timeout
for us according to the preferences network.http.response.timeout and
network.http.keep-alive.timeout.
Version 3.0.0 updates some dependencies and changes some polyfills
around. See https://github.com/Kinto/kinto-http.js/pull/158 for more
details.
MozReview-Commit-ID: 4eAwghOpqfE
--HG--
extra : rebase_source : 9657c3b3f9f0e905845933c123fe099c4408dd29
Some of our tests call `Svc.Prefs.resetBranch("")` or
`Service.startOver()`, which resets the validation prefs to their
default values. This patch makes it a bit more explicit which tests
perform a full sync, and should run validation.
MozReview-Commit-ID: 7a0wR8nuJeW
--HG--
extra : rebase_source : 18ff025671c9dd5e2e30d441940f5c2fb804cae3
This is the "repair responder" - it handles a "repairRequest" command sent
by another client and attempts to take the list of IDs that client lists as
missing and upload whatever records are necessary such that the requesting
client would then be likely to find a complete and valid tree on the server.
MozReview-Commit-ID: 4xw19nH6EfL
This patch defines the concept of a "doctor" for collections. The doctor is
responsible for running all validators and deciding whether or not to initiate
a repair request based on the validation results.
MozReview-Commit-ID: 6NLRE6L0OpA
This creates a collection_repair module, somewhat analogous to the existing
collection_validator module. This defines the public interface to request a
new repair and respond to a remote repair request, and also includes changes
to clients.js to call this public interface.
MozReview-Commit-ID: 9JPpRrLgFoR
This patch is generated by the following sed script:
find . ! -wholename '*/.hg*' -type f \( -iname '*.html' -o -iname '*.xhtml' -o -iname '*.xul' -o -iname '*.js' \) -exec sed -i -e 's/\(\(text\|application\)\/javascript\);version=1.[0-9]/\1/g' {} \;
MozReview-Commit-ID: AzhtdwJwVNg
--HG--
extra : rebase_source : e8f90249454c0779d926f87777f457352961748d
Get rid of the ugly hack where test code monkeypatches a singleton to
convince the rest of the ExtensionStorageSync code that a fake user is
logged in. Instead, take a handle to the fxAccounts service at
construction time. Test code can provide any kind of fxAccounts it
wants, including one that has a hard-coded user.
This provokes a bunch of changes:
- ExtensionStorageSync is now no longer a singleton, but a class. You
have to instantiate it in order to do anything with it. A global
instance extensionStorageSync is provided for use by Sync code as
well as WebExtension code.
- CryptoCollection is now also a class, and each ExtensionStorageSync
instance has its own CryptoCollection. This dependency should maybe
also be injected, but for the time being it doesn't provide us any
value to do so.
- Converting singletons with asynchronous methods to classes is a pain
in the neck. We convert async method foo from `foo:
Task.async(....)` to `async foo() { .... }`. While we're here,
convert KeyRingEncryptionRemoteTransformer#decode to async/await to
eliminate a need for `let self = this`.
- Update Sync code and WebExtension code to use extensionStorageSync.
- There's a cyclic dependency where CryptoCollection#sync depends on
ExtensionStorageSync#_syncCollection which depends on
CryptoCollection#getKeyRing. As a short-term hack, we now require an
ExtensionStorageSync argument to CryptoCollection#sync.
- KeyRingEncryptionRemoteTransformer now takes a handle to fxAccounts
on construction time as well. Because this is the only
EncryptionRemoteTransformer subclass that accesses fxAccounts, we
can get rid of the hack where the tests monkeypatch something in the
EncryptionRemoteTransformer prototype.
- CollectionKeyEncryptionRemoteTransformer now takes a handle to a
CryptoCollection, rather than relying on a global one.
- A bunch of methods that previously assumed access to fxAccounts now
check if fxAccounts is present (i.e. if we're on Android). Strictly
speaking, this isn't required by this change, but it helps make
certain kinds of failure a little easier to diagnose.
- Update tests, including extracting a domain-specific helper method
to hide the use of CollectionKeyEncryptionRemoteTransformer. We now
no longer monkeypatch in our mock fxaService, but pass it to the
test so that it can do whatever it wants with it. We also provide an
ExtensionStorageSync instance for convenience. Access to the global
cryptoCollection is now done through an ExtensionStorageSync
instance.
To summarize, we've gone from a situation where a bunch of singletons
had implicit dependencies on other singletons in a shared global
namespace, to a situation where dependencies are provided explicitly
using method/constructor arguments. This highlights some of the
dependencies present:
- ExtensionStorageSync depends on CryptoCollection and fxAccounts if
it needs to sync
- Every collection created via openCollection needs a handle to
CryptoCollection so it can correctly create its remote transformers
- CryptoCollection needs a handle to fxAccounts so it can create its
own remote transformer for its special collection
Most of this is only possible, or at least much easier, because we no
longer try to juggle Sqlite connections but just keep one around
forever.
However, please note:
- CryptoCollection needs a handle to ExtensionStorageSync to actually
perform syncing logic because that's where we foolishly put the
logic to make requests
- There's still a backing Sqlite store which is shared by everything
- There's still a singleton tracking contexts that opened extensions
which we manage to try to clean contexts up correctly
MozReview-Commit-ID: DGIzyRTdYZ1
--HG--
extra : rebase_source : 305a98b8e1ece3ae7a5e8a2fdb5c58d8e7c496d9
Since services/sync doesn't ship on Android, this meant
conditionally-defining some variables such as `cryptoCollection` and
`CollectionKeyEncryptionRemoteTransformer` depending on whether or not
we were on Android. However, none of these definitions really rely on
functionality that isn't present on Android (although you can't really
use them yet either). Move the dependency together with the dependant
code so we can simplify a bit. This lets us remove conditional uses of
`cryptoCollection` and `CollectionKeyEncryptionRemoteTransformer`.
Because the WebExtensions source directory has more stringent eslint
rules, we end up reformatting and commenting a bit in addition to
moving.
MozReview-Commit-ID: 2ddDeymYFNi
--HG--
rename : services/sync/tests/unit/test_extension_storage_crypto.js => toolkit/components/extensions/test/xpcshell/test_ext_storage_sync_crypto.js
extra : rebase_source : f84c0810b6a447f2104f96c9dc9d6139743f269c
This does a sha256 of record IDs, the same way we do for collection
IDs, during encryption.
The way we compute the new ID (using an overridden method) is a little
bit of a hack, but we use the new ID as part of the HMAC. This also
invalidates a previous assumption, which is that we kept record IDs
the same during decryption.
MozReview-Commit-ID: Gbzlo9OE1he
--HG--
extra : rebase_source : 3a303cce50e60ecf0cd709da4ebef5ff98bff314
extra : intermediate-source : 033163eeac84466553d4f9636093c3b824d7cf2b
extra : source : c5c115baff4a729b064e74e87821f7ee3b9d3edb
Camouflage deletes as updates, according to the rules given by the new
RemoteTransformer contract.
MozReview-Commit-ID: CwVJSs2jOil
--HG--
extra : rebase_source : 88f8e1b4db0b6b7aea4ae5dcfc67af906cb01009
This allows us to transform deletes. See
https://github.com/Kinto/kinto.js/pull/640/.
MozReview-Commit-ID: 3EWcxW4Ny6s
--HG--
extra : rebase_source : 67cd78dcde8a220c7939486599f315c93b9092dc
This patch also refactors `compareServerWithClient` and
`inspectServerRecords` to periodically yield back to the main thread,
since validating a large number of bookmarks in a tight loop causes
jank. Yielding every 50 records for 50ms is somewhat arbitrary, but
keeps things snappy for my profile (2947 records).
MozReview-Commit-ID: 99DJtNvNeHI
--HG--
extra : rebase_source : cc0991106c17ce906b8779afbe9eced0e27cc4b4
For components also referencing it in code, see the blockers of bug 1336311.
MozReview-Commit-ID: 4tUZ24HKBWy
--HG--
extra : rebase_source : ec16149f525b9b7eaca7f96f1369929d21497121
This patch switches to using preferences written by the clients engine to
determine the number of clients so it's correct before the first sync for
a browser session.
It also fixes another bug I discovered - if a mobile device dropped from the
client list, the preference for the number of mobile devices would remain at
1 rather than resetting to zero.
MozReview-Commit-ID: IPythSQcorx
--HG--
extra : rebase_source : 4ce465e3958d037f90312553a0c353cdc161c0fe
Mark Hammond
Wes Kocher
David Rajchenbach-Teller
Carsten "Tomcat" Book
Florian Quèze
Joseph Cameron
Edouard Oger
BharatR123
Joel Maher
Ethan Glasser-Camp
Kit Cambridge
ffxbld
Zibi Braniecki
Thom Chiovoloni
Kris Maglione
Mathieu Leplatre
Gijs Kruitbosch
Masatoshi Kimura
Jorg K
Phil Ringnalda
Mayank Srivastav
Jonathan Hao
Aman Dwivedi
Ahsan
Sebastian Hengst
stefanh@inbox.com