eb8a547f22
Avoid undefined property warnings by checking for null first instead of comparing.
2009-11-04 15:12:29 -08:00
c803c9f499
bug 485108 - drop status dialog, show status including engines on statusbar, remove dead status bits
2009-10-28 03:42:35 -04:00
cafcb53243
Ignore sortindex when comparing if records are different because it's only used for ordering of records.
2009-10-16 16:18:38 -07:00
7fcf4fdde6
Bug 522077 - Download a certain total number of items each sync instead of fixed "catch up"
...
Limit the initial the first fetch of new items by a total number of fetch and subtract the number of items processed. Use the difference to keep fetching more items from the backlog in chunks.
2009-10-13 11:20:28 -07:00
3f44aa854b
Split multiple id deletes into 100-id chunks instead of hitting max apache URI length.
2009-10-12 18:11:31 -07:00
53ada6a89f
Bug 520215 - Only wipe if data is able to be restored from the server
...
For each engine, try decrypting a record from the server before wiping local data.
2009-10-12 16:22:54 -07:00
39f90ad4f7
Simplify logic for Engines.getAll/Enabled and use Enabled for main sync loop.
2009-10-08 13:51:22 -07:00
9c2378f997
Don't unconditionally forceGC on processIncoming especially when there's nothing to process.
2009-09-25 18:39:24 -07:00
e55480bf35
Sort the sync timing report to show certain functions first.
2009-09-25 18:38:47 -07:00
0d568192bf
Bug 518972 - Only upload history records that have more than one visit
...
Allow each engine to provide a custom Collection object and have History provide a collection that filters out certain data. This is inefficient because we have to first create then encrypt the record before we can filter it out.
2009-09-25 16:52:12 -07:00
736efc571f
Bug 518018 - Default to serverURL instead of userAPI on 404 cluster check
...
Rework server/user/misc prefs to allow relative paths and full urls for generating API paths. Cache string properties of generated URLs under the storageAPI instead of using dynamic getters.
2009-09-21 17:13:41 -07:00
56b977bd62
Merge weave to about-weave.
2009-09-16 19:16:48 -07:00
dea85e923c
Remove meta serialization now that resource takes objects.
2009-09-16 19:15:55 -07:00
2625b8a1fb
initial patch
...
--HG--
extra : rebase_source : 250c8f0d85483300147a6f0d691e0214db6d837b
2009-09-15 21:54:05 -04:00
cc109400c6
bug 511549 - make detailedStatus much smarter about errors and backoff, r=edilee
...
--HG--
extra : rebase_source : 42e3eb9c89d2567d18c02e7583202e284dd69b0e
2009-09-15 21:38:52 -04:00
8973822877
Fix exception name.
2009-09-11 13:10:19 -07:00
a1a35caae8
Add some comments to partial download code and remove other comments.
2009-09-11 08:24:42 -07:00
1a4b100a73
Merge weave to weave-partial-view.
2009-09-10 23:18:33 -07:00
439d963fbf
Bug 507429 - Partial sync download support
...
Only fetch a limited number of items on first/update syncs and if we get the same number, ask the server for the ids to fetch later. Also on every download, process some of the backlog and save the list of GUIDs to disk as json for cross-session support.
2009-09-10 23:11:33 -07:00
da76f65a99
Remove some unnecessary logs that follow right after the notify/event now that we only show verbose. Only log non-success response body for Trace.
2009-09-10 22:57:36 -07:00
a364a3c310
Merge weave to weave-partial-view.
2009-09-10 12:42:29 -07:00
695967c675
Update lastSync to the lastModified time from info/collections instead of updating times on each record. This is to prepare for fetching a list of GUIDs and processing a list of GUIDs.
2009-09-10 12:41:38 -07:00
554e7266d1
Bug 515676 - Allow giving objects to PUT/POST without Filters to avoid stringify([parse(stringify(stringify(obj)))])
...
Get rid of Filters and automatically JSON.stringify PUT/POST data that aren't strings, so plain Records can be passed in to PUT and POST. This leverages toJSON of Records to provide an object that can be serialized. Fix up client record serialize/deserialize to still escape/unescape non-ASCII.
2009-09-10 11:05:13 -07:00
890c3d342c
Merge weave to weave-partial-view.
2009-09-08 23:35:21 -07:00
57b041bbe1
Expose a _handleDupe on engines and provide a custom one for bookmarks that tracks GUID changes so that it can keep an alias mapping to fix incoming item properties (id, parent, predecessor). Move out _reparentOrphans so that it is triggered on update and not just create because folders can change ids to the right parent.
2009-09-08 23:33:15 -07:00
54ad191966
Bug 514323 - Use info/collections to check if there's new data. r=thunder
...
Fetch info/collections before syncing engines and set the lastModified property on the engine so that they can check if they need to process incoming.
2009-09-03 21:30:40 -07:00
15ce110350
Bug 514600 - Track how long various parts of sync take
...
Instrument all functions that are part of the sync engine (except some constructors, etc.) and generate statistics (min/max/sum/num/avg) for processing. For now with the default appender, implement toString to report just the total time.
2009-09-03 20:11:36 -07:00
e95c1122bc
Switch to 0.5 server API for deleting multiple ids from a collection.
2009-08-31 18:30:44 -07:00
5f8f947cfd
Merge weave to weave-partial-view w/ conflicts.
2009-08-31 18:03:35 -07:00
b0e2b6f92e
Pick a canonical guid to keep in a way that all machines can agree without having all data -- "smaller" guid.
2009-08-31 17:51:26 -07:00
60f942b3ea
Implement multiple ids deletion for use with 0.3 APIs.
2009-08-31 17:50:23 -07:00
d09d8a37e6
Switch end-of-sync deletes to something SyncEngine is aware of and runs at _syncFinish.
2009-08-31 17:30:34 -07:00
6fc3d9dd75
Convert uses of recordLike/findLikeId to findDupe except Bookmarks is unimplemented.
2009-08-31 17:27:30 -07:00
7c9ca97a64
Resolve lastChannel removal and 0.5 api changes. r=thunder
2009-08-26 16:09:48 -07:00
9be9fdd6c3
Merge weave to weave-0.5-api w/ conflicts.
2009-08-26 15:42:05 -07:00
771c9805e7
Bug 511746 - Resource.foo shouldn't throw except in exceptional cases. r=thunder
...
Get rid of lastChannel and return a String object from _request with additional properties of status, succeeded, headers -- even if the response was handled by cache. Update engines to check for non-success and throw the failure. Update tests to use these additional properties instead of lastChannel, etc.
2009-08-26 15:32:46 -07:00
852bb71758
Backed out changeset 129ca9a54aed due to burning
...
test_auth_manager: FAIL
test_resource: FAIL
2009-08-26 01:50:36 -07:00
adf361843a
Change Resource.get() semantics and support X-Weave-Alert (bug #478330 )
...
--HG--
extra : rebase_source : e0eb1e34f85ddd041005d780590640005dc0b434
2009-08-26 00:05:57 -07:00
6dce762690
Read out the timestamp of the POST response header because the modified time is not part of the response data.
2009-08-25 18:04:46 -07:00
c37a0a3d85
Bug 507433 - Update client to use the weave 0.5 server. r-Mardak
...
Initial 0.5 server API switch not quite working.
2009-08-25 17:06:13 -07:00
fbdd3f2a85
Don't include changes to special places Library "exclude from backup" items.
2009-08-25 16:15:05 -07:00
aec89471ca
Conditionally log trace records to avoid always doing record.toString().
...
--HG--
extra : rebase_source : 65d8025324ed95eac29d530e2699a2f29c8a2cb7
2009-08-20 17:00:15 -07:00
26702591c1
Print out the stack when failing to apply an incoming record.
2009-08-16 12:39:23 -07:00
783223115f
Get rid of depth and sort on index instead.
2009-08-15 00:56:27 -07:00
5f6b7af2ff
Remove meta/mini records. Bye!
2009-08-13 18:50:54 -07:00
b0e99b50a1
Bug 506795 - Handle adding duplicate logins. r=thunder
...
Refactor reconcile to call findLikeId that by default will look for recordLike in the outgoing queue as it does now. Override findLikeId for password engine to search local logins.
2009-07-28 10:06:02 -07:00
6293c974de
Convert various debug messages to trace.
2009-07-22 21:40:18 -07:00
54314c829e
Bug 505906: Don't do a last POST if there is nothing left to send. Regression from bug 481347. r=mardak
2009-07-22 17:48:47 -07:00
ac355ae207
Bug 485573 - Fennec runs out of memory and crashes if too many history items to sync. r=thunder
...
Incrementally process records as the collection finds record boundaries and converts them to records for the engine to use. Get rid of the collection iterator and original RecordParser. Add tests for incremental record parsing and remove old iter tests.
2009-07-22 16:38:34 -07:00
ddefe825fc
Bug 481347: Split uploads into multiple chunks to get around server upload limit and curtail memory usage
2009-07-22 16:21:33 -07:00
ac496a8ff4
Bug 504387 - Cache keys after creating/PUTting them instead of after GETting. r=thunder
...
Set the record manager for CryptoMetas, PubKeys, PrivKeys after uploading keys.
2009-07-16 23:13:58 -07:00
c75387d5ab
Bug 504230 - Use Engine's logging level for Store and Tracker
...
Use the log.logger.engine.* prefs to determine what the Store and Tracker should log with. This requires putting the same engine name on each Store/Tracker object, so there's some duplicate string values right now.
2009-07-14 18:34:03 -07:00
21ad9fd3dc
Bug 504212 - Have javascript stack traces show [object Object] for various CryptoWrapper functions
...
Pass the ID object instead of reading out the password and passing it in to encrypt/decrypt.
2009-07-14 16:51:04 -07:00
5abbdfd658
Bug 504177 - Don't block the UI when syncing up/down
...
Sync.sleep(0) to let the main thread do UI, etc. after every record encrypt or decrypt.
2009-07-14 14:01:26 -07:00
baa6609a31
merge weave. resolve addition of kSpecialIds to bookmarks.js near removal of Async.sugar.
2009-06-23 22:34:57 -07:00
ce951c538a
Bug 493816 - Timestamps need to be saved as string prefs
...
Internally store .lastSync as a string but keep exposing set/getters as float values. parseFloat takes both strings and numbers and gives a number.
--HG--
extra : rebase_source : b657dd3f732b9f3b3b0ef208ee53e799aa87d2c6
2009-06-16 17:22:59 -07:00
4ac23d2cab
Remove async.js and wrap.js and remove remaining references to Async.sugar, etc.
2009-06-06 18:24:30 -05:00
51c0ce9b4b
Remove Async.sugar from engines.js.
2009-06-05 15:34:39 -07:00
4ed0e2e9db
Engine_wipeServer: async -> sync. SyncEngine__wipeServer: async/yield -> sync.
2009-06-05 15:34:37 -07:00
301d62f2bb
Engine_wipeClient: async -> sync. Engine__wipeClient: async/yield -> sync.
2009-06-05 15:34:35 -07:00
a5bc6c458e
Engine_resetClient: async -> sync. SyncEngine__resetClient: async/yield -> sync.
2009-06-05 15:34:34 -07:00
ddede5e342
Engine_sync: async -> sync. SyncEngine__sync: async/yield -> sync.
2009-06-05 15:34:32 -07:00
e5c7999e86
Temporarily convert _notify to _notifyAsync and add a Utils.notify.
2009-06-05 15:34:23 -07:00
6d842a391a
SyncEngine__processIncoming: async/yield -> sync.
2009-06-05 00:39:35 -07:00
a242dd68b6
CollectionIterator_next: async + async/yield -> sync.
2009-06-05 00:38:23 -07:00
667c064e18
Store_applyIncoming, SyncEngine__applyIncoming: async + async/yield -> sync.
2009-06-05 00:36:11 -07:00
cb6f639b26
SyncEngine__syncStartup, SyncEngine__uploadOutgoing, SyncEngine__syncFinish all async/yield -> sync.
2009-06-04 23:48:27 -07:00
8f9a249f1f
CryptoWrapper_decrypt: async + async/yield -> sync. Fix up bookmark/client decrypt.
2009-06-04 19:06:57 -07:00
e7babd76eb
CryptoWrapper_encrypt: async + async/yield -> sync.
2009-06-04 17:58:37 -07:00
26e278509c
CryptoMeta_addUnwrappedKey: async + async/yield -> sync.
2009-06-04 17:36:34 -07:00
18ddfe6cf0
PubKeyManager_getDefaultKey: async + async/yield -> sync.
2009-06-04 17:04:51 -07:00
de24904b92
RecordMgr_get: async + async/yield -> sync.
2009-06-04 16:50:57 -07:00
6f2fce527b
Change SyncEngine._reconcile to not be async/yield.
2009-06-04 14:18:04 -07:00
3e935d30d1
Fix Resource.* call sites used for wiping the server and initial sync (+ key gen upload)
2009-06-04 10:17:27 -07:00
5220bc6208
Change Resource.get() to be sync (no callback) and fix up call sites used for login + sync now.
2009-06-04 10:16:07 -07:00
a76192895b
Bug 493256 - Reconcile doesn't handle removed items correctly. r=thunder
...
Have engines check if the deleted flag is the same for _isEqual.
2009-05-15 13:18:16 -05:00
a5f88e66c2
Bug 493001 - Dynamically generate weave prefs engine list based on registered engines
...
Remove existing hardcoded engine list + prefs and build checkboxes based on registered engines. Engines that give null for enabled (instead of true/false) are considered disabled. Update base Engine enabled get/set to give null and allow setting.
2009-05-14 11:38:15 -05:00
0d0030a423
skip getting meta records for records already in the meta list
2009-04-08 18:08:45 -07:00
d6b09727c8
Bug 486810 - Engines (and their trackers) are created with every new window. r=thunder
...
Register the built-in engines on service start-up instead of from the overlay, and have Engines.register check if the engine has already been registered.
2009-04-08 14:39:14 -05:00
60afb4ea7c
Bug 487308 - Allow registering of an array of engines
...
Import engines to the Weave global object and use them to register engines, which checks if the arg is an array. To support handling of errors (unused), the engine is returned on register failure.
2009-04-07 16:45:41 -05:00
9a4da9c571
Add a Utils.isArray and use it
2009-04-07 16:45:29 -05:00
a9b1c2cc31
Bug 482670 - WBORecord should implement a .deleted property
...
Expose a .deleted property that engines can set to true to store an even thinner deleted payload (empty string "" instead of "[null]") on the server. Handle deserializing of deleted records by setting the property. Note: Engines must set their payloads to something JSON-able if it's not a delete record.
2009-04-03 12:38:47 -05:00
3cead7f04b
Bug 481741 - Switch to JSON from nsIJSON
2009-03-31 14:09:38 -05:00
04f5b8828b
Bug 482898 - Allow an optional array of engines for wipeRemote, wipeServer, resetClient, etc.. r=thunder
2009-03-20 00:13:16 -07:00
e82eb9e0da
self.cb not this.cb for engine._wipeClient
2009-03-11 14:35:23 -05:00
f40f88e033
Bug 478327: Send minimal '[null]' payload for deleted records
2009-03-11 01:40:04 -07:00
1a083160c8
Bug 480457 - Expose a resetClient API for weave service and engines. r=thunder
2009-02-26 22:36:14 -08:00
ef43c50062
add some trace-level debugging to reconciler; ignore old (changed) IDs while reconciling
2009-03-03 01:29:35 -08:00
85834524e0
Switch away from __defineGetter__ due to bug 481104; explicitly cache logins in password engine; whitespace fixes
2009-03-02 18:55:26 -08:00
fd1d0e8eef
Remove getter that fixes the 'two-store' problem
2009-03-03 01:15:48 +01:00
7365b0d561
Bug 480461: skip items that cannot be decrypted
2009-02-26 17:31:06 -08:00
1557586ebb
remove unused observer service getter
2009-02-26 13:52:12 -08:00
e8c14a7b1b
reset lastSync pref before setting it, as some clients have mismatched types saved
2009-02-23 19:33:40 -08:00
34b3a01ca1
Wrap.notify now takes the topic prefix as an argument, instead of requiring this._osPrefix to be set; use Observers.js in several places (sync.js, status.js, etc); some event topics have changed, beware; fix up status window, now prints some useful status as sync progresses
2009-02-22 00:04:58 -08:00
0e3361e842
shorten engine logging pref names, set defaults for them
2009-02-20 01:51:20 -08:00
9a6bb0d0f4
pass the cryptoMetaURL to the store's createRecord
2009-02-19 00:36:55 -08:00
393dc449e1
make sure to only save the last sync timestamp as a number
2009-02-18 20:17:30 -08:00
5cf57ecddd
add a method for resetting the engine's lastSync pref
2009-02-17 13:20:02 -08:00
0458699895
add '-engine' to engine name in observer notifications
2009-02-11 21:51:24 -08:00
c8d063e84a
remove sm-weave-proxy01 hack
2009-02-11 13:47:51 -08:00
8a357105f7
multi-cluster support; use new 'newer' flag instead of 'modified' for collection searches
2009-02-10 00:57:16 -08:00
ec75cd2b3d
set record IDs inside the store; fix bookmarks tracker, it was ignoring change events (e.g., changed title and such)
2009-01-27 16:54:54 -08:00
cb5343de85
only url-encode weave IDs when constructing a URL with them
2009-01-27 16:36:00 -08:00
3cedb09d82
Make records not extend Resource, instead they have a serialize() method that can be used to create a Resource for uploading if necessary.
...
Use global service instances under Svc.*
Consolidate various record managers (caches/convenience factories) to extend the same base object
Log exceptions caught by notify wrapper
2009-01-27 13:35:10 -08:00
64399820c2
cast modified timestamp to an int if it's a string
2009-01-21 19:02:38 -08:00
0b5d7072fd
Merged
2009-01-15 14:06:00 -08:00
bb5eead71b
Added a method to historyStore called tempTableExists which does a SELECT (rather than using tableExists) to correctly tell whether a temp table is there or not, so that historyEngine will work correctly on both Fennec and Firefox (See bug 472963)
2009-01-15 14:05:50 -08:00
1420201b13
add some extra log info to engine; have tracker ignore all changes when applying a change to prevent the tracker from generating new guids for new items before the store has a chance to set the right one
2009-01-14 22:01:04 -08:00
5a6cb633ba
ignore/unignore specific weave IDs instead of a blanket enable/disable of the tracker during sync.
2009-01-13 14:43:21 -08:00
928336472f
skip part of reconciliation for incoming deleted records
2009-01-08 21:33:37 -08:00
3e51b66d87
fail recordLike if either record is null (deleted)
2009-01-07 13:05:05 -08:00
3079faacf5
add clients sync (list of clients with names and types); make it simpler to create Engine subclasses; remove the 2nd 'authenticator' parameter for Resources; resetServer is now wipeServer; consider 0 an invalid place ID (fixes bug when calculating item depths); add a Svc global exported from utils.js where commonly used services can live
2009-01-06 13:54:18 -08:00
f706c5158c
fix depth/index record generation
2009-01-02 21:13:32 -08:00
84f66ce8dc
drop _isEqual debug statements to trace level
2009-01-02 17:35:47 -08:00
73622e4087
consider sortindex when checking if a remote record is the same as a local one
2009-01-02 17:27:45 -08:00
9d56d21abe
remove extra debug code
2009-01-02 16:17:36 -08:00
d591998b22
fix/further simplify reconciliation
2009-01-02 15:51:35 -08:00
0414d1f146
remove 'outgoing' record cache (generate outgoing records at send time); clear the store cache and force a GC after reconciliation; push depth+index (metadata) records after regular (full) records; simplify + better comment reconciliation function; fix low mem check
2009-01-02 13:51:38 -08:00
b91c0d5409
typo fix
2008-12-30 23:56:53 -08:00
08400bd281
add a generic cache class for storing records during reconciliation. cache has 100 item limit, and is cleared before and after reconciliation so the OS can reclaim memory after GC
2008-12-30 23:52:20 -08:00
90bae8f088
make engine keep less records in memory by limiting the outgoing queue to a maximum of 100 records, and fetch the rest from the store each time
2008-12-29 23:28:17 -08:00
bf5efd53ba
switch to generating individual bookmark records directly from the places store, without using a cache (to improve memory performance); create objects for bookmark types; temporarily disable history sync
2008-12-28 19:59:44 -08:00
1543c71b58
force GC, makes sync of many items possible on fennec
2008-12-26 21:49:17 -08:00
5c873c8695
change serverURL to use 0.3 url schema
2008-12-23 13:51:30 -08:00
3bf24e2364
Various engine/bookmark changes:
...
* Rely on the server to sort incoming records, remove all sorting code client-side.
* Streamline sync to be able to process incoming records one at a time, as soon as they are downloaded. This changes reconciliation to be able to process a single incoming record.
* Engine base class will automatically convert parentid, depth, and sortindex into toplevel WBO objects. This is good for now but kinda broken, engine subclasses should really be generating WBO records themselves.
* Since index is now a toplevel WBO property, there is no need for the bookmarks sync code to subclass recordLike.
* Refactor bookmarks store to be a little cleaner and work directly with records instead of "commands".
2008-12-23 11:30:31 -08:00
b043c0298e
change engine to process (download, reconcile, apply changes) incoming records one by one
2008-12-19 11:48:09 -08:00
18166fe10e
implement the awe (or tears) inducing places-style sql queries -- history sync works pretty well now
2008-12-18 22:39:32 -08:00
23baae2006
don't stop applying changes when one fails to do so
2008-12-12 13:53:48 -08:00
6bf68420c8
enable history sync (not working yet)
2008-12-09 12:26:14 -08:00
8e9523ee72
make changeItemID exclusively a store method
2008-12-08 13:21:25 -08:00
e11a3250a5
remove sync cores, they are no longer used; remove sharing code from bookmarks (we can bring it back when we need it again, needs work); have the engine give hints to the store about when to wrap all items and cache them (to make subsequent calls to wrap one item fast); move serializeItem/getAllIDs into the store
2008-12-08 09:53:32 -08:00
810cd9f6fe
clear changed IDs before adding all current ones in first-sync case; edit the ID list directly instead of enabling/disabling the tracker
2008-12-06 00:08:54 -08:00
08e95437cf
parentGUID -> parentid
2008-12-05 03:36:27 -08:00
3cb919b02a
re-enable tracker in the first-sync case just to add all the guids; add some debugging messages; when reconciling throw out records which are identical on the client and server (even if we thought they had changed)
2008-12-05 03:28:17 -08:00
dfc77578a0
oops, always disable tracker during sync
2008-12-05 00:58:25 -08:00
0d6ea4b75d
disable tracker for entire sync run. make sure tracker is re-enabled at the end
2008-12-05 00:55:19 -08:00
a7fe7a2dff
use callbacks instead of snapshot diffs to figure out what changes to send to the server
2008-12-05 00:39:54 -08:00
a10a4d3bfd
add logic to detect when the same item is in both incoming & outgoing queues, but with different IDs - change the local ID in that case
2008-11-27 00:25:28 +09:00
669b52e0de
decrypt incoming items all in one go so as to sort them by index (which is inside the payload); don't attempt to change the index of an item if we're also changing its folder; only change index/parent if it's different from current value
2008-11-26 14:07:18 +09:00
b61f0c6a28
comment depth function for clarity, remove incorrect warning
2008-11-25 16:37:19 +09:00
fe41e68365
sort incoming records by depth, so that folders are created before their containing items
2008-11-25 16:12:00 +09:00
94a267ae9b
use the new modified property returned after a server POST
2008-11-25 14:48:22 +09:00
e33f7592d2
add some debug chatter
2008-11-25 01:48:38 +09:00
89b35d0700
clean up sync function, get last timestamp properly from post result, save correct snapshot at the end, only attempt to post changes to server if there are any changes at all
2008-11-25 01:04:14 +09:00
393262eead
enable batch upload, yay!
2008-11-20 10:38:53 +09:00
f5d4bdee2f
more NewEngine impl
2008-11-20 09:20:25 +09:00
06155f7fae
crypto changes: private keys get their own iv and salt, crypto meta records get a separate iv and no salt. various fixes in crypto code and other wbo related objects. fix crypto tests to match.
2008-11-08 02:00:33 -08:00
5ad74e339a
make sure records always have a data field that represents their state (e.g. for serialization); add some pki routines (createKeypair); add incoming/outgoing queue to NewEngine
2008-11-06 23:23:35 -08:00
928d63611b
add a temp NewEngine class that will talk to the 0.3 server
2008-11-06 19:18:07 -08:00
25a8c72d97
fix log4moz calls (new api), fix module imports for renamed modules
2008-11-03 15:00:38 -08:00
Edward Lee
Mike Connor
Dan Mills
Anant Narayanan
Anant Narayanan
Edward Lee
jonathandicarlo@jonathan-dicarlos-macbook-pro.local