gecko-dev/toolkit/components/places
Kit Cambridge 064ed56076 Bug 1368209 - Refactor `Engine::_processIncoming` into three stages. r=eoger,tcsc
* In the first stage, we fetch changed records, newest first, up to the
  download limit. We keep track of the oldest record modified time we
  see.
* Once we've fetched all records, we reconcile, noting records that
  fail to decrypt or reconcile for the next sync. We then ask the store
  to apply all remaining records. Previously, `applyIncomingBatchSize`
  specified how many records to apply at a time. I removed this because
  it added an extra layer of indirection that's no longer necessary,
  now that download batching buffers all records in memory, and all
  stores are async.
* In the second stage, we fetch IDs for all remaining records changed
  between the last sync and the oldest modified time we saw in the
  first stage. We *don't* set the download limit here, to ensure we
  add *all* changed records to our backlog, and we use the `"oldest"`
  sort order instead of `"index"`.
* In the third stage, we backfill as before. We don't want large deltas
  to delay other engines from syncing, so we still only take IDs up to
  the download limit from the backlog, and include failed IDs from the
  previous sync. On subsequent syncs, we'll keep fetching from the
  backlog until it's empty.

Other changes to note in this patch:

* `Collection::_rebuildURL` now allows callers to specify both `older`
  and `newer`. According to :rfkelly, this is explicitly and
  intentionally supported.
* Tests that exercise `applyIncomingBatchSize` are gone, since that's
  no longer a thing.
* The test server now shuffles records if the sort order is
  unspecified.

MozReview-Commit-ID: 4EXvNOa8mIo

--HG--
extra : rebase_source : f382f0a883c5aa1f6a4466fefe22ad1a88ab6d20
2017-11-01 11:09:57 -07:00
..
tests Backed out changeset e17a2bca5391 (bug 1411891) for failing xpcshell in toolkit/modules/tests/xpcshell/test_NewTabUtils.js r=backout on CLOSED TREE 2017-10-31 14:19:52 +02:00
BookmarkHTMLUtils.jsm Bug 1411368 - Automatically fix space-unary-ops issues raised when using ESLint 4. r=mossop 2017-10-26 13:14:13 +01:00
BookmarkJSONUtils.jsm Bug 1411368 - Automatically fix no-multi-spaces issues raised when using ESLint 4. r=mossop 2017-10-26 11:47:01 +01:00
Bookmarks.jsm Backed out changeset e17a2bca5391 (bug 1411891) for failing xpcshell in toolkit/modules/tests/xpcshell/test_NewTabUtils.js r=backout on CLOSED TREE 2017-10-31 14:19:52 +02:00
ClusterLib.js
ColorAnalyzer.js
ColorAnalyzer_worker.js
ColorConversion.js
Database.cpp Bug 1356531 - Remove moz_places_faviconindex and old Places migrations. r=Paolo 2017-10-20 12:48:37 +02:00
Database.h Bug 1356531 - Remove moz_places_faviconindex and old Places migrations. r=Paolo 2017-10-20 12:48:37 +02:00
ExtensionSearchHandler.jsm Bug 1408777 - Automatically fix instances of missing semicolons in the tree. r=Standard8 2017-10-15 20:50:39 +01:00
FaviconHelpers.cpp Bug 1409969 - The root domain icon should not be preferred if 2 icons for the same size exist. r=standard8 2017-10-21 17:45:24 +02:00
FaviconHelpers.h Bug 1247843 - Part 2: Set request context ID for the channel used to load favicon. r=mak 2017-09-20 20:09:00 -04:00
Helpers.cpp Bug 1412142 - Expose `PlacesUtils.history.hashURL`. r=mak 2017-08-16 16:01:40 -07:00
Helpers.h Bug 1412142 - Expose `PlacesUtils.history.hashURL`. r=mak 2017-08-16 16:01:40 -07:00
History.cpp Bug 1409249: Require singleton constructors to return explicit already_AddRefed. r=froydnj 2017-10-16 21:08:42 -07:00
History.h Bug 1409249: Require singleton constructors to return explicit already_AddRefed. r=froydnj 2017-10-16 21:08:42 -07:00
History.jsm Bug 1376149 - Speed up orphan favicons cleanups on history removals. r=Paolo 2017-10-24 01:35:07 +02:00
PageIconProtocolHandler.js
PlaceInfo.cpp
PlaceInfo.h
PlacesBackups.jsm Bug 1408777 - Automatically fix instances of missing semicolons in the tree. r=Standard8 2017-10-15 20:50:39 +01:00
PlacesCategoriesStarter.js
PlacesDBUtils.jsm Bug 1376149 - Speed up orphan favicons cleanups on history removals. r=Paolo 2017-10-24 01:35:07 +02:00
PlacesRemoteTabsAutocompleteProvider.jsm Bug 1408777 - Automatically fix instances of missing semicolons in the tree. r=Standard8 2017-10-15 20:50:39 +01:00
PlacesSearchAutocompleteProvider.jsm Bug 1408777 - Automatically fix instances of missing semicolons in the tree. r=Standard8 2017-10-15 20:50:39 +01:00
PlacesSyncUtils.jsm Bug 1368209 - Refactor `Engine::_processIncoming` into three stages. r=eoger,tcsc 2017-11-01 11:09:57 -07:00
PlacesTransactions.jsm Backed out changeset e17a2bca5391 (bug 1411891) for failing xpcshell in toolkit/modules/tests/xpcshell/test_NewTabUtils.js r=backout on CLOSED TREE 2017-10-31 14:19:52 +02:00
PlacesUtils.jsm Bug 1412141 - Correctly forward change sources when updating keywords. r=mak 2017-10-10 10:37:00 -07:00
SQLFunctions.cpp Bug 1380606 - Add an `IS_VALID_GUID()` SQL function to Places. r=mak 2017-10-05 11:32:49 -07:00
SQLFunctions.h Bug 1380606 - Add an `IS_VALID_GUID()` SQL function to Places. r=mak 2017-10-05 11:32:49 -07:00
Shutdown.cpp
Shutdown.h
UnifiedComplete.js Bug 1408777 - Automatically fix instances of missing semicolons in the tree. r=Standard8 2017-10-15 20:50:39 +01:00
VisitInfo.cpp
VisitInfo.h
jar.mn
moz.build
mozIAsyncFavicons.idl Bug 1247843 - Part 2: Set request context ID for the channel used to load favicon. r=mak 2017-09-20 20:09:00 -04:00
mozIAsyncHistory.idl
mozIAsyncLivemarks.idl
mozIColorAnalyzer.idl
mozIPlacesAutoComplete.idl
mozIPlacesPendingOperation.idl
nsAnnoProtocolHandler.cpp Bug 1402888 - Remove nsIInputStreamPump::Init() offset and limit params, r=smaug 2017-09-26 07:21:44 +02:00
nsAnnoProtocolHandler.h
nsAnnotationService.cpp Bug 1405317 - When inserting trees of bookmarks, avoid updating the last modified date when marking annotations. r=mak 2017-10-03 21:18:10 +01:00
nsAnnotationService.h
nsFaviconService.cpp Bug 1347532 - Allow to set a default icon size for the page-icon protocol. r=standard8 2017-10-17 16:24:55 +02:00
nsFaviconService.h Bug 1347532 - Allow to set a default icon size for the page-icon protocol. r=standard8 2017-10-17 16:24:55 +02:00
nsIAnnotationService.idl Bug 1405317 - When inserting trees of bookmarks, avoid updating the last modified date when marking annotations. r=mak 2017-10-03 21:18:10 +01:00
nsIBrowserHistory.idl Bug 1405687 - Remove some deprecated code from Places related code. r=standard8 2017-10-04 16:33:46 +02:00
nsIFaviconService.idl Bug 1347532 - Allow to set a default icon size for the page-icon protocol. r=standard8 2017-10-17 16:24:55 +02:00
nsINavBookmarksService.idl
nsINavHistoryService.idl Bug 1412142 - Expose `PlacesUtils.history.hashURL`. r=mak 2017-08-16 16:01:40 -07:00
nsITaggingService.idl
nsLivemarkService.js Bug 1411368 - Automatically fix no-multi-spaces issues raised when using ESLint 4. r=mossop 2017-10-26 11:47:01 +01:00
nsMaybeWeakPtr.h
nsNavBookmarks.cpp Bug 1405317 - When inserting trees of bookmarks, avoid updating the last modified date when marking annotations. r=mak 2017-10-03 21:18:10 +01:00
nsNavBookmarks.h
nsNavHistory.cpp Bug 1404198: Part 2f - Switch to NS_NewTimer* in toolkit. r=njn 2017-10-15 23:14:00 -07:00
nsNavHistory.h
nsNavHistoryQuery.cpp
nsNavHistoryQuery.h
nsNavHistoryResult.cpp Bug 1395333 - Remove an unexpected warning from nsNavHistoryResult.cpp. r=standard8 2017-09-01 13:55:01 +02:00
nsNavHistoryResult.h
nsPIPlacesDatabase.idl
nsPlacesExpiration.js Bug 1376149 - Speed up orphan favicons cleanups on history removals. r=Paolo 2017-10-24 01:35:07 +02:00
nsPlacesIndexes.h Bug 1396282 - Add query for getting Highlights (recent bookmarks and recent history with metadata). r=mak 2017-09-02 13:15:04 -07:00
nsPlacesMacros.h
nsPlacesModule.cpp
nsPlacesTables.h
nsPlacesTriggers.h
nsTaggingService.js Bug 1411368 - Automatically fix no-multi-spaces issues raised when using ESLint 4. r=mossop 2017-10-26 11:47:01 +01:00
toolkitplaces.manifest
unifiedcomplete-top-urls.json