Schema migration is simple: it adds localVersion and syncVersion columns to bookmarks table.
Default values are:
- localVersion=1
- syncVersion=0
... indicating that a record needs to be uploaded.
Data migration is also relatively simple: we need to ensure that records which are "up-to-date",
according to the old timestamp-based tracking, are not marked for an upload, and vice versa - records which are
either "new" or "changed" are marked for an upload.
Since our default during schema migration is to mark everything for upload, the data migration
concerns itself with the records which are considered as "up-to-date".
See detailed description in the comments in the data migration function.
MozReview-Commit-ID: J6aglurYwlo
--HG--
extra : rebase_source : 299a4eb8ea28ddfd71e98e005af1e7f89468dd80
Versioned syncing work later in the patch series introduces functionality that is best suited to live
within the RepositorySession inheritance chain.
We'd like to introduce a new RepositorySession subclass which individual RepositorySessions are able to
inherit. And that's when the current inheritance structure gets in the way: history and bookmarks both share
a superclass, and we'd like to only introduce this new functionality for bookmarks.
This makes our task, as stated, impossible without breaking apart the current inheritance structure.
This patch introduces a few "delegate" objects:
- SessionHelper
--> HistorySessionHelper
--> BookmarksSessionHelper
... which absorb most of the functionality from AndroidBrowserRepositorySession (removed) and bookmark and history
repository sessions.
This change is not functional - everything remains as before otherwise.
MozReview-Commit-ID: 7WwUmY3Wql7
--HG--
extra : rebase_source : a8cd49fd14cdc76b9e2906d4ee8c2052b9152413
As part of moving toward versioned syncing, we need to start decoupling change tracking
concepts from parts of the system that facilitate flow of records. This allows us to track
what changed differently for different data types, while maintaining a consistent and predictable API.
A move toward that is to let repositories own determinining that a record has been modified.
Repositories are now asked to provide modified records, instead of a very specific "records modified since".
This patch does not change behaviour of the system: every repository still uses timestamp-based
change tracking to actually provide modified records to the caller. A changeover to version
tracking will come later in this series for bookmarks, and as part of Bug 1383894 for other repositories.
MozReview-Commit-ID: LQuWYdlNHpt
--HG--
extra : rebase_source : 5552d74d4a967ce85af09aaa57ca438fe5b949f3
We're moving toward version-based syncing. This is one of the bricks in that road,
removing an unused timestamp-based interface for accessing changed records.
MozReview-Commit-ID: CYUyASWXrMW
--HG--
extra : rebase_source : b95687409bc5f3e8e21fb9b084efdcd14a975d01
So far, we haven't tested session restoring and saving in combination, which led to bug 1379374 where restored background tabs weren't being written back to disk after upgrading to Firefox 55.
MozReview-Commit-ID: 7CUbis7u8fb
--HG--
extra : rebase_source : 2429ab8841ff5fb61765e73ac0988fc093be6817
extra : source : 35ec7ccb1b5ef71f875fa492bbcd571a5366602b
This combines getBaseDomain & getHostSecondLevel domain and allows us to be
more flexible.
MozReview-Commit-ID: 7FdWsfZvGFt
--HG--
extra : rebase_source : eed9361fef02609776c772d21a004004a1d9d543
Error reporting, and especially the split between per-stage and global session errors,
are a bit of a mess; at the very least, this patch unifies things a little bit, and
ensures we're not passing around nulls unexpectedly.
MozReview-Commit-ID: JTSp7GuOji0
--HG--
extra : rebase_source : 19bbd2c98776b32b803d7febb55549bc430cbc3e
This mechanically replaces nsILocalFile with nsIFile in
*.js, *.jsm, *.sjs, *.html, *.xul, *.xml, and *.py.
MozReview-Commit-ID: 4ecl3RZhOwC
--HG--
extra : rebase_source : 412880ea27766118c38498d021331a3df6bccc70
Grisha showed me the link from Android API level to SQLite version and
presumably all 3.7.* releases are forwards compatible so we can relax the
requirement of which version of Android the test DBs need to be created on: I
updated the comment accordingly.
MozReview-Commit-ID: Fhu95671kSK
--HG--
extra : rebase_source : 65bb4f565c1e5c64d91292077084324ba6f40dd6
This will be used to join with the PageMetadata table so we can extract
PageMetadata values in our top sites queries.
MozReview-Commit-ID: EmwjyFM1qUf
--HG--
extra : rebase_source : 54888226238ad4d25bbe6e8c1c34bcaef2522288
The bootstrap catalog is outdated and downloading files from it returns in 404 errors from the CDN.
It's impossible to keep the bootstrap catalog updated. Instead let's start with an empty catalog
and rely on the latest catalog that we synchronize from Kinto.
MozReview-Commit-ID: FduZWJBvj16
--HG--
extra : rebase_source : 46f6d1492d6c291254261dc04a0fb75f78ca1cdf
- use EOJ's handy .equals() to compare JSON structures
- generated DSA signature prefix seem to have changed post Java 1.8 update
MozReview-Commit-ID: JwQLb998Kro
--HG--
extra : rebase_source : 802045c6ad6f2c46e34c9765022c5707c65ee3e6
After the previous changeset, some numbers stood out:
- HighlightsRanking.extractFeatures: 44.9%
- HighlightCandidate.getFeatureValue: 19.4%
- Collections.secondaryHash: 17.3%
- HashMap.get: 11.7%
My hypothesis was that our HighlightCandidate.features implementation was slow:
it was mapping FeatureNames -> values in a HashMap but HashMap look-ups are
slower than a direct memory access.
I replaced the implementation with a direct access from an array - about as
fast as we can get. This encouraged me to make some changes with the following
benefits:
- Rewrote HighlightsRanking.normalize to save iterations and allocations.
- Rm code from HighlightsRanking.scoreEntries: we no longer need to iterate to
construct the filtered items, we just index directly into the list
- Rewrote HighlightsRanking.decay(), which I think is a little clearer now.
- Saved a few iterator/object allocations inside inner loops in places.
The tests pass and we have coverage for the normalize changes but not for
scoreEntries.
---
For perf, my changes affected multiple methods so the percentages are no longer
reliable but I can verify absolute runtime changes. I ran three tests, the best
of which showed an overall 33% runtime compared to the previous changeset and
the other two profiles showed a 66% overall runtime. In particular, for the
middle run, the changes for affected methods go from X microseconds to Y
microseconds:
- Features.get: 3,554,796 -> 322,145
- secondaryHash: 3,165,785 -> 35,253
- HighlightsRanking.normalize: 6,578,481 -> 1,734,078
- HighlightsRanking.scoreEntries: 3,017,272 -> 448,300
As far as I know, my changes should not have introduced any new inefficiencies
to the code.
MozReview-Commit-ID: 9THXe8KqBbB
--HG--
extra : rebase_source : a190bc2e7c0f3ed2b5cb65202b902dcd455b3aa8
After the previous changeset, some numbers stood out:
- HighlightsRanking.extractFeatures: 44.9%
- HighlightCandidate.getFeatureValue: 19.4%
- Collections.secondaryHash: 17.3%
- HashMap.get: 11.7%
My hypothesis was that our HighlightCandidate.features implementation was slow:
it was mapping FeatureNames -> values in a HashMap but HashMap look-ups are
slower than a direct memory access.
I replaced the implementation with a direct access from an array - about as
fast as we can get. This encouraged me to make some changes with the following
benefits:
- Rewrote HighlightsRanking.normalize to save iterations and allocations.
- Rm code from HighlightsRanking.scoreEntries: we no longer need to iterate to
construct the filtered items, we just index directly into the list
- Rewrote HighlightsRanking.decay(), which I think is a little clearer now.
- Saved a few iterator/object allocations inside inner loops in places.
The tests pass and we have coverage for the normalize changes but not for
scoreEntries.
---
For perf, my changes affected multiple methods so the percentages are no longer
reliable but I can verify absolute runtime changes. I ran three tests, the best
of which showed an overall 33% runtime compared to the previous changeset and
the other two profiles showed a 66% overall runtime. In particular, for the
middle run, the changes for affected methods go from X microseconds to Y
microseconds:
- Features.get: 3,554,796 -> 322,145
- secondaryHash: 3,165,785 -> 35,253
- HighlightsRanking.normalize: 6,578,481 -> 1,734,078
- HighlightsRanking.scoreEntries: 3,017,272 -> 448,300
As far as I know, my changes should not have introduced any new inefficiencies
to the code.
MozReview-Commit-ID: 9THXe8KqBbB
--HG--
extra : rebase_source : 0c3f4c4f1e854d96c77bace60392fc4c7274e6e7
After the previous changeset, some numbers stood out:
- HighlightsRanking.extractFeatures: 44.9%
- HighlightCandidate.getFeatureValue: 19.4%
- Collections.secondaryHash: 17.3%
- HashMap.get: 11.7%
My hypothesis was that our HighlightCandidate.features implementation was slow:
it was mapping FeatureNames -> values in a HashMap but HashMap look-ups are
slower than a direct memory access.
I replaced the implementation with a direct access from an array - about as
fast as we can get. This encouraged me to make some changes with the following
benefits:
- Rewrote HighlightsRanking.normalize to save iterations and allocations.
- Rm code from HighlightsRanking.scoreEntries: we no longer need to iterate to
construct the filtered items, we just index directly into the list
- Rewrote HighlightsRanking.decay(), which I think is a little clearer now.
- Saved a few iterator/object allocations inside inner loops in places.
The tests pass and we have coverage for the normalize changes but not for
scoreEntries.
---
For perf, my changes affected multiple methods so the percentages are no longer
reliable but I can verify absolute runtime changes. I ran three tests, the best
of which showed an overall 33% runtime compared to the previous changeset and
the other two profiles showed a 66% overall runtime. In particular, for the
middle run, the changes for affected methods go from X microseconds to Y
microseconds:
- Features.get: 3,554,796 -> 322,145
- secondaryHash: 3,165,785 -> 35,253
- HighlightsRanking.normalize: 6,578,481 -> 1,734,078
- HighlightsRanking.scoreEntries: 3,017,272 -> 448,300
As far as I know, my changes should not have introduced any new inefficiencies
to the code.
MozReview-Commit-ID: 9THXe8KqBbB
--HG--
extra : rebase_source : 2358fe83acebaf04a61d912e88f8cf420b7df3d7
This reduces the calls to `getColumnIndexOrThrow` to 9 (from 1.6k) and
HighlightsRanking.extractFeatures goes from 77.1% inclusive CPU time -> 40.8%,
14.6k ms -> 7.1k ms.
MozReview-Commit-ID: L6HqvBK5I4i
--HG--
extra : rebase_source : e3f3997abb8c625bf121927be3140e43378f644e
This is used by the iOS implementation of AS to get the highlight titles.
MozReview-Commit-ID: 1p5Lf9OBcfD
--HG--
extra : rebase_source : f2a71e2a27c56fadff6710d3e321678a71994c8a
This should not be run in AS yet because each time it's called, it reads from
the disk and even the tests on desktop take 3s+ to run.
MozReview-Commit-ID: 5h4BcH3myCn
--HG--
extra : rebase_source : cf4c9465a211f369d0ceb6b0e56a22c5f0ad6c1c
I verified the behavior is the same by replacing StringUtils.join with
TextUtils.join in the StringUtils.join test (which passed) before deleting it
all.
MozReview-Commit-ID: 1pmSdTuG0LU
--HG--
extra : rebase_source : 3c20a5ceccaebd3d30a935b3ea20c87ace7d628f
extra : source : 9cdc0fcca44abc8f2ec1ea8f853ca31f8d59ce10
Grigory Kruglov
Wes Kocher
Yoshi Huang
Jim Chen
Jan Henning
Michael Comella
Edouard Oger
Dan Banner
Sebastian Kaspari
Jing-wei Wu
Sebastian Hengst
Masatoshi Kimura
kimsaehun
James Willcox
Ryan VanderMeulen
Bob Clary