This adds synchronization to the global database used by previous local storage implementation.
This patch was enhanced by asuth to bind attached database path.
Places has shown that argument binding is necessary. (Profiles may include usernames in their path which can have cool emoji and such.)
There's now an upper limit for snapshot prefilling. The value is configurable and is currently set to 4096 bytes.
Snapshots can operate in multiple modes depending on if all items have been loaded or all keys have been received. This should provide the best performance for each specific state.
This patch also adds support for creating explicit snapshots which can be used for testing.
This patch was enhanced by asuth to bind attached database path.
Places has shown that argument binding is necessary. (Profiles may include usernames in their path which can have cool emoji and such.)
Datastores are preloaded only for content principals. The preloading is triggered as soon as possible to lower the chance of blocking the main thread in content process. If there is no physical database on disk for given origin, datastore is not created. Preloaded datastores are kept alive for 20 seconds.
New methods open() and close() are added to the Storage WebIDL interface. They are only available when a pref is set and are only intended for testing. There's also a new method resetStoragesForPrincipal() which is used as a callback for close() since datastores don't release directory locks immediately. resetStoragesForPrincipal() requests an exclusive lock for given origin, so it must wait for any exising shared locks to be released.
An attribute for checking if the next gen local storage implementation is enabled is exposed via a new interface nsILocalStorageManager which should be used for any other local storage specific stuff.
This adds a new quota client implementation, but only implements ShutdownWorkThreads.
At shutdown we wait for all running operations to finish including database actors which are closed by using an extra IPC message which avoids races between the parent and child.
Databases are dropped on the child side as soon as they are not used (e.g. after unlinking by the cycle collector).
The implementation is based on a cache (datastore) living in the parent process and sync IPC calls initiated from content processes.
IPC communication is done using per principal/origin database actors which connect to the datastore.
The synchronous blocking of the main thread is done by creating a nested event target and spinning the event loop.
LocalStorage needs to be exposed in every context except for sandboxes and
NullPrincipals (data: URLs, for instance). But we need to keep data
separate in some scenarios: private-browsing and trackers.
In private-browsing, LocalStorage keeps data in memory, and it shares
StorageEvents just with other origins in the same private-browsing
environment.
For Trackers, we expose a partitioned LocalStorage, which doesn't share
data with other contexts, and it's just in memory. Partitioned localStorage
is available only for trackers listed in the
privacy.restrict3rdpartystorage.partitionedHosts pref. See
nsContentUtils::IsURIInPrefList to know the syntax for the pref value.
Most of the times when we automatically create nsThread wrappers for threads
that don't already have them, we don't actually need the event targets, since
those threads don't run XPCOM event loops. Aside from wasting memory, actually
creating these event loops can lead to leaks if a thread tries to dispatch a
runnable to the queue which creates a reference cycle with the thread.
Not creating the event queues for threads that don't actually need them helps
avoid those foot guns, and also makes it easier to figure out which treads
actually run XPCOM event loops.
MozReview-Commit-ID: Arck4VQqdne
--HG--
extra : source : a03a61d6d724503c3b7c5e31fe32ced1f5d1c219
extra : intermediate-source : 5152af6ab3e399216ef6db8f060c257b2ffbd330
extra : histedit_source : ef06000344416e0919f536d5720fa979d2d29c66%2C4671676b613dc3e3ec762edf5d72a2ffbe6fca3f
Most of the times when we automatically create nsThread wrappers for threads
that don't already have them, we don't actually need the event targets, since
those threads don't run XPCOM event loops. Aside from wasting memory, actually
creating these event loops can lead to leaks if a thread tries to dispatch a
runnable to the queue which creates a reference cycle with the thread.
Not creating the event queues for threads that don't actually need them helps
avoid those foot guns, and also makes it easier to figure out which treads
actually run XPCOM event loops.
MozReview-Commit-ID: Arck4VQqdne
--HG--
extra : rebase_source : fcf8fa50e748c4b54c3bb1997575d9ffd4cbaae1
extra : source : a03a61d6d724503c3b7c5e31fe32ced1f5d1c219
Most of the times when we automatically create nsThread wrappers for threads
that don't already have them, we don't actually need the event targets, since
those threads don't run XPCOM event loops. Aside from wasting memory, actually
creating these event loops can lead to leaks if a thread tries to dispatch a
runnable to the queue which creates a reference cycle with the thread.
Not creating the event queues for threads that don't actually need them helps
avoid those foot guns, and also makes it easier to figure out which treads
actually run XPCOM event loops.
MozReview-Commit-ID: Arck4VQqdne
--HG--
extra : rebase_source : 02c5572b92ee48c11697d90941336e10c03d49cf
Everything that goes in a PLDHashtable (and its derivatives, like
nsTHashtable) needs to inherit from PLDHashEntryHdr. But through a lack
of enforcement, copy constructors for these derived classes didn't
explicitly invoke the copy constructor for PLDHashEntryHdr (and the
compiler didn't invoke the copy constructor for us). Instead,
PLDHashTable explicitly copied around the bits that the copy constructor
would have.
The current setup has two problems:
1) Derived classes should be using move construction, not copy
construction, since anything that's shuffling hash table keys/entries
around will be using move construction.
2) Derived classes should take responsibility for transferring bits of
superclass state around, and not rely on something else to handle that.
The second point is not a huge problem for PLDHashTable (PLDHashTable
only has to copy PLDHashEntryHdr's bits in a single place), but future
hash table implementations that might move entries around more
aggressively would have to insert compensation code all over the
place. Additionally, if moving entries is implemented via memcpy (which
is quite common), PLDHashTable copying around bits *again* is
inefficient.
Let's fix all these problems in one go, by:
1) Explicitly declaring the set of constructors that PLDHashEntryHdr
implements (and does not implement). In particular, the copy
constructor is deleted, so any derived classes that attempt to make
themselves copyable will be detected at compile time: the compiler
will complain that the superclass type is not copyable.
This change on its own will result in many compiler errors, so...
2) Change any derived classes to implement move constructors instead of
copy constructors. Note that some of these move constructors are,
strictly speaking, unnecessary, since the relevant classes are moved
via memcpy in nsTHashtable and its derivatives.
This switches over the few remaining usages of the deprecated
BeginWriting/EndWriting(iterator&) functions to the more standard
BeginWriting/EndWriting() functions.
--HG--
extra : rebase_source : 3c54621d4921eb45157ec4edce0b693bdd7f02d5
nsPrintfCString creates a formatted string, but doesn't print it.
ExtractOriginData is called for its constructor's scope and origin out parameters. This ExtractOriginData object doesn't really need a name because it is unused after we have scope and origin, but this change makes clang-tidy happy and makes the code clear that ExtractOriginData is an object and not a funcion being called.
MozReview-Commit-ID: nxUBwzL4Uq
--HG--
extra : rebase_source : 5fd3a592ac628b14b8221bf2df4fc442fb407106
extra : source : 1dfaf199d2ede7a445b40ec7585c66010f5a1a6b
This patch adds a new IPDL protocol PBackgroundLocalStorageCache. It is used by LocalStorageCache object to broadcast changes in local storage cache to other content processes. Each origin has its own PBackgroundLocalStorageCache, so now we can notify content processes that actually have a local storage cache for given origin. This greatly improves performance and reduces memory footprint especialy when local storage changes carry big strings and/or happen very quickly (before this patch all child processes were blindly notified).
This was used to label IndexedDB work and work in storage/mozStorage*.
I don't think this deserves its own category; categories are most useful for
the main thread, and most of the time-consuming database-related work happens
on helper threads. The main thread pieces are mostly for asynchronicity-
coordination and don't usually take up time.
This patch labels IndexedDB work as DOM instead (which is maybe debatable) and
the rest as OTHER.
MozReview-Commit-ID: 3UYhFFbi3Ry
--HG--
extra : rebase_source : 5c88dfd67274103de01fe44191f49776017738f9
This was used to label IndexedDB work and work in storage/mozStorage*.
I don't think this deserves its own category; categories are most useful for
the main thread, and most of the time-consuming database-related work happens
on helper threads. The main thread pieces are mostly for asynchronicity-
coordination and don't usually take up time.
This patch labels IndexedDB work as DOM instead (which is maybe debatable) and
the rest as OTHER.
MozReview-Commit-ID: 3UYhFFbi3Ry
--HG--
extra : rebase_source : f83946138d8311ea5aa91f537a1d8e420e784068
This was used to label IndexedDB work and work in storage/mozStorage*.
I don't think this deserves its own category; categories are most useful for
the main thread, and most of the time-consuming database-related work happens
on helper threads. The main thread pieces are mostly for asynchronicity-
coordination and don't usually take up time.
This patch labels IndexedDB work as DOM instead (which is maybe debatable) and
the rest as OTHER.
MozReview-Commit-ID: 3UYhFFbi3Ry
--HG--
extra : rebase_source : 16abc3c4bd8ed9ac55b5c188bd10ee26b0566330
Margareta Eliza Balazs
Ehsan Akhgari
Sylvestre Ledru
Andrew Sutherland
Jan Varga
Gabriele Svelto
Andrea Marchesini
Gerald Squelart
Kris Maglione
Narcis Beleuzu
Noemi Erli
Markus Stange
Nathan Froyd
Eric Rahm
Boris Zbarsky
Daniel Holbert
Sebastian Hengst
Masatoshi Kimura
Chris Peterson
Andi-Bogdan Postelnicu
Jeff Gilbert
Csoregi Natalia