This patch adds a timer for synchronous local storage requests. Once the timer
fires the request is canceled. The parent reports debugging information upon receiving cancellation.
Differential Revision: https://phabricator.services.mozilla.com/D24006
***
Bug 1514594: Part 3a - Change ChromeUtils.import to return an exports object; not pollute global. r=mccr8
This changes the behavior of ChromeUtils.import() to return an exports object,
rather than a module global, in all cases except when `null` is passed as a
second argument, and changes the default behavior not to pollute the global
scope with the module's exports. Thus, the following code written for the old
model:
ChromeUtils.import("resource://gre/modules/Services.jsm");
is approximately the same as the following, in the new model:
var {Services} = ChromeUtils.import("resource://gre/modules/Services.jsm");
Since the two behaviors are mutually incompatible, this patch will land with a
scripted rewrite to update all existing callers to use the new model rather
than the old.
***
Bug 1514594: Part 3b - Mass rewrite all JS code to use the new ChromeUtils.import API. rs=Gijs
This was done using the followng script:
https://bitbucket.org/kmaglione/m-c-rewrites/src/tip/processors/cu-import-exports.jsm
***
Bug 1514594: Part 3c - Update ESLint plugin for ChromeUtils.import API changes. r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D16747
***
Bug 1514594: Part 3d - Remove/fix hundreds of duplicate imports from sync tests. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16748
***
Bug 1514594: Part 3e - Remove no-op ChromeUtils.import() calls. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16749
***
Bug 1514594: Part 3f.1 - Cleanup various test corner cases after mass rewrite. r=Gijs
***
Bug 1514594: Part 3f.2 - Cleanup various non-test corner cases after mass rewrite. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16750
--HG--
extra : rebase_source : 359574ee3064c90f33bf36c2ebe3159a24cc8895
extra : histedit_source : b93c8f42808b1599f9122d7842d2c0b3e656a594%2C64a3a4e3359dc889e2ab2b49461bab9e27fc10a7
This patch uses categorical keyed Historgram to collect data. While the key is
used to determine whether is it an external error or an internal error (Note:
the external error is referred to low level failure, for example: database
corrupt, OS API errors, ... etc; the internal error is referred to errors, like:
not handle file properly, unexpected filenames, ... etc), the labels for
categorical indicates where the error happens.
Furthermore, this patch make QuotaManager keep traversing the profile even if
an error happens so that we can get more information in the telemetry data.
Please note that these things should only happen in the Nightly Channel.
Differential Revision: https://phabricator.services.mozilla.com/D15908
--HG--
extra : moz-landing-system : lando
Summary: Really sorry for the size of the patch. It's mostly automatic
s/nsIDocument/Document/ but I had to fix up in a bunch of places manually to
add the right namespacing and such.
Overall it's not a very interesting patch I think.
nsDocument.cpp turns into Document.cpp, nsIDocument.h into Document.h and
nsIDocumentInlines.h into DocumentInlines.h.
I also changed a bunch of nsCOMPtr usage to RefPtr, but not all of it.
While fixing up some of the bits I also removed some unneeded OwnerDoc() null
checks and such, but I didn't do anything riskier than that.
This patch slightly speedups origin initialization by having a special column in
the database table for current data usage.
This patch also fixes a problem with length computation of some unicode strings.
This improves performance by keeping snapshots around for some time if there are no changes done by other processes. If a snapshot is not destroyed immediately after getting into the stable state then there's a chance that it won't have to be synchronously created again when a new opeartion is requested.
Items are now cached also in an array (besides a hashtable). This gives us very fast snapshot initizilization for datastores that fit into the prefill limit. String buffers are reference counted, so memory footprint is only affected by the size of nsString.
This patch also introduces a WriteOptimizer which is an abstraction for collecting, coalescing and applying write operations.
Before this patch, it was only possible to initialize a snapshot to the Partial state or AllOrderedItems state. Now there's a third state AllOrderedKeys.
This improves performance by eliminating sync calls to parent process when we know nothing about a key in content process (in that case we have to use a sync call to the parent process to see if there's a value for it).
With this patch we always try to send all keys to content when a snapshot is being initialized. For this to work efficiently, we cache the size of all keys.
Having cached size of all keys also allows us to just iterate the mValues hashtable when the size of keys is bigger than snapshot prefill threshold (instead of iterating over the mKeys array and joining with mValues for each particular key).
There's some additional cleanup in snapshot info construction and Datastore::SetItem/RemoveItem/Clear methods.
This adds synchronization to the global database used by previous local storage implementation.
This patch was enhanced by asuth to bind attached database path.
Places has shown that argument binding is necessary. (Profiles may include usernames in their path which can have cool emoji and such.)
There's now an upper limit for snapshot prefilling. The value is configurable and is currently set to 4096 bytes.
Snapshots can operate in multiple modes depending on if all items have been loaded or all keys have been received. This should provide the best performance for each specific state.
This patch also adds support for creating explicit snapshots which can be used for testing.
This improves performance a lot in cases when multiple operations are invoked by a single JS function (number of sync IPC calls is reduced to a minimum). It also improves correctness since changes are not visible to other content processes until a JS function finishes.
The patch implements core infrastructure, all items are sent to content when a snapshot is initialized and everything is fully working. However, sending of all items at once is not optimal for bigger databases. Support for lazy loading of items is implemented in a following patch.
If a database actor already exists for given origin, reuse it instead of creating a new one. This improves memory footprint a bit and also eliminates some round trips to the parent process.
This patch was enhanced by asuth to bind attached database path.
Places has shown that argument binding is necessary. (Profiles may include usernames in their path which can have cool emoji and such.)
Datastores are preloaded only for content principals. The preloading is triggered as soon as possible to lower the chance of blocking the main thread in content process. If there is no physical database on disk for given origin, datastore is not created. Preloaded datastores are kept alive for 20 seconds.
New methods open() and close() are added to the Storage WebIDL interface. They are only available when a pref is set and are only intended for testing. There's also a new method resetStoragesForPrincipal() which is used as a callback for close() since datastores don't release directory locks immediately. resetStoragesForPrincipal() requests an exclusive lock for given origin, so it must wait for any exising shared locks to be released.
An attribute for checking if the next gen local storage implementation is enabled is exposed via a new interface nsILocalStorageManager which should be used for any other local storage specific stuff.
Storage events are fired either directly after getting response from synchronous SetItem call or through observers. When a new onstorage event listener is added, we sycnhronously register an observer in the parent process. There's always only one observer actor per content process.
PBackgroundLSDatabase is now managed by a new PBackgroundLSObject protocol. PBackgroundLSObject is needed to eliminate the need to pass the principal info and document URI everytime a write operation occurs.
Preparation of an observer shares some states with preparation of a datastore, so common stuff now lives in LSRequestBase and preparation of a datastore now implements a nested state machine.
This patch was enhanced by asuth to drop observers only when the last storage listener is removed.
EventListenerRemoved is invoked on any removal, not just the final removal, so we need to make sure it's the final removal before dropping observer.
Since we keep/cache data in memory anyway, private browsing support is mostly about avoiding any persistence related calls and clearing private browsing datastores when we get a notification. The separation between normal and private browsing datastores is done by the privateBrowsingId attribute which is part of the origin string.
Introduced a Connection and a ConnectioThread object.
All I/O requests are processed by a new single thread managed by ConnectionThread object.
Connection objects are prepared by the prepare datastore operation and then kept alive by datastores just like directory locks.
All datastore I/O operations are wrapped in a transaction which automaticaly commits after 5 seconds.
Datastore preparation now also loads all items from the database.
Expose the nested main event target, so it can be used in the single process case by the parent side to process runnables which need to run on the main thread.
After this change we don't have use hacks like getting profile directory path on the child side and sending it to the parent. The parent side can now do it freely even in the single process case.
This patch was enhanced by asuth to not tunnel the nested main event target through IPC.
Preparation of datastores now creates real database files on disk. The LocalStorage directory is protected by a directory lock.
Infrastructure for database schema upgrades is in place too.
Database actors are now explicitely tracked by the datastore. When the last actor finishes the directory lock is released.
Added also implementation for QuotaClient::GetUsageForOrigin() and QuotaClient::AbortOperations().
This adds a new quota client implementation, but only implements ShutdownWorkThreads.
At shutdown we wait for all running operations to finish including database actors which are closed by using an extra IPC message which avoids races between the parent and child.
Databases are dropped on the child side as soon as they are not used (e.g. after unlinking by the cycle collector).
The implementation is based on a cache (datastore) living in the parent process and sync IPC calls initiated from content processes.
IPC communication is done using per principal/origin database actors which connect to the datastore.
The synchronous blocking of the main thread is done by creating a nested event target and spinning the event loop.
Jan Varga
Sebastian Hengst
Sylvestre Ledru
Yaron Tausky
Gurzau Raul
Boris Zbarsky
Oana Pop Rus
Kris Maglione
Ehsan Akhgari
Tom Tung
Emilio Cobos Álvarez
Mark Banner
Narcis Beleuzu
Alex Gaynor
Andrew Sutherland