Includes baddataDB.sqlite which is a copy of goodDB.sqlite but with what appears
to be the row count inflated beyond the end of the file. This causes loading the
database to succeed but queries to fail.
This increments a scalar for every database open and for every database query
where a query is the full execution of a statement from start to completion. If
a statement is re-used then the scalar will be incremented once for each use.
Differential Revision: https://phabricator.services.mozilla.com/D73938
Includes baddataDB.sqlite which is a copy of goodDB.sqlite but with what appears
to be the row count inflated beyond the end of the file. This causes loading the
database to succeed but queries to fail.
This increments a scalar for every database open and for every database query
where a query is the full execution of a statement from start to completion. If
a statement is re-used then the scalar will be incremented once for each use.
Differential Revision: https://phabricator.services.mozilla.com/D73938
mozStorage used to read prefs on service init, because they could only be read
on the main-thread. When service init was moved earlier, it started trying
to read prefs too early, before the profile was set up, thus it ended up always
reading the default value.
This patch moves the only relevant pref to mirrored StaticPrefs that can be accessed
from different threads, and removes two preferences that apparently are not necessary
(they have been broken from a long time) for now.
In particular, providing a global synchronous setting is a footgun, each consumer should
decide about their synchronous needs, rather than abusing a dangerous "go fast" setting.
The page size is something we don't change from quite some time, and it's unlikely to be
used to run experiments in the wild before doing local measurements first, for which Try
builds are enough.
The remaining exclusiveLock pref is a bit controversial, because in general exclusive lock
is better for various reasons, and mostly it is necessary to use WAL on network shares.
Though developers may find it useful for debugging, and some third parties are doing
dangerous things (like copying over databases) to work around it, for which it's safer to
provide a less dangerous alternative.
Note exclusive lock only works on Unix-derived systems for now (no Windows implementation).
Finally, this introduces a fallback to exclusive lock, so that if a third party is using our
databases, so that we can't get an exclusive lock, we'll fallback to normal locking.
Differential Revision: https://phabricator.services.mozilla.com/D82717
To be able to remove SystemGroup, NS_ReleaseOnMainThreadSystemGroup
needs to have its dependency on SystemGroup removed. Since all
releases using SystemGroup would've released on the main thread anyway
we can safely replace NS_ReleaseOnMainThreadSystemGroup with
NS_ReleaseOnMainThread.
Depends on D64390
Differential Revision: https://phabricator.services.mozilla.com/D67631
--HG--
extra : moz-landing-system : lando
I originally had this as a few patches, but the work to fix test
failures and get the whole thing into a complete working state
quickly tangled them up. Apologies for that. To summarize what's
going on here, however:
- We introduce two prefs: shutdown.fastShutdownStage and
shutdown.lateWriteChecksStage. The latter pref is set to 1, which
will leave the existing late write checking behavior unchanged.
However, we introduce this pref to make it simpler in the future
to bump the late write checks window earlier in the shutdown cycle.
- We introduce an AppShutdown class, which will house static methods
and a small amount of state for unifying some shutdown logic. Most
importantly, it will now manage the state for app initiated restarts,
as well as the logic for performing a safe fast shutdown.
- We refactored the existing restart code to call into the new
AppShutdown file, so that if we are configured to actually perform
a fast shutdown, we will be able to run the necessary restart logic
immediately before doing so. Previously, the restart logic occurred
later in the shutdown cycle than our late write checking, meaning
if we were to simply exit the process at that point in time, we
would never run the restart coe.
- Lastly, we updated two locations which called TerminateProcess and/or
_exit(0) to call into the AppShutdown method (DoFastShutdown).
Differential Revision: https://phabricator.services.mozilla.com/D59196
--HG--
extra : moz-landing-system : lando
The inclusions were removed with the following very crude script and the
resulting breakage was fixed up by hand. The manual fixups did either
revert the changes done by the script, replace a generic header with a more
specific one or replace a header with a forward declaration.
find . -name "*.idl" | grep -v web-platform | grep -v third_party | while read path; do
interfaces=$(grep "^\(class\|interface\).*:.*" "$path" | cut -d' ' -f2)
if [ -n "$interfaces" ]; then
if [[ "$interfaces" == *$'\n'* ]]; then
regexp="\("
for i in $interfaces; do regexp="$regexp$i\|"; done
regexp="${regexp%%\\\|}\)"
else
regexp="$interfaces"
fi
interface=$(basename "$path")
rg -l "#include.*${interface%%.idl}.h" . | while read path2; do
hits=$(grep -v "#include.*${interface%%.idl}.h" "$path2" | grep -c "$regexp" )
if [ $hits -eq 0 ]; then
echo "Removing ${interface} from ${path2}"
grep -v "#include.*${interface%%.idl}.h" "$path2" > "$path2".tmp
mv -f "$path2".tmp "$path2"
fi
done
fi
done
Differential Revision: https://phabricator.services.mozilla.com/D55444
--HG--
extra : moz-landing-system : lando
In retrospect, overloading `Connection::connectionReady` to mean
"is ready _and_ supports this operation" wasn't a good idea. This
commit reverts that change (cfd44c936a9b), and adds two new methods:
* `Connection::operationSupported`, to check if a connection supports
sync or async operations. This method is public.
* `Connection::ensureOperationSupported`, that asserts or returns an
error if the connection doesn't support an operation. This is
private.
`operationSupported` is used by callers like `Service::minimizeMemory`
to detect if the connection supports sync operations, since both sync
and async connections implement `mozIStorageConnection` now.
Finally, some callers used `!mDBConn` to check if the connection was
ready, while others used `connectionReady()`. This commit changes them
to use the latter.
Differential Revision: https://phabricator.services.mozilla.com/D24974
--HG--
extra : moz-landing-system : lando
This commit wraps just enough of the mozStorage API to support the
bookmarks mirror. It's not complete: for example, there's no way
to open, clone, or close a connection, because the mirror handles
that from JS. The wrapper also omits shutdown blocking and retrying on
`SQLITE_BUSY`.
This commit also changes the behavior of sync and async mozStorage
connections. Async (`mozIStorageAsyncConnection`) methods may be called
from any thread on any connection. Sync (`mozIStorageConnection`)
methods may be called from any thread on a sync connection, and from
background threads on an async connection. All connections now QI
to `mozIStorageConnection`, but attempting to call a sync method on
an async connection from the main thread throws.
Finally, this commit exposes an `OpenedConnection::unsafeRawConnection`
getter in Sqlite.jsm, for JS code to access the underlying connection.
Differential Revision: https://phabricator.services.mozilla.com/D20073
--HG--
extra : moz-landing-system : lando
Those tests rely on the build not being a cross-compile, which is fine
most of the time, but more importantly, rely on the fact that the
version of sqlite that Firefox is being built against is the same as the
one it will be running against, which is not likely to be true.
So, all in all, it's better to do the checks at runtime.
And while they could be restricted to when building against system
sqlite, we still run them for in-tree sqlite, to ensure that we actually
built it with the expected options.
Differential Revision: https://phabricator.services.mozilla.com/D18057
--HG--
extra : moz-landing-system : lando
Bug 730495 broke them by moving the code to another directory without
moving the corresponding #define, but it also fixed why this was
disabled in some configurations so we can now just enable when
MOZ_MEMORY is defined (jemalloc used).
Differential Revision: https://phabricator.services.mozilla.com/D18049
--HG--
extra : moz-landing-system : lando
This introduces the machinery needed to generate crash annotations from a YAML
file. The relevant C++ functions are updated to take a typed enum. JavaScript
calls are unaffected but they will throw if the string argument does not
correspond to one of the known entries in the C++ enum. The existing whitelists
and blacklists of annotations are also generated from the YAML file and all
duplicate code related to them has been consolidated. Once written out to the
.extra file the annotations are converted in string form and are no different
than the existing ones.
All existing annotations have been included in the list (and some obsolete ones
have been removed) and all call sites have been updated including tests where
appropriate.
--HG--
extra : source : 4f6c43f2830701ec5552e08e3f1b06fe6d045860
Because of the storage::Service's connection list, it's possible for the
refcount for a non-main-thread Connection to experience transient increases
and decreases at any time, dooming logic in Release() that assumes the
refcount isn't changing.
This patch adopts use of an Atomic<bool> so that we execute cleanup logic
exactly once when the refcount falls to 1 at some point. Care is taken to
ensure that the failsafe Close() occurs on the correct thread.
SpinningSynchronousClose() is still dangerous and can still potentially
nest deeply on the stack. If we see instances of that in the future, we
may want to adopt use of PushEventQueue so that we can avoid re-entrancy
in our event loop spinning.
MozReview-Commit-ID: A835HBec50H
--HG--
extra : rebase_source : af2f63e8f050b7a0275e39f73e59133958e29f19
This is a follow-up to bug 1409249. There are a lot of places where our
factory singleton constructors either don't correctly handle their returned
references being released by the component manager, or do handle it, but in
ways that are not obvious.
This patch handles a few places where we can sometimes wind up with dangling
singleton pointers, adds some explanatory comments and sanity check
assertions, and replaces some uses of manual refcounting with StaticRefPtr and
ClearOnShutdown.
There are still some places where we may wind up with odd behavior if the
first QI for a getService call fails. In those cases, we wind up destroying
the first instance of a service that we create, and re-creating a new one
later.
MozReview-Commit-ID: ANYndvd7aZx
--HG--
extra : rebase_source : acfb0611a028fef6b9387eb5d1d9e285782fbc7c
Right now, NS_GENERIC_FACTORY_SINGLETON_CONSTRUCTOR expects singleton
constructors to return already-addrefed raw pointers, and while it accepts
constructors that return already_AddRefed, most existing don't do so.
Meanwhile, the convention elsewhere is that a raw pointer return value is
owned by the callee, and that the caller needs to addref it if it wants to
keep its own reference to it.
The difference in convention makes it easy to leak (I've definitely caused
more than one shutdown leak this way), so it would be better if we required
the singleton getters to return an explicit already_AddRefed, which would
behave the same for all callers.
This also cleans up several singleton constructors that left a dangling
pointer to their singletons when their initialization methods failed, when
they released their references without clearing their global raw pointers.
MozReview-Commit-ID: 9peyG4pRYcr
--HG--
extra : rebase_source : 2f5bd89c17cb554541be38444672a827c1392f3f
NS_ProxyRelease will immediately release the provided reference if already on
the right thread unless aAlwaysRelease=true (the default is false). We need to
set it to be true to avoid Close() and its potential to, out of desperation
when faced with misuse, spin a nested event loop.
--HG--
extra : rebase_source : 91c70a6fa081fdf20a8dfabf5d69bccbef1ba33c
The observer references were the only thing guranteed to keep the Service
alive, leading to potential use-after-free during the iteration loop to
make sure all the connections were closed. (Ironically, if they were
fully closed and their instances destroyed, that's when bad things would
happen.)
--HG--
extra : rebase_source : 6c8d6f9e0b75751b10166d7e2c63d5a3cb27d28e
Simon Giesecke
Jan Varga
Dave Townsend
Razvan Maries
Marco Bonardo
Andreas Farre
Eric Rahm
shindli
Doug Thayer
Gabriele Svelto
Ehsan Akhgari
Sylvestre Ledru
Lina Cambridge
Mike Hommey
David Major
Andrew Sutherland
Kris Maglione
Andrew McCreight
Bob Owen
Sebastian Hengst
Kai Engert
Andrea Marchesini
Olli Pettay
Bill McCloskey
Carsten "Tomcat" Book
Wes Kocher