I originally had this as a few patches, but the work to fix test
failures and get the whole thing into a complete working state
quickly tangled them up. Apologies for that. To summarize what's
going on here, however:
- We introduce two prefs: shutdown.fastShutdownStage and
shutdown.lateWriteChecksStage. The latter pref is set to 1, which
will leave the existing late write checking behavior unchanged.
However, we introduce this pref to make it simpler in the future
to bump the late write checks window earlier in the shutdown cycle.
- We introduce an AppShutdown class, which will house static methods
and a small amount of state for unifying some shutdown logic. Most
importantly, it will now manage the state for app initiated restarts,
as well as the logic for performing a safe fast shutdown.
- We refactored the existing restart code to call into the new
AppShutdown file, so that if we are configured to actually perform
a fast shutdown, we will be able to run the necessary restart logic
immediately before doing so. Previously, the restart logic occurred
later in the shutdown cycle than our late write checking, meaning
if we were to simply exit the process at that point in time, we
would never run the restart coe.
- Lastly, we updated two locations which called TerminateProcess and/or
_exit(0) to call into the AppShutdown method (DoFastShutdown).
Differential Revision: https://phabricator.services.mozilla.com/D59196
--HG--
extra : moz-landing-system : lando
Done with:
./mach static-analysis check --checks="-*, modernize-concat-nested-namespaces" --fix .
and then clang-format on the files
Differential Revision: https://phabricator.services.mozilla.com/D58217
--HG--
extra : moz-landing-system : lando
Done with:
./mach static-analysis check --checks="-*, modernize-concat-nested-namespaces" --fix .
and then clang-format on the files
Differential Revision: https://phabricator.services.mozilla.com/D58217
--HG--
extra : moz-landing-system : lando
This changeset is a simple find and replace of `MOZ_FALLTHROUGH` and `[[fallthrough]]`.
Unfortunately, the MOZ_FALLTHROUGH_ASSERT macro (to assert on case fallthrough in debug builds) is still necessary after switching from [[clang::fallthrough]] to [[fallthrough]] because:
* MOZ_ASSERT(false) followed by [[fallthrough]] triggers a -Wunreachable-code warning in DEBUG builds
* but MOZ_ASSERT(false) without [[fallthrough]] triggers a -Wimplicit-fallthrough warning in NDEBUG builds.
Differential Revision: https://phabricator.services.mozilla.com/D56440
--HG--
extra : moz-landing-system : lando
The inclusions were removed with the following very crude script and the
resulting breakage was fixed up by hand. The manual fixups did either
revert the changes done by the script, replace a generic header with a more
specific one or replace a header with a forward declaration.
find . -name "*.idl" | grep -v web-platform | grep -v third_party | while read path; do
interfaces=$(grep "^\(class\|interface\).*:.*" "$path" | cut -d' ' -f2)
if [ -n "$interfaces" ]; then
if [[ "$interfaces" == *$'\n'* ]]; then
regexp="\("
for i in $interfaces; do regexp="$regexp$i\|"; done
regexp="${regexp%%\\\|}\)"
else
regexp="$interfaces"
fi
interface=$(basename "$path")
rg -l "#include.*${interface%%.idl}.h" . | while read path2; do
hits=$(grep -v "#include.*${interface%%.idl}.h" "$path2" | grep -c "$regexp" )
if [ $hits -eq 0 ]; then
echo "Removing ${interface} from ${path2}"
grep -v "#include.*${interface%%.idl}.h" "$path2" > "$path2".tmp
mv -f "$path2".tmp "$path2"
fi
done
fi
done
Differential Revision: https://phabricator.services.mozilla.com/D55444
--HG--
extra : moz-landing-system : lando
This is a wrapper around the `sqlite3_limit` interface that returns the
binding parameter limit. Adding this getter lets us clean up the inline
`SQLITE_MAX_VARIABLE_NUMBER` constants scattered around Places.
Differential Revision: https://phabricator.services.mozilla.com/D49071
--HG--
extra : moz-landing-system : lando
We want to remove flat strings (JSFlatString). With this patch we only expose
linear strings (JSLinearString) to API consumers.
This is very mechanical for the most part, because code typically only cares
about linear strings and not the null-termination aspect.
CTypes's Library.cpp has some Windows-specific code where we relied on null-terminated
strings. This patch adds JS_CopyStringCharsZ for that use case.
Differential Revision: https://phabricator.services.mozilla.com/D48314
--HG--
extra : moz-landing-system : lando
A low-frequency error that showed up in telemetry for new bookmark
sync is "Storage operation failed with NS_ERROR_CANNOT_CONVERT_DATA",
which occurs when we try to convert a variant to the wrong type. It
would be helpful to include the type and column names for this case;
for example, "Can't get i32 for column lastModified". Neither the type
nor column name are PII; we want them to show up in telemetry.
Differential Revision: https://phabricator.services.mozilla.com/D46433
--HG--
extra : moz-landing-system : lando
This patch adds a fixed-size array of client usages to OriginInfo and modifies
quota tracking APIs to require the client type to be passed in.
A new method ResetUsageForClient is implemented. The method is used during
client-specific origin clearing. ResetUsageForClient is much faster than calling
GetUsageForOrigin and calling DecreaseUsageForOrigin after that.
LockedUsage now has an assertion that verifies that the total sum of client
usages matches total origin usage. This method should be called instead of
touching mUsage directly.
A new assertion is added to GetQuotaObject which verifies that passed file
belongs to the given persistence type, origin, and client.
Differential Revision: https://phabricator.services.mozilla.com/D38028
--HG--
extra : moz-landing-system : lando
This commit updates mozStorage to always:
* Pass the length, using sqlite3_{column, value}_bytes16, when
creating an nsDependentString from a pointer.
* Call sqlite3_{column, value}_bytes{16} after
sqlite3_{column, value}_{text, blob, text16}, per the
recommendation in https://www.sqlite.org/c3ref/column_blob.html.
Some callers did this before, or in unclear order, since C++ doesn't
specify one for evaluating function arguments.
* Pass the byte length to sqlite3_result_text16.
Differential Revision: https://phabricator.services.mozilla.com/D26848
--HG--
extra : moz-landing-system : lando
The `nsresult` codes that mozStorage returns are often too generic. For
example, `NS_ERROR_FAILURE` might be anything from a SQL syntax error
to an invalid column name in a trigger.
This commit adds a `Conn::call_and_wrap_error` helper that checks the
last SQLite error, and returns that instead of `nsresult`. Not all
errors are SQLite errors, however, so we only use this for mozStorage
methods that return `convertResultCode`.
Differential Revision: https://phabricator.services.mozilla.com/D25179
--HG--
extra : moz-landing-system : lando
In retrospect, overloading `Connection::connectionReady` to mean
"is ready _and_ supports this operation" wasn't a good idea. This
commit reverts that change (cfd44c936a9b), and adds two new methods:
* `Connection::operationSupported`, to check if a connection supports
sync or async operations. This method is public.
* `Connection::ensureOperationSupported`, that asserts or returns an
error if the connection doesn't support an operation. This is
private.
`operationSupported` is used by callers like `Service::minimizeMemory`
to detect if the connection supports sync operations, since both sync
and async connections implement `mozIStorageConnection` now.
Finally, some callers used `!mDBConn` to check if the connection was
ready, while others used `connectionReady()`. This commit changes them
to use the latter.
Differential Revision: https://phabricator.services.mozilla.com/D24974
--HG--
extra : moz-landing-system : lando
This commit wraps just enough of the mozStorage API to support the
bookmarks mirror. It's not complete: for example, there's no way
to open, clone, or close a connection, because the mirror handles
that from JS. The wrapper also omits shutdown blocking and retrying on
`SQLITE_BUSY`.
This commit also changes the behavior of sync and async mozStorage
connections. Async (`mozIStorageAsyncConnection`) methods may be called
from any thread on any connection. Sync (`mozIStorageConnection`)
methods may be called from any thread on a sync connection, and from
background threads on an async connection. All connections now QI
to `mozIStorageConnection`, but attempting to call a sync method on
an async connection from the main thread throws.
Finally, this commit exposes an `OpenedConnection::unsafeRawConnection`
getter in Sqlite.jsm, for JS code to access the underlying connection.
Differential Revision: https://phabricator.services.mozilla.com/D20073
--HG--
extra : moz-landing-system : lando
This commit adds a `storage_variant::HashPropertyBag` type that
exposes an idiomatic Rust interface for `nsIWritablePropertyBag`.
Differential Revision: https://phabricator.services.mozilla.com/D21062
--HG--
extra : moz-landing-system : lando
This is a follow-up to the previous part, which actually changes one of
these callers to use Array<nsIIDRef> instead of [array] nsIIDPtr.
From doing this patch, it seems like we should consider changing
the type `nsIIDRef` to instead simply be `nsIID`, and treat it more like
the `AString` types from the POV of XPIDL. `nsIIDPtr` would then
continue to exist for backwards compatibility, but we can probably
remove almost all current consumers over time.
Depends on D19175
Differential Revision: https://phabricator.services.mozilla.com/D19176
--HG--
extra : moz-landing-system : lando
Those tests rely on the build not being a cross-compile, which is fine
most of the time, but more importantly, rely on the fact that the
version of sqlite that Firefox is being built against is the same as the
one it will be running against, which is not likely to be true.
So, all in all, it's better to do the checks at runtime.
And while they could be restricted to when building against system
sqlite, we still run them for in-tree sqlite, to ensure that we actually
built it with the expected options.
Differential Revision: https://phabricator.services.mozilla.com/D18057
--HG--
extra : moz-landing-system : lando
Bug 730495 broke them by moving the code to another directory without
moving the corresponding #define, but it also fixed why this was
disabled in some configurations so we can now just enable when
MOZ_MEMORY is defined (jemalloc used).
Differential Revision: https://phabricator.services.mozilla.com/D18049
--HG--
extra : moz-landing-system : lando
***
Bug 1514594: Part 3a - Change ChromeUtils.import to return an exports object; not pollute global. r=mccr8
This changes the behavior of ChromeUtils.import() to return an exports object,
rather than a module global, in all cases except when `null` is passed as a
second argument, and changes the default behavior not to pollute the global
scope with the module's exports. Thus, the following code written for the old
model:
ChromeUtils.import("resource://gre/modules/Services.jsm");
is approximately the same as the following, in the new model:
var {Services} = ChromeUtils.import("resource://gre/modules/Services.jsm");
Since the two behaviors are mutually incompatible, this patch will land with a
scripted rewrite to update all existing callers to use the new model rather
than the old.
***
Bug 1514594: Part 3b - Mass rewrite all JS code to use the new ChromeUtils.import API. rs=Gijs
This was done using the followng script:
https://bitbucket.org/kmaglione/m-c-rewrites/src/tip/processors/cu-import-exports.jsm
***
Bug 1514594: Part 3c - Update ESLint plugin for ChromeUtils.import API changes. r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D16747
***
Bug 1514594: Part 3d - Remove/fix hundreds of duplicate imports from sync tests. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16748
***
Bug 1514594: Part 3e - Remove no-op ChromeUtils.import() calls. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16749
***
Bug 1514594: Part 3f.1 - Cleanup various test corner cases after mass rewrite. r=Gijs
***
Bug 1514594: Part 3f.2 - Cleanup various non-test corner cases after mass rewrite. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16750
--HG--
extra : rebase_source : 359574ee3064c90f33bf36c2ebe3159a24cc8895
extra : histedit_source : b93c8f42808b1599f9122d7842d2c0b3e656a594%2C64a3a4e3359dc889e2ab2b49461bab9e27fc10a7
Do not require that SQLite has been built with support for custom FTS3
tokenizers enabled by default. This allows to use system SQLite in
distributions which provide SQLite configured in this way (which is SQLite
upstream's default configuration due to security concerns).
Requires exposing the sqlite3_db_config symbol in bundled SQLite.
Disable no longer needed setting of SQLITE_ENABLE_FTS3_TOKENIZER macro in
bundled SQLite build.
This patch slightly speedups origin initialization by having a special column in
the database table for current data usage.
This patch also fixes a problem with length computation of some unicode strings.
- modify line wrap up to 80 chars; (tw=80)
- modify size of tab to 2 chars everywhere; (sts=2, sw=2)
--HG--
extra : rebase_source : 7eedce0311b340c9a5a1265dc42d3121cc0f32a0
extra : amend_source : 9cb4ffdd5005f5c4c14172390dd00b04b2066cd7
These maps hold strong references which complicate nsThread lifetime handling
considerably, and only have a couple of fringe uses. We have a linked list of
active threads that the thread manager can use for its internal enumeration
purposes, and the external uses are easily done away with, so there doesn't
seem to be much reason to keep the map around.
MozReview-Commit-ID: x7dsj6C4x8
--HG--
extra : source : 5f870621361012ba459943212d8c68a9ff81cb16
extra : intermediate-source : 89a0c0874d400dd324df6fc3627c0c47d130df19
extra : histedit_source : bbd7900e3d754bde925a411c10aa30a1d6e22edd
These maps hold strong references which complicate nsThread lifetime handling
considerably, and only have a couple of fringe uses. We have a linked list of
active threads that the thread manager can use for its internal enumeration
purposes, and the external uses are easily done away with, so there doesn't
seem to be much reason to keep the map around.
MozReview-Commit-ID: x7dsj6C4x8
--HG--
extra : rebase_source : 897e2d32d1dfee24d51459065925fb9b41fa543a
extra : source : 5f870621361012ba459943212d8c68a9ff81cb16
These maps hold strong references which complicate nsThread lifetime handling
considerably, and only have a couple of fringe uses. We have a linked list of
active threads that the thread manager can use for its internal enumeration
purposes, and the external uses are easily done away with, so there doesn't
seem to be much reason to keep the map around.
MozReview-Commit-ID: x7dsj6C4x8
--HG--
extra : rebase_source : 88c56fa4f5da97f33ade08d892c3d8c42666307e
We move the XPConnect() singleton accessor to nsIXConnect to make it available for consumers outside of XPConnect. Most of the consumers of the singleton accessor just need the nsIXPConnect public interface, except for the IsShuttingDown() member which this patch adds to nsIXPConnect as well.
Differential Revision: https://phabricator.services.mozilla.com/D5151
We move the XPConnect() singleton accessor to nsIXConnect to make it available for consumers outside of XPConnect. Most of the consumers of the singleton accessor just need the nsIXPConnect public interface, except for the IsShuttingDown() member which this patch adds to nsIXPConnect as well.
Differential Revision: https://phabricator.services.mozilla.com/D5151
This makes it much easier to update existing consumers of
XPCOMUtils.enumerateCategoryEntries to use the category manager directly.
It also, unfortunately, requires updating existing category manager consumers
to use the Services getter in order to avoid ESLint errors.
Differential Revision: https://phabricator.services.mozilla.com/D4278
--HG--
extra : rebase_source : fb9fd9b21db80af472ff6250a2e9a35e8d538147
This introduces the machinery needed to generate crash annotations from a YAML
file. The relevant C++ functions are updated to take a typed enum. JavaScript
calls are unaffected but they will throw if the string argument does not
correspond to one of the known entries in the C++ enum. The existing whitelists
and blacklists of annotations are also generated from the YAML file and all
duplicate code related to them has been consolidated. Once written out to the
.extra file the annotations are converted in string form and are no different
than the existing ones.
All existing annotations have been included in the list (and some obsolete ones
have been removed) and all call sites have been updated including tests where
appropriate.
--HG--
extra : source : 4f6c43f2830701ec5552e08e3f1b06fe6d045860
Summary: GTest is permafailing because of this test (GTest is always running over 1200s).
Reviewers: froydnj
Reviewed By: froydnj
Bug #: 1473531
Differential Revision: https://phabricator.services.mozilla.com/D2032
--HG--
extra : rebase_source : 07413c4d7ba4eb6bd1439f292bd489a775dab61b
Use the exclusive VFS on unix systems, so that:
1. we can avoid the memory mapped -shm files in wal mode
2. we gain more compatibility with nfs shares
3. we gain some protection from third parties touching open dbs
On the other side it won't be possible anymore to use an open database from a
different process (like the Sqlite command line), for which we provide an hidden
pref: storage.multiProcessAccess.enabled
Differential Revision: https://phabricator.services.mozilla.com/D1964
--HG--
extra : moz-landing-system : lando
Use the exclusive VFS on unix systems, so that:
1. we can avoid the memory mapped -shm files in wal mode
2. we gain more compatibility with nfs shares
3. we gain some protection from third parties touching open dbs
On the other side it won't be possible anymore to use an open database from a
different process (like the Sqlite command line), for which we provide an hidden
pref: storage.multiProcessAccess.enabled
Differential Revision: https://phabricator.services.mozilla.com/D1964
--HG--
extra : moz-landing-system : lando
This patch is an automatic replacement of s/NS_NOTREACHED/MOZ_ASSERT_UNREACHABLE/. Reindenting long lines and whitespace fixups follow in patch 6b.
MozReview-Commit-ID: 5UQVHElSpCr
--HG--
extra : rebase_source : 4c1b2fc32b269342f07639266b64941e2270e9c4
extra : source : 907543f6eae716f23a6de52b1ffb1c82908d158a
Generalizes NetworkActivity so it can be used for sockets but also disk files.
The host/port data becomes a single location string prefixed with socket://
or file:// and we're not using the FD as the identifier anymore.
IOActivityMonitor is now used in three places:
- nsFileStreams for plain files
- TelemetryVFS for sqlite files
- nsSocketTransport & nsUDPSocket for UDP & TCP sockets
MozReview-Commit-ID: GNu5o400PaV
--HG--
rename : netwerk/base/NetworkActivityMonitor.cpp => netwerk/base/IOActivityMonitor.cpp
rename : netwerk/base/NetworkActivityMonitor.h => netwerk/base/IOActivityMonitor.h
rename : netwerk/base/nsINetworkActivityData.idl => netwerk/base/nsIIOActivityData.idl
extra : rebase_source : 55a1c51b261ffbe16f88671d55445d1b0d9106b6
This was used to label IndexedDB work and work in storage/mozStorage*.
I don't think this deserves its own category; categories are most useful for
the main thread, and most of the time-consuming database-related work happens
on helper threads. The main thread pieces are mostly for asynchronicity-
coordination and don't usually take up time.
This patch labels IndexedDB work as DOM instead (which is maybe debatable) and
the rest as OTHER.
MozReview-Commit-ID: 3UYhFFbi3Ry
--HG--
extra : rebase_source : 5c88dfd67274103de01fe44191f49776017738f9
This was used to label IndexedDB work and work in storage/mozStorage*.
I don't think this deserves its own category; categories are most useful for
the main thread, and most of the time-consuming database-related work happens
on helper threads. The main thread pieces are mostly for asynchronicity-
coordination and don't usually take up time.
This patch labels IndexedDB work as DOM instead (which is maybe debatable) and
the rest as OTHER.
MozReview-Commit-ID: 3UYhFFbi3Ry
--HG--
extra : rebase_source : f83946138d8311ea5aa91f537a1d8e420e784068
This was used to label IndexedDB work and work in storage/mozStorage*.
I don't think this deserves its own category; categories are most useful for
the main thread, and most of the time-consuming database-related work happens
on helper threads. The main thread pieces are mostly for asynchronicity-
coordination and don't usually take up time.
This patch labels IndexedDB work as DOM instead (which is maybe debatable) and
the rest as OTHER.
MozReview-Commit-ID: 3UYhFFbi3Ry
--HG--
extra : rebase_source : 16abc3c4bd8ed9ac55b5c188bd10ee26b0566330
This also removes any redundant Ci.nsISupports elements in the interface
lists.
This was done using the following script:
acecb401b7/processors/chromeutils-generateQI.jsm
MozReview-Commit-ID: AIx10P8GpZY
--HG--
extra : rebase_source : a29c07530586dc18ba040f19215475ac20fcfb3b
Marco Bonardo ext:(%20%3Cmbonardo%40mozilla.com%3E)
Doug Thayer
Simon Giesecke
Sylvestre Ledru
Dorel Luca
Chris Peterson
Jeff Walden
Gabriele Svelto
Tom Tung
Lina Cambridge
Jan de Mooij
Steve Fink
Boris Zbarsky
Gijs Kruitbosch
Jan Varga
Mike Hommey
Olli Pettay
Victor Porof
Eden Chuang
Ehsan Akhgari
Andrea Marchesini
Andreea Pavel
Mark Banner
Ryan Hunt
Nika Layzell
Gurzau Raul
Kris Maglione
arthur.iakab
Myk Melez
Arfrever Frehtes Taifersar Arahesis
Tooru Fujisawa
Benjamin Bouvier
David Major
Nicholas Nethercote
Narcis Beleuzu
Noemi Erli
shindli
aceman
Robert Bartlensky
Calixte Denizet
Jeff Gilbert
Andi-Bogdan Postelnicu
Tarek Ziadé
Markus Stange
Csoregi Natalia