This patch replaces |ResetTables|(clear table's in-memory and on-disk
data) with |DeleteTables|(clear table's on-disk data) in GetLookupCache to avoid infinite
loop.
We can just delete on-disk data when file corruption is detected in |GetLookupCache|
without clearing the cache's internal data and refreshing current
active caches because in that scenario, the lookup cache failing to read
database has not yet added to the active caches list.
Differential Revision: https://phabricator.services.mozilla.com/D43181
--HG--
extra : moz-landing-system : lando
Whereas previously MozDescribeCodeAddress would have handled demangling,
we need to explicitly do that from our new GetFunction method. The string we
generate is now more useful for the profiler to merge -- having dropped the
address in the previous patch, and the file & line number and library in this
patch.
While we're at it, try to demangle Rust symbols too.
Ideally we'd add Rust symbol handling to DemangleSymbol in
StackWalk.cpp, but that lives in mozglue, which currently cannot have
any Rust crate dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D43142
--HG--
extra : moz-landing-system : lando
Bug 1573566 moved libxul in a subdirectory, which means the list of
files it links are now relative to that directory, rather than the
directory where buildid.cpp is. So the dependencies for buildid.cpp
need to account for that.
Differential Revision: https://phabricator.services.mozilla.com/D43032
--HG--
extra : moz-landing-system : lando
Marking these constructors as `constexpr` encourages the compiler to
constant-initialize most of the `gScalars` array, rather than generating
a giant glob of code to move the (constant) values into the array at
runtime.
Differential Revision: https://phabricator.services.mozilla.com/D42842
--HG--
extra : moz-landing-system : lando
This prevents a whole host of OOM issues from occurring intermittently
when running these tests.
Differential Revision: https://phabricator.services.mozilla.com/D42654
--HG--
extra : moz-landing-system : lando
This Change removes all call to Exists() in Directory Provider component, which creates the possibility for the componenet to return an empty list. SearchService.jsm is modified to handle this possibility.
Differential Revision: https://phabricator.services.mozilla.com/D42772
--HG--
extra : moz-landing-system : lando
* Add an extra param to promptToChangePassword to indicate whether a duped/superceded login should be deleted
* Add a raft of tests for promptToChangePassword as it applies to generated passwords and auto-saved logins
Differential Revision: https://phabricator.services.mozilla.com/D40115
--HG--
extra : moz-landing-system : lando
Until now, `WinUtils::PreparePathForTelemetry` did not whitelist paths
originating from the Windows system directory. This is unfortunate, as those
paths are not going to contain any PII yet may be useful for troubleshooting.
By adding `%SystemRoot%` to the whitelist, such paths will be included after
substituting the real system directory with the name of the environment
variable.
Finally, we need to lazily build the whitelist in a thread-safe manner.
Whitelist construction has been refactored so that the whitelist is a static
local variable that is atomically constructed.
Differential Revision: https://phabricator.services.mozilla.com/D42916
--HG--
extra : moz-landing-system : lando
This picks up various improvements, especially to how configurations are handled and some new rules.
Differential Revision: https://phabricator.services.mozilla.com/D37271
--HG--
extra : moz-landing-system : lando
This is the last message that WebProgressChild was sending to the
RemoteWebProgress in the parent process, so we can remove the module entirely.
Differential Revision: https://phabricator.services.mozilla.com/D35091
--HG--
extra : moz-landing-system : lando
We intentionally only clean out the prefs registry key during uninstall, unlike
the other registry keys that the installer creates, because we might want to do
experiments that involve the procedures that normally clean out registry keys.
This leaves open the possibility that an experiment causes some problem that
by its nature makes it difficult to disable the experiment, since it wouldn't
be disabled by either a normal update or a paveover install, but it's
necessary to make sure that experiments persist for longer than e.g. a single
update cycle.
Depends on D40327
Differential Revision: https://phabricator.services.mozilla.com/D40328
--HG--
extra : moz-landing-system : lando
This adds a field to about:webrtc which indicates whether an associated
candidate is behind a proxy or not.
Differential Revision: https://phabricator.services.mozilla.com/D39974
--HG--
extra : moz-landing-system : lando
This prevents a whole host of OOM issues from occurring intermittently
when running these tests.
Differential Revision: https://phabricator.services.mozilla.com/D42654
--HG--
extra : moz-landing-system : lando
This test currently opens 72 browser windows, which is a bit excessive.
This is because the test has been written procedurally in order to
demonstrate the testing logic, and it has been implemented such that
each test execution happens in its own window.
However this isn't necessary. This patch rewrites this test such that
the procedural expression of the test is untouched, however as the
procedural expression is running we only compile a list of test
scenarios to run through, and later on based on those create individual
test cases the test only opens on private and one non-private window,
and runs through all of the test cases.
This both improves the speed of the test massively and will hopefully
address issues such as intermittent OOM errors.
Differential Revision: https://phabricator.services.mozilla.com/D42595
--HG--
extra : moz-landing-system : lando
This style is used consistently throughout telemetry code, but was
missed in these places.
Differential Revision: https://phabricator.services.mozilla.com/D42830
--HG--
extra : moz-landing-system : lando
Before this patch, when Safe Browsing updating process discovers an error, it
quits and resets the table failing to update.
After this patch, updating process will continue to run when an error
occurs to find all the tables failing to apply an update.
Differential Revision: https://phabricator.services.mozilla.com/D42615
--HG--
extra : moz-landing-system : lando
Patch P2 & P3 refine how Safe Browsing handles Safe Browsing database
loading failure.
Safe Browsing databases are read in 3 scenarios:
1. |GetLookupCache| is called on startup. Safe Browsing reads prefix
files in this case. Metadata for updates(.sbstore, .metadata) are not
read in this scenario.
2. |TableRequest| is called before applying an update, Safe Browsing
reads update metadata to apply a partial update.
3. During an update, Safe Browsing reads both prefix files and metadata
in order to merge the update result.
For Case 1, we reset a table's database only when it returns FILE_CORRUPTED
while loading prefixes from the prefix file(.vlpset).
For Case 2, we reset a table's database when the table fails to load its
metadata file or prefix file. This is because we need to make sure both
files are complete so we can correctly perform a partial update.
Note that in this case, we don't just reset the database when "FILE_CORRUPTED"
is detected, we reset the database as long as an error occurs while loading the
database.
For Case 3, For all the tables failing to load their database during an
updating process, the databases of those tables will be reset.
Case 1 and Case 2 are done in Patch P2; Case 3 is done in Patch P3
Differential Revision: https://phabricator.services.mozilla.com/D42614
--HG--
extra : moz-landing-system : lando
This Change removes all call to Exists() in Directory Provider component, which creates the possibility for the componenet to return an empty list. SearchService.jsm is modified to handle this possibility.
Differential Revision: https://phabricator.services.mozilla.com/D42772
--HG--
extra : moz-landing-system : lando
In bug 1542830 I need to generate an import library from a .DEF file. The
`llvm-dlltool` utility is the tool to support this.
This change adds detection for the aforementioned utility and also configures
the required flags.
Differential Revision: https://phabricator.services.mozilla.com/D41817
--HG--
extra : moz-landing-system : lando
Collect telemetry for the number of pending style and layout flush requests per
flush and the number of style and layout flushes per nsRefreshDriver::Tick. A
style flush reports only style requests, but a layout flush reports style and
layout requests since flushing layout implies a style flush also.
Differential Revision: https://phabricator.services.mozilla.com/D40756
--HG--
extra : moz-landing-system : lando
In the old way, if referrer is ommited we will not be able to go to
download page. In that case, we should use originalReferrer in
ReferrerInfo object
Differential Revision: https://phabricator.services.mozilla.com/D39875
--HG--
extra : moz-landing-system : lando
The test starts to fail more often with Fission session history (which is still using sync IPC).
The change should guarantee that there is time to send and receive options-loaded message.
Differential Revision: https://phabricator.services.mozilla.com/D42574
--HG--
extra : moz-landing-system : lando
* Add a new extensionStorage actor to enable inspection of data stored by an extension using the WebExtension storage.local API in the Storage panel client.
* The actor is only listed when the developer toolbox is targeting an extension process. For multi-process Firefox (e10s), this applies to only the toolbox accessed in about:debugging.
* The actor is gated behind a preference: devtools.storage.extensionStorage.enabled. This preference is set to false by default.
* The Storage panel displays storage item values as strings. If a storage item value is not JSON-stringifiable, it will be displayed in the table as "Object".
* It should be noted that extension storage.local’s storage backend is in the process of migrating from a JSON file to IndexedDB as of Firefox 66 for performance reasons. This actor only works for extensions that have migrated to the IndexedDB storage backend.
* In-line comments referencing Bugs 1542038 and 1542039 indicate places where the implementation may differ for local storage versus the other storage areas in the actor.
Differential Revision: https://phabricator.services.mozilla.com/D34415
--HG--
extra : moz-landing-system : lando
* Add an extra param to promptToChangePassword to indicate whether a duped/superceded login should be deleted
* Add a raft of tests for promptToChangePassword as it applies to generated passwords and auto-saved logins
Differential Revision: https://phabricator.services.mozilla.com/D40115
--HG--
extra : moz-landing-system : lando
This patch also includes some spot fixes for the storage principal
support on the XUL browser element.
Differential Revision: https://phabricator.services.mozilla.com/D42203
--HG--
extra : moz-landing-system : lando
For launching with an external protocol handler on Windows, we validate a uri
before sending it to `ShellExecute`, by converting a string into `PIDL` using
`SHParseDisplayName` and extract a string back from PIDL using
`IShellFolder::GetDisplayNameOf`. The problem was that if a fragment, a
string following a hash mark (#), is always dropped after this validation.
This is caused by the intended design of Windows.
A proposed fix is to use `CreateUri` for validation, which is used behind
`IShellFolder::GetDisplayNameOf`. However, we also keep `SHParseDisplayName`
because there are cases where `CreateUri` succeeds while `SHParseDisplayName`
fails such as a non-existent `file:` uri and we want to keep the same
validation result for those cases.
Adding `CreateUri` broke MinGW build because of our toolkit issue. We use
dynamic linking for MinGW build in the meantime.
This patch adds a new unittest to make sure the new validation logic
behaves the same as the old one except the fragment issue.
Differential Revision: https://phabricator.services.mozilla.com/D42041
--HG--
extra : moz-landing-system : lando
While working on porting the (audio-playback indicators) bug 1562990 to fission, we saw the potential to delete some methods in nsIDOMWindowUtils because they were not used anymore in our codebase except in a couple of tests files. So now, we should only mute/unmute or change the volume in the parent process. As such, interfaces are added in SpecialPowers to change media muted or volume state from content processes.
Differential Revision: https://phabricator.services.mozilla.com/D41782
--HG--
extra : moz-landing-system : lando
There are two interrelated problems with `InitializeGlobalState`. The
first is that the function itself is enormous; it clocks in at around
90K in my x86-64 Linux Nightly. (For comparison, the core interpreter
of the JS engine is smaller than this function.) This comes about
mostly because of:
```
gOriginHashesList = MakeUnique<OriginHashesList>(OriginHashesList{
});
```
which has to:
1. Construct a Tuple.
2. Append that Tuple to a temporary `std::initializer_list`.
3. Repeat for ~2500 origins.
4. Hand the `std::initializer_list` off to an `nsTArray`.
5. Move that temporary `nsTArray` into the newly `MakeUnique`'d `nsTArray`.
6. Destroy all the temporary objects created in the process.
You can imagine that this process is not exactly efficient.
The second problem is that creating this giant mass of code means that
TelemetryOrigin.cpp takes ~30s to compile on my home machine (x86-64
Android, because that's what I took the measurements on). This amount
of time is...excessive.
We're going to attack both problems with a single patch. The core idea
is that we're going to make three statically constructed arrays from
TelemetryOriginData.inc:
1. An array of (`originLength`, `hashLength`) pairs.
2. A large string constant containing the origins, in TelemetryOriginData.inc
declared order.
3. Likewise for the hashes.
The length of the first array is therefore the number of origins that
we're dealing with. It's important to note that items 2 and 3 are
storing data that we'd have to store anyway; they're just storing them
in a slightly different format. The first array is ~5K, but we're
willing to pay that to cut down on codesize.
To access the i'th origin, we need to sum the [0, i) `originLengths`
members; this sum then gives us the offset into the giant string
constant for the origins for the i'th origin. We can do the same
computation for the hashes, but using `hashLength` and the giant string
constant for the hashes. Fortunately, as we iterate through
the (`originLength`, `hashLength`) array, we can maintain a running
total of these members for nominal cost.
We can then build `gOriginHashesList` in the same loop that initializes
`gOriginToIndexMap` and `gHashToIndexMap`, for significantly less work.
As a bonus, we can make inserting items into those two maps cheaper,
because we have easy access to the lengths for the keys, rather than
computing them at runtime.
All of these changes combine to shrink `InitializeGlobalState` to less
than 2K of code on a local build. Compilation time is similarly reduced.
Differential Revision: https://phabricator.services.mozilla.com/D42344
--HG--
extra : moz-landing-system : lando
x509.subjectPublicKeyInfo was returning undefined in Firefox while in Certainly-Something it was returning the expected value. There was a problem in PublicKeyInfo.js of pkijs library, this condition `if(this.algorithm.algorithmParams instanceof asn1js.ObjectIdentifier)` was returning false, but it shouldn't. This problem was fixed in the new version of the library.
Differential Revision: https://phabricator.services.mozilla.com/D42035
--HG--
extra : moz-landing-system : lando
This patch fixes a number of miscellaneous issues with permission setting.
When we move a directory, recreate it, and move the contents back, those contents may be in use (possibly by antivirus). To address this, we now fall back to copying the data back and removing the originals.
The maximum number of backups to make when moving conflicting files is now 3 instead of 9. 9 seemed a bit excessive.
There is now an additional level of SetPermissionsOf, FilesAndDirsWithBadPerms. This new value causes permissions not to be fixed if we are unable to read them. This should prevent unnecessary permission fixes while still allowing us to aggressively set them when necessary.
Differential Revision: https://phabricator.services.mozilla.com/D42066
--HG--
extra : moz-landing-system : lando
Some tests check that the remote type is "web", but with Fission it
will instead start with "webIsolated=".
I fixed some of the errors in
browser_new_web_tab_in_file_process_pref.js and
browser_httpResponseProcessSelection.js, but there are other failures,
so they remain disabled.
Differential Revision: https://phabricator.services.mozilla.com/D42354
--HG--
extra : moz-landing-system : lando
MANUAL PUSH: Lando appears to be confused about which branch it needs to push to (landing blocked on "Repository is not supported by Lando.").
Differential Revision: https://phabricator.services.mozilla.com/D42466
--HG--
extra : rebase_source : b70568422d12aedd3d371139d2505f863ff29ee2
extra : amend_source : 54d289dae1fe85bdf70893ed165ac990ac8dc35d
This changes provide basic support for webextenion tabs and webNavigation listeners by implementing missing objects on which Fennec implementation was relying.
Differential Revision: https://phabricator.services.mozilla.com/D36575
--HG--
extra : moz-landing-system : lando
mInputStream holds a reference to the current opened HashStore file.
While resetting the file, mInputStream should be closed first, otherwise,
the file->Remove returns failure code NS_ERROR_FILE_IS_LOCKED.
(This only happens in Windows platform)
Differential Revision: https://phabricator.services.mozilla.com/D42460
--HG--
extra : moz-landing-system : lando
The current setup, where gtest/libxul uses the static library in
the same directory as the shared libxul, and somehow the backend ignores
gkrust for gtest/libxul, is fragile.
Differential Revision: https://phabricator.services.mozilla.com/D42246
--HG--
rename : toolkit/library/dependentlibs.py => toolkit/library/build/dependentlibs.py
extra : moz-landing-system : lando
Don't show the login in about:logins if the username or password cannot be decrypted.
Differential Revision: https://phabricator.services.mozilla.com/D40845
--HG--
extra : moz-landing-system : lando
For launching with an external protocol handler on Windows, we validate a uri
before sending it to `ShellExecute`, by converting a string into `PIDL` using
`SHParseDisplayName` and extract a string back from PIDL using
`IShellFolder::GetDisplayNameOf`. The problem was that if a fragment, a
string following a hash mark (#), is always dropped after this validation.
This is caused by the intended design of Windows.
A proposed fix is to use `CreateUri` for validation, which is used behind
`IShellFolder::GetDisplayNameOf`. However, we also keep `SHParseDisplayName`
because there are cases where `CreateUri` succeeds while `SHParseDisplayName`
fails such as a non-existent `file:` uri and we want to keep the same
validation result for those cases.
This patch adds a new unittest to make sure the new validation logic
behaves the same as the old one except the fragment issue.
Differential Revision: https://phabricator.services.mozilla.com/D42041
--HG--
extra : moz-landing-system : lando
The `bits` library needs to be updated to prefix traits with `dyn` in order to compile with Rust 1.37 in which they've deprecated non-prefixed traits.
Differential Revision: https://phabricator.services.mozilla.com/D42187
--HG--
extra : moz-landing-system : lando
Bug 1498518 adds credui.dll to use OS's authenticate dialog for WebAuth support, but this isn't required at start up etc. So we should move this to delay load DLL list.
Differential Revision: https://phabricator.services.mozilla.com/D42079
--HG--
extra : moz-landing-system : lando
The remaining uses all need adjustements to in-tree mozconfigs, so they
all need to be done at once.
However, to make things slightly more intelligible, we do this in two
steps. This is step 1: we modify the use_toolchain transform to take care of
the transformation, while keeping the task definitions intact, so that
we only deal with mozconfig and build script adjustements here.
Differential Revision: https://phabricator.services.mozilla.com/D41890
With the removal of the old Chromium file_util code, we should no longer
be using temporary files with names starting with "org.chromium.", so the
crash reporter and main thread I/O test no longer need to recognize that
prefix.
Differential Revision: https://phabricator.services.mozilla.com/D34629
--HG--
extra : moz-landing-system : lando
This will allow these heuristics to be used on mobile.
This depends on GeckoView supporting a JavaScript module
at resource:///modules/BrowserUsageTelemetry.jsm with the
interface described in nsIBrowserUsage.idl. Bug 1573238 has
been filed in order to implement this module on GeckoView.
Differential Revision: https://phabricator.services.mozilla.com/D41621
--HG--
extra : moz-landing-system : lando
When working on GeckoView, there's no need to produce a Fennec APK.
This commit avoids doing that work at `mach package` time. There are
many other things we'd like to stop doing as we remove Fennec from the
tree, so we add a general flag to guard such things.
Depends on D41447
Differential Revision: https://phabricator.services.mozilla.com/D41448
--HG--
extra : moz-landing-system : lando
`SyncedBookmarksMirror::merge` is racy: the storage thread might finish
merging before it gets the cancellation signal. However, `apply` also
notifies observers and fetches records, which happen after we've
aborted our JS controller. Even if the Rust op isn't canceled in time,
we should still abort the remaining operations.
Differential Revision: https://phabricator.services.mozilla.com/D42065
--HG--
extra : moz-landing-system : lando
Also remove most C++-side optimizations for avoiding calls to Rust
for short strings now that we have LTO between C++ and Rust. Since
LTO still leaves the overhead of one function call layer, inlined
function call avoidance optimization is left in place in the
IsUTF8 and in the 8-bit IsASCII cases for which perfherder flags
the difference as significant for the length 15.
Differential Revision: https://phabricator.services.mozilla.com/D40999
--HG--
extra : moz-landing-system : lando
It was used as an intermediate static library to ensure an order in
libxul-gtest wrt StaticXULComponents*, but those were removed in bug
1541792.
Differential Revision: https://phabricator.services.mozilla.com/D41098
--HG--
rename : toolkit/library/gtest/static/TestUCRTDepends.cpp => toolkit/library/gtest/TestUCRTDepends.cpp
extra : moz-landing-system : lando
When a local or remote item changed, we'd potentially scan three tables
(with an expensive `LEFT JOIN`!) to check if anything changed...then
scan the same tables again to build the local and remote trees. This
check was originally meant to avoid unnecessary merges. However, the
bottleneck isn't merging now; it's reading from the database.
Since the merger has been rewritten in Rust, is synchronous, doesn't
keep a transaction open for the entire merge (see the
`total_sync_changes` check), and only emits ops for items that actually
changed, it's more efficient to build and merge optimistically, and
bail before applying if nothing changed.
This commit also moves `validateLocalRoots` into Rust.
Differential Revision: https://phabricator.services.mozilla.com/D41690
--HG--
extra : moz-landing-system : lando
This commit changes `mozISyncedBookmarksMerger.merge` to return a
cancelable operation, and adds an abort `signal` option to
`SyncedBookmarksMirror.apply`. Aborting the signal's
`AbortController`, or finalizing the mirror, interrupts the merge.
Differential Revision: https://phabricator.services.mozilla.com/D41310
--HG--
extra : moz-landing-system : lando
In the content blocking log, these can be represented by different entries
in the events array.
Differential Revision: https://phabricator.services.mozilla.com/D41784
--HG--
extra : moz-landing-system : lando
This changes provide basic support for webextenion tabs and webNavigation listeners by implementing missing objects on which Fennec implementation was relying.
Differential Revision: https://phabricator.services.mozilla.com/D36575
--HG--
extra : moz-landing-system : lando
The `xul:menulist` appears a little offset but this is the same as it was prior to the fission patch that completely broke it.
If you have an alternate approach or a way to get the correct position for the `xul:menulist` I am all ears.
Differential Revision: https://phabricator.services.mozilla.com/D40805
--HG--
extra : moz-landing-system : lando
This makes prefs definition simpler, more consistent, and less error-prone.
The patch also changes the form of the "not Android" condition to one used more
widely in all.js.
Differential Revision: https://phabricator.services.mozilla.com/D41299
--HG--
extra : moz-landing-system : lando
This makes prefs definition simpler, more consistent, and less error-prone.
Differential Revision: https://phabricator.services.mozilla.com/D41298
--HG--
extra : moz-landing-system : lando
This changes the way crash reports for child processes happening too early
during the child process' startup. Before bug 1547698 we wrote a partial
.extra file with those crashes that lacked the process type. The user would
not be notified of those crashes until she restarted Firefox and even when
submitted those crashes would be erroneously labeled as browser crashes.
After bug 1547698 we stopped writing .extra files entirely for those crashes
which left orphaned .dmp files among the pending crash reports.
This patch does three things to improve the situation:
* It writes a partial .extra file so that the crashes are detected at the next
startup. So the user is still not notified directly of these crashes but she
can report them later.
* It adds the process type to the .extra file so that the crash reporters are
labelled correctly.
* It fixes a leak in the `pidToMinidump` hash-map. Since the crashes were
not finalized the `ChildProcessData` strucutre associated with them would
never be fred.
Differential Revision: https://phabricator.services.mozilla.com/D40810
--HG--
extra : moz-landing-system : lando
dlee
Cameron McCormack
Brindusan Cristian
Will Hawkins
rdalal
Mark Striemer
Mike Hommey
Dustin J. Mitchell
Robert Strong
Luca Greco
Coroiu Cristina
Dorel Luca
Luke Crouch
Ben Campbell
Drew Willcoxon
Gabriele Svelto
Nathan Froyd
Ehsan Akhgari
Emilio Cobos Álvarez
arthur.iakab
Landry Breuil
Markus Stange
ruthra kumar
lesleynorton
Noemi Erli
Sam Foster
Jan Odvarko
tanhengyeow
Olli Pettay
Aaron Klotz
Jan-Erik Rediger
Matthew Noorenberghe
Shane Caraveo
Mark Banner
Oana Pop Rus
Barret Rennie
Razvan Maries
Mike Conley
Matt Howell
Ryan Alderete
Thomas Nguyen
Carolina
Sylvestre Ledru
Nick Alexander
Brian Hackett
Dan Glastonbury
Gijs Kruitbosch
Kirk Steuber
Nihanth Subramanya
Micah Tigley
André Bargull
preyunk
Robert Helmer
harry
dleblanccyr
Bianca Danforth
Geoff Lankow
Toshihito Kikuchi
Abdoulaye O. Ly
Cosmin Sabou
Tim Nguyen
Andrew McCreight
Jan Varga
Ciure Andrei
chrmod
Megha
Moritz Birghan
Alexander Surkov
Kashav Madan
Eric Rahm
Daniel Varga
Jonathan Kew
Henri Sivonen
Matt Woodrow
Makoto Kato
singuliere
Paul Zuehlcke
Chris H-C
Andrea Marchesini
Trishul
Bogdan Tara
Jed Davis
Mihai Alexandru Michis
Lina Cambridge
Sebastian Streich
Michael Cooper
J.C. Jones
Johann Hofmann
Dan Minor
Alessio Placitelli
Itiel
Michael Ratcliffe
Nicholas Nethercote
Erica Wright
Liang-Heng Chen