* Improves UTF-8 validation performance.
* Improves UTF-8 to UTF-16 decode performance.
* Improves non-Latin and Latin1-ish Latin single-byte encode performance.
* Improves code quality by addressing some clippy lints.
The optional legacy CJK encoder changes are not used by Firefox.
Differential Revision: https://phabricator.services.mozilla.com/D12514
--HG--
extra : moz-landing-system : lando
The current rust panic hook keeps a string for the crash reporter, and
goes on calling the default rust panic hook, which prints out a crash
stack... when RUST_BOOTSTRAP is set *and* when that works. Notably, on
both mac and Windows, it only really works for local builds, but fails
for debug builds from automation, although on automation itself, we also
do stackwalk from crash minidumps, which alleviates the problem.
Artifact debug builds are affected, though.
More importantly, C++ calls to e.g. MOZ_CRASH have a similar but
different behavior, in that they dump a stack trace on debug builds, by
default (with exceptions, see below for one). The format of those stack
traces is understood by the various fix*stack*py scripts under
tools/rb/, that are used by the various test harnesses both on
automation and locally.
Additionally, the current rust panic hook, as it calls the default rust
panic hook, ends up calling abort() on non-Windows platforms, which ends
up being verbosely redirected to mozalloc_abort per
https://dxr.mozilla.org/mozilla-central/rev/237e4c0633fda8e227b2ab3ab57e417c980a2811/memory/mozalloc/mozalloc_abort.cpp#79
which then calls MOZ_CRASH. Theoretically, /that/ would also print a
stack trace, but doesn't because currently the stack trace printing code
lives in libxul, and MOZ_CRASH only calls it when compiled from
libxul-code, which mozalloc_abort is not part of.
With this change, we make the rust panic handler call back into
MOZ_CRASH directly. This has multiple advantages:
- This is more consistent cross-platforms (Windows is not special
anymore).
- This is more consistent between C++ and rust (stack traces all look
the same, and can all be post-processed by fix*stack*py if need be)
- This is more consistent in behavior, where debug builds will show
those stack traces without caring about environment variables.
- It demangles C++ symbols in rust-initiated stack traces (for some
reason that didn't happen with the rust panic handler)
A few downsides:
- the loss of demangling for some rust symbols.
- the loss of addresses in the stacks, although they're not entirely
useful
- extra empty lines.
The first should be fixable later one. The latter two are arguably
something that should be consistent across C++ and rust, and should be
changed if necessary, independently of this patch.
Depends on D11719
Depends on D11719
Differential Revision: https://phabricator.services.mozilla.com/D11720
--HG--
extra : moz-landing-system : lando
The current rust panic hook keeps a string for the crash reporter, and
goes on calling the default rust panic hook, which prints out a crash
stack... when RUST_BOOTSTRAP is set *and* when that works. Notably, on
both mac and Windows, it only really works for local builds, but fails
for debug builds from automation, although on automation itself, we also
do stackwalk from crash minidumps, which alleviates the problem.
Artifact debug builds are affected, though.
More importantly, C++ calls to e.g. MOZ_CRASH have a similar but
different behavior, in that they dump a stack trace on debug builds, by
default (with exceptions, see below for one). The format of those stack
traces is understood by the various fix*stack*py scripts under
tools/rb/, that are used by the various test harnesses both on
automation and locally.
Additionally, the current rust panic hook, as it calls the default rust
panic hook, ends up calling abort() on non-Windows platforms, which ends
up being verbosely redirected to mozalloc_abort per
https://dxr.mozilla.org/mozilla-central/rev/237e4c0633fda8e227b2ab3ab57e417c980a2811/memory/mozalloc/mozalloc_abort.cpp#79
which then calls MOZ_CRASH. Theoretically, /that/ would also print a
stack trace, but doesn't because currently the stack trace printing code
lives in libxul, and MOZ_CRASH only calls it when compiled from
libxul-code, which mozalloc_abort is not part of.
With this change, we make the rust panic handler call back into
MOZ_CRASH directly. This has multiple advantages:
- This is more consistent cross-platforms (Windows is not special
anymore).
- This is more consistent between C++ and rust (stack traces all look
the same, and can all be post-processed by fix*stack*py if need be)
- This is more consistent in behavior, where debug builds will show
those stack traces without caring about environment variables.
- It demangles C++ symbols in rust-initiated stack traces (for some
reason that didn't happen with the rust panic handler)
A few downsides:
- the loss of demangling for some rust symbols.
- the loss of addresses in the stacks, although they're not entirely
useful
- extra empty lines.
The first should be fixable later one. The latter two are arguably
something that should be consistent across C++ and rust, and should be
changed if necessary, independently of this patch.
Depends on D11719
Differential Revision: https://phabricator.services.mozilla.com/D11720
--HG--
extra : moz-landing-system : lando
The current rust panic hook keeps a string for the crash reporter, and
goes on calling the default rust panic hook, which prints out a crash
stack... when RUST_BOOTSTRAP is set *and* when that works. Notably, on
both mac and Windows, it only really works for local builds, but fails
for debug builds from automation, although on automation itself, we also
do stackwalk from crash minidumps, which alleviates the problem.
Artifact debug builds are affected, though.
More importantly, C++ calls to e.g. MOZ_CRASH have a similar but
different behavior, in that they dump a stack trace on debug builds, by
default (with exceptions, see below for one). The format of those stack
traces is understood by the various fix*stack*py scripts under
tools/rb/, that are used by the various test harnesses both on
automation and locally.
Additionally, the current rust panic hook, as it calls the default rust
panic hook, ends up calling abort() on non-Windows platforms, which ends
up being verbosely redirected to mozalloc_abort per
https://dxr.mozilla.org/mozilla-central/rev/237e4c0633fda8e227b2ab3ab57e417c980a2811/memory/mozalloc/mozalloc_abort.cpp#79
which then calls MOZ_CRASH. Theoretically, /that/ would also print a
stack trace, but doesn't because currently the stack trace printing code
lives in libxul, and MOZ_CRASH only calls it when compiled from
libxul-code, which mozalloc_abort is not part of.
With this change, we make the rust panic handler call back into
MOZ_CRASH directly. This has multiple advantages:
- This is more consistent cross-platforms (Windows is not special
anymore).
- This is more consistent between C++ and rust (stack traces all look
the same, and can all be post-processed by fix*stack*py if need be)
- This is more consistent in behavior, where debug builds will show
those stack traces without caring about environment variables.
- It demangles C++ symbols in rust-initiated stack traces (for some
reason that didn't happen with the rust panic handler)
A few downsides:
- the loss of demangling for some rust symbols.
- the loss of addresses in the stacks, although they're not entirely
useful
- extra empty lines.
The first should be fixable later one. The latter two are arguably
something that should be consistent across C++ and rust, and should be
changed if necessary, independently of this patch.
Depends on D11719
Differential Revision: https://phabricator.services.mozilla.com/D11720
--HG--
extra : moz-landing-system : lando
This is the equivalent of the rustc-workspace-hack used by the rust build to
ensure cargo and RLS see the same set of features for dependencies so that
these dependencies may be reused by invocations of cargo for these two
projects. The trivial crate added specifies the union of the set of
features activated for a particular crate for each time it appears in the
dependency tree so that cargo will understand these dependencies to be
re-usable across cargo implementations. This eliminates re-building jsrust
and some of its dependencies twice, and reduces the number of crates compiled
in the tree by about 90 in testing on linux.
Differential Revision: https://phabricator.services.mozilla.com/D9041
This pulls a new nom version, which is slightly unfortunate, but I do want some
of the fixes upstream, and it's build-only, so I think it's not a huge deal.
Differential Revision: https://phabricator.services.mozilla.com/D9362
Updating rkv to 0.5 enables us to un-vendor new-ordered-float, as rkv 0.4 is the last crate in the tree that depends on it.
It also enables us to un-vendor version 0.5 of uuid. We previously needed that version because multiple third-party crates depended on it, and we have limited control over third-party sub-dependencies. But rkv 0.4 was the last third-party crate that still depended on version 0.5 of uuid; rkv 0.5 depends on version 0.6 of uuid.
There would still be two internal crates that depend on version 0.5 of uuid: geckodriver and webrender_bindings. But we have more control over internal sub-dependencies, and we can update those two internal crates to depend on version 0.6 of uuid. This patch does so.
To summarize, this patch makes the following changes:
* rkv: 0.4 -> 0.5
* new-ordered-float: un-vendored
* geckodriver: uuid dependency 0.5 -> 0.6
* webrender_bindings: uuid dependency 0.5 -> 0.6
* uuid 0.5: un-vendored
* uuid 0.6: remains in tree
Differential Revision: https://phabricator.services.mozilla.com/D9160
--HG--
extra : moz-landing-system : lando
This is the equivalent of the rustc-workspace-hack used by the rust build to
ensure cargo and RLS see the same set of features for dependencies so that
these dependencies may be reused by invocations of cargo for these two
projects. The trivial crate added specifies the union of the set of
features activated for a particular crate for each time it appears in the
dependency tree so that cargo will understand these dependencies to be
re-usable across cargo implementations. This eliminates re-building jsrust
and some of its dependencies twice, and reduces the number of crates compiled
in the tree by about 90 in testing on linux.
Differential Revision: https://phabricator.services.mozilla.com/D9041
--HG--
extra : moz-landing-system : lando
Summary: The previous two changesets bump up a few dependencies. This is the companion mach rust vendor.
Test Plan: It builds.
Reviewers: ted
Tags: #secure-revision
Bug #: 1497446
--HG--
extra : amend_source : 6eeef28181e1e72891e1f3ad1d67b70cdf926e21
extra : histedit_source : e6af1b38e2272656c543f6c4f9778e80e6c75fd9
Only difference with current versions is the ability to use the crates
with stable rust without setting RUSTC_BOOTSTRAP globally ourselves.
Differential Revision: https://phabricator.services.mozilla.com/D7679
--HG--
extra : moz-landing-system : lando
* Update encoding_rs to 0.8.8.
* Change U+FEFD and U+FEFE to RTL in IS_RTL_PRESENTATION_FORM to make the
Rust and C++ code agree on what's RTL.
MozReview-Commit-ID: CuK6fN4pojG
Differential Revision: https://phabricator.services.mozilla.com/D7285
--HG--
extra : moz-landing-system : lando
Most importantly, this picks up "object" and "goblin" for ELF binary parsing.
We only use the ELF code from goblin, so the mach-O parsing code gets
eliminated by the linker. Overall, this increases the Android installer size
by 20KB.
Try pushes for reference:
before: https://treeherder.mozilla.org/#/jobs?repo=try&revision=834b56dc5ab3d63a43a32f740ee8212296ac726d&selectedJob=201600899
after: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6983b27e8d3cb715d3b7e6cbd276683f6466e3cc&selectedJob=201600475
installer size: 34524820 -> 34542861 (34.52MB -> 34.54MB)
$ mach vendor rust
Updating registry `https://github.com/rust-lang/crates.io-index`
Adding goblin v0.0.17
Adding memmap v0.6.2
Adding miniz-sys v0.1.10
Adding object v0.10.0
Adding parity-wasm v0.31.3
Adding plain v0.2.3
Adding profiler_helper v0.1.0 (file:///Users/mstange/code/mozilla/tools/profiler/rust-helper)
Adding scroll v0.9.1
Adding scroll_derive v0.9.5
Adding syn v0.15.5
Adding thin-vec v0.1.0
Adding uuid v0.6.5
0:30.11 The following files exceed the filesize limit of 102400:
third_party/rust/miniz-sys/miniz.c
third_party/rust/syn-0.14.6/src/expr.rs
third_party/rust/syn-0.14.6/src/gen/fold.rs
third_party/rust/syn-0.14.6/src/gen/visit.rs
third_party/rust/syn-0.14.6/src/gen/visit_mut.rs
The syn dependency is not compiled for goblin, as far as I can tell - it's only
needed for the 'syn' feature of scroll_derive, and scroll does not ask for
scroll_derive/syn.
object -> goblin -> scroll -> scroll_derive -/-> syn
But it looks like other versions of syn were already in the tree.
Depends on D7021
Differential Revision: https://phabricator.services.mozilla.com/D7023
--HG--
rename : third_party/rust/syn/src/parsers.rs => third_party/rust/syn-0.14.6/src/parsers.rs
rename : third_party/rust/syn/src/verbatim.rs => third_party/rust/syn-0.14.6/src/verbatim.rs
rename : third_party/rust/uuid/.travis.yml => third_party/rust/uuid-0.5.1/.travis.yml
rename : third_party/rust/uuid/src/rustc_serialize.rs => third_party/rust/uuid-0.5.1/src/rustc_serialize.rs
rename : third_party/rust/uuid/src/serde.rs => third_party/rust/uuid-0.5.1/src/serde.rs
extra : moz-landing-system : lando
Most importantly, this picks up "object" and "goblin" for ELF binary parsing.
We only use the ELF code from goblin, so the mach-O parsing code gets
eliminated by the linker. Overall, this increases the Android installer size
by 20KB.
Try pushes for reference:
before: https://treeherder.mozilla.org/#/jobs?repo=try&revision=834b56dc5ab3d63a43a32f740ee8212296ac726d&selectedJob=201600899
after: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6983b27e8d3cb715d3b7e6cbd276683f6466e3cc&selectedJob=201600475
installer size: 34524820 -> 34542861 (34.52MB -> 34.54MB)
$ mach vendor rust
Updating registry `https://github.com/rust-lang/crates.io-index`
Adding goblin v0.0.17
Adding memmap v0.6.2
Adding miniz-sys v0.1.10
Adding object v0.10.0
Adding parity-wasm v0.31.3
Adding plain v0.2.3
Adding profiler_helper v0.1.0 (file:///Users/mstange/code/mozilla/tools/profiler/rust-helper)
Adding scroll v0.9.1
Adding scroll_derive v0.9.5
Adding syn v0.15.5
Adding thin-vec v0.1.0
Adding uuid v0.6.5
0:30.11 The following files exceed the filesize limit of 102400:
third_party/rust/miniz-sys/miniz.c
third_party/rust/syn-0.14.6/src/expr.rs
third_party/rust/syn-0.14.6/src/gen/fold.rs
third_party/rust/syn-0.14.6/src/gen/visit.rs
third_party/rust/syn-0.14.6/src/gen/visit_mut.rs
The syn dependency is not compiled for goblin, as far as I can tell - it's only
needed for the 'syn' feature of scroll_derive, and scroll does not ask for
scroll_derive/syn.
object -> goblin -> scroll -> scroll_derive -/-> syn
But it looks like other versions of syn were already in the tree.
Depends on D7021
Differential Revision: https://phabricator.services.mozilla.com/D7023
--HG--
rename : third_party/rust/syn/src/parsers.rs => third_party/rust/syn-0.14.6/src/parsers.rs
rename : third_party/rust/syn/src/verbatim.rs => third_party/rust/syn-0.14.6/src/verbatim.rs
rename : third_party/rust/uuid/.travis.yml => third_party/rust/uuid-0.5.1/.travis.yml
rename : third_party/rust/uuid/src/rustc_serialize.rs => third_party/rust/uuid-0.5.1/src/rustc_serialize.rs
rename : third_party/rust/uuid/src/serde.rs => third_party/rust/uuid-0.5.1/src/serde.rs
extra : moz-landing-system : lando
We had a mix of 0.6.2 and 0.6.5 (which is the current release),
this unifies to the latest version. It also enables the union
feature which removes the discriminant, reducing memory usage.
This cherry-picks servo/servo#21746.
log 0.3.9 is still there because tokio-uds 0.1.7, which is a
dependency of audioipc, still depends on ^0.3. A separate bug will
be filed on WebRTC to upgrade their tokio-uds dependency.
Because all Rust crates in the tree are vendored, using the wildcard
"*" version dependency could have unintended reprecussions on
rsdparsa if another crate changes its log version dependency.
This patch, along with the others associated with this bug, upgrades
the log crate to 0.4.* throughout. This has the benefit that we
can get rid of the duplicate vendored log crates in third_party/rust.
By relaxing the version dependency on the log crate we are able to
get rid of duplicate vendored crates.
This particular version number, combined with rsdparsa depending
on "*" (anything) meant that a change to this version number in
audioipc would have reprecussions on the dependencies of rsdparsa.
I will address the wildcard version in a separate patch on rsdparsa.
The fix for bug 1482810 requires an upstream change to the cc crate: if CFLAGS is defined, then don't enable warnings by default. That change was included in cc crate version 1.0.23, and this change vendors that version of the cc crate into mozilla-central.
MozReview-Commit-ID: Krfrs1dSN9d
Differential Revision: https://phabricator.services.mozilla.com/D4699
--HG--
extra : moz-landing-system : lando
winapi 0.2.8 is used by a number of crates in our dependency graph. The
newest version of winapi is 0.3.4, which is a significant restructuring
and also more amenable to further development, e.g. adding AArch64
support. This patch does the easy work of updating as many things as
possible to winapi 0.3.4 via a simple `cargo update`:
cargo update -p atty:0.2.2 -p fs2 -p msdos_time -p parking_lot_core -p aho-corasick
and then vendoring the results of those changes.
To support that, this patch also does the following.
- Removes the insert(), remove() and might_contain() methods, because they are
specialized versions of insert_hash(), remove_hash(), and
might_contain_hash(), and they are only used by tests within this file.
- Moves hash() from the top level into create_and_insert_some_stuff().
- Changes create_and_insert_some_stuff() so that instead of hashing consecutive
integers, it instead hashes stringified consecutive integers, which matches
real usage a little better.
- Raises the false_positives limit a little to account for the above changes.
--HG--
extra : rebase_source : f839ff86986d5c8a17506a808ba676f69c7ef407
This moves most of the code to be Rust, except potentially some evaluator
functions, and allows to unblock the use case from any-hover / any-pointer and
remove nsMediaFeatures.
Differential Revision: https://phabricator.services.mozilla.com/D2976
Correctness improvements:
* UTF errors are handled safely per spec instead of dangerously truncating
strings.
* There are fewer converter implementations.
Performance improvements:
* The old code did exact buffer length math, which meant doing UTF math twice
on each input string (once for length calculation and another time for
conversion). Exact length math is more complicated when handling errors
properly, which the old code didn't do. The new code does UTF math on the
string content only once (when converting) but risks allocating more than
once. There are heuristics in place to lower the probability of
reallocation in cases where the double math avoidance isn't enough of a
saving to absorb an allocation and memcpy.
* Previously, in UTF-16 <-> UTF-8 conversions, an ASCII prefix was optimized
but a single non-ASCII code point pessimized the rest of the string. The
new code tries to get back on the fast ASCII path.
* UTF-16 to Latin1 conversion guarantees less about handling of out-of-range
input to eliminate an operation from the inner loop on x86/x86_64.
* When assigning to a pre-existing string, the new code tries to reuse the
old buffer instead of first releasing the old buffer and then allocating a
new one.
* When reallocating from the new code, the memcpy covers only the data that
is part of the logical length of the old string instead of memcpying the
whole capacity. (For old callers old excess memcpy behavior is preserved
due to bogus callers. See bug 1472113.)
* UTF-8 strings in XPConnect that are in the Latin1 range are passed to
SpiderMonkey as Latin1.
New features:
* Conversion between UTF-8 and Latin1 is added in order to enable faster
future interop between Rust code (or otherwise UTF-8-using code) and text
node and SpiderMonkey code that uses Latin1.
MozReview-Commit-ID: JaJuExfILM9
The cc crate had a bug where it would ignore compiler arguments in compilers
specified in environment variables. We update to a copy with that issue fixed:
9eaa56536e
MozReview-Commit-ID: FE8AywUEMoc
--HG--
extra : rebase_source : 565b3464a1be4278b211312c8703aadc53cab777
WR Updater Bot
Henri Sivonen
Mike Hommey
Dorel Luca
Coroiu Cristina
Cosmin Sabou
Dan Gohman
Emilio Cobos Álvarez
Gurzau Raul
Nathan Froyd
Margareta Eliza Balazs
Noemi Erli
Chris Manchester
Sebastian Hengst
Myk Melez
David Teller
Jeff Muizelaar
Alex Gaynor
J.C. Jones
Andreas Tolfsen
Markus Stange
shindli
Benjamin Bouvier
Matt Brubeck
Andre Bogus
Matt Brubeck
Tooru Fujisawa
Bastien Orivel
Jean-Yves Avenard
Henrik Skupin
Andreea Pavel
Nicholas Nethercote
James Graham
Daniel Varga
Kartikaya Gupta
Cameron McCormack
Ted Mielczarek