зеркало из https://github.com/mozilla/gecko-dev.git
Bug 1431356 - Update encoding_rs to 0.7.2 and simd to 0.2.1. r=emk.
MozReview-Commit-ID: Lp3zyF2rLxN --HG-- extra : rebase_source : 81b515206ca5d28623cbaead16244ef258da2088
This commit is contained in:
Henri Sivonen
Родитель
0e3d8ae3b5
Коммит
733ced6d8d
Различия файлов скрыты, потому что одна или несколько строк слишком длинны
|
|
@ -37,8 +37,8 @@ rustc.
|
|||
|
||||
## rustfmt
|
||||
|
||||
Please install [`rustfmt`](https://github.com/rust-lang-nursery/rustfmt) and
|
||||
run `cargo fmt` before creating a pull request.
|
||||
The `rustfmt` version used for this code is 0.8.4. Please either use that
|
||||
version or avoid using `rustfmt` (so as not to reformat all the code).
|
||||
|
||||
## Unit tests
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@
|
|||
|
||||
[package]
|
||||
name = "encoding_rs"
|
||||
version = "0.7.1"
|
||||
version = "0.7.2"
|
||||
authors = ["Henri Sivonen <hsivonen@hsivonen.fi>"]
|
||||
description = "A Gecko-oriented implementation of the Encoding Standard"
|
||||
homepage = "https://docs.rs/encoding_rs/"
|
||||
|
|
@ -24,6 +24,9 @@ license = "MIT/Apache-2.0"
|
|||
repository = "https://github.com/hsivonen/encoding_rs"
|
||||
[profile.release]
|
||||
lto = true
|
||||
[dependencies.cfg-if]
|
||||
version = "0.1.0"
|
||||
|
||||
[dependencies.serde]
|
||||
version = "1.0"
|
||||
optional = true
|
||||
|
|
@ -31,20 +34,17 @@ optional = true
|
|||
[dependencies.simd]
|
||||
version = "0.2.0"
|
||||
optional = true
|
||||
[dev-dependencies.bincode]
|
||||
version = "0.8"
|
||||
|
||||
[dependencies.cfg-if]
|
||||
version = "0.1.0"
|
||||
[dev-dependencies.serde_derive]
|
||||
version = "1.0"
|
||||
|
||||
[dev-dependencies.serde_json]
|
||||
version = "1.0"
|
||||
|
||||
[dev-dependencies.bincode]
|
||||
version = "0.8"
|
||||
|
||||
[features]
|
||||
simd-accel = ["simd"]
|
||||
no-static-ideograph-encoder-tables = []
|
||||
simd-accel = ["simd"]
|
||||
[badges.travis-ci]
|
||||
repository = "hsivonen/encoding_rs"
|
||||
|
|
|
|||
|
|
@ -9,6 +9,11 @@ encoding_rs an implementation of the (non-JavaScript parts of) the
|
|||
[Encoding Standard](https://encoding.spec.whatwg.org/) written in Rust and
|
||||
used in Gecko (starting with Firefox 56).
|
||||
|
||||
Additionally, the `mem` module provides various operations for dealing with
|
||||
in-RAM text (as opposed to data that's coming from or going to an IO boundary).
|
||||
The `mem` module is a module instead of a separate crate due to internal
|
||||
implementation detail efficiencies.
|
||||
|
||||
## Functionality
|
||||
|
||||
Due to the Gecko use case, encoding_rs supports decoding to and encoding from
|
||||
|
|
@ -43,6 +48,26 @@ Specifically, encoding_rs does the following:
|
|||
workloads than the standard library; hopefully will get upstreamed some
|
||||
day) and ASCII.
|
||||
|
||||
Additionally, `encoding_rs::mem` does the following:
|
||||
|
||||
* Checks if a byte buffer contains only ASCII.
|
||||
* Checks if a potentially-invalid UTF-16 buffer contains only Basic Latin (ASCII).
|
||||
* Checks if a valid UTF-8, potentially-invalid UTF-8 or potentially-invalid UTF-16
|
||||
buffer contains only Latin1 code points (below U+0100).
|
||||
* Checks if a valid UTF-8, potentially-invalid UTF-8 or potentially-invalid UTF-16
|
||||
buffer or a code point or a UTF-16 code unit can trigger right-to-left behavior
|
||||
(suitable for checking if the Unicode Bidirectional Algorithm can be optimized
|
||||
out).
|
||||
* Combined versions of the above two checks.
|
||||
* Converts valid UTF-8, potentially-invalid UTF-8 and Latin1 to UTF-16.
|
||||
* Converts potentially-invalid UTF-16 and Latin1 to UTF-8.
|
||||
* Converts UTF-8 and UTF-16 to Latin1 (if in range).
|
||||
* Finds the first invalid code unit in a buffer of potentially-invalid UTF-16.
|
||||
* Makes a mutable buffer of potential-invalid UTF-16 contain valid UTF-16.
|
||||
* Copies ASCII from one buffer to another up to the first non-ASCII byte.
|
||||
* Converts ASCII to UTF-16 up to the first non-ASCII byte.
|
||||
* Converts UTF-16 to ASCII up to the first non-Basic Latin code unit.
|
||||
|
||||
## Licensing
|
||||
|
||||
Please see the file named
|
||||
|
|
@ -63,6 +88,8 @@ using the C++ standard library and [GSL](https://github.com/Microsoft/GSL/) type
|
|||
For the Gecko context, there's a
|
||||
[C++ wrapper using the MFBT/XPCOM types](https://searchfox.org/mozilla-central/source/intl/Encoding.h#100).
|
||||
|
||||
These bindings do not cover the `mem` module.
|
||||
|
||||
## Sample programs
|
||||
|
||||
* [Rust](https://github.com/hsivonen/recode_rs)
|
||||
|
|
@ -133,9 +160,9 @@ decode-optimized tables. With realistic work loads, this seemed fast enough
|
|||
not to be user-visibly slow on Raspberry Pi 3 (which stood in for a phone
|
||||
for testing) in the Web-exposed encoder use cases.
|
||||
|
||||
A framework for measuring performance is [available separately][1].
|
||||
A framework for measuring performance is [available separately][2].
|
||||
|
||||
[1]: https://github.com/hsivonen/encoding_bench/
|
||||
[2]: https://github.com/hsivonen/encoding_bench/
|
||||
|
||||
## Rust Version Compatibility
|
||||
|
||||
|
|
@ -193,6 +220,12 @@ used in Firefox.
|
|||
|
||||
## Release Notes
|
||||
|
||||
### 0.7.2
|
||||
|
||||
* Add the `mem` module.
|
||||
* Refactor SIMD code which can affect performance outside the `mem`
|
||||
module.
|
||||
|
||||
### 0.7.1
|
||||
|
||||
* When encoding from invalid UTF-16, correctly handle U+DC00 followed by
|
||||
|
|
|
|||
|
|
@ -24,6 +24,14 @@
|
|||
#[cfg(all(feature = "simd-accel", any(target_feature = "sse2", all(target_endian = "little", target_arch = "aarch64"))))]
|
||||
use simd_funcs::*;
|
||||
|
||||
// `as` truncates, so works on 32-bit, too.
|
||||
#[allow(dead_code)]
|
||||
pub const ASCII_MASK: usize = 0x80808080_80808080u64 as usize;
|
||||
|
||||
// `as` truncates, so works on 32-bit, too.
|
||||
#[allow(dead_code)]
|
||||
pub const BASIC_LATIN_MASK: usize = 0xFF80FF80_FF80FF80u64 as usize;
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! ascii_naive {
|
||||
($name:ident,
|
||||
|
|
@ -212,6 +220,62 @@ macro_rules! basic_latin_alu {
|
|||
});
|
||||
}
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! latin1_alu {
|
||||
($name:ident,
|
||||
$src_unit:ty,
|
||||
$dst_unit:ty,
|
||||
$stride_fn:ident) => (
|
||||
#[cfg_attr(feature = "cargo-clippy", allow(never_loop))]
|
||||
#[inline(always)]
|
||||
pub unsafe fn $name(src: *const $src_unit, dst: *mut $dst_unit, len: usize) {
|
||||
let mut offset = 0usize;
|
||||
// This loop is only broken out of as a `goto` forward
|
||||
loop {
|
||||
let mut until_alignment = {
|
||||
if ::std::mem::size_of::<$src_unit>() < ::std::mem::size_of::<$dst_unit>() {
|
||||
// unpack
|
||||
let src_until_alignment = (ALIGNMENT - ((src as usize) & ALIGNMENT_MASK)) & ALIGNMENT_MASK;
|
||||
if (dst.offset(src_until_alignment as isize) as usize) & ALIGNMENT_MASK != 0 {
|
||||
break;
|
||||
}
|
||||
src_until_alignment
|
||||
} else {
|
||||
// pack
|
||||
let dst_until_alignment = (ALIGNMENT - ((dst as usize) & ALIGNMENT_MASK)) & ALIGNMENT_MASK;
|
||||
if (src.offset(dst_until_alignment as isize) as usize) & ALIGNMENT_MASK != 0 {
|
||||
break;
|
||||
}
|
||||
dst_until_alignment
|
||||
}
|
||||
};
|
||||
if until_alignment + STRIDE_SIZE <= len {
|
||||
while until_alignment != 0 {
|
||||
let code_unit = *(src.offset(offset as isize));
|
||||
*(dst.offset(offset as isize)) = code_unit as $dst_unit;
|
||||
offset += 1;
|
||||
until_alignment -= 1;
|
||||
}
|
||||
let len_minus_stride = len - STRIDE_SIZE;
|
||||
loop {
|
||||
$stride_fn(src.offset(offset as isize) as *const usize,
|
||||
dst.offset(offset as isize) as *mut usize);
|
||||
offset += STRIDE_SIZE;
|
||||
if offset > len_minus_stride {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
while offset < len {
|
||||
let code_unit = *(src.offset(offset as isize));
|
||||
*(dst.offset(offset as isize)) = code_unit as $dst_unit;
|
||||
offset += 1;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! ascii_simd_check_align {
|
||||
($name:ident,
|
||||
|
|
@ -294,6 +358,89 @@ macro_rules! ascii_simd_check_align {
|
|||
});
|
||||
}
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! latin1_simd_check_align {
|
||||
($name:ident,
|
||||
$src_unit:ty,
|
||||
$dst_unit:ty,
|
||||
$stride_both_aligned:ident,
|
||||
$stride_src_aligned:ident,
|
||||
$stride_dst_aligned:ident,
|
||||
$stride_neither_aligned:ident) => (
|
||||
#[inline(always)]
|
||||
pub unsafe fn $name(src: *const $src_unit, dst: *mut $dst_unit, len: usize) {
|
||||
let mut offset = 0usize;
|
||||
if STRIDE_SIZE <= len {
|
||||
let len_minus_stride = len - STRIDE_SIZE;
|
||||
// XXX Should we first process one stride unconditinoally as unaligned to
|
||||
// avoid the cost of the branchiness below if the first stride fails anyway?
|
||||
// XXX Should we just use unaligned SSE2 access unconditionally? It seems that
|
||||
// on Haswell, it would make sense to just use unaligned and not bother
|
||||
// checking. Need to benchmark older architectures before deciding.
|
||||
let dst_masked = (dst as usize) & ALIGNMENT_MASK;
|
||||
if ((src as usize) & ALIGNMENT_MASK) == 0 {
|
||||
if dst_masked == 0 {
|
||||
loop {
|
||||
$stride_both_aligned(src.offset(offset as isize),
|
||||
dst.offset(offset as isize));
|
||||
offset += STRIDE_SIZE;
|
||||
if offset > len_minus_stride {
|
||||
break;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
loop {
|
||||
$stride_src_aligned(src.offset(offset as isize),
|
||||
dst.offset(offset as isize));
|
||||
offset += STRIDE_SIZE;
|
||||
if offset > len_minus_stride {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if dst_masked == 0 {
|
||||
loop {
|
||||
$stride_dst_aligned(src.offset(offset as isize),
|
||||
dst.offset(offset as isize));
|
||||
offset += STRIDE_SIZE;
|
||||
if offset > len_minus_stride {
|
||||
break;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
loop {
|
||||
$stride_neither_aligned(src.offset(offset as isize),
|
||||
dst.offset(offset as isize));
|
||||
offset += STRIDE_SIZE;
|
||||
if offset > len_minus_stride {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
while offset < len {
|
||||
let code_unit = *(src.offset(offset as isize));
|
||||
// On x86_64, this loop autovectorizes but in the pack
|
||||
// case there are instructions whose purpose is to make sure
|
||||
// each u16 in the vector is truncated before packing. However,
|
||||
// since we don't care about saturating behavior of SSE2 packing
|
||||
// when the input isn't Latin1, those instructions are useless.
|
||||
// Unfortunately, using the `assume` intrinsic to lie to the
|
||||
// optimizer doesn't make LLVM omit the trunctation that we
|
||||
// don't need. Possibly this loop could be manually optimized
|
||||
// to do the sort of thing that LLVM does but without the
|
||||
// ANDing the read vectors of u16 with a constant that discards
|
||||
// the high half of each u16. As far as I can tell, the
|
||||
// optimization assumes that doing a SIMD read past the end of
|
||||
// the array is OK.
|
||||
*(dst.offset(offset as isize)) = code_unit as $dst_unit;
|
||||
offset += 1;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! ascii_simd_unalign {
|
||||
($name:ident,
|
||||
|
|
@ -328,6 +475,34 @@ macro_rules! ascii_simd_unalign {
|
|||
});
|
||||
}
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! latin1_simd_unalign {
|
||||
($name:ident,
|
||||
$src_unit:ty,
|
||||
$dst_unit:ty,
|
||||
$stride_neither_aligned:ident) => (
|
||||
#[inline(always)]
|
||||
pub unsafe fn $name(src: *const $src_unit, dst: *mut $dst_unit, len: usize) {
|
||||
let mut offset = 0usize;
|
||||
if STRIDE_SIZE <= len {
|
||||
let len_minus_stride = len - STRIDE_SIZE;
|
||||
loop {
|
||||
$stride_neither_aligned(src.offset(offset as isize),
|
||||
dst.offset(offset as isize));
|
||||
offset += STRIDE_SIZE;
|
||||
if offset > len_minus_stride {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
while offset < len {
|
||||
let code_unit = *(src.offset(offset as isize));
|
||||
*(dst.offset(offset as isize)) = code_unit as $dst_unit;
|
||||
offset += 1;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! ascii_to_ascii_simd_stride {
|
||||
($name:ident,
|
||||
|
|
@ -336,7 +511,7 @@ macro_rules! ascii_to_ascii_simd_stride {
|
|||
#[inline(always)]
|
||||
pub unsafe fn $name(src: *const u8, dst: *mut u8) -> bool {
|
||||
let simd = $load(src);
|
||||
if !is_ascii(simd) {
|
||||
if !simd_is_ascii(simd) {
|
||||
return false;
|
||||
}
|
||||
$store(dst, simd);
|
||||
|
|
@ -352,7 +527,7 @@ macro_rules! ascii_to_basic_latin_simd_stride {
|
|||
#[inline(always)]
|
||||
pub unsafe fn $name(src: *const u8, dst: *mut u16) -> bool {
|
||||
let simd = $load(src);
|
||||
if !is_ascii(simd) {
|
||||
if !simd_is_ascii(simd) {
|
||||
return false;
|
||||
}
|
||||
let (first, second) = simd_unpack(simd);
|
||||
|
|
@ -362,6 +537,20 @@ macro_rules! ascii_to_basic_latin_simd_stride {
|
|||
});
|
||||
}
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! unpack_simd_stride {
|
||||
($name:ident,
|
||||
$load:ident,
|
||||
$store:ident) => (
|
||||
#[inline(always)]
|
||||
pub unsafe fn $name(src: *const u8, dst: *mut u16) {
|
||||
let simd = $load(src);
|
||||
let (first, second) = simd_unpack(simd);
|
||||
$store(dst, first);
|
||||
$store(dst.offset(8), second);
|
||||
});
|
||||
}
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! basic_latin_to_ascii_simd_stride {
|
||||
($name:ident,
|
||||
|
|
@ -371,7 +560,7 @@ macro_rules! basic_latin_to_ascii_simd_stride {
|
|||
pub unsafe fn $name(src: *const u16, dst: *mut u8) -> bool {
|
||||
let first = $load(src);
|
||||
let second = $load(src.offset(8));
|
||||
if is_basic_latin(first | second) {
|
||||
if simd_is_basic_latin(first | second) {
|
||||
$store(dst, simd_pack(first, second));
|
||||
true
|
||||
} else {
|
||||
|
|
@ -380,23 +569,40 @@ macro_rules! basic_latin_to_ascii_simd_stride {
|
|||
});
|
||||
}
|
||||
|
||||
#[allow(unused_macros)]
|
||||
macro_rules! pack_simd_stride {
|
||||
($name:ident,
|
||||
$load:ident,
|
||||
$store:ident) => (
|
||||
#[inline(always)]
|
||||
pub unsafe fn $name(src: *const u16, dst: *mut u8) {
|
||||
let first = $load(src);
|
||||
let second = $load(src.offset(8));
|
||||
$store(dst, simd_pack(first, second));
|
||||
});
|
||||
}
|
||||
|
||||
cfg_if! {
|
||||
if #[cfg(all(feature = "simd-accel", target_endian = "little", target_arch = "aarch64"))] {
|
||||
// SIMD with the same instructions for aligned and unaligned loads and stores
|
||||
|
||||
pub const STRIDE_SIZE: usize = 16;
|
||||
|
||||
const ALIGNMENT: usize = 8;
|
||||
// pub const ALIGNMENT: usize = 8;
|
||||
|
||||
ascii_to_ascii_simd_stride!(ascii_to_ascii_stride_neither_aligned, load16_unaligned, store16_unaligned);
|
||||
|
||||
ascii_to_basic_latin_simd_stride!(ascii_to_basic_latin_stride_neither_aligned, load16_unaligned, store8_unaligned);
|
||||
unpack_simd_stride!(unpack_stride_neither_aligned, load16_unaligned, store8_unaligned);
|
||||
|
||||
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_neither_aligned, load8_unaligned, store16_unaligned);
|
||||
pack_simd_stride!(pack_stride_neither_aligned, load8_unaligned, store16_unaligned);
|
||||
|
||||
ascii_simd_unalign!(ascii_to_ascii, u8, u8, ascii_to_ascii_stride_neither_aligned);
|
||||
ascii_simd_unalign!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_neither_aligned);
|
||||
ascii_simd_unalign!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_neither_aligned);
|
||||
latin1_simd_unalign!(unpack_latin1, u8, u16, unpack_stride_neither_aligned);
|
||||
latin1_simd_unalign!(pack_latin1, u16, u8, pack_stride_neither_aligned);
|
||||
} else if #[cfg(all(feature = "simd-accel", target_feature = "sse2"))] {
|
||||
// SIMD with different instructions for aligned and unaligned loads and stores.
|
||||
//
|
||||
|
|
@ -406,7 +612,7 @@ cfg_if! {
|
|||
|
||||
pub const STRIDE_SIZE: usize = 16;
|
||||
|
||||
const ALIGNMENT_MASK: usize = 15;
|
||||
pub const ALIGNMENT_MASK: usize = 15;
|
||||
|
||||
ascii_to_ascii_simd_stride!(ascii_to_ascii_stride_both_aligned, load16_aligned, store16_aligned);
|
||||
ascii_to_ascii_simd_stride!(ascii_to_ascii_stride_src_aligned, load16_aligned, store16_unaligned);
|
||||
|
|
@ -418,31 +624,37 @@ cfg_if! {
|
|||
ascii_to_basic_latin_simd_stride!(ascii_to_basic_latin_stride_dst_aligned, load16_unaligned, store8_aligned);
|
||||
ascii_to_basic_latin_simd_stride!(ascii_to_basic_latin_stride_neither_aligned, load16_unaligned, store8_unaligned);
|
||||
|
||||
unpack_simd_stride!(unpack_stride_both_aligned, load16_aligned, store8_aligned);
|
||||
unpack_simd_stride!(unpack_stride_src_aligned, load16_aligned, store8_unaligned);
|
||||
unpack_simd_stride!(unpack_stride_dst_aligned, load16_unaligned, store8_aligned);
|
||||
unpack_simd_stride!(unpack_stride_neither_aligned, load16_unaligned, store8_unaligned);
|
||||
|
||||
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_both_aligned, load8_aligned, store16_aligned);
|
||||
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_src_aligned, load8_aligned, store16_unaligned);
|
||||
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_dst_aligned, load8_unaligned, store16_aligned);
|
||||
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_neither_aligned, load8_unaligned, store16_unaligned);
|
||||
|
||||
pack_simd_stride!(pack_stride_both_aligned, load8_aligned, store16_aligned);
|
||||
pack_simd_stride!(pack_stride_src_aligned, load8_aligned, store16_unaligned);
|
||||
pack_simd_stride!(pack_stride_dst_aligned, load8_unaligned, store16_aligned);
|
||||
pack_simd_stride!(pack_stride_neither_aligned, load8_unaligned, store16_unaligned);
|
||||
|
||||
ascii_simd_check_align!(ascii_to_ascii, u8, u8, ascii_to_ascii_stride_both_aligned, ascii_to_ascii_stride_src_aligned, ascii_to_ascii_stride_dst_aligned, ascii_to_ascii_stride_neither_aligned);
|
||||
ascii_simd_check_align!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_both_aligned, ascii_to_basic_latin_stride_src_aligned, ascii_to_basic_latin_stride_dst_aligned, ascii_to_basic_latin_stride_neither_aligned);
|
||||
ascii_simd_check_align!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_both_aligned, basic_latin_to_ascii_stride_src_aligned, basic_latin_to_ascii_stride_dst_aligned, basic_latin_to_ascii_stride_neither_aligned);
|
||||
latin1_simd_check_align!(unpack_latin1, u8, u16, unpack_stride_both_aligned, unpack_stride_src_aligned, unpack_stride_dst_aligned, unpack_stride_neither_aligned);
|
||||
latin1_simd_check_align!(pack_latin1, u16, u8, pack_stride_both_aligned, pack_stride_src_aligned, pack_stride_dst_aligned, pack_stride_neither_aligned);
|
||||
} else if #[cfg(all(target_endian = "little", target_pointer_width = "64"))] {
|
||||
// Aligned ALU word, little-endian, 64-bit
|
||||
|
||||
pub const STRIDE_SIZE: usize = 16;
|
||||
|
||||
const ALIGNMENT: usize = 8;
|
||||
pub const ALIGNMENT: usize = 8;
|
||||
|
||||
const ALIGNMENT_MASK: usize = 7;
|
||||
pub const ALIGNMENT_MASK: usize = 7;
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn ascii_to_basic_latin_stride_little_64(src: *const usize, dst: *mut usize) -> bool {
|
||||
let word = *src;
|
||||
let second_word = *(src.offset(1));
|
||||
// Check if the words contains non-ASCII
|
||||
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
unsafe fn unpack_alu(word: usize, second_word: usize, dst: *mut usize) {
|
||||
let first = ((0x00000000_FF000000usize & word) << 24) |
|
||||
((0x00000000_00FF0000usize & word) << 16) |
|
||||
((0x00000000_0000FF00usize & word) << 8) |
|
||||
|
|
@ -463,18 +675,10 @@ cfg_if! {
|
|||
*(dst.offset(1)) = second;
|
||||
*(dst.offset(2)) = third;
|
||||
*(dst.offset(3)) = fourth;
|
||||
true
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn basic_latin_to_ascii_stride_little_64(src: *const usize, dst: *mut usize) -> bool {
|
||||
let first = *src;
|
||||
let second = *(src.offset(1));
|
||||
let third = *(src.offset(2));
|
||||
let fourth = *(src.offset(3));
|
||||
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
unsafe fn pack_alu(first: usize, second: usize, third: usize, fourth: usize, dst: *mut usize) {
|
||||
let word = ((0x00FF0000_00000000usize & second) << 8) |
|
||||
((0x000000FF_00000000usize & second) << 16) |
|
||||
((0x00000000_00FF0000usize & second) << 24) |
|
||||
|
|
@ -493,28 +697,18 @@ cfg_if! {
|
|||
(0x00000000_000000FFusize & third);
|
||||
*dst = word;
|
||||
*(dst.offset(1)) = second_word;
|
||||
true
|
||||
}
|
||||
|
||||
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_little_64);
|
||||
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_little_64);
|
||||
} else if #[cfg(all(target_endian = "little", target_pointer_width = "32"))] {
|
||||
// Aligned ALU word, little-endian, 32-bit
|
||||
|
||||
pub const STRIDE_SIZE: usize = 8;
|
||||
|
||||
const ALIGNMENT: usize = 4;
|
||||
pub const ALIGNMENT: usize = 4;
|
||||
|
||||
const ALIGNMENT_MASK: usize = 3;
|
||||
pub const ALIGNMENT_MASK: usize = 3;
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn ascii_to_basic_latin_stride_little_32(src: *const usize, dst: *mut usize) -> bool {
|
||||
let word = *src;
|
||||
let second_word = *(src.offset(1));
|
||||
// Check if the words contains non-ASCII
|
||||
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
unsafe fn unpack_alu(word: usize, second_word: usize, dst: *mut usize) {
|
||||
let first = ((0x0000FF00usize & word) << 8) |
|
||||
(0x000000FFusize & word);
|
||||
let second = ((0xFF000000usize & word) >> 8) |
|
||||
|
|
@ -527,18 +721,10 @@ cfg_if! {
|
|||
*(dst.offset(1)) = second;
|
||||
*(dst.offset(2)) = third;
|
||||
*(dst.offset(3)) = fourth;
|
||||
return true;
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn basic_latin_to_ascii_stride_little_32(src: *const usize, dst: *mut usize) -> bool {
|
||||
let first = *src;
|
||||
let second = *(src.offset(1));
|
||||
let third = *(src.offset(2));
|
||||
let fourth = *(src.offset(3));
|
||||
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
unsafe fn pack_alu(first: usize, second: usize, third: usize, fourth: usize, dst: *mut usize) {
|
||||
let word = ((0x00FF0000usize & second) << 8) |
|
||||
((0x000000FFusize & second) << 16) |
|
||||
((0x00FF0000usize & first) >> 8) |
|
||||
|
|
@ -549,28 +735,18 @@ cfg_if! {
|
|||
(0x000000FFusize & third);
|
||||
*dst = word;
|
||||
*(dst.offset(1)) = second_word;
|
||||
return true;
|
||||
}
|
||||
|
||||
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_little_32);
|
||||
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_little_32);
|
||||
} else if #[cfg(all(target_endian = "big", target_pointer_width = "64"))] {
|
||||
// Aligned ALU word, big-endian, 64-bit
|
||||
|
||||
pub const STRIDE_SIZE: usize = 16;
|
||||
|
||||
const ALIGNMENT: usize = 8;
|
||||
pub const ALIGNMENT: usize = 8;
|
||||
|
||||
const ALIGNMENT_MASK: usize = 7;
|
||||
pub const ALIGNMENT_MASK: usize = 7;
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn ascii_to_basic_latin_stride_big_64(src: *const usize, dst: *mut usize) -> bool {
|
||||
let word = *src;
|
||||
let second_word = *(src.offset(1));
|
||||
// Check if the words contains non-ASCII
|
||||
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
unsafe fn unpack_alu(word: usize, second_word: usize, dst: *mut usize) {
|
||||
let first = ((0xFF000000_00000000usize & word) >> 8) |
|
||||
((0x00FF0000_00000000usize & word) >> 16) |
|
||||
((0x0000FF00_00000000usize & word) >> 24) |
|
||||
|
|
@ -591,18 +767,10 @@ cfg_if! {
|
|||
*(dst.offset(1)) = second;
|
||||
*(dst.offset(2)) = third;
|
||||
*(dst.offset(3)) = fourth;
|
||||
return true;
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn basic_latin_to_ascii_stride_big_64(src: *const usize, dst: *mut usize) -> bool {
|
||||
let first = *src;
|
||||
let second = *(src.offset(1));
|
||||
let third = *(src.offset(2));
|
||||
let fourth = *(src.offset(3));
|
||||
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
unsafe fn pack_alu(first: usize, second: usize, third: usize, fourth: usize, dst: *mut usize) {
|
||||
let word = ((0x00FF0000_00000000usize & first) << 8) |
|
||||
((0x000000FF_00000000usize & first) << 16) |
|
||||
((0x00000000_00FF0000usize & first) << 24) |
|
||||
|
|
@ -621,28 +789,18 @@ cfg_if! {
|
|||
(0x00000000_000000FFusize & fourth);
|
||||
*dst = word;
|
||||
*(dst.offset(1)) = second_word;
|
||||
return true;
|
||||
}
|
||||
|
||||
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_big_64);
|
||||
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_big_64);
|
||||
} else if #[cfg(all(target_endian = "big", target_pointer_width = "32"))] {
|
||||
// Aligned ALU word, big-endian, 32-bit
|
||||
|
||||
pub const STRIDE_SIZE: usize = 8;
|
||||
|
||||
const ALIGNMENT: usize = 4;
|
||||
pub const ALIGNMENT: usize = 4;
|
||||
|
||||
const ALIGNMENT_MASK: usize = 3;
|
||||
pub const ALIGNMENT_MASK: usize = 3;
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn ascii_to_basic_latin_stride_big_32(src: *const usize, dst: *mut usize) -> bool {
|
||||
let word = *src;
|
||||
let second_word = *(src.offset(1));
|
||||
// Check if the words contains non-ASCII
|
||||
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
unsafe fn unpack_alu(word: usize, second_word: usize, dst: *mut usize) {
|
||||
let first = ((0xFF000000usize & word) >> 8) |
|
||||
((0x00FF0000usize & word) >> 16);
|
||||
let second = ((0x0000FF00usize & word) << 8) |
|
||||
|
|
@ -655,18 +813,10 @@ cfg_if! {
|
|||
*(dst.offset(1)) = second;
|
||||
*(dst.offset(2)) = third;
|
||||
*(dst.offset(3)) = fourth;
|
||||
return true;
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn basic_latin_to_ascii_stride_big_32(src: *const usize, dst: *mut usize) -> bool {
|
||||
let first = *src;
|
||||
let second = *(src.offset(1));
|
||||
let third = *(src.offset(2));
|
||||
let fourth = *(src.offset(3));
|
||||
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
unsafe fn pack_alu(first: usize, second: usize, third: usize, fourth: usize, dst: *mut usize) {
|
||||
let word = ((0x00FF0000usize & first) << 8) |
|
||||
((0x000000FFusize & first) << 16) |
|
||||
((0x00FF0000usize & second) >> 8) |
|
||||
|
|
@ -677,11 +827,7 @@ cfg_if! {
|
|||
(0x000000FFusize & fourth);
|
||||
*dst = word;
|
||||
*(dst.offset(1)) = second_word;
|
||||
return true;
|
||||
}
|
||||
|
||||
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_big_32);
|
||||
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_big_32);
|
||||
} else {
|
||||
ascii_naive!(ascii_to_ascii, u8, u8);
|
||||
ascii_naive!(ascii_to_basic_latin, u8, u16);
|
||||
|
|
@ -716,7 +862,7 @@ cfg_if! {
|
|||
let len_minus_stride = len - STRIDE_SIZE;
|
||||
loop {
|
||||
let simd = unsafe { load16_unaligned(src.offset(offset as isize)) };
|
||||
if !is_ascii(simd) {
|
||||
if !simd_is_ascii(simd) {
|
||||
break;
|
||||
}
|
||||
offset += STRIDE_SIZE;
|
||||
|
|
@ -787,9 +933,51 @@ cfg_if! {
|
|||
None
|
||||
}
|
||||
} else {
|
||||
// `as` truncates, so works on 32-bit, too.
|
||||
const ASCII_MASK: usize = 0x80808080_80808080u64 as usize;
|
||||
const BASIC_LATIN_MASK: usize = 0xFF80FF80_FF80FF80u64 as usize;
|
||||
#[inline(always)]
|
||||
unsafe fn unpack_latin1_stride_alu(src: *const usize, dst: *mut usize) {
|
||||
let word = *src;
|
||||
let second_word = *(src.offset(1));
|
||||
unpack_alu(word, second_word, dst);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn pack_latin1_stride_alu(src: *const usize, dst: *mut usize) {
|
||||
let first = *src;
|
||||
let second = *(src.offset(1));
|
||||
let third = *(src.offset(2));
|
||||
let fourth = *(src.offset(3));
|
||||
pack_alu(first, second, third, fourth, dst);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn ascii_to_basic_latin_stride_alu(src: *const usize, dst: *mut usize) -> bool {
|
||||
let word = *src;
|
||||
let second_word = *(src.offset(1));
|
||||
// Check if the words contains non-ASCII
|
||||
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
unpack_alu(word, second_word, dst);
|
||||
true
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn basic_latin_to_ascii_stride_alu(src: *const usize, dst: *mut usize) -> bool {
|
||||
let first = *src;
|
||||
let second = *(src.offset(1));
|
||||
let third = *(src.offset(2));
|
||||
let fourth = *(src.offset(3));
|
||||
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
|
||||
return false;
|
||||
}
|
||||
pack_alu(first, second, third, fourth, dst);
|
||||
true
|
||||
}
|
||||
|
||||
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_alu);
|
||||
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_alu);
|
||||
latin1_alu!(unpack_latin1, u8, u16, unpack_latin1_stride_alu);
|
||||
latin1_alu!(pack_latin1, u16, u8, pack_latin1_stride_alu);
|
||||
|
||||
#[inline(always)]
|
||||
unsafe fn ascii_to_ascii_stride(src: *const usize, dst: *mut usize) -> Option<usize> {
|
||||
|
|
|
|||
|
|
@ -394,9 +394,7 @@ fn gbk_encode_non_unified(bmp: u16) -> Option<(usize, usize)> {
|
|||
// PUA between Hanzi Levels
|
||||
let bmp_minus_pua_between_hanzi = bmp.wrapping_sub(0xE810);
|
||||
if bmp_minus_pua_between_hanzi < 5 {
|
||||
return Some(
|
||||
(0x81 + 0x56, 0xFF - 5 + bmp_minus_pua_between_hanzi as usize),
|
||||
);
|
||||
return Some((0x81 + 0x56, 0xFF - 5 + bmp_minus_pua_between_hanzi as usize));
|
||||
}
|
||||
None
|
||||
}
|
||||
|
|
@ -595,6 +593,14 @@ mod tests {
|
|||
|
||||
// 0xFF
|
||||
decode_gb18030(b"\xFF\x40", "\u{FFFD}\u{0040}");
|
||||
decode_gb18030(b"\xE3\xFF\x9A\x33", "\u{FFFD}\u{FFFD}"); // not \u{FFFD}\u{FFFD}\u{0033} !
|
||||
decode_gb18030(b"\xFF\x32\x9A\x33", "\u{FFFD}\u{0032}\u{FFFD}"); // not \u{FFFD}\u{0032}\u{FFFD}\u{0033} !
|
||||
decode_gb18030(b"\xFF\x40\x00", "\u{FFFD}\u{0040}\u{0000}");
|
||||
decode_gb18030(b"\xE3\xFF\x9A\x33\x00", "\u{FFFD}\u{FFFD}\u{0033}\u{0000}");
|
||||
decode_gb18030(
|
||||
b"\xFF\x32\x9A\x33\x00",
|
||||
"\u{FFFD}\u{0032}\u{FFFD}\u{0033}\u{0000}",
|
||||
);
|
||||
|
||||
// Four bytes
|
||||
decode_gb18030(b"\x81\x30\x81\x30", "\u{0080}");
|
||||
|
|
@ -605,7 +611,7 @@ mod tests {
|
|||
decode_gb18030(b"\xE3\x32\x9A\x36\x81\x30", "\u{FFFD}\u{FFFD}");
|
||||
decode_gb18030(b"\xE3\x32\x9A\x36\x81\x40", "\u{FFFD}\u{4E02}");
|
||||
decode_gb18030(b"\xE3\x32\x9A", "\u{FFFD}"); // not \u{FFFD}\u{0032}\u{FFFD} !
|
||||
|
||||
decode_gb18030(b"\xE3\x32\x9A\x00", "\u{FFFD}\u{0032}\u{FFFD}\u{0000}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
// except according to those terms.
|
||||
|
||||
#![cfg_attr(feature = "cargo-clippy", allow(doc_markdown, inline_always, new_ret_no_self))]
|
||||
#![doc(html_root_url = "https://docs.rs/encoding_rs/0.7.1")]
|
||||
#![doc(html_root_url = "https://docs.rs/encoding_rs/0.7.2")]
|
||||
|
||||
//! encoding_rs is a Gecko-oriented Free Software / Open Source implementation
|
||||
//! of the [Encoding Standard](https://encoding.spec.whatwg.org/) in Rust.
|
||||
|
|
@ -17,6 +17,10 @@
|
|||
//! streamability goals are browser-oriented, and that FFI-friendliness is a
|
||||
//! goal.
|
||||
//!
|
||||
//! Additionally, the `mem` module provides functions that are useful for
|
||||
//! applications that need to be able to deal with legacy in-memory
|
||||
//! representations of Unicode.
|
||||
//!
|
||||
//! # Availability
|
||||
//!
|
||||
//! The code is available under the
|
||||
|
|
@ -491,7 +495,7 @@
|
|||
//! </tbody>
|
||||
//! </table>
|
||||
|
||||
#![cfg_attr(feature = "simd-accel", feature(cfg_target_feature, platform_intrinsics))]
|
||||
#![cfg_attr(feature = "simd-accel", feature(cfg_target_feature, platform_intrinsics, core_intrinsics))]
|
||||
|
||||
#[macro_use]
|
||||
extern crate cfg_if;
|
||||
|
|
@ -539,6 +543,8 @@ mod handles;
|
|||
mod data;
|
||||
mod variant;
|
||||
|
||||
pub mod mem;
|
||||
|
||||
use variant::*;
|
||||
use utf_8::utf8_valid_up_to;
|
||||
use ascii::ascii_valid_up_to;
|
||||
|
|
@ -2030,20 +2036,20 @@ static ENCODINGS_IN_LABEL_SORT: [&'static Encoding; 219] = [&WINDOWS_1252_INIT,
|
|||
/// # Streaming vs. Non-Streaming
|
||||
///
|
||||
/// When you have the entire input in a single buffer, you can use the
|
||||
/// methods [`decode()`][1], [`decode_with_bom_removal()`][2],
|
||||
/// [`decode_without_bom_handling()`][3],
|
||||
/// [`decode_without_bom_handling_and_without_replacement()`][4] and
|
||||
/// [`encode()`][5]. (These methods are available to Rust callers only and are
|
||||
/// methods [`decode()`][3], [`decode_with_bom_removal()`][3],
|
||||
/// [`decode_without_bom_handling()`][5],
|
||||
/// [`decode_without_bom_handling_and_without_replacement()`][6] and
|
||||
/// [`encode()`][7]. (These methods are available to Rust callers only and are
|
||||
/// not available in the C API.) Unlike the rest of the API available to Rust,
|
||||
/// these methods perform heap allocations. You should the `Decoder` and
|
||||
/// `Encoder` objects when your input is split into multiple buffers or when
|
||||
/// you want to control the allocation of the output buffers.
|
||||
///
|
||||
/// [1]: #method.decode
|
||||
/// [2]: #method.decode_with_bom_removal
|
||||
/// [3]: #method.decode_without_bom_handling
|
||||
/// [4]: #method.decode_without_bom_handling_and_without_replacement
|
||||
/// [5]: #method.encode
|
||||
/// [3]: #method.decode
|
||||
/// [4]: #method.decode_with_bom_removal
|
||||
/// [5]: #method.decode_without_bom_handling
|
||||
/// [6]: #method.decode_without_bom_handling_and_without_replacement
|
||||
/// [7]: #method.encode
|
||||
///
|
||||
/// # Instances
|
||||
///
|
||||
|
|
@ -2222,6 +2228,7 @@ impl Encoding {
|
|||
/// unsafe fallback for labels that `for_label()` maps to `Some(REPLACEMENT)`.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn for_label_no_replacement(label: &[u8]) -> Option<&'static Encoding> {
|
||||
match Encoding::for_label(label) {
|
||||
None => None,
|
||||
|
|
@ -2246,6 +2253,7 @@ impl Encoding {
|
|||
/// or UTF-16BE BOM or `None` otherwise.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn for_bom(buffer: &[u8]) -> Option<(&'static Encoding, usize)> {
|
||||
if buffer.starts_with(b"\xEF\xBB\xBF") {
|
||||
Some((UTF_8, 3))
|
||||
|
|
@ -2264,6 +2272,7 @@ impl Encoding {
|
|||
/// `document.characterSet` property.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn name(&'static self) -> &'static str {
|
||||
self.name
|
||||
}
|
||||
|
|
@ -2272,6 +2281,7 @@ impl Encoding {
|
|||
/// `char`. (Only true if the output encoding is UTF-8.)
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn can_encode_everything(&'static self) -> bool {
|
||||
self.output_encoding() == UTF_8
|
||||
}
|
||||
|
|
@ -2280,12 +2290,14 @@ impl Encoding {
|
|||
/// U+0000...U+007F and vice versa.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn is_ascii_compatible(&'static self) -> bool {
|
||||
!(self == REPLACEMENT || self == UTF_16BE || self == UTF_16LE || self == ISO_2022_JP)
|
||||
}
|
||||
|
||||
/// Checks whether the bytes 0x00...0x7F map mostly to the characters
|
||||
/// U+0000...U+007F and vice versa.
|
||||
#[inline]
|
||||
fn is_potentially_borrowable(&'static self) -> bool {
|
||||
!(self == REPLACEMENT || self == UTF_16BE || self == UTF_16LE)
|
||||
}
|
||||
|
|
@ -2294,6 +2306,7 @@ impl Encoding {
|
|||
/// UTF-16BE, UTF-16LE and replacement and the encoding itself otherwise.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn output_encoding(&'static self) -> &'static Encoding {
|
||||
if self == REPLACEMENT || self == UTF_16BE || self == UTF_16LE {
|
||||
UTF_8
|
||||
|
|
@ -2336,6 +2349,7 @@ impl Encoding {
|
|||
/// `usize`.
|
||||
///
|
||||
/// Available to Rust only.
|
||||
#[inline]
|
||||
pub fn decode<'a>(&'static self, bytes: &'a [u8]) -> (Cow<'a, str>, &'static Encoding, bool) {
|
||||
let (encoding, without_bom) = match Encoding::for_bom(bytes) {
|
||||
Some((encoding, bom_length)) => (encoding, &bytes[bom_length..]),
|
||||
|
|
@ -2378,6 +2392,7 @@ impl Encoding {
|
|||
/// `usize`.
|
||||
///
|
||||
/// Available to Rust only.
|
||||
#[inline]
|
||||
pub fn decode_with_bom_removal<'a>(&'static self, bytes: &'a [u8]) -> (Cow<'a, str>, bool) {
|
||||
let without_bom = if self == UTF_8 && bytes.starts_with(b"\xEF\xBB\xBF") {
|
||||
&bytes[3..]
|
||||
|
|
@ -2689,6 +2704,7 @@ impl Encoding {
|
|||
/// for UTF-8, UTF-16LE or UTF-16BE instead of this encoding.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn new_decoder(&'static self) -> Decoder {
|
||||
Decoder::new(self, self.new_variant_decoder(), BomHandling::Sniff)
|
||||
}
|
||||
|
|
@ -2702,6 +2718,7 @@ impl Encoding {
|
|||
/// encoding.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn new_decoder_with_bom_removal(&'static self) -> Decoder {
|
||||
Decoder::new(self, self.new_variant_decoder(), BomHandling::Remove)
|
||||
}
|
||||
|
|
@ -2717,6 +2734,7 @@ impl Encoding {
|
|||
/// instead of this method to cause the BOM to be removed.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn new_decoder_without_bom_handling(&'static self) -> Decoder {
|
||||
Decoder::new(self, self.new_variant_decoder(), BomHandling::Off)
|
||||
}
|
||||
|
|
@ -2724,6 +2742,7 @@ impl Encoding {
|
|||
/// Instantiates a new encoder for the output encoding of this encoding.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn new_encoder(&'static self) -> Encoder {
|
||||
let enc = self.output_encoding();
|
||||
enc.variant.new_encoder(enc)
|
||||
|
|
@ -2767,6 +2786,7 @@ impl Encoding {
|
|||
}
|
||||
|
||||
impl PartialEq for Encoding {
|
||||
#[inline]
|
||||
fn eq(&self, other: &Encoding) -> bool {
|
||||
(self as *const Encoding) == (other as *const Encoding)
|
||||
}
|
||||
|
|
@ -2775,12 +2795,14 @@ impl PartialEq for Encoding {
|
|||
impl Eq for Encoding {}
|
||||
|
||||
impl Hash for Encoding {
|
||||
#[inline]
|
||||
fn hash<H: Hasher>(&self, state: &mut H) {
|
||||
(self as *const Encoding).hash(state);
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for Encoding {
|
||||
#[inline]
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
|
||||
write!(f, "Encoding {{ {} }}", self.name)
|
||||
}
|
||||
|
|
@ -2788,6 +2810,7 @@ impl std::fmt::Debug for Encoding {
|
|||
|
||||
#[cfg(feature = "serde")]
|
||||
impl Serialize for Encoding {
|
||||
#[inline]
|
||||
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
|
||||
where S: Serializer
|
||||
{
|
||||
|
|
@ -3054,6 +3077,7 @@ impl Decoder {
|
|||
/// of the decoder.
|
||||
///
|
||||
/// Available via the C wrapper.
|
||||
#[inline]
|
||||
pub fn encoding(&self) -> &'static Encoding {
|
||||
self.encoding
|
||||
}
|
||||
|
|
@ -3769,12 +3793,14 @@ impl Encoder {
|
|||
}
|
||||
|
||||
/// The `Encoding` this `Encoder` is for.
|
||||
#[inline]
|
||||
pub fn encoding(&self) -> &'static Encoding {
|
||||
self.encoding
|
||||
}
|
||||
|
||||
/// Returns `true` if this is an ISO-2022-JP encoder that's not in the
|
||||
/// ASCII state and `false` otherwise.
|
||||
#[inline]
|
||||
pub fn has_pending_state(&self) -> bool {
|
||||
self.variant.has_pending_state()
|
||||
}
|
||||
|
|
@ -4111,6 +4137,16 @@ fn in_range16(i: u16, start: u16, end: u16) -> bool {
|
|||
i.wrapping_sub(start) < (end - start)
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn in_range32(i: u32, start: u32, end: u32) -> bool {
|
||||
i.wrapping_sub(start) < (end - start)
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn in_inclusive_range8(i: u8, start: u8, end: u8) -> bool {
|
||||
i.wrapping_sub(start) <= (end - start)
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn in_inclusive_range16(i: u16, start: u16, end: u16) -> bool {
|
||||
i.wrapping_sub(start) <= (end - start)
|
||||
|
|
|
|||
Разница между файлами не показана из-за своего большого размера
Загрузить разницу
|
|
@ -21,6 +21,7 @@ pub unsafe fn load16_unaligned(ptr: *const u8) -> u8x16 {
|
|||
simd
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[inline(always)]
|
||||
pub unsafe fn load16_aligned(ptr: *const u8) -> u8x16 {
|
||||
*(ptr as *const u8x16)
|
||||
|
|
@ -31,6 +32,7 @@ pub unsafe fn store16_unaligned(ptr: *mut u8, s: u8x16) {
|
|||
::std::ptr::copy_nonoverlapping(&s as *const u8x16 as *const u8, ptr, 16);
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[inline(always)]
|
||||
pub unsafe fn store16_aligned(ptr: *mut u8, s: u8x16) {
|
||||
*(ptr as *mut u8x16) = s;
|
||||
|
|
@ -43,6 +45,7 @@ pub unsafe fn load8_unaligned(ptr: *const u16) -> u16x8 {
|
|||
simd
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[inline(always)]
|
||||
pub unsafe fn load8_aligned(ptr: *const u16) -> u16x8 {
|
||||
*(ptr as *const u16x8)
|
||||
|
|
@ -53,6 +56,7 @@ pub unsafe fn store8_unaligned(ptr: *mut u16, s: u16x8) {
|
|||
::std::ptr::copy_nonoverlapping(&s as *const u16x8 as *const u8, ptr as *mut u8, 16);
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[inline(always)]
|
||||
pub unsafe fn store8_aligned(ptr: *mut u16, s: u16x8) {
|
||||
*(ptr as *mut u16x8) = s;
|
||||
|
|
@ -89,7 +93,7 @@ cfg_if! {
|
|||
cfg_if! {
|
||||
if #[cfg(target_feature = "sse2")] {
|
||||
#[inline(always)]
|
||||
pub fn is_ascii(s: u8x16) -> bool {
|
||||
pub fn simd_is_ascii(s: u8x16) -> bool {
|
||||
unsafe {
|
||||
let signed: i8x16 = ::std::mem::transmute_copy(&s);
|
||||
x86_mm_movemask_epi8(signed) == 0
|
||||
|
|
@ -101,16 +105,42 @@ cfg_if! {
|
|||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn is_ascii(s: u8x16) -> bool {
|
||||
pub fn simd_is_ascii(s: u8x16) -> bool {
|
||||
unsafe {
|
||||
aarch64_vmaxvq_u8(s) < 0x80
|
||||
}
|
||||
}
|
||||
} else {
|
||||
#[inline(always)]
|
||||
pub fn is_ascii(s: u8x16) -> bool {
|
||||
let highest_ascii = u8x16::splat(0x7F);
|
||||
!s.gt(highest_ascii).any()
|
||||
pub fn simd_is_ascii(s: u8x16) -> bool {
|
||||
let above_ascii = u8x16::splat(0x80);
|
||||
s.lt(above_ascii).all()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
cfg_if! {
|
||||
if #[cfg(target_feature = "sse2")] {
|
||||
#[inline(always)]
|
||||
pub fn simd_is_str_latin1(s: u8x16) -> bool {
|
||||
if simd_is_ascii(s) {
|
||||
return true;
|
||||
}
|
||||
let above_str_latin1 = u8x16::splat(0xC4);
|
||||
s.lt(above_str_latin1).all()
|
||||
}
|
||||
} else if #[cfg(target_arch = "aarch64")]{
|
||||
#[inline(always)]
|
||||
pub fn simd_is_str_latin1(s: u8x16) -> bool {
|
||||
unsafe {
|
||||
aarch64_vmaxvq_u8(s) < 0xC4
|
||||
}
|
||||
}
|
||||
} else {
|
||||
#[inline(always)]
|
||||
pub fn simd_is_str_latin1(s: u8x16) -> bool {
|
||||
let above_str_latin1 = u8x16::splat(0xC4);
|
||||
s.lt(above_str_latin1).all()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -122,20 +152,107 @@ cfg_if! {
|
|||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn is_basic_latin(s: u16x8) -> bool {
|
||||
pub fn simd_is_basic_latin(s: u16x8) -> bool {
|
||||
unsafe {
|
||||
aarch64_vmaxvq_u16(s) < 0x80
|
||||
}
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn simd_is_latin1(s: u16x8) -> bool {
|
||||
unsafe {
|
||||
aarch64_vmaxvq_u16(s) < 0x100
|
||||
}
|
||||
}
|
||||
} else {
|
||||
#[inline(always)]
|
||||
pub fn is_basic_latin(s: u16x8) -> bool {
|
||||
let highest_ascii = u16x8::splat(0x7F);
|
||||
!s.gt(highest_ascii).any()
|
||||
pub fn simd_is_basic_latin(s: u16x8) -> bool {
|
||||
let above_ascii = u16x8::splat(0x80);
|
||||
s.lt(above_ascii).all()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn simd_is_latin1(s: u16x8) -> bool {
|
||||
// For some reason, on SSE2 this formulation
|
||||
// seems faster in this case while the above
|
||||
// function is better the other way round...
|
||||
let highest_latin1 = u16x8::splat(0xFF);
|
||||
!s.gt(highest_latin1).any()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn contains_surrogates(s: u16x8) -> bool {
|
||||
let mask = u16x8::splat(0xF800);
|
||||
let surrogate_bits = u16x8::splat(0xD800);
|
||||
(s & mask).eq(surrogate_bits).any()
|
||||
}
|
||||
|
||||
cfg_if! {
|
||||
if #[cfg(target_arch = "aarch64")]{
|
||||
macro_rules! aarch64_return_false_if_below_hebrew {
|
||||
($s:ident) => ({
|
||||
unsafe {
|
||||
if aarch64_vmaxvq_u16($s) < 0x0590 {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
macro_rules! non_aarch64_return_false_if_all {
|
||||
($s:ident) => ()
|
||||
}
|
||||
} else {
|
||||
macro_rules! aarch64_return_false_if_below_hebrew {
|
||||
($s:ident) => ()
|
||||
}
|
||||
|
||||
macro_rules! non_aarch64_return_false_if_all {
|
||||
($s:ident) => ({
|
||||
if $s.all() {
|
||||
return false;
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! in_range16x8 {
|
||||
($s:ident, $start:expr, $end:expr) => ({
|
||||
// SIMD sub is wrapping
|
||||
($s - u16x8::splat($start)).lt(u16x8::splat($end - $start))
|
||||
})
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn is_u16x8_bidi(s: u16x8) -> bool {
|
||||
// We try to first quickly refute the RTLness of the vector. If that
|
||||
// fails, we do the real RTL check, so in that case we end up wasting
|
||||
// the work for the up-front quick checks. Even the quick-check is
|
||||
// two-fold in order to return `false` ASAP if everything is below
|
||||
// Hebrew.
|
||||
|
||||
aarch64_return_false_if_below_hebrew!(s);
|
||||
|
||||
let below_hebrew = s.lt(u16x8::splat(0x0590));
|
||||
|
||||
non_aarch64_return_false_if_all!(below_hebrew);
|
||||
|
||||
if (below_hebrew | in_range16x8!(s, 0x0900, 0x200F) | in_range16x8!(s, 0x2068, 0xD802)).all() {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Quick refutation failed. Let's do the full check.
|
||||
|
||||
(in_range16x8!(s, 0x0590, 0x0900) | in_range16x8!(s, 0xFB50, 0xFE00) |
|
||||
in_range16x8!(s, 0xFE70, 0xFF00) | in_range16x8!(s, 0xD802, 0xD804) |
|
||||
in_range16x8!(s, 0xD83A, 0xD83C) | s.eq(u16x8::splat(0x200F)) |
|
||||
s.eq(u16x8::splat(0x202B)) | s.eq(u16x8::splat(0x202E)) | s.eq(u16x8::splat(0x2067)))
|
||||
.any()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn simd_unpack(s: u8x16) -> (u16x8, u16x8) {
|
||||
unsafe {
|
||||
|
|
@ -206,7 +323,7 @@ mod tests {
|
|||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_basic_latin_success() {
|
||||
fn test_simd_is_basic_latin_success() {
|
||||
let ascii: [u8; 16] = [0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70, 0x71,
|
||||
0x72, 0x73, 0x74, 0x75, 0x76];
|
||||
let basic_latin: [u16; 16] = [0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70,
|
||||
|
|
@ -216,7 +333,7 @@ mod tests {
|
|||
let mut vec = Vec::with_capacity(16);
|
||||
vec.resize(16, 0u8);
|
||||
let ptr = vec.as_mut_ptr();
|
||||
assert!(is_basic_latin(first | second));
|
||||
assert!(simd_is_basic_latin(first | second));
|
||||
unsafe {
|
||||
store16_unaligned(ptr, simd_pack(first, second));
|
||||
}
|
||||
|
|
@ -224,46 +341,46 @@ mod tests {
|
|||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_basic_latin_c0() {
|
||||
fn test_simd_is_basic_latin_c0() {
|
||||
let input: [u16; 16] = [0x61, 0x62, 0x63, 0x81, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70, 0x71,
|
||||
0x72, 0x73, 0x74, 0x75, 0x76];
|
||||
let first = unsafe { load8_unaligned(input.as_ptr()) };
|
||||
let second = unsafe { load8_unaligned(input.as_ptr().offset(8)) };
|
||||
assert!(!is_basic_latin(first | second));
|
||||
assert!(!simd_is_basic_latin(first | second));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_basic_latin_0fff() {
|
||||
fn test_simd_is_basic_latin_0fff() {
|
||||
let input: [u16; 16] = [0x61, 0x62, 0x63, 0x0FFF, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70,
|
||||
0x71, 0x72, 0x73, 0x74, 0x75, 0x76];
|
||||
let first = unsafe { load8_unaligned(input.as_ptr()) };
|
||||
let second = unsafe { load8_unaligned(input.as_ptr().offset(8)) };
|
||||
assert!(!is_basic_latin(first | second));
|
||||
assert!(!simd_is_basic_latin(first | second));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_basic_latin_ffff() {
|
||||
fn test_simd_is_basic_latin_ffff() {
|
||||
let input: [u16; 16] = [0x61, 0x62, 0x63, 0xFFFF, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70,
|
||||
0x71, 0x72, 0x73, 0x74, 0x75, 0x76];
|
||||
let first = unsafe { load8_unaligned(input.as_ptr()) };
|
||||
let second = unsafe { load8_unaligned(input.as_ptr().offset(8)) };
|
||||
assert!(!is_basic_latin(first | second));
|
||||
assert!(!simd_is_basic_latin(first | second));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_ascii_success() {
|
||||
fn test_simd_is_ascii_success() {
|
||||
let ascii: [u8; 16] = [0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70, 0x71,
|
||||
0x72, 0x73, 0x74, 0x75, 0x76];
|
||||
let simd = unsafe { load16_unaligned(ascii.as_ptr()) };
|
||||
assert!(is_ascii(simd));
|
||||
assert!(simd_is_ascii(simd));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_ascii_failure() {
|
||||
fn test_simd_is_ascii_failure() {
|
||||
let input: [u8; 16] = [0x61, 0x62, 0x63, 0x64, 0x81, 0x66, 0x67, 0x68, 0x69, 0x70, 0x71,
|
||||
0x72, 0x73, 0x74, 0x75, 0x76];
|
||||
let simd = unsafe { load16_unaligned(input.as_ptr()) };
|
||||
assert!(!is_ascii(simd));
|
||||
assert!(!simd_is_ascii(simd));
|
||||
}
|
||||
|
||||
#[cfg(target_feature = "sse2")]
|
||||
|
|
|
|||
|
|
@ -22,13 +22,20 @@ pub fn decode(encoding: &'static Encoding, bytes: &[u8], expect: &str) {
|
|||
}
|
||||
vec.extend_from_slice(bytes);
|
||||
string.push_str(expect);
|
||||
decode_without_padding(encoding, &vec[..], &string[..]);
|
||||
decode_without_padding_impl(encoding, &vec[..], &string[..], i);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn decode_without_padding(encoding: &'static Encoding, bytes: &[u8], expect: &str) {
|
||||
decode_to_utf8(encoding, bytes, expect);
|
||||
decode_to_utf16(encoding, bytes, &utf16_from_utf8(expect)[..]);
|
||||
decode_without_padding_impl(encoding, bytes, expect, 0);
|
||||
}
|
||||
|
||||
fn decode_without_padding_impl(encoding: &'static Encoding,
|
||||
bytes: &[u8],
|
||||
expect: &str,
|
||||
padding: usize) {
|
||||
decode_to_utf8_impl(encoding, bytes, expect, padding);
|
||||
decode_to_utf16_impl(encoding, bytes, &utf16_from_utf8(expect)[..], padding);
|
||||
decode_to_string(encoding, bytes, expect);
|
||||
}
|
||||
|
||||
|
|
@ -56,40 +63,116 @@ pub fn encode_without_padding(encoding: &'static Encoding, string: &str, expect:
|
|||
}
|
||||
|
||||
pub fn decode_to_utf16(encoding: &'static Encoding, bytes: &[u8], expect: &[u16]) {
|
||||
decode_to_utf16_impl(encoding, bytes, expect, 0);
|
||||
}
|
||||
|
||||
pub fn decode_to_utf16_impl(encoding: &'static Encoding,
|
||||
bytes: &[u8],
|
||||
expect: &[u16],
|
||||
padding: usize) {
|
||||
for i in padding..bytes.len() {
|
||||
let (head, tail) = bytes.split_at(i);
|
||||
decode_to_utf16_with_boundary(encoding, head, tail, expect);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn decode_to_utf16_with_boundary(encoding: &'static Encoding,
|
||||
head: &[u8],
|
||||
tail: &[u8],
|
||||
expect: &[u16]) {
|
||||
let mut decoder = encoding.new_decoder();
|
||||
let mut dest: Vec<u16> =
|
||||
Vec::with_capacity(decoder.max_utf16_buffer_length(bytes.len()).unwrap());
|
||||
let mut dest: Vec<u16> = Vec::with_capacity(
|
||||
decoder
|
||||
.max_utf16_buffer_length(head.len() + tail.len())
|
||||
.unwrap()
|
||||
);
|
||||
let capacity = dest.capacity();
|
||||
dest.resize(capacity, 0u16);
|
||||
let (complete, read, written, _) = decoder.decode_to_utf16(bytes, &mut dest, true);
|
||||
match complete {
|
||||
CoderResult::InputEmpty => {}
|
||||
CoderResult::OutputFull => {
|
||||
unreachable!();
|
||||
let mut total_read = 0;
|
||||
let mut total_written = 0;
|
||||
{
|
||||
let (complete, read, written, _) = decoder.decode_to_utf16(head, &mut dest, false);
|
||||
match complete {
|
||||
CoderResult::InputEmpty => {}
|
||||
CoderResult::OutputFull => {
|
||||
unreachable!();
|
||||
}
|
||||
}
|
||||
total_read += read;
|
||||
total_written += written;
|
||||
}
|
||||
assert_eq!(read, bytes.len());
|
||||
assert_eq!(written, expect.len());
|
||||
dest.truncate(written);
|
||||
{
|
||||
let (complete, read, written, _) =
|
||||
decoder.decode_to_utf16(tail, &mut dest[total_written..], true);
|
||||
match complete {
|
||||
CoderResult::InputEmpty => {}
|
||||
CoderResult::OutputFull => {
|
||||
unreachable!();
|
||||
}
|
||||
}
|
||||
total_read += read;
|
||||
total_written += written;
|
||||
}
|
||||
assert_eq!(total_read, head.len() + tail.len());
|
||||
assert_eq!(total_written, expect.len());
|
||||
dest.truncate(total_written);
|
||||
assert_eq!(&dest[..], expect);
|
||||
}
|
||||
|
||||
pub fn decode_to_utf8(encoding: &'static Encoding, bytes: &[u8], expect: &str) {
|
||||
decode_to_utf8_impl(encoding, bytes, expect, 0);
|
||||
}
|
||||
|
||||
pub fn decode_to_utf8_impl(encoding: &'static Encoding,
|
||||
bytes: &[u8],
|
||||
expect: &str,
|
||||
padding: usize) {
|
||||
for i in padding..bytes.len() {
|
||||
let (head, tail) = bytes.split_at(i);
|
||||
decode_to_utf8_with_boundary(encoding, head, tail, expect);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn decode_to_utf8_with_boundary(encoding: &'static Encoding,
|
||||
head: &[u8],
|
||||
tail: &[u8],
|
||||
expect: &str) {
|
||||
let mut decoder = encoding.new_decoder();
|
||||
let mut dest: Vec<u8> =
|
||||
Vec::with_capacity(decoder.max_utf8_buffer_length(bytes.len()).unwrap());
|
||||
let mut dest: Vec<u8> = Vec::with_capacity(
|
||||
decoder
|
||||
.max_utf8_buffer_length(head.len() + tail.len())
|
||||
.unwrap()
|
||||
);
|
||||
let capacity = dest.capacity();
|
||||
dest.resize(capacity, 0u8);
|
||||
let (complete, read, written, _) = decoder.decode_to_utf8(bytes, &mut dest, true);
|
||||
match complete {
|
||||
CoderResult::InputEmpty => {}
|
||||
CoderResult::OutputFull => {
|
||||
unreachable!();
|
||||
let mut total_read = 0;
|
||||
let mut total_written = 0;
|
||||
{
|
||||
let (complete, read, written, _) = decoder.decode_to_utf8(head, &mut dest, false);
|
||||
match complete {
|
||||
CoderResult::InputEmpty => {}
|
||||
CoderResult::OutputFull => {
|
||||
unreachable!();
|
||||
}
|
||||
}
|
||||
total_read += read;
|
||||
total_written += written;
|
||||
}
|
||||
assert_eq!(read, bytes.len());
|
||||
assert_eq!(written, expect.len());
|
||||
dest.truncate(written);
|
||||
{
|
||||
let (complete, read, written, _) =
|
||||
decoder.decode_to_utf8(tail, &mut dest[total_written..], true);
|
||||
match complete {
|
||||
CoderResult::InputEmpty => {}
|
||||
CoderResult::OutputFull => {
|
||||
unreachable!();
|
||||
}
|
||||
}
|
||||
total_read += read;
|
||||
total_written += written;
|
||||
}
|
||||
assert_eq!(total_read, head.len() + tail.len());
|
||||
assert_eq!(total_written, expect.len());
|
||||
dest.truncate(total_written);
|
||||
assert_eq!(&dest[..], expect.as_bytes());
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -34,21 +34,21 @@ cfg_if! {
|
|||
}
|
||||
}
|
||||
|
||||
const UTF8_NORMAL_TRAIL: u8 = 1 << 3;
|
||||
pub const UTF8_NORMAL_TRAIL: u8 = 1 << 3;
|
||||
|
||||
const UTF8_THREE_BYTE_SPECIAL_LOWER_BOUND_TRAIL: u8 = 1 << 4;
|
||||
pub const UTF8_THREE_BYTE_SPECIAL_LOWER_BOUND_TRAIL: u8 = 1 << 4;
|
||||
|
||||
const UTF8_THREE_BYTE_SPECIAL_UPPER_BOUND_TRAIL: u8 = 1 << 5;
|
||||
pub const UTF8_THREE_BYTE_SPECIAL_UPPER_BOUND_TRAIL: u8 = 1 << 5;
|
||||
|
||||
const UTF8_FOUR_BYTE_SPECIAL_LOWER_BOUND_TRAIL: u8 = 1 << 6;
|
||||
pub const UTF8_FOUR_BYTE_SPECIAL_LOWER_BOUND_TRAIL: u8 = 1 << 6;
|
||||
|
||||
const UTF8_FOUR_BYTE_SPECIAL_UPPER_BOUND_TRAIL: u8 = 1 << 7;
|
||||
pub const UTF8_FOUR_BYTE_SPECIAL_UPPER_BOUND_TRAIL: u8 = 1 << 7;
|
||||
|
||||
// BEGIN GENERATED CODE. PLEASE DO NOT EDIT.
|
||||
// Instead, please regenerate using generate-encoding-data.py
|
||||
|
||||
/// Bit is 1 if the trail is invalid.
|
||||
static UTF8_TRAIL_INVALID: [u8; 256] =
|
||||
pub static UTF8_TRAIL_INVALID: [u8; 256] =
|
||||
[248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248,
|
||||
248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248,
|
||||
248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248,
|
||||
|
|
@ -433,16 +433,18 @@ pub struct Utf8Decoder {
|
|||
}
|
||||
|
||||
impl Utf8Decoder {
|
||||
pub fn new_inner() -> Utf8Decoder {
|
||||
Utf8Decoder {
|
||||
code_point: 0,
|
||||
bytes_seen: 0,
|
||||
bytes_needed: 0,
|
||||
lower_boundary: 0x80u8,
|
||||
upper_boundary: 0xBFu8,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn new() -> VariantDecoder {
|
||||
VariantDecoder::Utf8(
|
||||
Utf8Decoder {
|
||||
code_point: 0,
|
||||
bytes_seen: 0,
|
||||
bytes_needed: 0,
|
||||
lower_boundary: 0x80u8,
|
||||
upper_boundary: 0xBFu8,
|
||||
}
|
||||
)
|
||||
VariantDecoder::Utf8(Utf8Decoder::new_inner())
|
||||
}
|
||||
|
||||
fn extra_from_state(&self) -> usize {
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
{"files":{".travis.yml":"e2c720c3633b7671efce49147c62b12bcbf630d7c5d6fc65cd97620bfa4ddcea","Cargo.toml":"608aad04f17a524ee21048fa2ce9f656ae344e0473dd0e331dc954f0f9677c63","LICENSE-APACHE":"a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2","LICENSE-MIT":"6d3a9431e65e69c73a8923e6517b889d17549b23db406b9ec027710d16af701f","README.md":"249294a9a5f63c64c0f7fe4a607060f43f3507dce2378067aa59d25fb3ae681d","benches/mandelbrot.rs":"051b5199e66bca6cf7774e9024915fd4e1349ab39726a10a14e06b60d65d87a4","benches/matrix.rs":"048a21dacdb62365e0105d00d2c8cd6bd2396ac81134f2bff7eb4f7d095fb735","examples/axpy.rs":"4307626045d64ec08361c97c9c72c5dc8d361bdc88f64453b97ac0212041a1b2","examples/convert.rs":"8e658fde050f8a0d8b84ad7570446b10fcf544afbd551b940ca340474f324840","examples/dot-product.rs":"6fe2e007c147af5353804173a593c5b9d57dbccec156e1da37e9e32537363f91","examples/fannkuch-redux-nosimd.rs":"7b2fbde35e8666929d14d67328471cb0483d038a5325232f8db148b30865312b","examples/fannkuch-redux.rs":"ea21fdbd2274488a62cc984acad6e0b65d52f24fb4ff63b7057a3a667e9c8aae","examples/mandelbrot.rs":"8b8fdca1edac50e5a33e0e0592bd41eb75114f31839ccd40d485c61a9a664380","examples/matrix-inverse.rs":"a378d20ef20c2119bb10a86de27c92fec2c2f77f374e6bfd36707c9825a5fe92","examples/nbody-nosimd.rs":"2c8e0a7feacd202fdd65eeceb6420d6e9f43340b81f20a8e532704a587a2796b","examples/nbody.rs":"a864311affab262024479d6348ff51af43d809e9ad332ec30ea4aacceaa2eae1","examples/ops.rs":"1316f915d0afcfa98fdc4077e965ccccf6b4b21c433cbe487ff0cdc60df3cd39","examples/spectral-norm-nosimd.rs":"ffc8512ecde779078ea467f38f423a0ea623c63da7078193f9dd370200773f79","examples/spectral-norm.rs":"edb09c9d477f83939098cfb77a27cc298bc7a0c8a8e29cece0cccae0d70d890e","src/aarch64/mod.rs":"83f52775364c98de0cecb7e1509530c18972e932469f5f1522aa24a735d0fa37","src/aarch64/neon.rs":"1fe769979e07d8e2bc3c78ce116e05d735860744efe097a894cc9421153257fb","src/arm/mod.rs":"dcdd90bc0b39abaf86a0c8946d442b16313563fbae1ff03248628275c74d8617","src/arm/neon.rs":"51cc509856200e80f8e4cc2c982586e6d1cef593ec4537e153dce0cfe31d3428","src/common.rs":"62f4e7e0fefb52ad190d0f2191bc435ac4deab3f2bc70dc427f2a7f9ccb7856e","src/lib.rs":"25f0b39c038fa85af858318135dfd87865be26c33bb4bd1438aec96a1e68d8b5","src/sixty_four.rs":"510a9e00189a61e4f0a5beb7052d5dee37fc8261f94a2af45ef10327e0f3b7df","src/v256.rs":"2e328e49034876d535e0627c7a62191da2b4fb156a657614bf531a5fc75b1385","src/x86/avx.rs":"c66140abefca634b48eae307c3ec8cf5a40f2279b10e246a7e2ac602a2a2bb28","src/x86/avx2.rs":"efe3006b13a13261a3dec3d37dc1d8cb53950f3803c420069231803374949937","src/x86/mod.rs":"0acc5a5e2672e2a0fddc11065663be8b8fa2da87320ea291fa86ff8c2f33edf5","src/x86/sse2.rs":"5ceda75a401958a135fc9d851b22075314cdeed69fd483b6a7be4f11373f40da","src/x86/sse3.rs":"9bd01a4f08069ca4f445952e744d651efe887e3835b18872e757375f0d053bd2","src/x86/sse4_1.rs":"9ceb80dd70a7e7dfeef508cb935e1a2637175bc87a3b090f5dea691ff6aa0516","src/x86/sse4_2.rs":"c59321aed8decdce4d0d8570cff46aed02e1a8265647ef7702e9b180fc581254","src/x86/ssse3.rs":"2290f0269bae316b8e0491495645ee38a9bd73525c8572759c1328341c3bdb4c"},"package":"7a94d14a2ae1f1f110937de5fb69e494372560181c7e1739a097fcc2cee37ba0"}
|
||||
{"files":{".travis.yml":"e2c720c3633b7671efce49147c62b12bcbf630d7c5d6fc65cd97620bfa4ddcea","Cargo.toml":"27c6a208f0c6253c4580508311d49bb421944abd272a7f9a5a38b51ef657aec2","LICENSE-APACHE":"a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2","LICENSE-MIT":"6d3a9431e65e69c73a8923e6517b889d17549b23db406b9ec027710d16af701f","README.md":"249294a9a5f63c64c0f7fe4a607060f43f3507dce2378067aa59d25fb3ae681d","benches/mandelbrot.rs":"051b5199e66bca6cf7774e9024915fd4e1349ab39726a10a14e06b60d65d87a4","benches/matrix.rs":"048a21dacdb62365e0105d00d2c8cd6bd2396ac81134f2bff7eb4f7d095fb735","examples/axpy.rs":"4307626045d64ec08361c97c9c72c5dc8d361bdc88f64453b97ac0212041a1b2","examples/convert.rs":"8e658fde050f8a0d8b84ad7570446b10fcf544afbd551b940ca340474f324840","examples/dot-product.rs":"6fe2e007c147af5353804173a593c5b9d57dbccec156e1da37e9e32537363f91","examples/fannkuch-redux-nosimd.rs":"7b2fbde35e8666929d14d67328471cb0483d038a5325232f8db148b30865312b","examples/fannkuch-redux.rs":"ea21fdbd2274488a62cc984acad6e0b65d52f24fb4ff63b7057a3a667e9c8aae","examples/mandelbrot.rs":"71be242543c1e487145d7f16341c05d05d86109de4d9e94c5d6bc9a9c6ed9766","examples/matrix-inverse.rs":"93dbc55c66a72e5f7bc730072f35682523fa20dd362755d8443ad6982143cb5d","examples/nbody-nosimd.rs":"9cf46ea02e266c20f811318f1c5856d5afb9575b2d48d552fbd978f5c1856bdb","examples/nbody.rs":"a864311affab262024479d6348ff51af43d809e9ad332ec30ea4aacceaa2eae1","examples/ops.rs":"b08ea83583df71d0052895d677320a9888da5b6729c9b70636d31ede5128bb7f","examples/spectral-norm-nosimd.rs":"ffc8512ecde779078ea467f38f423a0ea623c63da7078193f9dd370200773f79","examples/spectral-norm.rs":"edb09c9d477f83939098cfb77a27cc298bc7a0c8a8e29cece0cccae0d70d890e","src/aarch64/mod.rs":"83f52775364c98de0cecb7e1509530c18972e932469f5f1522aa24a735d0fa37","src/aarch64/neon.rs":"3c05ea43b7261b9af9c0d904b37de01c2ba99caedcb464700f16617b672965a1","src/arm/mod.rs":"dcdd90bc0b39abaf86a0c8946d442b16313563fbae1ff03248628275c74d8617","src/arm/neon.rs":"00aed2c94455b7ff5755b7598fb166a94c7242ad9adf4e5379560ab04af560e7","src/common.rs":"c5a7b937c5cd8c3bccf0fb20d5d77770c0d9b0dd9fa06a661c6f2ddf118e65c0","src/lib.rs":"08c345b6a2ad641daa3c1a40b1dcc6e4f9047939414bd81b05051fc74a563fec","src/sixty_four.rs":"d168776d02acf943bda8044b24e644b7a9584197a223eba1a7c3024b205dc87d","src/v256.rs":"34bfde3676e23f6925db5d0408ae838e3aab7706128fd7c33e855b8579c69318","src/x86/avx.rs":"efcf2120a904a89b0adf2d3d3bdd0ca17df2ec058410af23fb7e81915873f808","src/x86/avx2.rs":"3bcb3f391ad5f16f0a6da0bc1301329beb478ad6265bd3b2c9c124fc2e6198e5","src/x86/mod.rs":"0acc5a5e2672e2a0fddc11065663be8b8fa2da87320ea291fa86ff8c2f33edf5","src/x86/sse2.rs":"8807fb04bbfb404e17fcacf1e21d22616f8b377540a227b1fd03c121879122dd","src/x86/sse3.rs":"9bd01a4f08069ca4f445952e744d651efe887e3835b18872e757375f0d053bd2","src/x86/sse4_1.rs":"9ceb80dd70a7e7dfeef508cb935e1a2637175bc87a3b090f5dea691ff6aa0516","src/x86/sse4_2.rs":"c59321aed8decdce4d0d8570cff46aed02e1a8265647ef7702e9b180fc581254","src/x86/ssse3.rs":"2290f0269bae316b8e0491495645ee38a9bd73525c8572759c1328341c3bdb4c"},"package":"3dd0805c7363ab51a829a1511ad24b6ed0349feaa756c4bc2f977f9f496e6673"}
|
||||
|
|
@ -1,25 +1,36 @@
|
|||
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
|
||||
#
|
||||
# When uploading crates to the registry Cargo will automatically
|
||||
# "normalize" Cargo.toml files for maximal compatibility
|
||||
# with all versions of Cargo and also rewrite `path` dependencies
|
||||
# to registry (e.g. crates.io) dependencies
|
||||
#
|
||||
# If you believe there's an error in this file please file an
|
||||
# issue against the rust-lang/cargo repository. If you're
|
||||
# editing this file be aware that the upstream Cargo.toml
|
||||
# will likely look very different (and much more reasonable)
|
||||
|
||||
[package]
|
||||
name = "simd"
|
||||
version = "0.2.0"
|
||||
version = "0.2.1"
|
||||
authors = ["Huon Wilson <dbau.pp+github@gmail.com>"]
|
||||
|
||||
repository = "https://github.com/rust-lang-nursery/simd"
|
||||
description = "`simd` offers limited cross-platform access to SIMD instructions on\nCPUs, as well as raw interfaces to platform-specific instructions.\n"
|
||||
documentation = "https://rust-lang-nursery.github.io/simd/doc/simd"
|
||||
license = "MIT/Apache-2.0"
|
||||
keywords = ["simd", "data-parallel"]
|
||||
readme = "README.md"
|
||||
keywords = ["simd", "data-parallel"]
|
||||
license = "MIT/Apache-2.0"
|
||||
repository = "https://github.com/rust-lang-nursery/simd"
|
||||
[package.metadata.docs.rs]
|
||||
features = ["doc"]
|
||||
[dependencies.serde]
|
||||
version = "1.0"
|
||||
optional = true
|
||||
|
||||
description = """
|
||||
`simd` offers limited cross-platform access to SIMD instructions on
|
||||
CPUs, as well as raw interfaces to platform-specific instructions.
|
||||
"""
|
||||
|
||||
[dependencies]
|
||||
serde = { version = "0.8", optional = true }
|
||||
serde_derive = { version = "0.8", optional = true }
|
||||
|
||||
[dev-dependencies]
|
||||
cfg-if = "0.1"
|
||||
[dependencies.serde_derive]
|
||||
version = "1.0"
|
||||
optional = true
|
||||
[dev-dependencies.cfg-if]
|
||||
version = "0.1"
|
||||
|
||||
[features]
|
||||
doc = []
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
#![feature(step_by, test)]
|
||||
#![feature(iterator_step_by, test)]
|
||||
|
||||
extern crate test;
|
||||
extern crate simd;
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ fn mul(x: &[f32x4; 4], y: &[f32x4; 4]) -> [f32x4; 4] {
|
|||
]
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
fn inverse_naive(x: &[[f32; 4]; 4]) -> [[f32; 4]; 4] {
|
||||
let mut t = [[0_f32; 4]; 4];
|
||||
for i in 0..4 {
|
||||
|
|
|
|||
|
|
@ -66,7 +66,7 @@ struct Planet {
|
|||
}
|
||||
|
||||
fn advance(bodies: &mut [Planet;N_BODIES], dt: f64, steps: i32) {
|
||||
for _ in (0..steps) {
|
||||
for _ in 0..steps {
|
||||
let mut b_slice: &mut [_] = bodies;
|
||||
loop {
|
||||
let bi = match shift_mut_ref(&mut b_slice) {
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ extern crate simd;
|
|||
|
||||
use simd::*;
|
||||
|
||||
#[allow(unused_variables)]
|
||||
fn main() {
|
||||
let x = i32x4::splat(1_i32);
|
||||
let y = -x;
|
||||
|
|
|
|||
|
|
@ -630,7 +630,7 @@ impl Aarch64I8x16 for i8x16 {
|
|||
#[doc(hidden)]
|
||||
pub mod common {
|
||||
use super::super::super::*;
|
||||
use std::mem;
|
||||
use core::mem;
|
||||
|
||||
#[inline]
|
||||
pub fn f32x4_sqrt(x: f32x4) -> f32x4 {
|
||||
|
|
|
|||
|
|
@ -473,7 +473,7 @@ impl u8x8 {
|
|||
pub mod common {
|
||||
use super::super::super::*;
|
||||
use super::*;
|
||||
use std::mem;
|
||||
use core::mem;
|
||||
|
||||
#[inline]
|
||||
pub fn f32x4_sqrt(x: f32x4) -> f32x4 {
|
||||
|
|
|
|||
|
|
@ -9,8 +9,7 @@ use super::{
|
|||
|
||||
Unalign, bitcast,
|
||||
};
|
||||
use std::mem;
|
||||
use std::ops;
|
||||
use core::{mem,ops};
|
||||
|
||||
#[cfg(any(target_arch = "x86",
|
||||
target_arch = "x86_64"))]
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
//! `simd` offers a basic interface to the SIMD functionality of CPUs.
|
||||
#![no_std]
|
||||
|
||||
#![feature(cfg_target_feature, repr_simd, platform_intrinsics, const_fn)]
|
||||
#![allow(non_camel_case_types)]
|
||||
|
|
@ -9,6 +10,8 @@ extern crate serde;
|
|||
#[macro_use]
|
||||
extern crate serde_derive;
|
||||
|
||||
use core::mem;
|
||||
|
||||
/// Boolean type for 8-bit integers.
|
||||
#[cfg_attr(feature = "with-serde", derive(Serialize, Deserialize))]
|
||||
#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
|
||||
|
|
@ -172,9 +175,9 @@ simd! {
|
|||
#[allow(dead_code)]
|
||||
#[inline]
|
||||
fn bitcast<T: Simd, U: Simd>(x: T) -> U {
|
||||
assert_eq!(std::mem::size_of::<T>(),
|
||||
std::mem::size_of::<U>());
|
||||
unsafe {std::mem::transmute_copy(&x)}
|
||||
assert_eq!(mem::size_of::<T>(),
|
||||
mem::size_of::<U>());
|
||||
unsafe {mem::transmute_copy(&x)}
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
|
|
@ -207,9 +210,15 @@ extern "platform-intrinsic" {
|
|||
fn simd_xor<T: Simd>(x: T, y: T) -> T;
|
||||
}
|
||||
#[repr(packed)]
|
||||
#[derive(Debug, Copy, Clone)]
|
||||
#[derive(Copy)]
|
||||
struct Unalign<T>(T);
|
||||
|
||||
impl<T: Clone> Clone for Unalign<T> {
|
||||
fn clone(&self) -> Unalign<T> {
|
||||
Unalign(unsafe { self.0.clone() })
|
||||
}
|
||||
}
|
||||
|
||||
#[macro_use]
|
||||
mod common;
|
||||
mod sixty_four;
|
||||
|
|
|
|||
|
|
@ -11,8 +11,7 @@ use super::{
|
|||
|
||||
Unalign, bitcast,
|
||||
};
|
||||
use std::mem;
|
||||
use std::ops;
|
||||
use core::{mem,ops};
|
||||
|
||||
/// Boolean type for 64-bit integers.
|
||||
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
#![allow(dead_code)]
|
||||
use std::ops;
|
||||
use std::mem;
|
||||
use core::{mem,ops};
|
||||
#[allow(unused_imports)]
|
||||
use super::{
|
||||
Simd,
|
||||
|
|
@ -329,6 +328,19 @@ impl i32x8 {
|
|||
}
|
||||
}
|
||||
|
||||
impl f32x8 {
|
||||
/// Convert each lane to a signed integer.
|
||||
#[inline]
|
||||
pub fn to_i32(self) -> i32x8 {
|
||||
unsafe {simd_cast(self)}
|
||||
}
|
||||
/// Convert each lane to an unsigned integer.
|
||||
#[inline]
|
||||
pub fn to_u32(self) -> u32x8 {
|
||||
unsafe {simd_cast(self)}
|
||||
}
|
||||
}
|
||||
|
||||
impl i16x16 {
|
||||
/// Convert each lane to an unsigned integer.
|
||||
#[inline]
|
||||
|
|
|
|||
|
|
@ -54,7 +54,7 @@ extern "platform-intrinsic" {
|
|||
#[doc(hidden)]
|
||||
pub mod common {
|
||||
use super::*;
|
||||
use std::mem;
|
||||
use core::mem;
|
||||
|
||||
macro_rules! bools {
|
||||
($($ty: ty, $all: ident, $any: ident, $testc: ident, $testz: ident;)*) => {
|
||||
|
|
|
|||
|
|
@ -42,7 +42,7 @@ extern "platform-intrinsic" {
|
|||
fn x86_mm256_packus_epi32(x: i32x8, y: i32x8) -> u16x16;
|
||||
fn x86_mm256_permutevar8x32_epi32(x: i32x8, y: i32x8) -> i32x8;
|
||||
fn x86_mm256_permutevar8x32_ps(x: f32x8, y: i32x8) -> f32x8;
|
||||
fn x86_mm256_sad_epu8(x: u8x32, y: u8x32) -> u8x32;
|
||||
fn x86_mm256_sad_epu8(x: u8x32, y: u8x32) -> u64x4;
|
||||
fn x86_mm256_shuffle_epi8(x: i8x32, y: i8x32) -> i8x32;
|
||||
fn x86_mm256_sign_epi8(x: i8x32, y: i8x32) -> i8x32;
|
||||
fn x86_mm256_sign_epi16(x: i16x16, y: i16x16) -> i16x16;
|
||||
|
|
|
|||
|
|
@ -48,7 +48,7 @@ extern "platform-intrinsic" {
|
|||
#[doc(hidden)]
|
||||
pub mod common {
|
||||
use super::super::super::*;
|
||||
use std::mem;
|
||||
use core::mem;
|
||||
|
||||
#[inline]
|
||||
pub fn f32x4_sqrt(x: f32x4) -> f32x4 {
|
||||
|
|
|
|||
|
|
@ -420,25 +420,25 @@ name = "encoding_c"
|
|||
version = "0.8.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
dependencies = [
|
||||
"encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding_glue"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"nserror 0.1.0",
|
||||
"nsstring 0.1.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding_rs"
|
||||
version = "0.7.1"
|
||||
version = "0.7.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
dependencies = [
|
||||
"cfg-if 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"simd 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"simd 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
|
@ -1208,7 +1208,7 @@ dependencies = [
|
|||
|
||||
[[package]]
|
||||
name = "simd"
|
||||
version = "0.2.0"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
|
||||
[[package]]
|
||||
|
|
@ -1659,7 +1659,7 @@ dependencies = [
|
|||
"checksum dwrote 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)" = "a207eb7b40e25d1d28dc679f451d321fb6954b73ceaa47986702575865469461"
|
||||
"checksum either 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "18785c1ba806c258137c937e44ada9ee7e69a37e3c72077542cd2f069d78562a"
|
||||
"checksum encoding_c 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "93ec52324ca72f423237a413ca0e1c60654c8b3d0934fcd5fd888508dfcc4ba7"
|
||||
"checksum encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f5215aabf22b83153be3ee44dfe3f940214541b2ce13d419c55e7a115c8c51a9"
|
||||
"checksum encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "98fd0f24d1fb71a4a6b9330c8ca04cbd4e7cc5d846b54ca74ff376bc7c9f798d"
|
||||
"checksum env_logger 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3ddf21e73e016298f5cb37d6ef8e8da8e39f91f9ec8b0df44b7deb16a9f8cd5b"
|
||||
"checksum error-chain 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ff511d5dc435d703f4971bc399647c9bc38e20cb41452e3b9feb4765419ed3f3"
|
||||
"checksum euclid 0.16.0 (registry+https://github.com/rust-lang/crates.io-index)" = "926c639bfdff1f3063f76bb66245f6d2b691aa20fdbaabecc38b2947a13a4eba"
|
||||
|
|
@ -1734,7 +1734,7 @@ dependencies = [
|
|||
"checksum serde 1.0.27 (registry+https://github.com/rust-lang/crates.io-index)" = "db99f3919e20faa51bb2996057f5031d8685019b5a06139b1ce761da671b8526"
|
||||
"checksum serde_derive 1.0.27 (git+https://github.com/gankro/serde?branch=deserialize_from_enums4)" = "<none>"
|
||||
"checksum serde_derive_internals 0.19.0 (git+https://github.com/gankro/serde?branch=deserialize_from_enums4)" = "<none>"
|
||||
"checksum simd 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7a94d14a2ae1f1f110937de5fb69e494372560181c7e1739a097fcc2cee37ba0"
|
||||
"checksum simd 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3dd0805c7363ab51a829a1511ad24b6ed0349feaa756c4bc2f977f9f496e6673"
|
||||
"checksum siphasher 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "2ffc669b726f2bc9a3bcff66e5e23b56ba6bf70e22a34c3d7b6d0b3450b65b84"
|
||||
"checksum slab 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "17b4fcaed89ab08ef143da37bc52adbcc04d4a69014f4c1208d6b51f0c47bc23"
|
||||
"checksum smallbitvec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "79b776f00dfe01df905fa3b2eaa1659522e99e3fc4a7b1334171622205c4bdcf"
|
||||
|
|
|
|||
|
|
@ -420,25 +420,25 @@ name = "encoding_c"
|
|||
version = "0.8.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
dependencies = [
|
||||
"encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding_glue"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"nserror 0.1.0",
|
||||
"nsstring 0.1.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding_rs"
|
||||
version = "0.7.1"
|
||||
version = "0.7.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
dependencies = [
|
||||
"cfg-if 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"simd 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
"simd 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
|
@ -1196,7 +1196,7 @@ dependencies = [
|
|||
|
||||
[[package]]
|
||||
name = "simd"
|
||||
version = "0.2.0"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
|
||||
[[package]]
|
||||
|
|
@ -1671,7 +1671,7 @@ dependencies = [
|
|||
"checksum dwrote 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)" = "a207eb7b40e25d1d28dc679f451d321fb6954b73ceaa47986702575865469461"
|
||||
"checksum either 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "18785c1ba806c258137c937e44ada9ee7e69a37e3c72077542cd2f069d78562a"
|
||||
"checksum encoding_c 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "93ec52324ca72f423237a413ca0e1c60654c8b3d0934fcd5fd888508dfcc4ba7"
|
||||
"checksum encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f5215aabf22b83153be3ee44dfe3f940214541b2ce13d419c55e7a115c8c51a9"
|
||||
"checksum encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "98fd0f24d1fb71a4a6b9330c8ca04cbd4e7cc5d846b54ca74ff376bc7c9f798d"
|
||||
"checksum env_logger 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3ddf21e73e016298f5cb37d6ef8e8da8e39f91f9ec8b0df44b7deb16a9f8cd5b"
|
||||
"checksum error-chain 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ff511d5dc435d703f4971bc399647c9bc38e20cb41452e3b9feb4765419ed3f3"
|
||||
"checksum euclid 0.16.0 (registry+https://github.com/rust-lang/crates.io-index)" = "926c639bfdff1f3063f76bb66245f6d2b691aa20fdbaabecc38b2947a13a4eba"
|
||||
|
|
@ -1746,7 +1746,7 @@ dependencies = [
|
|||
"checksum serde 1.0.27 (registry+https://github.com/rust-lang/crates.io-index)" = "db99f3919e20faa51bb2996057f5031d8685019b5a06139b1ce761da671b8526"
|
||||
"checksum serde_derive 1.0.27 (git+https://github.com/gankro/serde?branch=deserialize_from_enums4)" = "<none>"
|
||||
"checksum serde_derive_internals 0.19.0 (git+https://github.com/gankro/serde?branch=deserialize_from_enums4)" = "<none>"
|
||||
"checksum simd 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7a94d14a2ae1f1f110937de5fb69e494372560181c7e1739a097fcc2cee37ba0"
|
||||
"checksum simd 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3dd0805c7363ab51a829a1511ad24b6ed0349feaa756c4bc2f977f9f496e6673"
|
||||
"checksum siphasher 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "2ffc669b726f2bc9a3bcff66e5e23b56ba6bf70e22a34c3d7b6d0b3450b65b84"
|
||||
"checksum slab 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "17b4fcaed89ab08ef143da37bc52adbcc04d4a69014f4c1208d6b51f0c47bc23"
|
||||
"checksum smallbitvec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "79b776f00dfe01df905fa3b2eaa1659522e99e3fc4a7b1334171622205c4bdcf"
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче