gecko-dev/intl
Henri Sivonen 3edc601325 Bug 1402247 - Use encoding_rs for XPCOM string encoding conversions. r=Nika,erahm,froydnj.
Correctness improvements:

 * UTF errors are handled safely per spec instead of dangerously truncating
   strings.

 * There are fewer converter implementations.

Performance improvements:

 * The old code did exact buffer length math, which meant doing UTF math twice
   on each input string (once for length calculation and another time for
   conversion). Exact length math is more complicated when handling errors
   properly, which the old code didn't do. The new code does UTF math on the
   string content only once (when converting) but risks allocating more than
   once. There are heuristics in place to lower the probability of
   reallocation in cases where the double math avoidance isn't enough of a
   saving to absorb an allocation and memcpy.

 * Previously, in UTF-16 <-> UTF-8 conversions, an ASCII prefix was optimized
   but a single non-ASCII code point pessimized the rest of the string. The
   new code tries to get back on the fast ASCII path.

 * UTF-16 to Latin1 conversion guarantees less about handling of out-of-range
   input to eliminate an operation from the inner loop on x86/x86_64.

 * When assigning to a pre-existing string, the new code tries to reuse the
   old buffer instead of first releasing the old buffer and then allocating a
   new one.

 * When reallocating from the new code, the memcpy covers only the data that
   is part of the logical length of the old string instead of memcpying the
   whole capacity. (For old callers old excess memcpy behavior is preserved
   due to bogus callers. See bug 1472113.)

 * UTF-8 strings in XPConnect that are in the Latin1 range are passed to
   SpiderMonkey as Latin1.

New features:

 * Conversion between UTF-8 and Latin1 is added in order to enable faster
   future interop between Rust code (or otherwise UTF-8-using code) and text
   node and SpiderMonkey code that uses Latin1.

MozReview-Commit-ID: JaJuExfILM9
2018-08-14 14:43:42 +03:00
..
build Bug 1470608: Remove StringBundleOverride service. r=gandalf 2018-06-23 22:56:19 -07:00
chardet Bug 1453795 - I18N - Initialize member fields in classes/ structures. r=jfkthame 2018-06-14 22:32:49 +03:00
docs Bug 1467377 - Fix typos in Intl and L10n documentation r=gandalf 2018-06-07 08:20:49 +02:00
encoding_glue Bug 1402247 - Use encoding_rs for XPCOM string encoding conversions. r=Nika,erahm,froydnj. 2018-08-14 14:43:42 +03:00
gtest Bug 1425217 - Remove Encoding::ForName. r=hsivonen 2017-12-08 00:31:55 +09:00
hyphenation Bug 1462937: Update callers to use nsIFile::GetDirectoryEntries as a nsIDirectoryEnumerator. r=froydnj 2018-05-19 20:17:45 -07:00
icu Bug 1466471 - Part 6: Import patch to fix case conversion bug. r=Waldo 2018-07-05 07:14:07 -07:00
icu-patches Bug 1466471 - Part 6: Import patch to fix case conversion bug. r=Waldo 2018-07-05 07:14:07 -07:00
l10n Bug 1480881 - Upgrade Gecko to Fluent 0.6. r=stas 2018-08-07 00:08:29 +00:00
locale Bug 1479606 - Sync BCP47 language tags between processes, not lang tags. r=jfkthame 2018-07-31 21:42:37 +00:00
locales
lwbrk Bug 1402247 - Use encoding_rs for XPCOM string encoding conversions. r=Nika,erahm,froydnj. 2018-08-14 14:43:42 +03:00
strres Bug 1473135: Stop using ReentrantMonitor for string bundle mutexes. r=gandalf 2018-07-20 17:20:00 -07:00
tzdata Bug 1459285: Update tzdata in ICU data files to 2018e. r=Waldo 2018-05-09 01:12:01 -07:00
uconv Bug 1444329 - Remove nsIScriptableUnicodeConverter::convertFromByteArray. r=hsivonen 2018-06-04 11:25:54 +01:00
unicharutil Bug 1482002 - add nsCaseInsensitiveStringComparator default constructor r=bz 2018-08-09 14:37:30 +10:00
Encoding.h Bug 1425217 - Remove Encoding::ForName. r=hsivonen 2017-12-08 00:31:55 +09:00
icu_sources_data.py Bug 1468273 - Fix flake8/pep8 in intl/ r=froydnj 2018-06-10 14:25:47 +02:00
moz.build Bug 1438687 - Add Developer documentation for LocaleService. r=jfkthame,Pike 2018-04-26 14:18:34 -07:00
update-icu.sh Bug 1466471 - Part 6: Import patch to fix case conversion bug. r=Waldo 2018-07-05 07:14:07 -07:00
update-tzdata.sh Bug 1324240 - Also set LANGUAGE environment variable to ensure svn uses English instead of the OS locale. r=Waldo 2017-04-05 10:41:42 +02:00