servo: Merge #19096 - Avoid decoding XHR type="json" responses as UTF-16BE/LE (from hsivonen:no-utf16-json); r=avadacatavra

https://infra.spec.whatwg.org/#parse-json-from-bytes says to use
"UTF-8 decode" rather than "decode", so UTF-16BE/LE BOM should
not be honored.

CC @SimonSapin

<!-- Please describe your changes on the following line: -->

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #__ (github issue number if applicable).

<!-- Either: -->
- [x] There are tests for these changes OR
- [ ] These changes do not require tests because _____

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

Source-Repo: https://github.com/servo/servo
Source-Revision: e64083ccc624cd508d4001c5456ac7e04aa049a6

--HG--
extra : subtree_source : https%3A//hg.mozilla.org/projects/converted-servo-linear
extra : subtree_revision : edbeae61eba86ab74d3d5e2c8aa764dc818ae242

servo/components/script/dom/xmlhttprequest.rs

@@ -1164,8 +1164,8 @@ impl XMLHttpRequest {
             return NullValue();
         }
         // Step 4
-        fn decode_to_utf16(bytes: &[u8], encoding: &'static Encoding) -> Vec<u16> {
-            let mut decoder = encoding.new_decoder();
+        fn decode_to_utf16_with_bom_removal(bytes: &[u8], encoding: &'static Encoding) -> Vec<u16> {
+            let mut decoder = encoding.new_decoder_with_bom_removal();
             let capacity = decoder.max_utf16_buffer_length(bytes.len()).expect("Overflow");
             let mut utf16 = Vec::with_capacity(capacity);
             let extra = unsafe {
@@ -1179,7 +1179,12 @@ impl XMLHttpRequest {
             }
             utf16
         }
-        let json_text = decode_to_utf16(&bytes, UTF_8);
+        // https://xhr.spec.whatwg.org/#json-response refers to
+        // https://infra.spec.whatwg.org/#parse-json-from-bytes which refers to
+        // https://encoding.spec.whatwg.org/#utf-8-decode which means
+        // that the encoding is always UTF-8 and the UTF-8 BOM is removed,
+        // if present, but UTF-16BE/LE BOM must not be honored.
+        let json_text = decode_to_utf16_with_bom_removal(&bytes, UTF_8);
         // Step 5
         rooted!(in(cx) let mut rval = UndefinedValue());
         unsafe {