<!-- Please describe your changes on the following line: -->
fixes a panic and aligns with spec
I've also added checks around each mutable borrow of the tokenizer to see if we can catch any other panics
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix#18930 (github issue number if applicable).
<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because _____
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Source-Repo: https://github.com/servo/servo
Source-Revision: 712d75af01e02fe9689d18d7a94c89318e39f52b
--HG--
extra : subtree_source : https%3A//hg.mozilla.org/projects/converted-servo-linear
extra : subtree_revision : 66a16b12a0302d6a108f7b6345a0554794c0f8de
An explanation of the Mozilla Source Code Directory Structure and links to
project pages with documentation can be found at:
https://developer.mozilla.org/en/Mozilla_Source_Code_Directory_Structure
For information on how to build Mozilla from the source code, see:
https://developer.mozilla.org/en/docs/Build_Documentation
To have your bug fix / feature added to Mozilla, you should create a patch and
submit it to Bugzilla (https://bugzilla.mozilla.org). Instructions are at:
https://developer.mozilla.org/en/docs/Creating_a_patch
https://developer.mozilla.org/en/docs/Getting_your_patch_in_the_tree
If you have a question about developing Mozilla, and can't find the solution
on https://developer.mozilla.org, you can try asking your question in a
mozilla.* Usenet group, or on IRC at irc.mozilla.org. [The Mozilla news groups
are accessible on Google Groups, or news.mozilla.org with a NNTP reader.]
You can download nightly development builds from the Mozilla FTP server.
Keep in mind that nightly builds, which are used by Mozilla developers for
testing, may be buggy. Firefox nightlies, for example, can be found at:
https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/
- or -
https://nightly.mozilla.org/
247165dd5b