When we encounter an invalid unicode escape within a regular
expression, we now pass that error on to Onigmo as if it didn't
exist in the parser (which matches the upstream parser's behavior).
We do this because there are tests that specify that you are
allowed to have invalid Unicode escapes if they are within the
context of a regular expression comment for a regular expression
in extended mode. That looks like:
/# \u /x
Note that this _only_ applies to Unicode escapes (as opposed to
applying to hex or meta/control escapes as well). Importantly it
also only applies if the regular expression is terminated. An
unterminated regular expression will still get error handling done
in the parser. That would look like:
/# \u
that would result in the same error handling we have today.
https://github.com/ruby/prism/commit/fb98034806
Add the ability to receive a callback when the parser encounters a
shebang that contains additional switches after the Ruby engine.
This is necessary because some command-line flags may be present
there that will alter the parse.
https://github.com/ruby/prism/commit/afc5000331
`Prism.lex` and `Prism.lex_file` return `ParseLexResult` instead of `Array`.
`Prism::parse_lex` and `Prism::parse_lex_file` return `ParseLexResult` instead of `ParseResult`.
This PR updates the documentation to reflect these return values.
https://github.com/ruby/prism/commit/ee331941c0
> ..., and on other POSIX systems we'll use `read`.
As `pm_string_mapped_init`'s doc comment says, it should fall back to
`read(2)`-based implementation on platforms without memory-mapped files
like WASI, but it didn't. This commit fixes it by calling `pm_string_file_init`
in the fallback case.
Also `defined(_POSIX_MAPPED_FILES)` check for `read(2)`-based path is
unnecessary, and it prevents the fallback from being executed, so this
change removes it.
https://github.com/ruby/prism/commit/b3d9064b71
* Note that we could shift the flags by 2 on serialize & deserialize
but it does not seems worth it as it does not save serialized size
in any significant amount, i.e. average was 0.799 before #2924.
* $ bundle exec rake serialized_size:topgems
Before:
Total sizes for top 100 gems:
total source size: 90207647
total serialized size: 69477115
total serialized/total source: 0.770
Stats of ratio serialized/source per file:
average: 0.844
median: 0.825
1st quartile: 0.597
3rd quartile: 1.064
min - max: 0.078 - 3.792
After:
Total sizes for top 100 gems:
total source size: 90207647
total serialized size: 66150209
total serialized/total source: 0.733
Stats of ratio serialized/source per file:
average: 0.800
median: 0.779
1st quartile: 0.568
3rd quartile: 1.007
min - max: 0.076 - 3.675
https://github.com/ruby/prism/commit/e012072f70
* $ bundle exec rake serialized_size:topgems
Before:
Total sizes for top 100 gems:
total source size: 90207647
total serialized size: 86284647
total serialized/total source: 0.957
Stats of ratio serialized/source per file:
average: 0.952
median: 0.937
1st quartile: 0.669
3rd quartile: 1.206
min - max: 0.080 - 4.065
After:
Total sizes for top 100 gems:
total source size: 90207647
total serialized size: 69477115
total serialized/total source: 0.770
Stats of ratio serialized/source per file:
average: 0.844
median: 0.825
1st quartile: 0.597
3rd quartile: 1.064
min - max: 0.078 - 3.792
https://github.com/ruby/prism/commit/cf90fe5759
This PR tweaked the documentation to correct an error encountered
when running the example code of `Prism::Dispatcher`.
This aims to make understanding the example smoother.
https://github.com/ruby/prism/commit/165a1a0e78
This commit prevents the following unary operators from being accepted
as the value prefix of a block's optional parameter:
- `+`
- `-`
- `~`
- `!`
For example, `f { |a = +b| }` will now raise a syntax error.
https://github.com/ruby/prism/commit/3024bee60c
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
Kevin Newton
Benoit Daloze
eileencodes
Alexander Momchilov
Nobuyoshi Nakada
Koichi ITO
Yuta Saito
Stan Lo