This eliminates the subnode on RationalNode and replaces it with two
integer fields, which represent the ratio for the rational. It also
reduces those two integers if they both fit into 32 bits.
Importantly, this PR does not implement bignum reduction. That's something
I'd like to consider for the future, but it's simple enough for now to
leave them unreduced, which makes it more useful than it used to be.
https://github.com/ruby/prism/commit/86e06c7068
Fixes https://github.com/ruby/prism/pull/2617.
There was an issue with the lexer as follows.
The following are valid regexp options:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/io").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :EOF]
```
The following are invalid regexp options. Unnecessary the `IDENTIFIER` token is appearing:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/az").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :IDENTIFIER, :EOF]
```
As a behavior of Ruby, when given `A` to `Z` and `a` to `z`, they act as invalid regexp options. e.g.,
```console
$ ruby -e '/regexp/az'
-e:1: unknown regexp options - az
/regexp/az
-e: compile error (SyntaxError)
```
Thus, it should probably not be construed as `IDENTIFIER` token.
Therefore, `pm_byte_table` has been adapted to accept those invalid regexp option values.
Whether it is a valid regexp option or not is checked by `pm_regular_expression_flags_create`.
For invalid regexp options, `PM_ERR_REGEXP_UNKNOWN_OPTIONS` is added to diagnostics.
https://github.com/ruby/prism/commit/d2a6096fcf
- Add `x` prefix to malloc, calloc, realloc, and free
(eg: malloc -> xmalloc)
- By default, they are replaced with stdlib's functions at build
- You can use custom functions by defining `PRISM_CUSTOM_ALLOCATOR` macro
https://github.com/ruby/prism/commit/7a878af619
Check that multibyte characters are valid using pm_strpbrk. We need
to add a couple of codepaths to ensure all encodings are covered.
Importantly this doesn't check regular expressions, because
apparently you're allowed to have invalid multibyte characters
inside regular expression comment groups/extended mode.
https://github.com/ruby/prism/commit/2857d3e1b5
This PR implements proper file parsing error handling. Previously
`file_options` would call `pm_string_mapped_init` which would print an
error from `perror`. However this wouldn't raise a proper Ruby error so
it was just a string output. I've done the following:
- Raise an error from `rb_syserr_fail` with the filepath in
`file_options`.
- No longer return `Qnil` if `file_options` returns false (because now
it will raise)
- Update `file_options` to return `static void` instead of `static
bool`.
- Update `file_options` and `profile_file` to check the type so when
passing `nil` we see a `TypeError`.
- Delete `perror` from `pm_string_mapped_init`
- Update `FFI` backend to raise appropriate errors when calling
`pm_string_mapped_init`.
- Add tests for `dump_file`, `lex_file`, `parse_file`,
`parse_file_comments`, `parse_lex_file`, and `parse_file_success?`
when a file doesn't exist and for `nil`.
- Updates the `bin/parse` script to no longer raise it's own
`ArgumentError` now that we raise a proper error.
Fixes: ruby/prism#2207https://github.com/ruby/prism/commit/b2f7494ff5
Kevin Newton
Herwin
Koichi ITO
Nobuyoshi Nakada
tompng
cui fliter
HASUMI Hitoshi
eileencodes