Refactor parser compile functions to reduce the dependence
on ruby functions.
This commit includes these changes
1. Refactor `gets`, `input` and `gets_` of `parser_params`
Parser needs two different data structure to get next line, function (`gets`) and input data (`input`).
However `gets_` is used for both function (`call`) and input data (`ptr`).
`call` is used for managing general callback function when `rb_ruby_parser_compile_generic` is used.
`ptr` is used for managing the current pointer on String when `parser_compile_string` is used.
This commit changes parser to used only `gets` and `input` then removes `gets_`.
2. Move parser_compile functions and `gets` functions from parse.y to ruby_parser.c
This change reduces the dependence on ruby functions from parser.
3. Change ruby_parser and ripper to take care of `VALUE input` GC mark
Move the responsibility of calling `rb_gc_mark` for `VALUE input` from parser to ruby_parser and ripper.
`input` is arbitrary data pointer from the viewpoint of parser.
4. Introduce rb_parser_compile_array function
Caller of `rb_parser_compile_generic` needs to take care about GC because ruby_parser doesn’t know
about the detail of `lex_gets` and `input`.
Introduce `rb_parser_compile_array` to reduce the complexity of ast.c.
This commit simplifies warnings for hash keys duplication and when clause duplication,
based on the discussion of https://bugs.ruby-lang.org/issues/20331.
Warnings are reported only when strings are same to ohters.
If an exception is raised the FBuffer is leaked.
For example, the following script leaks memory:
o = Object.new
def o.to_json(a) = raise
10.times do
100_000.times do
begin
JSON(o)
rescue
end
end
puts `ps -o rss= -p #{$$}`
end
Before:
31824
35696
40240
44304
47424
50944
54000
58384
62416
65296
After:
24416
24640
24640
24736
24736
24736
24736
24736
24736
24736
https://github.com/flori/json/commit/44df509dc2
Some extensions (like stringio) may need to differentiate between
chilled strings and frozen strings.
They can now use rb_str_chilled_p but must check for its presence since
the function will be removed when chilled strings are removed.
[Bug #20389]
[Feature #20205]
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
The simple names in `default_exclude_exts` do not match extension
paths under gems.
Extract each gem name from the gemspec file at the top level of each
gem directory. For example, if `ext` is `syslog-0.1.2/ext/syslog/`,
find out `syslog-0.1.2/syslog.gemspec` and take the base name `syslog`
without the suffix `.gemspec`.
[Feature #20205]
As a path toward enabling frozen string literals by default in the future,
this commit introduce "chilled strings". From a user perspective chilled
strings pretend to be frozen, but on the first attempt to mutate them,
they lose their frozen status and emit a warning rather than to raise a
`FrozenError`.
Implementation wise, `rb_compile_option_struct.frozen_string_literal` is
no longer a boolean but a tri-state of `enabled/disabled/unset`.
When code is compiled with frozen string literals neither explictly enabled
or disabled, string literals are compiled with a new `putchilledstring`
instruction. This instruction is identical to `putstring` except it marks
the String with the `STR_CHILLED (FL_USER3)` and `FL_FREEZE` flags.
Chilled strings have the `FL_FREEZE` flag as to minimize the need to check
for chilled strings across the codebase, and to improve compatibility with
C extensions.
Notes:
- `String#freeze`: clears the chilled flag.
- `String#-@`: acts as if the string was mutable.
- `String#+@`: acts as if the string was mutable.
- `String#clone`: copies the chilled flag.
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
This frees FL_USER0 on both T_MODULE and T_CLASS.
Note: prior to this, FL_SINGLETON was never set on T_MODULE,
so checking for `FL_SINGLETON` without first checking that
`FL_TYPE` was `T_CLASS` was valid. That's no longer the case.
Nobuyoshi Nakada
yui-knk
Jean byroot Boussier
Peter Zhu
Hiroshi SHIBATA
tompng
Étienne Barrié
S-H-GAMELINKS
Benoit Daloze
Jean Boussier