This patch removes the `VALUE flags` member from the `rb_ast_t` structure making `rb_ast_t` no longer an IMEMO object.
## Background
We are trying to make the Ruby parser generated from parse.y a universal parser that can be used by other implementations such as mruby.
To achieve this, it is necessary to exclude VALUE and IMEMO from parse.y, AST, and NODE.
## Summary (file by file)
- `rubyparser.h`
- Remove the `VALUE flags` member from `rb_ast_t`
- `ruby_parser.c` and `internal/ruby_parser.h`
- Use TypedData_Make_Struct VALUE which wraps `rb_ast_t` `in ast_alloc()` so that GC can manage it
- You can retrieve `rb_ast_t` from the VALUE by `rb_ruby_ast_data_get()`
- Change the return type of `rb_parser_compile_XXXX()` functions from `rb_ast_t *` to `VALUE`
- rb_ruby_ast_new() which internally `calls ast_alloc()` is to create VALUE vast outside ruby_parser.c
- `iseq.c` and `vm_core.h`
- Amend the first parameter of `rb_iseq_new_XXXX()` functions from `rb_ast_body_t *` to `VALUE`
- This keeps the VALUE of AST on the machine stack to prevent being removed by GC
- `ast.c`
- Almost all change is replacement `rb_ast_t *ast` with `VALUE vast` (sorry for the big diff)
- Fix `node_memsize()`
- Now it includes `rb_ast_local_table_link`, `tokens` and script_lines
- `compile.c`, `load.c`, `node.c`, `parse.y`, `proc.c`, `ruby.c`, `template/prelude.c.tmpl`, `vm.c` and `vm_eval.c`
- Follow-up due to the above changes
- `imemo.{c|h}`
- If an object with `imemo_ast` appears, considers it a bug
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Types like `Type::CString` really only assert that at one point the object had
its class field equal to `String`. Once a singleton class is created for any
strings, the type makes no assertion about any class field anymore, and becomes
the same as `Type::TString`.
Previously, the `--yjit-verify-ctx` option wasn't allowing objects of these
kind that have have singleton classes to pass verification even though the code
generators handle it just fine.
Found through `ruby/spec`.
https://bugs.ruby-lang.org/issues/20228 started freeing `stk_base` to
avoid a memory leak. But `stk_base` is sometimes stack allocated (using
`xalloca`), so the free only works if the regex stack has grown enough
to hit `stack_double` (which uses `xmalloc` and `xrealloc`).
To reproduce the problem on master and 3.3.1:
```ruby
Regexp.timeout = 0.001
/^(a*)x$/ =~ "a" * 1000000 + "x"'
```
Some details about this potential fix:
`stk_base == stk_alloc` on
[init](dde99215f2/regexec.c (L1153)),
so if `stk_base != stk_alloc` we can be sure we called
[`stack_double`](dde99215f2/regexec.c (L1210))
and it's safe to free. It's also safe to free if we've
[saved](dde99215f2/regexec.c (L1187-L1189))
the stack to `msa->stack_p`, since we do the `stk_base != stk_alloc`
check before saving.
This matches the check we do inside
[`stack_double`](dde99215f2/regexec.c (L1221))
* Revert "Revert "YJIT: Optimize local variables when EP == BP" (#10584)"
This reverts commit c878344195.
* YJIT: Take care of GC references in ISEQ invariants
Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>
---------
Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>
We would previously cause a stack overflow if we parsed a file that
was too deeply nested when we were calling inspect. Instead, we now
use a queue of commands to do it linearly so we don't.
https://github.com/ruby/prism/commit/0f21f5bfe1
Some helpers, like Rails console's `app`, requires memoization of the
helper's ivars. To support it IRB needs to memoize helper method instances
as well.
https://github.com/ruby/irb/commit/169a9a2c30
Add a specialized codegen function for `Class#superclass`.
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Takashi Kokubun (k0kubun) <takashikkbn@gmail.com>
Co-authored-by: Randy Stauner <randy.stauner@shopify.com>
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
yui-knk
Nobuyoshi Nakada
dependabot[bot]
HASUMI Hitoshi
Alan Wu
Peter Zhu
Kevin Newton
Josh Nichols
Takashi Kokubun
tomoya ishida
Daniel Colson
David Rodriguez
卜部昌平
Xi Ruoyao
Eugene Kenny
Aaron Patterson
Stan Lo
Kevin Menard
Hiroshi SHIBATA
Stan Lo