Граф коммитов

715 Коммитов

Автор SHA1 Сообщение Дата
John Hawthorn c18edc5b5d Avoid underflow of rb_yjit_live_iseq_count
This value is only incremented when rb_iseq_translate_threaded_code is
called, which doesn't happen for iseqs which result in a syntax error.

This is easy to hit by running a debug build with RUBY_FREE_AT_EXIT=1,
but any build and options could underflow this value by running enough
evals.
2023-12-21 20:43:01 -08:00
Nobuyoshi Nakada 2f595c744e
Adjust styles [ci skip] 2023-12-17 00:21:00 +09:00
HParker 55326a915f Introduce --parser runtime flag
Introduce runtime flag for specifying the parser,

```
ruby --parser=prism
```

also update the description:

```
$ ruby --parser=prism --version
ruby 3.3.0dev (2023-12-08T04:47:14Z add-parser-runtime.. 0616384c9f) +PRISM [x86_64-darwin23]
```

[Bug #20044]
2023-12-15 13:42:19 -05:00
eileencodes 049a9bd62f [PRISM] Fix `compile_prism` when src is a file
`compile_prism` can take a source and file (and other arguments) or a
file as the source. `compile` checks if the source is a file and if it
is converts it. `compile_prism` is now doing the same thing.

On the Ruby side `compile` handles a file
[here](https://github.com/ruby/ruby/blob/master/iseq.c#L1159-L1162).

Before:

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(26,21)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] name@0
0000 putstring                              "Prism"                   (  25)[Li]
0002 setlocal                               name@0, 0
0005 putself                                                          (  26)[Li]
0006 putobject                              "hello, "
0008 getlocal                               name@0, 0
0011 dup
0012 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0014 anytostring
0015 concatstrings                          2
0017 send                                   <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, nil
0020 leave
hello, Prism

"********* PRISM *************"
./test.rb:13:in `compile_prism': wrong argument type File (expected String) (TypeError)
	from ./test.rb:13:in `<main>'
make: *** [run] Error 1
```

After:

```
"********* Ruby *************"
== disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(26,21)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] name@0
0000 putstring                              "Prism"                   (  25)[Li]
0002 setlocal                               name@0, 0
0005 putself                                                          (  26)[Li]
0006 putobject                              "hello, "
0008 getlocal                               name@0, 0
0011 dup
0012 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0014 anytostring
0015 concatstrings                          2
0017 send                                   <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, nil
0020 leave

"********* PRISM *************"
== disasm: #<ISeq:<compiled>@test_code.rb:24 (24,0)-(25,21)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] name@0
0000 putstring                              "Prism"                   (  24)[Li]
0002 setlocal                               name@0, 0
0005 putself                                                          (  25)[Li]
0006 putobject                              "hello, "
0008 getlocal                               name@0, 0
0011 dup
0012 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0014 anytostring
0015 concatstrings                          2
0017 send                                   <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, nil
0020 leave                                                            (  24)
```

Fixes ruby/prism#1609
2023-12-15 10:27:44 -05:00
Adam Hess 6816e8efcf Free everything at shutdown
when the RUBY_FREE_ON_SHUTDOWN environment variable is set, manually free memory at shutdown.

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Peter Zhu <peter@peterzhu.ca>
2023-12-07 15:52:35 -05:00
Nobuyoshi Nakada 582c202c23
Adjust styles [ci skip] 2023-12-06 15:14:59 +09:00
HParker b8b319dd1a Revert "allow enabling Prism via flag or env var"
This reverts commit 9b76c7fc89.
2023-12-06 10:21:12 +09:00
Nobuyoshi Nakada c146da50bd
Adjust styles [ci skip] 2023-12-06 09:43:10 +09:00
HParker 9b76c7fc89 allow enabling Prism via flag or env var
Enable Prism using either --prism

    ruby --prism test.rb

or via env var

    RUBY_PRISM=1 ruby test.rb
2023-12-05 12:17:14 -05:00
Peter Zhu d1691617d6 Pin instruction storage
The operands in each instruction needs to be pinned because if
auto-compaction runs in iseq_set_sequence, then the objects could exist
on the generated_iseq buffer, which would not be reference updated which
can lead to T_MOVED (and subsequently T_NONE) objects on the iseq.
2023-12-02 09:06:03 -05:00
Kevin Newton 323bec6295 RubyVM::InstructionSequence.compile_file_prism
* Provide a new API compile_file_prism which mirrors compile_file
but uses prism to parse/compile.
* Provide the ability to run test-all with RUBY_ISEQ_DUMP_DEBUG set
to "prism". If it is, we'll use the new compile_file_prism API to
load iseqs during the test run.
2023-11-20 12:45:29 -08:00
Nobuyoshi Nakada ad72d96fcc
Escape and quote non-local variable names 2023-11-15 17:52:40 +09:00
Kevin Newton ec86b2eb39
[prism] Update to use new options APIs 2023-11-03 10:13:50 -04:00
Jemma Issroff 8f71a5c530 [PRISM] Move scope_node itself to CRuby, create prism_compile.h 2023-10-25 18:18:35 -03:00
Jemma Issroff 0abf2d86b9 [PRISM] Move pm_scope_node_init to prism_compile.c
pm_scope_node_init is only used for CRuby, so should not live in the
ruby/prism repo. We will merge the changes here first so they're
not breaking, and will then remove from ruby/prism
2023-10-25 18:18:35 -03:00
Jemma Issroff 10c5063704 Address PR comments 2023-10-18 17:16:11 -07:00
Jemma Issroff 1701b79b54 Put line change back 2023-10-18 17:16:11 -07:00
Jemma Issroff 252b0fc715 Temporarily removed location code on scope nodes 2023-10-18 17:16:11 -07:00
Jemma Issroff ba3a99acaf Remove pm_compile_context_t, move the context onto ScopeNode
We changed ScopeNodes to point to their parent (previous) ScopeNodes.
Accordingly, we can remove pm_compile_context_t, and store all
necessary context in ScopeNodes, allowing us to access locals from
outer scopes.
2023-10-18 17:16:11 -07:00
Alan Wu d2b0c9da2e
YJIT: Add a live ISeq counter
It's an estimator for application size and could be used as a
compilation heuristic later.

Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
2023-10-18 23:46:35 +00:00
Matt Valentine-House b6390a21eb [ci skip] More docs for InstructionSequence.compile
This commit documents that you can also pass a `File` object to
`RubyVM::InstructionSequence.compile`, instead of a string, and this
will behave in a similar way to
`RubyVM::InstructionSequence.compile_file`

e.g.

```
❯ ./ruby -e "puts RubyVM::InstructionSequence.compile(File.open('test.rb')).disasm"
== disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(2,21)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] name@0
0000 putstring                              "Ruby"                    (   1)[Li]
0002 setlocal_WC_0                          name@0
0004 putself                                                          (   2)[Li]
0005 putobject                              "Hello, "
0007 getlocal_WC_0                          name@0
0009 dup
0010 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0012 anytostring
0013 concatstrings                          2
0015 opt_send_without_block                 <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE>
0017 leave

~/git/ruby master* ≡ ⇡
❯ ./ruby -e "puts RubyVM::InstructionSequence.compile(File.open('test.rb').read).disasm"
== disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(2,21)>
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] name@0
0000 putstring                              "Ruby"                    (   1)[Li]
0002 setlocal_WC_0                          name@0
0004 putself                                                          (   2)[Li]
0005 putobject                              "Hello, "
0007 getlocal_WC_0                          name@0
0009 dup
0010 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0012 anytostring
0013 concatstrings                          2
0015 opt_send_without_block                 <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE>
0017 leave
```

This is explicitly allowed by this code path in
`rb_iseq_compile_with_option` so we should document it.

```
    if (RB_TYPE_P(src, T_FILE)) {
        parse = rb_parser_compile_file_path;
    }
    else {
        parse = rb_parser_compile_string_path;
        StringValue(src);
    }
```
2023-10-02 12:31:34 +01:00
Kevin Newton 3d0a46796b Rename YARP symbols to prism 2023-09-27 13:57:38 -04:00
Adam Hess ef59175a68 fix iseq kwargs table and original_iseq leaks
[bug #19903]

Co-authored-by: Peter Zhu <peter@peterzhu.ca>
2023-09-26 11:12:21 -04:00
Nobuyoshi Nakada 4c040fe850
Copy compile options from AST directly without intermediate Hash 2023-09-01 14:06:42 +09:00
Nobuyoshi Nakada 1fbc8cdf06
Copy `rb_compile_option_t` only if needed
Use `COMPILE_OPTION_DEFAULT` if nothing to change.
2023-09-01 13:39:36 +09:00
Nobuyoshi Nakada 61f1657f68
Use macro argument not the variable directly 2023-09-01 13:36:15 +09:00
Kevin Newton 49dff732e8
Update YARP APIs to handle uint8_t 2023-08-30 14:41:39 -04:00
Jemma Issroff 80dc570a45
Compile more YARP node types (#8322)
* Add several more node simple types to YARP's compiler:

Nodes include: DefinedNode, EmbeddedStatementsNode,
LocalVariableReadNode, LocalVariableWriteNode, MultiWriteNode,
OptionalParameterNode, SplatNode, YieldNode

* Add AssocSplatNode, RangeNode

* Add RangeNode, other helpers for future nodes

* Add ArrayNode, HashNode, static literal helpers

* Add branch conditionals

* Add IfNode, UnlessNode

* Add ScopeNode

* NEW_ISEQ and NEW_CHILD_ISEQ implemented for YARP

* Add nodes that depend on ScopeNode

* Addressed PR comments
2023-08-29 13:13:15 -07:00
Jemma Issroff 535045ab3b
[YARP] Compile basic types (#8311)
* Add a compile_context arg to yp_compile_node

The compile_context will allow us to pass around the parser, and
the constants and lookup table (to be used in future commits).

* Compile yp_program_node_t and yp_statements_node_t

Add the compilation for program and statements node so that we can
successfully compile an empty program with YARP.

* Helper functions for parsing numbers, strings, and symbols

* Compile basic numeric / boolean node types in YARP

* Compile StringNode and SymbolNodes in YARP

* Compile several basic node types in YARP

* Added error return for missing node
2023-08-29 09:27:00 -07:00
Jemma Issroff 3b815ed7da
Add yarp/yarp_compiler.c (#8042)
* Add yarp/yarp_compiler.c as stencil for compiling YARP

This commit adds yarp/yarp_compiler.c, and changes the sync script
to ensure that yarp/yarp_compiler.c will not get overwritten

* [Misc #119772] Create and expose RubyVM::InstructionSequence.compile_yarp

This commit creates the stencil for a compile_yarp function, which
we will continue to fill out. It allows us to check the output
of compiled YARP code against compiled code without using YARP.
2023-08-28 13:55:58 -07:00
Nobuyoshi Nakada 6aa16f9ec1 Move SCRIPT_LINES__ away from parse.y 2023-08-25 18:23:05 +09:00
Koichi Sasada d68c01fd31 support `rescue` event for TracePoint
fix [Feature #19572]
2023-08-01 22:46:17 +09:00
Ruby c330037c1a `cc->cme` should not be marked.
cc is callcache.

cc->klass (klass) should not be marked because if the klass is
free'ed, the cc->klass will be cleared by `vm_cc_invalidate()`.

cc->cme (cme) should not be marked because if cc is invalidated
when cme is free'ed.
- klass marks cme if klass uses cme.
- caller classe's ccs->cme marks cc->cme.
- if cc is invalidated (klass doesn't refer the cc),
  cc is invalidated by `vm_cc_invalidate()` and cc->cme is
  not be accessed.
- On the multi-Ractors, cme will be collected with global GC
  so that it is safe if GC is not interleaving while accessing
  cc and cme.

fix [Bug #19436]

```ruby
10_000.times{|i|
  # p i if (i%1_000) == 0

  str = "x" * 1_000_000
  def str.foo = nil
  eval "def call#{i}(s) = s.foo"
  send "call#{i}", str
}
```

Without this patch:

```
real    1m5.639s
user    0m6.637s
sys     0m58.292s
```

and with this patch:

```
real    0m2.045s
user    0m1.627s
sys     0m0.164s
```
2023-07-28 10:51:11 +09:00
Takashi Kokubun 38be9a9b72
Clean up OPT_STACK_CACHING (#8132) 2023-07-27 17:27:05 -07:00
Peter Zhu 58386814a7 Don't check for null pointer in calls to free
According to the C99 specification section 7.20.3.2 paragraph 2:

> If ptr is a null pointer, no action occurs.

So we do not need to check that the pointer is a null pointer.
2023-06-30 09:13:31 -04:00
yui-knk 19c62b400d Replace parser & node compile_option from Hash to bit field
This commit reduces dependency to CRuby object.
2023-06-17 16:41:08 +09:00
yui-knk b481b673d7 [Feature #19719] Universal Parser
Introduce Universal Parser mode for the parser.
This commit includes these changes:

* Introduce `UNIVERSAL_PARSER` macro. All of CRuby related functions
  are passed via `struct rb_parser_config_struct` when this macro is enabled.
* Add CI task with 'cppflags=-DUNIVERSAL_PARSER' for ubuntu.
2023-06-12 18:23:48 +09:00
yui-knk 98637d421d Move `ruby_node_name` to node.c and rename prefix of the function 2023-05-23 18:05:35 +09:00
eileencodes ce99e50ede Move `catch_except_p` to `compile_data`
The `catch_except_p` flag is used for communicating between parent and
child iseq's that a throw instruction was emitted. So for example if a
child iseq has a throw in it and the parent wants to catch the throw, we
use this flag to communicate to the parent iseq that a throw instruction
was emitted.

This flag is only useful at compile time, it only impacts the
compilation process so it seems to be fine to move it from the iseq body
to the compile_data struct.

Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
2023-04-11 10:47:58 -07:00
Takashi Kokubun df1b007fbd Remove unused VM_CALL_BLOCKISEQ flag 2023-04-01 10:22:47 -07:00
Koichi Sasada c9fd81b860 `vm_call_single_noarg_inline_builtin`
If the iseq only contains `opt_invokebuiltin_delegate_leave` insn and
the builtin-function (bf) is inline-able, the caller doesn't need to
build a method frame.

`vm_call_single_noarg_inline_builtin` is fast path for such cases.
2023-03-23 14:03:12 +09:00
Takashi Kokubun 9ad19069f9 Remove obsoleted functions in rjit.c 2023-03-07 23:59:50 -08:00
Takashi Kokubun 23ec248e48 s/mjit/rjit/ 2023-03-06 23:44:01 -08:00
Takashi Kokubun 2e875549a9 s/MJIT/RJIT/ 2023-03-06 23:44:01 -08:00
Takashi Kokubun 233ddfac54 Stop exporting symbols for MJIT 2023-03-06 21:59:23 -08:00
Takashi Kokubun 32e6f15beb Store MJIT blocks on each ISEQ 2023-03-05 23:28:59 -08:00
Takashi Kokubun 63d96ccbcd Invalidate everything on GC.compact 2023-03-05 23:28:59 -08:00
John Bampton 2f7270c681
Fix spelling (#7389) 2023-02-27 09:56:06 -08:00
Matt Valentine-House 72aba64fff Merge gc.h and internal/gc.h
[Feature #19425]
2023-02-09 10:32:29 -05:00
Peter Zhu 861d70e383 Rename iseq_mark_and_update to iseq_mark_and_move
The new name is more consistent.
2023-02-08 12:43:25 -05:00