Граф коммитов

729 Коммитов

Автор SHA1 Сообщение Дата
Yusuke Endoh 68e1dc5172 iseq.c: Make ast_line_count return 0 when syntax error occurred
This broke coverage CI

```
  1) Failure:
TestRequire#test_load_syntax_error [/home/runner/work/actions/actions/ruby/test/ruby/test_require.rb:228]:
Exception(SyntaxError) with message matches to /unexpected/.
[SyntaxError] exception expected, not #<TypeError: no implicit conversion of false into Integer>.
```
https://github.com/ruby/actions/runs/2914743968?check_suite_focus=true
2021-06-26 00:15:16 +09:00
Yusuke Endoh 0a36cab1b5 Enable USE_ISEQ_NODE_ID by default
... which is formally called EXPERIMENTAL_ISEQ_NODE_ID.

See also ff69ef27b0.

https://bugs.ruby-lang.org/issues/17930
2021-06-18 03:35:38 +09:00
Yusuke Endoh dfba87cd62 Make it possible to get AST::Node from Thread::Backtrace::Location
RubyVM::AST.of(Thread::Backtrace::Location) returns a node that
corresponds to the location. Typically, the node is a method call, but
not always.

This change also includes iseq's dump/load support of node_ids for each
instructions.
2021-06-18 03:35:38 +09:00
Yusuke Endoh fb01411ae8 node.h: Reduce struct size to fit with Ruby object size (five VALUEs)
by merging `rb_ast_body_t#line_count` and `#script_lines`.

Fortunately `line_count == RARRAY_LEN(script_lines)` was always
satisfied. When script_lines is saved, it has an array of lines, and
when not saved, it has a Fixnum that represents the old line_count.
2021-06-18 02:34:27 +09:00
Takashi Kokubun 070caf54d2
Refactor rb_vm_insn_addr2insn calls
It's been a way too much amount of ifdefs.
2021-06-02 01:16:50 -07:00
Yusuke Endoh ff69ef27b0 compile.c: Pass node instead of nd_line(node) to ADD_INSN* functions
... then, new_insn_core extracts nd_line(node).

Also, if a macro "EXPERIMENTAL_ISEQ_NODE_ID" is defined, this changeset
keeps nd_node_id(node) for each instruction. This is intended for
TypeProf to identify what AST::Node corresponds to each instruction.

This patch is originally authored by @yui-knk for showing which column a
NoMethodError occurred.

https://github.com/ruby/ruby/compare/master...yui-knk:feature/node_id

Co-Authored-By: Yuichiro Kaneko <yui-knk@ruby-lang.org>
2021-05-07 17:02:15 +09:00
Aaron Patterson 8359821870 Use rb_fstring for "defined" strings.
We can take advantage of fstrings to de-duplicate the defined strings.
This means we don't need to keep the list of defined strings on the VM
(or register them as mark objects)
2021-03-17 10:55:37 -07:00
Aaron Patterson 17bf478de1 Store strings for `defined` in the iseqs
We can know the string used for "defined" calls at compile time, then
store the string in the instruction sequences
2021-03-17 10:55:37 -07:00
Koichi Sasada 954d6c7432 remove invalidated cc
if cc is invalidated, cc should be released from iseq.
2021-01-06 14:57:48 +09:00
Koichi Sasada e7fc353f04 enable constant cache on ractors
constant cache `IC` is accessed by non-atomic manner and there are
thread-safety issues, so Ruby 3.0 disables to use const cache on
non-main ractors.

This patch enables it by introducing `imemo_constcache` and allocates
it by every re-fill of const cache like `imemo_callcache`.
[Bug #17510]

Now `IC` only has one entry `IC::entry` and it points to
`iseq_inline_constant_cache_entry`, managed by T_IMEMO object.

`IC` is atomic data structure so `rb_mjit_before_vm_ic_update()` and
`rb_mjit_after_vm_ic_update()` is not needed.
2021-01-05 02:27:58 +09:00
Jeremy Evans 4a5c42db88 Make RubyVM::InstructionSequence.compile_file use same encoding as load
This switches the internal function from rb_parser_compile_file_path
to rb_parser_load_file, which is the same internal method that
Kernel#load uses.

Fixes [Bug #17308]
2020-11-19 07:12:50 +09:00
Koichi Sasada 084e7e31b2 remain enabled and line specified trace points
If two or more tracepoints enabled with the same target and with
different target lines, the only last line is activated.
This patch fixes this issue by remaining existing trace instructions.
[Bug #17302]
2020-11-17 07:33:38 +09:00
Nobuyoshi Nakada 799253dc46
strip trailing spaces [ci skip] 2020-10-30 12:26:59 +09:00
Koichi Sasada 07c03bc309 check isolated Proc more strictly
Isolated Proc prohibit to access outer local variables, but it was
violated by binding and so on, so they should be error.
2020-10-29 23:42:55 +09:00
Nobuyoshi Nakada 081cc4eb28
Dump FrozenCore specially 2020-10-20 23:52:19 +09:00
Koichi Sasada f6661f5085 sync RClass::ext::iv_index_tbl
iv_index_tbl manages instance variable indexes (ID -> index).
This data structure should be synchronized with other ractors
so introduce some VM locks.

This patch also introduced atomic ivar cache used by
set/getinlinecache instructions. To make updating ivar cache (IVC),
we changed iv_index_tbl data structure to manage (ID -> entry)
and an entry points serial and index. IVC points to this entry so
that cache update becomes atomically.
2020-10-17 08:18:04 +09:00
Alan Wu 1d8b689b9e Remove unused field in rb_iseq_constant_body
This was introduced in 191ce5344e
and has been unused since beae6cbf0f
2020-07-23 01:17:59 -04:00
Koichi Sasada a0f12a0258
Use ID instead of GENTRY for gvars. (#3278)
Use ID instead of GENTRY for gvars.

Global variables are compiled into GENTRY (a pointer to struct
rb_global_entry). This patch replace this GENTRY to ID and
make the code simple.

We need to search GENTRY from ID every time (st_lookup), so
additional overhead will be introduced.
However, the performance of accessing global variables is not
important now a day and this simplicity helps Ractor development.
2020-07-03 16:56:44 +09:00
Koichi Sasada 8070cb56db
fix return event and opt_invokebuiltin_delegate_leave (#3256)
If :return event is specified for a opt_invokebuiltin_delegate_leave
and leave combination, the instructions should be
  opt_invokebuiltin_delegate
  trace_return
instructions. To make it, opt_invokebuiltin_delegate_leave
instruction will be changed to opt_invokebuiltin_delegate even if
it is not an event target instruction.
2020-06-26 10:21:56 +09:00
Takashi Kokubun 737da8d383
Add another missing cast 2020-06-23 23:57:26 -07:00
Takashi Kokubun 6ecef1199e
Add missing cast 2020-06-23 23:50:31 -07:00
Takashi Kokubun 3e02cd518f
Trace :return of builtin methods
using opt_invokebuiltin_delegate_leave insn.

Since Ruby 2.7, :return of methods using builtin have not been traced properly.
2020-06-23 23:42:38 -07:00
Koichi Sasada b06d7c5521
ISeq created with callback is special, translation cannot be applied 2020-06-17 08:18:45 +09:00
卜部昌平 77293cef91 vm_ci_markable: added
CIs are created on-the-fly, which increases GC pressure.  However they
include no references to other objects, and those on-the-fly CIs tend to
be short lived.  Why not skip allocation of them.  In doing so we need
to add a flag denotes the CI object does not reside inside of objspace.
2020-06-09 09:52:46 +09:00
卜部昌平 9e41a75255 sed -i 's|ruby/impl|ruby/internal|'
To fix build failures.
2020-05-11 09:24:08 +09:00
卜部昌平 d7f4d732c1 sed -i s|ruby/3|ruby/impl|g
This shall fix compile errors.
2020-05-11 09:24:08 +09:00
Nobuyoshi Nakada 69b3e0ac59 Create succ_index_table as a part of `iseq_setup`
With compiling `CPDEBUG >= 2`, `rb_iseq_disasm` segfaults if this
table has not been created.  Also `ibf_load_iseq_each` calls
`rb_iseq_insns_info_encode_positions`.
2020-04-15 16:06:48 +09:00
Nobuyoshi Nakada f9822d1738
Shrink diassembled result string 2020-04-15 12:17:45 +09:00
Nobuyoshi Nakada e474c189da
Suppress -Wswitch warnings 2020-04-08 15:13:37 +09:00
卜部昌平 9e6e39c351
Merge pull request #2991 from shyouhei/ruby.h
Split ruby.h
2020-04-08 13:28:13 +09:00
Jeremy Evans d2c41b1bff Reduce allocations for keyword argument hashes
Previously, passing a keyword splat to a method always allocated
a hash on the caller side, and accepting arbitrary keywords in
a method allocated a separate hash on the callee side.  Passing
explicit keywords to a method that accepted a keyword splat
did not allocate a hash on the caller side, but resulted in two
hashes allocated on the callee side.

This commit makes passing a single keyword splat to a method not
allocate a hash on the caller side.  Passing multiple keyword
splats or a mix of explicit keywords and a keyword splat still
generates a hash on the caller side.  On the callee side,
if arbitrary keywords are not accepted, it does not allocate a
hash.  If arbitrary keywords are accepted, it will allocate a
hash, but this commit uses a callinfo flag to indicate whether
the caller already allocated a hash, and if so, the callee can
use the passed hash without duplicating it.  So this commit
should make it so that a maximum of a single hash is allocated
during method calls.

To set the callinfo flag appropriately, method call argument
compilation checks if only a single keyword splat is given.
If only one keyword splat is given, the VM_CALL_KW_SPLAT_MUT
callinfo flag is not set, since in that case the keyword
splat is passed directly and not mutable.  If more than one
splat is used, a new hash needs to be generated on the caller
side, and in that case the callinfo flag is set, indicating
the keyword splat is mutable by the callee.

In compile_hash, used for both hash and keyword argument
compilation, if compiling keyword arguments and only a
single keyword splat is used, pass the argument directly.

On the caller side, in vm_args.c, the callinfo flag needs to
be recognized and handled.  Because the keyword splat
argument may not be a hash, it needs to be converted to a
hash first if not.  Then, unless the callinfo flag is set,
the hash needs to be duplicated.  The temporary copy of the
callinfo flag, kw_flag, is updated if a hash was duplicated,
to prevent the need to duplicate it again.  If we are
converting to a hash or duplicating a hash, we need to update
the argument array, which can including duplicating the
positional splat array if one was passed.  CALLER_SETUP_ARG
and a couple other places needs to be modified to handle
similar issues for other types of calls.

This includes fairly comprehensive tests for different ways
keywords are handled internally, checking that you get equal
results but that keyword splats on the caller side result in
distinct objects for keyword rest parameters.

Included are benchmarks for keyword argument calls.
Brief results when compiled without optimization:

  def kw(a: 1) a end
  def kws(**kw) kw end
  h = {a: 1}

  kw(a: 1)       # about same
  kw(**h)        # 2.37x faster
  kws(a: 1)      # 1.30x faster
  kws(**h)       # 2.19x faster
  kw(a: 1, **h)  # 1.03x slower
  kw(**h, **h)   # about same
  kws(a: 1, **h) # 1.16x faster
  kws(**h, **h)  # 1.14x faster
2020-03-17 12:09:43 -07:00
Takashi Kokubun 8562bfd150
Move code to mark jit_unit's cc_entries to mjit.c 2020-03-12 22:21:32 -07:00
Takashi Kokubun da4b97a0e3
Pin and inline cme in JIT-ed method calls
```
$ benchmark-driver benchmark.yml -v --rbenv 'before --jit;after --jit' --repeat-count=12 --output=all
before --jit: ruby 2.8.0dev (2020-03-11T07:43:12Z master e89ebdcb87) +JIT [x86_64-linux]
after --jit: ruby 2.8.0dev (2020-03-11T07:54:18Z master 143776a0da) +JIT [x86_64-linux]
Calculating -------------------------------------
                                 before --jit           after --jit
Optcarrot Lan_Master.nes    73.86976729561439     77.20184819316513 fps
                            74.46997176460742     78.43493030231805
                            77.59686308754307     78.55714131655935
                            78.53693921126656     79.08984255596820
                            80.10158944910573     79.17751731838183
                            80.12254974411167     79.60853122429181
                            80.28678655204945     79.74674066871896
                            80.38690681095379     79.90624544440300
                            80.79223498756919     80.57881084206193
                            80.82857188422419     80.70677614429169
                            81.06447745878245     81.03868541295149
                            81.21620802278490     82.16354660940607
```
2020-03-11 00:59:34 -07:00
Takashi Kokubun 9511b4c8fa
Optimize away call data refs in JIT-ed method calls
According to ko1, `cd->cc != cc` was for GC.compact guard.
As we pin cc by rb_gc_mark(), we don't need the check.

```
$ benchmark-driver benchmark.yml -v --rbenv 'before --jit;after --jit' --repeat-count=12 --output=all
before --jit: ruby 2.8.0dev (2020-03-11T05:36:48Z master da6948753e) +JIT [x86_64-linux]
after --jit: ruby 2.8.0dev (2020-03-11T06:26:34Z master 36b20b8b4a) +JIT [x86_64-linux]
Calculating -------------------------------------
                                 before --jit           after --jit
Optcarrot Lan_Master.nes    74.03480698689405     71.63404803273507 fps
                            74.15085286586992     73.43923328104295
                            75.51738277744781     75.75465268365384
                            76.24922600109410     76.74071607861318
                            76.45513422802325     77.47521029238116
                            76.86617230739330     78.14759496269018
                            77.71509137131933     79.14051571125866
                            77.72839157096146     79.35884822673313
                            78.25218904561633     79.92538876408051
                            78.72521071333249     79.98075556706726
                            78.79950460165091     80.51747831497875
                            79.43884960720381     80.97973166525254
```
2020-03-10 23:29:50 -07:00
Takashi Kokubun 33b78b89ac
Eliminate unnecessary mjit_iseq_cc_entries calls
just in case.
2020-02-26 00:34:02 -08:00
Takashi Kokubun 69f377a3d6
Internalize rb_mjit_unit definition again
Fixed a TODO in b9007b6c54
2020-02-26 00:27:29 -08:00
Koichi Sasada 84d1a99a3f should be initialize jit_unit->cc_entries.
GC can invoke just after allocation of jit_unit->cc_entries so
it should be zero-cleared.
2020-02-25 13:37:52 +09:00
Koichi Sasada f744d80106 check USE_MJIT
iseq->body->jit_unit is not available if USE_MJIT==0 .
2020-02-22 11:54:19 +09:00
Koichi Sasada b9007b6c54 Introduce disposable call-cache.
This patch contains several ideas:

(1) Disposable inline method cache (IMC) for race-free inline method cache
    * Making call-cache (CC) as a RVALUE (GC target object) and allocate new
      CC on cache miss.
    * This technique allows race-free access from parallel processing
      elements like RCU.
(2) Introduce per-Class method cache (pCMC)
    * Instead of fixed-size global method cache (GMC), pCMC allows flexible
      cache size.
    * Caching CCs reduces CC allocation and allow sharing CC's fast-path
      between same call-info (CI) call-sites.
(3) Invalidate an inline method cache by invalidating corresponding method
    entries (MEs)
    * Instead of using class serials, we set "invalidated" flag for method
      entry itself to represent cache invalidation.
    * Compare with using class serials, the impact of method modification
      (add/overwrite/delete) is small.
    * Updating class serials invalidate all method caches of the class and
      sub-classes.
    * Proposed approach only invalidate the method cache of only one ME.

See [Feature #16614] for more details.
2020-02-22 09:58:59 +09:00
Koichi Sasada f2286925f0 VALUE size packed callinfo (ci).
Now, rb_call_info contains how to call the method with tuple of
(mid, orig_argc, flags, kwarg). Most of cases, kwarg == NULL and
mid+argc+flags only requires 64bits. So this patch packed
rb_call_info to VALUE (1 word) on such cases. If we can not
represent it in VALUE, then use imemo_callinfo which contains
conventional callinfo (rb_callinfo, renamed from rb_call_info).

iseq->body->ci_kw_size is removed because all of callinfo is VALUE
size (packed ci or a pointer to imemo_callinfo).

To access ci information, we need to use these functions:
vm_ci_mid(ci), _flag(ci), _argc(ci), _kwarg(ci).

struct rb_call_info_kw_arg is renamed to rb_callinfo_kwarg.

rb_funcallv_with_cc() and rb_method_basic_definition_p_with_cc()
is temporary removed because cd->ci should be marked.
2020-02-22 09:58:59 +09:00
卜部昌平 b223a78a71 this ternary operator is an undefined behaviour
Let me quote ISO/IEC 9899:2018 section 6.5.15:

> Constraints
>
> The first operand shall have scalar type.
> One of the following shall hold for the second and third operands:
> — both operands have arithmetic type;
> — both operands have the same structure or union type;
> — both operands have void type;
(snip)

Here, `*option` is a const struct rb_compile_option_struct. OTOH
`COMPILE_OPTION_DEFAULT` is a struct rb_compile_option_struct, without
const.   These two are _not_ the "same structure or union type".  Hence
the expression renders undefined behaviour.  COMPILE_OPTION_DEFAULT is
not a const because `RubyVM::InstructionSequence.compile_option=`
touches its internals on-the-fly.  There is no way to meet the
constraints quoted above.

Using ternary operator here was a mistake at the first place.  Let's
just replace it with a normal `if` statement.
2020-02-06 11:46:51 +09:00
0x005c 461db352c2 Rename RUBY_MARK_NO_PIN_UNLESS_NULL to RUBY_MARK_MOVABLE_UNLESS_NULL 2020-01-23 00:11:03 +13:00
卜部昌平 5e22f873ed decouple internal.h headers
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead.  This would significantly
speed up incremental builds.

We take the following inclusion order in this changeset:

1.  "ruby/config.h", where _GNU_SOURCE is defined (must be the very
    first thing among everything).
2.  RUBY_EXTCONF_H if any.
3.  Standard C headers, sorted alphabetically.
4.  Other system headers, maybe guarded by #ifdef
5.  Everything else, sorted alphabetically.

Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
2019-12-26 20:45:12 +09:00
Nobuyoshi Nakada db16629008
Fixed misspellings
Fixed misspellings reported at [Bug #16437], only in ruby and rubyspec.
2019-12-20 09:32:42 +09:00
Yusuke Endoh 60c53ff6ee vm_core.h (iseq_unique_id): prefer uintptr_t instead of unsigned long
It produced a warning about type cast in LLP64 (i.e., windows).
2019-12-10 17:12:21 +09:00
Yusuke Endoh 156fb72d70 vm_args.c (rb_warn_check): Use iseq_unique_id instead of its pointer
(This is the second try of 036bc1da6c6c9b0fa9b7f5968d897a9554dd770e.)

If iseq is GC'ed, the pointer of iseq may be reused, which may hide a
deprecation warning of keyword argument change.

http://ci.rvm.jp/results/trunk-test1@phosphorus-docker/2474221

```
1) Failure:
TestKeywordArguments#test_explicit_super_kwsplat [/tmp/ruby/v2/src/trunk-test1/test/ruby/test_keyword.rb:549]:
--- expected
+++ actual
@@ -1 +1 @@
-/The keyword argument is passed as the last hash parameter.* for `m'/m
+""
```

This change ad-hocly adds iseq_unique_id for each iseq, and use it
instead of iseq pointer.  This covers the case where caller is GC'ed.
Still, the case where callee is GC'ed, is not covered.

But anyway, it is very rare that iseq is GC'ed.  Even when it occurs, it
just hides some warnings.  It's no big deal.
2019-12-09 15:22:48 +09:00
Yusuke Endoh 3cdb37d9db Revert "vm_args.c (rb_warn_check): Use iseq_unique_id instead of its pointer"
This reverts commit 036bc1da6c.

This caused a failure on iseq_binary mode.
http://ci.rvm.jp/results/trunk-iseq_binary@silicon-docker/2474587

Numbering iseqs is not trivial due to dump/load.
2019-12-09 13:49:24 +09:00
Yusuke Endoh 036bc1da6c vm_args.c (rb_warn_check): Use iseq_unique_id instead of its pointer
If iseq is GC'ed, the pointer of iseq may be reused, which may hide a
deprecation warning of keyword argument change.

http://ci.rvm.jp/results/trunk-test1@phosphorus-docker/2474221

```
  1) Failure:
TestKeywordArguments#test_explicit_super_kwsplat [/tmp/ruby/v2/src/trunk-test1/test/ruby/test_keyword.rb:549]:
--- expected
+++ actual
@@ -1 +1 @@
-/The keyword argument is passed as the last hash parameter.* for `m'/m
+""
```

This change ad-hocly adds iseq_unique_id for each iseq, and use it
instead of iseq pointer.  This covers the case where caller is GC'ed.
Still, the case where callee is GC'ed, is not covered.

But anyway, it is very rare that iseq is GC'ed.  Even when it occurs, it
just hides some warnings.  It's no big deal.
2019-12-09 12:04:58 +09:00
Aaron Patterson 2c8d186c6e
Introduce an "Inline IVAR cache" struct
This commit introduces an "inline ivar cache" struct.  The reason we
need this is so compaction can differentiate from an ivar cache and a
regular inline cache.  Regular inline caches contain references to
`VALUE` and ivar caches just contain references to the ivar index.  With
this new struct we can easily update references for inline caches (but
not inline var caches as they just contain an int)
2019-12-05 13:37:02 -08:00
卜部昌平 0e8219f591 make functions static
These functions are used from within a compilation unit so we can
make them static, for better binary size.  This changeset reduces
the size of generated ruby binary from 26,590,128 bytes to
26,584,472 bytes on my macihne.
2019-11-19 12:36:19 +09:00
Samuel Williams 78e266da1d
Clarify documentation for `InstructionSequence#compile`.
We incorrectly assumed that the `file` argument should be the file name and
caused https://github.com/scoutapp/scout_apm_ruby/issues/307 because
exception backtrace did not contain correct path. This documentation
clarifies the role of the different arguments and provides extra
examples.
2019-11-19 11:40:00 +09:00
Jeremy Evans c5c05460ac Warn on access/modify of $SAFE, and remove effects of modifying $SAFE
This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.

This modifies some internal functions that took a safe level argument
to no longer take the argument.

rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.

One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd.  We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.
2019-11-18 01:00:25 +02:00
卜部昌平 c9ffe751d1 delete unused functions
Looking at the list of symbols inside of libruby-static.a, I found
hundreds of functions that are defined, but used from nowhere.

There can be reasons for each of them (e.g. some functions are
specific to some platform, some are useful when debugging, etc).
However it seems the functions deleted here exist for no reason.

This changeset reduces the size of ruby binary from 26,671,456
bytes to 26,592,864 bytes on my machine.
2019-11-14 20:35:48 +09:00
Dylan Thacker-Smith ac112f2b5d Avoid top-level search for nested constant reference from nil in defined?
Fixes [Bug #16332]

Constant access was changed to no longer allow top-level constant access
through `nil`, but `defined?` wasn't changed at the same time to stay
consistent.

Use a separate defined type to distinguish between a constant
referenced from the current lexical scope and one referenced from
another namespace.
2019-11-13 15:36:58 +09:00
Takashi Kokubun 5c168c7e7f
Support RB_BUILTIN in ISeq#to_a 2019-11-09 21:40:38 -08:00
Nobuyoshi Nakada 0ad0a8ff58
builtin.h must be included *AFTER* vm_core.h 2019-11-08 14:26:21 +09:00
Koichi Sasada 46acd0075d support builtin features with Ruby and C.
Support loading builtin features written in Ruby, which implement
with C builtin functions.
[Feature #16254]

Several features:

(1) Load .rb file at boottime with native binary.

Now, prelude.rb is loaded at boottime. However, this file is contained
into the interpreter as a text format and we need to compile it.
This patch contains a feature to load from binary format.

(2) __builtin_func() in Ruby call func() written in C.

In Ruby file, we can write `__builtin_func()` like method call.
However this is not a method call, but special syntax to call
a function `func()` written in C. C functions should be defined
in a file (same compile unit) which load this .rb file.

Functions (`func` in above example) should be defined with
  (a) 1st parameter: rb_execution_context_t *ec
  (b) rest parameters (0 to 15).
  (c) VALUE return type.
This is very similar requirements for functions used by
rb_define_method(), however `rb_execution_context_t *ec`
is new requirement.

(3) automatic C code generation from .rb files.

tool/mk_builtin_loader.rb creates a C code to load .rb files
needed by miniruby and ruby command. This script is run by
BASERUBY, so *.rb should be written in BASERUBY compatbile
syntax. This script load a .rb file and find all of __builtin_
prefix method calls, and generate a part of C code to export
functions.

tool/mk_builtin_binary.rb creates a C code which contains
binary compiled Ruby files needed by ruby command.
2019-11-08 09:09:29 +09:00
Lourens Naudé 4480d68931 Right size the compile option hash 2019-10-29 11:31:15 +09:00
Aaron Patterson bbf3de22b6
Pin labels during disassembly
We need to ensure that labels are pinned while disassembling.  If the
compactor runs during disassembly, references to these labels could go
bad, so this commit just ensures that the labels can't move until we're
done.
2019-10-28 12:15:05 -07:00
Alan Wu 89e7997622 Combine call info and cache to speed up method invocation
To perform a regular method call, the VM needs two structs,
`rb_call_info` and `rb_call_cache`. At the moment, we allocate these two
structures in separate buffers. In the worst case, the CPU needs to read
4 cache lines to complete a method call. Putting the two structures
together reduces the maximum number of cache line reads to 2.

Combining the structures also saves 8 bytes per call site as the current
layout uses separate two pointers for the call info and the call cache.
This saves about 2 MiB on Discourse.

This change improves the Optcarrot benchmark at least 3%. For more
details, see attached bugs.ruby-lang.org ticket.

Complications:
 - A new instruction attribute `comptime_sp_inc` is introduced to
 calculate SP increase at compile time without using call caches. At
 compile time, a `TS_CALLDATA` operand points to a call info struct, but
 at runtime, the same operand points to a call data struct. Instruction
 that explicitly define `sp_inc` also need to define `comptime_sp_inc`.
 - MJIT code for copying call cache becomes slightly more complicated.
 - This changes the bytecode format, which might break existing tools.

[Misc #16258]
2019-10-24 18:03:42 +09:00
卜部昌平 7e0ae1698d avoid overflow in integer multiplication
This changeset basically replaces `ruby_xmalloc(x * y)` into
`ruby_xmalloc2(x, y)`.  Some convenient functions are also
provided for instance `rb_xmalloc_mul_add(x, y, z)` which allocates
x * y + z byes.
2019-10-09 12:12:28 +09:00
Yusuke Endoh c8a18e25c1 iseq.c (rb_iseq_compile_on_base): Removed
ko1 cannot remember why he introduced the function.  And it is not used.

After it is removed, the argument "base_block" of
rb_iseq_compile_with_option is always zero.
2019-10-04 21:30:32 +09:00
Yusuke Endoh c3dd3b9553 iseq.c (rb_iseq_compile_with_option): dummy parent_iseq for the parser
The parsing of `RubyVM::InstructionSequence.compile` does not support an
outer scope currently.  So it specified NULL as parent_iseq for the
parser.  However, it resulted in the following false-positive warning.

```
RubyVM::InstructionSequence.compile(<<END)
  o = Object.new
  o #=> <compiled>:2: warning: possibly useless use of a variable in void context
END
```

This change specifies a dummy empty parent_iseq instead of NULL, which
suppresses the false positive.
2019-10-04 02:35:10 +09:00
Yusuke Endoh b43afa0a8f Make parser_params have parent_iseq instead of base_block
The parser needs to determine whether a local varaiable is defined or
not in outer scope.  For the sake, "base_block" field has kept the outer
block.

However, the whole block was actually unneeded; the parser used only
base_block->iseq.

So, this change lets parser_params have the iseq directly, instead of
the whole block.
2019-10-04 02:30:36 +09:00
Nobuyoshi Nakada 0c6f36668a
Adjusted spaces [ci skip] 2019-09-27 10:20:56 +09:00
Aaron Patterson 9b6460cacc
Remove mark array
We don't use this array anymore so we can remove it
2019-09-26 13:56:42 -07:00
Aaron Patterson 50fadefb7e
Scan the ISEQ arena for markables and mark them
This commit scans the ISEQ arena for objects that can be marked and
marks them.  This should make the mark array unnecessary.
2019-09-26 13:56:41 -07:00
Aaron Patterson 3cd8f76f7f
Introduce a secondary arena
We'll scan the secondary arena during GC mark. So, we should only
allocate "markable" instruction linked list nodes out of the secondary
arena.
2019-09-26 13:56:41 -07:00
Aaron Patterson bd017c633d
Extract allocation and free functions
Now we can allocate and free a secondary arena.
2019-09-26 13:56:41 -07:00
Jeremy Evans 3a23b71f0a Make Method/Proc#parameters handle **nil syntax
Use a [:nokey] entry in this case.
2019-08-30 12:39:31 -07:00
Nobuyoshi Nakada 896d9f967b
Constified local variable `translator` 2019-08-30 12:06:42 +09:00
Nobuyoshi Nakada e9da4f57b3
Adjust indent [ci skip] 2019-08-30 12:06:42 +09:00
卜部昌平 b8fd2e83e7 decouple compile.c usage of imemo_ifunc
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct.  This commit deletes ANYARGS from
struct vm_ifunc, but in doing so we also have to decouple the usage
of this struct in compile.c, which (I think) is an abuse of ANYARGS.
2019-08-27 15:52:26 +09:00
Alan Wu dc0e45e39b Update moved objects in original_iseq
Without doing this, enabling a TracePoint on a method could lead to use
of moved objects. This was found by running
`env RUBY_ISEQ_DUMP_DEBUG=to_binary make test-all`, which sets
orignal_iseq then runs the compaction tests and the tracepoint tests.

Please excuse the lack of tests. I was not able to figure out how to
reliably trigger a move on a specific iseq imemo to make a good
regression test.

To manually confirm the problem and this fix, you can run:
```
env RUBY_ISEQ_DUMP_DEBUG=to_binary make test-all \
  TESTOPTS="test/ruby/test_gc_compact.rb \
            test/gdbm/test_gdbm.rb \
            test/ruby/test_settracefunc.rb"
```

Or the following script:

```ruby
tp = TracePoint.new(:line) {}
1.times do # put it in a block to not keep these objects alive
  objects = 10_000.times.map { Object.new }
  objects.hash
end

1.times do
  # this allocation pattern can realistically happen in an app
  # at load time
  beek = 10_000.times.map do
    eval(<<-RUBY)
      def foo
        a + b
        1.times {
          4 + 234234
        }
        nil + 234
      end
    RUBY
    Object.new
    Object.new
  end
  beek.hash
end

tp.enable(target: self.:foo) { 234 } # allocate original iseq

GC.verify_compaction_references(toward: :empty)
GC.compact

tp.enable(target: self.:foo) { 234234 } # crash
```

[Bug #16098]
2019-08-19 12:44:38 -07:00
Benoit Daloze 39a43d9cd0 Make it as clear as possible that RubyVM is MRI-specific and only exists on MRI (#2113) [ci skip]
* Make it clear as possible that RubyVM is MRI-specific and only exists on MRI

* See [Bug #15743].
* Use "CRuby VM" instead of "Ruby VM" for clarity.

* Use YARV rather than "CRuby VM" for documenting RubyVM::InstructionSequence

* Avoid introducing a new "CRuby VM" term in documentation
2019-08-19 14:51:00 +09:00
Aaron Patterson aac4d9d6c7
Rename rb_gc_mark_no_pin -> rb_gc_mark_movable
Renaming this function.  "No pin" leaks some implementation details.  We
just want users to know that if they mark this object, the reference may
move and they'll need to update the reference accordingly.
2019-08-12 16:44:54 -04:00
git e688ab26c7 * expand tabs. 2019-08-13 01:34:36 +09:00
Aaron Patterson 76a928bac2
Unpin default value objects
We're already updating the location of default values, so we may as well
unpin them.
2019-08-12 12:34:09 -04:00
Jeremy Evans 96cec6b277 Document that RubyVM::InstructionSequence methods are implementation and version dependent
Fixes [Bug #6785]
2019-08-05 16:14:30 -07:00
Koichi Sasada c25ff7bb5d check iseq is executable 2019-07-23 08:42:20 +01:00
Lourens Naudé 90c4bd2d2b
Let memory sizes of the various IMEMO object types be reflected correctly
[Feature #15805]

Closes: https://github.com/ruby/ruby/pull/2140
2019-07-23 16:22:34 +09:00
Nobuyoshi Nakada b1aecef873
Use UNALIGNED_MEMBER_PTR
* internal.h (UNALIGNED_MEMBER_ACCESS, UNALIGNED_MEMBER_PTR):
  moved from eval_intern.h.

* compile.c iseq.c, vm.c: use UNALIGNED_MEMBER_PTR for `entries`
  in `struct iseq_catch_table`.

* vm_eval.c, vm_insnhelper.c: use UNALIGNED_MEMBER_PTR for `body`
  in `rb_method_definition_t`.
2019-05-31 16:04:16 +09:00
Kazuhiro NISHIYAMA aee36bf149
Fix Possible Control flow issues (DEADCODE)
Coverity Scan says `Execution cannot reach this statement: "poison_object(v);"`,
so do nothing when `ptr` is always 0 without address_sanitizer.
2019-05-29 13:16:05 +09:00
Urabe, Shyouhei 763989c6c5 prefix ASAN related inline functions asan_
requested by Ko1.
2019-05-23 17:24:53 +09:00
Aaron Patterson 154a67f140
Rename rb_gc_new_location to rb_gc_location
The function will return new or existing locations depending on whether
or not the object actually moved, so give it a more appropriate name.
2019-05-18 12:24:28 +03:00
Nobuyoshi Nakada 812a438145
iseq.c: removed unnecessary zero-fills 2019-04-29 11:31:18 +09:00
Lourens Naudé 99084f5401 Lazy allocate the compile data catch table array
Closes: https://github.com/ruby/ruby/pull/2119
2019-04-25 12:37:29 +09:00
git a8d4dc287f * expand tabs. 2019-04-23 10:19:47 +09:00
Aaron Patterson 98ee63693d
Symbols can move, so don't cache in static pointer
This changes the static pointers to use IDs then look up the symbols
with the ID.  Symbols can move, so we don't want to keep static
references to them.
2019-04-22 18:19:40 -07:00
tenderlove c784da1744 Update MJIT references
ISeq can move, so we need to tell MJIT where the new location is.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67624 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20 02:40:41 +00:00
tenderlove 91793b8967 Add `GC.compact` again.
🙏

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67620 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20 01:19:47 +00:00
tenderlove 744e5df715 Reverting compaction for now
For some reason symbols (or classes) are being overridden in trunk

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67598 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-17 09:41:41 +00:00
tenderlove 3c55b643ae Adding `GC.compact` and compacting GC support.
This commit adds the new method `GC.compact` and compacting GC support.
Please see this issue for caveats:

  https://bugs.ruby-lang.org/issues/15626

[Feature #15626]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-17 03:17:25 +00:00
nobu 47c82df27f Share the exception local ID table
[Fix GH-2115]

From: Lourens Naudé <lourens@bearmetal.eu>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67511 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-11 10:36:36 +00:00
svn 481481b81a * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67500 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-10 09:16:00 +00:00
kazu 25c1fd3b90 Reverting all commits from r67479 to r67496 because of CI failures
Because hard to specify commits related to r67479 only.
So please commit again.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-10 09:15:21 +00:00
nobu 5ae753dfa3 Adjusted styles
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-10 06:44:41 +00:00
tenderlove 3ef4db15e9 Adding `GC.compact` and compacting GC support.
This commit adds the new method `GC.compact` and compacting GC support.
Please see this issue for caveats:

  https://bugs.ruby-lang.org/issues/15626

[Feature #15626]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67479 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-09 20:32:04 +00:00
svn f4bff3a7f9 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67406 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-01 22:52:39 +00:00
tenderlove 1286674bb9 unpoison / poison objects while walking the heap
This fixes some ASAN errors

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67405 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-01 22:52:35 +00:00
nobu 78d6e33702 Fix styles [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66762 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-09 13:58:49 +00:00
k0kubun 4fb8d00436 iseq.c: avoid Null pointer dereference
detected by coverity scan

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66500 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-23 00:45:11 +00:00
nobu feb78840c8 iseq.c: resurrect literal hash operands
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66472 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-20 23:21:50 +00:00
ko1 d3aad15922 Return same ISeq object for one src.
* iseq.c: before this patch, RubyVM::InstructionSequence.of(src) (ISeq in
  short) returns different ISeq (wrapper) objects point to one ISeq internal
  object. This patch changes this behavior to cache created ISeq (wrapper)
  objects and return same ISeq object for an internal ISeq object.

* iseq.h (ISEQ_EXECUTABLE_P): introduced to check executable ISeq objects.

* iseq.h (ISEQ_COMPILE_DATA_ALLOC): reordr setting flag line to avoid
  ISEQ_USE_COMPILE_DATA but compiled_data == NULL case.

* vm_core.h (rb_iseq_t): introduce `rb_iseq_t::wrapper` and
  `rb_iseq_t::aux::exec`.  Move `rb_iseq_t::local_hooks` to
  `rb_iseq_t::aux::exec::local_hooks`.

* test/ruby/test_iseq.rb: add ISeq.of() tests.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66246 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-06 10:52:27 +00:00
nobu 98e65d9d92 Prefer rb_check_arity when 0 or 1 arguments
Especially over checking argc then calling rb_scan_args just to
raise an ArgumentError.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66238 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-06 07:49:24 +00:00
nobu b549d14c00 Add rb_typeddata_is_instance_of
Similar to rb_typeddata_is_kind_of, except for that inherited type
is not an instance.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66019 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-27 03:19:06 +00:00
ko1 2b44228d6f fix for ISeq.of(method).
* iseq.c (iseqw_s_of): `rb_method_iseq(method)` can return NULL.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66017 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-27 03:02:41 +00:00
svn 467c29820a * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66016 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-27 02:45:26 +00:00
ko1 33dc8a5c23 `RubyVM::ISeq.of` accept ISeq.
* iseq.c (iseqw_s_of): return given object if the given object is
  a `RubyVM::InstructionSequence`. We can specify ISeq for
  `TracePoint#enable(target:)`.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66015 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-27 02:45:25 +00:00
ko1 72e60a0437 `TracePoint#enable(target_line:)` is supported. [Feature #15289]
* vm_trace.c: `TracePoint#enable(target_line:)` is supported.
  This option enables a hook only at specified target_line.
  target_line should be combination with target and :line event.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 20:16:14 +00:00
ko1 d35bfdac74 fix for ansi-pedantic compile option.
* iseq.c (iseq_add_local_tracepoint_i): extract aggregate initialization.
  See https://travis-ci.org/ruby/ruby/jobs/459881277

* iseq.c (iseq_remove_local_tracepoint_i): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66006 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 18:48:48 +00:00
ko1 f6fea2b227 fix type error.
* iseq.c (iseq_iterate_children): should use cast to `int`.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66005 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 18:27:02 +00:00
svn 61c8247d55 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66004 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 18:16:54 +00:00
ko1 96990203b7 Support targetting TracePoint [Feature #15289]
* vm_trace.c (rb_tracepoint_enable_for_target): support targetting
  TracePoint. [Feature #15289]

  Tragetting TracePoint is only enabled on specified method, proc
  and so on, example: `tp.enable(target: code)`.

  `code` should be consisted of InstructionSeuqnece (iseq)
  (RubyVM::InstructionSeuqnece.of(code) should not return nil)
  If code is a tree of iseq, TracePoint is enabled on all of
  iseqs in a tree.

  Enabled tragetting TracePoints can not enabled again with
  and without target.

* vm_core.h (rb_iseq_t): introduce `rb_iseq_t::local_hooks`
  to store local hooks.
  `rb_iseq_t::aux::trace_events` is renamed to
  `global_trace_events` to contrast with `local_hooks`.

* vm_core.h (rb_hook_list_t): add `rb_hook_list_t::running`
  to represent how many Threads/Fibers are used this list.
  If this field is 0, nobody using this hooks and we can
  delete it.

  This is why we can remove code from cont.c.

* vm_core.h (rb_vm_t): because of above change, we can eliminate
  `rb_vm_t::trace_running` field.
  Also renamed from `rb_vm_t::event_hooks` to `global_hooks`.

* vm_core.h, vm.c (ruby_vm_event_enabled_global_flags): renamed
  from `ruby_vm_event_enabled_flags.

* vm_core.h, vm.c (ruby_vm_event_local_num): added to count
  enabled targetting TracePoints.

* vm_core.h, vm_trace.c (rb_exec_event_hooks): accepts
  hook list.

* vm_core.h (rb_vm_global_hooks): added for convinience.

* method.h (rb_method_bmethod_t): added to maintain Proc
  and `rb_hook_list_t` for bmethod (defined by define_method).

* prelude.rb (TracePoint#enable): extracet a keyword parameter
  (because it is easy than writing in C).
  It calls `TracePoint#__enable` internal method written in C.

* vm_insnhelper.c (vm_trace): check also iseq->local_hooks.

* vm.c (invoke_bmethod): check def->body.bmethod.hooks.

* vm.c (hook_before_rewind): check iseq->local_hooks
  and def->body.bmethod.hooks before rewind by exception.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66003 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 18:16:39 +00:00
svn 5164e2e43a * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65543 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-05 02:14:00 +00:00
yui-knk ce798d08de Implement `RubyVM::AST.of` [Feature #14836]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65542 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-05 02:13:45 +00:00
svn b5425ad780 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65226 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-20 10:45:55 +00:00
mame 6c9a705032 Remove tracecoverage instructions
The instructions were used only for branch coverage.
Instead, it now uses a trace framework [Feature #14104].

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65225 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-20 10:45:48 +00:00
mame bad9d2246f iseq.c (rb_iseq_trace_flag_cleared): Accept size_t for type consistency
It caused "implicit conversion loses integer precision" on clang.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65199 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-20 05:44:12 +00:00
svn 8076601976 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65196 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-20 05:33:13 +00:00
mame 47ea999b46 ext/coverage/: add the oneshot mode
This patch introduces "oneshot_lines" mode for `Coverage.start`, which
checks "whether each line was executed at least once or not", instead of
"how many times each line was executed".  A hook for each line is fired
at most once, and after it is fired, the hook flag was removed; it runs
with zero overhead.

See [Feature #15022] in detail.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65195 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-20 05:33:04 +00:00
nobu fa8b08b424 Prefer `rb_fstring_lit` over `rb_fstring_cstr`
The former states explicitly that the argument must be a literal,
and can optimize away `strlen` on all compilers.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65059 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-13 09:59:22 +00:00
ko1 daf194745b check argument type.
* iseq.c (iseqw_s_compile): check argument type (T_STRING) to
  avoid SEGV.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-04 15:22:53 +00:00
k0kubun 402001d6c7 iseq.c: prefix rb_ to non-static iseq functions
I assume we always prefix rb_ to non-static functions to avoid conflict.
These functions are not exported and safe to be renamed.

iseq.h: ditto
compile.c: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64736 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-13 13:59:25 +00:00
mame 7c9af3e196 iseq.c (rb_vm_encoded_insn_data_table_init): add a cast to build on clang
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64521 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-23 08:49:23 +00:00
mame 5b72097225 iseq.c (rb_iseq_trace_set): refactoring by using encoded_insn_data
Now it uses encoded_insn_data to identify and replace each encoded insn.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64519 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-23 08:32:31 +00:00
mame d574683c40 iseq.c: add a map from encoded insn to insn data
This enhances rb_vm_insn_addr2insn which retrieves a decoded insn number
from encoded insn.
The insn data table include not only decoded insn number, but also its
len, trace and non-trace version of encoded insn.
This table can be used to simplify trace instrumentation.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64518 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-23 08:32:30 +00:00
ko1 b85b10c11f check trace flags at loading [Bug #14702]
* iseq.c (iseq_init_trace): at ISeq loading time, we need to check
  `ruby_vm_event_enabled_flags` to turn on trace instructions.
  Seprate this checking code from `finish_iseq_build()` and make
  new function. `iseq_ibf_load()` calls this funcation after loading.

* test/ruby/test_iseq.rb: add a test for this fix.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64514 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-23 04:12:14 +00:00
mame d65f7458bc parse.y: remove coverage-related code fragments
The code fragments that initializes coverage data were scattered into
both parse.y and compile.c.  parse.y allocated a coverage data, and
compile.c initialize the data.

To remove this cross-cutting concern, this change moves the allocation
from "coverage" function of parse.y to "rb_iseq_new_top" of iseq.c.
For the sake, parse.y just counts the line number of the original source
code, and the number is passed via rb_ast_body_t.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64508 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-22 10:38:56 +00:00
nobu c5f50296b4 prototized
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64506 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-22 10:22:02 +00:00
mame 33af0429ea thread.c (rb_reset_coverages): remove coverage counters from all ISeqs
When coverage measurement is enabled, the compiler makes each iseq have
a reference to the counter array of coverage.
Even after coverage measurement is disabled, the reference is kept.
And, if coverage measurement is restarted, a coverage hook will increase
the counter.  This is completely meaningless; it brings just overhead.

To remove this meaninglessness, this change removes all the reference
when coverage measuement is stopped.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64504 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-22 05:24:50 +00:00
mame 2138f24c70 insns.def (invokesuper): remove a dummy receiever flag hack for ZSUPER
This is just a refactoring.

The receiver of "invokesuper" was a boolean to represent if it is ZSUPER
or not.  This was used in vm_search_super_method to prohibit ZSUPER call
in define_method. (It is currently prohibited because of the limitation
of the implementation.)

This change removes the hack by introducing an explicit flag,
VM_CALL_SUPER, to signal the information.  Now, the implementation of
"invokesuper" is consistent with "send" instruction.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64268 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-10 07:45:16 +00:00
k0kubun e1be448840 mjit.c: disable calling JIT-ed code
when TracePoint is enabled. We're cancelling JIT-ed code execution AFTER
each instruction, but there is no guard before the first insn of method.

To prevent spoiling performance, I don't want to modify the JIT-ed code
to fix this. So this commit replaces `mjit_enabled` check with `mjit_call_p`
check.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63734 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-23 14:11:19 +00:00
mame 75d25ede15 Make VM_INSN_INFO_TABLE_IMPL=1 work
rb_iseq_insns_info_decode_positions is used only when
VM_INSN_INFO_TABLE_IMPL=2.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63645 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-13 04:51:43 +00:00
normal 609939f8c6 rb_vm_insn_addr2insn: use st to perform addr2insn mapping
The current VM_INSTRUCTION_SIZE is 198, so the linear search
painful during a major GC phase.

I noticed rb_vm_insn_addr2insn2 showing up at the top of some
profiles while working on some malloc-related stuff, so I
decided to attack it.

Most notably, the benchmark/bm_vm3_gc.rb improves by over 40%:

  https://80x24.org/spew/20180602220554.GA9991@whir/raw

[ruby-core:87361] [Feature #14814]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63594 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-06 20:57:48 +00:00
yui-knk 548896c927 iseq.c: Fix segfault when debug is not 0
* iseq.c (get_insn_info_succinct_bitvector): If
  VM_CHECK_MODE is 0, `body->insns_info.positions` is
  freed in `rb_iseq_insns_info_encode_positions`.
  Print `position` only when VM_CHECK_MODE is set.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63468 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-19 03:43:00 +00:00
nobu a12bfb3bbd iseq body local variables
* compile.c, iseq.c: extract body and param.keyword in iseq as
  local variables.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63441 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-16 01:40:44 +00:00
nobu 610c6ba5d2 iseq.c: iseq body local variables
* iseq.c: extract body and param.keyword in iseq as local
  variables.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63404 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-12 01:24:18 +00:00
nobu 67221be89b iseq.c: refactoring rb_iseq_free
* iseq.c (rb_iseq_free): call mjit_free_iseq only if iseq->body is
  not NULL too, as the function accesses the body.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63403 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-12 01:24:16 +00:00
shyouhei 75d5cf55de RSTRING_PTR is not guaranteed to be VALUE-aligned (retry)
Don't abuse struct RString to hold arbitrary memory region.
Raw pointer should just suffice.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63368 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-09 05:42:06 +00:00
mame 18c40b16d0 tool/ruby_vm/views/_insn_name_info.erb: Auto-detect the longest insn name
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63318 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-02 00:57:50 +00:00
mame 8bfc46a117 iseq.c (rb_iseq_disasm_insn): Change the width of insn names
Currently "trace_opt_send_without_block" (28 letters) is the longest
insn.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63317 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-02 00:57:47 +00:00
nobu 0dd9f7cc5c iseq.c: consistent rb_bug messages
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63275 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-27 13:14:09 +00:00
nobu a3fe1034c4 insns.def: checktype
* insns.def (checktype): split branchiftype to checktype and
  branchif, to make branch condition negation possible.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63225 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-21 10:52:52 +00:00
tenderlove 5bafb6c596 Add write barrier calls for direct marking objects.
This commit adds write barriers for objects marked from `rb_iseq_mark`.
r62851 introduced direct marking from iseqs to:

* keyword arg default values
* catch table iseqs
* VALUEs embedded in encoded instructions

This patch adds missing write barrier calls to those references.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63147 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-13 21:36:51 +00:00
nobu bc6e61f149 compile.c: do not dump pointers
* compile.c (ibf_dump_iseq_each): do not dump succ_index_table
  pointer.  positions are dumped as integer arrays.  pointer
  values are meaningless outside the process.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-05 07:04:39 +00:00
nobu c2007e191b iseq.c: indent iseq dump
* iseq.c (rb_iseq_disasm): indent catch_table iseq entries.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-04 11:00:39 +00:00
nobu 5ff72d0569 iseq.c: strip trailing spaces in dump
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63084 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-04 04:28:47 +00:00
nobu c700bb1a2b iseq.c: constfied validate_get_insn_info [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62869 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-21 02:20:37 +00:00
nobu ce848356ae compile.c: fix ibf_load_code
* compile.c (ibf_load_iseq_each): manage iseq_size to point loaded
  objects in iseq_encoded.  now marking iseq scans iseq_encoded
  directly.

* test/ruby/test_iseq.rb (test_to_binary_with_objects): skip for
  now, but fix argument order of assert_equal.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62856 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-20 08:36:42 +00:00
tenderlove 9e26858e8c Reverting r62775, this should fix i686 builds
We need to mark default values for kwarg methods.  This also fixes
Bootsnap.  IBF iseq loading needed to mark iseqs as "having markable
objects".

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62851 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-19 18:21:54 +00:00
nobu c6e50e513c iseq.c: append local var index to name
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62831 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-19 00:32:52 +00:00
nobu a65345417d iseq.c: mark kwrest arg
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62829 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-19 00:32:51 +00:00
naruse 94c40622f5 Revert "Add direct marking on iseq operands"
This reverts commit r62706.

It causes SEGV on i686-linux (debian) and armv7l-linux-eabihf:
http://www.rubyist.net/~akr/chkbuild/debian/ruby-trunk/log/20180309T204300Z.diff.html.gz
http://rubyci.s3.amazonaws.com/scw-9d6766/ruby-trunk/log/20180309T211706Z.diff.html.gz

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62775 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-16 07:59:10 +00:00
naruse b099c5e620 Revert "Fix warning: cast from pointer to integer of different size"
This reverts commit r62709.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62773 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-16 07:59:08 +00:00
tenderlove cc7bd8e5ff Unconditionally define `TRACE_INSN_P`
`TRACE_INSN_P` doesn't need to know about encoded iseqs, it just needs
to look at decoded iseqs.  We have the decoded iseqs available, so no
reason to look at encoded ones.  This change allows us to clear
`original_iseq` from the iseq struct without any segvs (previously,
clearing `original_iseq` would cause the tests to crash).

* iseq.c (rb_iseq_trace_set): Only use decoded iseq with `TRACE_INSN_P`

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62750 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-14 23:27:10 +00:00
kazu af0d504f2d Fix warning: cast from pointer to integer of different size
```
.../ruby/iseq.c: In function ‘rb_vm_insn_null_translator’:
.../ruby/iseq.c:137:12: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
     return (int)addr;
            ^
```

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62709 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-10 00:34:24 +00:00
tenderlove 8952964976 Add direct marking on iseq operands
Directly marking iseq operands allows us to eliminate the "mark array"
stored on ISEQ objects, which will reduce the amount of memory ISEQ
objects consume.  This patch changes the iseq mark function to:

* Directly marks ISEQ operands
* Iterate over and mark child ISEQs

It also introduces two flags on the ISEQ object.  In order to mark
instruction operands, we have to disassemble the instructions and find
the instruction parameters and types.  Instructions may also be
translated to jump addresses.  Instruction sequences may get marked by
the GC *while* they're mid flight (being compiled).  The
`ISEQ_TRANSLATED` flag is used to indicate whether or not the
instructions have been translated to jump addresses so that when we
decode the instructions we know whether or not we need to go from jump
location back to original instruction or not.

Not all ISEQ objects have any markable objects embedded in their
instructions.  We can detect whether or not an ISEQ has markable objects
in the instructions at compile time.  If the instructions contain
markable objects, we set a flag `ISEQ_MARKABLE_ISEQ` on the ISEQ object.
This means that during the mark phase, we can skip decompilation if the
flag is *not* set.  In other words, we can avoid decompilation of we
know in advance there is nothing to mark.

`once` instructions have an operand that contains the result of a
one-time compilation of a regex.  Before this patch, that operand was
called an "inline cache", even though the struct was actually an "inline
storage".  This patch changes the operand to be an "inline storage" so
that we can differentiate between caches that need marking (the inline
storage) and caches that don't need marking (inline cache).

[ruby-core:84909]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62706 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-09 20:11:45 +00:00
k0kubun 33949a0320 compile.c: set catch_except_p flag
to be used for MJIT's optimization. It's not used for optimization
in this commit yet.

vm_core.h: added catch_except_p field.

iseq.c: show the flag in ISeq disasm for debugging.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-04 07:04:28 +00:00
k0kubun ed935aa5be mjit_compile.c: merge initial JIT compiler
which has been developed by Takashi Kokubun <takashikkbn@gmail> as
YARV-MJIT. Many of its bugs are fixed by wanabe <s.wanabe@gmail.com>.

This JIT compiler is designed to be a safe migration path to introduce
JIT compiler to MRI. So this commit does not include any bytecode
changes or dynamic instruction modifications, which are done in original
MJIT.

This commit even strips off some aggressive optimizations from
YARV-MJIT, and thus it's slower than YARV-MJIT too. But it's still
fairly faster than Ruby 2.5 in some benchmarks (attached below).

Note that this JIT compiler passes `make test`, `make test-all`, `make
test-spec` without JIT, and even with JIT. Not only it's perfectly safe
with JIT disabled because it does not replace VM instructions unlike
MJIT, but also with JIT enabled it stably runs Ruby applications
including Rails applications.

I'm expecting this version as just "initial" JIT compiler. I have many
optimization ideas which are skipped for initial merging, and you may
easily replace this JIT compiler with a faster one by just replacing
mjit_compile.c. `mjit_compile` interface is designed for the purpose.

common.mk: update dependencies for mjit_compile.c.

internal.h: declare `rb_vm_insn_addr2insn` for MJIT.

vm.c: exclude some definitions if `-DMJIT_HEADER` is provided to
compiler. This avoids to include some functions which take a long time
to compile, e.g. vm_exec_core. Some of the purpose is achieved in
transform_mjit_header.rb (see `IGNORED_FUNCTIONS`) but others are
manually resolved for now. Load mjit_helper.h for MJIT header.
mjit_helper.h: New. This is a file used only by JIT-ed code. I'll
refactor `mjit_call_cfunc` later.
vm_eval.c: add some #ifdef switches to skip compiling some functions
like Init_vm_eval.

win32/mkexports.rb: export thread/ec functions, which are used by MJIT.

include/ruby/defines.h: add MJIT_FUNC_EXPORTED macro alis to clarify
that a function is exported only for MJIT.

array.c: export a function used by MJIT.
bignum.c: ditto.
class.c: ditto.
compile.c: ditto.
error.c: ditto.
gc.c: ditto.
hash.c: ditto.
iseq.c: ditto.
numeric.c: ditto.
object.c: ditto.
proc.c: ditto.
re.c: ditto.
st.c: ditto.
string.c: ditto.
thread.c: ditto.
variable.c: ditto.
vm_backtrace.c: ditto.
vm_insnhelper.c: ditto.
vm_method.c: ditto.

I would like to improve maintainability of function exports, but I
believe this way is acceptable as initial merging if we clarify the
new exports are for MJIT (so that we can use them as TODO list to fix)
and add unit tests to detect unresolved symbols.
I'll add unit tests of JIT compilations in succeeding commits.

Author: Takashi Kokubun <takashikkbn@gmail.com>
Contributor: wanabe <s.wanabe@gmail.com>

Part of [Feature #14235]

---

* Known issues
  * Code generated by gcc is faster than clang. The benchmark may be worse
    in macOS. Following benchmark result is provided by gcc w/ Linux.
  * Performance is decreased when Google Chrome is running
  * JIT can work on MinGW, but it doesn't improve performance at least
    in short running benchmark.
  * Currently it doesn't perform well with Rails. We'll try to fix this
    before release.

---

* Benchmark reslts

Benchmarked with:
Intel 4.0GHz i7-4790K with 16GB memory under x86-64 Ubuntu 8 Cores

- 2.0.0-p0: Ruby 2.0.0-p0
- r62186: Ruby trunk (early 2.6.0), before MJIT changes
- JIT off: On this commit, but without `--jit` option
- JIT on: On this commit, and with `--jit` option

** Optcarrot fps

Benchmark: https://github.com/mame/optcarrot

|         |2.0.0-p0 |r62186   |JIT off  |JIT on   |
|:--------|:--------|:--------|:--------|:--------|
|fps      |37.32    |51.46    |51.31    |58.88    |
|vs 2.0.0 |1.00x    |1.38x    |1.37x    |1.58x    |

** MJIT benchmarks

Benchmark: https://github.com/benchmark-driver/mjit-benchmarks
(Original: https://github.com/vnmakarov/ruby/tree/rtl_mjit_branch/MJIT-benchmarks)

|           |2.0.0-p0 |r62186   |JIT off  |JIT on   |
|:----------|:--------|:--------|:--------|:--------|
|aread      |1.00     |1.09     |1.07     |2.19     |
|aref       |1.00     |1.13     |1.11     |2.22     |
|aset       |1.00     |1.50     |1.45     |2.64     |
|awrite     |1.00     |1.17     |1.13     |2.20     |
|call       |1.00     |1.29     |1.26     |2.02     |
|const2     |1.00     |1.10     |1.10     |2.19     |
|const      |1.00     |1.11     |1.10     |2.19     |
|fannk      |1.00     |1.04     |1.02     |1.00     |
|fib        |1.00     |1.32     |1.31     |1.84     |
|ivread     |1.00     |1.13     |1.12     |2.43     |
|ivwrite    |1.00     |1.23     |1.21     |2.40     |
|mandelbrot |1.00     |1.13     |1.16     |1.28     |
|meteor     |1.00     |2.97     |2.92     |3.17     |
|nbody      |1.00     |1.17     |1.15     |1.49     |
|nest-ntimes|1.00     |1.22     |1.20     |1.39     |
|nest-while |1.00     |1.10     |1.10     |1.37     |
|norm       |1.00     |1.18     |1.16     |1.24     |
|nsvb       |1.00     |1.16     |1.16     |1.17     |
|red-black  |1.00     |1.02     |0.99     |1.12     |
|sieve      |1.00     |1.30     |1.28     |1.62     |
|trees      |1.00     |1.14     |1.13     |1.19     |
|while      |1.00     |1.12     |1.11     |2.41     |

** Discourse's script/bench.rb

Benchmark: https://github.com/discourse/discourse/blob/v1.8.7/script/bench.rb

NOTE: Rails performance was somehow a little degraded with JIT for now.
We should fix this.
(At least I know opt_aref is performing badly in JIT and I have an idea
 to fix it. Please wait for the fix.)

*** JIT off
Your Results: (note for timings- percentile is first, duration is second in millisecs)

categories_admin:
  50: 17
  75: 18
  90: 22
  99: 29
home_admin:
  50: 21
  75: 21
  90: 27
  99: 40
topic_admin:
  50: 17
  75: 18
  90: 22
  99: 32
categories:
  50: 35
  75: 41
  90: 43
  99: 77
home:
  50: 39
  75: 46
  90: 49
  99: 95
topic:
  50: 46
  75: 52
  90: 56
  99: 101

*** JIT on
Your Results: (note for timings- percentile is first, duration is second in millisecs)

categories_admin:
  50: 19
  75: 21
  90: 25
  99: 33
home_admin:
  50: 24
  75: 26
  90: 30
  99: 35
topic_admin:
  50: 19
  75: 20
  90: 25
  99: 30
categories:
  50: 40
  75: 44
  90: 48
  99: 76
home:
  50: 42
  75: 48
  90: 51
  99: 89
topic:
  50: 49
  75: 55
  90: 58
  99: 99

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 11:22:28 +00:00
k0kubun fd44a5777f mjit.c: merge MJIT infrastructure
that allows to JIT-compile Ruby methods by generating C code and
using C compiler.  See the first comment of mjit.c to know what this
file does.

mjit.c is authored by Vladimir Makarov <vmakarov@redhat.com>.
After he invented great method JIT infrastructure for MRI as MJIT,
Lars Kanis <lars@greiz-reinsdorf.de> sent the patch to support MinGW
in MJIT. In addition to merging it, I ported pthread to Windows native
threads. Now this MJIT infrastructure can be compiled on Visual Studio.

This commit simplifies mjit.c to decrease code at initial merge. For
example, this commit does not provide multiple JIT threads support.
We can resurrect them later if we really want them, but I wanted to minimize
diff to make it easier to review this patch.

`/tmp/_mjitXXX` file is renamed to `/tmp/_ruby_mjitXXX` because non-Ruby
developers may not know the name "mjit" and the file name should make
sure it's from Ruby and not from some harmful programs.  TODO: it may be
better to store this to some temporary directory which Ruby is already using
by Tempfile, if it's not bad for performance.

mjit.h: New. It has `mjit_exec` interface similar to `vm_exec`, which is
for triggering MJIT. This drops interface for AOT compared to the original
MJIT.

Makefile.in: define macros to let MJIT know the path of MJIT header.
Probably we can refactor this to reduce the number of macros (TODO).
win32/Makefile.sub: ditto.

common.mk: compile mjit.o and mjit_compile.o. Unlike original MJIT, this
commit separates MJIT infrastructure and JIT compiler code as independent
object files. As initial patch is NOT going to have ultra-fast JIT compiler,
it's likely to replace JIT compiler, e.g. original MJIT's compiler or some
future JIT impelementations which are not public now.

inits.c: define MJIT module. This is added because `MJIT.enabled?` was
necessary for testing.
test/lib/zombie_hunter.rb: skip if `MJIT.enabled?`. Obviously this
wouldn't work with current code when JIT is enabled.
test/ruby/test_io.rb: skip this too. This would make no sense with MJIT.

ruby.c: define MJIT CLI options. As major difference from original MJIT,
"-j:l"/"--jit:llvm" are renamed to "--jit-cc" because I want to support
not only gcc/clang but also cl.exe (Visual Studio) in the future. But it
takes only "--jit-cc=gcc", "--jit-cc=clang" for now. And only long "--jit"
options are allowed since some Ruby committers preferred it at Ruby
developers Meeting on January, and some of options are renamed.
This file also triggers to initialize MJIT thread and variables.
eval.c: finalize MJIT worker thread and variables.
test/ruby/test_rubyoptions.rb: fix number of CLI options for --jit.

thread_pthread.c: change for pthread abstraction in MJIT. Prefix rb_ for
functions which are used by other files.
thread_win32.c: ditto, for Windows.  Those pthread porting is one of major
works that YARV-MJIT created, which is my fork of MJIT, in Feature 14235.
thread.c: follow rb_ prefix changes

vm.c: trigger MJIT call on VM invocation. Also trigger `mjit_mark` to avoid
SEGV by race between JIT and GC of ISeq. The improvement was provided by
wanabe <s.wanabe@gmail.com>.
In JIT compiler I created and am going to add in my next commit, I found
that having `mjit_exec` after `vm_loop_start:` is harmful because the
JIT-ed function doesn't proceed other ISeqs on RESTORE_REGS of leave insn.
Executing non-FINISH frame is unexpected for my JIT compiler and
`exception_handler` triggers executions of such ISeqs. So `mjit_exec`
here should be executed only when it directly comes from `vm_exec` call.
`RubyVM::MJIT` module and `.enabled?` method is added so that we can skip
some tests which don't expect JIT threads or compiler file descriptors.

vm_insnhelper.h: trigger MJIT on method calls during VM execution.

vm_core.h: add fields required for mjit.c. `bp` must be `cfp[6]` because
rb_control_frame_struct is likely to be casted to another struct. The
last position is the safest place to add the new field.
vm_insnhelper.c: save initial value of cfp->ep as cfp->bp. This is an
optimization which are done in both MJIT and YARV-MJIT. So this change
is added in this commit. Calculating bp from ep is a little heavy work,
so bp is kind of cache for it.

iseq.c: notify ISeq GC to MJIT. We should know which iseq in MJIT queue
is GCed to avoid SEGV.  TODO: unload some GCed units in some safe way.

gc.c: add hooks so that MJIT can wait GC, and vice versa. Simultaneous
JIT and GC executions may cause SEGV and so we should synchronize them.

cont.c: save continuation information in MJIT worker. As MJIT shouldn't
unload JIT-ed code which is being used, MJIT wants to know full list of
saved execution contexts for continuation and detect ISeqs in use.

mjit_compile.c: added empty JIT compiler so that you can reuse this commit
to build your own JIT compiler. This commit tries to compile ISeqs but
all of them are considered as not supported in this commit. So you can't
use JIT compiler in this commit yet while we added --jit option now.

Patch author: Vladimir Makarov <vmakarov@redhat.com>.

Contributors:
Takashi Kokubun <takashikkbn@gmail.com>.
wanabe <s.wanabe@gmail.com>.
Lars Kanis <lars@greiz-reinsdorf.de>.

Part of Feature 12589 and 14235.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62189 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 06:58:09 +00:00
nobu 5ce70a0831 iseq.c: internal lvar name
* iseq.c (local_var_name): name internal local variables as `?N`.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62100 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-30 02:08:30 +00:00
shyouhei e92684dae3 variable succ_index_table never actually used
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61941 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-19 03:19:58 +00:00
kazu ce7ad3a84e Use already casted variable
r61883

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61935 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-18 12:54:24 +00:00
nobu 2f67833473 iseq.c: dump as non-trace instructions
* iseq.c (iseq_data_to_ary): when OPT_CALL_THREADED_CODE is used,
  iseq_encoded is overwritten by instructions with trace and the
  original_iseq is not stored.  convert these instructions to the
  original instructions as external representation.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61890 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-17 15:28:37 +00:00
nobu 640c3dbfa1 iseq.c: flatten TS_NUM operand conditions
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-17 06:23:57 +00:00
nobu 6b5e0bd98c exclude flexible array size with old compilers
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61814 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-14 11:19:18 +00:00
mame 7ad9975fe7 iseq.h (struct iseq_catch_table_entry, iseq_compile_data_storage): Use FLEX_ARY_LEN
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61808 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-13 12:44:17 +00:00
mame 7da7716159 internal.h (FLEX_ARY_LEN): Add a macro to define a flexible array
Also, use it in iseq.c.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61804 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-13 11:45:25 +00:00
shyouhei 307547f02f flexible array member is a C99ism
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61801 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-13 10:47:32 +00:00
nobu 64be4cfb5e iseq.c: child iseq encoding
* iseq.c (iseq_inspect): preserve the encoding of child iseqs.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61767 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-11 08:34:55 +00:00
mame 0542d61e27 iseq.c: Succinct bitvector now supports 32-bit machine
Second try of succinct bitvector.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61757 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-10 05:57:15 +00:00
mame ab2c205c00 iseq.c: Add explicit casts from long to int (to pass compile on clang)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61740 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 14:11:39 +00:00
mame 83262f2489 iseq.c: Add a succinct bitvector implementation for insn_info_table
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61739 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 14:05:23 +00:00
mame 6d2de83bf0 iseq.c (iseq_data_to_ary): Avoid direct use of insns_info.positions
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61738 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 14:05:21 +00:00
mame 4928900814 Rename code_range to code_location
Because the name "code_range" is ambiguous with encoding's.
Abbreviations ("crange", and "cr") are also renamed to "loc".

The traditional "code_location" (a pair of lineno and column) is
renamed to "code_position".  Abbreviations are also renamed
(first_loc to beg_pos, and last_loc to end_pos).

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61721 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 08:45:35 +00:00
ko1 8a83cd100b use `getblockparamproxy` to pass blocks.
* compile.c (setup_args): use `getblockparamproxy` (`rb_block_param_proxy`)
  to represent a block parameter passing.

* vm_args.c (vm_caller_setup_arg_block): check `rb_block_param_proxy`
  instead of using `VM_CALL_ARGS_BLOCKARG_BLOCKPARAM` call flag.

* vm_core.h (VM_CALL_ARGS_BLOCKARG_BLOCKPARAM): removed.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61694 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-08 16:06:33 +00:00
mame 52057336b7 parse.y: Make consistent with the terms about code ranges and locations
"loc" was ambiguous; it might refer both a location and a code range.
This change uses "loc" for a location, and "crange" or "cr" for a code
range.

A location (abbr. loc) is a point in a program and consists of line
number and column number.  A code range (abbr. crange and cr) is a range
within a program and consists of a pair of locations which is the first
and the last.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61690 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-08 12:56:58 +00:00
mame 38c62063c0 node.h: remove NODE_PRELUDE
NODE_PRELUDE contains a `BEGIN` node, a main node, and compile_option.
This node is assumed that it must be located immediately under the root
NODE_SCOPE, but this strange assumption is not so good, IMO.

This change removes the assumtion; it integrates the former two nodes by
block_append, and moves compile_option into rb_ast_body_t.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61610 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-05 08:59:23 +00:00
mame 92b81dc597 make rb_iseq_new* accept rb_ast_body_t instead of NODE*
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61609 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-05 08:59:22 +00:00
mame 503b858cef node.h: define rb_ast_body_t and restructure rb_ast_t
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61608 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-05 08:59:20 +00:00
mame e1428e5c92 Refactor ISEQ_TYPE_DEFINED_GUARD out
This commit removes ISEQ_TYPE_DEFINED_GUARD because it is no longer
needed.  And this introduces ISEQ_TYPE_PLAIN which means that the iseq
does nothing special but just wrap an expression.  Currently, this is
used for once execution: `/foo#{ bar }baz/o`.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61601 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-05 00:49:41 +00:00
mame e743a35314 iseq.c: Refactor out rb_iseq_new_ifunc from rb_iseq_new_with_opt
It is too error-prone to pass IMEMO_IFUNC object as NODE*.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61592 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-04 07:07:49 +00:00
mame 1e4be7a852 iseq.c (rb_iseq_new_with_opt): handle the case where node is imemo_func
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61591 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-04 04:44:16 +00:00
nobu 9edca48cff insns_info.inc.tmpl: share large tablese
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61590 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-04 01:46:01 +00:00
mame 691b05e83c vm_core.h: make the algorithm of get_insn_info selectable
Currently, VM_INSN_INFO_TABLE_IMPL == 0 means linear search, and
VM_INSN_INFO_TABLE_IMPL == 1 means binary search.  I plan to add
succinct bitvector algorithm later.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61537 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-01 13:18:55 +00:00
mame 295838e6eb iseq.h: Extract position array from iseq_insn_info_entry
This makes TracePoint a bit fast by reducing cache misses of
`get_insn_info_binary_search`.

Also, I plan to use succinct bitvector algorithm for `get_insn_info`
instead of binary search.  This change will make it easy.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61536 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-01 12:51:21 +00:00
mame e1fce44ca6 iseq.c: fix build error when VM_CHECK_MODE is enabled
Follow up of r61534.  Sorry.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61535 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-01 09:30:06 +00:00
mame fd950df795 vm_core.h: refactoring of insns_info
This factors rb_iseq_constant_body#insns_info and #insns_info_size to
struct iseq_insn_info.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61534 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-01 09:16:27 +00:00
ko1 d7c6776f7c Disasm output contains code range.
* iseq.c (iseq_inspect): show also code range information.
  Note that `iseq_inspect` is used only for header of disasm.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61463 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-25 04:40:42 +00:00
ko1 cbac40b3e5 Remove "trace_instruction" compile option.
* iseq.h (rb_compile_option_struct): trace instruction is removed so that
  remove the trace_instruction compile option.
  Don't show warning (just ignore) for Ruby 2.5.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61450 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-24 17:07:53 +00:00
ko1 3044138bd7 RubyVM::InstructionSequence#trace_points.
* iseq.c (iseqw_trace_points): add `RubyVM::InstructionSequence#trace_points`
  method for tools which want to manipulate ISeq (and traces).

* test/ruby/test_iseq.rb: add a test for this method.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61427 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-23 14:46:59 +00:00
ko1 eb8c2773cb RubyVM::InstructionSequence#each_child.
* iseq.c (iseqw_each_child): add RubyVM::InstructionSequence#each_child
  method for tools which want to manipulate ISeq.

* test/ruby/test_iseq.rb: add a test for this method.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61425 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-23 12:48:24 +00:00
k0kubun 1942e92547 iseq.c: show first_lineno on iseq inspect
My motivation was to improve the output of `ruby --dump=insns xxx.rb`.
When one file has many iseqs, it's hard to find the one I want to read.

So I wanted `iseq_disasm` to show first_lineno. I unified the behavior
of `iseqw_disasm` for consistency.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61423 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-23 09:10:34 +00:00
kazu a747834a69 iseq.c: Remove unused macro in r61395
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61422 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-23 01:53:48 +00:00
nobu 2a723132e9 iseq.c: local variable names
* iseq.c (rb_insn_operand_intern): simplified the condition for
  local variable names, assume TS_LINDEX and TS_NUM combinations
  are only for local variables.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61421 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-23 00:55:29 +00:00
nobu 6e0074d0ff iseq.c (rb_insn_operand_intern): needs parenthesies
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61403 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-22 01:12:02 +00:00
nobu b4dc51ce10 iseq.c: defined insn operand
* iseq.c (rb_insn_operand_intern): improve operands of defined
  instruction.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61395 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-22 00:29:38 +00:00
yui-knk 69ae0f0f42 iseq.c (rb_iseq_code_range): Remove needless `;`
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61384 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21 07:09:03 +00:00
ko1 e9a184ed63 add experimental API.
* iseq.c (rb_iseq_code_range): added to access iseq's code range.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21 06:40:28 +00:00