Граф коммитов

750 Коммитов

Автор SHA1 Сообщение Дата
Koichi Sasada 07c03bc309 check isolated Proc more strictly
Isolated Proc prohibit to access outer local variables, but it was
violated by binding and so on, so they should be error.
2020-10-29 23:42:55 +09:00
Jeremy Evans dfb3605bbe
Add Thread.ignore_deadlock accessor
Setting this to true disables the deadlock detector.  It should
only be used in cases where the deadlock could be broken via some
external means, such as via a signal.

Now that $SAFE is no longer used, replace the safe_level_ VM flag
with ignore_deadlock for storing the setting.

Fixes [Bug #13768]
2020-10-28 15:27:00 -07:00
Koichi Sasada 99310e3eb5 Some global variables can be accessed from ractors
Some global variables should be used from non-main Ractors.
[Bug #17268]

```ruby
     # ractor-local (derived from created ractor): debug
     '$DEBUG' => $DEBUG,
     '$-d' => $-d,

     # ractor-local (derived from created ractor): verbose
     '$VERBOSE' => $VERBOSE,
     '$-w' => $-w,
     '$-W' => $-W,
     '$-v' => $-v,

     # process-local (readonly): other commandline parameters
     '$-p' => $-p,
     '$-l' => $-l,
     '$-a' => $-a,

     # process-local (readonly): getpid
     '$$'  => $$,

     # thread local: process result
     '$?'  => $?,

     # scope local: match
     '$~'  => $~.inspect,
     '$&'  => $&,
     '$`'  => $`,
     '$\''  => $',
     '$+'  => $+,
     '$1'  => $1,

     # scope local: last line
     '$_' => $_,

     # scope local: last backtrace
     '$@' => $@,
     '$!' => $!,

     # ractor local: stdin, out, err
     '$stdin'  => $stdin.inspect,
     '$stdout' => $stdout.inspect,
     '$stderr' => $stderr.inspect,
```
2020-10-20 15:38:54 +09:00
Koichi Sasada 319afed20f Use language TLS specifier if it is possible.
To access TLS, it is faster to use language TLS specifier instead
of using pthread_get/setspecific functions.

Original proposal is: Use native thread locals. #3665
2020-10-20 01:05:06 +09:00
Koichi Sasada f6661f5085 sync RClass::ext::iv_index_tbl
iv_index_tbl manages instance variable indexes (ID -> index).
This data structure should be synchronized with other ractors
so introduce some VM locks.

This patch also introduced atomic ivar cache used by
set/getinlinecache instructions. To make updating ivar cache (IVC),
we changed iv_index_tbl data structure to manage (ID -> entry)
and an entry points serial and index. IVC points to this entry so
that cache update becomes atomically.
2020-10-17 08:18:04 +09:00
Koichi Sasada ae693fff74 fix releasing timing.
(1) recorded_lock_rec > current_lock_rec should not be occurred
    on rb_ec_vm_lock_rec_release().
(2) should be release VM lock at EXEC_TAG(), not POP_TAG().
(3) some refactoring.
2020-10-14 16:36:55 +09:00
Koichi Sasada c3ba3fa8d0 support exception when lock_rec > 0
If a ractor getting a VM lock (monitor) raises an exception,
unlock can be skipped. To release VM lock correctly on exception
(or other jumps with JUMP_TAG), EC_POP_TAG() releases VM lock.
2020-10-14 14:02:06 +09:00
Samuel Williams 70f08f1eed Make `Thread#join` non-blocking. 2020-09-21 11:48:44 +12:00
Koichi Sasada 74ddac1c82 relax dependency
vm_sync.h does not need to include vm_core.h and ractor_pub.h.
2020-09-15 00:04:59 +09:00
Koichi Sasada f7ccb8dd88 restart Ractor.select on intterupt
signal can interrupt Ractor.select, but if there is no exception,
Ractor.select should restart automatically.
2020-09-15 00:04:59 +09:00
Koichi Sasada 79df14c04b Introduce Ractor mechanism for parallel execution
This commit introduces Ractor mechanism to run Ruby program in
parallel. See doc/ractor.md for more details about Ractor.
See ticket [Feature #17100] to see the implementation details
and discussions.

[Feature #17100]

This commit does not complete the implementation. You can find
many bugs on using Ractor. Also the specification will be changed
so that this feature is experimental. You will see a warning when
you make the first Ractor with `Ractor.new`.

I hope this feature can help programmers from thread-safety issues.
2020-09-03 21:11:06 +09:00
Jeremy Evans a0273d67d0 Avoid a use after free in VM assertion
If the thread for the current EC has been killed, don't check
the VM ptr for the EC (which gets it via the thread), as that will
have already been freed.

Fixes [Bug #16907]
2020-08-21 14:52:30 -07:00
Alan Wu 1d8b689b9e Remove unused field in rb_iseq_constant_body
This was introduced in 191ce5344e
and has been unused since beae6cbf0f
2020-07-23 01:17:59 -04:00
卜部昌平 215c6fa3d0 RUBY_CONST_ASSERT: use STATIC_ASSERT instead
Static assertions shall be done using STATIC_ASSERT these days.
2020-07-10 12:23:41 +09:00
Takashi Kokubun 7561db8c00
Introduce Primitive.attr! to annotate 'inline' (#3242)
[Feature #15589]
2020-06-20 17:13:03 -07:00
Samuel Williams 0e3b0fcdba
Thread scheduler for light weight concurrency. 2020-05-14 22:10:55 +12:00
卜部昌平 233c2018f1 drop varargs.h support
This header file is simply out of date (for decades since at least
1989).  It's the 21st century.  Just stop using it.
2020-05-11 14:56:51 +09:00
卜部昌平 9e41a75255 sed -i 's|ruby/impl|ruby/internal|'
To fix build failures.
2020-05-11 09:24:08 +09:00
卜部昌平 d7f4d732c1 sed -i s|ruby/3|ruby/impl|g
This shall fix compile errors.
2020-05-11 09:24:08 +09:00
卜部昌平 4ff3f20540 add #include guard hack
According to MSVC manual (*1), cl.exe can skip including a header file
when that:

- contains #pragma once, or
- starts with #ifndef, or
- starts with #if ! defined.

GCC has a similar trick (*2), but it acts more stricter (e. g. there
must be _no tokens_ outside of #ifndef...#endif).

Sun C lacked #pragma once for a looong time.  Oracle Developer Studio
12.5 finally implemented it, but we cannot assume such recent version.

This changeset modifies header files so that each of them include
strictly one #ifndef...#endif.  I believe this is the most portable way
to trigger compiler optimizations. [Bug #16770]

*1: https://docs.microsoft.com/en-us/cpp/preprocessor/once
*2: https://gcc.gnu.org/onlinedocs/cppinternals/Guard-Macros.html
2020-04-13 16:06:00 +09:00
卜部昌平 9e6e39c351
Merge pull request #2991 from shyouhei/ruby.h
Split ruby.h
2020-04-08 13:28:13 +09:00
Nobuyoshi Nakada b6833ff50d
Get rid of redefinition of `rb_execution_context_t`
Regardless of the order to include "vm_core.h" and "builtin.h".
2020-03-19 13:25:53 +09:00
Yusuke Endoh 0256e4f0f5 thread_pthread.c: allocate sigaltstack before pthread_create
A new (not-initialized-yet) pthread attempts to allocate sigaltstack by
using xmalloc.  It may cause GC, but because the thread is not
initialized yet, ruby_native_thread_p() returns false, which leads to
"[FATAL] failed to allocate memory" and exit.

In fact, we can observe the error message in the log of OpenBSD CI:
https://rubyci.org/logs/rubyci.s3.amazonaws.com/openbsd-current/ruby-master/log/20200306T083005Z.log.html.gz

This changeset allocates sigaltstack before pthread is created.
2020-03-06 21:41:34 +09:00
Koichi Sasada b9007b6c54 Introduce disposable call-cache.
This patch contains several ideas:

(1) Disposable inline method cache (IMC) for race-free inline method cache
    * Making call-cache (CC) as a RVALUE (GC target object) and allocate new
      CC on cache miss.
    * This technique allows race-free access from parallel processing
      elements like RCU.
(2) Introduce per-Class method cache (pCMC)
    * Instead of fixed-size global method cache (GMC), pCMC allows flexible
      cache size.
    * Caching CCs reduces CC allocation and allow sharing CC's fast-path
      between same call-info (CI) call-sites.
(3) Invalidate an inline method cache by invalidating corresponding method
    entries (MEs)
    * Instead of using class serials, we set "invalidated" flag for method
      entry itself to represent cache invalidation.
    * Compare with using class serials, the impact of method modification
      (add/overwrite/delete) is small.
    * Updating class serials invalidate all method caches of the class and
      sub-classes.
    * Proposed approach only invalidate the method cache of only one ME.

See [Feature #16614] for more details.
2020-02-22 09:58:59 +09:00
Koichi Sasada f2286925f0 VALUE size packed callinfo (ci).
Now, rb_call_info contains how to call the method with tuple of
(mid, orig_argc, flags, kwarg). Most of cases, kwarg == NULL and
mid+argc+flags only requires 64bits. So this patch packed
rb_call_info to VALUE (1 word) on such cases. If we can not
represent it in VALUE, then use imemo_callinfo which contains
conventional callinfo (rb_callinfo, renamed from rb_call_info).

iseq->body->ci_kw_size is removed because all of callinfo is VALUE
size (packed ci or a pointer to imemo_callinfo).

To access ci information, we need to use these functions:
vm_ci_mid(ci), _flag(ci), _argc(ci), _kwarg(ci).

struct rb_call_info_kw_arg is renamed to rb_callinfo_kwarg.

rb_funcallv_with_cc() and rb_method_basic_definition_p_with_cc()
is temporary removed because cd->ci should be marked.
2020-02-22 09:58:59 +09:00
卜部昌平 34f8e75f93 rb_vm_t::postponed_job_index shall be rb_atomic_t
Pointer to this field is passed to ATOMIC_CAS.  We have to use
rb_atomic_t for that purpose.
2020-02-06 11:46:51 +09:00
Lourens Naudé 40c57ad4a1 Let execution context local storage be an ID table 2020-01-11 14:40:36 +13:00
Jeremy Evans beae6cbf0f Fully separate positional arguments and keyword arguments
This removes the warnings added in 2.7, and changes the behavior
so that a final positional hash is not treated as keywords or
vice-versa.

To handle the arg_setup_block splat case correctly with keyword
arguments, we need to check if we are taking a keyword hash.
That case didn't have a test, but it affects real-world code,
so add a test for it.

This removes rb_empty_keyword_given_p() and related code, as
that is not needed in Ruby 3.  The empty keyword case is the
same as the no keyword case in Ruby 3.

This changes rb_scan_args to implement keyword argument
separation for C functions when the : character is used.
For backwards compatibility, it returns a duped hash.
This is a bad idea for performance, but not duping the hash
breaks at least Enumerator::ArithmeticSequence#inspect.

Instead of having RB_PASS_CALLED_KEYWORDS be a number,
simplify the code by just making it be rb_keyword_given_p().
2020-01-02 18:40:45 -08:00
卜部昌平 5e22f873ed decouple internal.h headers
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead.  This would significantly
speed up incremental builds.

We take the following inclusion order in this changeset:

1.  "ruby/config.h", where _GNU_SOURCE is defined (must be the very
    first thing among everything).
2.  RUBY_EXTCONF_H if any.
3.  Standard C headers, sorted alphabetically.
4.  Other system headers, maybe guarded by #ifdef
5.  Everything else, sorted alphabetically.

Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
2019-12-26 20:45:12 +09:00
Alan Wu 85a337f986 Kernel#lambda: return forwarded block as non-lambda proc
Before this commit, Kernel#lambda can't tell the difference between a
directly passed literal block and one passed with an ampersand.

A block passed with an ampersand is semantically speaking already a
non-lambda proc. When Kernel#lambda receives a non-lambda proc, it
should simply return it.

Implementation wise, when the VM calls a method with a literal block, it
places the code for the block on the calling control frame and passes a
pointer (block handler) to the callee. Before this commit, the VM
forwards block arguments by simply forwarding the block handler, which
leaves the slot for block code unused when a control frame forwards its
block argument. I use the vacant space to indicate that a frame has
forwarded its block argument and inspect that in Kernel#lambda to detect
forwarded blocks.

This is a very ad-hoc solution and relies *heavily* on the way block
passing works in the VM. However, it's the most self-contained solution
I have.

[Bug #15620]
2019-12-21 09:08:52 -05:00
卜部昌平 fa7fa5c86b delete rb_vm_call()
Nobody uses it any longer.
2019-12-18 14:14:51 +09:00
Koichi Sasada 9d3ffcfbfc disable assertion.
This assertion is not needed because we found the bug.
ba11a74745.
2019-12-17 20:23:15 +09:00
Nobuyoshi Nakada d2f04d332f Kernel#abort without arguments should print error info
[Bug #16424]
2019-12-16 14:55:59 +09:00
Yusuke Endoh 60c53ff6ee vm_core.h (iseq_unique_id): prefer uintptr_t instead of unsigned long
It produced a warning about type cast in LLP64 (i.e., windows).
2019-12-10 17:12:21 +09:00
Yusuke Endoh 156fb72d70 vm_args.c (rb_warn_check): Use iseq_unique_id instead of its pointer
(This is the second try of 036bc1da6c6c9b0fa9b7f5968d897a9554dd770e.)

If iseq is GC'ed, the pointer of iseq may be reused, which may hide a
deprecation warning of keyword argument change.

http://ci.rvm.jp/results/trunk-test1@phosphorus-docker/2474221

```
1) Failure:
TestKeywordArguments#test_explicit_super_kwsplat [/tmp/ruby/v2/src/trunk-test1/test/ruby/test_keyword.rb:549]:
--- expected
+++ actual
@@ -1 +1 @@
-/The keyword argument is passed as the last hash parameter.* for `m'/m
+""
```

This change ad-hocly adds iseq_unique_id for each iseq, and use it
instead of iseq pointer.  This covers the case where caller is GC'ed.
Still, the case where callee is GC'ed, is not covered.

But anyway, it is very rare that iseq is GC'ed.  Even when it occurs, it
just hides some warnings.  It's no big deal.
2019-12-09 15:22:48 +09:00
Yusuke Endoh 3cdb37d9db Revert "vm_args.c (rb_warn_check): Use iseq_unique_id instead of its pointer"
This reverts commit 036bc1da6c.

This caused a failure on iseq_binary mode.
http://ci.rvm.jp/results/trunk-iseq_binary@silicon-docker/2474587

Numbering iseqs is not trivial due to dump/load.
2019-12-09 13:49:24 +09:00
Yusuke Endoh 036bc1da6c vm_args.c (rb_warn_check): Use iseq_unique_id instead of its pointer
If iseq is GC'ed, the pointer of iseq may be reused, which may hide a
deprecation warning of keyword argument change.

http://ci.rvm.jp/results/trunk-test1@phosphorus-docker/2474221

```
  1) Failure:
TestKeywordArguments#test_explicit_super_kwsplat [/tmp/ruby/v2/src/trunk-test1/test/ruby/test_keyword.rb:549]:
--- expected
+++ actual
@@ -1 +1 @@
-/The keyword argument is passed as the last hash parameter.* for `m'/m
+""
```

This change ad-hocly adds iseq_unique_id for each iseq, and use it
instead of iseq pointer.  This covers the case where caller is GC'ed.
Still, the case where callee is GC'ed, is not covered.

But anyway, it is very rare that iseq is GC'ed.  Even when it occurs, it
just hides some warnings.  It's no big deal.
2019-12-09 12:04:58 +09:00
Aaron Patterson 2c8d186c6e
Introduce an "Inline IVAR cache" struct
This commit introduces an "inline ivar cache" struct.  The reason we
need this is so compaction can differentiate from an ivar cache and a
regular inline cache.  Regular inline caches contain references to
`VALUE` and ivar caches just contain references to the ivar index.  With
this new struct we can easily update references for inline caches (but
not inline var caches as they just contain an int)
2019-12-05 13:37:02 -08:00
卜部昌平 0e8219f591 make functions static
These functions are used from within a compilation unit so we can
make them static, for better binary size.  This changeset reduces
the size of generated ruby binary from 26,590,128 bytes to
26,584,472 bytes on my macihne.
2019-11-19 12:36:19 +09:00
Jeremy Evans c5c05460ac Warn on access/modify of $SAFE, and remove effects of modifying $SAFE
This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.

This modifies some internal functions that took a safe level argument
to no longer take the argument.

rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.

One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd.  We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.
2019-11-18 01:00:25 +02:00
卜部昌平 c9ffe751d1 delete unused functions
Looking at the list of symbols inside of libruby-static.a, I found
hundreds of functions that are defined, but used from nowhere.

There can be reasons for each of them (e.g. some functions are
specific to some platform, some are useful when debugging, etc).
However it seems the functions deleted here exist for no reason.

This changeset reduces the size of ruby binary from 26,671,456
bytes to 26,592,864 bytes on my machine.
2019-11-14 20:35:48 +09:00
Koichi Sasada 3141642380 __builtin_inline!
Add an experimental `__builtin_inline!(c_expression)` special intrinsic
which run a C code snippet.
In `c_expression`, you can access the following variables:
  * ec (rb_execution_context_t *)
  * self (const VALUE)
  * local variables (const VALUE)
Not that you can read these variables, but you can not write them.
You need to return from this expression and return value will be a
result of __builtin_inline!().

Examples:
  `def foo(x) __builtin_inline!('return rb_p(x);'); end` calls `p(x)`.
  `def double(x) __builtin_inline!('return INT2NUM(NUM2INT(x) * 2);')`
  returns x*2.
2019-11-11 16:47:50 +09:00
Koichi Sasada 46acd0075d support builtin features with Ruby and C.
Support loading builtin features written in Ruby, which implement
with C builtin functions.
[Feature #16254]

Several features:

(1) Load .rb file at boottime with native binary.

Now, prelude.rb is loaded at boottime. However, this file is contained
into the interpreter as a text format and we need to compile it.
This patch contains a feature to load from binary format.

(2) __builtin_func() in Ruby call func() written in C.

In Ruby file, we can write `__builtin_func()` like method call.
However this is not a method call, but special syntax to call
a function `func()` written in C. C functions should be defined
in a file (same compile unit) which load this .rb file.

Functions (`func` in above example) should be defined with
  (a) 1st parameter: rb_execution_context_t *ec
  (b) rest parameters (0 to 15).
  (c) VALUE return type.
This is very similar requirements for functions used by
rb_define_method(), however `rb_execution_context_t *ec`
is new requirement.

(3) automatic C code generation from .rb files.

tool/mk_builtin_loader.rb creates a C code to load .rb files
needed by miniruby and ruby command. This script is run by
BASERUBY, so *.rb should be written in BASERUBY compatbile
syntax. This script load a .rb file and find all of __builtin_
prefix method calls, and generate a part of C code to export
functions.

tool/mk_builtin_binary.rb creates a C code which contains
binary compiled Ruby files needed by ruby command.
2019-11-08 09:09:29 +09:00
Koichi Sasada 88135845f1 enable assertion for debug.
http://ci.rvm.jp/results/trunk-vm-asserts@silicon-docker/2340856
2019-10-25 17:58:54 +09:00
卜部昌平 356e203a3a more on struct rb_call_data
Replacing adjacent struct rb_call_info and struct rb_call_cache
into a struct rb_call_data.
2019-10-25 12:24:22 +09:00
Alan Wu 89e7997622 Combine call info and cache to speed up method invocation
To perform a regular method call, the VM needs two structs,
`rb_call_info` and `rb_call_cache`. At the moment, we allocate these two
structures in separate buffers. In the worst case, the CPU needs to read
4 cache lines to complete a method call. Putting the two structures
together reduces the maximum number of cache line reads to 2.

Combining the structures also saves 8 bytes per call site as the current
layout uses separate two pointers for the call info and the call cache.
This saves about 2 MiB on Discourse.

This change improves the Optcarrot benchmark at least 3%. For more
details, see attached bugs.ruby-lang.org ticket.

Complications:
 - A new instruction attribute `comptime_sp_inc` is introduced to
 calculate SP increase at compile time without using call caches. At
 compile time, a `TS_CALLDATA` operand points to a call info struct, but
 at runtime, the same operand points to a call data struct. Instruction
 that explicitly define `sp_inc` also need to define `comptime_sp_inc`.
 - MJIT code for copying call cache becomes slightly more complicated.
 - This changes the bytecode format, which might break existing tools.

[Misc #16258]
2019-10-24 18:03:42 +09:00
Nobuyoshi Nakada 29e6782f5d
Share ruby_sighandler_t definition 2019-10-09 23:39:58 +09:00
Yusuke Endoh 891cbd66a4 signal.c: save the original sighandlers for fatal signals
On Android, a signal handler that is not SIG_DFL is set by default for
SIGSEGV.  Ruby's install_sighandler inserts Ruby's handler only when the
signal has no handler, so it does not insert Ruby's SEGV report handler,
which caused some test failures.

This changeset forces to install Ruby's handler for some fatal signals
(sigbus, sigsegv, and sigill).  They keep the original handlers, and
call them when the interpreter receives the signals.
2019-10-09 23:22:15 +09:00
Yusuke Endoh dd477df411 error.c (rb_bug_for_fatal_signal): renamed from rb_bug_context
Just refactoring.

The name "rb_bug_context" is completely unclear for me.
(Can you see that "context" means "machine register context"?)
The context is available only when a fatal signal (sigbus, sigsegv, or
sigill) is received; in fact, the function is used only for fatal
signals.  So, I think the name should be changed.
2019-10-09 23:02:22 +09:00
Yusuke Endoh c8a18e25c1 iseq.c (rb_iseq_compile_on_base): Removed
ko1 cannot remember why he introduced the function.  And it is not used.

After it is removed, the argument "base_block" of
rb_iseq_compile_with_option is always zero.
2019-10-04 21:30:32 +09:00
卜部昌平 eb92159d72 Revert https://github.com/ruby/ruby/pull/2486
This reverts commits: 10d6a3aca7 8ba48c1b85 fba8627dc1 dd883de5ba
6c6a25feca 167e6b48f1 7cb96d41a5 3207979278 595b3c4fdd 1521f7cf89
c11c5e69ac cf33608203 3632a812c0 f56506be0d 86427a3219 .

The reason for the revert is that we observe ABA problem around
inline method cache.  When a cache misshits, we search for a
method entry.  And if the entry is identical to what was cached
before, we reuse the cache.  But the commits we are reverting here
introduced situations where a method entry is freed, then the
identical memory region is used for another method entry.  An
inline method cache cannot detect that ABA.

Here is a code that reproduce such situation:

```ruby
require 'prime'

class << Integer
  alias org_sqrt sqrt
  def sqrt(n)
    raise
  end

  GC.stress = true
  Prime.each(7*37){} rescue nil # <- Here we populate CC
  class << Object.new; end

  # These adjacent remove-then-alias maneuver
  # frees a method entry, then immediately
  # reuses it for another.
  remove_method :sqrt
  alias sqrt org_sqrt
end

Prime.each(7*37).to_a # <- SEGV
```
2019-10-03 12:45:24 +09:00
卜部昌平 cf33608203 refactor constify most of rb_method_definition_t
Most (if not all) of the fields of rb_method_definition_t are never
meant to be modified once after they are stored.  Marking them const
makes it possible for compilers to warn on unintended modifications.
2019-09-30 10:26:38 +09:00
Jeremy Evans 6fdd701472 Remove VM_NO_KEYWORDS, replace with RB_NO_KEYWORDS
VM_NO_KEYWORDS was introduced first in vm_core.h, but it is best
to only use a single definition for this.
2019-09-29 16:41:00 -07:00
Jeremy Evans 6b52959ef7 Fix keyword argument separation issues in Thread.new 2019-09-26 08:01:53 -07:00
Jeremy Evans 3b302ea8c9 Add Module#ruby2_keywords for passing keywords through regular argument splats
This approach uses a flag bit on the final hash object in the regular splat,
as opposed to a previous approach that used a VM frame flag.  The hash flag
approach is less invasive, and handles some cases that the VM frame flag
approach does not, such as saving the argument splat array and splatting it
later:

  ruby2_keywords def foo(*args)
    @args = args
    bar
  end
  def bar
    baz(*@args)
  end
  def baz(*args, **kw)
    [args, kw]
  end
  foo(a:1)    #=> [[], {a: 1}]
  foo({a: 1}, **{}) #=> [[{a: 1}], {}]

  foo({a: 1}) #=> 2.7: [[], {a: 1}] # and warning
  foo({a: 1}) #=> 3.0: [[{a: 1}], {}]

It doesn't handle some cases that the VM frame flag handles, such as when
the final hash object is replaced using Hash#merge, but those cases are
probably less common and are unlikely to properly support keyword
argument separation.

Use ruby2_keywords to handle argument delegation in the delegate library.
2019-09-25 12:33:52 -07:00
Fangrui Song e006b992c2 typedef rb_jmpbuf_t to void *[5] if __builtin_setjmp is used
The built-in version operates on a buffer of 5 words, much smaller than
the size of jmp_buf defined in libc.
Note, powerpc requires 5 words, while arm and x86_64 just require 3.
2019-09-21 13:24:58 +09:00
Nobuyoshi Nakada 5f5aca1b5f
Moved ruby_node_name declaration to node.h 2019-09-13 17:20:11 +09:00
Jeremy Evans 37a2c660aa Convert keyword argument to required positional hash argument for Class#new, Method#call, UnboundMethod#bind_call
Also add keyword argument separation warnings for Class#new and Method#call.

To allow for keyword argument to required positional hash converstion in
cfuncs, add a vm frame flag indicating the cfunc was called with an empty
keyword hash (which was removed before calling the cfunc).  The cfunc can
check this frame flag and add back an empty hash if it is passing its
arguments to another Ruby method.  Add rb_empty_keyword_given_p function
for checking if called with an empty keyword hash, and
rb_add_empty_keyword for adding back an empty hash to argv.

All of this empty keyword argument support is only for 2.7.  It will be
removed in 3.0 as Ruby 3 will not convert empty keyword arguments to
required positional hash arguments.  Comment all of the relevent code
to make it obvious this is expected to be removed.

Add rb_funcallv_kw as an public C-API function, just like rb_funcallv
but with a keyword flag.  This is used by rb_obj_call_init (internals
of Class#new).  This also required expected call_type enum with
CALL_FCALL_KW, similar to the recent addition of CALL_PUBLIC_KW.

Add rb_vm_call_kw as a internal function, used by call_method_data
(internals of Method#call and UnboundMethod#bind_call). Add tests
for UnboundMethod#bind_call keyword handling.
2019-09-06 19:41:23 -07:00
Jeremy Evans fd2ef1a9bf Add VM_NO_KEYWORDS
I think this is easier to read than using literal 0 with comments
in every case where it is used.
2019-09-05 17:47:12 -07:00
Yusuke Endoh ce04392d8d Propagate kw_splat information
The kw_splat flag is whether the original call passes keyword or not.
Some types of methods (e.g., bmethod and sym_proc) drops the
information.  This change tries to propagate the flag to the final
callee, as far as I can.
2019-09-05 17:47:12 -07:00
卜部昌平 dd2b9d4a96 hide rb_funcallv_with_cc from public
Requested by ko1.  Also, because now that this function is internal
use only, why not just directly use struct rb_call_cache to purge
the ZALLOC.
2019-09-05 12:13:07 +09:00
Jeremy Evans 39c3252cd1
Merge pull request #2422 from jeremyevans/rb_keyword_given_p
Add rb_keyword_given_p to the C-API
2019-09-03 11:32:02 -07:00
Takashi Kokubun a848b62819
Make the rb_vmdebug_debug_print_post declaration
consistent with the definition
2019-09-03 01:53:49 +09:00
Jeremy Evans 6a9ce1fea8 Support **nil syntax for specifying a method does not accept keyword arguments
This syntax means the method should be treated as a method that
uses keyword arguments, but no specific keyword arguments are
supported, and therefore calling the method with keyword arguments
will raise an ArgumentError.  It is still allowed to double splat
an empty hash when calling the method, as that does not pass
any keyword arguments.
2019-08-30 12:39:31 -07:00
卜部昌平 e3fc30564e rb_thread_create now free from ANYARGS
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct.  This commit deletes ANYARGS from
rb_thread_create, which seems very safe to do.
2019-08-27 15:52:26 +09:00
卜部昌平 703783324c rb_ensure now free from ANYARGS
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct.  This commit deletes ANYARGS from
rb_ensure, which also revealed many arity / type mismatches.
2019-08-27 15:52:26 +09:00
卜部昌平 b8fd2e83e7 decouple compile.c usage of imemo_ifunc
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct.  This commit deletes ANYARGS from
struct vm_ifunc, but in doing so we also have to decouple the usage
of this struct in compile.c, which (I think) is an abuse of ANYARGS.
2019-08-27 15:52:26 +09:00
Nobuyoshi Nakada 765eb18a45
Make VM_DEBUG_VERIFY_METHOD_CACHE derived from VMDEBUG
VM_DEBUG_MODE has been used only here.
2019-08-17 23:14:52 +09:00
Nobuyoshi Nakada 0c2d81dada
Renamed ruby_finalize_{0,1}
And pass rb_execution_context_t as an argument.
2019-08-13 09:47:08 +09:00
Yusuke Endoh 086ffe72c7 Revert "Revert "Add a specialized instruction for `.nil?` calls""
This reverts commit a0980f2446.

Retry for macOS Mojave.
2019-08-02 23:25:38 +09:00
Yusuke Endoh a0980f2446 Revert "Add a specialized instruction for `.nil?` calls"
This reverts commit 9faef3113f.

It seemed to cause a failure on macOS Mojave, though I'm unsure how.
https://rubyci.org/logs/rubyci.s3.amazonaws.com/osx1014/ruby-master/log/20190802T034503Z.fail.html.gz

This tentative revert is to check if the issue is actually caused by the
change or not.
2019-08-02 15:03:34 +09:00
Aaron Patterson 9faef3113f
Add a specialized instruction for `.nil?` calls
This commit adds a specialized instruction for called to `.nil?`.  It is
about 27% faster than master in the case where the object is nil or not
nil.  In the case where an object implements `nil?`, I think it may be
slightly slower.  Here is a benchmark:

```ruby
require "benchmark/ips"

class Niller
  def nil?; true; end
end

not_nil = Object.new
xnil = nil
niller = Niller.new

Benchmark.ips do |x|
  x.report("nil?")    { xnil.nil? }
  x.report("not nil") { not_nil.nil? }
  x.report("niller")   { niller.nil? }
end
```

On Ruby master:

```
[aaron@TC ~/g/ruby (master)]$ ./ruby compil.rb
Warming up --------------------------------------
                nil?   429.195k i/100ms
             not nil   437.889k i/100ms
              niller   437.935k i/100ms
Calculating -------------------------------------
                nil?     20.166M (± 8.1%) i/s -    100.002M in   5.002794s
             not nil     20.046M (± 7.6%) i/s -     99.839M in   5.020086s
              niller     22.467M (± 6.1%) i/s -    112.111M in   5.013817s
[aaron@TC ~/g/ruby (master)]$ ./ruby compil.rb
Warming up --------------------------------------
                nil?   449.660k i/100ms
             not nil   433.836k i/100ms
              niller   443.073k i/100ms
Calculating -------------------------------------
                nil?     19.997M (± 8.8%) i/s -     99.375M in   5.020458s
             not nil     20.529M (± 7.0%) i/s -    102.385M in   5.020689s
              niller     21.796M (± 8.0%) i/s -    108.110M in   5.002300s
[aaron@TC ~/g/ruby (master)]$ ./ruby compil.rb
Warming up --------------------------------------
                nil?   402.119k i/100ms
             not nil   438.968k i/100ms
              niller   398.226k i/100ms
Calculating -------------------------------------
                nil?     20.050M (±12.2%) i/s -     98.519M in   5.008817s
             not nil     20.614M (± 8.0%) i/s -    102.280M in   5.004531s
              niller     22.223M (± 8.8%) i/s -    110.309M in   5.013106s

```

On this branch:

```
[aaron@TC ~/g/ruby (specialized-nilp)]$ ./ruby compil.rb
Warming up --------------------------------------
                nil?   468.371k i/100ms
             not nil   456.517k i/100ms
              niller   454.981k i/100ms
Calculating -------------------------------------
                nil?     27.849M (± 7.8%) i/s -    138.169M in   5.001730s
             not nil     26.417M (± 8.7%) i/s -    131.020M in   5.011674s
              niller     21.561M (± 7.5%) i/s -    107.376M in   5.018113s
[aaron@TC ~/g/ruby (specialized-nilp)]$ ./ruby compil.rb
Warming up --------------------------------------
                nil?   477.259k i/100ms
             not nil   428.712k i/100ms
              niller   446.109k i/100ms
Calculating -------------------------------------
                nil?     28.071M (± 7.3%) i/s -    139.837M in   5.016590s
             not nil     25.789M (±12.9%) i/s -    126.470M in   5.011144s
              niller     20.002M (±12.2%) i/s -     98.144M in   5.001737s
[aaron@TC ~/g/ruby (specialized-nilp)]$ ./ruby compil.rb
Warming up --------------------------------------
                nil?   467.676k i/100ms
             not nil   445.791k i/100ms
              niller   415.024k i/100ms
Calculating -------------------------------------
                nil?     26.907M (± 8.0%) i/s -    133.755M in   5.013915s
             not nil     25.319M (± 7.9%) i/s -    125.713M in   5.007758s
              niller     19.569M (±11.8%) i/s -     96.286M in   5.008533s
```

Co-Authored-By: Ashe Connor <kivikakk@github.com>
2019-07-31 16:21:25 -07:00
Koichi Sasada c23e597674 respect RUBY_DEBUG.
see RUBY_DEBUG for each debug options.
2019-07-15 11:30:34 +09:00
Takashi Kokubun 45dfd4c09d
Make export declaration place more consistent 2019-07-14 18:09:10 +09:00
Samuel Williams e4cafa393f Ensure that vm_stack is cleared in `thread_cleanup_func_before_exec`.
If `vm_stack` is left dangling in a forked process, the gc attempts to scan
it, but it is invalid and will cause a segfault. Therefore, we clear it
before forking.

In order to simplify this, `rb_ec_clear_vm_stack` was introduced.
2019-06-20 16:44:50 +12:00
Samuel Williams c26c514494 Revert failed attempt at fixing invalid usage of vm_stack. 2019-06-20 15:30:29 +12:00
Samuel Williams dbc2b89bc0 Ensure `vm_stack` is cleared after fork. 2019-06-20 13:41:18 +12:00
Samuel Williams d17344cfc5 Remove IA64 support. 2019-06-19 23:30:04 +12:00
Samuel Williams cb5da39f20 Use shared implementation of `rb_ec_initialize_vm_stack`. 2019-06-19 20:39:10 +12:00
Aaron Patterson 6db2d6d852
Add compaction support for more types.
This commit adds compaction support for:

* Fibers
* Continuations
* Autoload Constants
2019-06-11 09:16:14 -07:00
Urabe, Shyouhei fa09acafde extend machine stacks when sanitizers are there
It seems sanitizers require extra amount of machine stacks.  Without
extending them the process tends to stack overflow.
2019-04-26 15:59:40 +09:00
Aaron Patterson ea520ca927
Prevent rb_define_(class|module) classes from moving
Before this commit, classes and modules would be registered with the
VM's `defined_module_hash`.  The key was the ID of the class, but that
meant that it was possible for hash collisions to occur.  The compactor
doesn't allow classes in the `defined_module_hash` to move, but if there
is a conflict, then it's possible a class would be removed from the hash
and not get pined.

This commit changes the key / value of the hash just to be the class
itself, thus preventing movement.
2019-04-22 20:08:01 -07:00
tenderlove 91793b8967 Add `GC.compact` again.
🙏

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67620 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20 01:19:47 +00:00
tenderlove 744e5df715 Reverting compaction for now
For some reason symbols (or classes) are being overridden in trunk

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67598 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-17 09:41:41 +00:00
tenderlove 3c55b643ae Adding `GC.compact` and compacting GC support.
This commit adds the new method `GC.compact` and compacting GC support.
Please see this issue for caveats:

  https://bugs.ruby-lang.org/issues/15626

[Feature #15626]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-17 03:17:25 +00:00
svn 481481b81a * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67500 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-10 09:16:00 +00:00
kazu 25c1fd3b90 Reverting all commits from r67479 to r67496 because of CI failures
Because hard to specify commits related to r67479 only.
So please commit again.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-10 09:15:21 +00:00
tenderlove 3ef4db15e9 Adding `GC.compact` and compacting GC support.
This commit adds the new method `GC.compact` and compacting GC support.
Please see this issue for caveats:

  https://bugs.ruby-lang.org/issues/15626

[Feature #15626]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67479 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-09 20:32:04 +00:00
k0kubun c92c0a5935 Prefer using vm_base_ptr rather than cfp->bp
in MJIT implementation.

This allows us to drop cfp->bp by just modifying vm_base_ptr in the
future.

No performance impact:

$ benchmark-driver benchmark.yml --rbenv='before::before --disable-gems --jit;bp_::after --disable-gems --jit;vm_env_ptr::ruby-svn --disable-gems --jit' -v --output=all --repeat-count=12
before: ruby 2.7.0dev (2019-03-24 trunk 67341) +JIT [x86_64-linux]
bp_: ruby 2.7.0dev (2019-03-24 trunk 67342) +JIT [x86_64-linux]
vm_env_ptr: ruby 2.7.0dev (2019-03-25 trunk 67343) +JIT [x86_64-linux]
last_commit=Prefer using vm_base_ptr rather than cfp->bp
Calculating -------------------------------------
                                       before                   bp_            vm_env_ptr
Optcarrot Lan_Master.nes    77.15059205092646     70.18873044267853     69.62171387083328 fps
                            78.75767783870441     77.49867689173411     75.43496867709587
                            79.60102690369321     77.78037687683523     79.36688927929428
                            80.25144236638835     78.74729849101701     80.42363742291455
                            82.22375417165489     80.44265482494045     80.90287243299306
                            82.29166786292619     80.51740049420938     81.81153053252902
                            83.35386925305345     80.91054205210609     81.93562989125176
                            83.39770634366975     81.34550754145043     82.24544621470430
                            83.88523450309972     81.60698516017347     82.76801860263230
                            84.17553130135879     82.69615943446324     83.02530407910871
                            84.42132328119858     83.00969158037691     83.19968539409922
                            84.60731429793329     83.32703363300098     83.81352746019631

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67344 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-25 14:26:11 +00:00
ko1 8dd0fb9039 use cfp->bp more.
cfp->bp was (re-)introduced by Kokubun san, but VM doesn't use it
because I (ko1) want to remove it in a future. But using it make
leave instruction fast because of sp consisntency check.
So now VM uses cfp->bp.

To use cfp->bp, I checked the value and I found that it is not a
"initial value of sp" but a "initial value of ep". Fix this problem
and fix all bp references (this is why bp is renamed to bp_).


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67342 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-25 06:58:50 +00:00
nobu ba03222da8 Pack rb_iseq_constant_body from 296 to 288 bytes
[Fix GH-2099]

From: Lourens Naudé <lourens@bearmetal.eu>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-18 01:42:20 +00:00
normal 23444302d9 introduce rb_nogvl C-API to mark ubf as async-signal-safe
zlib and bignum both contain unblocking functions which are
async-signal-safe and do not require spawning additional
threads.

We can execute those functions directly in signal handlers
without incurring overhead of extra threads, so provide C-API
users the ability to deal with that.  Other C-API users may
have similar need.

This flexible API can supercede existing uses of
rb_thread_call_without_gvl and rb_thread_call_without_gvl2 by
introducing a flags argument to control behavior.

Note: this API is NOT finalized.  It needs approval from other
committers.  I prefer shorter name than previous
rb_thread_call_without_gvl* functions because my eyes requires
big fonts.

[Bug #15499]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66712 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-04 13:14:11 +00:00
svn d3c0b20949 * remove trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66596 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-27 17:39:21 +00:00
ko1 496267023c fix missed script_compiled events. [Bug #15471]
* ruby.c (process_options): script_compiled events are missed on
  command line -e or specified file. this commit fix it.
  [Bug #15471]

  This patch should be backport to Ruby 2.6 branch.

* vm_core.h (rb_exec_event_hook_script_compiled): introduce utility
  function to invoke a script_compiled event.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66595 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-27 17:39:17 +00:00
ko1 d3aad15922 Return same ISeq object for one src.
* iseq.c: before this patch, RubyVM::InstructionSequence.of(src) (ISeq in
  short) returns different ISeq (wrapper) objects point to one ISeq internal
  object. This patch changes this behavior to cache created ISeq (wrapper)
  objects and return same ISeq object for an internal ISeq object.

* iseq.h (ISEQ_EXECUTABLE_P): introduced to check executable ISeq objects.

* iseq.h (ISEQ_COMPILE_DATA_ALLOC): reordr setting flag line to avoid
  ISEQ_USE_COMPILE_DATA but compiled_data == NULL case.

* vm_core.h (rb_iseq_t): introduce `rb_iseq_t::wrapper` and
  `rb_iseq_t::aux::exec`.  Move `rb_iseq_t::local_hooks` to
  `rb_iseq_t::aux::exec::local_hooks`.

* test/ruby/test_iseq.rb: add ISeq.of() tests.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66246 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-06 10:52:27 +00:00
normal eb38fb670b vm_trace.c: workqueue as thread-safe version of postponed_job
postponed_job is safe to use in signal handlers, but is not
thread-safe for MJIT.  Implement a workqueue for MJIT
thread-safety.

[Bug #15316]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66100 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-30 03:56:29 +00:00
ko1 72e60a0437 `TracePoint#enable(target_line:)` is supported. [Feature #15289]
* vm_trace.c: `TracePoint#enable(target_line:)` is supported.
  This option enables a hook only at specified target_line.
  target_line should be combination with target and :line event.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 20:16:14 +00:00
svn 61c8247d55 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66004 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 18:16:54 +00:00
ko1 96990203b7 Support targetting TracePoint [Feature #15289]
* vm_trace.c (rb_tracepoint_enable_for_target): support targetting
  TracePoint. [Feature #15289]

  Tragetting TracePoint is only enabled on specified method, proc
  and so on, example: `tp.enable(target: code)`.

  `code` should be consisted of InstructionSeuqnece (iseq)
  (RubyVM::InstructionSeuqnece.of(code) should not return nil)
  If code is a tree of iseq, TracePoint is enabled on all of
  iseqs in a tree.

  Enabled tragetting TracePoints can not enabled again with
  and without target.

* vm_core.h (rb_iseq_t): introduce `rb_iseq_t::local_hooks`
  to store local hooks.
  `rb_iseq_t::aux::trace_events` is renamed to
  `global_trace_events` to contrast with `local_hooks`.

* vm_core.h (rb_hook_list_t): add `rb_hook_list_t::running`
  to represent how many Threads/Fibers are used this list.
  If this field is 0, nobody using this hooks and we can
  delete it.

  This is why we can remove code from cont.c.

* vm_core.h (rb_vm_t): because of above change, we can eliminate
  `rb_vm_t::trace_running` field.
  Also renamed from `rb_vm_t::event_hooks` to `global_hooks`.

* vm_core.h, vm.c (ruby_vm_event_enabled_global_flags): renamed
  from `ruby_vm_event_enabled_flags.

* vm_core.h, vm.c (ruby_vm_event_local_num): added to count
  enabled targetting TracePoints.

* vm_core.h, vm_trace.c (rb_exec_event_hooks): accepts
  hook list.

* vm_core.h (rb_vm_global_hooks): added for convinience.

* method.h (rb_method_bmethod_t): added to maintain Proc
  and `rb_hook_list_t` for bmethod (defined by define_method).

* prelude.rb (TracePoint#enable): extracet a keyword parameter
  (because it is easy than writing in C).
  It calls `TracePoint#__enable` internal method written in C.

* vm_insnhelper.c (vm_trace): check also iseq->local_hooks.

* vm.c (invoke_bmethod): check def->body.bmethod.hooks.

* vm.c (hook_before_rewind): check iseq->local_hooks
  and def->body.bmethod.hooks before rewind by exception.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66003 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 18:16:39 +00:00
svn e4c9a70c96 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65776 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-18 01:37:45 +00:00