* Implement gen_newarray
* Implement newhash for n=0
* Add yjit tests for newhash/newarray
* Fix integer size warning on clang
* Save PC and SP in newhash and newarray
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
* Implement getblockparamproxy
* Parallel runner: wait for timeout thread to terminate after killing
Or else the leak cheaker could sees the thread as running and cause test
failures in test-tool.
* Add a comment, use jne
* Comment about where 0x3 comes from
* Implement send with blocks
Not that much extra work compared to `opt_send_without_block`.
Moved the stack over flow check because it could've exited after changes
are made to cfp.
* rename oswb counters
* Might as well implement sending block to cfuncs
* Disable sending blocks to cfuncs for now
* Reconstruct interpreter sp before calling into cfuncs
In case the callee cfunc calls a method or delegates to a block.
This also has the side benefit of letting call sites that sometimes are
iseq calls and sometimes cfunc call share the same successor.
* only sync with interpreter sp when passing a block
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Aaron Patterson <aaron.patterson@shopify.com>
* Use rb_ivar_get() for general case of getivar
Pretty straight forward. Buys about 1% coverage on railsbench.
* Update yjit_codegen.c
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
* Implement calls to methods with simple optional params
* Remove unnecessary MJIT_STATIC
See comment for MJIT_STATIC. I added it not knowing whether it's
required because the function next to it has it. Don't use it and wait
for problems to come up instead.
* Better naming, some comments
* Count bailing on kw only iseqs
On railsbench:
```
opt_send_without_block exit reasons:
bmethod 59729 (27.7%)
optimized_method 59137 (27.5%)
iseq_complex_callee 41362 (19.2%)
alias_method 33346 (15.5%)
callsite_not_simple 19170 ( 8.9%)
iseq_only_keywords 1300 ( 0.6%)
kw_splat 1299 ( 0.6%)
cfunc_ruby_array_varg 18 ( 0.0%)
```
Lazily compile out a chain of checks for different known classes and
whether `self` embeds its ivars or not.
* Remove trailing whitespaces
* Get proper addresss in Capstone disassembly
* Lowercase address in Capstone disassembly
Capstone uses lowercase for jump targets in generated listings. Let's
match it.
* Use the same successor in getivar guard chains
Cuts down on duplication
* Address reviews
* Fix copypasta error
* Add a comment
When a BOP is redefined, the BOP redefinition callback will invalidate
any blocks that depend on BOPS. This allows us to eliminate runtime
checks for BOP redefinition.
env_copy() uses rb_ary_delete_at() with a loop counting up while
iterating through the list of read only locals. rb_ary_delete_at() can
shift elements in the array to an index lesser than the loop index,
causing locals to be missed and set to Qfalse in the returned
environment.
Iterate through the locals in reverse instead, this way the shifting
never happens for locals that are yet to be visited and we process all
the locals in the array.
[Bug #18023]
rb_objspace_reachable_objects_from requires that the GC not be active.
Since the Ractor barrier is not executed for incremental sweeping,
Ractor may call rb_objspace_reachable_objects_from after sweeping
has started to share objects. This causes a crash that looks like
the following:
```
<internal:ractor>:627: [BUG] rb_objspace_reachable_objects_from() is not supported while during_gc == true
```
Co-authored-by: Vinicius Stock <vinicius.stock@shopify.com>
If the GC has been disabled we need to re-enable it so we can evacuate
the transient heap.
Fixes https://bugs.ruby-lang.org/issues/17985
[Bug #17985] [ruby-core:104260]
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Fixed the race condition when replacing `freelist` entry with its
chained next element. At acquiring an entry, hold the entry once
with the special value, then release by replacing it with the next
element again after acquired. If another thread is holding the
same entry at that time, spinning until the entry gets released.
Co-Authored-By: Koichi Sasada <ko1@atdot.net>
Ractor.yield(obj, move: true) and
Ractor.select(..., yield_value: obj, move: true) tried to yield a
value with move semantices, but if the trial is faild, the obj
should not become a moved object.
To keep this rule, `wait_moving` wait status is introduced.
New yield/take process:
(1) If a ractor tried to yield (move:true), make taking racotr's
wait status `wait_moving` and make a moved object by
`ractor_move(obj)` and wakeup taking ractor.
(2) If a ractor tried to take a message from a ractor waiting fo
yielding (move:true), wakeup the ractor and wait for (1).
because the name "MJIT" is an internal code name, it's inconsistent with
--jit while they are related to each other, and I want to discourage future
JIT implementation-specific (e.g. MJIT-specific) APIs by this rename.
[Feature #17490]
constant cache `IC` is accessed by non-atomic manner and there are
thread-safety issues, so Ruby 3.0 disables to use const cache on
non-main ractors.
This patch enables it by introducing `imemo_constcache` and allocates
it by every re-fill of const cache like `imemo_callcache`.
[Bug #17510]
Now `IC` only has one entry `IC::entry` and it points to
`iseq_inline_constant_cache_entry`, managed by T_IMEMO object.
`IC` is atomic data structure so `rb_mjit_before_vm_ic_update()` and
`rb_mjit_after_vm_ic_update()` is not needed.
Ractor.make_shareable(obj) tries to make obj a shareable object
by changing the attribute of obj and traversable objects from obj
(mainly freeze them).
"copy: true" option is more conservative approach by make deep
copied object and make it sharable. It doesn't affect any existing
objects.
ObjectSpace._id2ref(id) can return any objects even if they are
unshareable, so this patch raises RangeError if it runs on multi-ractor
mode and the found object is unshareable.
Thread's interrupt set Ractor's wakeup_status as interrupted, but
the status remains next Ractor communication API. This patch makes
to ignore the previous interrupt state.
[Bug #17366]
Also this patch solves the Thread#kill and Ractor#take issues.
ractor_copy() used rb_ary_modify() to make sure this array is not
sharing anything, but it also checks frozen flag. So frozen arrays
raises an error. To solve this issue, this patch introduces new
function rb_ary_cancel_sharing() which makes sure the array does not
share another array and it doesn't check frozen flag.
[Bug #17343]
A test is quoted from https://github.com/ruby/ruby/pull/3817
close_incoming by antoher ractor means there is no other messages
will be sent to the ractor, so Ractor.receive will block forever,
and it should raise and stop.
close_outgoing by antoher ractor means, ... I don't have good idea
to use it. It can be a private method.
Ractor#close calls both, but it does not make sense to call
different purpose methods, so I remove it.
a method defined by define_method with normal Proc can not cross
ractors because the normal Proc is not shareable. However,
shareable Proc can be crossed between ractors, so the method with
shareable Proc should be called correctly.
Ractor.make_shareable() supports Proc object if
(1) a Proc only read outer local variables (no assignments)
(2) read outer local variables are shareable.
Read local variables are stored in a snapshot, so after making
shareable Proc, any assignments are not affeect like that:
```ruby
a = 1
pr = Ractor.make_shareable(Proc.new{p a})
pr.call #=> 1
a = 2
pr.call #=> 1 # `a = 2` doesn't affect
```
[Feature #17284]
Accessing a shareable object is prohibitted because it can cause
race condition, but if the shareable object is frozen, there is no
problem to access ivars.
Introduce new method Ractor.make_shareable(obj) which tries to make
obj shareable object. Protocol is here.
(1) If obj is shareable, it is shareable.
(2) If obj is not a shareable object and if obj can be shareable
object if it is frozen, then freeze obj. If obj has reachable
objects (rs), do rs.each{|o| Ractor.make_shareable(o)}
recursively (recursion is not Ruby-level, but C-level).
(3) Otherwise, raise Ractor::Error. Now T_DATA is not a shareable
object even if the object is frozen.
If the method finished without error, given obj is marked as
a sharable object.
To allow makng a shareable frozen T_DATA object, then set
`RUBY_TYPED_FROZEN_SHAREABLE` as type->flags. On default,
this flag is not set. It means user defined T_DATA objects are
not allowed to become shareable objects when it is frozen.
You can make any object shareable by setting FL_SHAREABLE flag,
so if you know that the T_DATA object is shareable (== thread-safe),
set this flag, at creation time for example. `Ractor` object is one
example, which is not a frozen, but a shareable object.
On Solaris, it seems to access ENV in ``, so skip it now.
```
stderr output is not empty
Exception `NameError' at bootstraptest.tmp.rb:7 - can not access non-sharable objects in constant Object::ENV by non-main Ractor.
#<Thread:0x0044cdf0 run> terminated with exception (report_on_exception is true):
bootstraptest.tmp.rb:7:in ``': can not access non-sharable objects in constant Object::ENV by non-main Ractor. (NameError)
Exception `Ractor::RemoteError' at <internal:ractor>:130 - thrown by remote Ractor.
<internal:ractor>:130:in `take': thrown by remote Ractor. (Ractor::RemoteError)
from bootstraptest.tmp.rb:55:in `<main>'
bootstraptest.tmp.rb:7:in ``': can not access non-sharable objects in constant Object::ENV by non-main Ractor. (NameError)
from bootstraptest.tmp.rb:7:in `ractor_local_globals'
from bootstraptest.tmp.rb:54:in `block in <main>'
```
Unshareable objects should not be touched from multiple ractors
so ObjectSpace.each_object should be restricted. On multi-ractor
mode, ObjectSpace.each_object only iterates shareable objects.
[Feature #17270]
generic_ivtbl is a process global table to maintain instance variables
for non T_OBJECT/T_CLASS/... objects. So we need to protect them
for multi-Ractor exection.
Hint: we can make them Ractor local for unshareable objects, but
now it is premature optimization.
enc_table which manages Encoding information. rb_encoding_list
also manages Encoding objects. Both are accessed/modified by ractors
simultaneously so that they should be synchronized.
For enc_table, this patch introduced GLOBAL_ENC_TABLE_ENTER/LEAVE/EVAL
to access this table with VM lock. To make shortcut, three new global
variables global_enc_ascii, global_enc_utf_8, global_enc_us_ascii are
also introduced.
For rb_encoding_list, we split it to rb_default_encoding_list (256 entries)
and rb_additional_encoding_list. rb_default_encoding_list is fixed sized Array
so we don't need to synchronized (and most of apps only needs it). To manage
257 or more encoding objects, they are stored into rb_additional_encoding_list.
To access rb_additional_encoding_list., VM lock is needed.
Ractor#close_outgoing should cancel waiting Ractor.yield. However,
yield a value by the Ractor's block should not cancel (to recognize
terminating Ractor, introduce rb_ractor_t::yield_atexit flag).
This implementation has memory corruption errors so and
it causes BUG on rare occasions. This commit skips
suspect tests on Github actions Compiler tests.
This commit introduces Ractor mechanism to run Ruby program in
parallel. See doc/ractor.md for more details about Ractor.
See ticket [Feature #17100] to see the implementation details
and discussions.
[Feature #17100]
This commit does not complete the implementation. You can find
many bugs on using Ractor. Also the specification will be changed
so that this feature is experimental. You will see a warning when
you make the first Ractor with `Ractor.new`.
I hope this feature can help programmers from thread-safety issues.
On btest, stderr messages are not displayed if core files are
generated. There is no reason to skip it, so this patch display
stderr and check core files.
This changes the following warnings:
* warning: class variable access from toplevel
* warning: class variable @foo of D is overtaken by C
into RuntimeErrors. Handle defined?(@@foo) at toplevel
by returning nil instead of raising an exception (the previous
behavior warned before returning nil when defined? was used).
Refactor the specs to avoid the warnings even in older versions.
The specs were checking for the warnings, but the purpose of
the related specs as evidenced from their description is to
test for behavior, not for warnings.
Fixes [Bug #14541]
rb_uninterruptible() disables any interrupts using handle_interrupt
feature (This function is used by `p`).
After this function, pending interrupts should be checked correctly,
however there is no chance to setup interrupt flag of working
threads, it means that nobody checks pending interrupts.
For example, it ignores terminate signal delivered at the end
of main thread and program can't stop.
This patch set interrupt flag if there are pending interrupts.
On Deiban 9 environment, the thread tests failed and
this maximum threads information can finish up the machine
resources. To check it, I turned-off showing this information.
This behavior was deprecated in 2.7 and scheduled to be removed
in 3.0.
Calling yield in a class definition outside a method is now a
SyntaxError instead of a LocalJumpError, as well.
This changes object_id from being based on the objects location in
memory (or a nearby memory location in the case of a conflict) to be
based on an always increasing number.
This number is a Ruby Integer which allows it to overflow the size of a
pointer without issue (very unlikely to happen in real programs
especially on 64-bit, but a nice guarantee).
This changes obj_to_id_tbl and id_to_obj_tbl to both be maps of Ruby
objects to Ruby objects (previously they were Ruby object to C integer)
which simplifies updating them after compaction as we can run them
through gc_update_table_refs.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
This changes object_id from being based on the objects location in
memory (or a nearby memory location in the case of a conflict) to be
based on an always increasing number.
This number is a Ruby Integer which allows it to overflow the size of a
pointer without issue (very unlikely to happen in real programs
especially on 64-bit, but a nice guarantee).
This changes obj_to_id_tbl and id_to_obj_tbl to both be maps of Ruby
objects to Ruby objects (previously they were Ruby object to C integer)
which simplifies updating them after compaction as we can run them
through gc_update_table_refs.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
In addition to detect dead canary, we try to detect the very moment
when we smash the stack top. Requested by k0kubun:
https://twitter.com/k0kubun/status/1085180749899194368
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66981 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* bootstraptest/runner.rb (assert_normal_exit): check MJIT first
to support btest with ruby ~2.5. btest (bootstraptest) should be
enable to run with stable ruby interpreter because modified ruby
may not able to run runner.rb and we need to know why (this is why
we introduce btest).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66842 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
"hash_bulk_insert" first expands the table, but the target size was
wrong: it was calculated by "num_entries + (size to buld insert)", but
it was wrong when "num_entries < entries_bound", i.e., it has a deleted
entry. "hash_bulk_insert" adds the given entries from entries_bound,
which led to out-of-bounds write access. [Bug #15536]
As a simple fix, this commit changes the calculation to "entries_bound +
size". I'm afraid if this might be inefficient, but I think it is safe
anyway.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66832 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
because test_io.rb:33 randomly fails
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1519055
checking MJIT.enabled? on driver might not make sense for target, but as
long as the CI is -DMJIT_FORCE_ENABLE, I believe it works for now.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66344 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
John Hawthorn
Alan Wu
Maxime Chevalier-Boisvert
Aaron Patterson
Maxime Chevalier-Boisvert
Peter Zhu
Samuel Williams
Nobuyoshi Nakada
eileencodes
Jun Aruga
Koichi Sasada
Takashi Kokubun
Marc-Andre Lafortune
Benoit Daloze
Michael Lindley
Quang-Minh Nguyen
Kazuhiro NISHIYAMA
Jeremy Evans
卜部昌平
git
nobu
shyouhei
naruse
ko1
mame
k0kubun