Граф коммитов

132 Коммитов

Автор SHA1 Сообщение Дата
Koichi Sasada c9a9b8036c remove `Ractor::Selector` from Ruby level
`Ractor::Selector` is not approved by Matz so remove it from
Ruby-level.

The implementation is used by `Ractor.select` so most of implementation
was remaind and calling `rb_init_ractor_selector()`, `Ractor::Selector`
will be defined. I will provide `ractor-selector` gem to try it.
2023-12-16 01:00:01 +09:00
Peter Zhu 10f44dfeff Fix Ractor sharing for too complex Objects 2023-11-28 17:43:22 -05:00
Peter Zhu 6eb5a9cf8f Fix Ractor sharing for too complex generic ivars 2023-11-28 17:43:22 -05:00
Peter Zhu e2d950733e Add ST table to gen_ivtbl for complex shapes
On 32-bit systems, we must store the shape ID in the gen_ivtbl to not
lose the shape. If we directly store the ST table into the generic
ivar table, then we lose the shape. This makes it impossible to
determine the shape of the object and whether it is too complex or not.
2023-10-31 12:07:54 -04:00
Koichi Sasada be1bbd5b7d M:N thread scheduler for Ractors
This patch introduce M:N thread scheduler for Ractor system.

In general, M:N thread scheduler employs N native threads (OS threads)
to manage M user-level threads (Ruby threads in this case).
On the Ruby interpreter, 1 native thread is provided for 1 Ractor
and all Ruby threads are managed by the native thread.

From Ruby 1.9, the interpreter uses 1:1 thread scheduler which means
1 Ruby thread has 1 native thread. M:N scheduler change this strategy.

Because of compatibility issue (and stableness issue of the implementation)
main Ractor doesn't use M:N scheduler on default. On the other words,
threads on the main Ractor will be managed with 1:1 thread scheduler.

There are additional settings by environment variables:

`RUBY_MN_THREADS=1` enables M:N thread scheduler on the main ractor.
Note that non-main ractors use the M:N scheduler without this
configuration. With this configuration, single ractor applications
run threads on M:1 thread scheduler (green threads, user-level threads).

`RUBY_MAX_CPU=n` specifies maximum number of native threads for
M:N scheduler (default: 8).

This patch will be reverted soon if non-easy issues are found.

[Bug #19842]
2023-10-12 14:47:01 +09:00
Peter Zhu 1e7b67f733 [Feature #19730] Remove transient heap 2023-07-13 09:27:33 -04:00
S-H-GAMELINKS 092c9b266a Reuse rb_ractor_make_shareable function for rb_ractor_make_shareable_copy function 2023-07-10 19:06:18 +09:00
Nobuyoshi Nakada 39933496ae
Remove duplicate declarations 2023-06-30 23:59:05 +09:00
Nobuyoshi Nakada b934976024
Prefer `0` over `NULL` as function pointers
SunC warns use of `NULL`, pointer to data as function pointers.
2023-06-23 03:15:55 +09:00
Peter Zhu 5199f2aaf9 Implement Hash AR tables on VWA 2023-05-17 09:19:40 -04:00
Koichi Sasada ffce3117b6 add new debug log line on `rb_ractor_terminate_all`
to leave the trace.
2023-04-26 17:58:32 +09:00
Koichi Sasada f5b824c745 show debug log for `ractor_terminal_interrupt_all` 2023-03-30 14:56:37 +09:00
Koichi Sasada 30b43f4f1a `rb_ractor_thread_list()` only for current ractor
so that no need to lock the ractor.
2023-03-30 14:56:37 +09:00
Koichi Sasada 94e4182267 `rb_current_ractor_raw(b)`
`rb_current_ractor()` expects it has valid `ec` and `r`.
`rb_current_ractor_raw()` with a parameter `false` allows to return
NULL if `ec` is not available.
2023-03-30 14:56:23 +09:00
Aaron Patterson 54dbd8bea8 Use an st table for "too complex" objects
st tables will maintain insertion order so we can marshal dump / load
objects with instance variables in the same order they were set on that
particular instance

[ruby-core:112926] [Bug #19535]

Co-Authored-By: Jemma Issroff <jemmaissroff@gmail.com>
2023-03-20 13:54:18 -07:00
Nobuyoshi Nakada bcd0aa896f
Fix -Wclobbered warning from gcc 12
A variable modified in `EXEC_TAG` block should be `volatile`.

```
ractor.c: In function 'ractor_try_yield':
ractor.c:1251:97: warning: argument 'obj' might be clobbered by 'longjmp' or 'vfork' [-Wclobbered]
 1251 | ractor_try_yield(rb_execution_context_t *ec, rb_ractor_t *cr, struct rb_ractor_queue *ts, VALUE obj, VALUE move, bool exc, bool is_will)
      |                                                                                           ~~~~~~^~~
```
2023-03-19 21:57:22 +09:00
Nobuyoshi Nakada ce47ee00ae Fix indirect counter increment
`*pcnt++` just dereferences `pcnt` then increments the local variable,
but has no side effect.
2023-03-15 13:59:11 +09:00
Takashi Kokubun 23ec248e48 s/mjit/rjit/ 2023-03-06 23:44:01 -08:00
Takashi Kokubun 2e875549a9 s/MJIT/RJIT/ 2023-03-06 23:44:01 -08:00
Takashi Kokubun 233ddfac54 Stop exporting symbols for MJIT 2023-03-06 21:59:23 -08:00
Koichi Sasada 941d36d195 fix timing bug
passing will and closing notification can conflict and
`Ractor::Selector#empty?` can return wrong answer.
This patch fix it.

```
     s = Ractor::Selector.new
     s.add Ractor.new{10}
     s.add Ractor.new{20}
     r, v = s.wait
     vs = []
     vs << v
     r, v = s.wait
     vs << v
     [*vs.sort, s.empty?]
  #=> "[10, 20, false]" (expected "[10, 20, true]")
```
2023-03-03 10:19:37 +09:00
Koichi Sasada 5875fce6ce `Ractor::Selector#empty?`
It returns the waiting set is empty or not.

Also add Ractor::Selector's tests.
2023-03-03 00:08:02 +09:00
Koichi Sasada a4421bd73c Rewrite Ractor synchronization mechanism
This patch rewrites Ractor synchronization mechanism, send/receive
and take/yield.

* API
  * Ractor::Selector is introduced for lightweight waiting
    for many ractors.
* Data structure
  * remove `struct rb_ractor_waiting_list` and use
    `struct rb_ractor_queue takers_queue` to manage takers.
  * remove `rb_ractor_t::yield_atexit` and use
    `rb_ractor_t::sync::will_basket::type` to check the will.
  * add `rb_ractor_basket::p.take` to represent a taking ractor.
* Synchronization protocol
  * For the Ractor local GC, `take` can not make a copy object
    directly so ask to generate the copy from the yielding ractor.
  * The following steps shows what `r1.take` does on `r0`.
    * step1: (r0) register `r0` into `r1`'s takers.
    * step2: (r0) check `r1`'s status and wakeup r0 if `r1` is waiting
             for yielding a value.
    * step3: (r0) sleep until `r1` wakes up `r0`.
  * The following steps shows what `Ractor.yield(v)` on `r1`.
    * step1: (r1) check first takers of `r1` and if there is (`r0`),
             make a copy object of `v` and pass it to `r0` and
             wakes up `r0`.
    * step2: (r1) if there is no taker ractors, sleep until
             another ractor try to take.
2023-03-02 14:31:54 +09:00
Koichi Sasada 1f936d654a `ractor_queue_enq/deq` doesn't need `rq` param
`rq` is always `r`'s queue.
2023-02-15 15:42:51 +09:00
Matt Valentine-House 72aba64fff Merge gc.h and internal/gc.h
[Feature #19425]
2023-02-09 10:32:29 -05:00
Peter Zhu 2296b877d8 Remove rb_hash_st_table
It's a duplicate of RHASH_ST_TABLE.
2023-01-31 15:48:09 -05:00
Takashi Kokubun b9332ac8e7
MJIT: Cancel all on disastrous situations (#7019)
I noticed this while running test_yjit with --mjit-call-threshold=1, 
which redefines `Integer#<`. When Ruby is monkey-patched, 
MJIT itself could be broken.

Similarly, Ruby scripts could break MJIT in many different ways. I
prepared the same set of hooks as YJIT so that we could possibly
override it and disable it on those moments. Every constant under
RubyVM::MJIT is private and thus it's an unsupported behavior though.
2022-12-24 01:13:40 -08:00
Jemma Issroff c1ab6ddc9a Transition complex objects to "too complex" shape
When an object becomes "too complex" (in other words it has too many
variations in the shape tree), we transition it to use a "too complex"
shape and use a hash for storing instance variables.

Without this patch, there were rare cases where shape tree growth could
"explode" and cause performance degradation on what would otherwise have
been cached fast paths.

This patch puts a limit on shape tree growth, and gracefully degrades in
the rare case where there could be a factorial growth in the shape tree.

For example:

```ruby
class NG; end

HUGE_NUMBER.times do
  NG.new.instance_variable_set(:"@unique_ivar_#{_1}", 1)
end
```

We consider objects to be "too complex" when the object's class has more
than SHAPE_MAX_VARIATIONS (currently 8) leaf nodes in the shape tree and
the object introduces a new variation (a new leaf node) associated with
that class.

For example, new variations on instances of the following class would be
considered "too complex" because those instances create more than 8
leaves in the shape tree:

```ruby
class Foo; end
9.times { Foo.new.instance_variable_set(":@uniq_#{_1}", 1) }
```

However, the following class is *not* too complex because it only has
one leaf in the shape tree:

```ruby
class Foo
  def initialize
    @a = @b = @c = @d = @e = @f = @g = @h = @i = nil
  end
end
9.times { Foo.new }
``

This case is rare, so we don't expect this change to impact performance
of most applications, but it needs to be handled.

Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
2022-12-15 10:06:04 -08:00
S-H-GAMELINKS 1f4f6c9832 Using UNDEF_P macro 2022-11-16 18:58:33 +09:00
Nobuyoshi Nakada 131c31a920
[Bug #19081] Show the caller location in warning for Ractor
The internal location in ractor.rb is not usefull at all.
```
$ ruby -e 'Ractor.new {}'
<internal:ractor>:267: warning: Ractor is experimental, ...
```
2022-10-26 19:43:14 +09:00
Aaron Patterson f0654b1027 More precisely iterate over Object instance variables
Shapes provides us with an (almost) exact count of instance variables.
We only need to check for Qundef when an IV has been "undefined"
Prefer to use ROBJECT_IV_COUNT when iterating IVs
2022-10-15 10:44:10 -07:00
卜部昌平 406ebb25a0 obj_refer_only_sharables_p_i: need derefernce
Clang says "warning: variable 'pcnt' set but not used" here.  In fact it
doesn't. The intention is clear that we want to increment cnt, not pcnt.
Adding a * mark solves everything.
2022-09-21 11:44:09 +09:00
Aaron Patterson 28a3434634 Disable Ractor check on 32bit architectures
Ractor verification requires storing the ractor id in the top 32 bits of
the object header.  Unfortunately 32 bit machines only have 32 bits in
the object header.  The verification code has a 32 bit left shift which
doesn't work on i686 and will clobber existing flags.

This commit disables the verification code on i686 since i686 will crash
if it's enabled.

Co-Authored-By: John Hawthorn <john@hawthorn.email>
Co-Authored-By: Jemma Issroff <jemmaissroff@gmail.com>
2022-08-24 10:54:27 -07:00
Yusuke Endoh 8f7e188822 Add "rb_" prefixes to toplevel enum definitions
... as per ko1's request.
2022-07-22 23:10:24 +09:00
Takashi Kokubun 5b21e94beb Expand tabs [ci skip]
[Misc #18891]
2022-07-21 09:42:04 -07:00
Koichi Sasada 1c4fc0241d rename thread internal naming
Now GVL is not process *Global* so this patch try to use
another words.

* `rb_global_vm_lock_t` -> `struct rb_thread_sched`
  * `gvl->owner` -> `sched->running`
  * `gvl->waitq` -> `sched->readyq`
* `rb_gvl_init` -> `rb_thread_sched_init`
* `gvl_destroy` -> `rb_thread_sched_destroy`
* `gvl_acquire` -> `thread_sched_to_running` # waiting -> ready -> running
* `gvl_release` -> `thread_sched_to_waiting` # running -> waiting
* `gvl_yield`   -> `thread_sched_yield`
* `GVL_UNLOCK_BEGIN` -> `THREAD_BLOCKING_BEGIN`
* `GVL_UNLOCK_END` -> `THREAD_BLOCKING_END`

* removed
  * `rb_ractor_gvl`
  * `rb_vm_gvl_destroy` (not used)

There are GVL functions such as `rb_thread_call_without_gvl()` yet
but I don't have good name to replace them. Maybe GVL stands for
"Greate Valuable Lock" or something like that.
2022-04-22 07:54:09 +09:00
Nobuyoshi Nakada 7b1ece9b94
Get rid of type-punning pointer casts 2022-04-07 19:19:13 +09:00
Yusuke Endoh 23530d68cb ractor.c: Add a helper function to ensure the context is a main ractor 2022-03-30 16:50:46 +09:00
Nobuyoshi Nakada 42a0bed351
Prefix ccan headers (#4568)
* Prefixed ccan headers

* Remove unprefixed names in ccan/build_assert

* Remove unprefixed names in ccan/check_type

* Remove unprefixed names in ccan/container_of

* Remove unprefixed names in ccan/list

Co-authored-by: Samuel Williams <samuel.williams@oriontransfer.co.nz>
2022-03-30 20:36:31 +13:00
Alan Wu 51e98eab1f
Fix Ractor.receive_if + rb_vm_barrier() deadlock
I have this scripts that deadlocks after about
5 minutes if I repeatedly run it with a shell loop:

```ruby
$VERBOSE = nil
lamb = ->(main, gc) do
  gc.verify_internal_consistency
  gc.verify_internal_consistency
  main << 1
  gc.verify_internal_consistency
  gc.verify_internal_consistency
  main << 2
  gc.verify_internal_consistency
  gc.verify_internal_consistency
  main << 3
  gc.verify_internal_consistency
  gc.verify_internal_consistency
end

lamb[[], GC]
lamb[[], GC]

r = Ractor.new Ractor.current, GC, &lamb

a = []
a << Ractor.receive_if{|msg| msg == 2}
a << Ractor.receive_if{|msg| msg == 3}
a << Ractor.receive_if{|msg| msg == 1}
```

Shell loop:

```shell
while ./miniruby deadlock.rb; do date; done
```

Once it locks up, CTRL-C doesn't interrupt the process which
led me to infer `receive_if` is looping in `ractor_receive_if()`
without checking for interrupts. This can be confirmed by
attaching a debugger to the deadlocked miniruby.

The deadlock has one thread looping in `receive_if`
and another waiting in `rb_vm_barrier()`. The barrier relies
on interrupt checking to finish. Theoretically speaking the
`rb_vm_barrier()` could come from one thread naturally starting GC.
We found this while developing YJIT but it dead locks running
with YJIT disabled. YJIT currently relies on `rb_vm_barrier()`
to synchronize before changing memory protection.

This diff adds an interrupt check in the loop in `Ractor#receive_if`
which seems to fix the deadlock.

In addition, this commit allows interrupting the following single
ractor script with CTRL-C.

```shell
ruby -e 'Ractor.current.send(3); Ractor.receive_if { false }'
```
2022-03-28 17:00:45 -04:00
Koichi Sasada c9af8a32a0 `USE_RUBY_DEBUG_LOG` doesn't check `RUBY_DEVEL`
`USE_RUBY_DEBUG_LOG` was only defined when `RUBY_DEVEL` is defined.
This patch removes this dependency (`USE_RUBY_DEBUG_LOG` is defined
independently from `RUBY_DEVEL`).

Do not commit a patch which enables `USE_RUBY_DEBUG_LOG`.
2021-12-29 00:22:02 +09:00
Nobuyoshi Nakada 975a6efd7e Suppress undef warnings for USE_RUBY_DEBUG_LOG 2021-12-25 18:04:25 +09:00
Jose Narvaez 4e2eb7695e Yet Another Ruby JIT!
Renaming uJIT to YJIT. AKA s/ujit/yjit/g.
2021-10-20 18:19:31 -04:00
Alan Wu 57977ba30d uJIT: Implement opt_getinlinecache
* ujit: implement opt_getinlinecache

Aggressively bet that writes to constants don't happen and invalidate
all opt_getinlinecache blocks on any and all constant writes.

Use alignment padding on block_t to track this assumption. No change to
sizeof(block_t).

* Fix compile warnings when not RUBY_DEBUG
* Fix reversed condition
* Switch to st_table to keep track of assumptions

Co-authored-by: Aaron Patterson <aaron.patterson@gmail.com>
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
2021-10-20 18:19:30 -04:00
Nobuyoshi Nakada 0c15752556
Use `RUBY_FUNCTION_NAME_STRING` for old Visual C++
Probably `__func__` is supported since Visual C++ 2015 (= 14.0,
`_MSC_VER` = 1900).
2021-10-20 11:22:33 +09:00
Nobuyoshi Nakada 768ceb4ead
Cast to void pointer for `%p` in commented out code [ci skip] 2021-10-20 11:22:33 +09:00
Kazuhiro NISHIYAMA 9b18f1bffe Supress `warning: data argument not used by format string [-Wformat-extra-args]` 2021-10-20 07:48:30 +09:00
Nobuyoshi Nakada ff480f2953
Cast to void pointer to suppress -Wformat-pedantic in RUBY_DEBUG_LOG 2021-10-03 13:59:48 +09:00
Peter Zhu bbedd29b6e [Bug #18117] Fix Ractor race condition with GC
rb_objspace_reachable_objects_from requires that the GC not be active.
Since the Ractor barrier is not executed for incremental sweeping,
Ractor may call rb_objspace_reachable_objects_from after sweeping
has started to share objects. This causes a crash that looks like
the following:

```
<internal:ractor>:627: [BUG] rb_objspace_reachable_objects_from() is not supported while during_gc == true
```

Co-authored-by: Vinicius Stock <vinicius.stock@shopify.com>
2021-08-24 09:47:42 -04:00
Peter Zhu 4a627dbdfd [Bug #18014] Fix memory leak in GC when using Ractors
When a Ractor is removed, the freelist in the Ractor cache is not
returned to the GC, leaving the freelist permanently lost. This commit
recycles the freelist when the Ractor is destroyed, preventing a memory
leak from occurring.
2021-07-15 11:48:52 -04:00