This patch introduce M:N thread scheduler for Ractor system.
In general, M:N thread scheduler employs N native threads (OS threads)
to manage M user-level threads (Ruby threads in this case).
On the Ruby interpreter, 1 native thread is provided for 1 Ractor
and all Ruby threads are managed by the native thread.
From Ruby 1.9, the interpreter uses 1:1 thread scheduler which means
1 Ruby thread has 1 native thread. M:N scheduler change this strategy.
Because of compatibility issue (and stableness issue of the implementation)
main Ractor doesn't use M:N scheduler on default. On the other words,
threads on the main Ractor will be managed with 1:1 thread scheduler.
There are additional settings by environment variables:
`RUBY_MN_THREADS=1` enables M:N thread scheduler on the main ractor.
Note that non-main ractors use the M:N scheduler without this
configuration. With this configuration, single ractor applications
run threads on M:1 thread scheduler (green threads, user-level threads).
`RUBY_MAX_CPU=n` specifies maximum number of native threads for
M:N scheduler (default: 8).
This patch will be reverted soon if non-easy issues are found.
[Bug #19842]
* Add rb_io_path and rb_io_open_descriptor.
* Use rb_io_open_descriptor to create PTY objects
* Rename FMODE_PREP -> FMODE_EXTERNAL and expose it
FMODE_PREP I believe refers to the concept of a "pre-prepared" file, but
FMODE_EXTERNAL is clearer about what the file descriptor represents and
aligns with language in the IO::Buffer module.
* Ensure that rb_io_open_descriptor closes the FD if it fails
If FMODE_EXTERNAL is not set, then it's guaranteed that Ruby will be
responsible for closing your file, eventually, if you pass it to
rb_io_open_descriptor, even if it raises an exception.
* Rename IS_EXTERNAL_FD -> RUBY_IO_EXTERNAL_P
* Expose `rb_io_closed_p`.
* Add `rb_io_mode` to get IO mode.
---------
Co-authored-by: KJ Tsanaktsidis <ktsanaktsidis@zendesk.com>
98637d421d changes the name of
the function. However this function is exported as global,
then change the name to origin one for keeping compatibility.
Remove !USE_RVARGC code
[Feature #19579]
The Variable Width Allocation feature was turned on by default in Ruby
3.2. Since then, we haven't received bug reports or backports to the
non-Variable Width Allocation code paths, so we assume that nobody is
using it. We also don't plan on maintaining the non-Variable Width
Allocation code, so we are going to remove it.
Show native thread's serial on `RUBY_DEBUG_LOG`.
`nt->serial` is also stored into `ruby_nt_serial` if the compiler
supports `RB_THREAD_LOCAL_SPECIFIER`.
This commit implements Objects on Variable Width Allocation. This allows
Objects with more ivars to be embedded (i.e. contents directly follow the
object header) which improves performance through better cache locality.
`RUBY_DEBUG_LOG_FILTER` specified only function names but this
patch also check file names for each log events.
If you specify `file:` or `func:` prefix, it's only filter
file names or func names (otherwize check both).
foo
# show log when file or func names are mached with foo
func:foo
# show log when func name matches foo
file:foo
# show log when file name matches foo
-file:foo,func:bar
# show log when file name does not contains foo
# and func name matches bar
`rb_thread_t::serial` is auto-incremented serial number for
threads and it can overflow, it means the serial is not a ID
for each thread, it is only for debug print.
`RUBY_DEBUG_LOG` shows this information.
Also skip EC related information if EC is NULL. This patch
enable to use `RUBY_DEBUG_LOG` without setup EC.
`USE_RUBY_DEBUG_LOG` was only defined when `RUBY_DEVEL` is defined.
This patch removes this dependency (`USE_RUBY_DEBUG_LOG` is defined
independently from `RUBY_DEVEL`).
Do not commit a patch which enables `USE_RUBY_DEBUG_LOG`.
Without this patch, Ruby doesn't show ractor's information when
there is only 1 ractor. However it is hard to read the log when
some ractors are created and terminated. This patch makes to keep
showing ractor's information on multi-ractor mode.
To make some kind of Ractor related extensions, some functions
should be exposed.
* include/ruby/thread_native.h
* rb_native_mutex_*
* rb_native_cond_*
* include/ruby/ractor.h
* RB_OBJ_SHAREABLE_P(obj)
* rb_ractor_shareable_p(obj)
* rb_ractor_std*()
* rb_cRactor
and rm ractor_pub.h
and rename srcdir/ractor.h to srcdir/ractor_core.h
(to avoid conflict with include/ruby/ractor.h)
This commit introduces Ractor mechanism to run Ruby program in
parallel. See doc/ractor.md for more details about Ractor.
See ticket [Feature #17100] to see the implementation details
and discussions.
[Feature #17100]
This commit does not complete the implementation. You can find
many bugs on using Ractor. Also the specification will be changed
so that this feature is experimental. You will see a warning when
you make the first Ractor with `Ractor.new`.
I hope this feature can help programmers from thread-safety issues.
Old gcc (< 5 maybe?) warns that this variable is not initialized:
debug.c: In function 'ruby_debug_log':
debug.c:441:13: warning: 'r' may be used uninitialized in this function [-Wmaybe-uninitialized]
if (r && len < MAX_DEBUG_LOG_MESSAGE_LEN) {
^
I don't know if that is true, but adding "= 0" here must harm no one.
Now you can specify multiple filters for RUBY_DEBUG_LOG output
by RUBY_DEBUG_LOG_FILTER=a,b,c (in this case, logs that the
function name contains a, b or c).
* RUBY_DEBUG_LOG: Logging debug information mechanism
This feature provides a mechanism to store logging information
to a file, stderr or memory space with simple macros.
The following information will be stored.
* (1) __FILE__, __LINE__ in C
* (2) __FILE__, __LINE__ in Ruby
* (3) __func__ in C (message title)
* (4) given string with sprintf format
* (5) Thread number (if multiple threads are running)
This feature is enabled only USE_RUBY_DEBUG_LOG is enabled.
Release version should not enable it.
Running with the `RUBY_DEBUG_LOG` environment variable enables
this feature.
# logging into a file
RUBY_DEBUG_LOG=/path/to/file STDERR
# logging into STDERR
RUBY_DEBUG_LOG=stderr
# logging into memory space (check with a debugger)
# It will help if the timing is important.
RUBY_DEBUG_LOG=mem
RUBY_DEBUG_LOG_FILTER environment variable can specify the fileter string.
If "(3) __func__ in C (message title)" contains the specified string, the
infomation will be stored (example: RUBY_DEBUG_LOG_FILTER=str will enable
only on str related information).
In a MRI source code, you can use the following macros:
* RUBY_DEBUG_LOG(fmt, ...): Above (1) to (4) will be logged.
* RUBY_DEBUG_LOG2(file, line, fmt, ...):
Same as RUBY_DEBUG_LOG(), but (1) will be replaced with given file, line.
Now, rb_call_info contains how to call the method with tuple of
(mid, orig_argc, flags, kwarg). Most of cases, kwarg == NULL and
mid+argc+flags only requires 64bits. So this patch packed
rb_call_info to VALUE (1 word) on such cases. If we can not
represent it in VALUE, then use imemo_callinfo which contains
conventional callinfo (rb_callinfo, renamed from rb_call_info).
iseq->body->ci_kw_size is removed because all of callinfo is VALUE
size (packed ci or a pointer to imemo_callinfo).
To access ci information, we need to use these functions:
vm_ci_mid(ci), _flag(ci), _argc(ci), _kwarg(ci).
struct rb_call_info_kw_arg is renamed to rb_callinfo_kwarg.
rb_funcallv_with_cc() and rb_method_basic_definition_p_with_cc()
is temporary removed because cd->ci should be marked.
Peter Zhu
Yusuke Endoh
S.H
yui-knk
Koichi Sasada
Samuel Williams
NARUSE, Yui
Nobuyoshi Nakada
John Bampton
Matt Valentine-House
Takashi Kokubun
卜部昌平