The timer function used on windows system set timer interrupt
flag of current main ractor's executing ec and thread can detect
the end of time slice. However, to set all ec->interrupt_flag for
all running ractors, it is requires to synchronize with other ractors.
However, timer thread can not acquire the ractor-wide lock because
of some limitation.
To solve this issue, this patch introduces USE_VM_CLOCK compile option
to introduce rb_vm_t::clock. This clock will be incremented by the
timer thread and each thread can check the incrementing by comparison
with previous checked clock. At last, on windows platform this patch
introduces some overhead, but I think there is no critical performance
issue because of this modification.
To access TLS, it is faster to use language TLS specifier instead
of using pthread_get/setspecific functions.
Original proposal is: Use native thread locals. #3665
This commit introduces Ractor mechanism to run Ruby program in
parallel. See doc/ractor.md for more details about Ractor.
See ticket [Feature #17100] to see the implementation details
and discussions.
[Feature #17100]
This commit does not complete the implementation. You can find
many bugs on using Ractor. Also the specification will be changed
so that this feature is experimental. You will see a warning when
you make the first Ractor with `Ractor.new`.
I hope this feature can help programmers from thread-safety issues.
A new (not-initialized-yet) pthread attempts to allocate sigaltstack by
using xmalloc. It may cause GC, but because the thread is not
initialized yet, ruby_native_thread_p() returns false, which leads to
"[FATAL] failed to allocate memory" and exit.
In fact, we can observe the error message in the log of OpenBSD CI:
https://rubyci.org/logs/rubyci.s3.amazonaws.com/openbsd-current/ruby-master/log/20200306T083005Z.log.html.gz
This changeset allocates sigaltstack before pthread is created.
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct. This commit deletes ANYARGS from
rb_thread_create, which seems very safe to do.
When using UBF_TIMER_PTHREAD (the UBF handler on OpenBSD), the
timer_pthread_fn function will not signal the main thread with
SIGVTALRM in cases where timer_pthread is armed before
consume_communication_pipe is called. This is because
consume_communication_pipe will unarm the timer.
Fix this by checking the return value of consume_communication_pipe.
If it returns TRUE and the timer_pthread is disarmed, then signal
the main thread with SIGVTALRM.
On OpenBSD, this fixes TestThread#test_thread_timer_and_interrupt, and
fixes hangs in TestProcess#test_execopts_redirect_open_fifo_interrupt_raise
and TestProcess#test_execopts_redirect_open_fifo_interrupt_print.
It also fixes the use of Ctrl+C/SIGINT in irb on OpenBSD. It does not
cause any test failures on Linux when UBF_TIMER_PTHREAD is forced as
the UBF handler.
Fixes [Bug #15798]
zlib and bignum both contain unblocking functions which are
async-signal-safe and do not require spawning additional
threads.
We can execute those functions directly in signal handlers
without incurring overhead of extra threads, so provide C-API
users the ability to deal with that. Other C-API users may
have similar need.
This flexible API can supercede existing uses of
rb_thread_call_without_gvl and rb_thread_call_without_gvl2 by
introducing a flags argument to control behavior.
Note: this API is NOT finalized. It needs approval from other
committers. I prefer shorter name than previous
rb_thread_call_without_gvl* functions because my eyes requires
big fonts.
[Bug #15499]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66712 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We need another native thread to call some unblocking functions
which aren't RUBY_UBF_IO or RUBY_UBF_PROCESS. Instead of a
permanent thread in <= 2.5, we can now rely on the thread cache
feature to perform interrupts.
[ruby-core:90865] [Bug #15499]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66708 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The following race may happen if ubf_timer_destroy calls
timer_delete before ubf_timer_disarm gets called from
a different thread. Consider the following timelines:
ubf_timer_destroy | ubf_timer_disarm
-------------------------------------+-----------------------------
| CAS(ARM => DISARM)
CAS(DISARM => DEAD) |
timer_delete |
| timer_settime(disarm)
Another option may be to add an intermediate "RTIMER_DISARMING"
state to the transition, but I figure the EINVAL check is
simpler and less intrusive code-wise.
cf. http://ci.rvm.jp/results/trunk-iseq_binary@silicon-docker/1545794
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66457 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Constantly forking a single-threaded process in a loop leads to
a memory leak when using POSIX timers. This fixes the leak for
GNU/Linux systems running glibc.
v2: disarm before timer_delete
v3: ubf_timer_arm prevents double-arming
This unreverts r66291 / commit ab73ef6b70
Example Linux-only reproduction may be found in:
r66290 / commit 043047a8fd
Note: FreeBSD 11.2 still leaks, I'm not sure why, yet.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66413 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We don't use ubf_select after r65495 / 5de7b3b4f2
("thread_pthread.c (native_ppoll_sleep): new eventfd (or pipe) for ubf"),
so we don't need to unregister the thread from the ubf list.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Uncontended GVL waitqueue could mean a single CPU setup where
threads are starved and can't even insert themselves into our
waitqueue. So we force other threads to run upon releasing
the GVL in an uncontended state, in the hope that we can
avoid entering the slow path of ppoll and similar syscalls.
This should prevent test/ruby/test_thread.rb::test_signal_at_join
timeout problems on our single CPU FreeBSD CI machine.
[ruby-core:90417] [Bug #15398]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66381 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts r66290 / commit 043047a8fd.
Still fails on CI, and I'm not able to reproduce the failure, either :<
http://ci.rvm.jp/results/trunk-test@ruby-sky1/1508228
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66291 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Constantly forking a single-threaded process in a loop leads to
a memory leak when using POSIX timers.
v2: disarm before timer_delete
==> fork_leapfrog.rb <==
require 'io/wait'
Dir.chdir '/proc'
prev = 0
loop do
pid = fork
exit!(0) if pid
# show the number of 4K pages used (Linux-only)
n = File.read("#$$/statm").split(-' ')[1].to_i
if n > prev
puts "#{prev} => #{n}"
prev = n
end
# since Ctrl-C from a terminal can't stop this loop,
# allow the user to just hit any key to stop
break if STDIN.wait(0)
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Constantly forking a single-threaded process in a loop leads to
a memory leak when using POSIX timers.
==> fork_leapfrog.rb <==
require 'io/wait'
Dir.chdir '/proc'
prev = 0
loop do
pid = fork
exit!(0) if pid
# show the number of 4K pages used (Linux-only)
n = File.read("#$$/statm").split(-' ')[1].to_i
if n > prev
puts "#{prev} => #{n}"
prev = n
end
# since Ctrl-C from a terminal can't stop this loop,
# allow the user to just hit any key to stop
break if STDIN.wait(0)
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66288 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Negative-numbered FDs are never valid FDs on POSIX-like
platforms, and we initialize our self-pipes/eventfd values
to "-1", so stop treating -1 as a reserved FD if our system
is too low on resources to allocate FDs at startup.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65832 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I already forgot why we needed to jump through such hoops :x
[ruby-core:88102]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65648 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* vm_core.h (rb_thread_struct): introduce new fields `invoke_type`
and `invoke_arg`.
There are two types threads: invoking proc (normal Ruby thread
created by `Thread.new do ... end`) and invoking func, created
by C-API. `invoke_type` shows the types.
* thread.c (thread_do_start): copy `invoke_arg.proc.args` contents
from Array to ALLOCA stack memory if args length is enough small (<8).
We don't need to keep Array and don't need to cancel using transient heap.
* vm.c (thread_mark): For func invoking threads, they can pass (void *)
parameter (rb_thread_t::invoke_arg::func::arg). However, a rubyspec test
(thread_spec.c) passes an Array object and it expect to mark it.
Clealy it is out of scope (misuse of `rb_thread_create` C-API). However,
I'm not sure someone else has such kind of misunderstanding.
So now we mark conservatively this (void *) arg with rb_gc_mark_maybe.
This misuse is found by this error log.
http://ci.rvm.jp/results/trunk-theap-asserts@silicon-docker/1448164
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65622 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Relying on ubf_select + ubf_list for main thread is not
guaranteed to wake a process up as it does not acquire
sigwait_fd and all other threads may be sleeping.
native_cond_sleep and the sigwait_fd path are immune to TOCTOU
issues, but native_ppoll_sleep may have its wakeup stolen
by sigwait_fd sleeper and the RUBY_VM_INTERRUPTED check is
insufficient.
Note: for pthreads platforms without POSIX timers, this becomes
more expensive than Ruby 2.5, as six pipe FDs come into use.
Linux is best off with only two descriptors for eventfd.
[ruby-core:89655]
cf. http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1437559http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1437673
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65495 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
vm->gvl.lock can be held by another thread, we must not wait
on it when called by the MJIT worker thread when it migrates
work to another thread. ubf_select is designed to do retrying
anyways, so it has no obligation to wake up a timer thread.
cf. http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1437880
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65465 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* configure.ac: introduce new configure option `--enable-mjit` and
`--disable-mjit`. Default is "enable".
`--disable-mjit` disables all of MJIT features so that `ruby --jit`
can't enable MJIT.
This option affect a macro `USE_MJIT`.
This change remove `--enable/disable-install-mjit-header` option.
* Makefile.in: introduce the `ENABLE_MJIT` variable.
* common.mk: use `ENABLE_MJIT` option.
* internal.h: respect `USE_MJIT`. Same as other *.c, *.h.
* test/ruby/test_jit.rb: check `ENABLE_MJIT` key of rbconfg.rb.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65204 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_hrtime_t is a more pleasant type to use and this can make
future changes around sleeping/scheduling easier.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65182 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`acquired' was an old boolean variable, but nowadays it is a
rb_thread_t pointer; "gvl.owner" seems like a more appropriate
name. And document the contended path including waitq, timer,
and timer_err.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64581 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Nobuyoshi Nakada
Samuel Williams
NARUSE, Yui
Benoit Daloze
Koichi Sasada
Yusuke Endoh
卜部昌平
git
Jeremy Evans
normal
ko1
duerst