Constantly forking a single-threaded process in a loop leads to
a memory leak when using POSIX timers. This fixes the leak for
GNU/Linux systems running glibc.
v2: disarm before timer_delete
v3: ubf_timer_arm prevents double-arming
This unreverts r66291 / commit ab73ef6b70
Example Linux-only reproduction may be found in:
r66290 / commit 043047a8fd
Note: FreeBSD 11.2 still leaks, I'm not sure why, yet.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66413 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We don't use ubf_select after r65495 / 5de7b3b4f2
("thread_pthread.c (native_ppoll_sleep): new eventfd (or pipe) for ubf"),
so we don't need to unregister the thread from the ubf list.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Uncontended GVL waitqueue could mean a single CPU setup where
threads are starved and can't even insert themselves into our
waitqueue. So we force other threads to run upon releasing
the GVL in an uncontended state, in the hope that we can
avoid entering the slow path of ppoll and similar syscalls.
This should prevent test/ruby/test_thread.rb::test_signal_at_join
timeout problems on our single CPU FreeBSD CI machine.
[ruby-core:90417] [Bug #15398]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66381 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts r66290 / commit 043047a8fd.
Still fails on CI, and I'm not able to reproduce the failure, either :<
http://ci.rvm.jp/results/trunk-test@ruby-sky1/1508228
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66291 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Constantly forking a single-threaded process in a loop leads to
a memory leak when using POSIX timers.
v2: disarm before timer_delete
==> fork_leapfrog.rb <==
require 'io/wait'
Dir.chdir '/proc'
prev = 0
loop do
pid = fork
exit!(0) if pid
# show the number of 4K pages used (Linux-only)
n = File.read("#$$/statm").split(-' ')[1].to_i
if n > prev
puts "#{prev} => #{n}"
prev = n
end
# since Ctrl-C from a terminal can't stop this loop,
# allow the user to just hit any key to stop
break if STDIN.wait(0)
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Constantly forking a single-threaded process in a loop leads to
a memory leak when using POSIX timers.
==> fork_leapfrog.rb <==
require 'io/wait'
Dir.chdir '/proc'
prev = 0
loop do
pid = fork
exit!(0) if pid
# show the number of 4K pages used (Linux-only)
n = File.read("#$$/statm").split(-' ')[1].to_i
if n > prev
puts "#{prev} => #{n}"
prev = n
end
# since Ctrl-C from a terminal can't stop this loop,
# allow the user to just hit any key to stop
break if STDIN.wait(0)
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66288 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Negative-numbered FDs are never valid FDs on POSIX-like
platforms, and we initialize our self-pipes/eventfd values
to "-1", so stop treating -1 as a reserved FD if our system
is too low on resources to allocate FDs at startup.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65832 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I already forgot why we needed to jump through such hoops :x
[ruby-core:88102]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65648 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* vm_core.h (rb_thread_struct): introduce new fields `invoke_type`
and `invoke_arg`.
There are two types threads: invoking proc (normal Ruby thread
created by `Thread.new do ... end`) and invoking func, created
by C-API. `invoke_type` shows the types.
* thread.c (thread_do_start): copy `invoke_arg.proc.args` contents
from Array to ALLOCA stack memory if args length is enough small (<8).
We don't need to keep Array and don't need to cancel using transient heap.
* vm.c (thread_mark): For func invoking threads, they can pass (void *)
parameter (rb_thread_t::invoke_arg::func::arg). However, a rubyspec test
(thread_spec.c) passes an Array object and it expect to mark it.
Clealy it is out of scope (misuse of `rb_thread_create` C-API). However,
I'm not sure someone else has such kind of misunderstanding.
So now we mark conservatively this (void *) arg with rb_gc_mark_maybe.
This misuse is found by this error log.
http://ci.rvm.jp/results/trunk-theap-asserts@silicon-docker/1448164
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65622 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Relying on ubf_select + ubf_list for main thread is not
guaranteed to wake a process up as it does not acquire
sigwait_fd and all other threads may be sleeping.
native_cond_sleep and the sigwait_fd path are immune to TOCTOU
issues, but native_ppoll_sleep may have its wakeup stolen
by sigwait_fd sleeper and the RUBY_VM_INTERRUPTED check is
insufficient.
Note: for pthreads platforms without POSIX timers, this becomes
more expensive than Ruby 2.5, as six pipe FDs come into use.
Linux is best off with only two descriptors for eventfd.
[ruby-core:89655]
cf. http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1437559http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1437673
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65495 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
vm->gvl.lock can be held by another thread, we must not wait
on it when called by the MJIT worker thread when it migrates
work to another thread. ubf_select is designed to do retrying
anyways, so it has no obligation to wake up a timer thread.
cf. http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1437880
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65465 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* configure.ac: introduce new configure option `--enable-mjit` and
`--disable-mjit`. Default is "enable".
`--disable-mjit` disables all of MJIT features so that `ruby --jit`
can't enable MJIT.
This option affect a macro `USE_MJIT`.
This change remove `--enable/disable-install-mjit-header` option.
* Makefile.in: introduce the `ENABLE_MJIT` variable.
* common.mk: use `ENABLE_MJIT` option.
* internal.h: respect `USE_MJIT`. Same as other *.c, *.h.
* test/ruby/test_jit.rb: check `ENABLE_MJIT` key of rbconfg.rb.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65204 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_hrtime_t is a more pleasant type to use and this can make
future changes around sleeping/scheduling easier.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65182 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`acquired' was an old boolean variable, but nowadays it is a
rb_thread_t pointer; "gvl.owner" seems like a more appropriate
name. And document the contended path including waitq, timer,
and timer_err.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64581 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Introduce a new rb_thread_sleep_interruptible that does not
execute interrupts before sleeping. Skipping the interrupt
check before sleep is required for out-of-GVL ruby_waitpid_all
to function properly when setting waitpid_state.ret
Now that ubf_select can be called by the gvl.timer thread
without recursive locking gvl.lock, we can safely use
rb_threadptr_interrupt to deal with waking up sleeping
processes,
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This simplifies the locking logic somewhat.
While we're at it, designate_timer_thread is worthless in
ubf_select because gvl_acquire_common already guarantees there
is a gvl.timer if gvl->waitq is populated.
In the future (for auto-fiber), this will allow using
th->unblock.func for rb_waitpid callers (via rb_sigchld_handler).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64575 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We need to ensure Signal.trap handlers can function if the main
thread is sleeping after a subthread has grabbed sigwait_fd,
but later exited.
Consider the following timeline:
main_thread sub-thread
-----------------------------------------
Signal.trap() { ... }
get sigwait_fd
ppoll on sigwait_fd
native_cond_sleep
(via pthread_cond_wait)
ppoll times-out
put sigwait_fd
sub-thread exits
only thread alive
SIGNAL HITS
The problem is pthread_cond_wait cannot return EINTR,
so we can never run the Signal.trap handler. So we
will avoid using native_cond_sleep in the main thread
and always use ppoll to sleep when in the main thread.
This can guarantee the main thread remains aware of
signals; even if it cannot safely read off sigwait_fd
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64538 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Relying on "struct timespec" was too annoying API-wise and
used more stack space. "double" was a bit wacky w.r.t rounding
in the past, so now we'll switch to using a 64-bit type.
Unsigned 64-bit integer is able to give us over nearly 585
years of range with nanoseconds. This range is good enough
for the Linux kernel internal time representation, so it
ought to be good enough for us.
This reduces the stack usage of functions while GVL is held
(and thus subject to marking) on x86-64 Linux (with ppoll):
rb_wait_for_single_fd 120 => 104
do_select 120 => 88
[ruby-core:88582] [Misc #15014]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64533 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Based on r64478, any regular user creating more than 1024 pipes
on Linux will end up with tiny pipes with only a single page
capacity. So avoid wasting user resources and use lighter
eventfd on Linux.
[ruby-core:88563] [Misc #15011]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64527 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
TIMER_THREAD_CREATED_P already checks that pid, and glibc 2.25+
no longer caches getpid(2).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64524 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It's possible for the ubf_list_head to be populated with dead
threads at fork or the ubf_list_lock to be held, so reinitialize
both at startup.
And while we're at it, use a static initializer at startup
to save a library call and kill some ifdef.
[ruby-core:88578] [Bug #15013]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64485 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This matches the behavior of old timer thread more closely
and seems to fix [Bug #14999] when limited to a single CPU.
I cannot reproduce the error on a multi-core system unless
I use schedtool to force affinity to a single CPU:
schedtool -a 0x01 -e make test-spec \
MSPECOPT='-R1000 spec/ruby/library/conditionvariable/wait_spec.rb'
While it may be good enough to pass the spec, I don't have
huge degree of confidence in the interrupt handling robustness
under extremely heavy load (these may be ancient bugs, though).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64467 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This is not called in signal handlers, so there's no reason for
it. glibc 2.25+ no longer caches getpid(), so it will cost a
syscall for those users.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64403 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We do not need to rely on SIGVTALRM for non-sighandler wakeups.
This will reduce spurious wakeups in cases where sigwait_fd
is not grabbed again, soon.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
For (rare) blocking functions which are not affected by signals,
we need to call the appropriate unblocking function via
`threadptr_trap_interrupt'
While we're at it, handling waitpid/SIGCHLD from gvl.timer isn't
harmful, here.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64388 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This hopefully clarifies the roles of UBF_TIMER and vm->gvl.timer
[ruby-core:88475] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64377 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Do not waste extra memory for each thread, but make
thread_pthread.c easier-to-follow as a result.
[ruby-core:88475] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64375 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
These functions will not be exported outside of thread_pthread.c
and we need to clarify the timer here is used for ubf and not
timeslice.
[ruby-core:88475] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64373 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This data structure has nothing to do with timers or threads.
[ruby-core:88475] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64372 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Hopefully this makes the code easier-to-follow
[ruby-core:88475] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64371 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I'm not sure what's causing this failure in Solaris and only
on rubyspec, since rb_io_wait_readable is a well-exercised
code path in other places. But maybe using a pthread for
timing (similar to old timer-thread) can solve the issue.
cf. http://rubyci.s3.amazonaws.com/unstable11s/ruby-trunk/log/20180814T042506Z.fail.html.gz
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64357 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
timer_create does not seem to support CLOCK_MONOTONIC on Solaris,
and CLOCK_HIRES seems like it could fail with insufficient permissions:
https://docs.oracle.com/cd/E86824_01/html/E54766/timer-create-3c.html
(Only tested on Linux and FreeBSD)
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64356 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It looks like I forgot to account for a situation involving 3
threads.
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This closes race condition where GVL is uncontended and a thread
receives a signal immediately before calling the blocking
function when releasing GVL:
1) check interrupts
2) release GVL
3) blocking function
If signal fires after 1) but before 3), that thread may never
wake up if GVL is uncontended
We also need to wakeup the ubf_list unconditionally on
gvl_yield; because two threads can be yielding to each other
while waiting on IO#close while waiting on threads in IO#read or
IO#gets.
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64353 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit 194a6a2c68 (r64203).
Race conditions which caused the original reversion will be fixed
in the subsequent commit.
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64352 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We don't use pthreads cancellation ourselves and it's painful to
use correctly. Any cancelled threads would break
vm->living_threads, GVL, thread_sync.c, autoload, etc...
So don't bother caring; because we can't stop rogue extensions
from completely breaking the VM in other ways, either.
[ruby-core:88282] [Misc #14962]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
By holding into sigwait_fd until after we acquire GVL, we can
hit the faster native_cond_sleep path instead of ppoll when
another thread wants to start sleeping. ppoll-ing on sigwait_fd
isn't really useful in program where GVL is contended
This also allows reducing vm->gvl.lock mutex contention on
waitpid sleep migrations.
r64170 this patch
vm_thread_condvar1 0.921 1.356
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64193 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It seems to be unavailable on some platforms including my Android phone.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64166 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We must not allow reentry into ubf_list_head once we delete
ourselves, otherwise we could hang in there forever.
[ruby-core:88218] [Bug #14945]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64134 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Not having contention for GVL could mean everybody else is stuck
in blocking region without GVL, so we kick the ubf list in that
case.
I expect this to fix test_thread_fd_close timeout:
http://ci.rvm.jp/results/trunk-test@ruby-sky3/1173398
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64133 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* thread_pthread.c (USE_NATIVE_SLEEP_COND): revised wrongly removed
line with the ifndef guard.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Maybe some platforms have strange condition variable implementations
which have a "memory" of which mutexes they're associated with.
In any case, it makes documentation easier even on GNU/Linux and
FreeBSD.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64124 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We need to be able to perform periodic ubf_list wakeups when a
thread is sleeping and waiting on signals.
[ruby-core:88088] [Misc #14937] [Bug #5343]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64115 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It's possible for another thread to take vm->gvl.lock
during gvl_release at the end of thread_start_func_2
during VM shutdown, at least.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64114 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We can't always designate a timer thread, so any sleepers must
also perform ubf wakeups. Note: a similar change needs to be
made for rb_thread_fd_select and rb_wait_for_single_fd.
[ruby-core:88088] [Misc #14937] [Bug #5343]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64111 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
thread_pthread.c relies on ppoll for rb_sigwait_sleep, so ensure
the compatibility wrapper is available for it.
[Bug #14950]
Reported-by: SHIBATA Hiroshi <hsbt@ruby-lang.org>
Reported-by: Greg L <Greg.mpls@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64110 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We still need to to designate a timer thread after registering target
thread for the ubf list.
Oops :x
Note: I was never able to reproduce
test/thread/test_queue.rb::test_thr_kill failures on my on
Debian machines.
[ruby-core:88088] [Misc #14937] [Bug #5343]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64108 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
To reduce resource use and reduce CI failure; remove
timer-thread. Single-threaded Ruby processes (including forked
children) will never see extra thread overhead. This prevents
glibc and jemalloc from going into multi-threaded mode and
initializing locks or causing fragmentation via arena explosion.
The GVL is implements its own wait-queue as a ccan/list to
permit controlling wakeup order. Timeslice under contention is
handled by a designated timer thread (similar to choosing a
"patrol_thread" for current deadlock checking).
There is only one self-pipe, now, as wakeups for timeslice are
done independently using condition variables. This reduces FD
pressure slightly.
Signal handling is handled directly by a Ruby Thread (instead
of timer-thread) by exposing signal self-pipe to callers of
rb_thread_fd_select, native_sleep, rb_wait_for_single_fd, etc...
Acquiring, using, and releasing the self-pipe is exposed via 4
new internal functions:
1) rb_sigwait_fd_get - exclusively acquire timer_thread_pipe.normal[0]
2) rb_sigwait_fd_sleep - sleep and wait for signal (and no other FDs)
3) rb_sigwait_fd_put - release acquired result from rb_sigwait_fd_get
4) rb_sigwait_fd_migrate - migrate signal handling to another thread
after calling rb_sigwait_fd_put.
rb_sigwait_fd_migrate is necessary for waitpid callers because
only one thread can wait on self-pipe at a time, otherwise a
deadlock will occur if threads fight over the self-pipe.
TRAP_INTERRUPT_MASK is now set for the main thread directly in
signal handler via rb_thread_wakeup_timer_thread.
Originally, I wanted to use POSIX timers
(timer_create/timer_settime) for this. Unfortunately, this
proved unfeasible as Mutex#sleep resumes on spurious wakeups and
test/thread/test_cv.rb::test_condvar_timed_wait failed. Using
pthread_sigmask to mask out SIGVTALRM fixed that test, but
test/fiddle/test_function.rb::test_nogvl_poll proved there'd be
some unavoidable (and frequent) incompatibilities from that
approach.
Finally, this allows us to drop thread_destruct_lock and
interrupt current ec directly.
We don't need to rely on vm->thread_destruct_lock or a coherent
vm->running_thread on any platform. Separate timer-thread for
time slice and signal handling is relegated to thread_win32.c,
now.
[ruby-core:88088] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64107 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Otherwise, an altstack may live past ObjectSpace destruction
and xfree-ing the altstack will segfault.
[ruby-core:85621] [Feature #14487]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64102 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There's actually no need to close the pipes used by the
sleepy timer thread before forking, only to stop the timer
thread itself.
Instead, we only close the parent pipes in the child process,
either via close-on-exec flag or when reinitializing the timer
thread.
This change will be necessary when we allow
rb_wait_for_single_fd and rb_thread_fd_select to wait on the
timer_thread_pipe.normal[0] directly and eliminate timer thread.
I don't anticipate compatibility problems with this change
alone.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63960 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
timer-thread will continue to be supported, but future
"timer" implementation may not be a thread.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63948 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
pthread_atfork is not idempotent and repeatedly calling it
causes it to register the same hook repeatedly; leading to
unbound memory growth.
Ruby already has a (confusing-named) internal API for to call
in the forked child process: rb_thread_atfork
Call the MJIT child_after_fork hook inside that to prevent
unbound growth with the following loop:
loop do
RubyVM::MJIT.pause
RubyVM::MJIT.resume
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63884 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Reading win32/win32.c waitpid implementation, maybe waitpid(-1, ...)
on that platform will never conflict with mjit use of waitpid.
In any case, I've added WAITPID_USE_SIGCHLD macro to vm_core.h
so it can be easy for Linux/BSD users to test (hopefully!)
win32-compatible code.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The change is unstable on Windows. Please re-commit it when it correctly
supports Windows.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63852 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I love `container_of' for generic data structures, but
in this case it's unnecessary and slightly harder-to-read.
This will make "Timeout in VM" slightly easier-to-read:
https://bugs.ruby-lang.org/issues/14859
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63846 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
mwrap <https://80x24.org/mwrap/> interposes malloc functions and
checks for GVL existence to determine Ruby source locations of
malloc calls. pthread_getattr_np (from get_stack) may call
realloc to get the CPU set size; so when using the thread-cache,
ruby_thread_has_gvl_p() may hit a false positive on reused
threads with lingering rb_thread_t in thread-specific data.
This was causing mwrap to call rb_source_location_cstr() and
crash because it was pointed to a zero ec->cfp->iseq.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63836 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Some systems lack SIGCHLD or have incomplete SIGCHLD
implementations. So enable polling mode for them.
[ruby-core:87705] [Bug #14867]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63795 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Use a global SIGCHLD handler to guard all callers of rb_waitpid.
To work safely with multi-threaded programs, we introduce a
VM-wide waitpid_lock to be acquired BEFORE fork/vfork spawns the
process. This is to be combined with the new ruby_waitpid_locked
function used by mjit.c in a non-Ruby thread.
Ruby-level SIGCHLD handlers registered with Signal.trap(:CHLD)
continues to work as before and there should be no regressions
in any existing use cases.
Splitting the wait queues for PID > 0 and groups (PID <= 0)
ensures we favor PID > 0 callers.
The disabling of SIGCHLD in rb_f_system is longer necessary,
as we use deferred signal handling and no longer make ANY
blocking waitpid syscalls in other threads which could "beat"
the waitpid call made by rb_f_system.
We prevent SIGCHLD from firing in normal Ruby Threads and only
enable it in the timer-thread, to prevent spurious wakeups
from in test/-ext-/gvl/test_last_thread.rb with MJIT enabled.
I've tried to guard as much of the code for RUBY_SIGCHLD==0
using C "if" statements rather than CPP "#if" so to reduce
the likelyhood of portability problems as the compiler will
see more code.
We also work to suppress false-positives from
Process.wait(-1, Process::WNOHANG) to quiets warnings from
spec/ruby/core/process/wait2_spec.rb with MJIT enabled.
Lastly, we must implement rb_grantpt for ext/pty. We need a
MJIT-compatible way of supporting grantpt(3) which may spawn
the `pt_chown' binary and call waitpid(2) on it.
[ruby-core:87605] [Ruby trunk Bug#14867]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63758 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
thread_pthread.c: Drop pthread_attr_setscope usage. It seems that,
at least on Linux and macOS, PTHREAD_SCOPE_PROCESS is not supported
and thus PTHREAD_SCOPE_SYSTEM should be used by default.
Let's just stop calling this until we find some platform that needs
`pthread_attr_setscope(&attr, PTHREAD_SCOPE_SYSTEM)`.
[Misc #14854]
From: fd0 (Daisuke Fujimura)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63730 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It is unnecessary to clear unblock.arg once unblock.func is
cleared, and unblock_function_clear in thread.c doesn't
touch it, either.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63724 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This fixes bootstraptest/test_fork.rb for systems with
sleepy timer thread disabled.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63709 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
"gvl.waiting" is volatile, so the compiler won't perform
these optimizations for us.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63683 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I guess everybody has poll() and fcntl() nowadays, as
the non-sleepy timer thread build has been broken for
years, now.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63682 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Some operating systems will work without calling
pthread_condattr_init, but some won't (such as OpenBSD). Prior
to r63238, pthread_condattr_init was always called before
calling pthread_condattr_setclock.
From: Jeremy Evans <code@jeremyevans.net>
[ruby-core:87345] [Ruby trunk Bug#14807]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63548 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I suspect GC may free the rb_thread_t (th) pointer by the time
we call register_cached_thread_and_wait.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Since r62466 ("thread_pthread.c: shorten and fix thread cache implementation"),
our thread cache is no longer buggy with programs using fork.
This makes significant improvements in vm_thread_alive_check1
and vm_thread_create_join benchmarks and does not introduce
regressions.
Unlike old thread cache, I've changed the cache to only last 3
seconds since per-thread setup in most programs rarely takes
more than a few milliseconds to re-establish things like network
connections. This is configurable by changing the THREAD_CACHE_TIME
variable.
I hope this allows users to simplify their code by removing the
need for thread pools in many cases.
vm_thread_alive_check1 10.872 0.150
vm_thread_close 1.988 2.027
vm_thread_condvar1 0.751 0.767
vm_thread_condvar2 0.744 0.752
vm_thread_create_join 5.296 2.343
vm_thread_mutex1 1.911 1.892
vm_thread_mutex2 1.902 1.896
vm_thread_mutex3 2.389 2.313
vm_thread_pass 0.271 0.272
vm_thread_pass_flood 0.175 0.179
vm_thread_pipe 0.460 0.436
vm_thread_queue 0.453 0.446
vm_thread_sized_queue 0.547 0.547
vm_thread_sized_queue2 1.417 1.413
vm_thread_sized_queue3 1.410 1.426
vm_thread_sized_queue4 0.787 0.791
Speedup ratio: compare with the result of `trunk' (greater is better)
name built
vm_thread_alive_check1 72.456
vm_thread_close 0.981
vm_thread_condvar1 0.979
vm_thread_condvar2 0.990
vm_thread_create_join 2.260
vm_thread_mutex1 1.010
vm_thread_mutex2 1.003
vm_thread_mutex3 1.033
vm_thread_pass 0.994
vm_thread_pass_flood 0.980
vm_thread_pipe 1.055
vm_thread_queue 1.016
vm_thread_sized_queue 0.999
vm_thread_sized_queue2 1.003
vm_thread_sized_queue3 0.989
vm_thread_sized_queue4 0.995
[ruby-core:87030] [Feature #14757]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63498 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This allows native_sleep to use less stack (80 -> 64 bytes on
x86-64) for GVL_UNLOCK_BEGIN/END. For future APIs, we will pass
`ec` or `th` around anyways, so the BLOCKING_REGION change
should be beneficial in the future.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63448 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* thread_pthread.c (Init_native_thread): fallback to the default
CLOCK_REALTIME when failed to set to CLOCK_MONOTONIC, e.g. on
Solaris. [Misc #14497]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63244 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There's no reason to use CLOCK_REALTIME for any condvars in Ruby.
Indeed, we initialized all condvars with RB_CONDATTR_CLOCK_MONOTONIC
anyway; so simplify our code and reduce ifdefs.
[ruby-core:85639] [Misc #14497]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63238 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Instead of allocating and registering the altstack in different
places, do it together to reduce code and improve readability.
When thread cache is enabled, storing altstack in rb_thread_t
is wasteful and we may reuse altstack in the same pthread.
This also lets us clearly allow use of xmalloc to allow GC to
recover from ENOMEM.
[ruby-core:85621] [Feature #14487]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63213 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The previous logic would overwrite the error message, replacing the message with the `fd` number.
This tiny update will print the message in full.
(I'm trying to debug an issue with the timer thread on my machine and the lack of error messages makes it really hard).
[Fix GH-1829]
From: Bo <bo@bowild.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62661 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I noticed this because of https://bugs.ruby-lang.org/issues/14494
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62484 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
No need to hold a lock while destroying a condition variable.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62476 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This should be slightly cheaper on NPTL as it does not rely on
atomics to set pd->joinid. We already use pthread_attr_setdetachstate,
so it won't introduce new problems by using a function we did not
use before.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62475 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Update to use ccan/list for constant-time delete on expiry and
avoid malloc. We must also initialize th->thread_id upon thread
reuse so Thread#name= works immediately upon thread creation.
We must also reinitialize the cache mutex and list_head on
fork like we do with GVL and timer thread mutexes.
While we're at it, use monotonic clock for timeout to avoid
system time changes.
"make exam TESTS='-x test_time_tz'" passes with USE_THREAD_CACHE
enabled (but remains off, here).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62466 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Rely on getclockofday for CLOCK_MONOTONIC, avoid needless
variables, and rely on overflow protection from timespec_add
instead of coding our own.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62459 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* thread.c (timeval_for): tv_usec is suseconds_t which may be
smaller than long.
* thread_pthread.c (native_cond_timeout): ret is now used in
CLOCK_MONOTONIC case only.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62275 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This results in fewer conversion on common modern systems with
support for clock_gettime, pthread_cond_timedwait and ppoll.
gettimeofday is declared obsolete by POSIX.1-2008, so it is yet
another reason to move away from it. This also appears to result
in the reduction of compatibility code required for dealing
with inconsistent implementations of "struct timeval".tv_sec
In the future, this will also result in fewer conversions for
kqueue and pselect if we elect to use them.
[ruby-core:85416] [Feature #14452]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62272 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* thread_pthread.c (rb_thread_create_mjit_thread): cast
worker_func pointer to void pointer inside. adjusted to the
declaration in mjit.c and the definition in thread_win32.c.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62242 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
svn
normal
ko1
duerst
mame
nobu
naruse
k0kubun