and functions to clarify the intention and make sure it's not used in a
surprising way (like using 2, 3, ... other than 0, 1 even while it seems
to be a boolean).
This is a retry of r66775. It included some typos...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66778 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit bb1a1aeab0.
We hit something on ci.rvm.jp, reverting until investigation is done.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66776 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and functions to clarify the intention and make sure it's not used in a
surprising way (like using 2, 3, ... other than 0, 1 even while it seems
to be a boolean).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66775 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This affects test/ruby/test_process.rb (test_execopt_env_path).
Since MJIT uses vfork+execve in a separate thread, there can be
small window in-between vfork and execve where tmp_script.cmd is
held open by the vforked child. vfork only pauses the MJIT
thread, not any Ruby Threads, so our call to Process.spawn will
hit ETXTBUSY in that window unless we fork.
main thread | MJIT thread
----------------------------------------------------
fd = open(tmp) | |
| vfork for CC | CC running
write | | ---------------
fchmod | | sees "fd" here
close(fd) | |
Process.spawn called | |
vfork (spawn)| (new process) | |
| execve => TXTBUSY | |
| | | execve (FD_CLOEXEC on fd)
| | vfork returns |
Holding the waitpid_lock whenever we intend to spawn a process
prevents the MJIT thread from spawning a process while we are
spawning in Ruby-land.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66171 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
io.c has not used it since r36229, and we can re-export
it if we need it at another time.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66157 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
TypedData_* macros hide assignments and are confusing and too
long for users of giant fonts.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66156 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
because JIT-ed code may still be on stack at this time, unlike
in ruby_cleanup().
This hopes to fix: (take 2)
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480207
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480173
It tries to print C backtrace but fails. And core file on the server
seems to be stopping on the irrelevant place due to its own signal
handler for the dump.
And I failed to reproduce this SEGV on my machine.
I don't know why it's broken, so let me try this change to investigate
the reason of SEGV.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65997 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to prevent ruby from leaving MJIT-related files.
test_jit.rb: add a test to prevent that
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65994 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
All normal Ruby IO methods (IO#read, IO#gets, IO#write, ...) are
all capable of appearing to be "blocking" when presented with a
file description with the O_NONBLOCK flag set; so there is
little risk of incompatibility within Ruby-using programs.
The biggest compatibility risk is when spawning external
programs. As a result, stdin, stdout, and stderr are now always
made blocking before exec-family calls.
This change will make an event-oriented MJIT usable if it is
waiting on pipes on POSIX_like platforms.
It is ALSO necessary to take advantage of (proposed lightweight
concurrency (aka "auto-Fiber") or any similar proposal for
network concurrency: https://bugs.ruby-lang.org/issues/13618
Named-pipe (FIFO) are NOT yet non-blocking by default since
they are rarely-used and may introduce compatibility problems
and extra syscall overhead for a common path.
Please revert this commit if there are problems and if I am afk
since I am afk a lot, lately.
[ruby-core:89950] [Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65922 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
while child handler is disabled.
trying to fix [Bug #15320]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65817 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
it didn't work.
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468677
and skips broken tests for now. But this issue should be fixed soon.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65814 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to prevent from proceeding one for MJIT while it's not safe yet.
By that situation, MJIT worker could be waiting for compiler process forever
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468033
[Bug #15320]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65807 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
by launching MJIT worker thread in child Ruby process.
See the comment before `mjit_child_after_fork` for details.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65785 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c, internal.h: support theap for small Hash.
Introduce RHASH_ARRAY (li_table) besides st_table and small Hash
(<=8 entries) are managed by an array data structure.
This array data can be managed by theap.
If st_table is needed, then converting array data to st_table data.
For st_table using code, we prepare "stlike" APIs which accepts hash value
and are very similar to st_ APIs.
This work is based on the GSoC achievement
by tacinight <tacingiht@gmail.com> and refined by ko1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65454 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Following how mjit_worker.c currently works, rb_f_system
now ensures the VM-wide waitpid lists is locked before
creating a new process via fork/vfork.
This ensures other rb_waitpid callers cannot steal work and
there are no possible race conditions from toggling
ruby_nocldwait without the use of atomics.
This sets us up for implementing MJIT process management
logic using normal Ruby APIs prepares us for VM-wide
asynchronous/event-base waitpid which can allow MJIT to
work without worker threads.
Take 2: set waitpid_state.pid on platforms w/o fork.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65437 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Following how mjit_worker.c currently works, rb_f_system
now ensures the VM-wide waitpid lists is locked before
creating a new process via fork/vfork.
This ensures other rb_waitpid callers cannot steal work and
there are no possible race conditions from toggling
ruby_nocldwait without the use of atomics.
This sets us up for implementing MJIT process management
logic using normal Ruby APIs prepares us for VM-wide
asynchronous/event-base waitpid which can allow MJIT to
work without worker threads.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65434 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_hrtime_t is a more pleasant type to use and this can make
future changes around sleeping/scheduling easier.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65182 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (check_exec_redirect): use RARRAY_AREF() instead of
using RARRAY_PTR().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This is the safer option, as there seems to be cases where checking
waitpid_state.ret is insufficient in ensure. I'm not 100% sure
why this is, but this change was required for my work-in-progress
Thread::Light patch series, too...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64782 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Oddly, all existing test cases passed multiple times
before this patch (even with --jit-wait), so this seems
like a difficult failure to prove.
Fixes: r64576 ("process.c: simplify SIGCHLD-based waitpid")
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64577 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Introduce a new rb_thread_sleep_interruptible that does not
execute interrupts before sleeping. Skipping the interrupt
check before sleep is required for out-of-GVL ruby_waitpid_all
to function properly when setting waitpid_state.ret
Now that ubf_select can be called by the gvl.timer thread
without recursive locking gvl.lock, we can safely use
rb_threadptr_interrupt to deal with waking up sleeping
processes,
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It is identical to do_waitpid, and the win32 version will not
be needed for MJIT (since win32 does not suffer from the
waitpid(-1, ...) conflict where waits can get stolen.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64541 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This re-reverts commit r64447.
The issue was machine side problem.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (ruby_waitpid_all): nothing to do unless SIGCHLD is
available.
* signal.c (ruby_nocldwait): used only if SIGCHLD is available.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64407 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Arbitrarily closing file descriptors on exec breaks use cases
where a Ruby process sets up a descriptor for non-Ruby children
to use. For example, the "rake foo" target may spawn any number
of subprocesses (Ruby or not) which depends on parsing the "FOO"
environment variable for out_fd:99 and writing to foo.out
FOO=out_fd:99 rake foo 99>>foo.out
Unfortunately, this introduced one incompatibility in
test/lib/test/unit.rb and it now requires explicitly setting
IO#close_on_exec=true
[ruby-core:88007] [Misc #14907]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64399 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We do not need to rely on SIGVTALRM for non-sighandler wakeups.
This will reduce spurious wakeups in cases where sigwait_fd
is not grabbed again, soon.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit 194a6a2c68 (r64203).
Race conditions which caused the original reversion will be fixed
in the subsequent commit.
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64352 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
getgroups(2) may return a GID list that includes duplicated GIDs.
The behavior is totaly depends on what OS is used.
This commit fixes the example of Process.groups so that the example
is independent of this OS-dependent features.
Additonaly, this commit adds the description of such system-dependent
characteristics of Process.groups.
[ruby-dev:50603] [Bug #14969]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64265 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
by sharing it with vm.c in internal.h.
vm.c: ditto
internal.h: ditto
mjit.h: share more.
mjit.c: make sure the third arguemnt is not used
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64253 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
w->cond may be changed without our knowledge in waitpid_nogvl
without th->interrupt_lock
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64201 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Once a thread has acquired sigwait_fd, hold onto it until
waitpid is complete. This prevents unnecessary migration
and atomic operations.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64200 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We don't use pthreads cancellation ourselves and it's painful to
use correctly. Any cancelled threads would break
vm->living_threads, GVL, thread_sync.c, autoload, etc...
So don't bother caring; because we can't stop rogue extensions
from completely breaking the VM in other ways, either.
[ruby-core:88282] [Misc #14962]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We see several occurrence of "diagnostic push/pop" so why not
make them macros. Tested on GCC8 / Clang 6.
Note that ruby.h is intentionally left untouched because we don't
want to introduce new public macros.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64118 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We reuse sleep_cond for waitpid notifications as well as GVL
waiting. So we must take care to not hold onto sleep_cond
when we try to reacquire GVL.
[ruby-core:88183]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64117 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
k0kubun
nobu
normal
svn
ko1
stomar
naruse
kazu
mrkn
shyouhei