getaddrinfo_a() gets stuck after fork().
To avoid this, we need 1 second sleep to wait for internal
worker threads of getaddrinfo_a() to be finished, but that is unacceptable.
[Bug #17220] [Feature #17134] [Feature #17187]
Previously, rb_getaddrinfo_a_before_exec() is called from before_exec().
However, the function needs to be called only before fork().
The change moves it to before_fork().
We need stop worker threads in getaddrinfo_a() before fork().
This change adds a hook before fork() that cancel all outstanding requests
and wait for all ongoing requests. Then, it waits for all worker
threads to be finished.
Fixes [Bug #17220]
To make some kind of Ractor related extensions, some functions
should be exposed.
* include/ruby/thread_native.h
* rb_native_mutex_*
* rb_native_cond_*
* include/ruby/ractor.h
* RB_OBJ_SHAREABLE_P(obj)
* rb_ractor_shareable_p(obj)
* rb_ractor_std*()
* rb_cRactor
and rm ractor_pub.h
and rename srcdir/ractor.h to srcdir/ractor_core.h
(to avoid conflict with include/ruby/ractor.h)
This commit introduces Ractor mechanism to run Ruby program in
parallel. See doc/ractor.md for more details about Ractor.
See ticket [Feature #17100] to see the implementation details
and discussions.
[Feature #17100]
This commit does not complete the implementation. You can find
many bugs on using Ractor. Also the specification will be changed
so that this feature is experimental. You will see a warning when
you make the first Ractor with `Ractor.new`.
I hope this feature can help programmers from thread-safety issues.
Not every compilers understand that rb_raise does not return. When a
function does not end with a return statement, such compilers can issue
warnings. We would better tell them about reachabilities.
Allow the 'Dir.home' method to reliably locate the user's home directory when
all three of the following are true at the same time:
1. Ruby is running on a Unix-like OS
2. The $HOME environment variable is not set
3. The process is not a descendant of login(1) (or a work-alike)
The prior behavior was that the lookup could only work for login-descended
processes.
This is accomplished by looking up the user's record in the password database
by uid (getpwuid_r(3)) as a fallback to the lookup by name (getpwname_r(3))
which is still attempted first (based on the name, if any, returned by
getlogin_r(3)).
If getlogin_r(3), getpwnam_r(3), and/or getpwuid_r(3) is not available at
compile time, will fallback on using their respective non-*_r() variants:
getlogin(3), getpwnam(3), and/or getpwuid(3).
The rationale for attempting to do the lookup by name prior to doing it by uid
is to accommodate the possibility of multiple login names (each with its own
record in the password database, so each with a potentially different home
directory) being mapped to the same uid (as is explicitly allowed for by
POSIX; see getlogin(3posix)).
Preserves the existing behavior for login-descended processes, and adds the
new capability of having Dir.home being able to find the user's home directory
for non-login-descended processes.
Fixes [Bug #16787]
Related discussion:
https://bugs.ruby-lang.org/issues/16787https://github.com/ruby/ruby/pull/3034
not only when !w but also when w == WAITPID_LOCK_ONLY.
See also: f7c0cc3692 and a226434206.
We thought this change was an oversight in the latter commit.
Without this change, the test fails like:
$ make test-all TESTS="../test/ruby/test_process.rb -n test_exec_failure_leaves_no_child" RUN_OPTS="--jit"
...
1) Failure:
TestProcess#test_exec_failure_leaves_no_child [/home/k0kubun/src/github.com/ruby/ruby/test/ruby/test_process.rb:2493]:
Expected [[26799, #<Process::Status: pid 26799 exit 127>]] to be empty.
Co-Authored-By: Yusuke Endoh <mame@ruby-lang.org>
As fork(2) is deprecated, its calls must be guarded by
`COMPILER_WARNING_IGNORED(-Wdeprecated-declarations)`.
All usages of fork(2) in process have been alread guarded. A new call
to fork(2) was added in ruby.c with f22c4ff359.
This caused a build failure on Solaris 11.
It may hide a bug to guard big code unnecessarily, so this change
introduces a simple wrapper "rb_fork" whose definition is guarded, and
replaces all calls to fork(2) with the wrapper function.
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead. This would significantly
speed up incremental builds.
We take the following inclusion order in this changeset:
1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very
first thing among everything).
2. RUBY_EXTCONF_H if any.
3. Standard C headers, sorted alphabetically.
4. Other system headers, maybe guarded by #ifdef
5. Everything else, sorted alphabetically.
Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.
This modifies some internal functions that took a safe level argument
to no longer take the argument.
rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.
One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd. We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.
This function has been used wrongly always at first, "allocate a
buffer then wrap it with tmpbuf". This order can cause a memory
leak, as tmpbuf creation also can raise a NoMemoryError exception.
The right order is "create a tmpbuf then allocate&wrap a buffer".
So the argument of this function is rather harmful than just
useless.
TODO:
* Rename this function to more proper name, as it is not used
"temporary" (function local) purpose.
* Allocate and wrap at once safely, like `ALLOCV`.
We can check the function pointer passed to rb_define_global_function
like we do so in rb_define_method. It turns out that almost anybody
is misunderstanding the API.
We can check the function pointer passed to rb_define_module_function
like how we do so in rb_define_method. The difference is that this
changeset reveales lots of atiry mismatches.
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct. This commit uses rb_gvar_getter_t /
rb_gvar_setter_t for rb_define_hooked_variable /
rb_define_virtual_variable which revealed lots of function prototype
inconsistencies. Some of them were literally decades old, going back
to dda5dc00cf.
Often uid / gid are 16 bit or 32 bit integers, while VALUE are 32
to 64 bits. They tend to differ in size. Because rb_ensure expects
its callbacks to take VALUE arguments, narrowing must be done by
hand, otherwise data corruption can happen depending on machine ABI.
* process.c (rb_clock_getres): fix code markups, as `+` can not
include parentheses.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67369 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and functions to clarify the intention and make sure it's not used in a
surprising way (like using 2, 3, ... other than 0, 1 even while it seems
to be a boolean).
This is a retry of r66775. It included some typos...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66778 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit bb1a1aeab0.
We hit something on ci.rvm.jp, reverting until investigation is done.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66776 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and functions to clarify the intention and make sure it's not used in a
surprising way (like using 2, 3, ... other than 0, 1 even while it seems
to be a boolean).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66775 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This affects test/ruby/test_process.rb (test_execopt_env_path).
Since MJIT uses vfork+execve in a separate thread, there can be
small window in-between vfork and execve where tmp_script.cmd is
held open by the vforked child. vfork only pauses the MJIT
thread, not any Ruby Threads, so our call to Process.spawn will
hit ETXTBUSY in that window unless we fork.
main thread | MJIT thread
----------------------------------------------------
fd = open(tmp) | |
| vfork for CC | CC running
write | | ---------------
fchmod | | sees "fd" here
close(fd) | |
Process.spawn called | |
vfork (spawn)| (new process) | |
| execve => TXTBUSY | |
| | | execve (FD_CLOEXEC on fd)
| | vfork returns |
Holding the waitpid_lock whenever we intend to spawn a process
prevents the MJIT thread from spawning a process while we are
spawning in Ruby-land.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66171 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
io.c has not used it since r36229, and we can re-export
it if we need it at another time.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66157 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
TypedData_* macros hide assignments and are confusing and too
long for users of giant fonts.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66156 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
because JIT-ed code may still be on stack at this time, unlike
in ruby_cleanup().
This hopes to fix: (take 2)
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480207
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480173
It tries to print C backtrace but fails. And core file on the server
seems to be stopping on the irrelevant place due to its own signal
handler for the dump.
And I failed to reproduce this SEGV on my machine.
I don't know why it's broken, so let me try this change to investigate
the reason of SEGV.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65997 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to prevent ruby from leaving MJIT-related files.
test_jit.rb: add a test to prevent that
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65994 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
All normal Ruby IO methods (IO#read, IO#gets, IO#write, ...) are
all capable of appearing to be "blocking" when presented with a
file description with the O_NONBLOCK flag set; so there is
little risk of incompatibility within Ruby-using programs.
The biggest compatibility risk is when spawning external
programs. As a result, stdin, stdout, and stderr are now always
made blocking before exec-family calls.
This change will make an event-oriented MJIT usable if it is
waiting on pipes on POSIX_like platforms.
It is ALSO necessary to take advantage of (proposed lightweight
concurrency (aka "auto-Fiber") or any similar proposal for
network concurrency: https://bugs.ruby-lang.org/issues/13618
Named-pipe (FIFO) are NOT yet non-blocking by default since
they are rarely-used and may introduce compatibility problems
and extra syscall overhead for a common path.
Please revert this commit if there are problems and if I am afk
since I am afk a lot, lately.
[ruby-core:89950] [Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65922 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
while child handler is disabled.
trying to fix [Bug #15320]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65817 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
it didn't work.
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468677
and skips broken tests for now. But this issue should be fixed soon.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65814 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to prevent from proceeding one for MJIT while it's not safe yet.
By that situation, MJIT worker could be waiting for compiler process forever
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468033
[Bug #15320]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65807 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
by launching MJIT worker thread in child Ruby process.
See the comment before `mjit_child_after_fork` for details.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65785 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c, internal.h: support theap for small Hash.
Introduce RHASH_ARRAY (li_table) besides st_table and small Hash
(<=8 entries) are managed by an array data structure.
This array data can be managed by theap.
If st_table is needed, then converting array data to st_table data.
For st_table using code, we prepare "stlike" APIs which accepts hash value
and are very similar to st_ APIs.
This work is based on the GSoC achievement
by tacinight <tacingiht@gmail.com> and refined by ko1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65454 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Following how mjit_worker.c currently works, rb_f_system
now ensures the VM-wide waitpid lists is locked before
creating a new process via fork/vfork.
This ensures other rb_waitpid callers cannot steal work and
there are no possible race conditions from toggling
ruby_nocldwait without the use of atomics.
This sets us up for implementing MJIT process management
logic using normal Ruby APIs prepares us for VM-wide
asynchronous/event-base waitpid which can allow MJIT to
work without worker threads.
Take 2: set waitpid_state.pid on platforms w/o fork.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65437 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Following how mjit_worker.c currently works, rb_f_system
now ensures the VM-wide waitpid lists is locked before
creating a new process via fork/vfork.
This ensures other rb_waitpid callers cannot steal work and
there are no possible race conditions from toggling
ruby_nocldwait without the use of atomics.
This sets us up for implementing MJIT process management
logic using normal Ruby APIs prepares us for VM-wide
asynchronous/event-base waitpid which can allow MJIT to
work without worker threads.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65434 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_hrtime_t is a more pleasant type to use and this can make
future changes around sleeping/scheduling easier.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65182 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (check_exec_redirect): use RARRAY_AREF() instead of
using RARRAY_PTR().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This is the safer option, as there seems to be cases where checking
waitpid_state.ret is insufficient in ensure. I'm not 100% sure
why this is, but this change was required for my work-in-progress
Thread::Light patch series, too...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64782 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Oddly, all existing test cases passed multiple times
before this patch (even with --jit-wait), so this seems
like a difficult failure to prove.
Fixes: r64576 ("process.c: simplify SIGCHLD-based waitpid")
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64577 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Introduce a new rb_thread_sleep_interruptible that does not
execute interrupts before sleeping. Skipping the interrupt
check before sleep is required for out-of-GVL ruby_waitpid_all
to function properly when setting waitpid_state.ret
Now that ubf_select can be called by the gvl.timer thread
without recursive locking gvl.lock, we can safely use
rb_threadptr_interrupt to deal with waking up sleeping
processes,
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It is identical to do_waitpid, and the win32 version will not
be needed for MJIT (since win32 does not suffer from the
waitpid(-1, ...) conflict where waits can get stolen.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64541 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This re-reverts commit r64447.
The issue was machine side problem.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (ruby_waitpid_all): nothing to do unless SIGCHLD is
available.
* signal.c (ruby_nocldwait): used only if SIGCHLD is available.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64407 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Arbitrarily closing file descriptors on exec breaks use cases
where a Ruby process sets up a descriptor for non-Ruby children
to use. For example, the "rake foo" target may spawn any number
of subprocesses (Ruby or not) which depends on parsing the "FOO"
environment variable for out_fd:99 and writing to foo.out
FOO=out_fd:99 rake foo 99>>foo.out
Unfortunately, this introduced one incompatibility in
test/lib/test/unit.rb and it now requires explicitly setting
IO#close_on_exec=true
[ruby-core:88007] [Misc #14907]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64399 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We do not need to rely on SIGVTALRM for non-sighandler wakeups.
This will reduce spurious wakeups in cases where sigwait_fd
is not grabbed again, soon.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit 194a6a2c68 (r64203).
Race conditions which caused the original reversion will be fixed
in the subsequent commit.
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64352 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
getgroups(2) may return a GID list that includes duplicated GIDs.
The behavior is totaly depends on what OS is used.
This commit fixes the example of Process.groups so that the example
is independent of this OS-dependent features.
Additonaly, this commit adds the description of such system-dependent
characteristics of Process.groups.
[ruby-dev:50603] [Bug #14969]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64265 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
by sharing it with vm.c in internal.h.
vm.c: ditto
internal.h: ditto
mjit.h: share more.
mjit.c: make sure the third arguemnt is not used
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64253 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
w->cond may be changed without our knowledge in waitpid_nogvl
without th->interrupt_lock
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64201 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Once a thread has acquired sigwait_fd, hold onto it until
waitpid is complete. This prevents unnecessary migration
and atomic operations.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64200 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We don't use pthreads cancellation ourselves and it's painful to
use correctly. Any cancelled threads would break
vm->living_threads, GVL, thread_sync.c, autoload, etc...
So don't bother caring; because we can't stop rogue extensions
from completely breaking the VM in other ways, either.
[ruby-core:88282] [Misc #14962]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We see several occurrence of "diagnostic push/pop" so why not
make them macros. Tested on GCC8 / Clang 6.
Note that ruby.h is intentionally left untouched because we don't
want to introduce new public macros.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64118 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We reuse sleep_cond for waitpid notifications as well as GVL
waiting. So we must take care to not hold onto sleep_cond
when we try to reacquire GVL.
[ruby-core:88183]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64117 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
To reduce resource use and reduce CI failure; remove
timer-thread. Single-threaded Ruby processes (including forked
children) will never see extra thread overhead. This prevents
glibc and jemalloc from going into multi-threaded mode and
initializing locks or causing fragmentation via arena explosion.
The GVL is implements its own wait-queue as a ccan/list to
permit controlling wakeup order. Timeslice under contention is
handled by a designated timer thread (similar to choosing a
"patrol_thread" for current deadlock checking).
There is only one self-pipe, now, as wakeups for timeslice are
done independently using condition variables. This reduces FD
pressure slightly.
Signal handling is handled directly by a Ruby Thread (instead
of timer-thread) by exposing signal self-pipe to callers of
rb_thread_fd_select, native_sleep, rb_wait_for_single_fd, etc...
Acquiring, using, and releasing the self-pipe is exposed via 4
new internal functions:
1) rb_sigwait_fd_get - exclusively acquire timer_thread_pipe.normal[0]
2) rb_sigwait_fd_sleep - sleep and wait for signal (and no other FDs)
3) rb_sigwait_fd_put - release acquired result from rb_sigwait_fd_get
4) rb_sigwait_fd_migrate - migrate signal handling to another thread
after calling rb_sigwait_fd_put.
rb_sigwait_fd_migrate is necessary for waitpid callers because
only one thread can wait on self-pipe at a time, otherwise a
deadlock will occur if threads fight over the self-pipe.
TRAP_INTERRUPT_MASK is now set for the main thread directly in
signal handler via rb_thread_wakeup_timer_thread.
Originally, I wanted to use POSIX timers
(timer_create/timer_settime) for this. Unfortunately, this
proved unfeasible as Mutex#sleep resumes on spurious wakeups and
test/thread/test_cv.rb::test_condvar_timed_wait failed. Using
pthread_sigmask to mask out SIGVTALRM fixed that test, but
test/fiddle/test_function.rb::test_nogvl_poll proved there'd be
some unavoidable (and frequent) incompatibilities from that
approach.
Finally, this allows us to drop thread_destruct_lock and
interrupt current ec directly.
We don't need to rely on vm->thread_destruct_lock or a coherent
vm->running_thread on any platform. Separate timer-thread for
time slice and signal handling is relegated to thread_win32.c,
now.
[ruby-core:88088] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64107 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (before_fork_ruby, after_fork_ruby): used only if fork()
or daemon() is available.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
http://ci.rvm.jp/results/trunk-asserts-nopara@silicon-docker/1149270
```
/home/ko1/ruby/src/trunk-asserts-nopara/process.c: In function 'assert_close_on_exec':
/home/ko1/ruby/src/trunk-asserts-nopara/process.c:298:9: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result]
(void)write(2, m, sizeof(m) - 1);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
workaround from https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66425#c34
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64029 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* include/ruby/ruby.h (UNREACHABLE_RETURN): UNREACHABLE at the end
of non-void functions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64025 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There's actually no need to close the pipes used by the
sleepy timer thread before forking, only to stop the timer
thread itself.
Instead, we only close the parent pipes in the child process,
either via close-on-exec flag or when reinitializing the timer
thread.
This change will be necessary when we allow
rb_wait_for_single_fd and rb_thread_fd_select to wait on the
timer_thread_pipe.normal[0] directly and eliminate timer thread.
I don't anticipate compatibility problems with this change
alone.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63960 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We need to preserve "IGNORE" behavior from Ruby 2.5 and earlier.
We can't rely on SA_NOCLDWAIT any more, since we always need
system() and MJIT to work; so we fake that behavior using
dedicated reaper (currently in timer-thread).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63879 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Non-parallel "make test-spec" caused
spec/ruby/core/process/wait2_spec.rb failures because mspec
uses "exec" in single-process mode, so there's no chance
the post-exec state could know about the MJIT child process
from its pre-exec state.
[ruby-core:87846] [Bug #14867]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63877 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Reading win32/win32.c waitpid implementation, maybe waitpid(-1, ...)
on that platform will never conflict with mjit use of waitpid.
In any case, I've added WAITPID_USE_SIGCHLD macro to vm_core.h
so it can be easy for Linux/BSD users to test (hopefully!)
win32-compatible code.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The change is unstable on Windows. Please re-commit it when it correctly
supports Windows.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63852 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
For systems with lossy SIGCHLD, an infinitely sleeping timer
thread needs to be aware of rb_waitpid callers in the first
place before it can check and reset polling status.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63829 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Basically in win32, mjit.c seems to work directly on spawnvp
result while normal Ruby code wraps process handles to look
like *nix PIDs. I'm only guessing, here...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63797 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Some systems lack SIGCHLD or have incomplete SIGCHLD
implementations. So enable polling mode for them.
[ruby-core:87705] [Bug #14867]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63795 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It's a bit redundant, but we optimize for platforms with
SIGCHLD, not without.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We may be interrupted by another thread after setting ubf,
but before we re-acquire interrupt_lock again to sleep on
w->cond.
This should fix test_wait_interrupt in test/ruby/test_process.rb
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63789 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It is possible to have both MJIT and normal child processes
alive, so we cannot set ECHILD based on such a guess. We can
still elide waitpid(PID <= 0) calls if we have callers in
vm->waiting_pids, however.
For specs, ensure Process.waitall does not leak MJIT
PIDs to Rubyspace.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63764 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Use a global SIGCHLD handler to guard all callers of rb_waitpid.
To work safely with multi-threaded programs, we introduce a
VM-wide waitpid_lock to be acquired BEFORE fork/vfork spawns the
process. This is to be combined with the new ruby_waitpid_locked
function used by mjit.c in a non-Ruby thread.
Ruby-level SIGCHLD handlers registered with Signal.trap(:CHLD)
continues to work as before and there should be no regressions
in any existing use cases.
Splitting the wait queues for PID > 0 and groups (PID <= 0)
ensures we favor PID > 0 callers.
The disabling of SIGCHLD in rb_f_system is longer necessary,
as we use deferred signal handling and no longer make ANY
blocking waitpid syscalls in other threads which could "beat"
the waitpid call made by rb_f_system.
We prevent SIGCHLD from firing in normal Ruby Threads and only
enable it in the timer-thread, to prevent spurious wakeups
from in test/-ext-/gvl/test_last_thread.rb with MJIT enabled.
I've tried to guard as much of the code for RUBY_SIGCHLD==0
using C "if" statements rather than CPP "#if" so to reduce
the likelyhood of portability problems as the compiler will
see more code.
We also work to suppress false-positives from
Process.wait(-1, Process::WNOHANG) to quiets warnings from
spec/ruby/core/process/wait2_spec.rb with MJIT enabled.
Lastly, we must implement rb_grantpt for ext/pty. We need a
MJIT-compatible way of supporting grantpt(3) which may spawn
the `pt_chown' binary and call waitpid(2) on it.
[ruby-core:87605] [Ruby trunk Bug#14867]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63758 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We must block signals before stopping timer-thread, otherwise
signal handing may be delayed until (and if) another signal
is received after timer-thread is restarted.
[ruby-core:87622] [Bug #14868] [Bug #13916]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63741 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Improves readability to me, and there's no point in using
macros for this with decent compilers.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63739 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* configure.ac: clock_gettime or gettimeofday must exist.
* process.c (rb_clock_gettime): prefer clock_gettime over
gettimeofday, as the latter is obsolete in SUSv4.
* random.c (fill_random_seed): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63663 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit eliminates (char **)RSTRING_PTR(...) like usages.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63414 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* gc.c (rb_alloc_tmp_buffer_with_count): keep the order; allocate
an empty imemo first then xmalloc, to get rid of potential
memory leak when allocation imemo failed.
* parse.y (rb_parser_malloc, rb_parser_calloc, rb_parser_realloc):
ditto.
* process.c (rb_execarg_allocate_dup2_tmpbuf): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63385 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Don't abuse struct RString to hold arbitrary memory region.
Thanks to @mame we now have rb_imemo_alloc_auto_free_pointer
so use it instead.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63373 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_thread_sleep's argument is int, while rb_protect expects the function
to take VALUE. Depending on ABI this could be a problem.
We should wrap rb_thread_sleep here.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63339 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`rb_str_append` may trigger GC, and in that case eargp might be GCed.
Probably for protecting it, `RB_GC_GUARD(execarg_obj)` can be seen in
other places.
Hoping to fix:
http://ci.rvm.jp/results/trunk_gcc5@silicon-docker/569818
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62570 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (rb_exec_fillarg): share subsequence of argv_buf for
command_name, and copy the encoding from the command string.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62175 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
of `system` with `exception: true` like `Process::Status#inspect`
[Feature #14386] [ruby-core:85013]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62158 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Needlessly exporting can reduce performance locally and increase
binary size.
Increasing the footprint of our C-API larger is also detrimental
to our development as it encourages tighter coupling with our
internals; making it harder for us to preserve compatibility.
If some parts of the core codebase needs access to globals,
internal.h should be used instead of anything in include/ruby/*.
"Urabe, Shyouhei" <shyouhei@ruby-lang.org> wrote:
> On Thu, Jan 18, 2018 at 7:33 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > shyouhei@ruby-lang.org wrote:
> >> https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=61908
> >>
> >> export rb_mFConst
> >
> > Why are we exporting all these and making the public C-API bigger?
> > If anything, we should make these static. Thanks.
>
> No concrete reason, except they have already been externed in 2.5.
> These variables had lacked declarations so far, which resulted in their
> visibility to be that of extern. The commit is just confirming the status quo.
>
> I'm not against to turn them into static.
This reverts changes from r61910, r61909, r61908, r61907, and r61906.
* transcode.c (rb_eUndefinedConversionError): make static
(rb_eInvalidByteSequenceError): ditto
(rb_eConverterNotFoundError): ditto
* process.c (rb_mProcGID, rb_mProcUid, rb_mProcID_Syscall): ditto
* file.c (rb_mFConst): ditto
* error.c (rb_mWarning, rb_cWarningBuffer): ditto
* enumerator.c (rb_cLazy): ditto
[Misc #14381]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62029 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
POSIX only defines mode_t to be "an integer typea", and in fact
MacOS defines it to be uint16_t. We didn't have NUM2USHORT before
so it did not make sense but now that we have it. Why not check
apptopriately.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61950 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (get_clk_tck): prefer CLK_TCK over older HZ, next to
_SC_CLK_TCK.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61888 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (proc_s_last_status): add Process.last_status
[ruby-core:83514] [Feature #14043]
* test/ruby/test_process.rb (test_last_status): add a test case for
Process.last_status.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61143 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* eval_error.c (rb_threadptr_error_print): renamed to
rb_ec_error_print() and it accepts `ec`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60545 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to represent execution context [Feature #14038]
* vm_core.h (rb_thread_t): rb_thread_t::ec is now a pointer.
There are many code using `th` to represent execution context
(such as cfp, VM stack and so on). To access `ec`, they need to
use `th->ec->...` (adding one indirection) so that we need to
replace them by passing `ec` instead of `th`.
* vm_core.h (GET_EC()): introduced to access current ec. Also
remove `ruby_current_thread` global variable.
* cont.c (rb_context_t): introduce rb_context_t::thread_ptr instead of
rb_context_t::thread_value.
* cont.c (ec_set_vm_stack): added to update vm_stack explicitly.
* cont.c (ec_switch): added to switch ec explicitly.
* cont.c (rb_fiber_close): added to terminate fibers explicitly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60440 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* array.c (rb_to_array_type): make public to share common code
internally.
* hash.c (rb_to_hash_type): make public to share common code
internally.
* symbol.c (rb_to_symbol_type): make public to share common code
internally.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60438 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because NaCl and PNaCl are already sunset status.
see https://bugs.chromium.org/p/chromium/issues/detail?id=239656#c160
configure.ac: Patch for this file was provided by @nobu.
[Feature #14041][ruby-core:83497][fix GH-1726]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60374 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
As with forking for execve(2) in `spawn', we must block signals
to ensure they are handled correctly in a freshly `fork'-ed child.
* process.c (retry_fork_ruby): block/unblock signals around fork
(rb_fork_ruby): re-enable signals in forked child
* test/ruby/test_process.rb (test_forked_child_signal): new test
[ruby-core:82883] [Bug #13916]
Thanks to Russell Davis for the bug report and test case.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59975 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (rb_execarg_addopt_rlimit): hoist out of rb_execarg_addopt
(rlimit_type_by_sym): new wrapper for dynamic symbol
(rb_execarg_addopt): check for dsym via rlimit_type_by_sym
* test/ruby/test_process.rb (test_execopts_rlimit): check dsym w/o pindown
Add extra check for bogus rlimit args, too.
[ruby-core:82033] [Bug #13744]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59322 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* vm_core.h (rb_thread_t): move several fields which are copied at cont.c
to rb_execution_context_t.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59177 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
To convert the object implicitly, it has had two parts in convert_type() which are
1. lookink up the method's id
2. calling the method
Seems that strncmp() and strcmp() in convert_type() are slightly heavy to look up
the method's id for type conversion.
This patch will add and use internal APIs (rb_convert_type_with_id, rb_check_convert_type_with_id)
to call the method without looking up the method's id when convert the object.
Array#flatten -> 19 % up
Array#+ -> 3 % up
[ruby-dev:50024] [Bug #13341] [Fix GH-1537]
### Before
Array#flatten 104.119k (± 1.1%) i/s - 525.690k in 5.049517s
Array#+ 1.993M (± 1.8%) i/s - 10.010M in 5.024258s
### After
Array#flatten 124.005k (± 1.0%) i/s - 624.240k in 5.034477s
Array#+ 2.058M (± 4.8%) i/s - 10.302M in 5.019328s
### Test Code
require 'benchmark/ips'
class Foo
def to_ary
[1,2,3]
end
end
Benchmark.ips do |x|
ary = []
100.times { |i| ary << i }
array = [ary]
x.report "Array#flatten" do |i|
i.times { array.flatten }
end
x.report "Array#+" do |i|
obj = Foo.new
i.times { array + obj }
end
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
if getrusage(2) is available, to improve precision of Process.times and
its user like lib/benchmark.rb.
On macOS, since getrusage(2) has better precision than times(3),
they are much improved like:
* Before
Process.times
=> #<struct Process::Tms utime=0.56, stime=0.35, cutime=0.04, cstime=0.03>
puts Benchmark.measure { "a" * 1_000_000_000 }
0.340000 0.310000 0.650000 ( 0.674025)
* After
Process.times
=> #<struct Process::Tms utime=0.561899, stime=0.35076, cutime=0.046483, cstime=0.038929>
puts Benchmark.measure { "a" * 1_000_000_000 }
0.343223 0.310037 0.653260 ( 0.674025)
On Linux, since struct rusage from getrusage(2) is used instead of struct tms
from times(2), they are slightly improved like:
* Before
Process.times
=> #<struct Process::Tms utime=0.43, stime=0.11, cutime=0.0, cstime=0.0>
puts Benchmark.measure { "a" * 1_000_000_000 }
0.120000 0.040000 0.170000 ( 0.171621)
* After
Process.times
=> #<struct Process::Tms utime=0.432, stime=0.116, cutime=0.0, cstime=0.0>
puts Benchmark.measure { "a" * 1_000_000_000 }
0.124000 0.048000 0.172000 ( 0.171621)
[ruby-dev:49471] [Feature #11952]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58935 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (obj2uid, obj2gid): check the error number returned by
`getpwnam_r()` and `getgrnam_r()`, instead of `errno`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* process.c (obj2uid, obj2gid): use temporary string as the buffer
instead of `rb_alloc_tmp_buffer`, which is `NODE_ALLOCA` since
r51492. [ruby-core:81084] [Bug #13554]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58658 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We already check for __builtin_mul_overflow in configure but never
actually referred it before. Why not call it if available, because
that should render supposedly-optimial assembly outputs.
Optionally if __builtin_mul_overflow_p is available, which is the case
for recent GCC, use that to detect fixnum overflow. This is much
faster than the previous. On my machine generated assembly of
numeric.c:int_pow reduces from 480 to 448 bytes, according to nm(1).
Also on my machine, following script boosts from 7.819 to 6.929 sec.
time ./miniruby -e 'i=0; while i < 30_000_000 do i += 1; 7 ** 23; end'
Signed-off-by: Urabe, Shyouhei <shyouhei@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
CLOCK_UPTIME_RAW, and CLOCK_UPTIME_RAW_APPROX which are introduced
by macOS 10.12.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56200 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Samuel Williams
Masaki Matsushita
Takashi Kokubun
Koichi Sasada
Stefan Stüben
Kazuhiro NISHIYAMA
卜部昌平
Nobuyoshi Nakada
nicholas a. evans
Alan D. Salewski
Yusuke Endoh
Benoit Daloze
Marcus Stollsteimer
Jeremy Evans
zverok
Alan Wu
yuuji.yaginuma
nobu
akr
stomar
k0kubun
normal
svn
ko1
naruse
kazu
mrkn
shyouhei
mame
hsbt
watson1978
sorah