On my 32-bit x86 userspace, I get the following .text savings:
text data bss dec hex filename
152971 56 252 153279 256bf io.o.before
152863 56 252 153171 25653 io.o.after
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There seems to be a compatibility problems with Rails +
Rack::Deflater; so we revert this incompatibility.
This effectively reverts r65922; but keeps the bugfixes to
better support non-blocking sockets and pipes for future use.
[Bug #15356] [Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
On 64-bit Linux, fstat() needs to fill out a 144 byte struct
while F_GETFL only needs to return 8 bytes.
Fwiw, F_GETFD requires an additional rcu_read_lock and bitmap
check; so it's obviously more expensive than F_GETFL on Linux.
Reduce stack usage of rb_update_max_fd from 184 to 24 bytes.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66060 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
nogvl_copy_file_range and nogvl_copy_stream_sendfile each
used 344 bytes of stack before this change. Now, they are
inlined into nogvl_copy_stream_func which only uses 200 bytes
of stack.
"struct stat" is 144 bytes on my 64-bit Linux.
Note: this doesn't affect GC (yet) since GVL is released;
but increases safety if called from deep machine stacks.
It will affect GC if Thread::Light is merged.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66053 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Reduce the struct to 80 bytes (from 88) on amd64 to reduce
stack use.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66049 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Lets admit Windows will always be too different from POSIX-like
platforms and non-blocking may never work as well or consistently.
[ruby-core:90042] [ruby-core:90044] [Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65962 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Since non-blocking I/O is the default after [Bug #14968],
we will hit it more often and cause more acquisition/release
of GVL to wait on single FD.
This also lets us avoid touching the temporal string locking
as much and lets us clean up some test changes made for
[Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65948 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This may fix failures from TestIO#test_recycled_fd_close because
interrupts may be missed due to TOCTOU in other places.
cf. http://ci.rvm.jp/results/trunk-nopara@silicon-docker/1475034
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65939 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The previous ordering was:
a) notify waiting_fd threads of impending close
b) waiting on busy list from a)
c) invalidate fptr->fd
d) calling close()
However, it was possible for a new thread to enter
the waiting_fd list while scheduling on b), leading
to EBADF from those threads when we hit d).
Instead, we now avoid triggering EBADF in other threads by
reordering b) and c)
a) notify waiting_fd threads of impending close
c) invalidate fptr->fd
b) waiting on busy list from a)
d) calling close()
cf. http://ci.rvm.jp/results/trunk-nopara@silicon-docker/1474526
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65937 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
fptr->fd may become -1 while GVL is released in
rb_wait_for_single_fd, so we must check it after reacquiring
GVL. This should avoid EBADF errors exposed by making pipes
non-blocking by default:
http://ci.rvm.jp/results/trunk-test@ruby-sky3/1473710
[Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65931 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
All normal Ruby IO methods (IO#read, IO#gets, IO#write, ...) are
all capable of appearing to be "blocking" when presented with a
file description with the O_NONBLOCK flag set; so there is
little risk of incompatibility within Ruby-using programs.
The biggest compatibility risk is when spawning external
programs. As a result, stdin, stdout, and stderr are now always
made blocking before exec-family calls.
This change will make an event-oriented MJIT usable if it is
waiting on pipes on POSIX_like platforms.
It is ALSO necessary to take advantage of (proposed lightweight
concurrency (aka "auto-Fiber") or any similar proposal for
network concurrency: https://bugs.ruby-lang.org/issues/13618
Named-pipe (FIFO) are NOT yet non-blocking by default since
they are rarely-used and may introduce compatibility problems
and extra syscall overhead for a common path.
Please revert this commit if there are problems and if I am afk
since I am afk a lot, lately.
[ruby-core:89950] [Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65922 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The behaviour of IO#ungetbyte has been depending on the width of
Fixnums. Fixnums should be invisible nowadays. It must be a
bug. Fix [Bug #14359]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65802 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This respects VM_CHECK_MODE and is more consistent with
the rest of our code.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65777 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The former states explicitly that the argument must be a literal,
and can optimize away `strlen` on all compilers.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65059 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (argf_next_argv): convert filename to the OS encoding to be
dealt with by system calls. [ruby-dev:50607] [Bug #14970]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64243 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
GNU/Hurd has writev(2) but does not define IOV_MAX
[ruby-core:87417] [Bug #14827]
Reported-by: Paul Sonnenschein
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
FreeBSD 11.0+ supports ppoll, so we may use it after accounting
for portability differences in how it treats POLLOUT vs POLLHUP
events as mutually exclusive (as documented in the FreeBSD
poll(2) manpage).
For waiting on high-numbered single FDs, this should put
FreeBSD on equal footing with Linux and should allow cheaper
FD readiness checking with sleepy GC in the future.
* thread.c (USE_POLL, POLLERR_SET): define for FreeBSD 11.0+
(rb_wait_for_single_fd): return all requested events on POLLERR_SET
io.c (USE_POLL): define for FreeBSD 11.0+
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63427 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
However this function is listed in ruby/io.h. We cannot but
define a new, void-returning variant to use instead.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63337 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
While we cannot use LIST_HEAD since r63312, we can at
least use list_head_init to make our code more readable.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63314 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Address of a variable whose storage duration is `auto` is _not_ a
compile time constant, according to ISO 9899 section 6.4.
LIST_HEAD takes such thing. You can't use it to declare local
variables.
Interestingly, address of a static variable _is_ a compile time
constant. So a declaration like `static LIST_HEAD..` is
completely legal even in C90.
In C99 and newer, this is not a constraint violation.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63312 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (internal_write_func, internal_writev_func): retry at
unexpected EPROTOTYPE on macOS, to get rid of a kernel bug.
[ruby-core:86690] [Bug #14713]
* ext/socket/init.c (rsock_{sendto,send,write}_blocking): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63304 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It is unsafe to release GVL and call rb_notify_fd_close after
close(2) on any given FD. FDs (file descriptor) may be recycled
in other threads immediately after close() to point to a different
file description. Note the distinction between "file description"
and "file descriptor".
th-1 | th-2
-------------------------------+---------------------------------------
io_close_fptr |
rb_notify_fd_close(fd) |
fptr_finalize_flush |
close(fd) |
rb_thread_schedule |
| fd reused (via pipe/open/socket/etc)
rb_notify_fd_close(fd) |
| sees "stream closed" exception
| for DIFFERENT file description
* thread.c (rb_thread_io_blocking_region): adjust comment for list_del
* thread.c (rb_notify_fd_close): give busy list to caller
* thread.c (rb_thread_fd_close): loop on busy list
* io.c (io_close_fptr): do not call rb_thread_fd_close on invalid FD
* io.c (io_reopen): use rb_thread_fd_close
Fixes: r57422 ("io.c: close before wait")
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63216 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This is consistent with other implementations of .write
in openssl and stringio.
* io.c (io_write_m): return 0 on argc == 0
[ruby-core:86285] [Bug #14338]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62967 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
For security reasons, File.read, File.binread, File.write, File.binwrite,
File.foreach, and File.readlines should not invoke external commands even
if the path starts with the pipe character |.
[ruby-core:84495] [Feature #14245]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62857 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It can be specified from 2.0. Ref: https://bugs.ruby-lang.org/issues/7103
[Fix GH-1841]
From: yuuji.yaginuma <yuuji.yaginuma@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62793 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
io.c: Variable and label definition are necessary in both cases.
From: Lars Kanis <lars@greiz-reinsdorf.de>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62321 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (fptr_finalize_flush): removed unused assignments. if
noraise, err is never used after set.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62157 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (fptr_copy_finalizer): fix inverted condition. if
finalizer does not change, pipe_list should not change too.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62145 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (fptr_copy_finalizer): remove fptr from pipe_list when pipe
became ordinary file, to fix access after free. to be finalized
by pipe_finalize and being in pipe_list must match.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62123 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (pipe_register_fptr): get rid of double registration which
causes access after free and segfault.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62121 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (pipe_del_fptr): merged code for the case fptr is first to
the loop for the rest.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62120 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (rb_stderr_to_original_p): hoist out the condition to write
messages to the stderr FD directly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61646 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
While we can't recycle strings after giving them rb_funcall*,
we can reduce their malloc overhead by resizing them to zero.
This only affects cases where either `src' or `dst' is a non-IO
object and either `copy_length' is passed or there is
pre-existing data in the read buffer.
* io.c (copy_stream_fallback_body): clear when done with `copy_length'
(copy_stream_body): clear when done with pre-existing read buffer
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61632 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (nogvl_copy_file_range): ignore EPERM and fallback to
sendfile(2) or read/write. copy_file_range(2) may not exist
even if __NR_copy_file_range is defined in the build environment.
[Bug #14207]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61412 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Since we release GVL, we must freeze and duplicate the string buffer
to prevent other threads from modifying our buffer while we are
waiting on pwrite(2).
* io.c (rb_io_pwrite): use_rb_str_tmp_frozen_{acquire/release}
[Bug #14195]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61376 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (rb_io_open_generic): try to open the named file as usual,
if klass is not IO nor File, so that Errno::ENOENT will be
raised probably. calling on File will be same in the future.
From: Nobuyoshi Nakada <nobu@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61320 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (rb_io_open_generic): when external command will be invoked
as other than IO singleton method, probably unintentionally,
warn if it is File or raise ArgumentError.
From: Nobuyoshi Nakada <nobu@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61317 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (open_key_args): open by rb_io_open always also when
open_args: option is given.
From: Nobuyoshi Nakada <nobu@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61314 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
IO#putc is multi-byte character safe when a String is given as its argument.
[ruby-core:82019] [Bug #13741]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60991 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (rb_write_error2): call `rb_w32_write_console()` when the device is tty,
like `rb_write_error_str()`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60957 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (io_strip_bom): just abandon detecting UTF encoding by BOM
unless opened for reading.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60884 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c: [DOC] improve the description for the chomp option and
add examples to IO.readlines and IO#readlines; other small fixes.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60624 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (io_writev): honor buffered mode to get rid of broken pipe
error when stdout is redirected to a pipeline.
[ruby-core:83578] [Feature #14042]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60535 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (io_fwritev): needs conversion to string before accessing
the content, as well as single argument case, not to segfault.
[Feature #9323]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60534 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Linux copy_file_range(2) fails with EBADF if the destination FD
has O_APPEND set. Preserve existing (Ruby <= 2.4) behavior by
falling back to alternative copy mechanisms if this is the case
(instead of raising Errno::EBADF).
* io.c (nogvl_copy_file_range): do not raise on O_APPEND dst
* test/ruby/test_io.rb (test_copy_stream_append): new test
[Feature #13867]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60490 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* array.c (rb_to_array_type): make public to share common code
internally.
* hash.c (rb_to_hash_type): make public to share common code
internally.
* symbol.c (rb_to_symbol_type): make public to share common code
internally.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60438 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (rb_io_puts): write a newline together at once for each
argument. based on the patch by rohitpaulk (Rohit Kuruvilla) at
[ruby-core:83508]. [Feature #14042]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60417 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because NaCl and PNaCl are already sunset status.
see https://bugs.chromium.org/p/chromium/issues/detail?id=239656#c160
configure.ac: Patch for this file was provided by @nobu.
[Feature #14041][ruby-core:83497][fix GH-1726]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60374 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (io_writev): total may be a bignum. [Feature #9323]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60371 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (io_writev): fix local variable declarations, when
writev(2) is not available. [Feature #9323]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60351 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
io.c: make IO#write accept multiple arguments.
it uses writev(2) if possible.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60343 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (argf_next_argv): the standard conversion to path name
should take place.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60154 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (rb_file_initialize): [DOC] stated that non-tty file is
buffered by the default, and added links to related methods.
[ruby-core:83081] [Bug #13965]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60121 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (ruby_set_inplace_mode): check if null is contained. based
on the patch by tommy (Masahiro Tomita) in [ruby-dev:50272].
[Bug #13960]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60097 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (fptr_finalize_flush): add an argument to keep GVL.
* io.c (fptr_finalize): adjust for above change.
* io.c (io_close_fptr): closing without GVL causes another
exception while raising exception in another thread. This causes
segfault on Windows. Keep GVL while closing when another thread
raises.
[Bug #13856] [ruby-core:82602]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60055 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
File#path for a file opened with O_TMPFILE has no meaning.
A filepath returned by this method isn't guarranteed about its accuracy,
but files opened with O_TMPFILE are known its recorded path has no
meaning. So let them not to return any pathname.
After a discussion in ruby-core, just returning Qnil makes guessing the
root cause difficult. Instead, this patch makes the method to raise an
error.
Other consideration is calling fnctl(2) on rb_file_path, but it adds a
overhead, and it's difficult to determine O_TMPFILE status after fd has
been closed.
[Feature #13568]
* io.c(rb_file_open_generic): Set Qnil to fptr->pathv when opening a
file using O_TMPFILE
* file.c(rb_file_path): Raise IOError when fptr->pathv is Qnil
* file.c(rb_file_path): [DOC] Update for the new behavior
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59704 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (io_setstrbuf): return true if the buffer is newly created.
* io.c (io_set_read_length): shrink the read buffer if it is a new
object and is too large. [ruby-core:81370] [Bug #13597]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59701 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (rb_io_reopen): take a block and ensure the IO closed
[Feature #2631]
* test/ruby/test_io.rb: add a test
* NEWS: add an entry for this change
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
To convert the object implicitly, it has had two parts in convert_type() which are
1. lookink up the method's id
2. calling the method
Seems that strncmp() and strcmp() in convert_type() are slightly heavy to look up
the method's id for type conversion.
This patch will add and use internal APIs (rb_convert_type_with_id, rb_check_convert_type_with_id)
to call the method without looking up the method's id when convert the object.
Array#flatten -> 19 % up
Array#+ -> 3 % up
[ruby-dev:50024] [Bug #13341] [Fix GH-1537]
### Before
Array#flatten 104.119k (± 1.1%) i/s - 525.690k in 5.049517s
Array#+ 1.993M (± 1.8%) i/s - 10.010M in 5.024258s
### After
Array#flatten 124.005k (± 1.0%) i/s - 624.240k in 5.034477s
Array#+ 2.058M (± 4.8%) i/s - 10.302M in 5.019328s
### Test Code
require 'benchmark/ips'
class Foo
def to_ary
[1,2,3]
end
end
Benchmark.ips do |x|
ary = []
100.times { |i| ary << i }
array = [ary]
x.report "Array#flatten" do |i|
i.times { array.flatten }
end
x.report "Array#+" do |i|
obj = Foo.new
i.times { array + obj }
end
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Somebody may pass 1030 (the value of F_DUPFD_CLOEXEC) to IO#fcntl
because they copied code from somewhere else. Ensure we know
about FDs created that way.
* io.c (do_fcntl): update max FD for F_DUPFD_CLOEXEC, too
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58590 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
poll(fds, n, 0) mean no timeout and immediately return. If you want to
wait something, you need to use -1 instead.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58534 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
svn
normal
kazu
shyouhei
nobu
shugo
k0kubun
glass
naruse
usa
stomar
hsbt
akr
shirosaki
sorah
watson1978
kosaki