Resize string buffer only if some data is received in
BasicSocket#read_nonblock and some methods.
Co-Authored-By: Samuel Williams <samuel.williams@oriontransfer.co.nz>
This removes the related tests, and puts the related specs behind
version guards. This affects all code in lib, including some
libraries that may want to support older versions of Ruby.
There seems to be a compatibility problems with Rails +
Rack::Deflater; so we revert this incompatibility.
This effectively reverts r65922; but keeps the bugfixes to
better support non-blocking sockets and pipes for future use.
[Bug #15356] [Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Perhaps this fixes test failures reported by Greg and k0kubun.
However, the failure of certain tests to handle non-blocking I/O
seems to indicate pre-existing problems on win32 platforms.
Somebody knowledgeable about win32 should be able to fix it.
[ruby-core:89973] [ruby-core:89976] [ruby-core:89977] [Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We need to make sockets non-blocking for systems without
SOCK_CLOEXEC/SOCK_NONBLOCK macros at all.
[ruby-core:89965] [Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65925 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
All normal Ruby IO methods (IO#read, IO#gets, IO#write, ...) are
all capable of appearing to be "blocking" when presented with a
file description with the O_NONBLOCK flag set; so there is
little risk of incompatibility within Ruby-using programs.
The biggest compatibility risk is when spawning external
programs. As a result, stdin, stdout, and stderr are now always
made blocking before exec-family calls.
This change will make an event-oriented MJIT usable if it is
waiting on pipes on POSIX_like platforms.
It is ALSO necessary to take advantage of (proposed lightweight
concurrency (aka "auto-Fiber") or any similar proposal for
network concurrency: https://bugs.ruby-lang.org/issues/13618
Named-pipe (FIFO) are NOT yet non-blocking by default since
they are rarely-used and may introduce compatibility problems
and extra syscall overhead for a common path.
Please revert this commit if there are problems and if I am afk
since I am afk a lot, lately.
[ruby-core:89950] [Bug #14968]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65922 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This becomes necesary if sockets become non-blocking by
default <https://bugs.ruby-lang.org/issues/14968>; but it's
always been possible to make sockets non-blocking anyways.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65619 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (internal_write_func, internal_writev_func): retry at
unexpected EPROTOTYPE on macOS, to get rid of a kernel bug.
[ruby-core:86690] [Bug #14713]
* ext/socket/init.c (rsock_{sendto,send,write}_blocking): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63304 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
IO#read_nonblock and IO#write_nonblock take into account
buffered data, so the Linux-only BasicSocket#read_nonblock
and BasicSocket#write_nonblock methods must, too.
This bug was only introduced in r58400
("socket: avoid fcntl for read/write_nonblock on Linux")
and does not affect any stable release.
* ext/socket/basicsocket.c (rsock_init_basicsocket):
* ext/socket/init.c (rsock_s_recvfrom_nonblock):
* ext/socket/init.c (rsock_init_socket_init):
* ext/socket/lib/socket.rb (def read_nonblock):
* ext/socket/lib/socket.rb (def write_nonblock):
* ext/socket/rubysocket.h (static inline void rsock_maybe_wait_fd):
* test/socket/test_basicsocket.rb (def test_read_write_nonblock):
[Feature #13362]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60496 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (rb_readwrite_syserr_fail): works with the given errno than
thread local errno.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53265 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* file.c, io.c, util.c: prefer rb_syserr_fail with saved errno
over setting errno then call rb_sys_fail, not to be clobbered
potentially and to reduce thread local errno accesses.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53264 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
accept(2) documents ENOMEM as a possible error, handle it
consistent with all of our other FD-allocating wrappers.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52727 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* dir.c (dir_initialize): use rb_gc_for_fd for ENOMEM
* ext/socket/init.c (rsock_socket): ditto
* ext/socket/socket.c (rsock_socketpair): ditto
* internal.h (rb_gc_for_fd): prototype
* io.c (rb_gc_for_fd): remove static
[ruby-core:71623] [Feature #11727]
Manpages for opendir(2), socket(2), and socketpair(3posix)
describe ENOMEM as a possible error for each of these;
handle it consistently with our existing wrappers for
open(2)/pipe(2) etc...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52726 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/socket/init.c (is_socket): extract predicate to see if the
given fd is a socket.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52605 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/socket/init.c (rsock_s_recvfrom_nonblock):
avoid arg parsing with C API
[ruby-core:71439] [Feature #11339]
* ext/socket/basicsocket.c (bsock_recv_nonblock):
adjust for above change, make private
* ext/socket/socket.c (sock_recvfrom_nonblock): ditto
* ext/socket/udpsocket.c (udp_recvfrom_nonblock): ditto
* ext/socket/lib/socket.rb (BasicSocket#recv_nonblock):
new wrapper for private method, move RDoc
(Socket#recvfrom_nonblock): ditto
(UDPSocket#recvfrom_nonblock): ditto
Note, not adding bm_recv_nonblock.rb to benchmark/ directory
since it is non-portable. It is only in this commit message.
Benchmark results + code
target 0: a (ruby 2.3.0dev (2015-11-12 trunk 52540) [x86_64-linux])
target 1: b (ruby 2.3.0dev (2015-11-12 avoid-kwarg-capi 52540) [x86_64-linux]
-----------------------------------------------------------
recv_nonblock
require 'socket'
nr = 1000000
msg = 'hello world'
buf = ''
size = msg.bytesize
UNIXSocket.pair(:SEQPACKET) do |a, b|
nr.times do
a.sendmsg(msg)
b.recv_nonblock(size, 0, buf, exception: false)
end
end
-----------------------------------------------------------
raw data:
[["recv_nonblock",
[[1.83511221408844,
1.8703329525887966,
1.8448856547474861,
1.859263762831688,
1.8331583738327026],
[1.5637447573244572,
1.4062932096421719,
1.4247371144592762,
1.4108827747404575,
1.4802536629140377]]]]
Elapsed time: 16.530452496 (sec)
-----------------------------------------------------------
benchmark results:
minimum results in each 5 measurements.
Execution time (sec)
name a b
recv_nonblock 1.833 1.406
Speedup ratio: compare with the result of `a' (greater is better)
name b
recv_nonblock 1.304
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52598 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/socket/init.c (rsock_raise_socket_error): get rid of a glibc
bug. [ruby-core:71100] [Bug #11600]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52190 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reduces GC overhead and makes the API more consistent
with IO#read and IO#read_nonblock.
* ext/socket/basicsocket.c (bsock_recv): document outbuf
* ext/socket/unixsocket.c (unix_recvfrom): ditto
* ext/socket/init.c (rsock_strbuf, recvfrom_locktmp): new functions
(rsock_s_recvfrom): support destination buffer as 3rd arg
(rsock_s_recvfrom_nonblock): ditto
* string.c (rb_str_locktmp_ensure): export for internal ext
* test/socket/test_nonblock.rb: test recv_nonblock
* test/socket/test_unix.rb: test recv
[ruby-core:69543] [Feature #11242]
Benchmark results:
user system total real
alloc 0.130000 0.280000 0.410000 ( 0.420656)
extbuf 0.100000 0.220000 0.320000 ( 0.318708)
-------------------8<--------------------
require 'socket'
require 'benchmark'
nr = 100000
msg = ' ' * 16384
size = msg.bytesize
buf = ' ' * size
UNIXSocket.pair(:DGRAM) do |a, b|
Benchmark.bmbm do |x|
x.report('alloc') do
nr.times do
b.send(msg, 0)
a.recv(size, 0)
end
end
x.report('extbuf') do
nr.times do
b.send(msg, 0)
a.recv(size, 0, buf)
end
end
end
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
As documented before, exceptions are expensive and IO::Wait*able are too
common in socket applications to be the exceptional case. Datagram
sockets deserve the same API which stream sockets are allowed with
read_nonblock and write_nonblock.
Note: this does not offer a performance advantage under optimal
conditions when both ends are equally matched in speed, but it it
does make debug output cleaner by avoiding exceptions whenever
the receiver slows down.
* ext/socket/ancdata.c (bsock_sendmsg_internal, bsock_recvmsg_internal):
support "exception: false" kwarg
* ext/socket/init.c (rsock_s_recvfrom_nonblock):
ditto
* ext/socket/init.c (rsock_s_recvfrom_nonblock): use rsock_opt_false_p
* ext/socket/socket.c (sock_connect_nonblock): ditto
* ext/socket/rubysocket.h (rsock_opt_false_p): new function
* ext/socket/basicsocket.c (bsock_recv_nonblock): update rdoc
* ext/socket/udpsocket.c (udp_recvfrom_nonblock): ditto
* test/socket/test_nonblock.rb: new tests
[ruby-core:69542] [Feature #11229]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50910 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/socket/ancdata.c (bsock_sendmsg_internal,
bsock_recvmsg_internal):
avoid redundant fcntl on Linux
[ruby-core:69154] [Feature #11145]
* ext/socket/init.c (rsock_s_recvfrom_nonblock): ditto
* ext/socket/rubysocket.h (MSG_DONTWAIT_RELIABLE): new macro
MSG_DONTWAIT is enough to force non-blocking I/O under Linux,
so avoid changing the state of a socket. This will allow certain
threads to do a non-destructive non-blocking "peek" while others
block (without relying on an extra ppoll syscall).
We shall be conservative about enabling this feature since some
OSes may have incomplete support for MSG_DONTWAIT. I shall
defer to a FreeBSD expert to enable that for FreeBSD.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50666 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This saves a system call by allowing us to use SOCK_NONBLOCK in
Linux when accept4 is available.
Note: I do not agree accept_nonblock should always make accepted
sockets non-blocking, and will propose a future API to allow
controlling whether accepted sockets are non-blocking or not
regardless of how they were created.
* ext/socket/init.c (cloexec_accept): support nonblock flag and
use SOCK_NONBLOCK if possible
* ext/socket/init.c (rsock_s_accept_nonblock): update cloexec_accept call
* ext/socket/init.c (accept_blocking): ditto for blocking
* test/socket/test_nonblock.rb: check nonblock? on accepted socket
[Feature #11138]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50518 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/socket/init.c (rsock_s_accept_nonblock): use rb_hash_lookup2
* ext/openssl/ossl_ssl.c (get_no_exception): new function
(ossl_ssl_accept_nonblock): use get_no_exception
(ossl_ssl_read_internal): ditto
(ossl_ssl_write_nonblock): ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49955 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This is analogous to functionality found in IO#read_nonblock and
IO#wait_nonblock. Raising exceptions for common failures on
non-blocking servers is expensive and makes $DEBUG too noisy.
Benchmark results:
user system total real
default 2.790000 0.870000 3.660000 ( 3.671597)
exception: false 1.120000 0.800000 1.920000 ( 1.922032)
exception: false (cached arg) 0.820000 0.770000 1.590000 ( 1.589267)
--------------------- benchmark script ------------------------
require 'socket'
require 'benchmark'
require 'tmpdir'
nr = 1000000
Dir.mktmpdir('nb_bench') do |path|
sock_path = "#{path}/test.sock"
s = UNIXServer.new(sock_path)
Benchmark.bmbm do |x|
x.report("default") do
nr.times do
begin
s.accept_nonblock
rescue IO::WaitReadable
end
end
end
x.report("exception: false") do
nr.times do
begin
s.accept_nonblock(exception: false)
rescue IO::WaitReadable
abort "should not raise"
end
end
end
x.report("exception: false (cached arg)") do
arg = { exception: false }
nr.times do
begin
s.accept_nonblock(arg)
rescue IO::WaitReadable
abort "should not raise"
end
end
end
end
end
* ext/socket/init.c (rsock_s_accept_nonblock):
support exception: false
[ruby-core:66385] [Feature #10532]
* ext/socket/init.c (rsock_init_socket_init): define new symbols
* ext/socket/rubysocket.h: adjust prototype
* ext/socket/socket.c (sock_accept_nonblock): support exception: false
* ext/openssl/ossl_ssl.c (ossl_ssl_accept_nonblock): ditto
* ext/socket/socket.c (Init_socket): adjust accept_nonblock definition
* ext/openssl/ossl_ssl.c (Init_ossl_ssl): ditto
* ext/socket/tcpserver.c (rsock_init_tcpserver): ditto
* ext/socket/unixserver.c (rsock_init_unixserver): ditto
* ext/socket/tcpserver.c (tcp_accept_nonblock): adjust
rsock_s_accept_nonblock call
* ext/socket/unixserver.c (unix_accept_nonblock): ditto
* ext/openssl/ossl_ssl.c (ossl_start_ssl): support no_exception
* ext/openssl/ossl_ssl.c (ossl_ssl_connect): adjust ossl_start_ssl call
* ext/openssl/ossl_ssl.c (ossl_ssl_connect_nonblock): ditto
* ext/openssl/ossl_ssl.c (ossl_ssl_accept): ditto
* test/socket/test_nonblock.rb (test_accept_nonblock): test for
"exception :false"
* test/socket/test_tcp.rb (test_accept_nonblock): new test
* test/socket/test_unix.rb (test_accept_nonblock): ditto
* test/openssl/test_pair.rb (test_accept_nonblock_no_exception): ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49948 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/socket/init.c (rsock_connect): refactor for blocking
(wait_connectable): clear error before wait
[Bug #9356]
We no longer use non-blocking sockets to emulate blocking behavior,
so eliminate error-prone and confusing platform-dependent code.
According to POSIX, connect() only needs to be called once in the
face of EINTR, so do not loop on it.
Before waiting on connect, drop any pending errors, since
rb_wait_for_single_fd may not clear the existing error
properly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47617 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
writable to avoid infinite loops on FreeBSD and other platforms
which conforms to SUSv3. This problem cannot be reproduced with
loopback interfaces, so it's hard to write test code.
rsock_connect() and wait_connectable() are overly complicated, so
they should be refactored, but I commit this fix as a workaround
for the release of Ruby 1.9.3 scheduled on Feb 24.
[ruby-core:60940] [Bug #9547]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45084 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
RBASIC_CLASS(obj) macro which returns a class of `obj'.
This change is a part of RGENGC branch [ruby-trunk - Feature #8339].
* object.c: add new function rb_obj_reveal().
This function reveal interal (hidden) object by rb_obj_hide().
Note that do not change class before and after hiding.
Only permitted example is:
klass = RBASIC_CLASS(obj);
rb_obj_hide(obj);
....
rb_obj_reveal(obj, klass);
TODO: API design. rb_obj_reveal() should be replaced with others.
TODO: modify constified variables using cast may be harmful for
compiler's analysis and optimizaton.
Any idea to prohibt inserting RBasic::klass directly?
If rename RBasic::klass and force to use RBASIC_CLASS(obj),
then all codes such as `RBASIC(obj)->klass' will be
compilation error. Is it acceptable? (We have similar
experience at Ruby 1.9,
for example "RARRAY(ary)->ptr" to "RARRAY_PTR(ary)".
* internal.h: add some macros.
* RBASIC_CLEAR_CLASS(obj) clear RBasic::klass to make it internal
object.
* RBASIC_SET_CLASS(obj, cls) set RBasic::klass.
* RBASIC_SET_CLASS_RAW(obj, cls) same as RBASIC_SET_CLASS
without write barrier (planned).
* RCLASS_SET_SUPER(a, b) set super class of a.
* array.c, class.c, compile.c, encoding.c, enum.c, error.c, eval.c,
file.c, gc.c, hash.c, io.c, iseq.c, marshal.c, object.c,
parse.y, proc.c, process.c, random.c, ruby.c, sprintf.c,
string.c, thread.c, transcode.c, vm.c, vm_eval.c, win32/file.c:
Use above macros and functions to access RBasic::klass.
* ext/coverage/coverage.c, ext/readline/readline.c,
ext/socket/ancdata.c, ext/socket/init.c,
* ext/zlib/zlib.c: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40691 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
subclasses that include WaitReadable or WaitWritable rather than
extending them into the exception object each time.
* error.c: Capture EGAIN, EWOULDBLOCK, EINPROGRESS exceptions and
export them for use in WaitReadable/Writable exceptions.
* io.c: Create versions of EAGAIN, EWOULDBLOCK, EINPROGRESS that
include WaitReadable and WaitWritable. Add rb_readwrite_sys_fail
for nonblocking failures using those exceptions. Use that
function in io_getpartial and io_write_nonblock instead of
rb_mod_sys_fail
* ext/openssl/ossl_ssl.c: Add new SSLError subclasses that include
WaitReadable and WaitWritable. Use those classes for
write_would_block and read_would_block instead of rb_mod_sys_fail.
* ext/socket/ancdata.c: Use rb_readwrite_sys_fail instead of
rb_mod_sys_fail in bsock_sendmsg_internal and
bsock_recvmsg_internal.
* ext/socket/init.c: Use rb_readwrite_sys_fail instead of
rb_mod_sys_fail in rsock_s_recvfrom_nonblock and
rsock_s_connect_nonblock.
* ext/socket/socket.c: Use rb_readwrite_sys_fail instead of
rb_mod_sys_fail in sock_connect_nonblock.
* include/ruby/ruby.h: Export rb_readwrite_sys_fail for use instead
of rb_mod_sys_fail. Introduce new constants RB_IO_WAIT_READABLE and
RB_IO_WAIT_WRITABLE for first arg to rb_readwrite_sys_fail.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40195 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Debian GNU/kFreeBSD. Consider HAVE_ACCEPT4 is defined
but SOCK_CLOEXEC is not defined.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40136 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
socket address returned from recvmsg().
* ext/socket/init.c (recvfrom_blocking): ignore truncated part of
socket address returned from recvfrom().
(rsock_s_recvfrom_nonblock): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@39278 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
socket address.
* ext/socket/rubysocket.h (rsock_make_ipaddr): add an argument for
socket address length.
(rsock_ipaddr): ditto.
* ext/socket/ipsocket.c (ip_addr): pass length to rsock_ipaddr.
(ip_peeraddr): ditto.
(ip_s_getaddress): pass length to rsock_make_ipaddr.
* ext/socket/socket.c (make_addrinfo): pass length to rsock_ipaddr.
(sock_s_getnameinfo): pass actual address length to rb_getnameinfo.
(sock_s_unpack_sockaddr_in): pass length to rsock_make_ipaddr.
* ext/socket/init.c (rsock_s_recvfrom): pass length to rsock_ipaddr.
(rsock_s_recvfrom_nonblock): ditto.
* ext/socket/tcpsocket.c (tcp_sockaddr): pass length to
rsock_make_ipaddr.
* ext/socket/raddrinfo.c (make_ipaddr0): add an argument for socket
address length. pass the length to rb_getnameinfo.
(rsock_ipaddr): ditto.
(rsock_make_ipaddr): add an argument for socket address length.
pass the length to make_ipaddr0.
(make_inetaddr): pass length to make_ipaddr0.
a local variable renamed.
(host_str): a local variable renamed.
(port_str): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@39239 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
r36944. it breaks mswin/mingw ruby and brought into many many
crashes.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36946 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Samuel Williams
Yusuke Endoh
Jeremy Evans
normal
nobu
naruse
hsbt
shugo
akr
ko1
headius
usa