Reported by kivikakk (Ashe Connor) with tests and doc fix
Patch based on mame and fix by naruse
[Bug #14586]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62695 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/resolv.rb (UnconnectedUDP#lazy_initialize): store new
sockets before binding, so the sockets get closed when the
requester is closing.
* lib/resolv.rb (ConnectedUDP#lazy_initialize): ditto.
* lib/resolv.rb (UnconnectedUDP#close): synchronize to get rid of
race condition.
* lib/resolv.rb (ConnectedUDP#close): ditto.
[ruby-core:85901] [Bug #14571]
From: quixoten (Devin Christensen) <quixoten@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62671 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I changed my mind and thought branching ERB.new in all libraries is too
hard. Code becomes too ugly. I increased the warn level to 2, and the
old initializer will be removed when Ruby 2.5 becomes EOL.
-S option of erb(1) stays in the same policy: will be removed at Ruby 2.7.
NEWS: note about the direction
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62612 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* In Enumerable, Enumerator::Lazy, Array, Hash and Set
[Feature #13784] [ruby-core:82285]
* Share specs for the various #select#select! methods and
reuse them for #filter/#filter!.
* Add corresponding filter tests for select tests.
* Update NEWS.
[Fix GH-1824]
From: Alexander Patrick <adp90@case.edu>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62575 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Also, as it's in the middle of the list of 4 arguments, 3rd and 4th arguments
(trim_mode, eoutvar) are changed to keyword arguments.
Old ways to specify arguments are deprecated and warned now.
bin/erb: deprecate -S option.
We'll remove all of deprecated ones at Ruby 2.7+.
enc/make_encmake.rb: stopped using deprecated interface
ext/etc/mkconstants.rb: ditto
ext/socket/mkconstants.rb: ditto
sample/ripper/ruby2html.rb: ditto
spec/ruby/library/erb/defmethod/def_erb_method_spec.rb: ditto
spec/ruby/library/erb/new_spec.rb: ditto
test/erb/test_erb.rb: ditto
test/erb/test_erb_command.rb: ditto
tool/generic_erb.rb: ditto
tool/ruby_vm/helpers/dumper.rb: ditto
tool/transcode-tblgen.rb: ditto
lib/rdoc/erbio.rb: ditto
lib/rdoc/generator/darkfish.rb: ditto
[Feature #14256]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62529 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Patch by Yuuji Yaginuma. [Fixes GH-1808]
In other classes, `FrozenError` will be raised if change the frozen
object.
In order to match the behavior, I think that `FrozenError` should
use in `OpenStruct`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62271 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Thanks to Paul Kuruvilla <rohitpaulk@gmail.com> for the patch
* lib/net/http.rb: fix documentation for HTTP connection reuse
[ruby-core:84815] [Bug #14349]
From: Paul Kuruvilla <rohitpaulk@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62113 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Thanks to Michael Zimmerman for the bug report
* lib/net/pop.rb: make modified strings mutable
[ruby-core:85210] [Bug #14416]
* test/net/pop/test_pop.rb: new test
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Thanks to Sam Saffron for this patch, it shows
a nice reduction which affects many web applications:
require 'memory_profiler'
MemoryProfiler.report do
require 'uri'
end.pretty_print
Before:
Total allocated: 986643 bytes (15159 objects)
Total retained: 246370 bytes (2532 objects)
After:
Total allocated: 926903 bytes (13665 objects)
Total retained: 208570 bytes (1587 objects)
* lib/uri/common.rb: reduce allocations and retained objects
[ruby-core:85161] [Feature #14410]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62081 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/optparse.rb (OptionParser#summarize): use literal newline to
join option summaries as IO#puts does, not the special gloval
variable $/.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62035 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/optparse.rb (OptionParser#summarize): use literal newline to
join option summaries as IO#puts does, not the special gloval
variable $/.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62032 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/resolv.rb (Resolv::Hosts::DefaultFileName): fallback to NUL
device when Win32::Resolv.get_hosts_path() returned nil, to get
rid of TypeError in lazy_initialize.
[ruby-core:84907] [Bug #14369] [Fix GH-1791]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61898 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/resolv.rb (each_address): use safe navigation operator to
avoid extra hash lookups, as well as each_name since r56890.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61897 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Saves a few hundred bytes of bytecode in a frequently loaded module.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61894 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reduces both user and system CPU time for large
uploads with dynamically-generated request bodies.
user system total real
before: 0.393334 1.580000 1.973334 ( 1.971066)
after: 0.223334 0.976666 1.200000 ( 1.198514)
------
require 'socket'
require 'net/http'
require 'benchmark'
nr = 1024 * 1024 * 1024
s = TCPServer.new('127.0.0.1', 0)
addr = s.addr
at_exit { Process.waitall }
fork do
c = s.accept
# not exactly accurate but fast
IO.copy_stream(c, '/dev/null', nr + 500000)
begin
buf = c.readpartial(16384)
tmp = ''
until buf.end_with?(-"0\r\n\r\n")
buf << c.readpartial(16384, tmp)
end
rescue EOFError
end
c.write "HTTP/1.1 201 Created\r\nConnection:close\r\n\r\n"
c.close
end
r, w = IO.pipe
fork do
r.close
IO.copy_stream('/dev/zero', w, nr)
w.close
end
w.close
Net::HTTP.start(addr[3], addr[1]) do |http|
put = Net::HTTP::Put.new('/dev0/foo')
put['Content-Type'] = 'application/content-type'
put['Transfer-Encoding'] = 'chunked'
put.body_stream = r
puts(Benchmark.measure { http.request(put) })
end
------
* lib/net/http/generic_request.rb (write): use multi-arg write
* lib/net/protocol.rb (write): support multi-arg
(write0): ditto
[ruby-core:84845] [Feature #14339]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61812 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We no longer need the compressed data once the inflate block is
called; so clear it ASAP to reduce memory overhead. This is a
small chunk, so it only saves a few hundred kilobytes with the
script below.
before: RssAnon: 5976 kB
after: RssAnon: 5564 kB
------
require 'net/http'
require 'zlib'
response_gz = ARGV.shift or abort "#$0 TEMPORARY_FILE"
# pre-create response since compressing is slower than decompressing
unless File.readable?(response_gz)
nr = 16384 * 2
buf = ((0..255).map(&:chr).join * 128)
File.open(response_gz, 'wb') do |fp|
gzip = Zlib::GzipWriter.new(fp)
nr.times { gzip.write(buf) }
gzip.close
end
buf.clear
end
response_gz = File.open(response_gz)
s = TCPServer.new('127.0.0.1', 0)
pid = fork do
c = s.accept
c.readpartial(16384).clear
c.write("HTTP/1.1 200 OK\r\n" \
"Content-Length: #{response_gz.stat.size}\r\n" \
"Content-Encoding: gzip\r\n" \
"Accept-Ranges: bytes\r\n" \
"\r\n")
IO.copy_stream(response_gz, c)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get(-'/') do |res|
res.read_body(&:clear)
end
end
puts File.readlines(-'/proc/self/status').grep(/RssAnon/)[0]
Process.waitpid2(pid)
------
* lib/net/http/response.rb (inflate_adapter): clear compressed_chunk
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61684 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Since r58846 (in Ruby 2.5), it is safe to clear the string
yielded to Net::HTTPResponse#read_body methods. This
reduces malloc garbage (anonymous RSS) using the Linux-only
script below:
before: user system total real
0.030000 0.250000 0.280000 ( 0.280511)
RssAnon: 60240 kB
after: user system total real
0.050000 0.223333 0.273333 ( 0.273118)
RssAnon: 6676 kB
------
# warning this script requires 1G free space for buffering
require 'open-uri'
require 'socket'
require 'benchmark'
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
buf = ((0..255).map(&:chr).join * 128)
nr = len / buf.size
pid = fork do
c = s.accept
c.readpartial(16384).clear
c.write("HTTP/1.1 200 OK\r\n" \
"Content-Length: #{len}\r\n" \
"Content-Type: application/octet-stream\r\n" \
"\r\n")
buf.freeze # speeds up IO#write slightly
nr.times { c.write(buf) }
c.close
end
addr = s.addr
open("http://#{addr[3]}:#{addr[1]}/", "rb") do |fp|
bm = Benchmark.measure do
while fp.read(16384, buf)
end
end
puts bm
end
puts File.readlines("/proc/#$$/status").grep(/RssAnon/)[0]
Process.waitpid2(pid)
------
* lib/open-uri.rb: clear string yielded by Net::HTTPResponse#read_body
[ruby-core:84662] [Feature #14320]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61664 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There's no need to allocate a temporary string when @rbuf is
empty, we can use it as the read_nonblock destination buffer to
save both allocation overhead and avoid a later memcpy.
This results in a halving user CPU time and tiny memory
reduction with the script below:
user system total real
before 0.603333 0.539999 1.143332 ( 1.143347)
RssAnon: 5624 kB
after 0.283334 0.560000 0.843334 ( 0.846072)
RssAnon: 5592 kB
------
require 'net/http'
require 'benchmark'
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024 * 2
pid = fork do
c = s.accept
c.readpartial(16384).clear
c.send("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n", Socket::MSG_MORE)
IO.copy_stream('/dev/zero', c, len)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
puts(Benchmark.measure { res.read_body(&:clear) })
end
end
puts File.readlines("/proc/self/status").grep(/RssAnon/)[0]
Process.waitpid2(pid)
------
* lib/net/protocol.rb (rbuf_fill): avoid allocation if rbuf is empty
[ruby-core:84678] [Feature #14326]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61663 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Not an encoding expert, but this seems necessary for the next
change. All of the IO#read, IO#read_nonblock and related
methods will return a binary string when given a length
argument; so anything appended to these buffers via <<
will be binary.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Oops, not ready, yet (will work on this tomorrow :x).
This reverts commit r61638
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61640 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I noticed this bug while working on something else with
RUBYOPT=-d on, existing test cases all passed with it.
Note: I use String.new because it is the local style, here,
I prefer +'' (or ''.b, for a future commit)
* lib/net/ftp.rb (BufferedSocket#read): use String.new
* test/net/ftp/test_buffered_socket.rb (test_read_nil): new test
[ruby-core:84675] [Bug #14323]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Not an encoding expert, but this seems necessary for the next
change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There are several places where rbuf_consume is called with
@rbuf.size as its length arg; simplify that case by avoiding
the slow String#slice! operation in favor of a lightweight
replacement.
The following script exhibits reduced memory usage and
runtimes using the time(1) command:
2.9s => 2.6s
70MB => 12 MB
---------
require 'net/http'
require 'digest/md5'
Thread.abort_on_exception = true
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
th = Thread.new do
c = s.accept
c.readpartial(16384)
c.write("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n")
IO.copy_stream('/dev/zero', c, len)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
dig = Digest::MD5.new
res.read_body { |buf|
dig.update(buf)
# String#clear is important to reduce malloc overhead,
# but most Ruby programmers don't do this :<
buf.clear
}
puts dig.hexdigest
end
end
----------
* lib/net/protocol (rbuf_consume): optimize for @rbuf.size == len
[Feature #14268]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61602 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/rdoc/test_case.rb (setup): call `RDoc::Markup::PreProcess.reset`
at the end of `setup` method.
`RDoc::RDoc.new` requires `rdoc/generator/darkfish`
and requires `rdoc/tom_doc.rb` at last. It add post_proecssor
(at `add_post_processor`) and it fails
`TestRDocMarkupPreProcess#test_class_post_process`.
This issue occured only with sorted tests by alphabetical order.
`make test-all TESTS='rdoc/test_rdoc_markup_pre_process.rb --test-order=alpha`)
We can observe this failure with parallel test only a few times.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61521 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* vm_core.h (rb_vm_t): move `rb_execution_context_t::safe_level` to
`rb_vm_t::safe_level_` because `$SAFE` is a process (VM) global state.
* vm_core.h (rb_proc_t): remove `rb_proc_t::safe_level` because `Proc`
objects don't need to keep `$SAFE` at the creation.
Also make `is_from_method` and `is_lambda` as 1 bit fields.
* cont.c (cont_restore_thread): no need to keep `$SAFE` for Continuation.
* eval.c (ruby_cleanup): use `rb_set_safe_level_force()` instead of access
`vm->safe_level_` directly.
* eval_jump.c: End procs `END{}` doesn't keep `$SAFE`.
* proc.c (proc_dup): removed and introduce `rb_proc_dup` in vm.c.
* safe.c (rb_set_safe_level): don't check `$SAFE` 1 -> 0 changes.
* safe.c (safe_setter): use `rb_set_safe_level()`.
* thread.c (rb_thread_safe_level): `Thread#safe_level` returns `$SAFE`.
It should be obsolete.
* transcode.c (load_transcoder_entry): `rb_safe_level()` only returns
0 or 1 so that this check is not needed.
* vm.c (vm_proc_create_from_captured): don't need to keep `$SAFE` for Proc.
* vm.c (rb_proc_create): renamed to `proc_create`.
* vm.c (rb_proc_dup): moved from proc.c.
* vm.c (vm_invoke_proc): do not need to set and restore `$SAFE`
for `Proc#call`.
* vm_eval.c (rb_eval_cmd): rename a local variable to represent clearer
meaning.
* lib/drb/drb.rb: restore `$SAFE`.
* lib/erb.rb: restore `$SAFE`, too.
* test/lib/leakchecker.rb: check `$SAFE == 0` at the end of tests.
* test/rubygems/test_gem.rb: do not set `$SAFE = 1`.
* bootstraptest/test_proc.rb: catch up this change.
* spec/ruby/optional/capi/string_spec.rb: ditto.
* test/bigdecimal/test_bigdecimal.rb: ditto.
* test/fiddle/test_func.rb: ditto.
* test/fiddle/test_handle.rb: ditto.
* test/net/imap/test_imap_response_parser.rb: ditto.
* test/pathname/test_pathname.rb: ditto.
* test/readline/test_readline.rb: ditto.
* test/ruby/test_file.rb: ditto.
* test/ruby/test_optimization.rb: ditto.
* test/ruby/test_proc.rb: ditto.
* test/ruby/test_require.rb: ditto.
* test/ruby/test_thread.rb: ditto.
* test/rubygems/test_gem_specification.rb: ditto.
* test/test_tempfile.rb: ditto.
* test/test_tmpdir.rb: ditto.
* test/win32ole/test_win32ole.rb: ditto.
* test/win32ole/test_win32ole_event.rb: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61510 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
comment. Fix regression at r58948.
I even don't want to deprecate it because deprecation needs to lex all
embedded Ruby script using Ripper and it would be slow. So Let me just
keep this behavior of Ruby 2.4. No change is the best compatibility.
This commit stopped using String#-@ because it's harmful for "ambiguous
first argument" warning if we really want to maintain this behavior.
[Bug #14243]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61497 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
naruse
nobu
eregon
k0kubun
kazu
hsbt
marcandre
normal
ko1
stomar
shyouhei
mame