Because hard to specify commits related to r67479 only.
So please commit again.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit should fix Net::Protocol::BufferedIO#write when sending
large multi-byte string like following example.
```
$ ruby -rnet/http -rjson -v -e "Net::HTTP.post(URI('http://httpbin.org/post'), { text: 'あ'*100_000 }.to_json, 'Content-Type' => 'application/json')"
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
Traceback (most recent call last):
19: from -e:1:in `<main>'
18: from lib/ruby/2.6.0/net/http.rb:500:in `post'
17: from lib/ruby/2.6.0/net/http.rb:605:in `start'
16: from lib/ruby/2.6.0/net/http.rb:920:in `start'
15: from lib/ruby/2.6.0/net/http.rb:502:in `block in post'
14: from lib/ruby/2.6.0/net/http.rb:1281:in `post'
13: from lib/ruby/2.6.0/net/http.rb:1493:in `send_entity'
12: from lib/ruby/2.6.0/net/http.rb:1479:in `request'
11: from lib/ruby/2.6.0/net/http.rb:1506:in `transport_request'
10: from lib/ruby/2.6.0/net/http.rb:1506:in `catch'
9: from lib/ruby/2.6.0/net/http.rb:1507:in `block in transport_request'
8: from lib/ruby/2.6.0/net/http/generic_request.rb:123:in `exec'
7: from lib/ruby/2.6.0/net/http/generic_request.rb:189:in `send_request_with_body'
6: from lib/ruby/2.6.0/net/protocol.rb:247:in `write'
5: from lib/ruby/2.6.0/net/protocol.rb:265:in `writing'
4: from lib/ruby/2.6.0/net/protocol.rb:248:in `block in write'
3: from lib/ruby/2.6.0/net/protocol.rb:275:in `write0'
2: from lib/ruby/2.6.0/net/protocol.rb:275:in `each_with_index'
1: from lib/ruby/2.6.0/net/protocol.rb:275:in `each'
lib/ruby/2.6.0/net/protocol.rb:280:in `block in write0': undefined method `bytesize' for nil:NilClass (NoMethodError)
```
[Fix GH-2058]
From: Eito Katagiri <eitoball@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66582 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
So that classes which uses Net::HTTP with https can use OpenSSL
namespace for example exception classes like OpenSSL::SSL::SSLError.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66401 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Stringify after downcase, to restrict arguments.
[Bug#15394]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66321 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/net/protocol.rb (ReadTimeout, WriteTimeout): Net::ReadTimeout and Net::WriteTimeout should tell the cause socket
[Feature #14832] [ruby-core:87440]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/net/http.rb: Documentation for Net::HTTP claims that
multipart/form-data is not supported, but Net::HTTPHeader#set_form
supports it since 1.9.3.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65230 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When TLS 1.3 is in use, the session ticket may not have been sent yet
even though a handshake has finished. Also, the ticket could change if
multiple session ticket messages are sent by the server. Use
SSLContext#session_new_cb instead of calling SSLSocket#session
immediately after a handshake. This way also works with earlier protocol
versions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64234 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/net/http/exceptions.rb: revert a part of r63590. to deprecate
Net::ProtoServerError seems to be wrong.
see [ruby-core:87488] [Feature#14688]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63661 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`Net::HTTPServerException` has been deprecated since r63590. [Bug #14688]
And `net/http/responses.rb` uses the deprecated constant, so Ruby warns of the deprecation.
Example:
```bash
$ ruby -r net/http -e ''
/home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/net/http/responses.rb:22: warning: constant Net::HTTPServerException is deprecated
```
This change suppresses the warning.
From: Masataka Pocke Kuwabara <kuwabara@pocke.me>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63591 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Thanks to ShockwaveNN (Pavel Lobashov) for reporting the bug.
[ruby-core:86990] [Bug #14750]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63396 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Otherwise connections (commonly on IDLE, but it could be any
command) may never receive notifications of link errors.
[ruby-core:86628] [Feature #14703]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63243 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Thanks to Paul Kuruvilla <rohitpaulk@gmail.com> for the patch
* lib/net/http.rb: fix documentation for HTTP connection reuse
[ruby-core:84815] [Bug #14349]
From: Paul Kuruvilla <rohitpaulk@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62113 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Thanks to Michael Zimmerman for the bug report
* lib/net/pop.rb: make modified strings mutable
[ruby-core:85210] [Bug #14416]
* test/net/pop/test_pop.rb: new test
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reduces both user and system CPU time for large
uploads with dynamically-generated request bodies.
user system total real
before: 0.393334 1.580000 1.973334 ( 1.971066)
after: 0.223334 0.976666 1.200000 ( 1.198514)
------
require 'socket'
require 'net/http'
require 'benchmark'
nr = 1024 * 1024 * 1024
s = TCPServer.new('127.0.0.1', 0)
addr = s.addr
at_exit { Process.waitall }
fork do
c = s.accept
# not exactly accurate but fast
IO.copy_stream(c, '/dev/null', nr + 500000)
begin
buf = c.readpartial(16384)
tmp = ''
until buf.end_with?(-"0\r\n\r\n")
buf << c.readpartial(16384, tmp)
end
rescue EOFError
end
c.write "HTTP/1.1 201 Created\r\nConnection:close\r\n\r\n"
c.close
end
r, w = IO.pipe
fork do
r.close
IO.copy_stream('/dev/zero', w, nr)
w.close
end
w.close
Net::HTTP.start(addr[3], addr[1]) do |http|
put = Net::HTTP::Put.new('/dev0/foo')
put['Content-Type'] = 'application/content-type'
put['Transfer-Encoding'] = 'chunked'
put.body_stream = r
puts(Benchmark.measure { http.request(put) })
end
------
* lib/net/http/generic_request.rb (write): use multi-arg write
* lib/net/protocol.rb (write): support multi-arg
(write0): ditto
[ruby-core:84845] [Feature #14339]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61812 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We no longer need the compressed data once the inflate block is
called; so clear it ASAP to reduce memory overhead. This is a
small chunk, so it only saves a few hundred kilobytes with the
script below.
before: RssAnon: 5976 kB
after: RssAnon: 5564 kB
------
require 'net/http'
require 'zlib'
response_gz = ARGV.shift or abort "#$0 TEMPORARY_FILE"
# pre-create response since compressing is slower than decompressing
unless File.readable?(response_gz)
nr = 16384 * 2
buf = ((0..255).map(&:chr).join * 128)
File.open(response_gz, 'wb') do |fp|
gzip = Zlib::GzipWriter.new(fp)
nr.times { gzip.write(buf) }
gzip.close
end
buf.clear
end
response_gz = File.open(response_gz)
s = TCPServer.new('127.0.0.1', 0)
pid = fork do
c = s.accept
c.readpartial(16384).clear
c.write("HTTP/1.1 200 OK\r\n" \
"Content-Length: #{response_gz.stat.size}\r\n" \
"Content-Encoding: gzip\r\n" \
"Accept-Ranges: bytes\r\n" \
"\r\n")
IO.copy_stream(response_gz, c)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get(-'/') do |res|
res.read_body(&:clear)
end
end
puts File.readlines(-'/proc/self/status').grep(/RssAnon/)[0]
Process.waitpid2(pid)
------
* lib/net/http/response.rb (inflate_adapter): clear compressed_chunk
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61684 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There's no need to allocate a temporary string when @rbuf is
empty, we can use it as the read_nonblock destination buffer to
save both allocation overhead and avoid a later memcpy.
This results in a halving user CPU time and tiny memory
reduction with the script below:
user system total real
before 0.603333 0.539999 1.143332 ( 1.143347)
RssAnon: 5624 kB
after 0.283334 0.560000 0.843334 ( 0.846072)
RssAnon: 5592 kB
------
require 'net/http'
require 'benchmark'
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024 * 2
pid = fork do
c = s.accept
c.readpartial(16384).clear
c.send("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n", Socket::MSG_MORE)
IO.copy_stream('/dev/zero', c, len)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
puts(Benchmark.measure { res.read_body(&:clear) })
end
end
puts File.readlines("/proc/self/status").grep(/RssAnon/)[0]
Process.waitpid2(pid)
------
* lib/net/protocol.rb (rbuf_fill): avoid allocation if rbuf is empty
[ruby-core:84678] [Feature #14326]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61663 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Not an encoding expert, but this seems necessary for the next
change. All of the IO#read, IO#read_nonblock and related
methods will return a binary string when given a length
argument; so anything appended to these buffers via <<
will be binary.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Oops, not ready, yet (will work on this tomorrow :x).
This reverts commit r61638
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61640 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I noticed this bug while working on something else with
RUBYOPT=-d on, existing test cases all passed with it.
Note: I use String.new because it is the local style, here,
I prefer +'' (or ''.b, for a future commit)
* lib/net/ftp.rb (BufferedSocket#read): use String.new
* test/net/ftp/test_buffered_socket.rb (test_read_nil): new test
[ruby-core:84675] [Bug #14323]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Not an encoding expert, but this seems necessary for the next
change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There are several places where rbuf_consume is called with
@rbuf.size as its length arg; simplify that case by avoiding
the slow String#slice! operation in favor of a lightweight
replacement.
The following script exhibits reduced memory usage and
runtimes using the time(1) command:
2.9s => 2.6s
70MB => 12 MB
---------
require 'net/http'
require 'digest/md5'
Thread.abort_on_exception = true
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
th = Thread.new do
c = s.accept
c.readpartial(16384)
c.write("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n")
IO.copy_stream('/dev/zero', c, len)
c.close
end
addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
dig = Digest::MD5.new
res.read_body { |buf|
dig.update(buf)
# String#clear is important to reduce malloc overhead,
# but most Ruby programmers don't do this :<
buf.clear
}
puts dig.hexdigest
end
end
----------
* lib/net/protocol (rbuf_consume): optimize for @rbuf.size == len
[Feature #14268]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61602 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
like r59693, initialize_http_header also should raise error. [Bug #14208]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61359 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
If uplevel keyword is given, the warning message is prepended
with caller file and line information and the string "warning: ".
The use of the uplevel keyword makes Kernel#warn format output
similar to how rb_warn formats output.
This patch modifies net/ftp and net/imap to use Kernel#warn
instead of $stderr.puts or $stderr.printf, since they are used
for printing warnings.
This makes lib/cgi/core and tempfile use $stderr.puts instead of
warn for debug logging, since they are used for debug printing
and not for warning.
This does not modify bundler, rubygems, or rdoc, as those are
maintained outside of ruby and probably wish to remain backwards
compatible with older ruby versions.
rb_warn_m code is originally from nobu, but I've changed it
so that it only includes the path and lineno from uplevel
(not the method), and also prepends the string "warning: ",
to make it more similar to rb_warn.
From: Jeremy Evans code@jeremyevans.net
Signed-off-by: Urabe Shyouhei shyouhei@ruby-lang.org
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61155 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
knu
kazu
shugo
glass
nobu
naruse
usa
marcandre
stomar
aycabta
rhe
ko1
normal
nagachika
shyouhei