* Added a nil check in Net::HTTPHeader#initialize_http_header for keys in the header that do not have any value
* Returning nil from the content_range method instead of raising an error when the unit in the content-range header is not bytes
* Modified initialize_http_header to match trunk
fix [Bug #11450]
fix https://github.com/ruby/ruby/pull/1018
The ignore_eof setting on HTTPResponse makes it so an EOFError is
raised when reading bodies with a defined Content-Length, if the
body read was truncated due to the socket be closed.
The ignore_eof setting on HTTP sets the values used in responses
that are created by the object.
For backwards compatibility, the default is for both settings is
true. However, unless you are specifically tested for and handling
truncated responses, it's a good idea to set ignore_eof to false so
that errors are raised for truncated responses, instead of those
errors silently being ignored.
Fixes [Bug #14972]
https://github.com/ruby/net-http/commit/4d47e34995
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Don't delete content-length before yielding inflate body, as that
causes a switch to read the entire body instead of reading in
chunks.
Fixes [Bug #16672]
https://github.com/ruby/net-http/commit/58284e9710
Co-authored-by: st0012 <stan001212@gmail.com>
This allows for the ability to opt-in to a method to set the
encoding of response bodies. By setting the accessor to a String
or Encoding instance, it will use the specified encoding.
Setting the value of true will try to detect the encoding of the
response body, either using the Content-Type header (assuming it
specifies charset) or by scanning for a <meta> tag in the document
that specifies the encoding. The default is false in which case
no forcing of encoding will be done (same as before the patch).
Implements [Feature #2567]
Implements [Feature #15517]
https://github.com/ruby/net-http/commit/6233e6b7c1
Co-authored-by: Yui Naruse <naruse@ruby-lang.org>
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Fixes [Bug #16672]
https://github.com/ruby/net-http/commit/a7cb30124c
RFC 6066, section 3, explicitly disallows the use of an IP address
as an SNI server name. So check if the connection is being made
to an IP address using the resolv regexps, and do not set an SNI
hostname in that case.
Recent changes to LibreSSL make it more strictly follow RFC 6066,
resulting an s.hostname= raising an error if passed an IP address.
When such verions of LibreSSL are used, this change not only fixes
the net/http tests, it also fixes tests for webrick and open-uri,
which both make SSL connections to 127.0.0.1 using net/http in
their tests.
Avoid warning in the openssl extension by unsetting
@ssl_context.verify_hostname if connecting to an IP address.
Make changes so that the post_connection_check still happens
when connecting to an IP address, which is necessary to keep
checking that the certificate returned includes the IP address,
which one of the tests depends on.
Revert the previous change that modified the regexp used for
checking the error message.
https://github.com/ruby/net-http/commit/fa68e64bee
Server Name Indication does not allow IP addresses (RFC 6066, section 3:
`Literal IPv4 and IPv6 addresses are not permitted in "HostName".`).
Recent versions of LibreSSL enforce this restriction, which raises
when setting the hostname to an IP address (s.hostname=), before
attempting to setup the SSL connection.
Since the certificate used in the test is already for localhost,
we cannot connect to localhost. However, as all the test does is
check for a specific exception, just modify the regexp used so it
handles both cases.
https://github.com/ruby/net-http/commit/6ab399db45
52ab9bbee9 makes a failure.
```
1) Failure:
HTTPRequestTest#test_initialize_HEAD [/tmp/ruby/v3/src/trunk/test/net/http/test_http_request.rb:53]:
<{"accept"=>["*/*"], "user-agent"=>["Ruby"]}> expected but was
<{"accept-encoding"=>["gzip;q=1.0,deflate;q=0.6,identity;q=0.3"],
"accept"=>["*/*"],
"user-agent"=>["Ruby"]}>.
```
I'm not sure it is correct, so please review later.
The last_communicated timestamp is for HTTP persistent connection, to
decide whether the current TCP connection may be reused for the
subsequent requests or not. Naturally, the timer must be reset if the
connection is recreated since it is no longer relevant.
https://github.com/ruby/net-http/commit/0a013de42d
The existing implementation of `Net::HTTP#write_timeout` relies on
`Net::BefferedIO` to trigger the `Net::WriteTimeout` error. This commit
changes `send_request_with_body_stream` to remove the optimization that
was making `Net::HTTP#write_timeout` not work when `body_stream` is
used.
Open issue:
https://bugs.ruby-lang.org/issues/17933https://github.com/ruby/net-http/commit/a0fab1ab52
Fixes intermittent error as below:
[242/838] 5316=test_ftp
#<Thread:0x0000020aa8733f20 D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:2532 run> terminated with exception (report_on_exception is true):
D:/a/ruby/ruby/src/tool/lib/minitest/unit.rb:199:in `assert': Expected #<Errno::ECONNRESET: An existing connection was forcibly closed by the remote host.> to be nil. (MiniTest::Assertion)
from D:/a/ruby/ruby/src/tool/lib/test/unit/core_assertions.rb:504:in `assert'
from D:/a/ruby/ruby/src/tool/lib/minitest/unit.rb:299:in `assert_nil'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:430:in `ensure in block in test_list_read_timeout_exceeded'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:431:in `block in test_list_read_timeout_exceeded'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:2539:in `block in create_ftp_server'
D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:426:in `write': An existing connection was forcibly closed by the remote host. (Errno::ECONNRESET)
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:426:in `print'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:426:in `block (2 levels) in test_list_read_timeout_exceeded'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:420:in `each'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:420:in `each_with_index'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:420:in `block in test_list_read_timeout_exceeded'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:2539:in `block in create_ftp_server'
to try to suppress a failure on Solaris. This is the same as
19504d115dhttp://rubyci.s3.amazonaws.com/solaris11-sunc/ruby-master/log/20210504T070007Z.fail.html.gz
```
1) Error:
IMAPTest#test_idle_done_not_during_idle:
Errno::EINVAL: Invalid argument - connect(2) for [::1]:33839
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:1214:in `__connect_nonblock'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:1214:in `connect_nonblock'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:56:in `connect_internal'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:137:in `connect'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:642:in `block in tcp'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:227:in `each'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:227:in `foreach'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:632:in `tcp'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/lib/net/imap.rb:1223:in `tcp_socket'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/lib/net/imap.rb:1180:in `initialize'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/test/net/imap/test_imap.rb:289:in `new'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/test/net/imap/test_imap.rb:289:in `test_idle_done_not_during_idle'
```
On Solaris, Socket.tcp seems to fail with EINVAL if the server closes
the connection immediately after accpeted. I think this is a bug of
Socket.tcp, but seems difficult to fix soon.
http://rubyci.s3.amazonaws.com/solaris11-sunc/ruby-master/log/20210429T100007Z.fail.html.gz
```
1) Failure:
Net::TestSMTP#test_eof_error_backtrace [/export/home/chkbuild/chkbuild-sunc/tmp/build/20210429T100007Z/ruby/test/net/smtp/test_smtp.rb:193]:
[ruby-core:78550] [Bug #13018].
[EOFError] exception expected, not #<Net::ReadTimeout: Net::ReadTimeout>.
```
Reported by Alexandr Savca as a DoS vulnerability, but Net::FTP is a
client library and the impact of the issue is low, so I have decided
to fix it as a normal issue.
Based on patch by nobu.
https://github.com/ruby/net-ftp/commit/a93af636f8
Previously, the connection leaked in this case. This uses
begin/ensure and checking for an error in the ensure block.
An alternative approach would be to not even perform the
connection until after the RETR (or other) command has been
sent. However, I'm not sure all FTP servers support that.
The current behavior is:
* Send (PASV/EPSV)
* Connect to the host/port returned in 227/229 reply
* Send (RETR/other command)
Changing it to connect after the RETR could break things.
FTP servers might expect that the client has already
connected before sending the RETR. The alternative
approach is more likely to introduce backwards compatibility
issues, compared to the begin/ensure approach taken here.
Fixes Ruby Bug 17027
https://github.com/ruby/net-ftp/commit/6e8535f076
because the name "MJIT" is an internal code name, it's inconsistent with
--jit while they are related to each other, and I want to discourage future
JIT implementation-specific (e.g. MJIT-specific) APIs by this rename.
[Feature #17490]
Shishir Joshi
Nobuyoshi Nakada
Karol Bucek
Jeremy Evans
Kazuhiro NISHIYAMA
Hiroshi SHIBATA
Takashi Kokubun
Koichi Sasada
Kazuki Yamaguchi
Miguel Teixeira
nicholas a. evans
MSP-Greg
Yusuke Endoh
Lukas Eipert
Shugo Maeda
Benoit Daloze