It's questionable whether we want to allow rstrip to work for strings
where the broken coderange occurs before the trailing whitespace and
not after, but this approach is probably simpler, and I don't think
users should expect string operations like rstrip to work on broken
strings.
In some cases, this changes rstrip to raise
Encoding::CompatibilityError instead of ArgumentError. However, as
the problem is related to an encoding issue in the receiver, and due
not due to an issue with an argument, I think
Encoding::CompatibilityError is the more appropriate error.
Fixes [Bug #18931]
This list is out of date. At least OpenBSD since 2013 does not
allow one user to read the environment variables of a process
run by another user.
While we could try to keep the list updated, I think it's a bad
idea to not use the user/password from the environment, even if
another user on the system could read it. If http_proxy exists
in the environment, and other users can read it, it doesn't
make it more secure for Ruby to ignore it. You could argue that
it encourages poor security practices, but net/http should provide
mechanism, not policy.
Fixes [Bug #18908]
https://github.com/ruby/net-http/commit/1e4585153d
* Added a nil check in Net::HTTPHeader#initialize_http_header for keys in the header that do not have any value
* Returning nil from the content_range method instead of raising an error when the unit in the content-range header is not bytes
* Modified initialize_http_header to match trunk
fix [Bug #11450]
fix https://github.com/ruby/ruby/pull/1018
The ignore_eof setting on HTTPResponse makes it so an EOFError is
raised when reading bodies with a defined Content-Length, if the
body read was truncated due to the socket be closed.
The ignore_eof setting on HTTP sets the values used in responses
that are created by the object.
For backwards compatibility, the default is for both settings is
true. However, unless you are specifically tested for and handling
truncated responses, it's a good idea to set ignore_eof to false so
that errors are raised for truncated responses, instead of those
errors silently being ignored.
Fixes [Bug #14972]
https://github.com/ruby/net-http/commit/4d47e34995
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Don't delete content-length before yielding inflate body, as that
causes a switch to read the entire body instead of reading in
chunks.
Fixes [Bug #16672]
https://github.com/ruby/net-http/commit/58284e9710
Co-authored-by: st0012 <stan001212@gmail.com>
This allows for the ability to opt-in to a method to set the
encoding of response bodies. By setting the accessor to a String
or Encoding instance, it will use the specified encoding.
Setting the value of true will try to detect the encoding of the
response body, either using the Content-Type header (assuming it
specifies charset) or by scanning for a <meta> tag in the document
that specifies the encoding. The default is false in which case
no forcing of encoding will be done (same as before the patch).
Implements [Feature #2567]
Implements [Feature #15517]
https://github.com/ruby/net-http/commit/6233e6b7c1
Co-authored-by: Yui Naruse <naruse@ruby-lang.org>
Previously, the content-encoding header was removed and the body
was modified, but the content-length header was not modified,
resulting in the content-length header not matching the body
length.
Fixes [Bug #16672]
https://github.com/ruby/net-http/commit/a7cb30124c
RFC 6066, section 3, explicitly disallows the use of an IP address
as an SNI server name. So check if the connection is being made
to an IP address using the resolv regexps, and do not set an SNI
hostname in that case.
Recent changes to LibreSSL make it more strictly follow RFC 6066,
resulting an s.hostname= raising an error if passed an IP address.
When such verions of LibreSSL are used, this change not only fixes
the net/http tests, it also fixes tests for webrick and open-uri,
which both make SSL connections to 127.0.0.1 using net/http in
their tests.
Avoid warning in the openssl extension by unsetting
@ssl_context.verify_hostname if connecting to an IP address.
Make changes so that the post_connection_check still happens
when connecting to an IP address, which is necessary to keep
checking that the certificate returned includes the IP address,
which one of the tests depends on.
Revert the previous change that modified the regexp used for
checking the error message.
https://github.com/ruby/net-http/commit/fa68e64bee
Server Name Indication does not allow IP addresses (RFC 6066, section 3:
`Literal IPv4 and IPv6 addresses are not permitted in "HostName".`).
Recent versions of LibreSSL enforce this restriction, which raises
when setting the hostname to an IP address (s.hostname=), before
attempting to setup the SSL connection.
Since the certificate used in the test is already for localhost,
we cannot connect to localhost. However, as all the test does is
check for a specific exception, just modify the regexp used so it
handles both cases.
https://github.com/ruby/net-http/commit/6ab399db45
52ab9bbee9 makes a failure.
```
1) Failure:
HTTPRequestTest#test_initialize_HEAD [/tmp/ruby/v3/src/trunk/test/net/http/test_http_request.rb:53]:
<{"accept"=>["*/*"], "user-agent"=>["Ruby"]}> expected but was
<{"accept-encoding"=>["gzip;q=1.0,deflate;q=0.6,identity;q=0.3"],
"accept"=>["*/*"],
"user-agent"=>["Ruby"]}>.
```
I'm not sure it is correct, so please review later.
The last_communicated timestamp is for HTTP persistent connection, to
decide whether the current TCP connection may be reused for the
subsequent requests or not. Naturally, the timer must be reset if the
connection is recreated since it is no longer relevant.
https://github.com/ruby/net-http/commit/0a013de42d
The existing implementation of `Net::HTTP#write_timeout` relies on
`Net::BefferedIO` to trigger the `Net::WriteTimeout` error. This commit
changes `send_request_with_body_stream` to remove the optimization that
was making `Net::HTTP#write_timeout` not work when `body_stream` is
used.
Open issue:
https://bugs.ruby-lang.org/issues/17933https://github.com/ruby/net-http/commit/a0fab1ab52
Fixes intermittent error as below:
[242/838] 5316=test_ftp
#<Thread:0x0000020aa8733f20 D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:2532 run> terminated with exception (report_on_exception is true):
D:/a/ruby/ruby/src/tool/lib/minitest/unit.rb:199:in `assert': Expected #<Errno::ECONNRESET: An existing connection was forcibly closed by the remote host.> to be nil. (MiniTest::Assertion)
from D:/a/ruby/ruby/src/tool/lib/test/unit/core_assertions.rb:504:in `assert'
from D:/a/ruby/ruby/src/tool/lib/minitest/unit.rb:299:in `assert_nil'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:430:in `ensure in block in test_list_read_timeout_exceeded'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:431:in `block in test_list_read_timeout_exceeded'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:2539:in `block in create_ftp_server'
D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:426:in `write': An existing connection was forcibly closed by the remote host. (Errno::ECONNRESET)
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:426:in `print'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:426:in `block (2 levels) in test_list_read_timeout_exceeded'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:420:in `each'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:420:in `each_with_index'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:420:in `block in test_list_read_timeout_exceeded'
from D:/a/ruby/ruby/src/test/net/ftp/test_ftp.rb:2539:in `block in create_ftp_server'
to try to suppress a failure on Solaris. This is the same as
19504d115dhttp://rubyci.s3.amazonaws.com/solaris11-sunc/ruby-master/log/20210504T070007Z.fail.html.gz
```
1) Error:
IMAPTest#test_idle_done_not_during_idle:
Errno::EINVAL: Invalid argument - connect(2) for [::1]:33839
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:1214:in `__connect_nonblock'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:1214:in `connect_nonblock'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:56:in `connect_internal'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:137:in `connect'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:642:in `block in tcp'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:227:in `each'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:227:in `foreach'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/.ext/common/socket.rb:632:in `tcp'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/lib/net/imap.rb:1223:in `tcp_socket'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/lib/net/imap.rb:1180:in `initialize'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/test/net/imap/test_imap.rb:289:in `new'
/export/home/chkbuild/chkbuild-sunc/tmp/build/20210504T070007Z/ruby/test/net/imap/test_imap.rb:289:in `test_idle_done_not_during_idle'
```
On Solaris, Socket.tcp seems to fail with EINVAL if the server closes
the connection immediately after accpeted. I think this is a bug of
Socket.tcp, but seems difficult to fix soon.
http://rubyci.s3.amazonaws.com/solaris11-sunc/ruby-master/log/20210429T100007Z.fail.html.gz
```
1) Failure:
Net::TestSMTP#test_eof_error_backtrace [/export/home/chkbuild/chkbuild-sunc/tmp/build/20210429T100007Z/ruby/test/net/smtp/test_smtp.rb:193]:
[ruby-core:78550] [Bug #13018].
[EOFError] exception expected, not #<Net::ReadTimeout: Net::ReadTimeout>.
```
Reported by Alexandr Savca as a DoS vulnerability, but Net::FTP is a
client library and the impact of the issue is low, so I have decided
to fix it as a normal issue.
Based on patch by nobu.
https://github.com/ruby/net-ftp/commit/a93af636f8
Previously, the connection leaked in this case. This uses
begin/ensure and checking for an error in the ensure block.
An alternative approach would be to not even perform the
connection until after the RETR (or other) command has been
sent. However, I'm not sure all FTP servers support that.
The current behavior is:
* Send (PASV/EPSV)
* Connect to the host/port returned in 227/229 reply
* Send (RETR/other command)
Changing it to connect after the RETR could break things.
FTP servers might expect that the client has already
connected before sending the RETR. The alternative
approach is more likely to introduce backwards compatibility
issues, compared to the begin/ensure approach taken here.
Fixes Ruby Bug 17027
https://github.com/ruby/net-ftp/commit/6e8535f076
Takashi Kokubun
Hiroshi SHIBATA
Jean Boussier
Peter Zhu
Jeremy Evans
Shishir Joshi
Nobuyoshi Nakada
Karol Bucek
Kazuhiro NISHIYAMA
Koichi Sasada
Kazuki Yamaguchi
Miguel Teixeira
nicholas a. evans
MSP-Greg
Yusuke Endoh
Lukas Eipert
Shugo Maeda