Граф коммитов

60 Коммитов

Автор SHA1 Сообщение Дата
Tanaka Akira 8f7884761e The default charset of text/* media type is UTF-8.
Thanks for the patch  gareth (Gareth Adams).  [Bug #15933]

-------

Combines two small, but very related changes

1: Treat HTTPS the same as HTTP

Previously, OpenURI followed guidance in RFC2616/3.7.1:

> When no explicit charset parameter is provided by the sender, media
> subtypes of the "text" type are defined to have a default charset
> value of "ISO-8859-1" when received via HTTP.

However this RFC was written before TLS was established and OpenURI was
never updated to treat HTTPS traffic the same way. So, HTTPS documents
received a different default to HTTP documents.

This commit removes the scheme check so that all text/* documents
processed by OpenURI are treated the same way.

In theory this processing gets applied to FTP URIs too, but there's no
mechanism in OpenURI for FTP documents to have Content-Type metadata
appended to them, so this ends up being a no-op.

2: Change default charset for text/* to UTF-8

Replaces the default ISO-8859-1 charset previously defined in RFC2616 (now
obsoleted) with a UTF-8 charset as defined in RFC6838.

Fixes: https://bugs.ruby-lang.org/issues/15933
2019-07-15 09:36:52 +09:00
Tanaka Akira 05aac90a1b Warn open-uri's "open" method at Kernel.
Use URI.open instead.

Thanks for the patch by jeremyevans0 (Jeremy Evans) [Misc #15893].
2019-07-14 17:18:17 +09:00
Yusuke Endoh 43730256e8 open-uri: Regenerate server certificates for tests
OpenSSL 1.1.1 requires 2048 bits or more.  This change will fix:
https://rubyci.org/logs/rubyci.s3.amazonaws.com/debian/ruby-master/log/20190527T003004Z.fail.html.gz#test%2Fopen-uri
2019-05-27 12:58:08 +09:00
akr bf287424fd open-uri defines URI.open defined as an alias.
open-uri's Kernel.open will be deprecated in future.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61392 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21 14:15:04 +00:00
akr a2831955b2 lib/open-uri.rb: accept :encoding option as well as encoding in mode string.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60232 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 06:22:53 +00:00
nobu 3408e9e3c3 revert r57323
StringIO.new makes the buffer IO.default_external, while
StringIO.new("".dup) makes source encoding which is defaulted to
UTF-8.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57328 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-14 09:38:56 +00:00
nobu c8a4b57254 StringIO.new makes a RW buffer
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57323 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-14 02:06:22 +00:00
rhe 0cc0080a49 open-uri: don't use OpenSSL::TestUtils from open-uri tests
Follow net/http. Define own test DH parameters and use.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56935 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-11-29 15:48:46 +00:00
svn 3cb56e8e3a * remove trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55316 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-07 12:29:37 +00:00
rhe 06552d56d1 open-uri: regenerate server certificates used in tests
* test/open-uri/test_ssl.rb: Regenerate test certificates. The test CA
  certificate was incorrectly generated. A CA certificate must have the
  basic constraints extension with cA bit set to TRUE. OpenSSL <= 1.0.2
  allowed the error when the certificate is in the trusted store but
  OpenSSL 1.1.0 no longer does.
  [ruby-core:75225] [Feature #12324]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55315 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-07 12:29:36 +00:00
hsbt e79d163caf * lib/open-uri.rb: Use `userinfo` for authenticated proxy.
[fix GH-1148] Patch by @SokichiFujita
* test/open-uri/test_open-uri.rb: ditto.
  [fix GH-1309] Patch by @jdamick

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54432 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-03-30 00:46:05 +00:00
nobu 8d66627161 leakchecker.rb: remove temporary measure
* lib/webrick/utils.rb (WEBrick::Utils::TimeoutHandler#watcher):
  make watcher thread restartable.

* lib/webrick/utils.rb (WEBrick::Utils::TimeoutHandler#terminate):
  new method to terminate watcher thread.

* test/lib/leakchecker.rb (LeakChecker#find_threads): revert
  r46941.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53439 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-01-05 06:09:17 +00:00
nobu c81b224edc test: use String#b instead of dup.force_encoding
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53109 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-14 06:40:55 +00:00
akr 68ebbbfebe * lib/open-uri.rb: Remove indicator for "frozen_string_literal: true".
* lib/pp.rb: Ditto.

* lib/prettyprint.rb: Ditto.

* lib/resolv.rb: Ditto.

* lib/securerandom.rb: Ditto.

* lib/tmpdir.rb: Ditto.

* lib/unicode_normalize/tables.rb: Ditto.

* test/net/ftp/test_buffered_socket.rb: Ditto.

* test/net/ftp/test_mlsx_entry.rb: Ditto.

* test/open-uri/test_open-uri.rb: Ditto.

* test/open-uri/test_ssl.rb: Ditto.

* test/pathname/test_pathname.rb: Ditto.

* test/test_pp.rb: Ditto.

* test/test_prettyprint.rb: Ditto.

* tool/transcode-tblgen.rb: Ditto.

* ext/pathname/lib/pathname.rb: Ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52526 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-11-10 11:48:14 +00:00
akr 89db37cf77 * lib/open-uri.rb: Specify frozen_string_literal: true.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52186 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-10-18 12:17:52 +00:00
nobu ae042f21fb use assert_raise
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50727 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-02 02:18:44 +00:00
nobu 21a40e94ac skip old OpenSSL
* test/open-uri/test_ssl.rb, test/webrick/test_httpproxy.rb: also
  depends on test/openssl/utils.rb.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48825 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-13 07:12:56 +00:00
usa bf4b7f3b8c * test/net/http/test_https.rb
(TestNetHTTPS#test_certificate_verify_failure): on Windows,
  Errno::ECONNRESET will be raised when the verify is failure at the
  client side, and it'll be eaten by WEBrick.

* test/open-uri/test_ssl.rb (TestOpenURISSL#test_validation_failure):
  ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48751 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-09 17:51:43 +00:00
akr ad58f04833 * test/open-uri: Test server log in server thread.
* test/webrick: Ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48347 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-09 14:01:20 +00:00
akr 070c310e87 * test/webrick: Store log in an array.
* test/net/http: Ditto.

* test/open-uri: Ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48341 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-09 11:51:06 +00:00
akr 8222432c9d Use assert_join_threads.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48317 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-08 13:41:49 +00:00
akr 600b913232 * test/open-uri/test_open-uri.rb: Check empty webrick log.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48300 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-06 12:23:22 +00:00
akr f4c848e158 * test/open-uri/test_open-uri.rb: Don't ignore webrick's log.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48287 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-05 14:29:46 +00:00
normal 9559b04e3d open-uri: accept :open_timeout option
* lib/open-uri.rb (OpenURI::Options): add :open_timeout default
* (def OpenURI.open_http): check :open_timeout option
* (module OpenURI): rdoc for :open_timeout
* test/open-uri/test_open-uri.rb (test_open_timeout): new test
  [Feature #10361]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47869 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-10 19:15:56 +00:00
hsbt d6f0bd2b84 * lib/open-uri.rb: remove needless condition for old ruby version.
* test/open-uri/test_open-uri.rb: ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-08-08 10:30:19 +00:00
naruse bb83f32dc3 support RFC3986 [Feature #2542]
* lib/uri/common.rb (URI::REGEXP): move to lib/uri/rfc2396_parser.rb.

* lib/uri/common.rb (URI::Parser): ditto.

* lib/uri/common.rb (URI.split): use RFC3986_Parser.

* lib/uri/common.rb (URI.parse): ditto.

* lib/uri/common.rb (URI.join): ditto.

* lib/uri/common.rb (URI.extract): deprecated.

* lib/uri/common.rb (URI.regexp): ditto.

* lib/uri/rfc2396_parser.rb: added.

* lib/uri/rfc3986_parser.rb: added.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46491 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-06-22 00:22:19 +00:00
naruse f977ebb1c9 suppress warnings: using default DH parameters.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-06-22 00:21:37 +00:00
akr f07096f7e1 Join threads.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46281 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-31 14:05:54 +00:00
akr 0c3d513185 * lib/open-uri.rb (OpenURI.open_uri): Call StringIO#close only if
the StringIO object is not closed yet.
  Reported by Jordi Massaguer Pla.  [ruby-core:42538] [Bug #6010]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45835 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-05 16:38:53 +00:00
akr 4329b14cfb * lib/open-uri.rb: Support multiple fields with same field
name (like Set-Cookie).
  (OpenURI::Meta#metas): New accessor to obtain fields as a Hash from
  field name (string) to field values (array of strings).
  [ruby-core:37734] [Bug #4964] reported by ren li.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40207 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-04-09 10:53:41 +00:00
usa a394604412 * test/open-uri/test_open-uri.rb (TestOpenURI#test_read_timeout): this
test expects that the server thread will be killed in sleep, but 0.01
  sec is too short to reach there.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-08-23 02:35:01 +00:00
drbrain 9416becda4 * lib/net/http.rb: Net::HTTP now automatically detects and uses
proxies from the environment.  A proxy may also be specified as
  before.

  Net::HTTP::Proxy still creates anonymous classes, but these classes
  are only used to store configuration information.  When an HTTP
  instance is created the configuration is now copied.

  Additionally, Net::HTTP::ProxyDelta is no longer used by Net::HTTP

  [Feature #6546]
* lib/open-uri.rb:  Moved URI::Generic#find_proxy to uri/generic.
* lib/uri/generic.rb:  Imported find_proxy from open-uri.
* test/open-uri/test_open-uri.rb:  Moved proxy-discovery tests to URI.
* test/uri/test_generic.rb:  Imported proxy-discovery tests from
  open-uri.
* test/net/http/test_http.rb:  Added tests for proxy behavior.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36476 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-20 01:56:21 +00:00
drbrain b1a0509b54 * lib/net/http/response.rb: Automatically inflate gzip and
deflate-encoded response bodies.  [Feature #6942]
* lib/net/http/generic_request.rb:  Automatically accept gzip and
  deflate content-encoding for requests.  [Feature #6494]
* lib/net/http/request.rb:  Updated documentation for #6494.
* lib/net/http.rb:  Updated documentation for #6492 and #6494, removed
  Content-Encoding handling now present in Net::HTTPResponse.
* test/net/http/test_httpresponse.rb:  Tests for #6492
* test/net/http/test_http_request.rb:  Tests for #6494
* test/open-uri/test_open-uri.rb (test_content_encoding):  Updated test
  for automatic content-encoding handling.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36473 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-19 22:43:38 +00:00
nobu 63a164f32e test: fix test conditions
* test/csv/test_features.rb (TestCSV#test_gzip_reader_bug_fix): test
  only if zlib is available.

* test/csv/test_features.rb (TestCSV#test_gzip_writer_bug_fix): ditto.

* test/open-uri/test_open-uri.rb (TestOpenURI#test_content_encoding):
  ditto.

* test/rexml/test_order.rb (OrderTester#test_more_ordering): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35592 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-05-08 06:50:03 +00:00
naruse b712455f98 Wait until the server stops.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35446 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-23 14:05:49 +00:00
nobu 77b1d88513 skip OpenSSL dependent tests if not available
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35440 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-23 13:48:41 +00:00
naruse eab4183f14 Wait until the proxy stops.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35429 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-22 15:04:44 +00:00
naruse 1d6d4d2461 r35304 changes the exception.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-12 02:51:02 +00:00
naruse 66fd3a4eb5 Wait the server until it stops.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35284 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-10 10:19:28 +00:00
naruse 3f443e6115 Use mount_proc to avoid the delay of writing data to files.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35239 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-04 12:49:12 +00:00
naruse 223e0212e9 Sleep 0.5sec to wait the file is written on server.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35237 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-04 07:58:19 +00:00
naruse 58a9bf0c4c Don't call f.close if f is nil or already closed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35229 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-03 19:30:10 +00:00
nobu 3a47cf3395 * remove trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31573 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-05-15 11:55:52 +00:00
usa d815916071 * test/ruby/test_process.rb (TestProcess#test_no_curdir): skip silently on
Windows, because this tests a platform specific feature and it'll never be
  supported on ruby on Windows.

* test/ruby/test_dir_m17n.rb (TestDir_M17N#test_filename_extutf8_invalid,
  TestDir_M17N#test_filename_as_bytes_extutf8): ditto.

* test/open-uri/test_open-uri.rb
  (TestOpenURI#test_find_proxy_case_sensitive_env): ditto.

* test/dl/test_handle.rb (DL::TestHandle#test_NEXT,
  DL::TestHandle#test_DEFAULT): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-03-25 02:50:17 +00:00
tarui e1a52d82b0 * test/open-uri/test_open-uri.rb (test_progress, test_progress_chunked):
fix typo.
	


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-06-11 21:47:41 +00:00
tarui c4f3a04cc1 * test/open-uri/test_open-uri.rb (test_progress, test_progress_chunked):
'size' is a accumulated transferred size in bytes.
	


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28288 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-06-11 21:34:44 +00:00
usa dd32962cba * test/open-uri/test_open-uri.rb (test_find_proxy_case_sensitive_env):
environment variable name is not case sensitive on Windows


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28021 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-05-26 08:25:15 +00:00
akr 136c25ebf0 unset HTTP_PROXY, etc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26064 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-12-10 10:26:55 +00:00
akr e82e2beb7c unset HTTP_PROXY and FTP_PROXY.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26063 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-12-10 10:13:03 +00:00
akr 0cde80d36a * lib/open-uri.rb (OpenURI::Meta#content_type_parse): strip quotes.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24885 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-13 02:29:47 +00:00