Граф коммитов

47 Коммитов

Автор SHA1 Сообщение Дата
Tanaka Akira 8f7884761e The default charset of text/* media type is UTF-8.
Thanks for the patch  gareth (Gareth Adams).  [Bug #15933]

-------

Combines two small, but very related changes

1: Treat HTTPS the same as HTTP

Previously, OpenURI followed guidance in RFC2616/3.7.1:

> When no explicit charset parameter is provided by the sender, media
> subtypes of the "text" type are defined to have a default charset
> value of "ISO-8859-1" when received via HTTP.

However this RFC was written before TLS was established and OpenURI was
never updated to treat HTTPS traffic the same way. So, HTTPS documents
received a different default to HTTP documents.

This commit removes the scheme check so that all text/* documents
processed by OpenURI are treated the same way.

In theory this processing gets applied to FTP URIs too, but there's no
mechanism in OpenURI for FTP documents to have Content-Type metadata
appended to them, so this ends up being a no-op.

2: Change default charset for text/* to UTF-8

Replaces the default ISO-8859-1 charset previously defined in RFC2616 (now
obsoleted) with a UTF-8 charset as defined in RFC6838.

Fixes: https://bugs.ruby-lang.org/issues/15933
2019-07-15 09:36:52 +09:00
Tanaka Akira 05aac90a1b Warn open-uri's "open" method at Kernel.
Use URI.open instead.

Thanks for the patch by jeremyevans0 (Jeremy Evans) [Misc #15893].
2019-07-14 17:18:17 +09:00
akr bf287424fd open-uri defines URI.open defined as an alias.
open-uri's Kernel.open will be deprecated in future.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61392 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21 14:15:04 +00:00
akr a2831955b2 lib/open-uri.rb: accept :encoding option as well as encoding in mode string.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60232 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 06:22:53 +00:00
nobu 3408e9e3c3 revert r57323
StringIO.new makes the buffer IO.default_external, while
StringIO.new("".dup) makes source encoding which is defaulted to
UTF-8.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57328 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-14 09:38:56 +00:00
nobu c8a4b57254 StringIO.new makes a RW buffer
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57323 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-14 02:06:22 +00:00
hsbt e79d163caf * lib/open-uri.rb: Use `userinfo` for authenticated proxy.
[fix GH-1148] Patch by @SokichiFujita
* test/open-uri/test_open-uri.rb: ditto.
  [fix GH-1309] Patch by @jdamick

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54432 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-03-30 00:46:05 +00:00
nobu 8d66627161 leakchecker.rb: remove temporary measure
* lib/webrick/utils.rb (WEBrick::Utils::TimeoutHandler#watcher):
  make watcher thread restartable.

* lib/webrick/utils.rb (WEBrick::Utils::TimeoutHandler#terminate):
  new method to terminate watcher thread.

* test/lib/leakchecker.rb (LeakChecker#find_threads): revert
  r46941.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53439 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-01-05 06:09:17 +00:00
nobu c81b224edc test: use String#b instead of dup.force_encoding
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53109 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-14 06:40:55 +00:00
akr 68ebbbfebe * lib/open-uri.rb: Remove indicator for "frozen_string_literal: true".
* lib/pp.rb: Ditto.

* lib/prettyprint.rb: Ditto.

* lib/resolv.rb: Ditto.

* lib/securerandom.rb: Ditto.

* lib/tmpdir.rb: Ditto.

* lib/unicode_normalize/tables.rb: Ditto.

* test/net/ftp/test_buffered_socket.rb: Ditto.

* test/net/ftp/test_mlsx_entry.rb: Ditto.

* test/open-uri/test_open-uri.rb: Ditto.

* test/open-uri/test_ssl.rb: Ditto.

* test/pathname/test_pathname.rb: Ditto.

* test/test_pp.rb: Ditto.

* test/test_prettyprint.rb: Ditto.

* tool/transcode-tblgen.rb: Ditto.

* ext/pathname/lib/pathname.rb: Ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52526 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-11-10 11:48:14 +00:00
akr 89db37cf77 * lib/open-uri.rb: Specify frozen_string_literal: true.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52186 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-10-18 12:17:52 +00:00
nobu ae042f21fb use assert_raise
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50727 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-02 02:18:44 +00:00
akr ad58f04833 * test/open-uri: Test server log in server thread.
* test/webrick: Ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48347 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-09 14:01:20 +00:00
akr 070c310e87 * test/webrick: Store log in an array.
* test/net/http: Ditto.

* test/open-uri: Ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48341 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-09 11:51:06 +00:00
akr 8222432c9d Use assert_join_threads.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48317 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-08 13:41:49 +00:00
akr 600b913232 * test/open-uri/test_open-uri.rb: Check empty webrick log.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48300 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-06 12:23:22 +00:00
akr f4c848e158 * test/open-uri/test_open-uri.rb: Don't ignore webrick's log.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48287 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-05 14:29:46 +00:00
normal 9559b04e3d open-uri: accept :open_timeout option
* lib/open-uri.rb (OpenURI::Options): add :open_timeout default
* (def OpenURI.open_http): check :open_timeout option
* (module OpenURI): rdoc for :open_timeout
* test/open-uri/test_open-uri.rb (test_open_timeout): new test
  [Feature #10361]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47869 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-10 19:15:56 +00:00
hsbt d6f0bd2b84 * lib/open-uri.rb: remove needless condition for old ruby version.
* test/open-uri/test_open-uri.rb: ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-08-08 10:30:19 +00:00
naruse bb83f32dc3 support RFC3986 [Feature #2542]
* lib/uri/common.rb (URI::REGEXP): move to lib/uri/rfc2396_parser.rb.

* lib/uri/common.rb (URI::Parser): ditto.

* lib/uri/common.rb (URI.split): use RFC3986_Parser.

* lib/uri/common.rb (URI.parse): ditto.

* lib/uri/common.rb (URI.join): ditto.

* lib/uri/common.rb (URI.extract): deprecated.

* lib/uri/common.rb (URI.regexp): ditto.

* lib/uri/rfc2396_parser.rb: added.

* lib/uri/rfc3986_parser.rb: added.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46491 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-06-22 00:22:19 +00:00
akr f07096f7e1 Join threads.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46281 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-31 14:05:54 +00:00
akr 0c3d513185 * lib/open-uri.rb (OpenURI.open_uri): Call StringIO#close only if
the StringIO object is not closed yet.
  Reported by Jordi Massaguer Pla.  [ruby-core:42538] [Bug #6010]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45835 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-05 16:38:53 +00:00
akr 4329b14cfb * lib/open-uri.rb: Support multiple fields with same field
name (like Set-Cookie).
  (OpenURI::Meta#metas): New accessor to obtain fields as a Hash from
  field name (string) to field values (array of strings).
  [ruby-core:37734] [Bug #4964] reported by ren li.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40207 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-04-09 10:53:41 +00:00
usa a394604412 * test/open-uri/test_open-uri.rb (TestOpenURI#test_read_timeout): this
test expects that the server thread will be killed in sleep, but 0.01
  sec is too short to reach there.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-08-23 02:35:01 +00:00
drbrain 9416becda4 * lib/net/http.rb: Net::HTTP now automatically detects and uses
proxies from the environment.  A proxy may also be specified as
  before.

  Net::HTTP::Proxy still creates anonymous classes, but these classes
  are only used to store configuration information.  When an HTTP
  instance is created the configuration is now copied.

  Additionally, Net::HTTP::ProxyDelta is no longer used by Net::HTTP

  [Feature #6546]
* lib/open-uri.rb:  Moved URI::Generic#find_proxy to uri/generic.
* lib/uri/generic.rb:  Imported find_proxy from open-uri.
* test/open-uri/test_open-uri.rb:  Moved proxy-discovery tests to URI.
* test/uri/test_generic.rb:  Imported proxy-discovery tests from
  open-uri.
* test/net/http/test_http.rb:  Added tests for proxy behavior.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36476 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-20 01:56:21 +00:00
drbrain b1a0509b54 * lib/net/http/response.rb: Automatically inflate gzip and
deflate-encoded response bodies.  [Feature #6942]
* lib/net/http/generic_request.rb:  Automatically accept gzip and
  deflate content-encoding for requests.  [Feature #6494]
* lib/net/http/request.rb:  Updated documentation for #6494.
* lib/net/http.rb:  Updated documentation for #6492 and #6494, removed
  Content-Encoding handling now present in Net::HTTPResponse.
* test/net/http/test_httpresponse.rb:  Tests for #6492
* test/net/http/test_http_request.rb:  Tests for #6494
* test/open-uri/test_open-uri.rb (test_content_encoding):  Updated test
  for automatic content-encoding handling.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36473 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-19 22:43:38 +00:00
nobu 63a164f32e test: fix test conditions
* test/csv/test_features.rb (TestCSV#test_gzip_reader_bug_fix): test
  only if zlib is available.

* test/csv/test_features.rb (TestCSV#test_gzip_writer_bug_fix): ditto.

* test/open-uri/test_open-uri.rb (TestOpenURI#test_content_encoding):
  ditto.

* test/rexml/test_order.rb (OrderTester#test_more_ordering): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35592 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-05-08 06:50:03 +00:00
naruse 1d6d4d2461 r35304 changes the exception.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-12 02:51:02 +00:00
naruse 66fd3a4eb5 Wait the server until it stops.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35284 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-10 10:19:28 +00:00
naruse 3f443e6115 Use mount_proc to avoid the delay of writing data to files.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35239 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-04 12:49:12 +00:00
naruse 223e0212e9 Sleep 0.5sec to wait the file is written on server.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35237 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-04 07:58:19 +00:00
naruse 58a9bf0c4c Don't call f.close if f is nil or already closed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35229 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-04-03 19:30:10 +00:00
usa d815916071 * test/ruby/test_process.rb (TestProcess#test_no_curdir): skip silently on
Windows, because this tests a platform specific feature and it'll never be
  supported on ruby on Windows.

* test/ruby/test_dir_m17n.rb (TestDir_M17N#test_filename_extutf8_invalid,
  TestDir_M17N#test_filename_as_bytes_extutf8): ditto.

* test/open-uri/test_open-uri.rb
  (TestOpenURI#test_find_proxy_case_sensitive_env): ditto.

* test/dl/test_handle.rb (DL::TestHandle#test_NEXT,
  DL::TestHandle#test_DEFAULT): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-03-25 02:50:17 +00:00
tarui e1a52d82b0 * test/open-uri/test_open-uri.rb (test_progress, test_progress_chunked):
fix typo.
	


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-06-11 21:47:41 +00:00
tarui c4f3a04cc1 * test/open-uri/test_open-uri.rb (test_progress, test_progress_chunked):
'size' is a accumulated transferred size in bytes.
	


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28288 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-06-11 21:34:44 +00:00
usa dd32962cba * test/open-uri/test_open-uri.rb (test_find_proxy_case_sensitive_env):
environment variable name is not case sensitive on Windows


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28021 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-05-26 08:25:15 +00:00
akr e82e2beb7c unset HTTP_PROXY and FTP_PROXY.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26063 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-12-10 10:13:03 +00:00
akr 0cde80d36a * lib/open-uri.rb (OpenURI::Meta#content_type_parse): strip quotes.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24885 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-13 02:29:47 +00:00
akr 945b604e3a extra commas removed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24877 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12 16:33:44 +00:00
akr 0466f33486 test read_timeout.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24876 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12 16:26:05 +00:00
akr db2be749b4 * lib/open-uri.rb (URI::FTP#buffer_open): fix the %2F handling.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24875 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12 16:18:03 +00:00
akr 33d5c13775 test invalid redirect location.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24872 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12 15:34:35 +00:00
akr 0ab4b21d81 fix tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24869 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12 15:22:44 +00:00
akr 797749c219 test ftp over http proxy.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24866 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12 14:19:58 +00:00
akr 188e9667dc * lib/open-uri.rb (URI::FTP#buffer_open): use the port specified in
the URI.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24865 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12 13:49:07 +00:00
akr 5ee445bc0c test proxy log.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24863 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12 11:07:05 +00:00
akr f34cf072a4 add https test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24862 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12 09:50:57 +00:00