lib/net/protocol: clear short-lived read buffer

Using a parallel Net::HTTP downloader, this reduced memory usage
from around 120MB to 50MB on my 32-bit x86 system.

* lib/net/protocol.rb (rbuf_fill): clear temporary buffer

Test script I used:

  require 'net/http'
  require 'uri'
  require 'digest/sha1'
  url = 'http://80x24.org/git-i-forgot-to-pack/objects/pack/pack-97b25a76c03b489d4cbbd85b12d0e1ad28717e55.idx'
  uri = URI(url)
  use_ssl = "https" == uri.scheme
  thrs = 30.times.map do
    Thread.start do
      cur = Thread.current.object_id
      Net::HTTP.start(uri.host, uri.port, use_ssl: use_ssl) do |http|
        req = Net::HTTP::Get.new(uri)
        http.request(req) do |res|
          dig = Digest::SHA1.new
          res.read_body do |buf|
            dig.update(buf)
            #buf.clear # most Ruby programmers don't do this :<
          end
          warn "#{Time.now} #{cur} #{dig.hexdigest}\n"
        end
      end
      :done
    end
  end

  p thrs.map(&:value)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58840 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit is contained in:
normal 2017-05-21 23:52:08 +00:00
Родитель 7406d7cffa
Коммит 8623ba8618
1 изменённых файлов: 3 добавлений и 1 удалений

Просмотреть файл

@ -171,7 +171,9 @@ module Net # :nodoc:
def rbuf_fill
case rv = @io.read_nonblock(BUFSIZE, exception: false)
when String
return @rbuf << rv
@rbuf << rv
rv.clear
return
when :wait_readable
@io.to_io.wait_readable(@read_timeout) or raise Net::ReadTimeout
# continue looping