tool/downloader.rb: retry downloads

because it's randomly failing on CI like
https://ci.appveyor.com/project/ruby/ruby/build/1.0.6724

Actually I'm not sure whether the exception class is Errno::ECONNREFUSED
or not. Please change the rescued exception to the correct one if it's
wrong. I changed to log exception class too in this commit.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61498 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit is contained in:
k0kubun 2017-12-27 13:32:59 +00:00
Родитель c46d3de0f5
Коммит ff6c540adb
1 изменённых файлов: 19 добавлений и 2 удалений

Просмотреть файл

@ -161,7 +161,9 @@ class Downloader
$stdout.flush $stdout.flush
end end
begin begin
data = url.read(options.merge(http_options(file, since.nil? ? true : since))) data = with_retry(3, Errno::ECONNREFUSED) do
url.read(options.merge(http_options(file, since.nil? ? true : since)))
end
rescue OpenURI::HTTPError => http_error rescue OpenURI::HTTPError => http_error
if http_error.message =~ /^304 / # 304 Not Modified if http_error.message =~ /^304 / # 304 Not Modified
if $VERBOSE if $VERBOSE
@ -207,7 +209,7 @@ class Downloader
end end
return file.to_path return file.to_path
rescue => e rescue => e
raise "failed to download #{name}\n#{e.message}: #{url}" raise "failed to download #{name}\n#{e.class}: #{e.message}: #{url}"
end end
def self.under(dir, name) def self.under(dir, name)
@ -264,6 +266,21 @@ class Downloader
end end
end end
end end
def self.with_retry(max_times, exception, &block)
times = 0
begin
block.call
rescue exception => e
times += 1
if times <= max_times
$stderr.puts "retrying #{e.class} (#{e.message}) after #{times ** 2} seconds..."
sleep(times ** 2)
retry
end
end
end
private_class_method :with_retry
end end
Downloader.https = https.freeze Downloader.https = https.freeze