When Zlib::Inflate#inflate or Zlib::Deflate#deflate is called
recursively inside the block, a crash can occur because of an
use-after-free bug.
https://github.com/ruby/zlib/commit/50fb8a0338
I'm not sure whether this handles all multithreaded use cases,
but this handles the example that crashes almost immediately
and does 10,000,000 total deflates using 100 separate threads.
To prevent the tests from taking forever, the committed test
for this uses only 10,000 deflates across 10 separate threads,
which still causes a segfault in the previous implementation
almost immediately.
Fixes [Bug #17803]
https://github.com/ruby/zlib/commit/4b1023b3f2
Seems like the test `TestZlibGzipFile#test_gzip_reader_zcat` fails when
the timestamp has `\n\n`.
https://ci.appveyor.com/project/ruby/ruby/builds/38597932
```
1) Error:
TestZlibGzipFile#test_gzip_reader_zcat:
Zlib::DataError: invalid distance too far back
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `initialize'
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `new'
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `zcat'
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `block (2 levels) in test_gzip_reader_zcat'
C:/projects/ruby/test/zlib/test_zlib.rb:521:in `open'
C:/projects/ruby/test/zlib/test_zlib.rb:521:in `block in test_gzip_reader_zcat'
C:/projects/ruby/lib/tempfile.rb:358:in `create'
C:/projects/ruby/test/zlib/test_zlib.rb:510:in `test_gzip_reader_zcat'
```
The test time is around 2021-04-08 04:40 +0900. Maybe the following time
should trigger the bug.
```
irb(main):001:0> Time.at(1617824266)
=> 2021-04-08 04:37:46 +0900
irb(main):002:0> [1617824266].pack("V")
=> "\n\nn`"
```
If a buffer keyword argument is given, it is used as the buffer,
instead of creating new strings. This can result in significantly
lower memory usage during inflation.
Implements #19https://github.com/ruby/zlib/commit/dac9a9b57d
Before this change, it was not possible to write out zero for the
timestamp part of a Gzip file's header, as calling GzipWriter#mtime with
zero was ignored.
Judging from the docs for `GzipWriter#mtime=`, it should be possible to
indicate that no timestamp is available by calling the method with zero.
https://github.com/ruby/zlib/commit/310be39cac
Ruby 2.7 deprecates taint and it no longer has an effect.
The lack of taint support should not cause a problem in
previous Ruby versions.
https://github.com/ruby/zlib/commit/21711ed0ce
Apparently, a component of Rails implements a buffering .write
method which keeps the String buffer around and makes it unsafe
for us to clear it after calling .write.
This caused Rack::Deflater to give empty results when enabled.
Fortunately, per r61631 / a55abcc0ca,
this misguided optimization was only worth a small (0.5MB) savings
and we still benefit from the majority of the memory savings in
that change.
Thanks to zunda for the bug report.
[ruby-core:90133] [Bug #15356]
Fixes: r61631 (commit a55abcc0ca)
("zlib: reduce garbage on gzip writes (deflate)")
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66268 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
patched by Andrew Haines <andrew@haines.org.nz> [ruby-core:81488]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59333 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/zlib/zlib.c (zstream): manage capacity and size of `buf`
instead of size and separated member `buf_filled`. reported by
Christian Jalio (jalio) at https://hackerone.com/reports/211958
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58526 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
StringIO.new makes the buffer IO.default_external, while
StringIO.new("".dup) makes source encoding which is defaulted to
UTF-8.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57328 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Skip two tests on AIX because zconf.h in zlib does not correctly
recognize _LARGE_FILES in AIX. The problem was already reported
to zlib, and skip these tests until it is fixed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When you change this to true, you may need to add more tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/zlib/zlib.c (gzfile_reset): preserve ZSTREAM_FLAG_GZFILE
[Bug #10101]
* test/zlib/test_zlib.rb (test_rewind): test each_byte
We must preserve the ZSTREAM_FLAG_GZFILE flag to prevent
zstream_detach_buffer from:
a) returning Qnil and breaking out of the `each_byte' loop
b) yielding a large string to each_byte
Note: the test case in bug report takes a long time. I found this
bug because I noticed the massive time descrepancy between
`each_byte' and `readbyte' loop before this patch. With this patch,
`each_byte' and `readbyte' both take very long.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47327 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
need a dictionary but are being decompressed by Zlib::Inflate.inflate
(which has no option to set a dictionary). Now Zlib::NeedDict is
raised instead of crashing. [ruby-trunk - Bug #8829]
* test/zlib/test_zlib.rb (TestZlibInflate): Test for the above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42720 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This allows zlib streams to be processed without huge memory growth.
[Feature #6612]
* NEWS: ditto
* ext/zlib/zlib.c (zstream_expand_buffer): Uses rb_yield when a block
is given for streaming support. Refactored to use
zstream_expand_buffer_into to remove duplicate code.
* ext/zlib/zlib.c (zstream_expand_buffer_protect): Added wrapper
function to pass jump state back through GVL-free section to allow
zstream clean-up before terminating the ruby call.
* ext/zlib/zlib.c (zstream_expand_buffer_without_gvl): Acquire GVL to
yield processed chunk of output stream.
* ext/zlib/zlib.c (zstream_detach_buffer): When a block is given,
returns Qnil mid-stream and yields the output buffer at the end of
the stream.
* test/zlib/test_zlib.rb: Updated tests
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36356 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
processing. rb_block_given_p() is not callable without the GVL.
* ext/zlib/extconf.rb: ditto
* NEWS: ditto
* test/zlib/test_zlib.rb: ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This allows zlib streams to be processed without huge memory growth.
[Feature #6612]
* NEWS: ditto
* ext/zlib/zlib.c (zstream_expand_buffer): Uses rb_yield when a block
is given for streaming support. Refactored to use
zstream_expand_buffer_into to remove duplicate code.
* ext/zlib/zlib.c (zstream_expand_buffer_protect): Added wrapper
function to pass jump state back through GVL-free section to allow
zstream clean-up before terminating the ruby call.
* ext/zlib/zlib.c (zstream_expand_buffer_without_gvl): Acquire GVL to
yield processed chunk of output stream.
* ext/zlib/zlib.c (zstream_detach_buffer): When a block is given,
returns Qnil mid-stream and yields the output buffer at the end of
the stream.
* ext/zlib/extconf.rb: Update INCFLAGS to find internal.h for
rb_thread_call_with_gvl
* test/zlib/test_zlib.rb: Updated tests
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36349 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Nobuyoshi Nakada
Samuel Williams
Hiroshi SHIBATA
Peter Zhu
Jeremy Evans
Yusuke Endoh
Sorah Fukumori
Alan Wu
normal
mame
nobu
naruse
kazu
odaira
glass
akr
hsbt
drbrain