This accounts for a significant number of string allocations when reading rubygems, but we can avoid that in many places by only copying into the outbuf when present
https://github.com/ruby/zlib/commit/d25ef406c1
This commit fixes the test failures on the zlib in Ubuntu jammy s390x.
According to the <https://packages.ubuntu.com/jammy-updates/zlib1g> -
`zlib_1.2.11.dfsg-2ubuntu9.2.debian.tar.xz`,
the zlib deb package is applying the patch 410.patch (madler/zlib#410), and
configured by `./configure --dfltcc` on Ubuntu jammy s390x. The `--dfltcc`
is to enable the deflate algorithm in hardware.
It produces a different (but still valid) compressed byte stream, and causes
the test failures in ruby/zlib. As a workaround, set the environment variable
`DFLTCC=0` disabling the implementation in zlib on s390x to the failing tests.
Note we need to test in a child Ruby process with `assert_separately` to test
on the `DFLTCC=0` set by the parent Ruby process.
https://github.com/ruby/zlib/commit/9f3b9c470c
(https://github.com/ruby/zlib/pull/55)
zstream_discard_input was encoding and character-aware when given input is user-provided, so this discards `len` chars instead of `len` bytes.
Also Zlib.gunzip explains in its rdoc that it is equivalent with the following code, but this doesn't fail for UTF-8 String.
```ruby
string = %w[1f8b0800c28000000003cb48cdc9c9070086a6103605000000].pack("H*").force_encoding('UTF-8')
sio = StringIO.new(string)
p gz.read #=> "hello"
gz&.close
p Zlib.gunzip(string) #=> Zlib::DataError
```
Reported and discovered by eagletmt at https://twitter.com/eagletmt/status/1689692467929694209https://github.com/ruby/zlib/commit/c5e58bc62a
When Zlib::Inflate#inflate or Zlib::Deflate#deflate is called
recursively inside the block, a crash can occur because of an
use-after-free bug.
https://github.com/ruby/zlib/commit/50fb8a0338
I'm not sure whether this handles all multithreaded use cases,
but this handles the example that crashes almost immediately
and does 10,000,000 total deflates using 100 separate threads.
To prevent the tests from taking forever, the committed test
for this uses only 10,000 deflates across 10 separate threads,
which still causes a segfault in the previous implementation
almost immediately.
Fixes [Bug #17803]
https://github.com/ruby/zlib/commit/4b1023b3f2
Seems like the test `TestZlibGzipFile#test_gzip_reader_zcat` fails when
the timestamp has `\n\n`.
https://ci.appveyor.com/project/ruby/ruby/builds/38597932
```
1) Error:
TestZlibGzipFile#test_gzip_reader_zcat:
Zlib::DataError: invalid distance too far back
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `initialize'
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `new'
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `zcat'
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `block (2 levels) in test_gzip_reader_zcat'
C:/projects/ruby/test/zlib/test_zlib.rb:521:in `open'
C:/projects/ruby/test/zlib/test_zlib.rb:521:in `block in test_gzip_reader_zcat'
C:/projects/ruby/lib/tempfile.rb:358:in `create'
C:/projects/ruby/test/zlib/test_zlib.rb:510:in `test_gzip_reader_zcat'
```
The test time is around 2021-04-08 04:40 +0900. Maybe the following time
should trigger the bug.
```
irb(main):001:0> Time.at(1617824266)
=> 2021-04-08 04:37:46 +0900
irb(main):002:0> [1617824266].pack("V")
=> "\n\nn`"
```
If a buffer keyword argument is given, it is used as the buffer,
instead of creating new strings. This can result in significantly
lower memory usage during inflation.
Implements #19https://github.com/ruby/zlib/commit/dac9a9b57d
Before this change, it was not possible to write out zero for the
timestamp part of a Gzip file's header, as calling GzipWriter#mtime with
zero was ignored.
Judging from the docs for `GzipWriter#mtime=`, it should be possible to
indicate that no timestamp is available by calling the method with zero.
https://github.com/ruby/zlib/commit/310be39cac
Ruby 2.7 deprecates taint and it no longer has an effect.
The lack of taint support should not cause a problem in
previous Ruby versions.
https://github.com/ruby/zlib/commit/21711ed0ce
Apparently, a component of Rails implements a buffering .write
method which keeps the String buffer around and makes it unsafe
for us to clear it after calling .write.
This caused Rack::Deflater to give empty results when enabled.
Fortunately, per r61631 / a55abcc0ca,
this misguided optimization was only worth a small (0.5MB) savings
and we still benefit from the majority of the memory savings in
that change.
Thanks to zunda for the bug report.
[ruby-core:90133] [Bug #15356]
Fixes: r61631 (commit a55abcc0ca)
("zlib: reduce garbage on gzip writes (deflate)")
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66268 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
patched by Andrew Haines <andrew@haines.org.nz> [ruby-core:81488]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59333 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/zlib/zlib.c (zstream): manage capacity and size of `buf`
instead of size and separated member `buf_filled`. reported by
Christian Jalio (jalio) at https://hackerone.com/reports/211958
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58526 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
StringIO.new makes the buffer IO.default_external, while
StringIO.new("".dup) makes source encoding which is defaulted to
UTF-8.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57328 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Skip two tests on AIX because zconf.h in zlib does not correctly
recognize _LARGE_FILES in AIX. The problem was already reported
to zlib, and skip these tests until it is fixed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When you change this to true, you may need to add more tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/zlib/zlib.c (gzfile_reset): preserve ZSTREAM_FLAG_GZFILE
[Bug #10101]
* test/zlib/test_zlib.rb (test_rewind): test each_byte
We must preserve the ZSTREAM_FLAG_GZFILE flag to prevent
zstream_detach_buffer from:
a) returning Qnil and breaking out of the `each_byte' loop
b) yielding a large string to each_byte
Note: the test case in bug report takes a long time. I found this
bug because I noticed the massive time descrepancy between
`each_byte' and `readbyte' loop before this patch. With this patch,
`each_byte' and `readbyte' both take very long.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47327 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
need a dictionary but are being decompressed by Zlib::Inflate.inflate
(which has no option to set a dictionary). Now Zlib::NeedDict is
raised instead of crashing. [ruby-trunk - Bug #8829]
* test/zlib/test_zlib.rb (TestZlibInflate): Test for the above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42720 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Samuel Giddins
Nobuyoshi Nakada
Martin Emde
Jun Aruga
Jun Aruga
Benoit Daloze
Sorah Fukumori
Samuel Williams
Hiroshi SHIBATA
Peter Zhu
Jeremy Evans
Yusuke Endoh
Alan Wu
normal
mame
nobu
naruse
kazu
odaira
glass
akr
hsbt
drbrain