28a1c4f33e seems to call an improper
ensure clause. [Bug #20655]
Than fixing it properly, I bet it would be much better to simply revert
that commit. It reduces the unneeded complexity. Jumping into a block
called by a C function like Hash#each with callcc is user's fault.
It does not need serious support.
The macro SafeStringValue() became just StringValue() in c5c05460ac,
and it is deprecated nowadays.
This patch replaces remaining macro usage. Some occurrences are left in
ext/stringio and ext/win32ole, they should be fixed upstream.
The macro itself is not deleted, because it may be used in extensions.
[Bug #20145]
Before this commit, both copy_compare_by_id and hash_copy will create a
copy of the ST table, so the ST table created in copy_compare_by_id will
be leaked.
h = { 1 => 2 }.compare_by_identity
10.times do
1_000_000.times do
h.select { false }
end
puts `ps -o rss= -p #{$$}`
end
Before:
110736
204352
300272
395520
460704
476736
542000
604704
682624
770528
After:
15504
16048
16144
16256
16320
16320
16752
16752
16752
16752
If a Hash is non-empty, there's no point calling `ar_force_convert_table`. We'll be immediately discarding that new st table, and replacing it with the new `identtable` st table that we're stealing out of `tmp`.
For non-empty Hashes, this function needs to rehash all the stored values (using the new `compare` and `hash` functions from `identhash`). It does so by writing into a newly allocated `tmp` Hash, and then transferring ownership of its st table into `self`.
For empty Hashes, we can skip allocating this `tmp`, because there's nothing to re-hash. We can just modify our new st table's `type` in-place.
* delete `ar_try_convert` but use `ar_force_convert_table`
to make program simple.
* `ar_force_convert_table` checks hash modification while
calling `#hash` method with the following strategy:
1. copy keys (and vals) of ar_table
2. calc hashes from keys
3. check copied keys and hash's keys. if not matched, repeat from 1
fix [Bug #20050]
st.c redefines malloc and free to be ruby_xmalloc and ruby_xfree, so
when this was copied into hash.c it ended up mismatching an xmalloc with
a regular free, which ended up inflating oldmalloc_increase_bytes when
hashes were freed by minor GC.
ar_table may be converted to st_table by `ar_force_convert_table`.
If the conversion occurs during the iteration of ar_table, the iteration
may lead to memory corruption.
This change prevents the catastrophy by throwing an exception when the
conversion is detected.
This issue is reported by [SuperS](https://hackerone.com/superss)
According to the C99 specification section 7.20.3.2 paragraph 2:
> If ptr is a null pointer, no action occurs.
So we do not need to check that the pointer is a null pointer.
st_copy allocates a st_table, which is not needed for hashes since it is
allocated by VWA and embedded, so this causes a memory leak.
The following script demonstrates the issue:
```ruby
20.times do
100_000.times do
{a: 1, b: 2, c: 3, d: 4, e: 5, f: 6, g: 7, h: 8, i: 9}
end
puts `ps -o rss= -p #{$$}`
end
```
Yusuke Endoh
Jean Boussier
Nobuyoshi Nakada
Takashi Kokubun
Peter Zhu
Burdette Lamar
Xavier Noria
Alexander Momchilov
Koichi Sasada
Alan Wu
John Hawthorn
Yuki Tsujimoto
Dorian Marié (perso)