Seems clang-7 has false-positives around memory sanitizer.
This line should not be reported because the memory region is
correctly unpoisoned beforehand:
https://travis-ci.org/ruby/ruby/jobs/524766381
Clang 8 seems to fix it.
rb_io_fptr_finalize_internal frees the memory region.
=================================================================
==85264==ERROR: AddressSanitizer: heap-use-after-free on address 0x610000000d8c at pc 0x5608e38077f7 bp 0x7ffee12d5440 sp 0x7ffee12d5438
READ of size 4 at 0x610000000d8c thread T0
#0 0x5608e38077f6 in rb_io_memsize io.c:4749:24
#1 0x5608e37a0481 in obj_memsize_of gc.c:3547:14
#2 0x5608e37a4f30 in check_rvalue_consistency gc.c:1107:2
#3 0x5608e37a2624 in RVALUE_OLD_P gc.c:1218:5
#4 0x5608e37a5bae in rb_gc_force_recycle gc.c:6652:18
#5 0x5608e38191f9 in rb_f_backquote io.c:9021:5
#6 0x5608e3d8aa14 in call_cfunc_1 vm_insnhelper.c:2058:12
#7 0x5608e3d6e23d in vm_call_cfunc_with_frame vm_insnhelper.c:2211:11
#8 0x5608e3d54a35 in vm_call_cfunc vm_insnhelper.c:2229:12
#9 0x5608e3d5253b in vm_call_method_each_type vm_insnhelper.c:2564:9
#10 0x5608e3d51f50 in vm_call_method vm_insnhelper.c:2701:13
#11 0x5608e3cf2de4 in vm_call_general vm_insnhelper.c:2734:12
#12 0x5608e3d79918 in vm_sendish vm_insnhelper.c:3627:11
#13 0x5608e3d06cf5 in vm_exec_core insns.def:789:11
#14 0x5608e3d43700 in rb_vm_exec vm.c:1892:22
#15 0x5608e3d47cbf in rb_iseq_eval_main vm.c:2151:11
#16 0x5608e37620ca in ruby_exec_internal eval.c:262:2
#17 0x5608e376198b in ruby_exec_node eval.c:326:12
#18 0x5608e37617d0 in ruby_run_node eval.c:318:25
#19 0x5608e35c9486 in main main.c:42:9
#20 0x7f62e9421b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
#21 0x5608e3522289 in _start (miniruby+0x15f289)
0x610000000d8c is located 76 bytes inside of 192-byte region [0x610000000d40,0x610000000e00)
freed by thread T0 here:
#0 0x5608e359a2ed in free (miniruby+0x1d72ed)
#1 0x5608e37af421 in objspace_xfree gc.c:9591:5
#2 0x5608e37af3da in ruby_sized_xfree gc.c:9687:2
#3 0x5608e3799ac8 in ruby_xfree gc.c:9694:5
#4 0x5608e380746d in rb_io_fptr_finalize_internal io.c:4728:5
#5 0x5608e38191ed in rb_f_backquote io.c:9020:5
#6 0x5608e3d8aa14 in call_cfunc_1 vm_insnhelper.c:2058:12
#7 0x5608e3d6e23d in vm_call_cfunc_with_frame vm_insnhelper.c:2211:11
#8 0x5608e3d54a35 in vm_call_cfunc vm_insnhelper.c:2229:12
#9 0x5608e3d5253b in vm_call_method_each_type vm_insnhelper.c:2564:9
#10 0x5608e3d51f50 in vm_call_method vm_insnhelper.c:2701:13
#11 0x5608e3cf2de4 in vm_call_general vm_insnhelper.c:2734:12
#12 0x5608e3d79918 in vm_sendish vm_insnhelper.c:3627:11
#13 0x5608e3d06cf5 in vm_exec_core insns.def:789:11
#14 0x5608e3d43700 in rb_vm_exec vm.c:1892:22
#15 0x5608e3d47cbf in rb_iseq_eval_main vm.c:2151:11
#16 0x5608e37620ca in ruby_exec_internal eval.c:262:2
#17 0x5608e376198b in ruby_exec_node eval.c:326:12
#18 0x5608e37617d0 in ruby_run_node eval.c:318:25
#19 0x5608e35c9486 in main main.c:42:9
#20 0x7f62e9421b96 in __libc_start_main
/build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
previously allocated by thread T0 here:
#0 0x5608e359a56d in malloc (miniruby+0x1d756d)
#1 0x5608e37aed12 in objspace_xmalloc0 gc.c:9416:5
#2 0x5608e37aebe7 in ruby_xmalloc0 gc.c:9600:12
#3 0x5608e37aea8b in ruby_xmalloc_body gc.c:9609:12
#4 0x5608e37a6d64 in ruby_xmalloc gc.c:11469:12
#5 0x5608e380e4b4 in rb_io_fptr_new io.c:8040:19
#6 0x5608e380e446 in rb_io_make_open_file io.c:8077:10
#7 0x5608e3850ea0 in pipe_open io.c:6707:5
#8 0x5608e384edb4 in pipe_open_s io.c:6772:12
#9 0x5608e381910b in rb_f_backquote io.c:9014:12
#10 0x5608e3d8aa14 in call_cfunc_1 vm_insnhelper.c:2058:12
#11 0x5608e3d6e23d in vm_call_cfunc_with_frame vm_insnhelper.c:2211:11
#12 0x5608e3d54a35 in vm_call_cfunc vm_insnhelper.c:2229:12
#13 0x5608e3d5253b in vm_call_method_each_type vm_insnhelper.c:2564:9
#14 0x5608e3d51f50 in vm_call_method vm_insnhelper.c:2701:13
#15 0x5608e3cf2de4 in vm_call_general vm_insnhelper.c:2734:12
#16 0x5608e3d79918 in vm_sendish vm_insnhelper.c:3627:11
#17 0x5608e3d06cf5 in vm_exec_core insns.def:789:11
#18 0x5608e3d43700 in rb_vm_exec vm.c:1892:22
#19 0x5608e3d47cbf in rb_iseq_eval_main vm.c:2151:11
#20 0x5608e37620ca in ruby_exec_internal eval.c:262:2
#21 0x5608e376198b in ruby_exec_node eval.c:326:12
#22 0x5608e37617d0 in ruby_run_node eval.c:318:25
#23 0x5608e35c9486 in main main.c:42:9
#24 0x7f62e9421b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
SUMMARY: AddressSanitizer: heap-use-after-free io.c:4749:24 in
rb_io_memsize
Shadow bytes around the buggy address:
0x0c207fff8160: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
0x0c207fff8170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c207fff8180: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
0x0c207fff8190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c207fff81a0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
=>0x0c207fff81b0: fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c207fff81c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c207fff81d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c207fff81e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c207fff81f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c207fff8200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==85264==ABORTING
ec->cfp->iseq might not exist at the very beginning of a thread.
=================================================================
==82954==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fc86f334810 at pc 0x55ceaf013125 bp 0x7ffe2eddbbf0 sp 0x7ffe2eddbbe8
READ of size 8 at 0x7fc86f334810 thread T0
#0 0x55ceaf013124 in vm_check_canary vm_insnhelper.c:217:24
#1 0x55ceaefb4796 in vm_push_frame vm_insnhelper.c:276:5
#2 0x55ceaf0124bd in th_init vm.c:2661:5
#3 0x55ceaf00d5eb in ruby_thread_init vm.c:2690:5
#4 0x55ceaf00d4b1 in rb_thread_alloc vm.c:2703:5
#5 0x55ceaef0038b in thread_s_new thread.c:872:20
#6 0x55ceaf04d8c1 in call_cfunc_m1 vm_insnhelper.c:2041:12
#7 0x55ceaf03118d in vm_call_cfunc_with_frame vm_insnhelper.c:2207:11
#8 0x55ceaf017985 in vm_call_cfunc vm_insnhelper.c:2225:12
#9 0x55ceaf01548b in vm_call_method_each_type vm_insnhelper.c:2560:9
#10 0x55ceaf014c96 in vm_call_method vm_insnhelper.c:2686:13
#11 0x55ceaefb5de4 in vm_call_general vm_insnhelper.c:2730:12
#12 0x55ceaf03c868 in vm_sendish vm_insnhelper.c:3623:11
#13 0x55ceaefc95bb in vm_exec_core insns.def:771:11
#14 0x55ceaf006700 in rb_vm_exec vm.c:1892:22
#15 0x55ceaf00acbf in rb_iseq_eval_main vm.c:2151:11
#16 0x55ceaea250ca in ruby_exec_internal eval.c:262:2
#17 0x55ceaea2498b in ruby_exec_node eval.c:326:12
#18 0x55ceaea247d0 in ruby_run_node eval.c:318:25
#19 0x55ceae88c486 in main main.c:42:9
#20 0x7fc874330b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
#21 0x55ceae7e5289 in _start (miniruby+0x15f289)
0x7fc86f334810 is located 16 bytes to the right of 1048576-byte region [0x7fc86f234800,0x7fc86f334800)
allocated by thread T0 here:
#0 0x55ceae85d56d in malloc (miniruby+0x1d756d)
#1 0x55ceaea71d12 in objspace_xmalloc0 gc.c:9416:5
#2 0x55ceaea71cd2 in ruby_xmalloc2_body gc.c:9623:12
#3 0x55ceaea7d09c in ruby_xmalloc2 gc.c:11479:12
#4 0x55ceaf00c3b7 in rb_thread_recycle_stack vm.c:2462:12
#5 0x55ceaf012256 in th_init vm.c:2656:29
#6 0x55ceaf00d5eb in ruby_thread_init vm.c:2690:5
#7 0x55ceaf00d4b1 in rb_thread_alloc vm.c:2703:5
#8 0x55ceaef0038b in thread_s_new thread.c:872:20
#9 0x55ceaf04d8c1 in call_cfunc_m1 vm_insnhelper.c:2041:12
#10 0x55ceaf03118d in vm_call_cfunc_with_frame vm_insnhelper.c:2207:11
#11 0x55ceaf017985 in vm_call_cfunc vm_insnhelper.c:2225:12
#12 0x55ceaf01548b in vm_call_method_each_type vm_insnhelper.c:2560:9
#13 0x55ceaf014c96 in vm_call_method vm_insnhelper.c:2686:13
#14 0x55ceaefb5de4 in vm_call_general vm_insnhelper.c:2730:12
#15 0x55ceaf03c868 in vm_sendish vm_insnhelper.c:3623:11
#16 0x55ceaefc95bb in vm_exec_core insns.def:771:11
#17 0x55ceaf006700 in rb_vm_exec vm.c:1892:22
#18 0x55ceaf00acbf in rb_iseq_eval_main vm.c:2151:11
#19 0x55ceaea250ca in ruby_exec_internal eval.c:262:2
#20 0x55ceaea2498b in ruby_exec_node eval.c:326:12
#21 0x55ceaea247d0 in ruby_run_node eval.c:318:25
#22 0x55ceae88c486 in main main.c:42:9
#23 0x7fc874330b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
SUMMARY: AddressSanitizer: heap-buffer-overflow vm_insnhelper.c:217:24 in vm_check_canary
Shadow bytes around the buggy address:
0x0ff98de5e8b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0ff98de5e8c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0ff98de5e8d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0ff98de5e8e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0ff98de5e8f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0ff98de5e900: fa fa[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0ff98de5e910: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0ff98de5e920: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0ff98de5e930: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0ff98de5e940: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0ff98de5e950: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==82954==ABORTING
This header is poisoned to detect unintentional buffer overrun.
However in this (and forthcoming) function, we are intentionally
looking at the header. We have to unpoison before anything.
memo.float_value might change inside of hash_sum. In case it
flipped from false to true there, and the calculated sum is Inf,
memo.c might not be initialized at all. This is bad.
Found using memory sanitizer:
==55293==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x55dfb8d6c529 in rb_float_new_inline internal.h:1814:53
#1 0x55dfb8d1b30c in enum_sum enum.c:4017:18
#2 0x55dfb86d75ad in call_cfunc_m1 vm_insnhelper.c:2041:12
#3 0x55dfb864b141 in vm_call_cfunc_with_frame vm_insnhelper.c:2207:11
#4 0x55dfb85e843d in vm_call_cfunc vm_insnhelper.c:2225:12
#5 0x55dfb85e08f3 in vm_call_method_each_type vm_insnhelper.c:2560:9
#6 0x55dfb85de9c7 in vm_call_method vm_insnhelper.c:2686:13
#7 0x55dfb849eac6 in vm_call_general vm_insnhelper.c:2730:12
#8 0x55dfb8686103 in vm_sendish vm_insnhelper.c:3623:11
#9 0x55dfb84dc29e in vm_exec_core insns.def:789:11
autoload_reset() can read this state.result. Because autoload_reset
is a function passed to rb_ensure, there is a chance when an
execption raises before actually filling this memory region.
test/ruby/test_defined.rb:test_autoload_noload is one of such case.
Found using memory sanitizer.
==54014==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x557a683f3e5a in autoload_reset variable.c:2372:9
#1 0x557a6707a93b in rb_ensure eval.c:1084:5
#2 0x557a683efbf5 in rb_autoload_load variable.c:2475:14
#3 0x557a685fc460 in vm_get_ev_const vm_insnhelper.c:938:4
#4 0x557a68448e0a in vm_exec_core insns.def:267:11
Depending on architectures, setjmp might not fully fill a jmp_buf.
On such machines the union can contain wobbly bits. They are then
scanned during mark_locations_array(). This is bad.
These assertions check if a newly allocated object (which is marked
as an uninitialized memory region in MSAN) is in fact a T_NONE.
Thus they intentionally read uninitialized memory regions, which do
not interface well with MSAN. Just disalbe them.
* parse.y (internal_id): number the ID serial for internal use by
counting down from the neary maximum value, not to accidentally
match permanent IDs.
[Bug #15768]
Details: https://github.com/ruby/ruby/pull/2150
Note that this introduction is discussed with @aycabta who is allowed to
make some changes to IRB by the IRB maintainer, keiju.
A pull request based on an old commit may rewind too many files,
even if unnecessary. As rewinding some files, e.g., common header
files, configure.ac, will result in full-rebuild, rebase in a
separate directory to get rid of such rewind.
Urabe, Shyouhei
Nobuyoshi Nakada
git
Takashi Kokubun
Pocket7878
Kazuhiro NISHIYAMA
Lourens Naudé
Kouhei Sutou
Masatoshi SEKI