Before, if dln_open failed to open RUBY_GC_LIBRARY_PATH, it would segfault
because it would try to raise an error, which cannot happen because the
GC has not been initialized yet.
This commit changes dln_open to return the error that occurred so the
caller can handle the error.
When we're searching for CCs, compare the argc and flags for CI rather
than comparing pointers. This means we don't need to store a reference
to the CI, and it also naturally "de-duplicates" CC objects.
We can observe the effect with the following code:
```ruby
require "objspace"
hash = {}
p ObjectSpace.memsize_of(Hash)
eval ("a".."zzz").map { |key|
"hash.merge(:#{key} => 1)"
}.join("; ")
p ObjectSpace.memsize_of(Hash)
```
On master:
```
$ ruby -v test.rb
ruby 3.4.0dev (2024-04-15T16:21:41Z master d019b3baec) [arm64-darwin23]
test.rb:3: warning: assigned but unused variable - hash
3424
527736
```
On this branch:
```
$ make runruby
compiling vm.c
linking miniruby
builtin_binary.inc updated
compiling builtin.c
linking static-library libruby.3.4-static.a
ln -sf ../../rbconfig.rb .ext/arm64-darwin23/rbconfig.rb
linking ruby
ld: warning: ignoring duplicate libraries: '-ldl', '-lobjc', '-lpthread'
RUBY_ON_BUG='gdb -x ./.gdbinit -p' ./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems ./test.rb
2240
2368
```
Co-authored-by: John Hawthorn <jhawthorn@github.com>
RGENGC_CHECK_MODE >=3 fails with an incinsistency in the old object
count during ec_finalization.
This is due to inconsistency introduced to the object graph using T_DATA
finalizers.
This is explained in commit 79df14c04b,
which disabled gc during finalization to work around this.
```
/* prohibit GC because force T_DATA finalizers can break an object graph consistency */
dont_gc_on()
```
This object graph inconsistency also seems to break RGENGC_CHECK_MODE >=
3, when it attempt to verify the object age relationships during
finalization at VM shutdown (gc_enter is called during finalization).
This commit stops the internal consistency check during gc_enter only
when RGENGC_CHECK_MODE >= 3 and when gc is disabled.
This fixes `make btest` with `-DRGENGC_CHECK_MODE=3`
This PR moves `rb_copy_wb_protected_attribute` and
`rb_gc_copy_finalizer` into a single function called
`rb_gc_copy_attributes` to be called by `init_copy`. This reduces the
surface area of the GC API.
Co-authored-by: Peter Zhu <peter@peterzhu.ca>
Currently, we check the values on the machine stack & register state to
see if they're actually a pointer to an ASAN fake stack, and mark the
values on the fake stack too if required. However, we are only doing
that for the _current_ thread (the one actually running the GC), not for
any other thread in the program.
Make rb_gc_mark_machine_context (which is called for marking non-current
threads) perform the same ASAN fake stack handling that
mark_current_machine_context performs.
[Bug #20310]
This `st_table` is used to both mark and pin classes
defined from the C API. But `vm->mark_object_ary` already
does both much more efficiently.
Currently a Ruby process starts with 252 rooted classes,
which uses `7224B` in an `st_table` or `2016B` in an `RArray`.
So a baseline of 5kB saved, but since `mark_object_ary` is
preallocated with `1024` slots but only use `405` of them,
it's a net `7kB` save.
`vm->mark_object_ary` is also being refactored.
Prior to this changes, `mark_object_ary` was a regular `RArray`, but
since this allows for references to be moved, it was marked a second
time from `rb_vm_mark()` to pin these objects.
This has the detrimental effect of marking these references on every
minors even though it's a mostly append only list.
But using a custom TypedData we can save from having to mark
all the references on minor GC runs.
Addtionally, immediate values are now ignored and not appended
to `vm->mark_object_ary` as it's just wasted space.
This frees FL_USER0 on both T_MODULE and T_CLASS.
Note: prior to this, FL_SINGLETON was never set on T_MODULE,
so checking for `FL_SINGLETON` without first checking that
`FL_TYPE` was `T_CLASS` was valid. That's no longer the case.
is_markable_object is called by rb_objspace_markable_object_p, which
may pass a T_NONE object. check_rvalue_consistency will fail if a T_NONE
object is passed in.
The name is misleading, as it seems like the function checks whether the
object is swept or not. But the function only checks whether the page is
before or after sweeping.
The FL_FINALIZE flag is set when there are finalizers for the object. We
can improver performance by not looking up in the table if the flag is
not set.
Using the following C extension:
#include "ruby/ruby.h"
static void data_free(void *_ptr) {}
static const rb_data_type_t data_type = {
"my_type",
{
NULL,
data_free,
},
0, 0, 0
};
static VALUE data_alloc(VALUE klass) {
return TypedData_Wrap_Struct(klass, &data_type, (void *)1);
}
void Init_myext(void) {
VALUE my_klass = rb_define_class("MyClass", rb_cObject);
rb_define_alloc_func(my_klass, data_alloc);
}
And the following benchmark:
require "benchmark"
final_objs = 1_000_000.times.map do
o = Object.new
ObjectSpace.define_finalizer(o, proc {})
o
end
puts(Benchmark.measure do
100_000_000.times do
MyClass.new
end
end)
Before:
10.974190 0.355037 11.329227 ( 11.416772)
After:
7.664310 0.347598 8.011908 ( 8.268969)
Rather than exposing that an imemo has a flag and four fields, this
changes the implementation to only expose one field (the klass) and
fills the rest with 0. The type will have to fill in the values themselves.
The switch statement is not exhaustive, meaning the "unreachable"
comment was not correct. This commit fixes it by making the list
exhaustive and adding an rb_bug in the default case.
Previously every call to vm_ci_new (when the CI was not packable) would
result in a different callinfo being returned this meant that every
kwarg callsite had its own CI.
When calling, different CIs result in different CCs. These CIs and CCs
both end up persisted on the T_CLASS inside cc_tbl. So in an eval loop
this resulted in a memory leak of both types of object. This also likely
resulted in extra memory used, and extra time searching, in non-eval
cases.
For simplicity in this commit I always allocate a CI object inside
rb_vm_ci_lookup, but ideally we would lazily allocate it only when
needed. I hope to do that as a follow up in the future.
rb_objspace_call_finalizer didn't free fibers and neither did
rb_objspace_free_objects, which caused fibers to be reported as leaked
when using RUBY_FREE_AT_EXIT. This commit changes rb_objspace_free_objects
to free all remaining Ruby objects.
This returns whether or not _any_ piece of memory in the range is
poisoned, not if _all_ of it is. That means that currently, with ASAN
enabled, pages which contain a single poisoned object are skipped
entirely from being iterated with objspace_each* family of functions.
[Bug #20220]
ASAN leaves a pointer to the fake frame on the stack; we can use the
__asan_addr_is_in_fake_stack API to work out the extent of the fake
stack and thus mark any VALUEs contained therein.
[Bug #20001]
T_DATA without a pointer or free function may still have ivars set on
them that need to be freed. The following leaked generic ivars for
example:
converter = Encoding::Converter.allocate
converter.instance_variable_set(:@foo, 1)
STACK OF 1 INSTANCE OF 'ROOT LEAK: <malloc in objspace_xmalloc0>':
<snip>
12 miniruby 0x10286ec50 ivar_set + 140 variable.c:1850
11 miniruby 0x102876afc generic_ivar_set + 136 variable.c:1668
ASAN leaves a pointer to the fake frame on the stack; we can use the
__asan_addr_is_in_fake_stack API to work out the extent of the fake
stack and thus mark any VALUEs contained therein.
[Bug #20001]
I was trying to debug an (unrelated) issue in the GC, and wanted to turn
on the trace-level GC output by compiling it with -DRGENGC_DEBUG=5.
Unfortunately, this actually causes a crash in newobj_init() because the
code there tries to log the obj_info() of the newly created object.
However, the object is not actually sufficiently set up for some of the
things that obj_info() tries to do:
* The instance variable table for a class is not yet initialized, and
when using variable-length RVALUES, said ivar table is embedded in
as-yet unitialized memory after the struct RValue. Attempting to read
this, as obj_info() does, causes a crash.
* T_DATA variables need to dereference their ->type field to print out
the underlying C type name, which is not set up until newobj_fill() is
called.
To fix this, create a new method `obj_info_basic`, which dumps out only
the parts of the object that are valid before the object is fully
initialized.
[Fixes#18795]
The for loops for marking and reference updating declaratively marked
TypedData objects did not mark/reference update the very last element.
When RGENGC_CHECK_MODE is turned on, this caused the test in Enumerator
to fail with:
tool/lib/test/unit/testcase.rb:173:in `rescue in run': failed to allocate memory (NoMemoryError)
This commit adds `GC.auto_compact = :empty` which will run
auto-compaction sorting pages by empty slots so the most amount of
objects will be moved. This will make it easier to write tests for
auto-compaction.
pinned_slots is not being reset every GC, which causes this assertion to
fail:
```
Assertion Failed: gc.c:7076:gc_pin:GET_HEAP_PAGE(obj)->pinned_slots <= GET_HEAP_PAGE(obj)->total_slots
```
This commit changes it to reset it at the beginning of every compaction
GC cycle.
Previously with RUBY_FREE_ON_EXIT, ractors where being xfree-ed which is incorrect since they are not xmalloced.
Instead we can free ractors with ractor free during shutdown. This change only effects main ractor freeing when RUBY_FREE_ON_EXIT is set.
Co-authored-by: John Hawthorn <john@hawthorn.email>
Previously, T_DATA and T_FILE objects did not have their instance
variables freed on exit which would be reported as a memory leak with
RUBY_FREE_ON_EXIT. This commit changes it to use obj_free which also
frees the generic instance variables.
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
When compiling with cppflags=-DRGENGC_CHECK_MODE, the following crashes:
```
$ RUBY_FREE_ON_EXIT=1 ./miniruby -e 0
-e: [BUG] obj_free: RVALUE_MARKED(0x0000000103570020 [3LM ] T_CLASS (anon)) != FALSE
```
This commit clears the mark bits when rb_free_on_exit is enabled.
Our current implementation of rb_postponed_job_register suffers from
some safety issues that can lead to interpreter crashes (see bug #1991).
Essentially, the issue is that jobs can be called with the wrong
arguments.
We made two attempts to fix this whilst keeping the promised semantics,
but:
* The first one involved masking/unmasking when flushing jobs, which
was believed to be too expensive
* The second one involved a lock-free, multi-producer, single-consumer
ringbuffer, which was too complex
The critical insight behind this third solution is that essentially the
only user of these APIs are a) internal, or b) profiling gems.
For a), none of the usages actually require variable data; they will
work just fine with the preregistration interface.
For b), generally profiling gems only call a single callback with a
single piece of data (which is actually usually just zero) for the life
of the program. The ringbuffer is complex because it needs to support
multi-word inserts of job & data (which can't be atomic); but nobody
actually even needs that functionality, really.
So, this comit:
* Introduces a pre-registration API for jobs, with a GVL-requiring
rb_postponed_job_prereigster, which returns a handle which can be
used with an async-signal-safe rb_postponed_job_trigger.
* Deprecates rb_postponed_job_register (and re-implements it on top of
the preregister function for compatability)
* Moves all the internal usages of postponed job register
pre-registration
when the RUBY_FREE_ON_SHUTDOWN environment variable is set, manually free memory at shutdown.
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Peter Zhu <peter@peterzhu.ca>
need_major_gc is set when a major GC is required. However, if
gc_stress_no_major is also set, then it will not actually run a major
GC.
For example, the following script will sometimes crash:
```
GC.stress = 1
50000.times.map { [] }
```
With the following message:
```
[BUG] cannot create a new page after major GC
```
The intention of GC.verify_compaction_references is, I believe, to force
every single movable object to be moved, so that it's possible to debug
native extensions which not correctly updating their references to
objects they mark as movable.
To do this, it doubles the number of allocated pages for each size pool,
and sorts the heap pages so that the free ones are swept first; thus,
every object in an old page should be moved into a free slot in one of
the new pages.
This worked fine until movement of objects _between_ size pools during
compaction was implemented. That causes some problems for
verify_compaction_references:
* We were doubling the number of pages in each size pool, but actually
if some objects need to move into a _different_ pool, there's no
guarantee that they'll be enough room in that one.
* It's possible for the sweep & compact cursors to meet in one size pool
before all the objects that want to move into that size pool from
another are processed by the compaction.
You can see these problems by changing some of the movement tests in
test_gc_compact.rb to try and move e.g. 50,000 objects instead of
500; the test is not able to actually move all of the objects in a
single compaction run.
To fix this, we do two things in verify_compaction_references:
* Firstly, we add enough pages to every size pool to make them the same
size. This ensures that their compact cursors will all have space to
move during compaction (even if that means empty pages are
pointlessly compacted)
* Then, we examine every object and determine where it _wants_ to be
compacted into. We use this information to add additional pages to
each size pool to handle all objects which should live there.
With these two changes, we can move arbitrary amounts of objects into
the correct size pool in a single call to verify_compaction_references.
My _motivation_ for performing this work was to try and fix some test
stability issues in test_gc_compact.rb. I now no longer think that we
actually see this particular bug in rubyci.org, but I also think
verify_compaction_references should do what it says on the tin, so it's
worth keeping.
[Bug #20022]
This works like objspace_each_obj, except instead of being called with
the start & end address of each page, it's called with the page
structure itself.
[Bug #20022]
Objects with the same shape must always have the same "embeddedness"
(either embedded or heap allocated) because YJIT assumes so. However,
using remove_instance_variable, it's possible that some objects are
embedded and some are heap allocated because it does not re-embed heap
allocated objects.
This commit changes remove_instance_variable to re-embed Object
instance variables when it becomes small enough.
Embedded shared strings cannot be moved because strings point into the
slot of the shared string. There may be code using the RSTRING_PTR on
the stack, which would pin the string but not pin the shared string,
causing it to move.
When generic instance variable has a shape, it is marked movable. If it
it transitions to too complex, it needs to update references otherwise
it may have incorrect references.
This is required for the same reason that super CC needs it.
See 36023d5cb7.
Reproducer:
def cached_foo_callsite(obj) = obj.foo
class Foo
def foo = :v1
module R
refine Foo do
def foo = :unused
end
end
end
obj = Foo.new
cached_foo_callsite(obj) # set up cc with cme for foo=:v1
class Foo
def foo = :v2
end
GC.start # cme for foo=:v1 collected, if not reachable by cached_foo_callsite
cached_foo_callsite(obj)
[Bug #19994]
On large Ruby applications, shutdown may be slow if a major GC has just
started because rb_objspace_call_finalizer completes the GC.
This commit adds gc_abort which discards the mark stack if during
incremental marking and stops sweeping if during lazy sweeping.
Previously, because gc_update_object_references() did not update the
VALUEs in the too_complex ivar st_table for T_CLASS and T_MODULE
objects, GC compaction could finish with corrupted objects.
- start with `klass`, not too_complex
- GC incremental step marks `klass` and its ivars
- ruby code makes `klass` too_complex
- GC compaction runs and move `klass` ivars, but because `klass` is
too_complex, its ivars are not updated by gc_update_object_references(),
leaving T_NONE or T_MOVED objects in the ivar table.
Co-authored-by: Peter Zhu <peter@peterzhu.ca>
Marking both keys and values versus marking just values is an important
distinction, but previously, gc_update_tbl_refs() and gc_update_table_refs()
had names that were too similar.
The st_table storing ivars for too_complex T_OBJECTs have IDs as keys,
but we were marking the IDs unnecessary previously, maybe due to the
confusing naming.
Previously, it tripped the assert about too_complex in
ROBJECT_IV_CAPACITY(). This fixes double faults for some crashes and
helps with use during development.
Since the callback defined in the objspace module might give up the GVL,
we need to make sure the right cr->mfd value is set back after the GVL
is re-obtained.
The previous implementation was using the pointer given
by `DATA_PTR` in all cases. But in the case of an embedded
TypedData, that pointer is garbage, we need to use RTYPEDDATA_GET_DATA
to get the proper data pointer.
Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
This commit adds a new flag RUBY_TYPED_EMBEDDABLE that allows the data
of a TypedData object to be embedded after the object itself. This will
improve cache locality and allow us to save the 8 byte data pointer.
Co-Authored-By: Jean Boussier <byroot@ruby-lang.org>
Tracks other callinfo that references the same kwargs and frees them when all references are cleared.
[bug #19906]
Co-authored-by: Peter Zhu <peter@peterzhu.ca>
Peter Zhu
Matt Valentine-House
Aaron Patterson
Nobuyoshi Nakada
KJ Tsanaktsidis
eileencodes
Jean Boussier
John Hawthorn
Alan Wu
KJ Tsanaktsidis
Rian McGuire
HParker
Koichi Sasada
Adam Hess
Soutaro Matsumoto
Jean Boussier
Yusuke Endoh