Граф коммитов

30 Коммитов

Автор SHA1 Сообщение Дата
Jemma Issroff 5246f4027e Transition shape when object's capacity changes
This commit adds a `capacity` field to shapes, and adds shape
transitions whenever an object's capacity changes. Objects which are
allocated out of a bigger size pool will also make a transition from the
root shape to the shape with the correct capacity for their size pool
when they are allocated.

This commit will allow us to remove numiv from objects completely, and
will also mean we can guarantee that if two objects share shapes, their
IVs are in the same positions (an embedded and extended object cannot
share shapes). This will enable us to implement ivar sets in YJIT using
object shapes.

Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
2022-11-10 10:11:34 -05:00
Matt Valentine-House a6dd859aff Add expand_heap option to GC.verify_compaction_references
In order to reliably test compaction we need to be able to move objects
between size pools.

In order for this to happen there must be pages in a size pool into
which we can allocate.

The existing implementation of `double_heap` only doubled the existing
number of pages in the heap, so if a size pool had a low number of pages
(or 0) it's not guaranteed that enough space will be created to move
objects into that size pool.

This commit deprecates the `double_heap` option and replaces it with
`expand_heap` instead.

expand heap will expand each heap by enough pages to hold a number of
slots defined by `GC_HEAP_INIT_SLOTS` or by `heap->total_pags` whichever
is larger.

If both `double_heap` and `expand_heap` are present, a deprecation
warning will be shown for `double_heap` and the `expand_heap` behaviour
will take precedence

Given that this is an API intended for debugging and testing GC
compaction I'm not concerned about the extra memory usage or time taken
to create the pages. However, for completeness:

Running the following `test.rb` and using `time` on my Macbook Pro shows
the following memory usage and time impact:

pp "RSS (kb): #{`ps -o rss #{Process.pid}`.lines.last.to_i}"
GC.verify_compaction_references(double_heap: true, toward: :empty)
pp "RSS (kb): #{`ps -o rss #{Process.pid}`.lines.last.to_i}"

❯ time make run
./miniruby -I./lib -I. -I.ext/common  -r./arm64-darwin21-fake  ./test.rb
"RSS (kb): 24000"
<internal:gc>:251: warning: double_heap is deprecated and will be removed
"RSS (kb): 25232"

________________________________________________________
Executed in  124.37 millis    fish           external
   usr time   82.22 millis    0.09 millis   82.12 millis
   sys time   28.76 millis    2.61 millis   26.15 millis

❯ time make run
./miniruby -I./lib -I. -I.ext/common  -r./arm64-darwin21-fake  ./test.rb
"RSS (kb): 24000"
"RSS (kb): 49040"

________________________________________________________
Executed in  150.13 millis    fish           external
   usr time  103.32 millis    0.10 millis  103.22 millis
   sys time   35.73 millis    2.59 millis   33.14 millis
2022-07-11 09:00:03 -04:00
Nobuyoshi Nakada b96a3a6fd2
Move `GC.verify_compaction_references` [Bug #18779]
Define `GC.verify_compaction_references` as a built-in ruby method,
according to GC compaction support via `GC::OPTS`.
2022-06-02 15:32:00 +09:00
Mike Dalessio 0de1495f35 Move compaction-related methods into gc.c
These methods are removed from gc.rb and added to gc.c:

- GC.compact
- GC.auto_compact
- GC.auto_compact=
- GC.latest_compact_info
- GC.verify_compaction_references

This is a prefactor to allow setting these methods to
`rb_f_notimplement` in a followup commit.
2022-05-24 09:40:03 -07:00
Peter Zhu 615e9b2865 [Feature #18364] Add GC.stat_heap to get stats for memory heaps
GC.stat_heap will return stats for memory heaps. This is
used for the Variable Width Allocation feature.
2022-01-04 09:46:36 -05:00
Peter Zhu 52525b527b [Bug #18451] [ci skip] Update documentation for GC.stat
Adds documentation for keys `time`, `compact_count`,
`read_barrier_faults`, `total_moved_objects`.
2021-12-29 16:20:50 -05:00
zverok cf83ba1da7 [DOC] Make sure new GC methods are documented 2021-12-24 10:36:29 +09:00
Peter Zhu f8ead3a72f [ci skip] Update documentation for GC.stat 2021-11-29 09:40:30 -05:00
Peter Zhu b0bbcaedc7 Revert "Add GC.stat_size_pool to get stats for a size pool"
This reverts commit 6157619bb6.

We'll wait for comments in the open ticket: https://bugs.ruby-lang.org/issues/18364
2021-11-25 11:01:50 -05:00
Peter Zhu 6157619bb6 Add GC.stat_size_pool to get stats for a size pool
GC.stat_size_pool will return stats for a particular size pool. This is
used for the Variable Width Allocation feature.
2021-11-25 10:33:17 -05:00
Koichi Sasada 6c64013978 use ULL2NUM directly.
@nobu pointed out that ULL (unsigned long long) should have at
least 64 bits so ULL2NUM(uint64_t) is not problem.
2021-11-19 08:32:07 +09:00
Koichi Sasada c347038d4e GC measurement feature
* `GC.measure_total_time = true` enables total time measurement (default: true)
* `GC.measure_total_time` returns current flag.
* `GC.total_time` returns measured total time in nano seconds.
* `GC.stat(:time)` (and Hash) returns measured total time in milli seconds.
2021-11-19 08:32:07 +09:00
Peter Zhu a5b6598192 [Feature #18239] Implement VWA for strings
This commit adds support for embedded strings with variable capacity and
uses Variable Width Allocation to allocate strings.
2021-10-25 13:26:23 -04:00
Jemma Issroff 2a9876a885
Enhance documentation on GC.stat (#4843) 2021-09-20 14:40:55 -04:00
Aaron Patterson 895b10ad02
Add documentation about GC.compact
[Misc #16443][ruby-core:96395]
2020-12-17 13:26:53 -08:00
Aaron Patterson c32218de1b
Disable auto compaction on platforms that can't support it
Both explicit compaction routines (gc_compact and the verify references form)
need to clear the heap before executing compaction.  Otherwise some
objects may not be alive, and we'll need the read barrier.  The heap
must only contain *live* objects if we want to disable the read barrier
during explicit compaction.

The previous commit was missing the "clear the heap" phase from the
"verify references" explicit compaction function.

Fixes [Bug #17306]
2020-11-25 11:29:14 -08:00
Aaron Patterson fed67fe6b2
Revert "Disable auto compaction on platforms that can't support it"
This reverts commit 63ad55cd88.

Revert "Disable read barrier on explicit compaction request"

This reverts commit 490b57783d.
2020-11-24 21:30:13 -08:00
Aaron Patterson 490b57783d Disable read barrier on explicit compaction request
We don't need a read barrier when the user calls `GC.compact` because we
don't allow allocations during GC, and all references should be "live"
2020-11-24 12:38:05 -08:00
Aaron Patterson ab5f2fa4fb
Refactor verification method
Combine everything in to one C function
2020-11-05 11:13:04 -08:00
Aaron Patterson 67b2c21c32
Add `GC.auto_compact= true/false` and `GC.auto_compact`
* `GC.auto_compact=`, `GC.auto_compact` can be used to control when
  compaction runs.  Setting `auto_compact=` to true will cause
  compaction to occurr duing major collections.  At the moment,
  compaction adds significant overhead to major collections, so please
  test first!

[Feature #17176]
2020-11-02 14:42:48 -08:00
Nobuyoshi Nakada 63aadc237f [Feature #16254] Use `Primitive.func` style 2020-06-19 18:46:55 +09:00
Nobuyoshi Nakada d863f4bccd [Feature #16254] Use `__builtin.func` style 2020-06-19 18:46:55 +09:00
Aaron Patterson 62ce8f96cd
Revert "Combine sweeping and moving"
This reverts commit 02b216e5a7.
This reverts commit 9b8825b6f9.

I found that combining sweep and move is not safe.  I don't think that
we can do compaction concurrently with _anything_ unless there is a read
barrier installed.

Here is a simple example.  A class object is freed, and during it's free
step, it tries to remove itself from its parent's subclass list.
However, during the sweep step, the parent class was moved and the
"currently being freed" class didn't have references updated yet.  So we
get a segv like this:

```
(lldb) bt
* thread #1, name = 'ruby', stop reason = signal SIGSEGV
  * frame #0: 0x0000560763e344cb ruby`rb_st_lookup at st.c:320:43
    frame #1: 0x0000560763e344cb ruby`rb_st_lookup(tab=0x2f7469672f6e6f72, key=3809, value=0x0000560765bf2270) at st.c:1010
    frame #2: 0x0000560763e8f16a ruby`rb_search_class_path at variable.c:99:9
    frame #3: 0x0000560763e8f141 ruby`rb_search_class_path at variable.c:145
    frame #4: 0x0000560763e8f141 ruby`rb_search_class_path(klass=94589785585880) at variable.c:191
    frame #5: 0x0000560763ec744e ruby`rb_vm_bugreport at vm_dump.c:996:17
    frame #6: 0x0000560763f5b958 ruby`rb_bug_for_fatal_signal at error.c:675:5
    frame #7: 0x0000560763e27dad ruby`sigsegv(sig=<unavailable>, info=<unavailable>, ctx=<unavailable>) at signal.c:955:5
    frame #8: 0x00007f8b891d33c0 libpthread.so.0`___lldb_unnamed_symbol1$$libpthread.so.0 + 1
    frame #9: 0x0000560763efa8bb ruby`rb_class_remove_from_super_subclasses(klass=94589790314280) at class.c:93:56
    frame #10: 0x0000560763d10cb7 ruby`gc_sweep_step at gc.c:2674:2
    frame #11: 0x0000560763d1187b ruby`gc_sweep at gc.c:4540:2
    frame #12: 0x0000560763d101f0 ruby`gc_start at gc.c:6797:6
    frame #13: 0x0000560763d15153 ruby`rb_gc_compact at gc.c:7479:12
    frame #14: 0x0000560763eb4eb8 ruby`vm_exec_core at vm_insnhelper.c:5183:13
    frame #15: 0x0000560763ea9bae ruby`rb_vm_exec at vm.c:1953:22
    frame #16: 0x0000560763eac08d ruby`rb_yield at vm.c:1132:9
    frame #17: 0x0000560763edb4f2 ruby`rb_ary_collect at array.c:3186:9
    frame #18: 0x0000560763e9ee15 ruby`vm_call_cfunc_with_frame at vm_insnhelper.c:2575:12
    frame #19: 0x0000560763eb2e66 ruby`vm_exec_core at vm_insnhelper.c:4177:11
    frame #20: 0x0000560763ea9bae ruby`rb_vm_exec at vm.c:1953:22
    frame #21: 0x0000560763eac08d ruby`rb_yield at vm.c:1132:9
    frame #22: 0x0000560763edb4f2 ruby`rb_ary_collect at array.c:3186:9
    frame #23: 0x0000560763e9ee15 ruby`vm_call_cfunc_with_frame at vm_insnhelper.c:2575:12
    frame #24: 0x0000560763eb2e66 ruby`vm_exec_core at vm_insnhelper.c:4177:11
    frame #25: 0x0000560763ea9bae ruby`rb_vm_exec at vm.c:1953:22
    frame #26: 0x0000560763ceee01 ruby`rb_ec_exec_node(ec=0x0000560765afa530, n=0x0000560765b088e0) at eval.c:296:2
    frame #27: 0x0000560763cf3b7b ruby`ruby_run_node(n=0x0000560765b088e0) at eval.c:354:12
    frame #28: 0x0000560763cee4a3 ruby`main(argc=<unavailable>, argv=<unavailable>) at main.c:50:9
    frame #29: 0x00007f8b88e560b3 libc.so.6`__libc_start_main + 243
    frame #30: 0x0000560763cee4ee ruby`_start + 46
(lldb) f 9
frame #9: 0x0000560763efa8bb ruby`rb_class_remove_from_super_subclasses(klass=94589790314280) at class.c:93:56
   90
   91  		*RCLASS_EXT(klass)->parent_subclasses = entry->next;
   92  		if (entry->next) {
-> 93  		    RCLASS_EXT(entry->next->klass)->parent_subclasses = RCLASS_EXT(klass)->parent_subclasses;
   94  		}
   95  		xfree(entry);
   96  	    }
(lldb) command script import -r misc/lldb_cruby.py
lldb scripts for ruby has been installed.
(lldb) rp entry->next->klass
(struct RMoved) $1 = (flags = 30, destination = 94589792806680, next = 94589784369160)
(lldb)
```
2020-06-09 13:53:18 -07:00
Aaron Patterson 9b8825b6f9
Update NEWS / documentation with GC.start(compact:true) 2020-05-29 15:59:42 -07:00
Aaron Patterson 02b216e5a7
Combine sweeping and moving
This commit combines the sweep step with moving objects.  With this
commit, we can do:

```ruby
GC.start(compact: true)
```

This code will do the following 3 steps:

1. Fully mark the heap
2. Sweep + Move objects
3. Update references

By default, this will compact in order that heap pages are allocated.
In other words, objects will be packed towards older heap pages (as
opposed to heap pages with more pinned objects like `GC.compact` does).
2020-05-29 15:24:32 -07:00
Kazuhiro NISHIYAMA 0b2c7473dd
Fix call-seq of GC.verify_compaction_references [ci skip] 2020-01-27 18:58:39 +09:00
Nobuyoshi Nakada af899503a6
Moved `GC.verify_compaction_references` to gc.rb
And fixed a segfault by coercion of `Qundef`, when any keyword
argument without `toward:` option is given.
2020-01-27 10:52:37 +09:00
Koichi Sasada eb4c86a698 Add option hash doc for GC stats.
Add a description about optional hash objects for GC.stat and
GC.latest_gc_info. [Bug #14408]

The patch is provided by sho-h (Sho Hashimoto).
Thank you so much.
2020-01-15 10:20:19 +09:00
Koichi Sasada 0b29662606 fix line break code (fix to LF) 2019-11-08 15:32:01 +09:00
Koichi Sasada 054cb6d389 forget to add gc.rb 2019-11-08 15:29:59 +09:00