Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
`lldb_cruby.py` manages lldb custom commands using functions. The file
is a large list of Python functions, and an init handler to map some of
the Python functions into the debugger, to enable execution of custom
logic during a debugging session.
Since LLDB 3.7 (September 2015) there has also been support for using
python classes rather than bare functions, as long as those classes
implement a specific interface.
This PR Introduces some more defined structure to the LLDB helper
functions by switching from the function based implementation to the
class based one, and providing an auto-loading mechanism by which new
functions can be loaded.
The intention behind this change is to make working with the LLDB
helpers easier, by reducing code duplication, providing a consistent
structure and a clearer API for developers.
The current function based approach has some advantages and
disadvantages
Advantages:
- Adding new code is easy.
- All the code is self contained and searchable.
Disadvantages:
- No visible organisation of the file contents. This means
- Hard to tell which functions are utility functions and which are
available to you in a debugging session
- Lots of code duplication within lldb functions
- Large files quickly become intimidating to work with - for example,
`lldb_disasm.py` was implemented as a seperate Python module because
it was easier to start with a clean slate than add significant amounts
of code to `lldb_cruby.py`
This PR attempts, to fix the disadvantages of the current approach and
maintain, or enhance, the benefits. The new structure of a command looks
like this;
```
class TestCommand(RbBaseCommand):
# program is the keyword the user will type in lldb to execute this command
program = "test"
# help_string will be displayed in lldb when the user uses the help functions
help_string = "This is a test command to show how to implement lldb commands"
# call is where our command logic will be implemented
def call(self, debugger, command, exe_ctx, result):
pass
```
If the command fulfils the following criteria it will then be
auto-loaded when an lldb session is started:
- The package file must exist inside the `commands` directory and the
filename must end in `_command.py`
- The package must implement a class whose name ends in `Command`
- The class inherits from `RbBaseCommand` or at minimum a class that
shares the same interface as `RbBaseCommand` (at minimum this means
defining `__init__` and `__call__`, and using `__call__` to call
`call` which is defined in the subclasses).
- The class must have a class variable `package` that is a String. This
is the name of the command you'll call in the `lldb` debugger.
This commit makes `rp` report the correct array length in lldb.
When USING_RVARGC is set we use 7 bits of the flags to store the array
len rather than the usual 2, so they need to be part of the mask when
calculating the length in lldb.
When calculating whether rvargc is enabled I've used the same approach
that's used by `GC.using_rvargc?` which is to detect whether there is
more than one size pool in the current objspace.
rb_backtrace relies on the existend of RUBY_T_MASK. This is set up by
the global loading code in lldb_init()
rb_backtrace does not call lldb_init previously, and therefore would
only work if called after another lldb function that _did_ load the
globals.
Now that classes are using VWA, the RCLASS_PTR uses an offset to get the
rb_classext_t object. Doing this all the time in lldb is boring. So
script lldb to do it for us
Commit dde164e968 decoupled incremental
marking from page sizes. This commit changes Ruby heap page sizes to
64KiB. Doing so will have several benefits:
1. We can use compaction on systems with 64KiB system page sizes (e.g.
PowerPC).
2. Larger page sizes will allow Variable Width Allocation to increase
slot sizes and embed larger objects.
3. Since commit 002fa28599, macOS has 64
KiB pages. Making page sizes 64 KiB will bring these systems to
parity.
I have attached some bechmark results below.
Discourse:
On Discourse, we saw much better p99 performance (e.g. for "categories"
it went from 214ms on master to 134ms on branch, for "home" it went
from 265ms to 251ms). We don’t see much change in p60, p75, and p90
performance. We also see a slight decrease in memory usage by 1.04x.
Branch RSS: 354.9MB
Master RSS: 368.2MB
railsbench:
On rails bench, we don’t see a big change in RPS or p99
performance. We don’t see a big difference in memory usage.
Branch RPS: 826.27
Master RPS: 824.85
Branch p99: 1.67
Master p99: 1.72
Branch RSS: 88.72MB
Master RSS: 88.48MB
liquid:
We don’t see a significant change in liquid performance.
Branch parse & render: 28.653 I/s
Master parse & render: 28.563 i/s
This commits implements size classes in the GC for the Variable Width
Allocation feature. Unless `USE_RVARGC` compile flag is set, only a
single size class is created, maintaining current behaviour. See the
redmine ticket for more details.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
This commit removes T_PAYLOAD since the new VWA implementation no longer
requires T_PAYLOAD types.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
This commits implements size classes in the GC for the Variable Width
Allocation feature. Unless `USE_RVARGC` compile flag is set, only a
single size class is created, maintaining current behaviour. See the
redmine ticket for more details.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
This commit removes T_PAYLOAD since the new VWA implementation no longer
requires T_PAYLOAD types.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
rather than having to do this in a two step process:
1. heap_page obj
2. dump_page $2 (or whatever lldb variable heap_page set)
we can now just
dump_page_rvalue obj
Update the lldb script so it can mostly recover a Ruby stack trace from
a core file. It's still missing line numbers and dealing with CFUNCs,
but you use it like this:
```
(lldb) rbbt ec
rb_control_frame_t TYPE
0x7f6fd6555fa0 EVAL ./bootstraptest/runner.rb error!!
0x7f6fd6555f68 METHOD ./bootstraptest/runner.rb main
0x7f6fd6555f30 METHOD ./bootstraptest/runner.rb in_temporary_working_directory
0x7f6fd6555ef8 METHOD /home/aaron/git/ruby/lib/tmpdir.rb mktmpdir
0x7f6fd6555ec0 BLOCK ./bootstraptest/runner.rb block in in_temporary_working_directory
0x7f6fd6555e88 CFUNC
0x7f6fd6555e50 BLOCK ./bootstraptest/runner.rb block (2 levels) in in_temporary_working_directory
0x7f6fd6555e18 BLOCK ./bootstraptest/runner.rb block in main
0x7f6fd6555de0 METHOD ./bootstraptest/runner.rb exec_test
0x7f6fd6555da8 CFUNC
0x7f6fd6555d70 BLOCK ./bootstraptest/runner.rb block in exec_test
0x7f6fd6555d38 CFUNC
0x7f6fd6555d00 TOP /home/aaron/git/ruby/bootstraptest/test_insns.rb error!!
0x7f6fd6555cc8 CFUNC
0x7f6fd6555c90 BLOCK /home/aaron/git/ruby/bootstraptest/test_insns.rb block in <top (required)>
0x7f6fd6555c58 METHOD ./bootstraptest/runner.rb assert_equal
0x7f6fd6555c20 METHOD ./bootstraptest/runner.rb assert_check
0x7f6fd6555be8 METHOD ./bootstraptest/runner.rb show_progress
0x7f6fd6555bb0 METHOD ./bootstraptest/runner.rb with_stderr
0x7f6fd6555b78 BLOCK ./bootstraptest/runner.rb block in show_progress
0x7f6fd6555b40 BLOCK ./bootstraptest/runner.rb block in assert_check
0x7f6fd6555b08 METHOD ./bootstraptest/runner.rb get_result_string
0x7f6fd6555ad0 METHOD ./bootstraptest/runner.rb make_srcfile
0x7f6fd6555a98 CFUNC
0x7f6fd6555a60 BLOCK ./bootstraptest/runner.rb block in make_srcfile
```
Getting the main execution context is difficult (it is stored in a
thread local) so for now you must supply an ec and this will make a
backtrace
Jemma Issroff
Aaron Patterson
Matt Valentine-House
Peter Zhu
Nobuyoshi Nakada