I noticed this while running test_yjit with --mjit-call-threshold=1,
which redefines `Integer#<`. When Ruby is monkey-patched,
MJIT itself could be broken.
Similarly, Ruby scripts could break MJIT in many different ways. I
prepared the same set of hooks as YJIT so that we could possibly
override it and disable it on those moments. Every constant under
RubyVM::MJIT is private and thus it's an unsupported behavior though.
Make the WASI_SDK_PATH variable mandatory when building for wasi host.
This requirement prevents developers from being stuck due to unfriendly
configuration's error message.
Before this change, if enable_shared was true, the directory would have a suffix of -static and be deleted on each make install. This would cause a second make install to fail.
Cases like this:
```ruby
obj = Object.new
loop do
obj.instance_variable_set(:@foo, 1)
obj.remove_instance_variable(:@foo)
end
```
can cause us to use many more shapes than we want (and even run out).
This commit changes the code such that when an instance variable is
removed, we'll walk up the shape tree, find the shape, then rebuild any
child nodes that happened to be below the "targetted for removal" IV.
This also requires moving any instance variables so that indexes derived
from the shape tree will work correctly.
Co-Authored-By: Jemma Issroff <jemmaissroff@gmail.com>
Co-authored-by: John Hawthorn <jhawthorn@github.com>
LD_PRELOAD sometimes forces loading libraries into unrelated
executables. For example, macOS on recent Apple Silicon can execute
arm64 and arm64e binaries by default, and /usr/bin/clang is built as
arm64e. If Ruby is built as arm64, and mkmf launched through runruby.rb
spawns /usr/bin/clang, dynamic loader tries to load libruby (arm64e)
into clang (arm64). This force-load causes library load failure.
In theory, we don't need both LD_PRELOAD and LD_LIBRARY_PATH at the same
time, because executables requiring libruby already have libruby
dependency, so LD_LIBRARY_PATH is enough for this case.
instead of FILE*.
Using C.fprintf is slower than String manipulation on memory. I'm going
to change the way MJIT writes files, and this is a prerequisite for it.
`conv` proc is used before and after the `url` variable is updated. So
this script didn't seem to behave correctly for the "Close #xxx" syntax.
Reusing the same variable name for different things seems prone to errors.
As there should be no modified files just affter `git cherry-pick`
succeeded in `sync_default_gems_with_commits`, reset to the previous
revision once to pick up the committed files.
This reverts commit bd148a2bdd.
ERB#result_with_hash does not work on Ruby 2.2
https://ci.appveyor.com/project/ruby/ruby/builds/45420170
```
../tool/generic_erb.rb:33:in `block (2 levels) in <main>': undefined method `result_with_hash' for #<ERB:0x0000000002516650> (NoMethodError)
```
access properly. Because the libclang node had two children, it wasn't
handled well by the pattern matching for the bit field case.
In addition to that, this bit field has a non-1 width. Because we're
returning true/false for a width-1 bit field, I added another behavior
that works like a char value for bit fields with width 2-8.
Bundler's backups changes environment variables starting with
BUNDLER_ORIG_. This causes a lot of noise in tests as the leakchecker
reports them as changed.
When a test worker hangs and timeouts, the test runner crashes with the
following stack trace:
ruby/tool/lib/test/unit.rb:1747:in `puke': undefined method `backtrace' for Timeout::Error:Class (NoMethodError)
from ruby/tool/lib/test/unit.rb:790:in `block in _run_parallel'
from ruby/tool/lib/test/unit.rb:788:in `each'
This commit adds handling for Timeout::Error and outputs a message.
When `--suppress_not_found` option is given, no revision information
is available. And remove extraneous newline, when result is empty or
ends with a newline.
Now revision.tmp will be regenerated always and every times, even if
the recent file exists in the source directory, as far as using git.
On the other hand, VirtualBox mounts shared folders as root, and git
rejects the repository there as dubious ownership.
Line with substituted issue references with URLs are often very long.
Although Git (and GitHub) recommends folding subject lines less than
50 columns, but many commits ignore this, so fold at 68 columns for
now.
Search VCS directory after other options are in effective, i.e.,
`--srcdir=nonexitent --suppress_not_found` options, as well as the
reverse order case, should print the current date only and exit
successfully.
The reference generated by using RDoc without the proper `--page-dir`
option (or `.rdoc_options`) file may contain `/doc/`. Since these
URIs are redirected by the server now, replace such URIs with the
corresponding rdoc-refs too.
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
except for bit fields.
I made a risky assumption on leading bit fields and just gave up
non-leading bit fields for now. I'll change it to let C code access bit
fields later.
The reason why this was commented out was because of gperf 3.0 vs 3.1
differences (see [Feature #13883]). Five years passed, I am pretty
confident that we can drop support of old versions here.
Ditto for uniname2ctype_p(), onig_jis_property(), and zonetab().
when -q is given.
One of the RubyCI servers, freebsd12, had a broken git environment:
```
$ git show
fatal: detected dubious ownership in repository at '/usr/home/chkbuild/chkbuild/tmp/build/20220917T123002Z/ruby'
To add an exception for this directory, call:
git config --global --add safe.directory /usr/home/chkbuild/chkbuild/tmp/build/20220917T123002Z/ruby
```
tool/lib/vcs.rb doesn't work normally for that server.
Even for such cases, we'd like to generate a usable revision.h.
So this patch lets revision.h fallback to default VCS.release_date
when VCS::NotFoundError is raised.
* So deprecated methods/constants/functions are dealt with early,
instead of many tests breaking suddenly when removing a deprecated
method/constant/function.
* Follows https://bugs.ruby-lang.org/issues/17591
I want to use more complicated macros with MJIT. For example:
```
# define SHAPE_MASK (((unsigned int)1 << SHAPE_BITS) - 1)
```
This commit adds a simple recursive descent parser that produces an AST
and a small visitor that converts the AST to Ruby.
Fixes id.h error during updating ripper.c by `make after-update`.
While it used to update id.h in the build directory, but was trying to
update ripper.c in the source directory. In principle, files in the
source directory can or should not depend on files in the build
directory.
Previously YARV bytecode implemented constant caching by having a pair
of instructions, opt_getinlinecache and opt_setinlinecache, wrapping a
series of getconstant calls (with putobject providing supporting
arguments).
This commit replaces that pattern with a new instruction,
opt_getconstant_path, handling both getting/setting the inline cache and
fetching the constant on a cache miss.
This is implemented by storing the full constant path as a
null-terminated array of IDs inside of the IC structure. idNULL is used
to signal an absolute constant reference.
$ ./miniruby --dump=insns -e '::Foo::Bar::Baz'
== disasm: #<ISeq:<main>@-e:1 (1,0)-(1,13)> (catch: FALSE)
0000 opt_getconstant_path <ic:0 ::Foo::Bar::Baz> ( 1)[Li]
0002 leave
The motivation for this is that we had increasingly found the need to
disassemble the instructions between the opt_getinlinecache and
opt_setinlinecache in order to determine the constant we are fetching,
or otherwise store metadata.
This disassembly was done:
* In opt_setinlinecache, to register the IC against the constant names
it is using for granular invalidation.
* In rb_iseq_free, to unregister the IC from the invalidation table.
* In YJIT to find the position of a opt_getinlinecache instruction to
invalidate it when the cache is populated
* In YJIT to register the constant names being used for invalidation.
With this change we no longe need disassemly for these (in fact
rb_iseq_each is now unused), as the list of constant names being
referenced is held in the IC. This should also make it possible to make
more optimizations in the future.
This may also reduce the size of iseqs, as previously each segment
required 32 bytes (on 64-bit platforms) for each constant segment. This
implementation only stores one ID per-segment.
There should be no significant performance change between this and the
previous implementation. Previously opt_getinlinecache was a "leaf"
instruction, but it included a jump (almost always to a separate cache
line). Now opt_getconstant_path is a non-leaf (it may
raise/autoload/call const_missing) but it does not jump. These seem to
even out.
I'm planning to introduce mjit_compiler.rb, and I want to make this
consistent with it. Consistency with compile.c doesn't seem important
for MJIT anyway.
The current MJIT relies on SIGCHLD and fork(2) to be performant, and
it's something mswin can't offer. You could run Linux MJIT on WSL
instead.
[Misc #18968]
Adds the `syntax_suggest` syntax error display tool to Ruby through the same mechanism as `error_highlight` and `did_you_mean`. Reference ticket: https://bugs.ruby-lang.org/issues/18159close#4845
## What is syntax_suggest?
When a syntax error is raised by requiring a file, dead_end will use a combination of indentation and lexing to identify the problem.
> Note: Previously this tool was named `dead_end`.
## Known issues
- SyntaxSearch's approach of showing syntax errors only works through integration with `require`, `load`, `autoload`, and `require_relative` (since it monkeypatches them to detect syntax errors). It does not work with direct Ruby file invocations https://github.com/zombocom/dead_end/issues/31.
- This causes failure in the test suite (test_expected_backtrace_location_when_inheriting_from_basic_object_and_including_kernel) and confusion when inspecting backtraces if there's a different error when trying to require a file such as measuring memory (https://github.com/zombocom/syntax_suggest/issues/124#issuecomment-1006705016).
- Discussed fix. We previously talked about opening up `SyntaxError` to be monkeypatched in the same way that other gems hook into `NoMethodError`. This is currently not possible and requires development work. When we last talked about it at RubyKaigi Nobu expressed an ability to make such a change.
- Split into `Ext` and `Lib` classes.
- `Ext#files` should not include built extension libraries.
- `Ext#files` should include scripts under its own `lib`.
- `Lib#files` should be prefixed with `lib/`.
Gemspec files generated by old bundler run `git` without changing the
working directory.
Or some gemspec files expect an owned file at the top exists ath the
current working directory.
Koichi Sasada
Hiroshi SHIBATA
David Rodríguez
Nobuyoshi Nakada
Takashi Kokubun
Soutaro Matsumoto
Yuta Saito
Yusuke Endoh
Samuel Williams
Jimmy Miller
Aaron Patterson
Alan Wu
Peter Zhu
John Hawthorn
Sergey Fedorov
Jemma Issroff
卜部昌平
Benoit Daloze
schneems
Burdette Lamar