Introduce Universal Parser mode for the parser.
This commit includes these changes:
* Introduce `UNIVERSAL_PARSER` macro. All of CRuby related functions
are passed via `struct rb_parser_config_struct` when this macro is enabled.
* Add CI task with 'cppflags=-DUNIVERSAL_PARSER' for ubuntu.
This instructs make to delete target files if the recipe fails midway, like
when make itself is interrupted. This is mostly for development since it
protects against corrupt builds that need a `make clean` to repair. Release
builds normally don't fail mid-recipe.
GNU make and BSD make support this.
From GNU make's manual:
> This is almost always what you want make to do, but it is not historical
> practice; so for compatibility, you must explicitly request it.
See https://innolitics.com/articles/make-delete-on-error/
Rust 1.58.0 unfortunately doesn't provide facilities to control symbol
visibility/presence, but we care about controlling the list of
symbols exported from libruby-static.a and libruby.so.
This commit uses `ld -r` to make a single object out of rustc's
staticlib output, libyjit.a. This moves libyjit.a out of MAINLIBS and adds
libyjit.o into COMMONOBJS, which obviates the code for merging libyjit.a
into libruby-static.a. The odd appearance of libyjit.a in SOLIBS is also
gone.
To filter out symbols we do not want to export on ELF platforms, we use
objcopy after the partial link. On darwin, we supply a symbol list to
the linker which takes care of hiding unprefixed symbols.
[Bug #19255]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
When the build is running with a base ruby then generating `x64-ucrt-ruby320.rc`
could fail due to a missing dependency to `x64-mingw-ucrt-fake.rb`.
This commit adds this dependency.
A failing build looks like so:
```
generating x64-mingw-ucrt-fake.rb
generating x64-ucrt-ruby320.rc
../snapshot-master/win32/resource.rb:in `require': cannot load such file -- ./x64-mingw-ucrt-fake (LoadError)
make: *** [GNUmakefile:57: x64-ucrt-ruby320.rc] Error 1
make: *** Waiting for unfinished jobs....
linking miniruby.exe
x64-mingw-ucrt-fake.rb updated
```
In principle, we have a DTrace probe in yjit.c, so yjit.o should be
in DTRACE_DEPENDENT_OBJS for DTRACE_REBUILD=yes builds. This commit
adds to the list.
In practice DTRACE_REBUILD=yes implies the system has a Solaris-like
DTrace and YJIT doesn't support those systems. YJIT_OBJ expands to
nothing, and yjit.c isn't compiled.
I tested on OmniOS v11 r151034m with:
$ ../src/configure --with-out-ext=psych MAKE=gmake AR=ar debugflags=-g
$ gmake -j
It builds before and after this change.
[Bug #18480]
Reduce duplicate replacements so that reflect macros in command lines
consitently. So that reflect macros in command lines. Others than
`nmake` have no problems with nested expansions.
Previously, we relied on shell word splitting, which leads
to passing the wrong arguments when there are white spaces
in the path.
Avoiding command substitution also makes this script more
likely to work under Solaris 10, where `/bin/sh` is not
POSIX compliant [1]. (Thanks you, `@znz` for fixing the syntax
error in 4210ae2158 though!)
The hack from c466f270b8
doesn't actually work so this commit reverts it. The shell
still needs to parse through all of the code, maybe because
make doesn't in fact send newlines to the shell.
By the way, we also use the `-exec` option in the `ext/distclean` task.
[1]: https://docs.oracle.com/cd/E26505_01/html/816-5165/sh-1.html
The shell in Solaris 10 has trouble understanding the syntax I used in
YJIT's library merging script.
This commit reduces the code the shell needs to parse before exiting on
non-YJIT builds to hopefully fix the error on Solaris.
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.
The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.
Because we want to be careful, YJIT is guarded behind a configure
option:
```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```
By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.
The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.
The CI tests have been updated and do not take any more resources than
before.
The development history of the Rust port is available at the following
commit for interested parties:
1fd9573d8b
Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.
[issue]: https://bugs.ruby-lang.org/issues/18481
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
Previously, `make test-spec` was not printing the description with +YJIT
even when YJIT was indeed enabled. It was confusing on CI. `fake.rb` was
changing the `RUBY_DESCRIPTION` constant incorrectly.
I suppose `make test-spec` mostly needs the `mkmf` faking and not the
faking of `RUBY_.*` constants, so maybe there is an opportunity to simplify
in the future.
This is another attempt to fix out-of-src build with
--with-static-linked-ext. The first attempt was
4f1888bda70981d9f5b1bf55ab692e0ce18e79f4 but reverted because it broke
out-of-src build from pre-generated sources via `make dist`.
This patch fixes the second trans C source gen, mentioned in the
previous commit message, by passing MINIRUBY as well as when invoking
from common.mk
* Rename --jit to --mjit
[Feature #18349]
* Fix a few more --jit references
* Fix MJIT Actions
* More s/jit/mjit/ and re-introduce --disable-jit
* Update NEWS.md
* Fix test_bug_reporter_add
Takashi Kokubun
Nobuyoshi Nakada
Janosch Müller
卜部昌平
Jemma Issroff
yui-knk
Alan Wu
Hiroshi SHIBATA
Lars Kanis
Jakub Kulík
Kazuhiro NISHIYAMA
Alan Wu
Yuta Saito
Peter Zhu
Naohisa Goto
Aaron Patterson
Jose Narvaez
Maxime Chevalier-Boisvert