Граф коммитов

867 Коммитов

Автор SHA1 Сообщение Дата
Takashi Kokubun e7c71c6c92
Make mjit_cont sharable with YJIT (#6556)
* Make mjit_cont sharable with YJIT

* Update dependencies

* Update YJIT binding
2022-10-17 09:27:59 -07:00
Nobuyoshi Nakada f178ff3933
Allow abbreviated dump options with additional options 2022-10-09 16:27:34 +09:00
yui-knk 52eaffde86 Support "+error-tolerant" as ruby option
[Feature #19013]
2022-10-08 17:59:11 +09:00
Takashi Kokubun 3767c6a90d
Ruby MJIT (#6028) 2022-09-04 21:53:46 -07:00
Takashi Kokubun cfa40e225a
Prefer stdbool for MJIT options
same motivation as d6f21b308b
2022-09-03 20:01:51 -07:00
schneems a50df1ab0e Setup SyntaxSuggest as default gem
Adds the `syntax_suggest` syntax error display tool to Ruby through the same mechanism as `error_highlight` and `did_you_mean`. Reference ticket: https://bugs.ruby-lang.org/issues/18159

close #4845

## What is syntax_suggest?

When a syntax error is raised by requiring a file, dead_end will use a combination of indentation and lexing to identify the problem.

> Note: Previously this tool was named `dead_end`. 

## Known issues

- SyntaxSearch's approach of showing syntax errors only works through integration with `require`, `load`, `autoload`, and `require_relative` (since it monkeypatches them to detect syntax errors). It does not work with direct Ruby file invocations https://github.com/zombocom/dead_end/issues/31.
  - This causes failure in the test suite (test_expected_backtrace_location_when_inheriting_from_basic_object_and_including_kernel) and confusion when inspecting backtraces if there's a different error when trying to require a file such as measuring memory (https://github.com/zombocom/syntax_suggest/issues/124#issuecomment-1006705016).
  - Discussed fix. We previously talked about opening up `SyntaxError` to be monkeypatched in the same way that other gems hook into `NoMethodError`. This is currently not possible and requires development work. When we last talked about it at RubyKaigi Nobu expressed an ability to make such a change.
2022-08-19 10:02:24 +09:00
Nobuyoshi Nakada ee864beb7c
Simplify around `USE_YJIT` macro (#6240)
* Simplify around `USE_YJIT` macro

- Use `USE_YJIT` macro only instead of `YJIT_BUILD`.
- An intermediate macro `YJIT_SUPPORTED_P` is no longer used.

* Bail out if YJIT is enabled on unsupported platforms
2022-08-15 13:05:12 -04:00
Nobuyoshi Nakada f2423be49d
Duplicate libruby self path
When LOAD_RELATIVE, as `sopath` is truncated to the prefix path, make
the duplicate before it.  Also make `rb_libruby_selfpath` frozen and
hidden.
2022-08-07 14:46:59 +09:00
Nobuyoshi Nakada 58c8b6e862
Adjust styles [ci skip] 2022-08-06 10:13:20 +09:00
Yuta Saito 00f411c58a Add `-bundle_loader` to mjit compilation args on macOS 2022-08-04 16:29:22 +09:00
Peter Zhu efb91ff19b Rename rb_ary_tmp_new to rb_ary_hidden_new
rb_ary_tmp_new suggests that the array is temporary in some way, but
that's not true, it just creates an array that's hidden and not on the
transient heap. This commit renames it to rb_ary_hidden_new.
2022-07-26 09:12:09 -04:00
Takashi Kokubun 5b21e94beb Expand tabs [ci skip]
[Misc #18891]
2022-07-21 09:42:04 -07:00
Nobuyoshi Nakada 58e7205c82
See the environment variable and then check if JIT options conflict 2022-07-08 19:10:18 +09:00
Nobuyoshi Nakada cd94842922 [Bug #18892] Reset `ARGF.lineno` after reading shebang 2022-07-07 11:33:07 +09:00
Nobuyoshi Nakada 829d726604 Fallback to the default JIT only when no JIT is enabled
Usually, command line options are given precedence first, environment
variables next, and fall back to configuration options at last.
2022-07-02 09:15:20 +09:00
Nobuyoshi Nakada 33100797c4
Remove redundant parentheses [ci skip] 2022-07-01 09:37:00 +09:00
Nobuyoshi Nakada f4d70e1cd8
Make `FEATURE_SET_P` macro to include `FEATURE_BIT` 2022-06-30 16:51:39 +09:00
Nobuyoshi Nakada bd8ed1f904
Do not call `exit()` directly 2022-06-29 20:10:29 +09:00
Alan Wu eb1a84a9c3 When YJIT is not built, hide options and use MJIT for --jit
YJIT is now a build-time opt-in so on platforms that YJIT could support
it could still be unavailable due to user discretion. Use MJIT for --jit
and don't display YJIT related command line options in --help when YJIT
is not included in the build.
2022-06-23 18:12:25 -04:00
Chris Seaton 31b2cd38c5 Include JIT information in crash reports
Since enabling YJIT or MJIT drastically changes what could go wrong at
runtime, it's good to be front and center about whether they are enabled
when dumping a crash report. Previously, `RUBY_DESCRIPTION` and the
description printed when crashing can be different when a JIT is on.

Introduce a new internal data global, `rb_dynamic_description`, and set
it to be the same as `RUBY_DESCRIPTION` during initialization; use it
when crashing.

 * version.c: Init_ruby_description(): Initialize and use
       `rb_dynamic_description`.
 * error.c: Change crash reports to use `rb_dynamic_description`.
 * ruby.c: Call `Init_ruby_description()` earlier. Slightly more work
       for when we exit right after printing the description but that
       was deemed acceptable.
 * include/ruby/version.h: Talk about how JIT info is not in
      `ruby_description`.
 * test/-ext-/bug_reporter/test_bug_reporter.rb: Remove handling for
       crash description being different from `RUBY_DESCRIPTION`.
 * test/ruby/test_rubyoptions.rb: ditto

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Alan Wu <alanwu@ruby-lang.org>
2022-06-20 17:18:29 -04:00
Alan Wu f90549cd38 Rust YJIT
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.

The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.

Because we want to be careful, YJIT is guarded behind a configure
option:

```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```

By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.

The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.

The CI tests have been updated and do not take any more resources than
before.

The development history of the Rust port is available at the following
commit for interested parties:
1fd9573d8b

Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.

[issue]: https://bugs.ruby-lang.org/issues/18481

Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2022-04-27 11:00:22 -04:00
Yuta Saito be1382b498 ruby.c: initialize extra optional extensions linked statically
Add a hook point to initialize extra extension libraries. The default
hook function is replaced when linking a strong `Init_extra_exts`
symbol. A builder can insert an object file that defines Init_extra_exts
by XLDFLAGS.
2022-01-19 11:19:06 +09:00
Peter Zhu ffda21b7ba [Feature #18491] Drop support for HP-UX
IA64 support was dropped in ticket #15894, so we can drop support for
HP-UX.
2022-01-18 09:52:15 -05:00
Nobuyoshi Nakada 2dff82bfca
`O_NONBLOCK` is not always a preprocessor constant on all platforms 2022-01-16 23:47:10 +09:00
Nobuyoshi Nakada c1bcfeec38
Transfer the responsibility for MJIT options to mjit.c 2022-01-15 18:57:33 +09:00
Nobuyoshi Nakada d853a7886e Define the default JIT feature bit 2022-01-15 18:55:47 +09:00
Nobuyoshi Nakada 26b4aa853f
Show JIT options only when supported 2022-01-12 17:51:14 +09:00
Kazuhiro NISHIYAMA b5310b8975
Fix compile errors
```
compiling ../ruby.c
../ruby.c:1547:17: error: implicit declaration of function 'setup_yjit_options' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                setup_yjit_options(s, &opt->yjit);
                ^
../ruby.c:1547:17: note: did you mean 'setup_mjit_options'?
../ruby.c:1122:1: note: 'setup_mjit_options' declared here
setup_mjit_options(const char *s, struct mjit_options *mjit_opt)
^
../ruby.c:1547:45: error: no member named 'yjit' in 'struct ruby_cmdline_options'; did you mean 'mjit'?
                setup_yjit_options(s, &opt->yjit);
                                            ^~~~
                                            mjit
../ruby.c:192:25: note: 'mjit' declared here
    struct mjit_options mjit;
                        ^
../ruby.c:1924:28: error: no member named 'yjit' in 'struct ruby_cmdline_options'; did you mean 'mjit'?
        rb_yjit_init(&opt->yjit);
                           ^~~~
                           mjit
../ruby.c:192:25: note: 'mjit' declared here
    struct mjit_options mjit;
                        ^
3 errors generated.
```
2022-01-11 17:33:01 +09:00
Nobuyoshi Nakada 3d675c72b9
Use `setup_yjit_options` only when supported 2022-01-11 17:26:21 +09:00
Nobuyoshi Nakada 1c9b5d452e
Reject command line option ending with `-` 2022-01-11 11:49:34 +09:00
Nobuyoshi Nakada 8b585f7460
Define YJIT options only when supported 2022-01-11 10:12:40 +09:00
Kazuhiro NISHIYAMA c764e368bd
Fix default --jit-max-cache in `ruby --help`
changed at 028f1887c2
2022-01-10 20:17:05 +09:00
Kazuhiro NISHIYAMA a7faca51ac
Add `=num` to yjit options
`--yjit-call-threshold` and `--yjit-max-versions` need an argument.
2022-01-09 12:39:34 +09:00
Peter Zhu 44379805b0 Fix crash on bootup when RGENGC_CHECK_MODE=2 with GC stress
We need to unshare the array loaded_features because the shared root
could be old gen (while loaded_features is young) which causes a WB
miss.
2022-01-04 14:46:20 -05:00
Takashi Kokubun 11b8aaa26a
Rename --jit to --mjit (#5248)
* Rename --jit to --mjit

[Feature #18349]

* Fix a few more --jit references

* Fix MJIT Actions

* More s/jit/mjit/ and re-introduce --disable-jit

* Update NEWS.md

* Fix test_bug_reporter_add
2021-12-13 16:08:01 -08:00
Yusuke Endoh a692a1597f ruby.c: Fix typo 2021-12-13 10:29:08 +09:00
Yusuke Endoh 8613c0c675 Introduce an option "--dump=insns_without_opt" for debugging purposes 2021-12-13 10:29:08 +09:00
Aaron Patterson 3b2b28d035
Revert "Force disable yjit on OpenBSD"
This reverts commit 119626da94.
2021-12-01 12:46:30 -08:00
Maxime Chevalier-Boisvert cdebf57ec6
Add --yjit-no-type-prop so we can test YJIT without type propagation (#5135)
* Add --yjit-no-type-prop so we can test YJIT without type propagation

* Fix typo in command line option

* Leave just two test workflows enable for YJIT
2021-11-18 10:44:31 -05:00
Peter Zhu a5b6598192 [Feature #18239] Implement VWA for strings
This commit adds support for embedded strings with variable capacity and
uses Variable Width Allocation to allocate strings.
2021-10-25 13:26:23 -04:00
Takashi Kokubun 6469038ae2
Fix TestRubyOptions#test_enable for -DMJIT_FORCE_ENABLE
--enable=all didn't work when cppflags=-DMJIT_FORCE_ENABLE was given.
2021-10-21 20:41:14 -07:00
Jeremy Evans 119626da94 Force disable yjit on OpenBSD
TestRubyOptions#test_enable was broken on OpenBSD after the yjit
merge. --yjit (and --enable-all, which enables --yjit) fails on
OpenBSD because yjit uses an insecure mmap call (both writable
and executable), in alloc_exec_mem, which OpenBSD does not allow.

This can probably be reverted if yjit switches to a more secure
mmap design (writable xor executable).  This would involve
initially calling mmap with PROT_READ | PROT_WRITE, and after writing
of executable code has finished, using mprotect to switch to
PROT_READ | PROT_EXEC. I believe Firefox uses this approach for
their Javascript engine since Firefox 46.
2021-10-21 08:26:02 -07:00
Alan Wu 40608d0b7b Fix indentation in ruby.c 2021-10-20 18:19:43 -04:00
Alan Wu 95e337f815 Warn when trying to use YJIT in --disable-jit-support builds 2021-10-20 18:19:43 -04:00
Alan Wu ff863f3818 Show --yjit in help message 2021-10-20 18:19:43 -04:00
Alan Wu 11096da6cc Improve YJIT command line option parsing
Previously, options such as "--yjit123" would enable YJIT. Additionally,
the error message for argument parsing mentioned "--jit-..." instead of
"--yjit-...".
2021-10-20 18:19:43 -04:00
Alan Wu 1f25bba7b7 YJIT_STATS instead of RUBY_DEBUG for --yjit-stats in help printout 2021-10-20 18:19:42 -04:00
Alan Wu d4fae3e390 Say YJIT is disabled by default in ruby --help 2021-10-20 18:19:42 -04:00
Alan Wu d355bf17ba Style fix in ruby.c 2021-10-20 18:19:42 -04:00
Noah Gibbs 21e58acef3 Don't enable YJIT by default. More tests on both Ubuntu and MacOS.
Add RUBY_YJIT_ENABLE env var and YJIT_FORCE_ENABLE compile-time constant.
Rename YJIT_STATS to RUBY_YJIT_STATS.
2021-10-20 18:19:42 -04:00
Noah Gibbs f021b35273 Reduce line length for YJIT option descriptions 2021-10-20 18:19:42 -04:00
Noah Gibbs fab469b678 Document YJIT options in --help 2021-10-20 18:19:42 -04:00
Aaron Patterson afc5db43ca is this right? 2021-10-20 18:19:40 -04:00
Aaron Patterson b78c50d800 Don't check MJIT if it's not enabled 2021-10-20 18:19:40 -04:00
Aaron Patterson 9d5beb6fde Disable YJIT by default if MJIT_FORCE_ENABLE is on
Compile time flag seems pretty forceful, so let MJIT turn on by default
if it is used.
2021-10-20 18:19:40 -04:00
Aaron Patterson ea453acf59 disable MJIT when --enable-all is set 2021-10-20 18:19:40 -04:00
Maxime Chevalier-Boisvert a1d42c37f4 Update ruby.c 2021-10-20 18:19:40 -04:00
Aaron Patterson 234ab816ba Exit if YJIT and MJIT are both enabled
YJIT and MJIT can't be running in the same process otherwise they'll
clobber each other.  We should show an error and exit if they're both
enabled.
2021-10-20 18:19:40 -04:00
Maxime Chevalier-Boisvert 7030cae969 Try running with more YJIT options in CI to surface more bugs 2021-10-20 18:19:36 -04:00
Maxime Chevalier-Boisvert b415ceb92e Increase default YJIT call threshold to 10. Add exec mem size arg. (#52) 2021-10-20 18:19:35 -04:00
Maxime Chevalier-Boisvert 96f4f918b0 Implement greedy versioning. Refactor versioning logic. (#10)
* Implement eager versioning. Refactor versioning logic.

* Add --version-limit and --greedy-versioning command-line args
2021-10-20 18:19:34 -04:00
Jose Narvaez 4e2eb7695e Yet Another Ruby JIT!
Renaming uJIT to YJIT. AKA s/ujit/yjit/g.
2021-10-20 18:19:31 -04:00
Maxime Chevalier-Boisvert 0710bec01e Implement --ujit-call-threshold 2021-10-20 18:19:30 -04:00
Maxime Chevalier-Boisvert a14015e9b5 Report unknown --ujit-* options 2021-10-20 18:19:28 -04:00
Alan Wu b7f93e81df Implement --ujit-stats and instructoin counting
VM and ujit instruction counting in debug builds.

shopify/ruby#19
2021-10-20 18:19:27 -04:00
Maxime Chevalier-Boisvert e4c65ec49c Refactor uJIT code into more files for readability 2021-10-20 18:19:26 -04:00
Alan Wu 726bdd4d35 Show whether MicroJIT is enabled in the version string 2021-10-20 18:19:24 -04:00
Alan Wu a8d784888d Fix mistake in ujit command line parsing 2021-10-20 18:19:24 -04:00
Alan Wu e8c914c250 Implement the --disable-ujit command line option 2021-10-20 18:19:24 -04:00
Nobuyoshi Nakada 842b0008c1 Skip broken strings as the locale encoding 2021-10-01 20:28:44 +09:00
Nobuyoshi Nakada 409dbc951b Replace expanded load path only when modified 2021-10-01 20:28:44 +09:00
Nobuyoshi Nakada b6534691a1 Copy path strings as interned strings 2021-10-01 20:28:44 +09:00
Nobuyoshi Nakada 60d0421ca8 Fix the encoding of loaded feature names [Bug #18191]
The feature names loaded from the default load paths should also
be in the file system encoding.
2021-10-01 20:28:44 +09:00
卜部昌平 f83b14af24 include/ruby/internal/interpreter.h: add doxygen
Must not be a bad idea to improve documents. [ci skip]

In fact many functions declared in the header file are already
documented more or less.  They were just copy & pasted, with applying
some style updates.
2021-09-10 20:00:06 +09:00
S-H-GAMELINKS bdd6d8746f Replace RBOOL macro 2021-09-05 23:01:27 +09:00
Nobuyoshi Nakada 3db7f6353f
Suppress unused-but-set-variable warning 2021-07-06 00:09:27 +09:00
Yusuke Endoh 9438c99590 Rename error_squiggle to error_highlight 2021-06-29 23:45:49 +09:00
Yusuke Endoh e946049665 [WIP] add error_squiggle gem
```
$ ./local/bin/ruby -e '1.time {}'
-e:1:in `<main>': undefined method `time' for 1:Integer (NoMethodError)

1.time {}
 ^^^^^
Did you mean?  times
```

https://bugs.ruby-lang.org/issues/17930
2021-06-29 23:45:49 +09:00
Samuel Williams 45e65f302b Deprecate and rework old (fd) centric functions. 2021-06-22 22:48:57 +12:00
Nobuyoshi Nakada 791d7d4947
Freeze command line script 2021-06-10 15:15:35 +09:00
Benoit Daloze 68d6bd0873 Fix trivial -Wundef warnings
* See [Feature #17752]

Co-authored-by: xtkoba (Tee KOBAYASHI) <xtkoba+ruby@gmail.com>
2021-05-04 14:56:55 +02:00
Nobuyoshi Nakada 1f08b0d18d
Removed dln_a_out
a.out format is considered extinct nowadays.
2021-03-24 20:13:55 +09:00
NARUSE, Yui d58daad37b [Feature #17684] Declare --disable-gems is for debugging 2021-03-17 15:36:05 +09:00
Nobuyoshi Nakada 90c12defb3
Constified variables for getenv 2021-03-12 17:13:53 +09:00
Yusuke Endoh 8ccc12118e Keep libpath length non-negative [Bug #16784]
When runtime_libruby_path does not include '/', it attempts to call
rb_str_resize with negative length. This change makes sure that the
length non-negative.

Co-Authored-By: xtkoba (Tee KOBAYASHI) <xtkoba+ruby@gmail.com>
2021-03-06 00:50:09 +09:00
YO4 (Yoshinao Muramatsu) b94b7965c4
Fixed fallback ENABLE_VIRTUAL_TERMINAL_PROCESSING value [Bug #17639] 2021-02-20 00:50:57 +09:00
Nobuyoshi Nakada d6e43e4723
Moved --verbose, --verbose and --help options to last [ci skip] 2021-01-24 14:59:41 +09:00
Nobuyoshi Nakada 6f6dfdcc68 Make warning values consistent [Bug #17523]
They should be affected, as well as `$VERBOSE`, by `-w`/`-W`
options, not only in the main script but in scripts loaded by `-r`
option too.
2021-01-13 18:12:49 +09:00
Nobuyoshi Nakada dc13bd22bb Ignore failure on unsupported fcntl to drop non-blocking mode
Fixes https://github.com/ruby/ruby/pull/3723
2020-12-26 09:45:06 +09:00
Nobuyoshi Nakada b9d00f42e6
Enable escape sequence on Windows10 console via pager too 2020-12-21 18:12:02 +09:00
Jeremy Evans 05313c914b Use category: :deprecated in warnings that are related to deprecation
Also document that both :deprecated and :experimental are supported
:category option values.

The locations where warnings were marked as deprecation warnings
was previously reviewed by shyouhei.

Comment a couple locations where deprecation warnings should probably
be used but are not currently used because deprecation warning
enablement has not occurred at the time they are called
(RUBY_FREE_MIN, RUBY_HEAP_MIN_SLOTS, -K).

Add assert_deprecated_warn to test assertions.  Use this to simplify
some tests, and fix failing tests after marking some warnings with
deprecated category.
2020-12-18 09:54:11 -08:00
Nobuyoshi Nakada 8b32191a31 Ignore failure on unsupported fcntl to drop non-blocking mode 2020-12-16 17:54:43 +09:00
Jeremy Evans dca13e2bf7 Make ruby -We the same as ruby -W -e
This changes the behavior, which I'm not sure is acceptable.
However, it's odd to allow an option to be combined, but change
the behavior of the option when combined.
2020-12-10 10:16:40 -08:00
Kenta Murata 146b3695c0
ruby.c: remove needless substitution of uenc
uenc has already been initialized the top of the function.
2020-12-08 02:10:03 +09:00
Lars Kanis 94b6933d1c
Set default for Encoding.default_external to UTF-8 on Windows (#2877)
* Use UTF-8 as default for Encoding.default_external on Windows
* Document UTF-8 change on Windows to Encoding.default_external

fix https://bugs.ruby-lang.org/issues/16604
2020-12-08 01:48:37 +09:00
Nobuyoshi Nakada afd765f2ee
Initialize UTF-8 encoding first as it is used always now 2020-11-23 09:45:17 +09:00
Koichi Sasada 99310e3eb5 Some global variables can be accessed from ractors
Some global variables should be used from non-main Ractors.
[Bug #17268]

```ruby
     # ractor-local (derived from created ractor): debug
     '$DEBUG' => $DEBUG,
     '$-d' => $-d,

     # ractor-local (derived from created ractor): verbose
     '$VERBOSE' => $VERBOSE,
     '$-w' => $-w,
     '$-W' => $-W,
     '$-v' => $-v,

     # process-local (readonly): other commandline parameters
     '$-p' => $-p,
     '$-l' => $-l,
     '$-a' => $-a,

     # process-local (readonly): getpid
     '$$'  => $$,

     # thread local: process result
     '$?'  => $?,

     # scope local: match
     '$~'  => $~.inspect,
     '$&'  => $&,
     '$`'  => $`,
     '$\''  => $',
     '$+'  => $+,
     '$1'  => $1,

     # scope local: last line
     '$_' => $_,

     # scope local: last backtrace
     '$@' => $@,
     '$!' => $!,

     # ractor local: stdin, out, err
     '$stdin'  => $stdin.inspect,
     '$stdout' => $stdout.inspect,
     '$stderr' => $stderr.inspect,
```
2020-10-20 15:38:54 +09:00
Nobuyoshi Nakada 996af2ce08 Disable deprecation warning by the default [Feature #16345]
And `-w` option turns it on.
2020-09-25 09:50:33 +09:00
Nobuyoshi Nakada 0dbf6e46fb
[Feature #17157] removed -T command line option 2020-09-07 00:59:53 +09:00
卜部昌平 7cf4625690 feature_option: do not goto into a branch
I'm not necessarily against every goto in general, but jumping into a
branch is definitely a bad idea.  Better refactor.
2020-06-29 11:05:41 +09:00
Nobuyoshi Nakada 9ce2066209
Improved error messages for mjit option
and default jit-verbose to 1 if no argument.
2020-06-23 10:01:24 +09:00
Nobuyoshi Nakada 2c3c6c96cf
Defer initialization
Defer initialization of extension libraries, loading prelude files
and requiring files, and skip if dump options are given.
2020-05-16 17:37:28 +09:00
Yusuke Endoh 39365b46e2
Merge pull request #3047 from mame/suppress-backtrace
Add `--suppress-backtrace=num` option to limit the backtrace length
2020-05-15 01:22:56 +09:00
卜部昌平 9e41a75255 sed -i 's|ruby/impl|ruby/internal|'
To fix build failures.
2020-05-11 09:24:08 +09:00
卜部昌平 d7f4d732c1 sed -i s|ruby/3|ruby/impl|g
This shall fix compile errors.
2020-05-11 09:24:08 +09:00
Yusuke Endoh 91e4e2403e internal/process.h: add a no-warning simple wrapper for fork(2)
As fork(2) is deprecated, its calls must be guarded by
`COMPILER_WARNING_IGNORED(-Wdeprecated-declarations)`.
All usages of fork(2) in process have been alread guarded.  A new call
to fork(2) was added in ruby.c with f22c4ff359.
This caused a build failure on Solaris 11.

It may hide a bug to guard big code unnecessarily, so this change
introduces a simple wrapper "rb_fork" whose definition is guarded, and
replaces all calls to fork(2) with the wrapper function.
2020-05-02 21:34:10 +09:00
Nobuyoshi Nakada a07cbacd23 Honor COLUMNS [Feature #16754] 2020-04-12 14:58:13 +09:00
Nobuyoshi Nakada cc68d2fbd0 Hightlight usage [Feature #16754] 2020-04-12 14:58:13 +09:00
Nobuyoshi Nakada 3825662d77 Set up environment variable for pager [Feature #16754] 2020-04-12 14:58:13 +09:00
Nobuyoshi Nakada e6551d835f PAGER without fork&exec too [Feature #16754] 2020-04-12 14:58:13 +09:00
Nobuyoshi Nakada f22c4ff359 View the help message with PAGER [Feature #16754]
View the help message wth pager designed by RUBY_PAGER or PAGER
environment variable, unless that value is empty.
2020-04-12 14:58:13 +09:00
卜部昌平 9e6e39c351
Merge pull request #2991 from shyouhei/ruby.h
Split ruby.h
2020-04-08 13:28:13 +09:00
Yusuke Endoh c6b26f5ccf io.c, ruby.c: include internal/variable.h for rb_gvar_readonly_setter
Same as 053f78e139.
emscripten requires a prototype declaration of rb_gvar_readonly_setter
if it is refered as a function pointer.
2020-01-09 23:57:46 +09:00
卜部昌平 5e22f873ed decouple internal.h headers
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead.  This would significantly
speed up incremental builds.

We take the following inclusion order in this changeset:

1.  "ruby/config.h", where _GNU_SOURCE is defined (must be the very
    first thing among everything).
2.  RUBY_EXTCONF_H if any.
3.  Standard C headers, sorted alphabetically.
4.  Other system headers, maybe guarded by #ifdef
5.  Everything else, sorted alphabetically.

Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
2019-12-26 20:45:12 +09:00
Nobuyoshi Nakada 3739ddc36f
Added `-W:experimental` to `--help` [Feature #16420] 2019-12-24 12:11:42 +09:00
Takashi Kokubun 4de2297d74
Re-apply f85e8d2a8b keeping characters < 80 2019-12-22 16:07:12 -08:00
Yusuke Endoh f156f70b31 Revert "Fixed misspellings"
This reverts commit f85e8d2a8b.

It violated the limit of width (> 80) and caused the test failure
2019-12-23 09:02:49 +09:00
Nobuyoshi Nakada f85e8d2a8b
Fixed misspellings
Fixed misspellings reported at [Bug #16437]
2019-12-23 08:41:34 +09:00
Nobuyoshi Nakada 484c1be895
Added `-W:experimental` command line option
[Feature #16420]
2019-12-20 23:51:17 +09:00
Nobuyoshi Nakada a84ad24386
Added -W: command line option
To manage `Warning[category]` flags.  Only `-W:deprecated` and
`-W:no-deprecated` are available now.  [Feature #16345]
2019-12-20 23:05:22 +09:00
Nobuyoshi Nakada 6876aa38f6
Added `FEATURE_SET_RESTORE` macro 2019-12-20 22:58:09 +09:00
Koichi Sasada 2c5c60754c use compiled binary for gem_prelude.rb.
`gem_prelude.rb` is not compiled yet. This patch compile it to
compiled binary.
2019-12-11 11:24:42 +09:00
Takashi Kokubun a19d625e66
Allow specifying arbitrary MJIT flags by --jit-debug
This is a secret feature for me. It's only for testing and any behavior
with this flag override is unsupported.

I needed this because I sometimes want to add debug options but do not
want to disable optimizations, for using Linux perf.
2019-12-01 00:58:47 -08:00
卜部昌平 7a9b2039b7 delete unused codes
Suppress compiler warnings.
2019-11-18 18:28:03 +09:00
Jeremy Evans ffd0820ab3 Deprecate taint/trust and related methods, and make the methods no-ops
This removes the related tests, and puts the related specs behind
version guards.  This affects all code in lib, including some
libraries that may want to support older versions of Ruby.
2019-11-18 01:00:25 +02:00
Jeremy Evans c5c05460ac Warn on access/modify of $SAFE, and remove effects of modifying $SAFE
This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.

This modifies some internal functions that took a safe level argument
to no longer take the argument.

rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.

One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd.  We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.
2019-11-18 01:00:25 +02:00
Yusuke Endoh 711c40ebdc Refactor parser_params by removing "in_main" flag
The relation between parser_param#base_block and #in_main were very
subtle.
A main script (that is passed via a command line) was parsed under
base_block = TOPLEVEL_BINDING and in_main = 1.
A script loaded by Kernel#require was parsed under
base_block = NULL and in_main = 0.
If base_block is non-NULL and in_main == 0, it is parsed by Kernel#eval
or family.

However, we know that TOPLEVEL_BINDING has no local variables when a
main script is parsed.  So, we don't have to parse a main script under
base_block = TOPLEVEL_BINDING.

Instead, this change parses a main script under base_block = 0.
If base_block is non-NULL, it is parsed by Kernel#eval or family.
By this simplication, "in_main" is no longer needed.
2019-10-04 02:30:36 +09:00
卜部昌平 7bcfd9189a drop-in type check for rb_define_global_function
We can check the function pointer passed to rb_define_global_function
like we do so in rb_define_method.  It turns out that almost anybody
is misunderstanding the API.
2019-08-29 18:34:09 +09:00
卜部昌平 ae2dc3f217 rb_define_hooked_variable now free from ANYARGS
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct.  This commit uses rb_gvar_getter_t /
rb_gvar_setter_t for rb_define_hooked_variable /
rb_define_virtual_variable which revealed lots of function prototype
inconsistencies.  Some of them were literally decades old, going back
to dda5dc00cf.
2019-08-27 15:52:26 +09:00
Yusuke Endoh 085d0e5ccb ruby.c (name_match_p): remove unnecessary condition
It always returns immediately when len was decremented to zero.
So len is always positive.  This change will suppress Coverity Scan
warning.
2019-07-14 21:58:22 +09:00
git 14b5ccf91b * expand tabs. 2019-06-04 09:36:32 +09:00
Nobuyoshi Nakada aa8d393d8a
EOF by 2 ^D on a TTY
Terminate the input from a TTY by 2 ^D at the middle of line, like
as many programs, `cat`, `perl` and so on, do.  By the first ^D,
the line will be sent without a newline, and then EOF will be send
by the next ^D.
2019-06-04 09:24:17 +09:00
Nobuyoshi Nakada ac00bdc8a8
Do not modify shared array
[Bug #15821]
2019-05-21 14:17:51 +09:00
nobu b84eed5dd8 ruby.c: respect features by command line
* ruby.c (process_options): feature options in command line
  arguments take precedence over options in RUBYOPT environment
  variable.  [ruby-core:92052] [Bug #15738]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67388 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-30 14:31:35 +00:00
nobu de0ef1a9df [DOC] fix markups [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-28 03:33:35 +00:00
k0kubun 0fa4a6a618 Change defaults of --jit options
* --jit-min-calls: 5 -> 10000

--jit-min-calls=5 obviously can compile non hotspot. This was not a
problem for MJIT-benchmarks and Optcarrot because the former has very
few hot optimiziable methods and the latter is likely to trigger
compilation of hotspot by its intensive calls to optimizable hotspot
methods and has a very short window to allow limited compilations.

In real-world applications, it has more time to compile more methods and
it pressures computer's limited resources like icache. We should avoid
compiling too many methods. Also compiling many methods exhausts time
budget for compilation in one ruby process lifetime and delays the "JIT
compaction" of Ruby 2.6.

JVM is known to use 1,500 for C1 (client) compiler and 10,000 for C2
(server) compiler for -XX:CompileThreshold by default.
https://docs.oracle.com/javase/8/embedded/develop-apps-platforms/codecache.htm

When things are called X,000 times, requiring 10,000 can eliminate
compilation of methods which are called only once in these X,000
iterations and obviously not hotspot. And in fact things like
unicorn-worker-killer restarts unicorn process every 4096 requests.
So I felt 10,000 is good for such an environment.

* --jit-max-cache: 1000 -> 100

By the same reason stated above, we should not allow compiling many
methods especially on MJIT which has a larger overhead due to poor code
locality by dlopen and whose code is also likely to be bigger by just
inlining many VM instructions with -O3 rather than directly generating
low-level code.

In JVM -XX:ReservedCodeCacheSize is 32M for reserved and 48M for maximum.
--jit-max-cache=1,000 could be closer to it, but in this case MJIT's
compilation is slow due to data synchronization between threads (to be
improved in Ruby 2.7 though) and we do not want to delay the "JIT
compaction" for a long time.

So I chose a really conservative number for this, but by having method
inlining in the future, wider range could be optimized even with this
value.

* Optcarrot

--disable-gems, --benchmark Lan_Master.nes 12 attempts.
No significant impact.

| r67276             | r67276 --jit      | after --jit       |
|:-------------------|:------------------|:------------------|
| 50.44369263063978  | 72.87390680773056 | 73.47873485047297 |
| 50.58788746124193  | 78.06820808947026 | 78.29723420171945 |
| 50.77509250801378  | 80.29010348842613 | 78.94689404460769 |
| 50.935361702064405 | 80.42796829926374 | 80.39539527351525 |
| 51.27352672981195  | 81.98758158033202 | 81.6754198664817  |
| 51.720715743242124 | 82.00118535811626 | 82.22960569251283 |
| 51.89643169822524  | 82.2290091613556  | 82.5013636146388  |
| 51.95895898113868  | 82.37318990939565 | 82.74002377794454 |
| 52.10124886807968  | 82.48796686037502 | 83.23354941183932 |
| 52.292280637519376 | 83.0265226541066  | 84.01552618012572 |
| 52.51856237784144  | 83.8797360318052  | 84.8588319093393  |
| 52.65076845986818  | 84.80037351256634 | 85.13577756273656 |

* Railsbench

`WARMUP=20000 BENCHMARK=1000 bin/bench` of https://github.com/k0kubun/railsbench.
It gets closer to --disable=jit.

|           | r67276 | r67276 | after  |
|           |        | --jit  | --jit  |
|:----------|:-------|:-------|:-------|
| req/s     | 891.3  | 742.2  | 841.5  |
|:----------|:-------|:-------|:-------|
| 50%ile ms | 1.00   | 1.21   | 1.08   |
| 66%ile ms | 1.02   | 1.24   | 1.09   |
| 75%ile ms | 1.03   | 1.28   | 1.10   |
| 80%ile ms | 1.03   | 1.30   | 1.11   |
| 90%ile ms | 1.09   | 1.42   | 1.15   |
| 95%ile ms | 1.32   | 1.65   | 1.27   |
| 98%ile ms | 4.79   | 2.23   | 1.81   |
| 99%ile ms | 5.68   | 7.52   | 6.64   |
|100%ile ms | 6.52   | 9.69   | 8.59   |

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67277 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-17 05:13:38 +00:00
nobu 50784a0a44 Defer escaping control char in error messages
* eval_error.c (print_errinfo): defer escaping control char in
  error messages until writing to stderr, instead of quoting at
  building the message.  [ruby-core:90853] [Bug #15497]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66753 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-08 09:08:31 +00:00
ko1 496267023c fix missed script_compiled events. [Bug #15471]
* ruby.c (process_options): script_compiled events are missed on
  command line -e or specified file. this commit fix it.
  [Bug #15471]

  This patch should be backport to Ruby 2.6 branch.

* vm_core.h (rb_exec_event_hook_script_compiled): introduce utility
  function to invoke a script_compiled event.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66595 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-27 17:39:17 +00:00
k0kubun b756b25c60 ruby.c: do not surface MJIT to user
In some places, both JIT and MJIT are being used, but it could be
confusing for new comers. We're not explaining MJIT on NEWS file or release
notes as well. So we consider MJIT as an internal term of implementation
like YARV.

configure.ac: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-19 14:00:49 +00:00
nobu f53bd45d6f AST.of -e script
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-10 11:43:02 +00:00
ko1 543acfabdc fix a warning message.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65264 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-21 02:38:26 +00:00
ko1 b710785f1a add disabling MJIT features option.
* configure.ac: introduce new configure option `--enable-mjit` and
  `--disable-mjit`. Default is "enable".
  `--disable-mjit` disables all of MJIT features so that `ruby --jit`
  can't enable MJIT.
  This option affect a macro `USE_MJIT`.
  This change remove `--enable/disable-install-mjit-header` option.

* Makefile.in: introduce the `ENABLE_MJIT` variable.

* common.mk: use `ENABLE_MJIT` option.

* internal.h: respect `USE_MJIT`. Same as other *.c, *.h.

* test/ruby/test_jit.rb: check `ENABLE_MJIT` key of rbconfg.rb.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65204 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-20 06:53:00 +00:00
nobu 4e133fdb7e Just a shebang is valid code
[ruby-core:89240] [Bug #15190]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64900 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-02 18:51:21 +00:00
nobu 6c70fede0c version.c: separate Init_ruby_description
* version.c (Init_ruby_description): separate to initialize
  RUBY_DESCRIPTION constant according to mjit.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64264 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-10 05:02:35 +00:00
nobu 9fbb66610c ruby.c: taint ARGV on Windows
* ruby.c (external_str_new_cstr): strings come from the external
  should be tainted.  [ruby-dev:50596] [Bug #14941]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64071 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-27 05:42:56 +00:00
nobu dff596be18 dladdr() is declared with non-const pointer on Solaris
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64064 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-26 09:34:30 +00:00
k0kubun 053cdaf7ee ruby.c: accept --disable-jit option
by promoting jit to feature flag.

mjit.h: update comment about mjit_opts.on

test_rubyoptions.rb: add test for switching JIT enablement

"--jit" flag usage may be deprecated later, but not discussed yet.

[Feature #14878]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63995 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-18 12:45:54 +00:00
nobu e39b2cff8a ruby.c: disable DidYouMean as gem
* ruby.c (process_options): as DidYouMean requires Rubygems, disable
  the former when the latter is disabled too.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63977 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-16 05:05:27 +00:00
nobu b53114dd99 set up mjit.on at initialization
* ruby.c (cmdline_options_init): set up mjit.on flag by
  MJIT_FORCE_ENABLE in the initialization function.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63822 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-02 07:42:24 +00:00
nobu 0534566839 ruby.c: making hidden objects
* ruby.c (add_modules): make hidden objects by particular
  functions.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63634 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-11 23:33:35 +00:00
nobu 7cdc78e284 ruby.c: localize a variable
* ruby.c (ruby_init_loadpath_safe): moved libdir to the block
  where it is used.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63477 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-21 12:25:45 +00:00
nobu 68b87aedde ruby.c: runtime_libruby_path
* ruby.c (runtime_libruby_path): hoisted out platform dependent
  routine to get the loaded runtime library path.
  cygwin_conv_path does path separator and WCHAR to UTF-8
  conversions too.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63476 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-21 12:24:47 +00:00
shyouhei 23c74845ed RSTRING_PTR is not guaranteed to be char*-aligned
We need to ensure aligned memory access by allocating
another memory region.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63388 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-10 02:54:04 +00:00
nobu 6dafb4c910 ruby.c: moved libdir
* ruby.c (ruby_init_loadpath_safe): moved libdir name inside
  LOAD_RELATIVE.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63271 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-27 05:20:40 +00:00
nobu 4eb2c1e121 ruby.c: fix compilation error
* ruby.c (ruby_init_loadpath_safe): fix compilation error when
  ENABLE_MULTIARCH but not universal binary.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63270 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-27 04:36:34 +00:00
nobu 67800ea9c7 mjit.c: prefix and archdir in init
* ruby.c (ruby_init_loadpath_safe): store prefix and archlibdir
  paths.

* mjit.c (compile_c_to_so, init_header_filename): use just one
  library path on Windows.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63269 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-27 02:02:29 +00:00
nobu 8036af48a5 ruby.c (ruby_init_loadpath_safe): constify
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63263 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-26 12:36:26 +00:00
nobu e68a8b1e9a ruby.c: exit by --version
* ruby.c (usage): stated exiting by `--version` option with
  nothing done.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62932 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-27 00:44:34 +00:00
nobu 2a99d5f9ef ruby.c: fix --verbose description in usage
* ruby.c (usage): fix the description of `--verbose` option, which
  does not print the version number unlike `-v` option.
  [ruby-core:86307] [Bug #14633]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62927 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-26 16:25:05 +00:00
k0kubun 2d1a415470 ruby.c: fix typo in r62530
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62535 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-22 15:29:33 +00:00
k0kubun 171c496e50 version.c: show +JIT when --jit is passed
in version output.
version.h: ditto
ruby.c: propagate option for it
common.mk: updated dependency for version.c

mjit.c: overwrites the RUBY_DESCRIPTION to have +JIT when --jit is passed

test/ruby/test_rubyoptions.rb: add test for them

Only `ruby --jit -v` will have "+JIT", but this is intentional.
This may not be convenient for debugging by ticket with `ruby -v`,
but it's convenient for benchmark tools that pass options (--jit)
when showing it. At least such behavior is planned for benchmark_driver.gem
and this behavior is designed for it. Other benchmark tools are
recommended to follow the behavior too if they show version.
RUBY_DESCRIPTION might be useful for it too.

The position of "+JIT" is changed from original proposal because other
platforms like JRuby and TruffleRuby end it with archtecture.
It's made similar to JRuby, but it's upper-cased because Matz made approval
for "+JIT" in the ticket.

Example:
$ ruby -v
ruby 2.6.0dev (2018-02-22 trunk 62529) [x86_64-linux]
$ ruby --jit -v
ruby 2.6.0dev (2018-02-22 trunk 62529) +JIT [x86_64-linux]

After --jit is made default in the future, this output may be removed.
So do not rely on this output if possible.

[Feature #14462]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62530 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-22 14:53:17 +00:00
nobu 9dc2d2855e no --jit-cc
* ruby.c (setup_mjit_options): removed --jit-cc option, since
  mjit header is affected by generated config.h which depends on
  the given compiler, so it cannot work with different compilers.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62379 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-12 03:39:18 +00:00
usa 714836c152 remove declaration of ruby_engine from internal.h
* internal.h (ruby_engine): remove declaration of ruby_engine because
	  it's declared at ruby/version.h.

	* ruby.c: include ruby/version.h for ruby_engine.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62274 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-07 04:04:33 +00:00
k0kubun 75eff5002d ruby.c: add MJIT_FORCE_ENABLE macro
to always enable MJIT for testing.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62264 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-06 16:03:14 +00:00
k0kubun 1fd0a7a660 mjit.c: determine prefix of MJIT header at runtime
so that MJIT can work if Ruby is distributed as prebuilt binary.

Now mjit_init() depends on the internal const TMP_RUBY_PREFIX which is
only available after ruby_init_loadpath_safe() (L1608) and before
ruby_init_prelude() (L1681). So the place of mjit_init() is moved.

Makefile.in: Removed static prefix from MJIT_HEADER_ISNTALL_DIR macro.
And this removes the unused LIBRUBY_LIBDIR macro as well.
win32/Makefile.sub: ditto.

Patch by: Lars Kanis <lars@greiz-reinsdorf.de>
[Bug #14445]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62238 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05 16:51:12 +00:00
k0kubun fd44a5777f mjit.c: merge MJIT infrastructure
that allows to JIT-compile Ruby methods by generating C code and
using C compiler.  See the first comment of mjit.c to know what this
file does.

mjit.c is authored by Vladimir Makarov <vmakarov@redhat.com>.
After he invented great method JIT infrastructure for MRI as MJIT,
Lars Kanis <lars@greiz-reinsdorf.de> sent the patch to support MinGW
in MJIT. In addition to merging it, I ported pthread to Windows native
threads. Now this MJIT infrastructure can be compiled on Visual Studio.

This commit simplifies mjit.c to decrease code at initial merge. For
example, this commit does not provide multiple JIT threads support.
We can resurrect them later if we really want them, but I wanted to minimize
diff to make it easier to review this patch.

`/tmp/_mjitXXX` file is renamed to `/tmp/_ruby_mjitXXX` because non-Ruby
developers may not know the name "mjit" and the file name should make
sure it's from Ruby and not from some harmful programs.  TODO: it may be
better to store this to some temporary directory which Ruby is already using
by Tempfile, if it's not bad for performance.

mjit.h: New. It has `mjit_exec` interface similar to `vm_exec`, which is
for triggering MJIT. This drops interface for AOT compared to the original
MJIT.

Makefile.in: define macros to let MJIT know the path of MJIT header.
Probably we can refactor this to reduce the number of macros (TODO).
win32/Makefile.sub: ditto.

common.mk: compile mjit.o and mjit_compile.o. Unlike original MJIT, this
commit separates MJIT infrastructure and JIT compiler code as independent
object files. As initial patch is NOT going to have ultra-fast JIT compiler,
it's likely to replace JIT compiler, e.g. original MJIT's compiler or some
future JIT impelementations which are not public now.

inits.c: define MJIT module. This is added because `MJIT.enabled?` was
necessary for testing.
test/lib/zombie_hunter.rb: skip if `MJIT.enabled?`. Obviously this
wouldn't work with current code when JIT is enabled.
test/ruby/test_io.rb: skip this too. This would make no sense with MJIT.

ruby.c: define MJIT CLI options. As major difference from original MJIT,
"-j:l"/"--jit:llvm" are renamed to "--jit-cc" because I want to support
not only gcc/clang but also cl.exe (Visual Studio) in the future. But it
takes only "--jit-cc=gcc", "--jit-cc=clang" for now. And only long "--jit"
options are allowed since some Ruby committers preferred it at Ruby
developers Meeting on January, and some of options are renamed.
This file also triggers to initialize MJIT thread and variables.
eval.c: finalize MJIT worker thread and variables.
test/ruby/test_rubyoptions.rb: fix number of CLI options for --jit.

thread_pthread.c: change for pthread abstraction in MJIT. Prefix rb_ for
functions which are used by other files.
thread_win32.c: ditto, for Windows.  Those pthread porting is one of major
works that YARV-MJIT created, which is my fork of MJIT, in Feature 14235.
thread.c: follow rb_ prefix changes

vm.c: trigger MJIT call on VM invocation. Also trigger `mjit_mark` to avoid
SEGV by race between JIT and GC of ISeq. The improvement was provided by
wanabe <s.wanabe@gmail.com>.
In JIT compiler I created and am going to add in my next commit, I found
that having `mjit_exec` after `vm_loop_start:` is harmful because the
JIT-ed function doesn't proceed other ISeqs on RESTORE_REGS of leave insn.
Executing non-FINISH frame is unexpected for my JIT compiler and
`exception_handler` triggers executions of such ISeqs. So `mjit_exec`
here should be executed only when it directly comes from `vm_exec` call.
`RubyVM::MJIT` module and `.enabled?` method is added so that we can skip
some tests which don't expect JIT threads or compiler file descriptors.

vm_insnhelper.h: trigger MJIT on method calls during VM execution.

vm_core.h: add fields required for mjit.c. `bp` must be `cfp[6]` because
rb_control_frame_struct is likely to be casted to another struct. The
last position is the safest place to add the new field.
vm_insnhelper.c: save initial value of cfp->ep as cfp->bp. This is an
optimization which are done in both MJIT and YARV-MJIT. So this change
is added in this commit. Calculating bp from ep is a little heavy work,
so bp is kind of cache for it.

iseq.c: notify ISeq GC to MJIT. We should know which iseq in MJIT queue
is GCed to avoid SEGV.  TODO: unload some GCed units in some safe way.

gc.c: add hooks so that MJIT can wait GC, and vice versa. Simultaneous
JIT and GC executions may cause SEGV and so we should synchronize them.

cont.c: save continuation information in MJIT worker. As MJIT shouldn't
unload JIT-ed code which is being used, MJIT wants to know full list of
saved execution contexts for continuation and detect ISeqs in use.

mjit_compile.c: added empty JIT compiler so that you can reuse this commit
to build your own JIT compiler. This commit tries to compile ISeqs but
all of them are considered as not supported in this commit. So you can't
use JIT compiler in this commit yet while we added --jit option now.

Patch author: Vladimir Makarov <vmakarov@redhat.com>.

Contributors:
Takashi Kokubun <takashikkbn@gmail.com>.
wanabe <s.wanabe@gmail.com>.
Lars Kanis <lars@greiz-reinsdorf.de>.

Part of Feature 12589 and 14235.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62189 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 06:58:09 +00:00
nobu 592dcccdf9 ruby.c: no VARIABLE_LIBPATH
* ruby.c (ruby_init_loadpath_safe): removed code using fixed size
  path buffer.  relative load path is supported only on platforms
  where dladdr is available, or on Windows, so !VARIABLE_LIBPATH
  code is not used.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-29 06:54:22 +00:00
normal a41386a2dc ruby.c (open_load_file): avoid shadowing variable for errno
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62027 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-24 20:50:29 +00:00
nobu e9cb552ec9 internal.h: remove dependecy on ruby/encoding.h
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61713 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 06:24:11 +00:00
mame 92b81dc597 make rb_iseq_new* accept rb_ast_body_t instead of NODE*
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61609 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-05 08:59:22 +00:00
mame 503b858cef node.h: define rb_ast_body_t and restructure rb_ast_t
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61608 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-05 08:59:20 +00:00
nobu b7d6b2299b ruby.c: script __dir__ encoding
* ruby.c (process_options): fallback to the encoding of the script
  name since rb_realpath_internal() cannot convert the encoding
  when it is ASCII-8BIT.

* test/ruby/test_rubyoptions.rb (test___dir__encoding): explicitly
  pass environment variables for locale as they are overriden by
  invoke_ruby.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61297 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-16 13:25:11 +00:00
ko1 85fcaf025d * node.h (ast_t): renamed to `rb_ast_t`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60565 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-29 15:51:23 +00:00
mame e35fe8d11b Revert "Revert "Manage AST NODEs out of GC""
This re-introduces r60485.
This reverts commit 5a176b75b1.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 16:44:57 +00:00
mame 5a176b75b1 Revert "Manage AST NODEs out of GC"
This reverts commit 620ba74778.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60486 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 16:02:34 +00:00
mame 620ba74778 Manage AST NODEs out of GC
NODEs in AST are no longer objects managed by GC.  This change will
remove the restriction imposed by the GC.  For example, a NODE can use
more than five words (this is my primary purpose; we want to store the
position data for each NODE, for coverage library), or even a NODE can
have variable length (some kinds of NODEs have unused fields).
To do this, however, we need more work, since Ripper still uses T_NODE
objects managed by the GC.

The life time of NODEs is more obvious than other kinds of objects; they
are created at parsing, and they become disused immediately after
compilation.  This change releases all NODEs by a few `xfree`s after
compilation, so performance will be improved a bit.  In extreme example,
`eval("x=1;" * 10000000)` runs much faster (40 sec. -> 7.8 sec. on my
machine).

The most important part of this change is `ast_t` struct, which has
three contents: (1) NODE buffer (malloc'ed memory), (2) a reference to
the root NODE, and (3) an array that contains objects that must be
marked during parsing (such as literal objects).  Some functions that
had received `NODE*` arguments, must now receive `ast_t*`.

* node.c, node.h: defines `ast_t` struct and related operations.
* gc.c, internal.h: defines `imemo_ast`.
* parse.y: makes `parser_params` struct have a reference to `ast_t`.
  Instead of `rb_node_newnode`, use `rb_ast_newnode` to create a NODE.
* iseq.c, load.c, ruby.c, template/prelude.c.tmpl: modifies some
  functions to handle `ast_t*` instead of `NODE*`.
* test/ruby/test_gc.rb: ad-hoc fix for a failed test.  The test assumes
  GC eden is increased at startup by NODE object creation.  However,
  this change now create no NODE object, so GC eden is not necessarily
  increased.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60485 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 15:59:02 +00:00
nobu d0d32ba1e8 ruby.c: fix r60393
* ruby.c (load_file_internal): set loop options after parsing
  shebang line.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60396 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-24 11:09:41 +00:00
nobu 30f5c55890 parse.y: rb_parser_set_options
* parse.y (yycompile0): append top-level addenda before appending
  prelude nodes.

* parse.y (rb_parser_set_options): set top-level addendum options
  before parsing.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60393 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-24 07:41:48 +00:00
usa 3e8c0c577c The encoding of __FILE__ and __dir__ should be same
* ruby.c (process_options): convert the real path of the script to locale
  encoding if its encoding is not locale (maybe UTF-8) on Windows/OS X.
  this change makes the encoding of __dir__ to the same encoding of __FILE__
  when the script name is passed from commandline.

* test/ruby/test_options.rb (test___dir__encoding): test for this change.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60300 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 15:43:05 +00:00
nobu f16f0441d6 ruby.c: reject NUL in $0
* ruby.c (ruby_setproctitle): raise if the argument contains NUL
  char.  process title is a NUL-terminated string.
  [ruby-core:82425] [Bug #13829]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60215 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-19 11:24:03 +00:00
sonots c18503c25a ruby.c: show help messages of --dump
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59973 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-20 05:01:09 +00:00
nobu 79e0a19353 ruby.c: paragraph mode by -00
* ruby.c (proc_options): set to paragraph mode, if -00 is given,
  as well as perl and -R0 option in 0.49.
  [ruby-core:81987] [Bug #13736]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59970 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-20 00:53:47 +00:00
nobu 157ee2bdbc ruby.c: ignore non-option in shebang line
* ruby.c (moreswitches): process all words as options only in
  an environment variable, but not in a shebang line.
  [ruby-core:82267] [Bug #13786]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59564 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-10 05:54:56 +00:00
nobu 8081826be6 ruby.c: origarg
* ruby.c (dladdr_path, ruby_set_argv): add guards for origarg.
  [ruby-core:82272] [Bug #13788]

* ruby.c (proc_options, process_options, ruby_process_options):
  set origarg if not set yet.

* ruby.c (process_options): prefer argv in the argument to origarg
  as program name.

* ruby.c (ruby_sysinit): set origarg only if argc and argv seem
  valid.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59549 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-09 11:03:55 +00:00
nobu 6af650256b RUBY_DEVEL flag
* configure.in: define RUBY_DEVEL only in the trunk.

* gc.c: enable runtime rgengc debug if RUBY_DEVEL

* ruby.c (debug_option): enable RUBY_DEBUG in --debug option only
  if RUBY_DEVEL.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-06-21 04:34:25 +00:00
nobu 793f5d8fe8 ruby.c: debug options in command line
* ruby.c (debug_option): parse options in --debug command line
  option same as RUBY_DEBUG env.  available only in the trunk.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59129 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-06-21 02:44:23 +00:00
nobu 238a6246ff ruby.c: script name in UTF-8
* ruby.c (process_options): keep script name in UTF-8 if UTF8_PATH
  to get rid of loss by conversion.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59041 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-06-08 02:07:42 +00:00
nobu 71b3abeba3 ruby.c: dladdr_path is not used on cygwin
* ruby.c (dladdr_path): dladdr is provided on recent cygwin, but
  GetModuleFileNameW is used instead of it.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59007 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-06-03 12:47:11 +00:00
duerst d0e7329df3 improve English for a warning message
On DOSish systems, there is a warning message for \r\n line endings on shebang line.
Improve this message from "shebang line ends with \r may cause a problem"
to "shebang line ending with \r may cause problems".

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58981 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-06-01 04:38:03 +00:00
nobu ab55cd9fb5 ruby.c: file in load_file argument
* ruby.c (load_file): move opened file to an argument, to reduce
  open/close calls in the near future.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58938 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-28 09:49:30 +00:00
nobu 3ade78fff6 ruby.c: shrink ruby_cmdline_options_t
* ruby.c (ruby_cmdline_options_t): reordered members and turned
  simple flags into bit fields to reduce the size (136->104 on
  LP64).

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58937 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-28 09:49:29 +00:00
nobu d9b9423ba8 ruby.c: encode script name
* ruby.c (process_options): encode script name to locale encoding
  instead of associate, if UTF-8 path.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58767 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-17 04:47:05 +00:00
nobu c7c1f371a4 ruby.c: defining DATA
* ruby.c (load_file_internal): define DATA here instead of ensure
  func.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57485 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-31 12:55:00 +00:00
nobu 3408e3b460 ruby.c: forbid options
* ruby.c (forbid_setid): constified.

* ruby.c (process_options): forbid if setid earlier.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57484 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-31 12:54:59 +00:00
usa 42d9712e78 declare `rb_w32_sysinit` in header.
* include/ruby/win32.h (rb_w32_sysinit): declare.
  [ruby-core:78444] [Bug #12994]

* ruby.c (ruby_sysinit): follow above change.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57072 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-12-13 05:17:48 +00:00
nobu 1db7b0ad7c ruby.c: dump option
* ruby.c (process_options): dump specified informations all, not
  only first one.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57017 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-12-07 14:39:24 +00:00
nobu 4791b2bc22 ruby.c: retry loading with GC
* ruby.c (open_load_file): retry after GC when the limit for open
  file descriptors reached.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56463 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-10-21 07:06:27 +00:00
usa e592a735cb * ruby.c (open_load_file): revert r56385. it introduced incompatibility
about `DATA.binmode?`.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56401 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-10-12 12:07:16 +00:00
nobu 21f1b89b9d ruby.c: fix macros
* ruby.c (open_load_file): FILE_ALT_SEPARATOR and EXEEXT are
  config.status variables, not available in config.h.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56400 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-10-12 08:24:36 +00:00
nobu 8c3af8ecc0 ruby.c: bind fd before waiting
* ruby.c (open_load_file): bind the open fd to an IO instance
  before waiting FIFO, not to leak the fd if interrupted.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56388 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-10-10 06:22:30 +00:00