with opt_send_without_block insn if call cache has valid ISeq.
If the receiver is not optimized target of opt_key (i.e. Hash or Array),
it triggers JIT cancel and it would be slow.
This change allows JIT to drop the check for Hash/Array and continue to
execute JIT even if the receiver is not Hash or Array.
See the following benchmark results. It's not improved so much, but it
would be effective when we achieve Ruby method inlining in
_mjit_compile_send.erb.
* Micro benchmark
Given the following bench.rb,
```
class HashWithIndifferentAccess < Hash
def []=(key, value)
super(key.to_s, value)
end
def [](key)
super(key.to_s)
end
end
indhash = HashWithIndifferentAccess.new
indhash[:foo] = 'bar'
key = 'foo'
100000000.times do
indhash[key]
end
```
** before
```
$ time ./ruby --disable-gems --jit-verbose=1 /tmp/bench.rb
JIT success (31.4ms): block in <main>@/tmp/bench.rb:15 -> /tmp/_ruby_mjit_p18206u0.c
JIT success (669.3ms): []@/tmp/bench.rb:6 -> /tmp/_ruby_mjit_p18206u1.c
Successful MJIT finish
./ruby --disable-gems --jit-verbose=1 /tmp/bench.rb 12.21s user 0.04s system 107% cpu 11.394 total
```
** after
```
$ time ./ruby --disable-gems --jit-verbose=1 /tmp/bench.rb
JIT success (41.0ms): block in <main>@/tmp/bench.rb:15 -> /tmp/_ruby_mjit_p17293u0.c
JIT success (679.0ms): []@/tmp/bench.rb:6 -> /tmp/_ruby_mjit_p17293u1.c
Successful MJIT finish
./ruby --disable-gems --jit-verbose=1 /tmp/bench.rb 11.54s user 0.06s system 108% cpu 10.726 total
```
The execution time is shortened.
* optcarrot benchmark
Optcarrot has no room to be improved by this change. Almost nothing is changed.
fps: 59.54 (before) -> 59.51 (after)
* discourse benchmark
I expected this to be improved a little, but it isn't too.
** before (JIT)
```
categories_admin:
50: 12
75: 13
90: 14
99: 22
home_admin:
50: 12
75: 13
90: 16
99: 22
topic_admin:
50: 12
75: 13
90: 15
99: 21
categories:
50: 18
75: 19
90: 23
99: 27
home:
50: 3
75: 4
90: 4
99: 12
topic:
50: 11
75: 11
90: 14
99: 20
```
** after (JIT)
```
categories_admin:
50: 12
75: 12
90: 16
99: 24
home_admin:
50: 12
75: 12
90: 14
99: 21
topic_admin:
50: 12
75: 13
90: 16
99: 21
categories:
50: 17
75: 18
90: 23
99: 32
home:
50: 3
75: 4
90: 4
99: 10
topic:
50: 11
75: 12
90: 13
99: 20
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62398 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
renamed from tool/ruby_vm/views/_mjit_compile_insn_line.erb.
Basically this file should handle everything about macro on JIT.
_mjit_compile_insn.erb: follow the refactoring
common.mk: follow the rename
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62384 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
in _mjit_compile_insn.erb to this file. As I'm going to add macro
expansions later, I want to separate such complex things from whole insn
compilation.
_mjit_compile_insn.erb: _mjit_compile_insn_line.erb part was removed.
common.mk: updated build system for them.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62383 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
mjit_compile.inc.erb: show unsupported insn name on --jit-verbose=1 too.
Also, removed osboleted workaround. Now some insn-related functions are
declared with MAYBE_UNUSED.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/ruby_vm/views/_mjit_compile_insn.erb: comment ID of
constant, calling method, and Symbol literal.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/ruby_vm/views/_mjit_compile_send.erb: `printf` modifier for
`rb_serial_t` which may not be `long long`, and '%ll' may not be
supported.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62351 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* mjit_compile.c (mjit_compile): name the original iseq pointer to
eliminate magic numbers.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62340 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I heard `error: conflicting types for 'restrict'` can be solved by
adding `-std=c99`.
Ideally we should use the same cflags which are used to compile vm.c,
but let me try this and see what happens on AIX.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62326 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
tool/ruby_vm/views/_insn_name_info.erb: on Linux, rb_vm_insn_name_offset
was needed to compile with --jit-debug (Usually --jit-debug requires
more symbols than the situation without --jit-debug because -O2 skips
some functions to compile).
vm.c: when running transform_mjit_header.rb with --jit-wait,
rb_source_location_cstr was repoted to be missing.
string.c: ditto, for rb_str_eql
numeric.c: ditto, for rb_float_eql
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62313 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to generate MJIT header.
Even if we can't build MJIT header, Ruby's build should success. And
compilers which are not explicitly supported are likely to fail to
transform MJIT header.
Also you can pass only gcc or clang to --jit-cc=xxx for now. Thus
generating header does never make sense.
So I decided to conservatively give up MJIT header generation.
But please feel free to add your favorite compiler's macro if you think
it's working. (Another workaround is passing -D__GNUC__ :p)
[Bug #14447] [Bug #14446]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62285 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and style of generated code.
I've used 2-space indentation at first but at some moment I started to
use insns.def contents for generated code. So the 4-space indentation
was introduced. But it does no longer make sense.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62255 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
AIX's header build is failing like:
./miniruby -I./lib -I. -I.ext/common ./tool/transform_mjit_header.rb "/usr/bin/gcc " rb_mjit_header.h .ext/include/powerpc-aix7.1.3.0/rb_mjit_min_header-2.6.0.h
error in initial header file:
/home/odaira/chkbuild/tmp/build/20180206T113302Z/tmp/20180206-15335556-aaiego.c:19:59: error: conflicting types for 'restrict'
extern size_t fread(void *restrict, size_t, size_t, FILE *restrict);
^
https://rubyci.org/logs/rubyci.s3.amazonaws.com/aix71_ppc/ruby-trunk/log/20180206T113302Z.log.html.gz#make
It's so hard to know the cause from current output. Let me add debug
output and see tomorrow's CI result.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62252 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because `Regexp#to_s` add `(?-mix:)`, it is not unnecessary.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62250 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit updates files so that aclocal.m4 generated by aclocal(1)
works well with our configure.ac
* ac_checking.m4: merged back to configure.ac because
aclocal(1) cannot handle this macro.
* ruby_append_options.m4: no longer used.
* ruby_check_va_copy.m4: define using AC_DEFUN so that
aclocal(1) can find this macro.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62243 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to whitelisted patterns. This fix is for NetBSD.
[Bug #14439]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62235 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I would like to stabilize RubyCI.
https://rubyci.org/logs/rubyci.s3.amazonaws.com/fedora26/ruby-trunk/log/20180205T123003Z.fail.html.gz
downloading did_you_mean-1.2.0.gem ... /home/hsbt/chkbuild/tmp/build/20180205T123003Z/tmp/ruby-snapshot20180205-23226-3201ha/ruby-2.6.0-r62226/tool/downloader.rb:212:in `rescue in download': failed to download did_you_mean-1.2.0.gem (RuntimeError)
SocketError: Failed to open TCP connection to rubygems.org:443 (getaddrinfo: Name or service not known): https://rubygems.org/downloads/did_you_mean-1.2.0.gem
from /home/hsbt/chkbuild/tmp/build/20180205T123003Z/tmp/ruby-snapshot20180205-23226-3201ha/ruby-2.6.0-r62226/tool/downloader.rb:123:in `download'
from /home/hsbt/chkbuild/tmp/build/20180205T123003Z/tmp/ruby-snapshot20180205-23226-3201ha/ruby-2.6.0-r62226/tool/downloader.rb:66:in `download'
from -e:4:in `<main>'
make[1]: *** [Makefile:1685: update-gems] Error 1
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62231 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
separating the macros. Applying the kept macros to code which is already
affected by the macros may cause errors in initial code.
This is hopefully the final fix for icc build failure.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62226 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
semi-automatically.
This is basically for icc's __DATE__, __TIME__, and many other families.
It causes an error by predefined macro redefinition.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62225 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/transform_mjit_header.rb (MJITHeader.check_code!): open
temporary file in binary mode too, not to include extra CRs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62222 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/vcs.rb (DebugSystem#system): pop option hash for old
version ruby which does not support `system` options.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62217 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/transform_mjit_header.rb: read and write as ASCII-8BIT to
make single-byte-optimizable always.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62215 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/transform_mjit_header.rb (separate_macro_and_code): return
macro and code separately as the name, and concat before output.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62214 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/transform_mjit_header.rb: print non-error messages to
STDOUT instead of STDERR. exit with false or abort instead of
exit 1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62205 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/transform_mjit_header.rb: add word boundary anchors and
match whole word to get rid of false `static` declarations,
e.g., rb_str_new_static.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62199 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
which has been developed by Takashi Kokubun <takashikkbn@gmail> as
YARV-MJIT. Many of its bugs are fixed by wanabe <s.wanabe@gmail.com>.
This JIT compiler is designed to be a safe migration path to introduce
JIT compiler to MRI. So this commit does not include any bytecode
changes or dynamic instruction modifications, which are done in original
MJIT.
This commit even strips off some aggressive optimizations from
YARV-MJIT, and thus it's slower than YARV-MJIT too. But it's still
fairly faster than Ruby 2.5 in some benchmarks (attached below).
Note that this JIT compiler passes `make test`, `make test-all`, `make
test-spec` without JIT, and even with JIT. Not only it's perfectly safe
with JIT disabled because it does not replace VM instructions unlike
MJIT, but also with JIT enabled it stably runs Ruby applications
including Rails applications.
I'm expecting this version as just "initial" JIT compiler. I have many
optimization ideas which are skipped for initial merging, and you may
easily replace this JIT compiler with a faster one by just replacing
mjit_compile.c. `mjit_compile` interface is designed for the purpose.
common.mk: update dependencies for mjit_compile.c.
internal.h: declare `rb_vm_insn_addr2insn` for MJIT.
vm.c: exclude some definitions if `-DMJIT_HEADER` is provided to
compiler. This avoids to include some functions which take a long time
to compile, e.g. vm_exec_core. Some of the purpose is achieved in
transform_mjit_header.rb (see `IGNORED_FUNCTIONS`) but others are
manually resolved for now. Load mjit_helper.h for MJIT header.
mjit_helper.h: New. This is a file used only by JIT-ed code. I'll
refactor `mjit_call_cfunc` later.
vm_eval.c: add some #ifdef switches to skip compiling some functions
like Init_vm_eval.
win32/mkexports.rb: export thread/ec functions, which are used by MJIT.
include/ruby/defines.h: add MJIT_FUNC_EXPORTED macro alis to clarify
that a function is exported only for MJIT.
array.c: export a function used by MJIT.
bignum.c: ditto.
class.c: ditto.
compile.c: ditto.
error.c: ditto.
gc.c: ditto.
hash.c: ditto.
iseq.c: ditto.
numeric.c: ditto.
object.c: ditto.
proc.c: ditto.
re.c: ditto.
st.c: ditto.
string.c: ditto.
thread.c: ditto.
variable.c: ditto.
vm_backtrace.c: ditto.
vm_insnhelper.c: ditto.
vm_method.c: ditto.
I would like to improve maintainability of function exports, but I
believe this way is acceptable as initial merging if we clarify the
new exports are for MJIT (so that we can use them as TODO list to fix)
and add unit tests to detect unresolved symbols.
I'll add unit tests of JIT compilations in succeeding commits.
Author: Takashi Kokubun <takashikkbn@gmail.com>
Contributor: wanabe <s.wanabe@gmail.com>
Part of [Feature #14235]
---
* Known issues
* Code generated by gcc is faster than clang. The benchmark may be worse
in macOS. Following benchmark result is provided by gcc w/ Linux.
* Performance is decreased when Google Chrome is running
* JIT can work on MinGW, but it doesn't improve performance at least
in short running benchmark.
* Currently it doesn't perform well with Rails. We'll try to fix this
before release.
---
* Benchmark reslts
Benchmarked with:
Intel 4.0GHz i7-4790K with 16GB memory under x86-64 Ubuntu 8 Cores
- 2.0.0-p0: Ruby 2.0.0-p0
- r62186: Ruby trunk (early 2.6.0), before MJIT changes
- JIT off: On this commit, but without `--jit` option
- JIT on: On this commit, and with `--jit` option
** Optcarrot fps
Benchmark: https://github.com/mame/optcarrot
| |2.0.0-p0 |r62186 |JIT off |JIT on |
|:--------|:--------|:--------|:--------|:--------|
|fps |37.32 |51.46 |51.31 |58.88 |
|vs 2.0.0 |1.00x |1.38x |1.37x |1.58x |
** MJIT benchmarks
Benchmark: https://github.com/benchmark-driver/mjit-benchmarks
(Original: https://github.com/vnmakarov/ruby/tree/rtl_mjit_branch/MJIT-benchmarks)
| |2.0.0-p0 |r62186 |JIT off |JIT on |
|:----------|:--------|:--------|:--------|:--------|
|aread |1.00 |1.09 |1.07 |2.19 |
|aref |1.00 |1.13 |1.11 |2.22 |
|aset |1.00 |1.50 |1.45 |2.64 |
|awrite |1.00 |1.17 |1.13 |2.20 |
|call |1.00 |1.29 |1.26 |2.02 |
|const2 |1.00 |1.10 |1.10 |2.19 |
|const |1.00 |1.11 |1.10 |2.19 |
|fannk |1.00 |1.04 |1.02 |1.00 |
|fib |1.00 |1.32 |1.31 |1.84 |
|ivread |1.00 |1.13 |1.12 |2.43 |
|ivwrite |1.00 |1.23 |1.21 |2.40 |
|mandelbrot |1.00 |1.13 |1.16 |1.28 |
|meteor |1.00 |2.97 |2.92 |3.17 |
|nbody |1.00 |1.17 |1.15 |1.49 |
|nest-ntimes|1.00 |1.22 |1.20 |1.39 |
|nest-while |1.00 |1.10 |1.10 |1.37 |
|norm |1.00 |1.18 |1.16 |1.24 |
|nsvb |1.00 |1.16 |1.16 |1.17 |
|red-black |1.00 |1.02 |0.99 |1.12 |
|sieve |1.00 |1.30 |1.28 |1.62 |
|trees |1.00 |1.14 |1.13 |1.19 |
|while |1.00 |1.12 |1.11 |2.41 |
** Discourse's script/bench.rb
Benchmark: https://github.com/discourse/discourse/blob/v1.8.7/script/bench.rb
NOTE: Rails performance was somehow a little degraded with JIT for now.
We should fix this.
(At least I know opt_aref is performing badly in JIT and I have an idea
to fix it. Please wait for the fix.)
*** JIT off
Your Results: (note for timings- percentile is first, duration is second in millisecs)
categories_admin:
50: 17
75: 18
90: 22
99: 29
home_admin:
50: 21
75: 21
90: 27
99: 40
topic_admin:
50: 17
75: 18
90: 22
99: 32
categories:
50: 35
75: 41
90: 43
99: 77
home:
50: 39
75: 46
90: 49
99: 95
topic:
50: 46
75: 52
90: 56
99: 101
*** JIT on
Your Results: (note for timings- percentile is first, duration is second in millisecs)
categories_admin:
50: 19
75: 21
90: 25
99: 33
home_admin:
50: 24
75: 26
90: 30
99: 35
topic_admin:
50: 19
75: 20
90: 25
99: 30
categories:
50: 40
75: 44
90: 48
99: 76
home:
50: 42
75: 48
90: 51
99: 89
topic:
50: 49
75: 55
90: 58
99: 99
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
compilation which is created by transforming a preprocessed vm.c.
This file will be used by JIT compiler's generated code which we are
going to have from succeeding commits.
Makefile.in: generate MJIT header for UNIX environments.
win32/Makefile.sub: generate MJIT header for mswin environments.
At initial merge, we're going to support only MinGW for Windows. So the
header installed by this file won't be used for short term, but we'll
add mswin support in a half year or so, for sure.
tool/transform_mjit_header.rb: New. This script was originally written as
minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for
Feature 12589.
Then I refactored a little so that it can conform CodeClimate CI which is
currently set for Ruby's GitHub repository, and fixed some bugs and ported
it to work on Windows.
Also, as original minimize_mjit_header.rb takes too long time to run,
this is modified to skip minimization step because having *static*
unused definitions does not waste compilation time on -O2 since compiler
can skip to compile unused static functions. So this does no longer
"minimize" the header and is renamed.
This header installation does NOT include a header to automatically
export symbols used by MJIT. That's because original MJIT code was
failing to export symbols in the import header in macOS environment.
But I would like to have the functionality for maintainability in the
future. I'll manually export things but it would be just an intemediate
solution.
Patch by: Vladimir N. Makarov <vmakarov@redhat.com>
Part of: Feature 12589 and 14235.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/ruby_vm/models/bare_instructions.rb (predefine_attributes):
`sp_inc` attribute which may return negative values must be
signed `rb_snum_t`, to be signed-expanded at type promotion.
* vm_insnhelper.h (ADJ_SP): removed the workaround for platforms
where rb_num_t is wider than int.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
k0kubun
nobu
kazu
shyouhei
naruse