It seems that `debug_counter_show_results_at_exit` could be called
earlier than `mjit_finish` sometimes.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67380 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and guard iseq from GC by marking iseq in mjit_copy_job.
This is a refactoring for implementing inlining later and
should not be fixing or introducing any bugs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67286 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* --jit-min-calls: 5 -> 10000
--jit-min-calls=5 obviously can compile non hotspot. This was not a
problem for MJIT-benchmarks and Optcarrot because the former has very
few hot optimiziable methods and the latter is likely to trigger
compilation of hotspot by its intensive calls to optimizable hotspot
methods and has a very short window to allow limited compilations.
In real-world applications, it has more time to compile more methods and
it pressures computer's limited resources like icache. We should avoid
compiling too many methods. Also compiling many methods exhausts time
budget for compilation in one ruby process lifetime and delays the "JIT
compaction" of Ruby 2.6.
JVM is known to use 1,500 for C1 (client) compiler and 10,000 for C2
(server) compiler for -XX:CompileThreshold by default.
https://docs.oracle.com/javase/8/embedded/develop-apps-platforms/codecache.htm
When things are called X,000 times, requiring 10,000 can eliminate
compilation of methods which are called only once in these X,000
iterations and obviously not hotspot. And in fact things like
unicorn-worker-killer restarts unicorn process every 4096 requests.
So I felt 10,000 is good for such an environment.
* --jit-max-cache: 1000 -> 100
By the same reason stated above, we should not allow compiling many
methods especially on MJIT which has a larger overhead due to poor code
locality by dlopen and whose code is also likely to be bigger by just
inlining many VM instructions with -O3 rather than directly generating
low-level code.
In JVM -XX:ReservedCodeCacheSize is 32M for reserved and 48M for maximum.
--jit-max-cache=1,000 could be closer to it, but in this case MJIT's
compilation is slow due to data synchronization between threads (to be
improved in Ruby 2.7 though) and we do not want to delay the "JIT
compaction" for a long time.
So I chose a really conservative number for this, but by having method
inlining in the future, wider range could be optimized even with this
value.
* Optcarrot
--disable-gems, --benchmark Lan_Master.nes 12 attempts.
No significant impact.
| r67276 | r67276 --jit | after --jit |
|:-------------------|:------------------|:------------------|
| 50.44369263063978 | 72.87390680773056 | 73.47873485047297 |
| 50.58788746124193 | 78.06820808947026 | 78.29723420171945 |
| 50.77509250801378 | 80.29010348842613 | 78.94689404460769 |
| 50.935361702064405 | 80.42796829926374 | 80.39539527351525 |
| 51.27352672981195 | 81.98758158033202 | 81.6754198664817 |
| 51.720715743242124 | 82.00118535811626 | 82.22960569251283 |
| 51.89643169822524 | 82.2290091613556 | 82.5013636146388 |
| 51.95895898113868 | 82.37318990939565 | 82.74002377794454 |
| 52.10124886807968 | 82.48796686037502 | 83.23354941183932 |
| 52.292280637519376 | 83.0265226541066 | 84.01552618012572 |
| 52.51856237784144 | 83.8797360318052 | 84.8588319093393 |
| 52.65076845986818 | 84.80037351256634 | 85.13577756273656 |
* Railsbench
`WARMUP=20000 BENCHMARK=1000 bin/bench` of https://github.com/k0kubun/railsbench.
It gets closer to --disable=jit.
| | r67276 | r67276 | after |
| | | --jit | --jit |
|:----------|:-------|:-------|:-------|
| req/s | 891.3 | 742.2 | 841.5 |
|:----------|:-------|:-------|:-------|
| 50%ile ms | 1.00 | 1.21 | 1.08 |
| 66%ile ms | 1.02 | 1.24 | 1.09 |
| 75%ile ms | 1.03 | 1.28 | 1.10 |
| 80%ile ms | 1.03 | 1.30 | 1.11 |
| 90%ile ms | 1.09 | 1.42 | 1.15 |
| 95%ile ms | 1.32 | 1.65 | 1.27 |
| 98%ile ms | 4.79 | 2.23 | 1.81 |
| 99%ile ms | 5.68 | 7.52 | 6.64 |
|100%ile ms | 6.52 | 9.69 | 8.59 |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67277 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and functions to clarify the intention and make sure it's not used in a
surprising way (like using 2, 3, ... other than 0, 1 even while it seems
to be a boolean).
This is a retry of r66775. It included some typos...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66778 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit bb1a1aeab0.
We hit something on ci.rvm.jp, reverting until investigation is done.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66776 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and functions to clarify the intention and make sure it's not used in a
surprising way (like using 2, 3, ... other than 0, 1 even while it seems
to be a boolean).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66775 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
get_uniq_filename never returns NULL but raises an exception on
failure.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66292 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Give the whole MJIT header path name by preloaded shared library
mjit_build_dir.so, than building the path from a given directory
name and the embedded base name.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66280 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It is not needed since we have LIST_HEAD_INIT initializer in
mjit_worker.c
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66203 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
in a new variable cc_common_args.
`cflags=-save-temps=obj` makes MJIT fail like:
https://rubyci.org/logs/www.rubyist.net/~akr/chkbuild/debian/ruby-trunk/log/20181203T095000Z.log.html.gz
This rubyci specifies -save-temps=obj in CFLAGS to use update-deps, and
the flag is harmful when we want to use -pipe flag.
mjit_worker.c: prefer cc_common_args over CC_COMMON_ARGS
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66164 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
because we mostly use cc_path now and the comment is obsolete (CC_PATH
is now absolute path and the TODO is already resolved).
mjit.c: use CC_COMMON_ARGS[0] directly
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66162 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
postponed_job is safe to use in signal handlers, but is not
thread-safe for MJIT. Implement a workqueue for MJIT
thread-safety.
[Bug #15316]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66100 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
-local variable to global variable.
Consider this case:
1. MJIT worker: dequeue ISeq (stop_worker_p was still FALSE)
2. Ruby thread: call Kernel#exec, which calls mjit_finish(FALSE),
sets `stop_worker_p = TRUE`, and fires RUBY_VM_CHECK_INTS() once
3. MJIT worker: register copy job, but found stop_worker_p is TRUE.
set `worker_stopped = TRUE` and the thread stops.
4. Function-local job variable expires by the thread stop (this is eliminated by this commit)
5. Ruby thread: find `worker_stopped` becamse TRUE, start Kernel#exec.
Kernel#exec fails but exception is rescued.
6. Ruby thread: call RUBY_VM_CHECK_INTS. copy job is dispatched but job variable
is already expired.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66035 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
because JIT-ed code may still be on stack at this time, unlike
in ruby_cleanup().
This hopes to fix: (take 2)
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480207
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480173
It tries to print C backtrace but fails. And core file on the server
seems to be stopping on the irrelevant place due to its own signal
handler for the dump.
And I failed to reproduce this SEGV on my machine.
I don't know why it's broken, so let me try this change to investigate
the reason of SEGV.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65997 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
it didn't work.
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468677
and skips broken tests for now. But this issue should be fixed soon.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65814 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to prevent from proceeding one for MJIT while it's not safe yet.
By that situation, MJIT worker could be waiting for compiler process forever
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468033
[Bug #15320]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65807 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
by launching MJIT worker thread in child Ruby process.
See the comment before `mjit_child_after_fork` for details.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65785 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Makefile.in (mjit_build_dir.so): separate MJIT_BUILD_DIR to
eliminate the feature for test-all after installation.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65587 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
-available region. reducing risk of SEGV in mjit_copy_job_handler() like
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1446117
I'm not sure which exact part is causing "[BUG] Segmentation fault at 0x0000000000000008"
on `(mjit_copy_job_handler+0x12) [0x564a6c4ce632] /home/ko1/ruby/src/trunk-mjit/mjit.c:26`...
mjit.c: ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65569 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Makefile.in (mjit_build_dir.so): separate MJIT_BUILD_DIR to
eliminate the feature for test-all after installation.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65563 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* mjit.c (init_header_filename): sticky-mode directory probably
would be less unsafe even if it is not owned.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_mjit_unit can either exist in unit_queue or active_units, but
not both. This will make state transitions for event-based MJIT
process management easier.
v2: recheck unit->iseq after GC wakeup
The iseq may be GC-ed while we were waiting for it since we
delete the unit from unit_queue during get_from_list
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65475 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
https://bugs.ruby-lang.org/issues/14867#note-112
I wanna touch similar places. To avoid our conflict, let me merge Eric's patch earlier.
Let's watch trunk-mjit / trunk-mjit-wait CIs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65473 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_mjit_unit can either exist in unit_queue or active_units, but
not both. This will make state transitions for event-based MJIT
process management easier.
[ruby-core:89654]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65468 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
For robustness against future changes. There should be no impact
at the moment,here, but we may call mjit_finish more than once
in a process lifetime in the future (implementing "stop"
instead of just "pause")
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65443 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Given that `copy_cache_from_main_thread()` breaks the loop when `stop_worker_p`
is TRUE, memory of `job` allocated by `alloca` may be invalid if `stop_worker_p`
is already TRUE.
mjit_worker.c: explain why `copy_cache_from_main_thread()` should not
stop checking `stop_worker_p`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65312 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
on VM_CHECK_INTS. Letting MJIT worker directly see inline cache which
may be being updated could result in inconsistent IC index and serial.
mjit_worker.c: request the copy job after dequeue, and receive the
result synchronously.
tool/ruby_vm/views/_mjit_compile_ivar.erb: use the copied IC
mjit_compile.c: change the interface to pass is_entries
mjit.h: ditto
=== Optcarrot Benchmark ===
Thankfully this didn't have major performance regression.
$ benchmark-driver benchmark.yml --rbenv 'before::before --disable-gems --jit;after::after --disable-gems --jit' -v --repeat-count 24
before: ruby 2.6.0dev (2018-10-21 trunk 65263) +JIT [x86_64-linux]
after: ruby 2.6.0dev (2018-10-21 trunk 65263) +JIT [x86_64-linux]
last_commit=mjit.c: copy inline cache values to MJIT worker
Calculating -------------------------------------
before after
Optcarrot Lan_Master.nes 85.421 85.454 fps
Comparison:
Optcarrot Lan_Master.nes
after: 85.5 fps
before: 85.4 fps - 1.00x slower
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65275 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* configure.ac: introduce new configure option `--enable-mjit` and
`--disable-mjit`. Default is "enable".
`--disable-mjit` disables all of MJIT features so that `ruby --jit`
can't enable MJIT.
This option affect a macro `USE_MJIT`.
This change remove `--enable/disable-install-mjit-header` option.
* Makefile.in: introduce the `ENABLE_MJIT` variable.
* common.mk: use `ENABLE_MJIT` option.
* internal.h: respect `USE_MJIT`. Same as other *.c, *.h.
* test/ruby/test_jit.rb: check `ENABLE_MJIT` key of rbconfg.rb.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65204 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
after the first compilation on --jit-wait.
Previously the assignment to `func` didn't have meaning for the behavior,
and the compiled code wasn't called immediately after the synchronous
compilation. It wasn't intentional.
Fixing this issue without impacting performance without --jit-wait is
not so obvious. Adding branch or goto to call func in mjit_exec spoiled
the performance without --jit-wait. Instead of that, I called the func
inside mjit_wait_call() (former mjit_get_iseq_func()) which is never
inlined to mjit_exec(). Thanks to that, this commit has no impact for
normal performance.
mjit.c: ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
k0kubun
kazu
nobu
normal
usa
ko1