I assumed that ISeq is never GC-ed by `in_jit` + `mjit_mark` on copy job
ISeq, but unfortunately I found SEGV on `mjit_copy_job_handler` in which
iseq->body was somehow Qnil. And it seems to be fixed by disabling the
job when `mjit_free_iseq` is called for the ISeq.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
as debugged in PR
[close https://github.com/ruby/ruby/pull/2130]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67549 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit a1b5d20068.
Revert "Revert "Skip recompiling tests on i686 Linux""
This reverts commit 7b88a9207b.
Revert "Simplify matrix for debugging"
This reverts commit e85d6c5c5e.
Sorry, these 3 commits were under debugging in
https://github.com/ruby/ruby/pull/2129 but accidentally merged by using
`git svn dcommit` instead of `git push` to the topic branch 🙇
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67538 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I'm writing `//` comments in newer MJIT code after C99 enablement
(because I write 1-line comments more often than multi-line comments
and `//` requires fewer chars on 1-line) and then they are mixed
with `/* */` now.
For consistency and to avoid the conversion in future changes, let me
finish the rewrite in MJIT-related code.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67533 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It seems that `debug_counter_show_results_at_exit` could be called
earlier than `mjit_finish` sometimes.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67380 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and guard iseq from GC by marking iseq in mjit_copy_job.
This is a refactoring for implementing inlining later and
should not be fixing or introducing any bugs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67286 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* --jit-min-calls: 5 -> 10000
--jit-min-calls=5 obviously can compile non hotspot. This was not a
problem for MJIT-benchmarks and Optcarrot because the former has very
few hot optimiziable methods and the latter is likely to trigger
compilation of hotspot by its intensive calls to optimizable hotspot
methods and has a very short window to allow limited compilations.
In real-world applications, it has more time to compile more methods and
it pressures computer's limited resources like icache. We should avoid
compiling too many methods. Also compiling many methods exhausts time
budget for compilation in one ruby process lifetime and delays the "JIT
compaction" of Ruby 2.6.
JVM is known to use 1,500 for C1 (client) compiler and 10,000 for C2
(server) compiler for -XX:CompileThreshold by default.
https://docs.oracle.com/javase/8/embedded/develop-apps-platforms/codecache.htm
When things are called X,000 times, requiring 10,000 can eliminate
compilation of methods which are called only once in these X,000
iterations and obviously not hotspot. And in fact things like
unicorn-worker-killer restarts unicorn process every 4096 requests.
So I felt 10,000 is good for such an environment.
* --jit-max-cache: 1000 -> 100
By the same reason stated above, we should not allow compiling many
methods especially on MJIT which has a larger overhead due to poor code
locality by dlopen and whose code is also likely to be bigger by just
inlining many VM instructions with -O3 rather than directly generating
low-level code.
In JVM -XX:ReservedCodeCacheSize is 32M for reserved and 48M for maximum.
--jit-max-cache=1,000 could be closer to it, but in this case MJIT's
compilation is slow due to data synchronization between threads (to be
improved in Ruby 2.7 though) and we do not want to delay the "JIT
compaction" for a long time.
So I chose a really conservative number for this, but by having method
inlining in the future, wider range could be optimized even with this
value.
* Optcarrot
--disable-gems, --benchmark Lan_Master.nes 12 attempts.
No significant impact.
| r67276 | r67276 --jit | after --jit |
|:-------------------|:------------------|:------------------|
| 50.44369263063978 | 72.87390680773056 | 73.47873485047297 |
| 50.58788746124193 | 78.06820808947026 | 78.29723420171945 |
| 50.77509250801378 | 80.29010348842613 | 78.94689404460769 |
| 50.935361702064405 | 80.42796829926374 | 80.39539527351525 |
| 51.27352672981195 | 81.98758158033202 | 81.6754198664817 |
| 51.720715743242124 | 82.00118535811626 | 82.22960569251283 |
| 51.89643169822524 | 82.2290091613556 | 82.5013636146388 |
| 51.95895898113868 | 82.37318990939565 | 82.74002377794454 |
| 52.10124886807968 | 82.48796686037502 | 83.23354941183932 |
| 52.292280637519376 | 83.0265226541066 | 84.01552618012572 |
| 52.51856237784144 | 83.8797360318052 | 84.8588319093393 |
| 52.65076845986818 | 84.80037351256634 | 85.13577756273656 |
* Railsbench
`WARMUP=20000 BENCHMARK=1000 bin/bench` of https://github.com/k0kubun/railsbench.
It gets closer to --disable=jit.
| | r67276 | r67276 | after |
| | | --jit | --jit |
|:----------|:-------|:-------|:-------|
| req/s | 891.3 | 742.2 | 841.5 |
|:----------|:-------|:-------|:-------|
| 50%ile ms | 1.00 | 1.21 | 1.08 |
| 66%ile ms | 1.02 | 1.24 | 1.09 |
| 75%ile ms | 1.03 | 1.28 | 1.10 |
| 80%ile ms | 1.03 | 1.30 | 1.11 |
| 90%ile ms | 1.09 | 1.42 | 1.15 |
| 95%ile ms | 1.32 | 1.65 | 1.27 |
| 98%ile ms | 4.79 | 2.23 | 1.81 |
| 99%ile ms | 5.68 | 7.52 | 6.64 |
|100%ile ms | 6.52 | 9.69 | 8.59 |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67277 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and functions to clarify the intention and make sure it's not used in a
surprising way (like using 2, 3, ... other than 0, 1 even while it seems
to be a boolean).
This is a retry of r66775. It included some typos...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66778 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit bb1a1aeab0.
We hit something on ci.rvm.jp, reverting until investigation is done.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66776 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and functions to clarify the intention and make sure it's not used in a
surprising way (like using 2, 3, ... other than 0, 1 even while it seems
to be a boolean).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66775 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
get_uniq_filename never returns NULL but raises an exception on
failure.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66292 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Give the whole MJIT header path name by preloaded shared library
mjit_build_dir.so, than building the path from a given directory
name and the embedded base name.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66280 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It is not needed since we have LIST_HEAD_INIT initializer in
mjit_worker.c
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66203 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
in a new variable cc_common_args.
`cflags=-save-temps=obj` makes MJIT fail like:
https://rubyci.org/logs/www.rubyist.net/~akr/chkbuild/debian/ruby-trunk/log/20181203T095000Z.log.html.gz
This rubyci specifies -save-temps=obj in CFLAGS to use update-deps, and
the flag is harmful when we want to use -pipe flag.
mjit_worker.c: prefer cc_common_args over CC_COMMON_ARGS
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66164 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
because we mostly use cc_path now and the comment is obsolete (CC_PATH
is now absolute path and the TODO is already resolved).
mjit.c: use CC_COMMON_ARGS[0] directly
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66162 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
postponed_job is safe to use in signal handlers, but is not
thread-safe for MJIT. Implement a workqueue for MJIT
thread-safety.
[Bug #15316]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66100 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
-local variable to global variable.
Consider this case:
1. MJIT worker: dequeue ISeq (stop_worker_p was still FALSE)
2. Ruby thread: call Kernel#exec, which calls mjit_finish(FALSE),
sets `stop_worker_p = TRUE`, and fires RUBY_VM_CHECK_INTS() once
3. MJIT worker: register copy job, but found stop_worker_p is TRUE.
set `worker_stopped = TRUE` and the thread stops.
4. Function-local job variable expires by the thread stop (this is eliminated by this commit)
5. Ruby thread: find `worker_stopped` becamse TRUE, start Kernel#exec.
Kernel#exec fails but exception is rescued.
6. Ruby thread: call RUBY_VM_CHECK_INTS. copy job is dispatched but job variable
is already expired.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66035 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
because JIT-ed code may still be on stack at this time, unlike
in ruby_cleanup().
This hopes to fix: (take 2)
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480207
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480173
It tries to print C backtrace but fails. And core file on the server
seems to be stopping on the irrelevant place due to its own signal
handler for the dump.
And I failed to reproduce this SEGV on my machine.
I don't know why it's broken, so let me try this change to investigate
the reason of SEGV.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65997 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
it didn't work.
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468677
and skips broken tests for now. But this issue should be fixed soon.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65814 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to prevent from proceeding one for MJIT while it's not safe yet.
By that situation, MJIT worker could be waiting for compiler process forever
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468033
[Bug #15320]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65807 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
by launching MJIT worker thread in child Ruby process.
See the comment before `mjit_child_after_fork` for details.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65785 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Makefile.in (mjit_build_dir.so): separate MJIT_BUILD_DIR to
eliminate the feature for test-all after installation.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65587 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
-available region. reducing risk of SEGV in mjit_copy_job_handler() like
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1446117
I'm not sure which exact part is causing "[BUG] Segmentation fault at 0x0000000000000008"
on `(mjit_copy_job_handler+0x12) [0x564a6c4ce632] /home/ko1/ruby/src/trunk-mjit/mjit.c:26`...
mjit.c: ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65569 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Makefile.in (mjit_build_dir.so): separate MJIT_BUILD_DIR to
eliminate the feature for test-all after installation.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65563 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* mjit.c (init_header_filename): sticky-mode directory probably
would be less unsafe even if it is not owned.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_mjit_unit can either exist in unit_queue or active_units, but
not both. This will make state transitions for event-based MJIT
process management easier.
v2: recheck unit->iseq after GC wakeup
The iseq may be GC-ed while we were waiting for it since we
delete the unit from unit_queue during get_from_list
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65475 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
https://bugs.ruby-lang.org/issues/14867#note-112
I wanna touch similar places. To avoid our conflict, let me merge Eric's patch earlier.
Let's watch trunk-mjit / trunk-mjit-wait CIs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65473 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_mjit_unit can either exist in unit_queue or active_units, but
not both. This will make state transitions for event-based MJIT
process management easier.
[ruby-core:89654]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65468 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
For robustness against future changes. There should be no impact
at the moment,here, but we may call mjit_finish more than once
in a process lifetime in the future (implementing "stop"
instead of just "pause")
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65443 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Given that `copy_cache_from_main_thread()` breaks the loop when `stop_worker_p`
is TRUE, memory of `job` allocated by `alloca` may be invalid if `stop_worker_p`
is already TRUE.
mjit_worker.c: explain why `copy_cache_from_main_thread()` should not
stop checking `stop_worker_p`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65312 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
k0kubun
kazu
nobu
normal
usa