Граф коммитов

187 Коммитов

Автор SHA1 Сообщение Дата
k0kubun 33eebfd301 mjit_worker.c: undefine CC_PATH
because we mostly use cc_path now and the comment is obsolete (CC_PATH
is now absolute path and the TODO is already resolved).

mjit.c: use CC_COMMON_ARGS[0] directly

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66162 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 12:00:27 +00:00
normal eb38fb670b vm_trace.c: workqueue as thread-safe version of postponed_job
postponed_job is safe to use in signal handlers, but is not
thread-safe for MJIT.  Implement a workqueue for MJIT
thread-safety.

[Bug #15316]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66100 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-30 03:56:29 +00:00
k0kubun 86cb3d319f mjit_worker.c: promote mjit_copy_job from function
-local variable to global variable.

Consider this case:
1. MJIT worker: dequeue ISeq (stop_worker_p was still FALSE)
2. Ruby thread: call Kernel#exec, which calls mjit_finish(FALSE),
                sets `stop_worker_p = TRUE`, and fires RUBY_VM_CHECK_INTS() once
3. MJIT worker: register copy job, but found stop_worker_p is TRUE.
                set `worker_stopped = TRUE` and the thread stops.
4. Function-local job variable expires by the thread stop (this is eliminated by this commit)
5. Ruby thread: find `worker_stopped` becamse TRUE, start Kernel#exec.
                Kernel#exec fails but exception is rescued.
6. Ruby thread: call RUBY_VM_CHECK_INTS. copy job is dispatched but job variable
                is already expired.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66035 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-27 12:00:51 +00:00
k0kubun 67485fee42 vm_trace.c: MJIT-limited thread-safety for postponed_job
[Bug #15316]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66001 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 15:47:20 +00:00
k0kubun 98a2b053f8 process.c: avoid dlclose before exec
because JIT-ed code may still be on stack at this time, unlike
in ruby_cleanup().

This hopes to fix: (take 2)
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480207

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 15:12:31 +00:00
k0kubun 5b12501163 Revert "process.c: try to workaroun SEGV by r65994"
This reverts commit 0e6aba22c6.

because it didn't help, at all.
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480207

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65998 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 15:12:30 +00:00
k0kubun 0e6aba22c6 process.c: try to workaroun SEGV by r65994
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1480173

It tries to print C backtrace but fails. And core file on the server
seems to be stopping on the irrelevant place due to its own signal
handler for the dump.

And I failed to reproduce this SEGV on my machine.

I don't know why it's broken, so let me try this change to investigate
the reason of SEGV.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65997 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 14:45:39 +00:00
k0kubun 2751b5366d mjit.c: avoid running copy job handler after ISeq GC
like this http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1471633

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65928 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-22 13:29:44 +00:00
kazu 1cf0af636c mjit.c: fix typos [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65825 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-20 03:03:02 +00:00
k0kubun 27322735af revert r65807
it didn't work.
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468677

and skips broken tests for now. But this issue should be fixed soon.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65814 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-19 22:10:12 +00:00
k0kubun e205cd80d2 process.c: do not run signal handler before fork
to prevent from proceeding one for MJIT while it's not safe yet.
By that situation, MJIT worker could be waiting for compiler process forever
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1468033

[Bug #15320]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65807 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-19 13:33:07 +00:00
k0kubun 524e4cc982 mjit.c: fix deadlock introduced by r65785
in mjit_pause() fired by before_fork_ruby(). When calling
RUBY_VM_CHECK_INTS() in mjit_pause()'s loop, other threads may call
start_worker() which sets `stop_worker_p = FALSE` and it could wait for
MJIT worker stop forever even while `stop_worker_p` is FALSE.

http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1466333
https://gist.github.com/ko1/a57ef1a03e1c7cfd22f1495e0864b63d

http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1466335
https://gist.github.com/ko1/96e867e36e6b75559b3d926f8c0bdaeb

https://app.wercker.com/ruby/ruby/runs/mjit-test1/5bf1221c183106002855989c?step=5bf1225987436a0006192c31

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-18 13:32:49 +00:00
k0kubun fe6974a8fc mjit_worker.c: support MJIT in forked Ruby process
by launching MJIT worker thread in child Ruby process.

See the comment before `mjit_child_after_fork` for details.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65785 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-18 08:25:48 +00:00
nobu f837601cdb mjit_build_dir: separate MJIT_BUILD_DIR
* Makefile.in (mjit_build_dir.so): separate MJIT_BUILD_DIR to
  eliminate the feature for test-all after installation.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65587 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-07 02:49:26 +00:00
k0kubun dbd90b2dff mjit.c: don't use mutex before checking availability
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65572 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-06 08:01:28 +00:00
k0kubun 7a2263fb38 mjit_worker.c: strictly control MJIT copy job
-available region. reducing risk of SEGV in mjit_copy_job_handler() like
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1446117

I'm not sure which exact part is causing "[BUG] Segmentation fault at 0x0000000000000008"
on `(mjit_copy_job_handler+0x12) [0x564a6c4ce632] /home/ko1/ruby/src/trunk-mjit/mjit.c:26`...

mjit.c: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65569 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-06 07:22:25 +00:00
nobu b11b804f8f Revert "mjit_build_dir: separate MJIT_BUILD_DIR"
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65566 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-06 05:14:36 +00:00
nobu a39080f59c mjit_build_dir: separate MJIT_BUILD_DIR
* Makefile.in (mjit_build_dir.so): separate MJIT_BUILD_DIR to
  eliminate the feature for test-all after installation.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65563 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-06 04:13:48 +00:00
nobu d05e04b825 Relax MJIT_BUILD_DIR restriction
* mjit.c (init_header_filename): sticky-mode directory probably
  would be less unsafe even if it is not owned.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-05 17:27:10 +00:00
nobu 6d497629b2 More verbose message at unsafe header
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65541 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-05 02:05:12 +00:00
nobu ef94a94d7d Check MJIT_BUILD_DIR strictly
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65536 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-04 12:46:50 +00:00
normal 143776f6fe mjit: get rid of rb_mjit_unit_node and use ccan/list
rb_mjit_unit can either exist in unit_queue or active_units, but
not both.  This will make state transitions for event-based MJIT
process management easier.

v2: recheck unit->iseq after GC wakeup

The iseq may be GC-ed while we were waiting for it since we
delete the unit from unit_queue during get_from_list

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65475 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-31 23:55:22 +00:00
k0kubun 883f93a12e Revert "revert r65471 and include Eric's patch as well"
This reverts commit ff5dc2cbbf.

Deadlock: http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1438883

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65474 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-31 13:12:39 +00:00
k0kubun ff5dc2cbbf revert r65471 and include Eric's patch as well
https://bugs.ruby-lang.org/issues/14867#note-112

I wanna touch similar places. To avoid our conflict, let me merge Eric's patch earlier.
Let's watch trunk-mjit / trunk-mjit-wait CIs.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65473 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-31 12:21:55 +00:00
normal dc8a744d6e Revert "mjit: get rid of rb_mjit_unit_node and use ccan/list"
This reverts commit c5177fa846. r65468

Many CI failures like:
 http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/1438415

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65471 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-31 06:44:42 +00:00
normal c5177fa846 mjit: get rid of rb_mjit_unit_node and use ccan/list
rb_mjit_unit can either exist in unit_queue or active_units, but
not both.  This will make state transitions for event-based MJIT
process management easier.

[ruby-core:89654]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65468 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-31 03:02:01 +00:00
normal b51724aac5 mjit.c (free_list): clear .length
For robustness against future changes. There should be no impact
at the moment,here, but we may call mjit_finish more than once
in a process lifetime in the future (implementing "stop"
instead of just "pause")

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65443 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 18:26:12 +00:00
k0kubun 5984aa84db mjit.c: prevent from accessing expired job
Given that `copy_cache_from_main_thread()` breaks the loop when `stop_worker_p`
is TRUE, memory of `job` allocated by `alloca` may be invalid if `stop_worker_p`
is already TRUE.

mjit_worker.c: explain why `copy_cache_from_main_thread()` should not
stop checking `stop_worker_p`.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65312 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-23 00:09:10 +00:00
k0kubun 13df05acfa mjit.c: copy call cache values to MJIT worker
same as r65275 but for call cache.

=== Optcarrot Benchmark ===
$ benchmark-driver benchmark.yml --rbenv 'before::before --disable-gems --jit;after::after --disable-gems --jit' -v --repeat-count 24
before: ruby 2.6.0dev (2018-10-21 trunk 65277) +JIT [x86_64-linux]
after: ruby 2.6.0dev (2018-10-21 trunk 65277) +JIT [x86_64-linux]
last_commit=mjit.c: copy call cache values to MJIT worker
Calculating -------------------------------------
                             before       after
Optcarrot Lan_Master.nes     85.372      85.359 fps

Comparison:
             Optcarrot Lan_Master.nes
                  before:        85.4 fps
                   after:        85.4 fps - 1.00x  slower

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65279 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-21 14:23:24 +00:00
k0kubun 1b5aa4f516 mjit.c: copy inline cache values to MJIT worker
on VM_CHECK_INTS. Letting MJIT worker directly see inline cache which
may be being updated could result in inconsistent IC index and serial.

mjit_worker.c: request the copy job after dequeue, and receive the
result synchronously.

tool/ruby_vm/views/_mjit_compile_ivar.erb: use the copied IC

mjit_compile.c: change the interface to pass is_entries
mjit.h: ditto

=== Optcarrot Benchmark ===
Thankfully this didn't have major performance regression.

$ benchmark-driver benchmark.yml --rbenv 'before::before --disable-gems --jit;after::after --disable-gems --jit' -v --repeat-count 24
before: ruby 2.6.0dev (2018-10-21 trunk 65263) +JIT [x86_64-linux]
after: ruby 2.6.0dev (2018-10-21 trunk 65263) +JIT [x86_64-linux]
last_commit=mjit.c: copy inline cache values to MJIT worker
Calculating -------------------------------------
                             before       after
Optcarrot Lan_Master.nes     85.421      85.454 fps

Comparison:
             Optcarrot Lan_Master.nes
                   after:        85.5 fps
                  before:        85.4 fps - 1.00x  slower

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65275 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-21 13:21:28 +00:00
ko1 b710785f1a add disabling MJIT features option.
* configure.ac: introduce new configure option `--enable-mjit` and
  `--disable-mjit`. Default is "enable".
  `--disable-mjit` disables all of MJIT features so that `ruby --jit`
  can't enable MJIT.
  This option affect a macro `USE_MJIT`.
  This change remove `--enable/disable-install-mjit-header` option.

* Makefile.in: introduce the `ENABLE_MJIT` variable.

* common.mk: use `ENABLE_MJIT` option.

* internal.h: respect `USE_MJIT`. Same as other *.c, *.h.

* test/ruby/test_jit.rb: check `ENABLE_MJIT` key of rbconfg.rb.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65204 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-20 06:53:00 +00:00
k0kubun 320d7c54ba mjit.c: don't let MJIT.pause hang on full active units
test/ruby/test_rubyvm_mjit.rb: add reproductive test

[Bug #15071]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65179 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-19 15:08:55 +00:00
k0kubun f13a2ad6df mjit.c: fix obsoleted comment [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64931 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-07 04:29:00 +00:00
k0kubun e889fad86c mjit.h: call compiled code immediately
after the first compilation on --jit-wait.

Previously the assignment to `func` didn't have meaning for the behavior,
and the compiled code wasn't called immediately after the synchronous
compilation. It wasn't intentional.

Fixing this issue without impacting performance without --jit-wait is
not so obvious. Adding branch or goto to call func in mjit_exec spoiled
the performance without --jit-wait. Instead of that, I called the func
inside mjit_wait_call() (former mjit_get_iseq_func()) which is never
inlined to mjit_exec(). Thanks to that, this commit has no impact for
normal performance.

mjit.c: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-07 04:17:59 +00:00
k0kubun c55e10a9ea mjit.c: reduce the number of variables
in mark_ec_units() to simplify code.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64321 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-12 12:59:47 +00:00
k0kubun 33d318a29e mjit.c: stop defining alias for a very limited use
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64307 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 14:48:10 +00:00
k0kubun 1bc2641b00 mjit.c: drop obsoleted duplicated declaration
of mjit_worker(). It was needed when mjit_worker.c is separated from
mjit.c, but it's now just included.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64306 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 14:44:31 +00:00
k0kubun 5cd84d247f mjit_worker.c: handle calloc failure
Unlike ZALLOC, it's not automatically handled.

mjit.c: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64304 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 14:34:05 +00:00
k0kubun d9260c5a50 mjit_worker.c: share MJIT warning logic
as mjit_warning().

mjit.c: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64298 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 14:05:34 +00:00
k0kubun 3d2e7b6158 mjit_worker.c: resurrect more static declarations
and remove old mjit_ prefixes again.

mjit.c: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64292 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 10:05:56 +00:00
k0kubun 3c442229d6 mjit.c: make some variables static again
and remove redundant mjit_ prefixes.

mjit_worker.c: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 09:48:07 +00:00
k0kubun a48d1d7991 mjit.c: include mjit_worker.c
instead of linking functions with mjit_worker.o.

In the r64285's structure, we needed to publish some variables with
mjit_ prefix. But ideally those variables should be completely private
in mjit.o (or old mjit_worker.o), and it was hard.

So I chose an approach similar to vm*.c for mjit.c and mjit_worker.c.
I believe mjit_compile.c is still nice to be separated.

After this commit, I'll remove the mjit_ prefix again...

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64289 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 09:37:14 +00:00
k0kubun 6306aa9222 mjit.c: exclude mjit_valid_class_serial_p
from mjit.c because it's executed only on MJIT worker thread.

Instead of that, `valid_class_serials` is shared with mjit_ prefix.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64288 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 08:34:25 +00:00
k0kubun b2e0d54024 mjit_worker.c: prefix mjit_ to pch_status
which was just forgotten.

mjit.c: ditto

mjit_internal.h: moved some macros only used by mjit_worker.c to it.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64287 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 08:07:13 +00:00
k0kubun ddcb40f595 mjit_worker.c: carve out worker-related code
The motivation of this change is to make sure rb_funcall or GC-related
functions are not called on worker-related code. Currently such
functions are used in some places and I believe it's partly because it's
hard to identify which part is called on MJIT worker thread.

Now, mjit.c is safe to use them but we know we need to safely deal with
mjit_compile.c, mjit_worker.c and mjit_internal.h.

mjit_compile.c: update the comment about it

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64285 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-11 07:57:58 +00:00
nobu 6c70fede0c version.c: separate Init_ruby_description
* version.c (Init_ruby_description): separate to initialize
  RUBY_DESCRIPTION constant according to mjit.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64264 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-10 05:02:35 +00:00
nobu 3ef25ed755 mjit.c: ruby_version is no longer used since r63279
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64261 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-10 04:33:11 +00:00
k0kubun 2e003f6c87 process.c: don't wait JIT queue flush on rb_f_exec
This wasn't intended in r64253.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64254 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-09 11:39:07 +00:00
k0kubun 212a77ed46 process.c: fix outdated mjit_pause declaration
by sharing it with vm.c in internal.h.

vm.c: ditto
internal.h: ditto
mjit.h: share more.

mjit.c: make sure the third arguemnt is not used

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64253 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-09 11:31:41 +00:00
k0kubun 510cd06c78 mjit.c: add :wait option to RubyVM::MJIT.pause
and wait until JIT queue is flushed when wait option is not passed or
`wait: true` is passed.

vm.c: ditto

test/ruby/test_rubyvm_mjit.rb: added test for pause/resume

test/lib/jit_support.rb: allow retrying MJIT on JITSupport level
test/ruby/test_jit.rb: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64250 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-09 09:58:07 +00:00