Граф коммитов

290 Коммитов

Автор SHA1 Сообщение Дата
Takashi Kokubun 474f79958b
Cast jit_func for Windows
https://ci.appveyor.com/project/ruby/ruby/builds/39542385/job/8b7aq951f9t01x4x
2021-06-10 00:40:59 -07:00
Takashi Kokubun c5e8a49bde
Avoid enqueueing the same ISeq twice
by a race condition by multiple Ractors.

Atmically incrementing body->total_calls may have its own cost, so for
now we intentionally leave the unreliable total_calls. So we allow an
ISeq to be never pushed when you use multiple Ractors. However, if you
enqueue a single ccan node twice, get_from_list loops infinitely. Thus
this patch takes care of such a situation.
2021-06-10 00:32:24 -07:00
Takashi Kokubun 7e14762159
Do not doubly hold an MJIT lock
This is a follow-up of 86c262541a.
CRITICAL_SECTION_START/FINISH are not needed when it's called from an
MJIT worker.

Also, ZALLOC needs to be calloc because ZALLOC may trigger GC, which an
MJIT worker must not do.
2021-06-02 23:59:33 -07:00
Takashi Kokubun 86c262541a
Fix a race condition around mjit_recompile
This fixes SEGVs like https://github.com/ruby/ruby/runs/2715166621?check_suite_focus=true.

When mjit_recompile is called when mjit_compile is compiling the exact
same iseq (and after it called mjit_capture_cc_entries), iseq->body->jit_unit
is re-created and its cc_entries becomes NULL. Then, when it tries to
lookup cc_entries through iseq->body->jit_unit, it fails.
2021-06-02 22:11:37 -07:00
Takashi Kokubun 028f1887c2
Change the default --jit-max-cache to 10000
This is useful for large applications like Rails.
https://k0kubun.medium.com/ruby-3-jit-can-make-rails-faster-756310f235a
2021-05-31 22:01:04 -07:00
Takashi Kokubun 07c05b6fe9
Drop JIT_ISEQ_SIZE_THRESHOLD
Compiling everything seems to contributed to improving the final
performance in general. MJIT's compilation is slow anyway, especially
when you need to wait for JIT compaction.

This might make sense for short-time benchmarks like Optcarrot with
default parameters, but it didn't give benefits in my local environment.
2021-05-31 21:12:48 -07:00
Takashi Kokubun 1aac0e8819
Mark inlined ISeqs during MJIT compilation (#4539)
[Bug #17584]
2021-05-30 21:42:02 -07:00
Koichi Sasada e7fc353f04 enable constant cache on ractors
constant cache `IC` is accessed by non-atomic manner and there are
thread-safety issues, so Ruby 3.0 disables to use const cache on
non-main ractors.

This patch enables it by introducing `imemo_constcache` and allocates
it by every re-fill of const cache like `imemo_callcache`.
[Bug #17510]

Now `IC` only has one entry `IC::entry` and it points to
`iseq_inline_constant_cache_entry`, managed by T_IMEMO object.

`IC` is atomic data structure so `rb_mjit_before_vm_ic_update()` and
`rb_mjit_after_vm_ic_update()` is not needed.
2021-01-05 02:27:58 +09:00
Takashi Kokubun 095972e799
Skip mjit_wait if iseq is not a target 2021-01-04 00:24:02 -08:00
Takashi Kokubun 4724bf856f
Avoid hanging on --jit-wait after MJIT.pause
When a worker is stopped, nobody will JIT a method for you.
2021-01-03 00:05:38 -08:00
Takashi Kokubun ac2df89113
Stop managing valid class serials
`mjit_valid_class_serial_p` has no longer been used since b9007b6c54.
2020-12-29 23:01:11 -08:00
Takashi Kokubun f26f905b28
Mark an ISeq being JIT-ed
This is to avoid SEGV on a CC reference in a normal compilation
https://github.com/ruby/ruby/runs/1586578023
2020-12-20 23:17:37 -08:00
Takashi Kokubun 1fdc97f1b7
Mark active_units
to avoid SEGV on mjit_recompile and compact_all_jit_code.

For some reason, ISeqs on stack are sometimes GC-ed (why?) and therefore
it may run mjit_recompile on a GC-ed ISeq, which I expected d07183ec85
to fix but apparently it may refer to random things if already GC-ed.
Marking active_units would workaround the situation.
http://ci.rvm.jp/results/trunk-mjit-wait@phosphorus-docker/3292740

Also, while compact_all_jit_code was executed, we saw some SEGVs where
CCs seemed to be already GC-ed, meaning their owner ISeq was not marked
properly. Even if units are still in active_units, it's not guaranteed
that their ISeqs are in use. So in this case we need to mark active_units
for a legitimate reason.
http://ci.rvm.jp/results/trunk-mjit-wait@phosphorus-docker/3293277
http://ci.rvm.jp/results/trunk-mjit-wait@phosphorus-docker/3293090
2020-12-20 22:58:45 -08:00
Takashi Kokubun a574df14e4
Stop marking unit_queue
The original motivation of this marking was https://github.com/k0kubun/yarv-mjit/issues/20.
As wanabe said, there are multiple options to mitigate the issue, and
Eric Wong introduced another fix at 143776f6fe by checking unit->iseq
inside the lock.

Therefore this particular condition has been covered in two ways, and
the script given by wanabe no longer crashes without mjit_mark().
2020-12-20 22:03:20 -08:00
Takashi Kokubun d07183ec85
Do not access jit_unit if NULL 2020-12-16 19:43:43 -08:00
Takashi Kokubun 53babf35ef
Inline getconstant on JIT (#3906)
* Inline getconstant on JIT

* Support USE_MJIT=0
2020-12-16 06:24:07 -08:00
Takashi Kokubun 5d8f227d0e
Lazily move units from active_units to stale_units
to avoid SEGV like
http://ci.rvm.jp/results/trunk-mjit@phosphorus-docker/3289588
by a race condition between mjit_recompile and compation around active_units
2020-12-16 00:22:50 -08:00
Takashi Kokubun 541e4340dd
Assert unit->iseq null out happens under the JIT/GC guard 2020-12-03 00:01:11 -08:00
Takashi Kokubun 122cd35939
Throttle unload_units
Because d80226e7bd often reduces the number of unloaded units, it
increases the number of unload_units calls, which are heavy.

To mitigate that, this throttles unload_units per `max_cache_size / 10`.

Also hoping to fix
https://ci.appveyor.com/project/ruby/ruby/builds/36552382/job/kjmjgw9cjyf2ksd7
2020-11-27 23:06:40 -08:00
Takashi Kokubun 16dab6b692
Run unload_units in the JIT worker thread
to avoid "Too many JIT code, but skipped unloading units for JIT compaction".
Now we can forget the `in_compact` locking.

Moving some functions from mjit.c to mjit_worker.c because mjit_worker.c
should have functions executed in the JIT worker.
2020-11-27 21:38:51 -08:00
Takashi Kokubun 237cb94cf5
Handle calloc failure
for cfd8c7e6ca.
2020-11-24 00:02:52 -08:00
Takashi Kokubun cfd8c7e6ca
Prefer calloc/free over ZALLOC/xfree
To avoid SEGV like
http://ci.rvm.jp/logfiles/brlog.trunk-mjit.20201124-061530
2020-11-23 23:48:06 -08:00
Takashi Kokubun 2700df3c9d
ruby/internal/config.h needs to be included first
to define USE_MJIT.
2020-11-22 21:02:01 -08:00
Takashi Kokubun 0a2b987696
Make --disable-jit-support compile
vm_core.h needs to be included to know rb_execution_context_t, etc.
I also added a trivial refactoring in mjit.c and missing dependency for
process.c.
2020-11-22 20:55:33 -08:00
Takashi Kokubun 01f38693aa
Remove obsoleted internal/mjit.h inclusion
🙇
2020-11-22 20:28:34 -08:00
Takashi Kokubun fa1250a506
Stop leaving .c files for JIT compaction in /tmp (#3802)
* Re-generate C files for JIT compaction every time

* Refactor in_jit return logic

* Just write code in a single file

* Add a TODO comment [ci skip]
2020-11-22 07:10:44 -08:00
Takashi Kokubun e0156bd396
Make sure all threads are scanned on unload_units
This has been a TODO since 79df14c04b. While adcf0316d1 covered the
root_fiber of the initial thread, it didn't cover root_fibers of other
threads. Now it's hooked properly in rb_threadptr_root_fiber_setup.

With regards to "XXX: Is this mjit_cont `mjit_cont_free`d?", when
rb_threadptr_root_fiber_release is called, although I'm not sure when
th->root_fiber is truthy, fiber_free seems to call cont_free and
mjit_cont_free. So mjit_conts of root_fibers seem to be freed properly.
2020-11-21 19:36:55 -08:00
Takashi Kokubun 8750d001c2
Fix wrong #ifdef usages with #if
Apparently #ifdef is always true
2020-11-20 23:48:43 -08:00
Takashi Kokubun 27d5af59a3
Unify some confusing macro usages
_MSC_VER used to be the macro to switch JIT compaction. However, since
d4381d2ceb, the correct macro to switch it was changed from _MSC_VER
to _WIN32. As I didn't properly replace all relevant _MSC_VER usages
to _WIN32, these macros have been used inconsistently.

nobu replaced _WIN32 with USE_HEADER_TRANSFORMATION in 5eb446d12f.
Therefore we had USE_HEADER_TRANSFORMATION and _MSC_VER. This commit
makes sure such inconsistent _MSC_VER usages will be unified to the new
header, also renaming it to USE_JIT_COMPACTION to be more precise about
the requirements. The header transformation itself is not quite relevant
to places changed in this commit.
2020-11-20 23:39:30 -08:00
Takashi Kokubun 0960f56a1d
Eliminate IVC sync between JIT and Ruby threads (#3799)
Thanks to Ractor (https://github.com/ruby/ruby/pull/2888 and https://github.com/ruby/ruby/pull/3662),
inline caches support parallel access now.
2020-11-20 22:18:37 -08:00
Takashi Kokubun 40bad72f31
Assert in_gc >= 0 instead of guarding it (#3687) 2020-10-22 21:59:13 -07:00
Aaron Patterson abf678a439 Use a lock level for a less granular lock.
We are seeing an error where code that is generated with MJIT contains
references to objects that have been moved.  I believe this is due to a
race condition in the compaction function.

`gc_compact` has two steps:

1. Run a full GC to pin objects
2. Compact / update references

Step one is executed with `garbage_collect`.  `garbage_collect` calls
`gc_enter` / `gc_exit`, these functions acquire a JIT lock and release a
JIT lock.  So a lock is held for the duration of step 1.

Step two is executed by `gc_compact_after_gc`.  It also holds a JIT
lock.

I believe the problem is that the JIT is free to execute between step 1
and step 2.  It copies call cache values, but doesn't pin them when it
copies them.  So the compactor thinks it's OK to move the call cache
even though it is not safe.

We need to hold a lock for the duration of `garbage_collect` *and*
`gc_compact_after_gc`.  This patch introduces a lock level which
increments and decrements.  The compaction function can increment and
decrement the lock level and prevent MJIT from executing during both
steps.
2020-10-22 07:59:06 -07:00
Koichi Sasada 79df14c04b Introduce Ractor mechanism for parallel execution
This commit introduces Ractor mechanism to run Ruby program in
parallel. See doc/ractor.md for more details about Ractor.
See ticket [Feature #17100] to see the implementation details
and discussions.

[Feature #17100]

This commit does not complete the implementation. You can find
many bugs on using Ractor. Also the specification will be changed
so that this feature is experimental. You will see a warning when
you make the first Ractor with `Ractor.new`.

I hope this feature can help programmers from thread-safety issues.
2020-09-03 21:11:06 +09:00
卜部昌平 f4950145cf mjit_mark_cc_entries: should consider VM_CALLCACHE_UNMARKABLE
Now that vm_empty_cc is VM_CALLCACHE_UNMARKABLE, it has to be properly
ruled out from being GCed.
2020-06-09 12:40:50 +09:00
Takashi Kokubun 61b14bb32b
Eliminate a call instruction on JIT cancel path
by calling combined functions specialized for each cancel type.

I'm hoping to improve locality of hot code, but this patch's impact should
be insignificant.
2020-05-26 23:01:52 -07:00
卜部昌平 9e41a75255 sed -i 's|ruby/impl|ruby/internal|'
To fix build failures.
2020-05-11 09:24:08 +09:00
卜部昌平 d7f4d732c1 sed -i s|ruby/3|ruby/impl|g
This shall fix compile errors.
2020-05-11 09:24:08 +09:00
Takashi Kokubun 818d6d3336
Deduplicate functions in compacted JIT code
to improve code locality.

Using benchmark-driver/sinatra with 100 methods JIT-ed,

[Before] 12149.97 rps
1.3M    /tmp/_ruby_mjit_p31171u145.so

[After] 12818.83 rps
260K    /tmp/_ruby_mjit_p32155u145.so

(VM is 13714.89 rps)
2020-05-01 01:38:16 -07:00
Takashi Kokubun e8a78d7df8
Do not stop the world during JIT compaction
Running C compiler for JIT compaction inside a critical section may lock
main thread for a long time when it triggers GC. As I'm planning to
increase this duration a bit, I'd like to make sure this doesn't stop
the world.

For now, I chose to give up unloading units when it's during JIT
compaction, assuming other calls may unload them later.
2020-04-30 21:38:50 -07:00
Takashi Kokubun a3f6f67967
Add MJIT_COUNTER macro to dump total_calls 2020-04-12 23:10:28 -07:00
卜部昌平 9e6e39c351
Merge pull request #2991 from shyouhei/ruby.h
Split ruby.h
2020-04-08 13:28:13 +09:00
Takashi Kokubun f6a54e6e46
Add debug counter for unload_units
changing add_iseq_to_process's debug counter name as well for comparison
2020-03-15 00:24:18 -07:00
Takashi Kokubun 43e18c68f4
Mark all cc_entries associated to compiled_iseq 2020-03-12 22:23:51 -07:00
Takashi Kokubun 8562bfd150
Move code to mark jit_unit's cc_entries to mjit.c 2020-03-12 22:21:32 -07:00
Takashi Kokubun 4bcd5981e8
Capture inlined iseq's cc entries in root iseq's
jit_unit to avoid marking wrong cc entries when inlined iseq is compiled
multiple times, resolving the TODO added by daf7c48d88.

This obviates pseudo jit_unit in inlined iseq introduced by 7ec2359374
and fixes memory leak of the adhoc unit.
2020-03-10 00:53:35 -07:00
Takashi Kokubun adcf0316d1
Prevent unloading methods used in root_fiber while calling another Fiber (#2939)
Fixing SEGVs like:
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2744905
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2744420
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2741400
2020-02-28 23:58:33 -08:00
Takashi Kokubun a8dcab7233
Avoid infinite loop on --jit-wait 2020-02-28 23:20:44 -08:00
Takashi Kokubun 69f377a3d6
Internalize rb_mjit_unit definition again
Fixed a TODO in b9007b6c54
2020-02-26 00:27:29 -08:00
Takashi Kokubun eb75f0d134
Remove an unused declaration
It was unnecessary in b9007b6c54
2020-02-25 23:24:25 -08:00
Koichi Sasada 84d1a99a3f should be initialize jit_unit->cc_entries.
GC can invoke just after allocation of jit_unit->cc_entries so
it should be zero-cleared.
2020-02-25 13:37:52 +09:00