symbol in return value of methods.
* test/win32ole/test_win32ole.rb ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64856 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
because r64849 seems to fix issues which we were confused about.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64850 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
as a workaround to fix the build pipeline broken by r64824,
because optimizing Ruby should be prioritized higher than supporting unused jokes.
In the current build system, exceeding 200 insns somehow crashes C
extension build on some of MinGW environments like "mingw32-make[1]:
*** No rule to make target 'note'. Stop."
https://ci.appveyor.com/project/ruby/ruby/build/9725/job/co4nu9jugm8qwdrp
and on some of Linux environments like "cannot load such file -- stringio (LoadError)"
```
build_install /home/ko1/ruby/src/trunk_gcc5/lib/rubygems/specification.rb:18:in `require': cannot load such file -- stringio (LoadError)
from /home/ko1/ruby/src/trunk_gcc5/lib/rubygems/specification.rb:18:in `<top (required)>'
from /home/ko1/ruby/src/trunk_gcc5/lib/rubygems.rb:1365:in `require'
from /home/ko1/ruby/src/trunk_gcc5/lib/rubygems.rb:1365:in `<module:Gem>'
from /home/ko1/ruby/src/trunk_gcc5/lib/rubygems.rb:116:in `<top (required)>'
from /home/ko1/ruby/src/trunk_gcc5/tool/rbinstall.rb:24:in `require'
from /home/ko1/ruby/src/trunk_gcc5/tool/rbinstall.rb:24:in `<main>'
make: *** [do-install-nodoc] Error 1
```
http://ci.rvm.jp/results/trunk_gcc5@silicon-docker/1353447
This commit removes "bitblt" and "trace_bitblt" insns, which reduces the
number of insns from 202 to 200 and fixes at least the latter build
failure. I hope this fixes the MinGW build failure as well. Let me
confirm the situation on AppVeyor CI.
Note that this is hard to fix because some MinGW environments (MSP-Greg's
MinGW CI on AppVeyor) don't reproduce this and some Linux environments
(including my local machine) don't reproduce it either. Make sure you
have the reproductive environment and confirm it's fixed when reverting
this commit.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64839 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit r64829. I'll prepare another temporary fix, but I'll
separately commit that to make it easier to revert that later.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64838 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/lib/test/unit.rb (Test::Unit::SubprocessOption#non_options):
set timeout scale after parsing options. the option value will be
set after returning from setup_options.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64827 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
not optimizing Array#& and Array#| because vm_insnhelper.c can't easily
inline it (large amount of array.c code would be needed in vm_insnhelper.c)
and the method body is a little complicated compared to Integer's ones.
So I thought only Integer#& and Integer#| have a significant impact,
and eliminating unnecessary branches would contribute to JIT's performance.
vm_insnhelper.c: ditto
tool/transform_mjit_header.rb: make sure these instructions are inlined
on JIT.
compile.c: compile vm_opt_and and vm_opt_or.
id.def: define id for them to be used in compile.c and vm*.c
vm.c: track redefinition of Integer#& and Integer#|
vm_core.h: allow detecting redefinition of & and |
test/ruby/test_jit.rb: test new insns
test/ruby/test_optimization.rb: ditto
* Optcarrot benchmark
This is a kind of experimental thing but I'm committing this since the
performance impact is significant especially on Optcarrot with JIT.
$ benchmark-driver benchmark.yml --rbenv 'before::before --disable-gems;before+JIT::before --disable-gems --jit;after::after --disable-gems;after+JIT::after --disable-gems --jit' -v --repeat-count 24
before: ruby 2.6.0dev (2018-09-24 trunk 64821) [x86_64-linux]
before+JIT: ruby 2.6.0dev (2018-09-24 trunk 64821) +JIT [x86_64-linux]
after: ruby 2.6.0dev (2018-09-24 opt_and 64821) [x86_64-linux]
last_commit=opt_or
after+JIT: ruby 2.6.0dev (2018-09-24 opt_and 64821) +JIT [x86_64-linux]
last_commit=opt_or
Calculating -------------------------------------
before before+JIT after after+JIT
Optcarrot Lan_Master.nes 51.460 66.315 53.023 71.173 fps
Comparison:
Optcarrot Lan_Master.nes
after+JIT: 71.2 fps
before+JIT: 66.3 fps - 1.07x slower
after: 53.0 fps - 1.34x slower
before: 51.5 fps - 1.38x slower
[close https://github.com/ruby/ruby/pull/1963]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64824 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/test_win32api.rb: suppress a warning, which we know already
well.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64816 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* vm_backtrace.c (rb_debug_inspector_open): escape all env using
`rb_vm_stack_to_heap()` before making bindings.
[Bug #15105]
There is a complicated story of this issue:
Without this patch, IFUNC frame does not escaped. A IFUNC frame
points to CFUNC ep as previous ep. However, CFUNC ep can be escaped
because of making bindings of Ruby level frames.
IFUNC's ep can points to invalidated ep and `rb_iter_break()` will
fail. This is why `any?` fails.
* test/-ext-/debug/test_debug.rb: add a test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64800 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I introduce a `union` method equivalent to the `|` operator, but which
accept more than array as argument. This improved readability, and it
is also coherent with the `+` operator, which has a similar `concat`
method. The method doesn't modify the original object and return a new
object instead. It is plan to introduce a `union!` method as well.
Tests and documentation are included.
It solves partially https://bugs.ruby-lang.org/issues/14097
[Fix GH-1747] [Feature #14097]
From: Ana María Martínez Gómez <ammartinez@suse.de>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64787 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (arg_blk_pass): preceeding arguments node may be NULL when
an empty keyword argument hash splat is optimized away.
[ruby-core:88890] [Bug #15087]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64786 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Hash#merge, merge!, and update could merge exactly two hashes.
Now, they accepts zero or more hashes as arguments so that it can merge
hashes more than two.
This patch was created by Koki Ryu <liukoki@gmail.com> at Ruby Hack
Challenge #5. Thank you!
[ruby-core:88970] [Feature #15111] [Fix GH-1951]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64777 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Could happen for some linearly dependent vectors.
Patch by Vasiliy Petrov. [Fix GH-1803]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64761 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c (rb_enc_get_index): external encoding may not be Data
object. [ruby-core:89016] [Bug #15122]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64753 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Remove the remainder of ROOT_FIBER_CONTEXT use and unnecessary
differences between the root and non-root fiber. This makes
it easier to follow new root fiber at fork time.
Multiple sources of truth often leads to bugs, as in this case.
We can determinte root fiber by checking a fiber against the root_fiber
of its owner thread. The new `fiber_is_root_p' function
supports that.
Now, we can care only about free-ing/recycling/munmap-ing stacks
as appropriate.
[Bug #15050]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64706 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Make sure Enumerator::ArithmeticSequence#each to work well
for a complex step value.
This reverts commit ca47fb329a.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64695 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Make sure Enumerator::ArithmeticSequence#each to work well
for a complex step value.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64692 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* range.c (range_cover): add code for range argument.
If the argument is a Range, check it is or is not
covered by the reciver. If it can be treated as a
sequence, this method treats it that way.
* test/ruby/test_range.rb (class TestRange): add tests
for this feature.
This patch is written by Owen Stephens. thank you!
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64640 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Interrupt timing is tricky and it's possible the target
thread is still stopped from the previous loop iteration.
[ruby-core:88732] [Bug #15043]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64590 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Otherwise, th->root_fiber can point to an invalid Fiber,
because Fibers do not live across fork. So consider
whatever Fiber is running the root fiber.
[ruby-core:88723] [Bug #15041]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64589 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Arch Linux CI still seems to timeout on this test...
Note, I can't reproduce the failures in these tests on a
FreeBSD 11.1 VM while infinite-looping, even without the
"th.join(0.001)". It doesn't seem related to the use of
rb_wait_for_single_fd (r64529).
cf. https://rubyci.org/logs/rubyci.s3.amazonaws.com/arch/ruby-trunk/log/20180826T090003Z.fail.html.gz
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64543 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We need to ensure Signal.trap handlers can function if the main
thread is sleeping after a subthread has grabbed sigwait_fd,
but later exited.
Consider the following timeline:
main_thread sub-thread
-----------------------------------------
Signal.trap() { ... }
get sigwait_fd
ppoll on sigwait_fd
native_cond_sleep
(via pthread_cond_wait)
ppoll times-out
put sigwait_fd
sub-thread exits
only thread alive
SIGNAL HITS
The problem is pthread_cond_wait cannot return EINTR,
so we can never run the Signal.trap handler. So we
will avoid using native_cond_sleep in the main thread
and always use ppoll to sleep when in the main thread.
This can guarantee the main thread remains aware of
signals; even if it cannot safely read off sigwait_fd
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64538 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
IO#sysread calls rb_wait_for_single_fd for compatibility, and
perhaps something is amiss with that (unrelated to timer-thread
elimination)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64529 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* iseq.c (iseq_init_trace): at ISeq loading time, we need to check
`ruby_vm_event_enabled_flags` to turn on trace instructions.
Seprate this checking code from `finish_iseq_build()` and make
new function. `iseq_ibf_load()` calls this funcation after loading.
* test/ruby/test_iseq.rb: add a test for this fix.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64514 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Line coverage was based on special instruction "tracecoverage".
Now, instead, it uses the mechanism of trace hook [Feature #14104].
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64509 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The code fragments that initializes coverage data were scattered into
both parse.y and compile.c. parse.y allocated a coverage data, and
compile.c initialize the data.
To remove this cross-cutting concern, this change moves the allocation
from "coverage" function of parse.y to "rb_iseq_new_top" of iseq.c.
For the sake, parse.y just counts the line number of the original source
code, and the number is passed via rb_ast_body_t.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64508 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
CI failures are still happening from these tests, but try
to break out of it earlier instead of holding up the job.
[Bug #14898]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64484 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
According to some runs in mjit-test (make test-all RUN_OPTS="--jit-wait"),
this test might not be the cause of its failure.
So, let me try running this again.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64426 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Arbitrarily closing file descriptors on exec breaks use cases
where a Ruby process sets up a descriptor for non-Ruby children
to use. For example, the "rake foo" target may spawn any number
of subprocesses (Ruby or not) which depends on parsing the "FOO"
environment variable for out_fd:99 and writing to foo.out
FOO=out_fd:99 rake foo 99>>foo.out
Unfortunately, this introduced one incompatibility in
test/lib/test/unit.rb and it now requires explicitly setting
IO#close_on_exec=true
[ruby-core:88007] [Misc #14907]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64399 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It is possible for Mutex#sleep (via ConditionVariable#wait)
to prematurely wake up under MJIT (because Mutex#sleep can't
handle spurious wakeups). This affects @idle_done_cond in
Net::IMAP#idle and means the response handler may never
set `in_idle' to `true`.
In any case, ensure the infinite looping `raiser' thread
stops running when the test is done.
Will work on reducing the effect of spurious wakeups from
MJIT...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64387 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I still seem to need more memory for parallel tests with MJIT...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64381 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
that times out with test-all w/ --jit-wait.
I'm running the following command on Wecker CI everyday:
```
make test-all TESTOPTS="--color=never --job-status=normal" RUN_OPTS="--disable-gems --jit-wait --jit-warnings" RUBY_FORCE_TEST_JIT=1
```
By running yesterday's all commits, r64354 ran successfully but r64355
didn't. So the test should be fixed to run with --jit-wait at first.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64374 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Let me skip this to make CI green first and take a look later...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64365 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit 52102f6ff5 (r64238).
It is no longer necessary if we use Tempfile.create in
WEBrick::HTTPAuth::Htgroup#flush (see next commit)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64362 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
EINTR seems unavoidable in real programs (or MJIT), so maybe
it's not worth dealing with. r64353 relies on POSIX timers
to signal.
Switching pipes and sockets to non-blocking by default would let
us get rid of POSIX timers, timer pthread and this hack:
https://bugs.ruby-lang.org/issues/14968
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64355 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit 194a6a2c68 (r64203).
Race conditions which caused the original reversion will be fixed
in the subsequent commit.
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64352 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Re-revert r64340, and take care about notimplemented case.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64343 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Module has many introspection methods for methods and constants that
either return an array or return true or false for whether the method
or constant is defined. Most of these methods support an optional
argument that controls whether to consider inheritance. Currently,
the following Module methods support such a argument:
* const_defined?
* constants
* instance_methods
* private_instance_methods
* protected_instance_methods
* public_instance_methods
and the following methods do not:
* method_defined?
* private_method_defined?
* protected_method_defined?
* public_method_defined?
This patch supports such an argument for the *method_defined?
methods.
While you can currently work around the lack of support via:
mod.instance_methods(false).include?(:method_name)
This patch allows the simpler and more efficient:
mod.method_defined?(:method_name, false)
One case where you want to exclude inheritance when checking
for a method definition is when you want to replace a method
that may already exist. To avoid a verbose warning, you want
to remove the method only if it is already defined:
remove_method(:foo) if method_defined?(:foo, false)
define_method(:foo){}
You can't call remove_method without checking for the method
definition, as that can raise a NameError, and you don't want
to include inheritance because remove_method will still raise
a NameError if the method is defined by an ancestor and not
by the module itself.
[ruby-core:88140] [Feature #14944]
From: Jeremy Evans <code@jeremyevans.net>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64337 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/ruby/test_env.rb (test_huge_value): Windows 8 seems having a
limit on single environment variable.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64303 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
failure. It's failing on mswinci but it's hard to know the cause without
out/err.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64299 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We should increase RUBY_TEST_SUBPROCESS_TIMEOUT_SCALE for slow
environments, and we configured some CI machines.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64295 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (ruby_setenv): do not check environment block size.
c.f. https://msdn.microsoft.com/en-us/library/windows/desktop/ms682653(v=vs.85).aspx
Starting with Windows Vista and Windows Server 2008, there is no
technical limitation on the size of the environment block.
[ruby-core:88400] [Bug #14979]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64293 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to fix the wrong line number on #caller_locations or rb_profile_frames.
Actually we would be able to move it only when method call (of
caller_locations) or C extension invocation (calling rb_profile_frames)
can happen.
This degrades performance. Optcarrot fps becomes...
before: 71.78976052783555
after: 67.65429356624131
I think I can lazily move it and fix the performance issue, even
improving the performance for the situation catch table exists.
But let me fix this bug first...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64283 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It's timed out on CI like this
https://gist.github.com/ko1/33df53d78fbec9e4156c017eeae023bb.
I'm not sure how long it would take (it finishes immediately on my machine),
so I chose 600s because r64271 says it took 635s on mswinci. (probably
it's improved by the commit, though.)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64278 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This fixes a bug of Enumerator::ArithmeticSequence#last in the case that
a non-integral argument is passed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64262 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Undefine new and allocate methods of Enumerator::ArithmeticSequence.
[ruby-core:82816] [Feature #13904]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64256 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Due to a bug in OpenSSL 1.1.0h[1] (it's only in this specific version;
it was introduced just before the release and is already fixed in their
stable branch), the callback set by SSLContext#session_new_cb= does not
get called for clients, making net/http and net/ftp not attempt session
resumption.
Let's disable the affected test cases for now. Another option would be
to fallback to using SSLSocket#session as we did before r64234. But
since only a single version is affected and hopefully a new stable
version containing the fix will be released in near future, I chose not
to add such workaround code to lib/.
[1] https://github.com/openssl/openssl/pull/5967
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64252 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and wait until JIT queue is flushed when wait option is not passed or
`wait: true` is passed.
vm.c: ditto
test/ruby/test_rubyvm_mjit.rb: added test for pause/resume
test/lib/jit_support.rb: allow retrying MJIT on JITSupport level
test/ruby/test_jit.rb: ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64250 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Running with TESTS=--no-retry, I sometimes get short reads
leading to failures due to the use of IO#readpartial instead of
IO#gets.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64248 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Apparently my tty can't keep up when I run test-all with
TESTS='-v --no-retry'
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64247 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (argf_next_argv): convert filename to the OS encoding to be
dealt with by system calls. [ruby-dev:50607] [Bug #14970]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64243 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Quiet a false-positive warning from TestHtgroup#test_htgroup
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64238 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When TLS 1.3 is in use, the session ticket may not have been sent yet
even though a handshake has finished. Also, the ticket could change if
multiple session ticket messages are sent by the server. Use
SSLContext#session_new_cb instead of calling SSLSocket#session
immediately after a handshake. This way also works with earlier protocol
versions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64234 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Sync with the current tip of master branch, 62436385306c of
ruby/openssl.git. Changes can be found at:
https://github.com/ruby/openssl/compare/v2.1.1...62436385306c
----------------------------------------------------------------
Brian Cunnie (1):
Correctly verify abbreviated IPv6 SANs
Janko Marohnić (1):
Reduce memory allocation when writing to SSLSocket
Jeremy Evans (1):
Move rb_global_variable call to directly after assignment
Kazuki Yamaguchi (7):
pkcs7: allow recipient's certificate to be omitted for PKCS7#decrypt
pkey: resume key generation after interrupt
tool/ruby-openssl-docker: update to latest versions
test/test_ssl: fix test failure with TLS 1.3
test/test_x509name: change script encoding to ASCII-8BIT
x509name: refactor OpenSSL::X509::Name#to_s
x509name: fix handling of X509_NAME_{oneline,print_ex}() return value
ahadc (1):
Update CONTRIBUTING.md
nobu (6):
no ID cache in Init functions
search winsock libraries explicitly
openssl: search winsock
openssl_missing.h: constified
reduce LibreSSL warnings
openssl/buffering.rb: no RS when output
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64233 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
By this commit's changes in other files, now MJIT started to work on VC++.
Unfortunately some features are still broken and they'll be fixed later.
This also suppresses cl.exe's default output to stdout because there
seems to be no option to do it. Tweaking some log messages as well.
vm_core.h: declare `__declspec(dllimport)` to export them correctly on mswin.
vm_insnhelper.h: ditto
mjit.h: ditto
test_jit.rb: skipped some pending tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64221 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit introduces new core class Enumerator::ArithmeticSequence.
Enumerator::ArithmeticSequence is a subclass of Enumerator, and
represents a number generator of an arithmetic sequence.
After this commit, Numeric#step and Range#step without blocks
returned an ArithmeticSequence object instead of an Enumerator.
This class introduces the following incompatibilities:
- You can create a zero-step ArithmeticSequence,
and its size is not ArgumentError, but Infinity.
- You can create a negative-step ArithmeticSequence from a range.
[ruby-core:82816] [Feature #13904]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64205 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Unfortunately, MJIT goes over the 100M RLIMIT_AS for this test
on x86-64 (Debian 9)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64204 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Trying to diagnose CI failures from i686-linux on Debian 7.
This also fixes a potential GC problem with Thread.start or
IO#read potentially closing the write-end of the pipe due
to allocation.
cf. http://www.rubyist.net/~akr/chkbuild/debian/ruby-trunk/log/20180805T080500Z.fail.html.gz
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64199 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
MJIT way generate an extra SIGCHLD, so we'll have to deal with it.
Any realistic Ruby program may hit unexpected SIGCHLD, too,
since any spawned subprocess could have extra grandchildren
which get reaped by the main Ruby process, and SIGCHLD may
be sent spuriously by an external process using kill(2) syscall
(via kill(1) or Process.kill).
cf. http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1194620
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64194 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We can reuse the sub-thread and exception with Thread#raise to
reproduce the old memory leak with less overhead. This allows
us to to run more iterations and improve reliability of the
actual test, particularly on platforms without USE_THREAD_CACHE.
For glibc and jemalloc, also limit arena count to avoid
inadvertant growth.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Use Thread.pass to let the sub-thread finish. This should avoid
running out of memory on resource-constrained systems.
(We don't want to use Thread#join to hit the exception)
This results in a significant memory usage reduction in this test:
make test-all TESTS='../test/ruby/test_thread.rb -v -n test_thread_interrupt_for_killed_thread'
Before:
3.46user 2.02system 0:05.29elapsed 103%CPU (0avgtext+0avgdata 88672maxresident)k
0inputs+0outputs (0major+143071minor)pagefaults 0swaps
After:
3.40user 1.83system 0:05.20elapsed 100%CPU (0avgtext+0avgdata 9368maxresident)k
0inputs+0outputs (0major+3059minor)pagefaults 0swaps
cf. https://rubyci.org/logs/rubyci.s3.amazonaws.com/scw-9d6766/ruby-trunk/log/20180803T231706Z.fail.html.gz
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64186 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Still looking into CI failures on P895 (and seemingly no other boxes):
http://ci.rvm.jp/results/trunk@P895/1190369
:<
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64184 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ast.c (Init_ast): undefine allocator of Node, as a method call
on an uninitialized Node causes segfault.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64168 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Actually unload_units is working on MinGW, but putiseq is behaving
badly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64163 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It looks like we need to retry test_timestampns in addition
to test_timestamp; so share some code while we're at it.
cf. http://ci.rvm.jp/results/trunk-test@frontier/1153126
[ruby-core:88104] [Bug #14898]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64157 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
when $MJIT_SEARCH_BUILD_DIR is set.
If prefix path is owned by root, `make install` needs to be run by root.
But in general we don't want to run `make test-all`, and also running
`make test-all` currently fails due to permission tests of rdoc and rubygems.
Thus, prior to this commit, specifying a prefix like "/usr/local" could
mean there was no way to pass test-all.
So we should not depend on `make install` for `make test-all`. Thus I
reverted r64104 and r64103, and applied this workaround to pass `make
test-all` without `make install`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64143 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/ruby/test_thread_{cv,queue}.rb: Move under the test/ruby directory.
and rename TestThread* from Test*.
* test/test_sync.rb: Move toplevel of test diretory because sync is still
standard library.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64137 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I can't reproduce the test failure and I'm still not sure what's
wrong, but maybe we can get a core dump from P895:
http://ci.rvm.jp/results/trunk@P895/1174672
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64135 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
To reduce resource use and reduce CI failure; remove
timer-thread. Single-threaded Ruby processes (including forked
children) will never see extra thread overhead. This prevents
glibc and jemalloc from going into multi-threaded mode and
initializing locks or causing fragmentation via arena explosion.
The GVL is implements its own wait-queue as a ccan/list to
permit controlling wakeup order. Timeslice under contention is
handled by a designated timer thread (similar to choosing a
"patrol_thread" for current deadlock checking).
There is only one self-pipe, now, as wakeups for timeslice are
done independently using condition variables. This reduces FD
pressure slightly.
Signal handling is handled directly by a Ruby Thread (instead
of timer-thread) by exposing signal self-pipe to callers of
rb_thread_fd_select, native_sleep, rb_wait_for_single_fd, etc...
Acquiring, using, and releasing the self-pipe is exposed via 4
new internal functions:
1) rb_sigwait_fd_get - exclusively acquire timer_thread_pipe.normal[0]
2) rb_sigwait_fd_sleep - sleep and wait for signal (and no other FDs)
3) rb_sigwait_fd_put - release acquired result from rb_sigwait_fd_get
4) rb_sigwait_fd_migrate - migrate signal handling to another thread
after calling rb_sigwait_fd_put.
rb_sigwait_fd_migrate is necessary for waitpid callers because
only one thread can wait on self-pipe at a time, otherwise a
deadlock will occur if threads fight over the self-pipe.
TRAP_INTERRUPT_MASK is now set for the main thread directly in
signal handler via rb_thread_wakeup_timer_thread.
Originally, I wanted to use POSIX timers
(timer_create/timer_settime) for this. Unfortunately, this
proved unfeasible as Mutex#sleep resumes on spurious wakeups and
test/thread/test_cv.rb::test_condvar_timed_wait failed. Using
pthread_sigmask to mask out SIGVTALRM fixed that test, but
test/fiddle/test_function.rb::test_nogvl_poll proved there'd be
some unavoidable (and frequent) incompatibilities from that
approach.
Finally, this allows us to drop thread_destruct_lock and
interrupt current ec directly.
We don't need to rely on vm->thread_destruct_lock or a coherent
vm->running_thread on any platform. Separate timer-thread for
time slice and signal handling is relegated to thread_win32.c,
now.
[ruby-core:88088] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64107 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This should fix CI failure under MJIT, 100ms wait may not be
enough for signal handler to fire:
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1169472
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
for the ease of MSP-Greg's CI. I'll track this failure as Bug#14948 and
fix it later.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When --jit-wait is specified, `unit_queue.length` is always 0 and it's
not a good metric.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64097 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When all compilation finishes or the number of JIT-ed code reaches
--jit-max-cache, this compacts all generated code to a single .so file
and re-loads all methods from it.
In the future, it may trigger compaction more frequently and/or limit
the maximum times of compaction to prevent unlimited memory usage.
So the current behavior is experimental, but at least the performance
improvement in this commit won't be removed.
=== Benchmark ===
In this benchmark, I'll compare following four conditions:
* trunk: r64082
* trunk JIT: r64082 w/ --jit
* single-so JIT: This commit w/ --jit
* objfcn JIT: This branch https://github.com/k0kubun/ruby/tree/objfcn w/ --jit,
which is shinh's objfcn https://github.com/shinh/ruby/tree/objfcn rebased from this commit
```
$ uname -a
Linux bionic 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
```
* Micro benchmark
Using this script https://gist.github.com/k0kubun/10e6d3387c9ab1b134622b2c9d76ef51,
calls some amount of different methods that just return `nil`. The following tables
are its average duration seconds of 3 measurements.
Smaller is better.
** 1 method (seconds)
| | trunk | trunk JIT | single-so JIT | objfcn JIT |
|:------|:------------------|:------------------|:------------------|:------------------|
| Time | 5.576067774333296 | 5.915551971666446 | 5.833641665666619 | 5.845915191666639 |
| Ratio | 1.00x | 1.06x | 1.05x | 1.05x |
** 50 methods (seconds)
| | trunk | trunk JIT | single-so JIT | objfcn JIT |
|:------|:------------------|:------------------|:------------------|:------------------|
| Time | 3.1661167996666677| 6.125825928333342 | 4.135432743666665 | 3.750358728333348 |
| Ratio | 1.00x | 1.93x | 1.31x | 1.18x |
** 1500 methods (seconds)
| | trunk | trunk JIT | single-so JIT | objfcn JIT |
|:------|:------------------|:------------------|:------------------|:------------------|
| Time | 5.971650823666664 | 19.579182102999994| 10.511108153999961| 10.854653588999932|
| Ratio | 1.00x | 3.28x | 1.76x | 1.82x |
* Discourse
Using the same benchmark strategy as https://bugs.ruby-lang.org/issues/14490 with
this branch https://github.com/k0kubun/discourse/commits/benchmark2 forked from discourse
v1.8.11 to support running trunk.
1. Run ruby script/bench.rb to warm up profiling database
2. Run RUBYOPT='--jit-verbose=1 --jit-max-cache=10000' RAILS_ENV=profile bin/puma -e production
3. WAIT 5-15 or so minutes for all jitting to stop so we have no cross talk
4. Run ab -n 100 http://localhost:9292/
5. Wait for all new jitting to finish
6. Run ab -n 100 http://localhost:9292/
** Response time (ms)
Here is the response time milliseconds for each percentile.
Skipping 99%ile because it's the same as 100%ile in 100 calls.
| | trunk| trunk|single|objfcn|
| | | JIT|so JIT| JIT|
|:----|:-----|:-----|:-----|:-----|
| 50% | 38 | 45 | 41 | 43 |
| 66% | 39 | 50 | 44 | 44 |
| 75% | 47 | 51 | 46 | 45 |
| 80% | 49 | 52 | 47 | 47 |
| 90% | 50 | 63 | 50 | 52 |
| 95% | 60 | 79 | 52 | 55 |
| 98% | 91 | 114 | 91 | 91 |
|100% | 97 | 133 | 96 | 99 |
** Ratio (smaller is better)
Here is the response time increase ratio against no-JIT trunk's one.
| | trunk| trunk|single|objfcn|
| | | JIT|so JIT| JIT|
|:----|:-----|:-----|:-----|:-----|
| 50% | 1.00x| 1.18x| 1.08x| 1.13x|
| 66% | 1.00x| 1.28x| 1.13x| 1.13x|
| 75% | 1.00x| 1.09x| 0.98x| 0.96x|
| 80% | 1.00x| 1.06x| 0.96x| 0.96x|
| 90% | 1.00x| 1.26x| 1.00x| 1.04x|
| 95% | 1.00x| 1.32x| 0.87x| 0.92x|
| 98% | 1.00x| 1.25x| 1.00x| 1.00x|
|100% | 1.00x| 1.37x| 0.99x| 1.02x|
While 50 and 60 %ile are still worse than no-JIT trunk, 75, 80, 90, 95,
98 and 100% are not slower than that.
So now it's a little harder to say "MJIT slows down Rails applications".
Probably I can close [Bug #14490] now. Let's start improving it.
Close https://github.com/ruby/ruby/pull/1921
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64094 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/unicode_normalize/normalize.rb: Fix the range check for trailing
Hangul jamo characters in Unicode normalization. Different from
leading or vowel jamos, where LBASE and VBASE are actual characters,
a value equal to TBASE expresses the absence of a trailing jamo.
This fix is technically correct, but there was no bug because
the regular expressions in lib/unicode_normalize/tables.rb
eliminate jamos equal to TBASE from normalization processing.
* test/test_unicode_normalize.rb: Add preventive test
test_no_trailing_jamo based on
d134809cd3
just for the case we ever get a regression.
This closes issue #14934, thanks to MaLin (Lin Ma) for reporting.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64087 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/test_unicode_normalize.rb: extract tests that do not depend
on NormalizationTest.txt data file from conditionally constructed
part of TestUnicodeNormalize class, to always run them even if
the data file isn't found.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/test_unicode_normalize.rb: search ucd directory
first if it exists. This change follows r61415.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64085 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and lazily delete them on termination.
This will be needed to create a large so file later.
The large number of .o files will be probably compacted before the large so
file is created.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64077 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ruby.c (external_str_new_cstr): strings come from the external
should be tainted. [ruby-dev:50596] [Bug #14941]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64071 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This adds a password_hash keyword argument to
WEBrick::HTTPAuth::Htpasswd#initialize. If set to :bcrypt, it
will create bcrypt hashes instead of crypt hashes, and will
raise an exception if the .htpasswd file uses crypt hashes.
If :bcrypt is used, then instead of calling
BasicAuth.make_passwd (which uses crypt),
WEBrick::HTTPAuth::Htpasswd#set_passwd will set the bcrypt
password directly. It isn't possible to change the
make_passwd API to accept the password hash format, as that
would break configurations who use Htpasswd#auth_type= to set
a custom auth_type.
This modifies WEBrick::HTTPAuth::BasicAuth to handle checking
both crypt and bcrypt hashes.
There are commented out requires for 'string/crypt', to handle
when String#crypt is deprecated and the undeprecated version is
moved to a gem.
There is also a commented out warning for the case when
the password_hash keyword is not specified and 'string/crypt'
cannot be required. I think the warning makes sense to nudge
users to using bcrypt.
I've updated the tests to test nil, :crypt, and :bcrypt values
for the password_hash keyword, skipping the bcrypt tests if the
bcrypt library cannot be required.
[ruby-core:88111] [Feature #14940]
From: Jeremy Evans <code@jeremyevans.net>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64060 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
in a short term, and add retries to prevent random CI failures by it.
I remember this and will address it later for sure.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64032 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* test/ruby/test_io.rb (test_select_leak): increase timeout seconds
to pass this test on a high-load machine.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64019 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* object.c (rb_f_integer): prefer `base` optional argument over
keyword arguments. this issue should be resolved more generally
by separating keyword arguments from hashes in the future.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64015 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
mrkn
suke
k0kubun
nobu
normal
svn
ko1
mame
hsbt
marcandre
kazu
tarui
usa
naruse
rhe
duerst