This reverts commit 87f6154bb4.
It turned out that the change fails to build on macOS
https://rubyci.org/logs/rubyci.s3.amazonaws.com/osx1014/ruby-master/log/20200304T074503Z.fail.html.gz
```
+ make 'TESTS=--hide-skip -v fiddle' RUBYOPT=-w test-all
dyld: lazy symbol binding failed: Symbol not found: _ffi_closure_alloc
Referenced from: /Users/hsbt/Documents/cb/tmp/build/20200304T074503Z/ruby/.ext/x86_64-darwin18/fiddle.bundle
Expected in: flat namespace
dyld: Symbol not found: _ffi_closure_alloc
Referenced from: /Users/hsbt/Documents/cb/tmp/build/20200304T074503Z/ruby/.ext/x86_64-darwin18/fiddle.bundle
Expected in: flat namespace
make: *** [yes-test-all] Abort trap: 6
```
This reverts commit efd641ffab.
This changeset seems to be needed to suppress a warning on Ubuntu 20.04
https://rubyci.org/logs/rubyci.s3.amazonaws.com/ubuntu/ruby-master/log/20200304T033004Z.log.html.gz
```
closure.c:264:5: warning: 'ffi_prep_closure' is deprecated: use ffi_prep_closure_loc instead [-Wdeprecated-declarations]
264 | result = ffi_prep_closure(pcl, cif, callback, (void *)self);
| ^~~~~~
```
I guess there was a reason why the commit was reverted (maybe some CIs
failed?), so try it again.
This patch contains several ideas:
(1) Disposable inline method cache (IMC) for race-free inline method cache
* Making call-cache (CC) as a RVALUE (GC target object) and allocate new
CC on cache miss.
* This technique allows race-free access from parallel processing
elements like RCU.
(2) Introduce per-Class method cache (pCMC)
* Instead of fixed-size global method cache (GMC), pCMC allows flexible
cache size.
* Caching CCs reduces CC allocation and allow sharing CC's fast-path
between same call-info (CI) call-sites.
(3) Invalidate an inline method cache by invalidating corresponding method
entries (MEs)
* Instead of using class serials, we set "invalidated" flag for method
entry itself to represent cache invalidation.
* Compare with using class serials, the impact of method modification
(add/overwrite/delete) is small.
* Updating class serials invalidate all method caches of the class and
sub-classes.
* Proposed approach only invalidate the method cache of only one ME.
See [Feature #16614] for more details.
Now, rb_call_info contains how to call the method with tuple of
(mid, orig_argc, flags, kwarg). Most of cases, kwarg == NULL and
mid+argc+flags only requires 64bits. So this patch packed
rb_call_info to VALUE (1 word) on such cases. If we can not
represent it in VALUE, then use imemo_callinfo which contains
conventional callinfo (rb_callinfo, renamed from rb_call_info).
iseq->body->ci_kw_size is removed because all of callinfo is VALUE
size (packed ci or a pointer to imemo_callinfo).
To access ci information, we need to use these functions:
vm_ci_mid(ci), _flag(ci), _argc(ci), _kwarg(ci).
struct rb_call_info_kw_arg is renamed to rb_callinfo_kwarg.
rb_funcallv_with_cc() and rb_method_basic_definition_p_with_cc()
is temporary removed because cd->ci should be marked.
The `cxx.try_compile` command in this file kicks `cxx.have_devel?`
internally, which recursively calls `cxx.try_link` with a different
source code. We don't want that happen (the source code compiled in
this file must be the first one). We need to fake the system.
Stop the special treatment of invalid hashAlgorithm of the message
imprint. Those invalid values can only appear after the object is
instantiated, before the user sets an actual message digest algorithm.
OpenSSL::Timestamp::TokenInfo#algorithm already does the same.
Also, remove the test case "test_create_request" since it does not make
much sense. Those fields are to be set by the user after creation of
the object and checking the initial value is pointless.
Fixes: https://github.com/ruby/openssl/issues/335https://github.com/ruby/openssl/commit/890a6476fa
It fails to build on Solaris:
https://rubyci.org/logs/rubyci.s3.amazonaws.com/solaris11-gcc/ruby-master/log/20200216T090008Z.log.html.gz
```
ossl_cipher.c: 関数 ‘ossl_cipher_init’ 内:
ossl_cipher.c:228:2: エラー: ‘EVP_md5’ is deprecated [-Werror=deprecated-declarations]
228 | EVP_BytesToKey(EVP_CIPHER_CTX_cipher(ctx), EVP_md5(), iv,
| ^~~~~~~~~~~~~~
In file included from /usr/include/openssl/x509.h:73,
from /usr/include/openssl/x509v3.h:63,
from ossl.h:23,
from ossl_cipher.c:10:
/usr/include/openssl/evp.h:732:26: 備考: ここで宣言されています
732 | DEPRECATED const EVP_MD *EVP_md5(void);
| ^~~~~~~
```
DEBUG: BUILDSTDERR: /usr/bin/ld: infinite_loop_dlsym.o: in function `native_loop_dlsym':
DEBUG: BUILDSTDERR: /builddir/build/BUILD/ruby-2.7.0/ext/-test-/popen_deadlock/infinite_loop_dlsym.c:16: undefined reference to `dlsym'
DEBUG: BUILDSTDERR: collect2: error: ld returned 1 exit status
Ruby was built with LibreSSL.
C++ keyword `nullptr` represents a null pointer (note also that NULL is
an integer in C++ due to its design flaw). Its type is `std::nullptr_t`,
defined in <cstddef> standard header. Why not support it when the
backend implementation can take a null pointer as an argument.
BSD make can run parallel more aggressively than GNU make. It communicate
with other make process through -J option in MAKEFLAGS environment variable
to notify a build failure happend in an other pararell make process.
https://www.freebsd.org/cgi/man.cgi?make
It usually works well but ext/-test-/cxxanyargs/Makefile has two targets
which are expected to fail (failure.o and failurem1.o).
Additional note:
To test and debug this issue, following command will speed up it.
`make -f exts.mk -j8 clean all`
This removes the warnings added in 2.7, and changes the behavior
so that a final positional hash is not treated as keywords or
vice-versa.
To handle the arg_setup_block splat case correctly with keyword
arguments, we need to check if we are taking a keyword hash.
That case didn't have a test, but it affects real-world code,
so add a test for it.
This removes rb_empty_keyword_given_p() and related code, as
that is not needed in Ruby 3. The empty keyword case is the
same as the no keyword case in Ruby 3.
This changes rb_scan_args to implement keyword argument
separation for C functions when the : character is used.
For backwards compatibility, it returns a duped hash.
This is a bad idea for performance, but not duping the hash
breaks at least Enumerator::ArithmeticSequence#inspect.
Instead of having RB_PASS_CALLED_KEYWORDS be a number,
simplify the code by just making it be rb_keyword_given_p().
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead. This would significantly
speed up incremental builds.
We take the following inclusion order in this changeset:
1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very
first thing among everything).
2. RUBY_EXTCONF_H if any.
3. Standard C headers, sorted alphabetically.
4. Other system headers, maybe guarded by #ifdef
5. Everything else, sorted alphabetically.
Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
* Default VMIN and VTIME to minimum input.
* Disable parity check bits explicitly.
* Disable all bits for flow control on input.
Co-Authored-By: NARUSE, Yui <naruse@airemix.jp>
https://github.com/ruby/io-console/commit/5ce201a686
TCSAFLUSH discards the buffer read before the mode change, which makes
IRB ignore the buffer input immediately after invoked. TCSANOW
preserves the buffer.
https://github.com/ruby/io-console/commit/b362920182
It fails to build on Solaris:
```
ossl_cipher.c: 関数 ‘ossl_cipher_init’ 内:
ossl_cipher.c:228:2: エラー: ‘EVP_md5’ is deprecated [-Werror=deprecated-declarations]
228 | EVP_BytesToKey(EVP_CIPHER_CTX_cipher(ctx), EVP_md5(), iv,
| ^~~~~~~~~~~~~~
In file included from /usr/include/openssl/x509.h:73,
from /usr/include/openssl/x509v3.h:63,
from ossl.h:23,
from ossl_cipher.c:10:
/usr/include/openssl/evp.h:732:26: 備考: ここで宣言されています
732 | DEPRECATED const EVP_MD *EVP_md5(void);
| ^~~~~~~
```
I agree that `-Werror=` is a good habit, but adding it by default is too
aggressive.
Asynchronous events such as signal trap, finalization timing,
thread switching and so on are managed by "interrupt_flag".
Ruby's threads check this flag periodically and if a thread
does not check this flag, above events doesn't happen.
This checking is CHECK_INTS() (related) macro and it is placed
at some places (laeve instruction and so on). However, at the end
of C methods, C blocks (IMEMO_IFUNC) etc there are no checking
and it can introduce uninterruptible thread.
To modify this situation, we decide to place CHECK_INTS() at
vm_pop_frame(). It increases interrupt checking points.
[Bug #16366]
This patch can introduce unexpected events...
Ruby 2.7 deprecates taint and it no longer has an effect.
The lack of taint support should not cause a problem in
previous Ruby versions.
Still untaint the tmpdir object on Ruby <2.7, as returning
a tainted string there could cause problems.
This removes the related tests, and puts the related specs behind
version guards. This affects all code in lib, including some
libraries that may want to support older versions of Ruby.
This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.
This modifies some internal functions that took a safe level argument
to no longer take the argument.
rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.
One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd. We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.
This removes the taint checking. Taint support is deprecated in
Ruby 2.7 and has no effect. I don't think removing the taint
checks in earlier ruby versions will cause any problems.
https://github.com/ruby/bigdecimal/commit/1918d466f3
Ruby 2.7 deprecates taint and it no longer has an effect.
The lack of taint support should not cause a problem in
previous Ruby versions.
I'm not sure if the untaint calls in deduplicate are still needed
after the removal of tainting in the parser. If they are not
needed, they should be removed.
https://github.com/ruby/psych/commit/73c1a2b4e0
I noticed that some files in rubygems were executable, and I could think
of no reason why they should be.
In general, I think ruby files should never have the executable bit set
unless they include a shebang, so I run the following command over the
whole repo:
```bash
find . -name '*.rb' -type f -executable -exec bash -c 'grep -L "^#!" $1 || chmod -x $1' _ {} \;
```
This gets the time zone abbreviations from
https://www.timeanddate.com/time/zones/, and adds unambiguous time
zones not already present in zonetab.list. See bin/update-abbr
for the program used.
This regenerates zonetab.h using prereq.mk (requires gperf).
Only one test line is added, just to make sure a new time zone
abbreviation is picked up.
Fixes Ruby Bug 16286
https://github.com/ruby/date/commit/702e8b3033
Before this change, it was not possible to write out zero for the
timestamp part of a Gzip file's header, as calling GzipWriter#mtime with
zero was ignored.
Judging from the docs for `GzipWriter#mtime=`, it should be possible to
indicate that no timestamp is available by calling the method with zero.
https://github.com/ruby/zlib/commit/310be39cac
Ruby 2.7 deprecates taint and it no longer has an effect.
The lack of taint support should not cause a problem in
previous Ruby versions.
https://github.com/ruby/gdbm/commit/f9aaa1a08d
Ruby 2.7 deprecates taint and it no longer has an effect.
The lack of taint support should not cause a problem in
previous Ruby versions.
https://github.com/ruby/zlib/commit/21711ed0ce
* More explanations/examples in class docs;
* Fix links to other methods (remove <code> tag);
* Fix wording of method docs (remove *stringio*
receiver name, as it is not rendered by modern
RDoc);
* Add option mention to linereading
methods (added in 2.4);
* Several other small fixes.
Ruby 2.7 deprecates taint and it no longer has an effect.
The lack of taint support should not cause a problem in
previous Ruby versions.
https://github.com/ruby/dbm/commit/1f0ff0bce1
* Use ffi_closure_free unconditionally.
The current conditionals reflect historic heritage of FFI. Usage of
ffi_closure_free should be better default nowadays, because libffi 3.0.5
fixing issues of ffi_closure_free should be widely available.
* RUBY_LIBFFI_MODVERSION is not used anymore.
Because `ffi_closure_free()` is not used unconditionally, there is no
other use for RUBY_LIBFFI_MODVERSION define, so drop its usage.
* Use more meaningful variable name.
`ver` variable used to be used to pupulate RUBY_LIBFFI_MODVERSION
define. Since the define was removed, the `libffi_dir` variable name
should better describe the remaining usage of the variable.
https://github.com/ruby/fiddle/commit/c49cc79eb8
If the first parameter to Fiddle::Function is a closure object (rather
than an interger), `rb_Integer` will cast it to an integer but not
maintain a reference to the closure. Then if the closure gets GC'd, we
have a segv. This commit keeps a reference to the original parameter to
initialize so that the object will not be GC'd.
Fixes: https://bugs.ruby-lang.org/issues/13286https://github.com/ruby/fiddle/commit/0fc697bbc5
Previously, the type of these arguments were not checked, leading to
NoMethodErrors in some cases, and TypeErrors in other cases, but not
showing what field was having the problems. This change makes it so
the field with the problem is included in the error message.
For the valid_*? methods, this changes them to return false if one
of the arguments that should be numeric is not.
Fixes Ruby Bug 11935
Fixes Ruby Misc 15298
https://github.com/ruby/date/commit/a2f4b665f8
Previously, julian dates would not round trip through to_time.to_date,
because Time is always considered gregorian. This converts the Date
instance from julian to gregorian before converting to Time, ensuring
that an equal date object will be returned if converting that Time
back to Date.
This does result in julian Date objects showing different day values
if converting to Time.
Fixes Ruby Bug 8428.
https://github.com/ruby/date/commit/d8df64555e
Tabs were expanded because previously the file did not have any tab indentation.
Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
Delegate MonitorMixin#synchronize body to Monitor#synchronize.
It makes guarantee interrupt safe (because Monitor#synchronize is
written in C). I thought Ruby implementation is also safe, but I
got stuck failure <http://ci.rvm.jp/results/trunk_test@P895/2327639>
so that I introduce this fix to guarantee interrupt safe.
Recent monitor.rb has performance problem because of interrupt
handlers. 'Monitor#synchronize' is frequently used primitive
so the performance of this method is important.
This patch rewrite 'monitor.rb' with 'monitor.so' (C-extension)
and make it faster. See [Feature #16255] for details.
Monitor class objects are normal object which include MonitorMixin.
This patch introduce a Monitor class which is implemented on C
and MonitorMixin uses Monitor object as re-entrant (recursive)
Mutex. This technique improve performance because we don't need
to care atomicity and we don't need accesses to instance variables
any more on Monitor class.
In where to convert Hash key to String for json, this patch will add shortcut for String/Symbol in Hash key.
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
json 65.000 i/100ms
Calculating -------------------------------------
json 659.576 (± 1.5%) i/s - 3.315k in 5.027127s
```
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
json 78.000 i/100ms
Calculating -------------------------------------
json 789.781 (± 2.7%) i/s - 3.978k in 5.041043s
```
```
require 'json'
require 'benchmark/ips'
obj = []
1000.times do |i|
obj << {
"id" => i,
:age => 42,
}
end
Benchmark.ips do |x|
x.report "json" do |iter|
count = 0
while count < iter
JSON.generate(obj)
count += 1
end
end
end
```
https://github.com/flori/json/commit/38c0f6dbe4
To convert Hash convert, this part was using following pseudo code
```
obj.keys.each do |key|
value = obj[key]
...
end
```
and `rb_funcall()` was called for `obj.keys`.
It might be slightly heavy to call the Ruby method.
This patch will iterate to convert Hash object about key/value using `rb_hash_foreach()` Ruby API instead of `rb_funcall()`.
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
json 55.000 i/100ms
Calculating -------------------------------------
json 558.501 (± 1.1%) i/s - 2.805k in 5.022986s
```
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
json 65.000 i/100ms
Calculating -------------------------------------
json 659.576 (± 1.5%) i/s - 3.315k in 5.027127s
```
```
require 'json'
require 'benchmark/ips'
obj = []
1000.times do |i|
obj << {
"id" => i,
:age => 42,
}
end
Benchmark.ips do |x|
x.report "json" do |iter|
count = 0
while count < iter
JSON.generate(obj)
count += 1
end
end
end
```
https://github.com/flori/json/commit/a73323dc5e
`rb_funcall` might be slightly heavy to call the Ruby method.
This patch will convert String encoding using `rb_str_encode()` instead of `rb_funcall()`.
## Before
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
json 78.000 i/100ms
Calculating -------------------------------------
json 789.781 (± 2.7%) i/s - 3.978k in 5.041043s
```
## After
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
json 129.000 i/100ms
Calculating -------------------------------------
json 1.300k (± 2.3%) i/s - 6.579k in 5.064656s
```
## Code
```
require 'json'
require 'benchmark/ips'
obj = []
1000.times do |i|
obj << {
"id" => i,
:age => 42,
}
end
Benchmark.ips do |x|
x.report "json" do |iter|
count = 0
while count < iter
JSON.generate(obj)
count += 1
end
end
end
```
https://github.com/flori/json/commit/9ae6d2969c
To get rid of a bug of `onig_region_set` which takes `int`s
instead of `OnigPosition`s, set elements of `beg` and `end`
members directly, for the time being.
Cited from mount(8):
```
strictatime
Always update the file access time when reading from a
file. Without this option the filesystem may default to a
less strict update mode, where some access time updates
are skipped for performance reasons. This option could be
ignored if it is not supported by the filesystem.
```
* ext/openssl/ossl_asn1.c (Init_ossl_asn1): prefer
`rb_gc_register_mark_object`, which is better for constant
objects, over `rb_gc_register_address` for global/static
variables which can be re-assigned at runtime. [Bug #16196]
* ext/openssl/ossl_asn1.c (Init_ossl_asn1): register the static
variable to grab an internal object, before creating the object.
otherwise the just-created object could get collected during the
global variable list allocation. [Bug #16196]
There have been some direct changes in parser.c which is automatically
generated from parser.rl. This updates parser.rl to sync the changes:
* 91793b8967
* 79ead821dd
* 80b5a0ff2a
This reverts commits: 10d6a3aca78ba48c1b85fba8627dc1dd883de5ba6c6a25feca167e6b48f17cb96d41a53207979278595b3c4fdd1521f7cf89c11c5e69accf336082033632a812c0f56506be0d86427a3219 .
The reason for the revert is that we observe ABA problem around
inline method cache. When a cache misshits, we search for a
method entry. And if the entry is identical to what was cached
before, we reuse the cache. But the commits we are reverting here
introduced situations where a method entry is freed, then the
identical memory region is used for another method entry. An
inline method cache cannot detect that ABA.
Here is a code that reproduce such situation:
```ruby
require 'prime'
class << Integer
alias org_sqrt sqrt
def sqrt(n)
raise
end
GC.stress = true
Prime.each(7*37){} rescue nil # <- Here we populate CC
class << Object.new; end
# These adjacent remove-then-alias maneuver
# frees a method entry, then immediately
# reuses it for another.
remove_method :sqrt
alias sqrt org_sqrt
end
Prime.each(7*37).to_a # <- SEGV
```
Currently, there is not a way to create a sized enumerator in C
with a different set of arguments than provided by Ruby, and
correctly handle keyword arguments. This function allows that.
The need for this is fairly uncommon, but it occurs at least in
Enumerator.produce, which takes arugments from Ruby but calls
rb_enumeratorize_with_size with a different set of arguments.
This adds rb_funcall_passing_block_kw, rb_funcallv_public_kw,
and rb_yield_splat_kw. This functions are necessary to easily
handle cases where rb_funcall_passing_block, rb_funcallv_public,
and rb_yield_splat are currently used and a keyword argument
separation warning is raised.
Now that we have eliminated most destructive operations over the
rb_method_entry_t / rb_callable_method_entry_t, let's make them
mostly immutabe and mark them const.
One exception is rb_export_method(), which destructively modifies
visibilities of method entries. I have left that operation as is
because I suspect that destructiveness is the nature of that
function.
This fixes instance_exec and similar methods. It also fixes
Enumerator::Yielder#yield, rb_yield_block, and a couple of cases
with Proc#{<<,>>}.
This support requires the addition of rb_yield_values_kw, similar to
rb_yield_values2, for passing the keyword flag.
Unlike earlier attempts at this, this does not modify the rb_block_call_func
type or add a separate function type. The functions of type
rb_block_call_func are called by Ruby with a separate VM frame, and we can
get the keyword flag information from the VM frame flags, so it doesn't need
to be passed as a function argument.
These changes require the following VM functions accept a keyword flag:
* vm_yield_with_cref
* vm_yield
* vm_yield_with_block
It's unlikely anyone would actually hit these. The methods are
private, you only hit this code path if calling these methods
before performing the SSL connection, and there is already a
verbose warning issued.
Cfuncs that use rb_scan_args with the : entry suffer similar keyword
argument separation issues that Ruby methods suffer if the cfuncs
accept optional or variable arguments.
This makes the following changes to : handling.
* Treats as **kw, prompting keyword argument separation warnings
if called with a positional hash.
* Do not look for an option hash if empty keywords are provided.
For backwards compatibility, treat an empty keyword splat as a empty
mandatory positional hash argument, but emit a a warning, as this
behavior will be removed in Ruby 3. The argument number check
needs to be moved lower so it can correctly handle an empty
positional argument being added.
* If the last argument is nil and it is necessary to treat it as an option
hash in order to make sure all arguments are processed, continue to
treat the last argument as the option hash. Emit a warning in this case,
as this behavior will be removed in Ruby 3.
* If splitting the keyword hash into two hashes, issue a warning, as we
will not be splitting hashes in Ruby 3.
* If the keyword argument is required to fill a mandatory positional
argument, continue to do so, but emit a warning as this behavior will
be going away in Ruby 3.
* If keyword arguments are provided and the last argument is not a hash,
that indicates something wrong. This can happen if a cfunc is calling
rb_scan_args multiple times, and providing arguments that were not
passed to it from Ruby. Callers need to switch to the new
rb_scan_args_kw function, which allows passing of whether keywords
were provided.
This commit fixes all warnings caused by the changes above.
It switches some function calls to *_kw versions with appropriate
kw_splat flags. If delegating arguments, RB_PASS_CALLED_KEYWORDS
is used. If creating new arguments, RB_PASS_KEYWORDS is used if
the last argument is a hash to be treated as keywords.
In open_key_args in io.c, use rb_scan_args_kw.
In this case, the arguments provided come from another C
function, not Ruby. The last argument may or may not be a hash,
so we can't set keyword argument mode. However, if it is a
hash, we don't want to warn when treating it as keywords.
In Ruby files, make sure to appropriately use keyword splats
or literal keywords when calling Cfuncs that now issue keyword
argument separation warnings through rb_scan_args. Also, make
sure not to pass nil in place of an option hash.
Work around Kernel#warn warnings due to problems in the Rubygems
override of the method. There is an open pull request to fix
these issues in Rubygems, but part of the Rubygems tests for
their override fail on ruby-head due to rb_scan_args not
recognizing empty keyword splats, which this commit fixes.
Implementation wise, adding rb_scan_args_kw is kind of a pain,
because rb_scan_args takes a variable number of arguments.
In order to not duplicate all the code, the function internals need
to be split into two functions taking a va_list, and to avoid passing
in a ton of arguments, a single struct argument is used to handle
the variables previously local to the function.