Since r63443, `-std=gnu99 -D_XOPEN_SOUCE=x00` is added to Solaris'
`CPPFLAGS`. `CPPFLAGS` is shared among `CC` / `CXX`. This results in
both `__STDC_VERSION__` and `__cplusplus` to be defined at the same time
for a C++ compilation, only on Solaris.
It seems the `CPPFLAGS` addition is intentional. We sould not touch
that part. Instead we need to reroute this by always check for
`__cplusplus` first.
* Add buffer protocol
* Modify for some review comments
* Per-object buffer availability
* Rename to MemoryView from Buffer and make compilable
* Support integral repeat count in memory view format
* Support 'x' for padding bytes
* Add rb_memory_view_parse_item_format
* Check type in rb_memory_view_register
* Update dependencies in common.mk
* Add test of MemoryView
* Add test of rb_memory_view_init_as_byte_array
* Add native size format test
* Add MemoryView test utilities
* Add test of rb_memory_view_fill_contiguous_strides
* Skip spaces in format string
* Support endianness specifiers
* Update documentation
* Support alignment
* Use RUBY_ALIGNOF
* Fix format parser to follow the pack format
* Support the _ modifier
* Parse count specifiers in get_format_size function.
* Use STRUCT_ALIGNOF
* Fix test
* Fix test
* Fix total size for the case with tail padding
* Fix rb_memory_view_get_item_pointer
* Fix rb_memory_view_parse_item_format again
Now that RUBY_ALIGNOF behaves like C11's _Alignof. This is not
necessarily the best stack arrangement. We can just give up using
__builtin_alloca_with_align(), and let alloca choose what is optimal.
It is reported that on a system of i386 System V ABI, GCC returns 8 for
__alignof__(double). OTOH the ABI defines alignments of double to be 4,
and ISO/IEC 9899:2011 reads that _Alignof(double) shall return 4 on such
machine. What we want in ruby is 4 instead of 8 there. We cannot use
__alignof__.
Additionally, both old GCC / old clang return 8 for _Alignof(double) on
such platforms. They are their bugs, and already fixed in recent
versions. But we have to support older compilers for a while. Shall
check sanity of _Alignof.
Added `WITH_REAL` versions to `RB_RANDOM_INTERFACE` macros. Also
these macros including "without real" versions no longer contain
the terminator (semicolon and comma).
* random.c: separate abstract rb_random_t and rb_random_mt_t for
Mersenne Twister implementation.
* include/ruby/random.h: the interface for extensions of Random
class.
* DLL imported symbol reference is not constant on Windows.
* check if properly initialized.
This commit introduces Ractor mechanism to run Ruby program in
parallel. See doc/ractor.md for more details about Ractor.
See ticket [Feature #17100] to see the implementation details
and discussions.
[Feature #17100]
This commit does not complete the implementation. You can find
many bugs on using Ractor. Also the specification will be changed
so that this feature is experimental. You will see a warning when
you make the first Ractor with `Ractor.new`.
I hope this feature can help programmers from thread-safety issues.
Commit 7aab062ef3 says:
> ruby_show_version() will no longer exits the process, if
> RUBY_SHOW_COPYRIGHT_TO_DIE is set to 0. This will be the default in
> the future.
3.0 is a good timing for that "future".
Because we check HAVE_STMT_AND_DECL_IN_EXPR in configure, it is peoven
to work in C. But C++ situation can be different. Oracle Developer
Studio is another example of such things.
Oracle Developer Studio's C++ preprocessor started to understand
__has_cpp_attribute since version 12.5, and is broken. After looking
around for a while I found Boost and ICU also had this issue before.
Let me add workaround for it.
Former ROBJECT_IV_INDEX_TBL macro included RCLASS_IV_INDEX_TBL, which is
not disclosed to extension libraies. The macro was kind of broken. Why
not just deprecate it, and convert the internal use into an inline
function.
RARRAY_AREF has been a macro for reasons. We might not be able to
change that for public APIs, but why not relax the situation internally
to make it an inline function.
- When NDEBUG is defined that shall be honoured.
- When RUBY_DEBUG is defined that shall be honoured.
- When both are defined and they conflict, warnings shall be rendered.
- When nothing is specified, nothing shall happen.
If __builtin_assume() is enables and RUBY_DEBUG=0, RUBY_ASSERT(expr)
will be compiled to __builtin_assume(expr) and compiler can assume
expr is true and apply aggressive optimizations. However we observed
doubtful behavior because of compiler optimizations, we introduce
new macro RUBY_ASSERT_NOASSUME to disable __builtin_assume().
With this macro, we can try without __builtin_assume().
Assertions in header files slows down an interpreter, so they should be
turned off by default (simple `make`). To enable them, define a macro
`RUBY_DEBUG=1` (e.g. `make cppflags=-DRUBY_DEBUG` or use `#define` at
the very beggining of the file. Note that even if `NDEBUG=1` is defined,
`RUBY_DEBUG=1` enables all assertions.
[Feature #16837]
related: https://github.com/ruby/ruby/pull/3120
`assert()` lines in MRI *.c is not disabled even if `RUBY_DEBUG=0` and
it can be disabled with `NDEBUG=1`. So please consider to use
`RUBY_ASSERT()` if you want to disable them when `RUBY_DEBUG=0`.
We observed combination of multiple __builtin_assume() can cause
strange compile results on clang-6 (-DNDEBUG exposed this issue).
To avoid this problem, __builtin_assume() is from clang-7.
https://github.com/ruby/ruby/pull/3120#issuecomment-630821333
https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html says:
> GCC never returns 1 when you call the inline function with a string
> constant or ...
Because rb_str_new and familiy take string constants, it was a bad idea
for them to contain RB_CONSTANT_P. We should instead do so in
corresponding macros. [ruby-core:98212]
[Bug #16465] [Bug #16801]
[Fix GH-2795] [Fix GH-2944] [Fix GH-3045] [Fix GH-3093]
Note: Backporting shouldn't modify object.h and instead can use
struct_new_kw which is basically a duplicate implementation of
rb_class_new_instance_pass_kw
Co-authored-by: Yusuke Endoh <mame@ruby-lang.org>
Co-authored-by: John Hawthorn <john@hawthorn.email>
Co-authored-by: Adam Hess <HParker@github.com>
Co-authored-by: Jose Cortinas <jacortinas@gmail.com>
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
As zero variadic macro argument, `rb_scan_args(argc, argv, "")`
(or `"0"`), means 0-arity method and should be defined so, that
case doesn't need to consider.
It seems gcc prior to 5 suffered from preprocessor bug. We have to
provide workarounds.
See https://github.com/ruby/ruby/runs/591138012
----
Here is something interesting. According to C99, strictly speaking,
the gcc behaviour was in fact legal(!) and everything else were
wrong. This was not a bug of gcc, rather a bug of the ISO C Standard.
This defect was reported to the committee as DR#412, and fixed
accordingly. All tested compilers now conform C17's updated
preprocessor description.
See http://www.open-std.org/JTC1/SC22/WG14/www/docs/dr_412.htm
This reverts commit 443389effc.
This reverts commit d94960f22e.
Inclusion of header files must be explicit. Every file shall directly
include what is necessary.
https://github.com/include-what-you-use/include-what-you-use says:
> When every file includes what it uses, then it is possible to edit any
> file and remove unused headers, without fear of accidentally breaking
> the upwards dependencies of that file. It also becomes easy to
> automatically track and update dependencies in the source code.
Though we don't use iwyu itself, the principle quoted above is a good
thing that we can agree.
Now that include guards were added to every and all of the headers
inside of our project this changeset does not increase compile time, at
least on my machine.
According to MSVC manual (*1), cl.exe can skip including a header file
when that:
- contains #pragma once, or
- starts with #ifndef, or
- starts with #if ! defined.
GCC has a similar trick (*2), but it acts more stricter (e. g. there
must be _no tokens_ outside of #ifndef...#endif).
Sun C lacked #pragma once for a looong time. Oracle Developer Studio
12.5 finally implemented it, but we cannot assume such recent version.
This changeset modifies header files so that each of them include
strictly one #ifndef...#endif. I believe this is the most portable way
to trigger compiler optimizations. [Bug #16770]
*1: https://docs.microsoft.com/en-us/cpp/preprocessor/once
*2: https://gcc.gnu.org/onlinedocs/cppinternals/Guard-Macros.html
Revert "Revert "include/ruby/3/core/rtypeddata.h: Use 0 instead of NULL for C++ compiler""
Revert "include/ruby/3/core/rtypeddata.h: Use 0 instead of NULL for C++ compiler"
Revert "include/ruby/3: Skip RUBY3_UNREACHABLE_RETURN for icc"
Revert "include/ruby/3: Stop RUBY3_UNREACHABLE_RETURN to suppress a SunC warning"
This reverts commit b965e7e05e.
This reverts commit d2bb2e066b.
This reverts commit 8ab4c55e7a.
This reverts commit 78eec3cdc5.
After https://github.com/ruby/ruby/pull/3011, we no longer need these
ifdef guards.
icc warns at this line.
> include/ruby/3/core/rstring.h(126): warning #413: variable "retval" has an uninitialized const field
> struct RString retval;
> ^
This must not be a problem because uninitialized const field is not used
at all. But the warnings are annoying. Let us suppress them.
icc warns side effects for RUBY3_ASSUME like this:
> ./include/ruby/3/value_type.h(202): warning #2261: __assume expression with side effects discarded
> RUBY3_ASSUME(RB_FLONUM_P(obj));
> ^
Which is a false positive (RB_FLONUM_P has no side effect). It seems
there is no way for us to tell icc that a functin is safe inside of
__assume. Just suppress the warning instead.
Without this patch, 20k files are opened (openat syscall) because
of duplicate includes. This patch reduced it to 3k and build time
was reduced compile time of range.o from 15sec -> 3sec on my machine.
[Bug #16772]
This reverts commit d2bb2e066b.
It didn't work:
https://rubyci.org/logs/rubyci.s3.amazonaws.com/icc-x64/ruby-master/log/20200409T050004Z.log.html.gz#end
```
In file included from ../../.././include/ruby/3/core.h(32),
from ../../.././include/ruby/ruby.h(28),
from ../../.././include/ruby.h(33),
from init.c(1):
../../.././include/ruby/3/core/rtypeddata.h(164): warning #967: conversion from "int" to "const struct rb_data_type_struct *"; sizes do not match
RUBY3_UNREACHABLE_RETURN(0);
```
I'm unsure why, but it fails on icc.
https://rubyci.org/logs/rubyci.s3.amazonaws.com/icc-x64/ruby-master/log/20200408T220004Z.log.html.gz
```
In file included from ../../.././include/ruby/3/core.h(32),
from ../../.././include/ruby/ruby.h(28),
from cxxanyargs.cpp(1):
../../.././include/ruby/3/core/rtypeddata.h(163): error: return value type does not match the function type
RUBY3_UNREACHABLE_RETURN(NULL);
```
I don't find how to suppress a false positive "unreachable" warning of
the Sun compiler:
```
"./include/ruby/3/core/rtypeddata.h", line 163: warning: statement not
reached
"./include/ruby/3/memory.h", line 256: warning: statement not reached
```
This warnings make mkmf test fail:
https://rubyci.org/logs/rubyci.s3.amazonaws.com/solaris10-sunc/ruby-master/log/20200408T170004Z.fail.html.gz
```
1) Failure:
TestMkmf::TestConvertible#test_typeof_builtin
[/export/home/users/chkbuild/cb-sunc/tmp/build/20200408T170004Z/ruby/test/mkmf/test_convertible.rb:9]:
convertible_int: checking for convertible type of short...
-------------------- short
--------------------
convertible_int: checking for convertible type of int...
-------------------- int
--------------------
convertible_int: checking for convertible type of long...
-------------------- long
--------------------
convertible_int: checking for convertible type of signed short...
-------------------- failed
...
"cc -I.
-I/export/home/users/chkbuild/cb-sunc/tmp/build/20200408T170004Z/ruby/.ext/include/sparc-solaris2.10
-I/export/home/users/chkbuild/cb-sunc/tmp/build/20200408T170004Z/ruby/include
-I./test -I/opt/csw/include -D_XOPEN_SOURCE=600 -DFFI_NO_RAW_API -g
-c conftest.c"
"/export/home/users/chkbuild/cb-sunc/tmp/build/20200408T170004Z/ruby/include/ruby/3/core/rtypeddata.h",
line 163: warning: statement not reached
"/export/home/users/chkbuild/cb-sunc/tmp/build/20200408T170004Z/ruby/include/ruby/3/memory.h",
line 256: warning: statement not reached
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: typedef signed short rbcv_typedef_;
4:
5: extern rbcv_typedef_ foo();
6: extern short foo();
/* end */
```
RUBY3_CAST is a macro to suppress g++/clang++ warnings about C-style
casts. Though Ruby core don't have to bother C++ situations, extension
libraries can benefit from this.
Commit e91c39f1c0 deleted definition of it.
Though I'm not sure if we can delete public API like this, it no longer
works nontheless. Having declaration without definiton is worse than
having nothing at all. Just delete the declartion too.
This compile-time option has been broken for years (at least since
commit 49369ef173, according to git
bisect). Let's delete codes that no longer works.
Comma at the end of enum is allowed in C since C99. We can use them
internally. However when it comes to extension libraries, they could be
written in different C++ versions. We cannot assume sometihng. Public
headers shall keep compatibilities.
1. By substituting `n_var` with its initializer, `0 < n_var` is
equivalent to `argc > argi + n_trail`.
2. As `argi` is non-negative, so `argi + n_trail >= n_trail`, and
the above expression is equivalent to `argc > n_trail`.
3. Therefore, `f_last` is always false, and `last_hash` is no
longer used.
Creative use of `@copydoc` Doxygen command and abusing its half-broken C
parser let us delete some lines of documentations, while preserving
document coverages.
C++ keyword `nullptr` represents a null pointer (note also that NULL is
an integer in C++ due to its design flaw). Its type is `std::nullptr_t`,
defined in <cstddef> standard header. Why not support it when the
backend implementation can take a null pointer as an argument.
C++ (and myself) hates macros. If we could do the same thing in both
preprocessor and template, we shall choose template. This particular
part of the ruby header is one of such situations.
Due to beae6cbf0f, the variable last_idx
is no longer changed and always -1. This change simplifies the code by
removing the variable. Coverity Scan pointed out this.
This removes the warnings added in 2.7, and changes the behavior
so that a final positional hash is not treated as keywords or
vice-versa.
To handle the arg_setup_block splat case correctly with keyword
arguments, we need to check if we are taking a keyword hash.
That case didn't have a test, but it affects real-world code,
so add a test for it.
This removes rb_empty_keyword_given_p() and related code, as
that is not needed in Ruby 3. The empty keyword case is the
same as the no keyword case in Ruby 3.
This changes rb_scan_args to implement keyword argument
separation for C functions when the : character is used.
For backwards compatibility, it returns a duped hash.
This is a bad idea for performance, but not duping the hash
breaks at least Enumerator::ArithmeticSequence#inspect.
Instead of having RB_PASS_CALLED_KEYWORDS be a number,
simplify the code by just making it be rb_keyword_given_p().
This is tentative. For the sake of simplicity we partially revert
commits e9cb552ec9, ee85a6e72b and 51edb30042. Will decouple them
once again when we are ready.
(old)
test.rb:4: warning: The last argument is used as the keyword parameter
test.rb:1: warning: for `foo' defined here; maybe ** should be added to the call?
(new)
test.rb:4: warning: The last argument is used as keyword parameters; maybe ** should be added to the call
test.rb:1: warning: The called method `foo' is defined here
... like we do so for struct timeval at several hundreds of lines above.
Depending on OS/Compiler, this can be the first place for the struct to
appear. To make sure the struct is global, we need a forward
declaration at this point.
rb_eval_cmd takes a safe level, and now that $SAFE is deprecated,
it should be deprecated as well.
Replace with rb_eval_cmd_kw, which takes a keyword flag. Switch
the two callers to this function.
This removes the related tests, and puts the related specs behind
version guards. This affects all code in lib, including some
libraries that may want to support older versions of Ruby.
This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.
This modifies some internal functions that took a safe level argument
to no longer take the argument.
rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.
One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd. We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.
to suppress the following warning:
```
compiling cxxanyargs.cpp
In file included from cxxanyargs.cpp:1:
In file included from ../../.././include/ruby/ruby.h:2150:
../../.././include/ruby/intern.h:56:19: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
void rb_mem_clear(register VALUE*, register long);
^~~~~~~~~
../../.././include/ruby/intern.h:56:36: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
void rb_mem_clear(register VALUE*, register long);
^~~~~~~~~
```
Currently, there is not a way to create a sized enumerator in C
with a different set of arguments than provided by Ruby, and
correctly handle keyword arguments. This function allows that.
The need for this is fairly uncommon, but it occurs at least in
Enumerator.produce, which takes arugments from Ruby but calls
rb_enumeratorize_with_size with a different set of arguments.
This adds rb_funcall_passing_block_kw, rb_funcallv_public_kw,
and rb_yield_splat_kw. This functions are necessary to easily
handle cases where rb_funcall_passing_block, rb_funcallv_public,
and rb_yield_splat are currently used and a keyword argument
separation warning is raised.
This fixes instance_exec and similar methods. It also fixes
Enumerator::Yielder#yield, rb_yield_block, and a couple of cases
with Proc#{<<,>>}.
This support requires the addition of rb_yield_values_kw, similar to
rb_yield_values2, for passing the keyword flag.
Unlike earlier attempts at this, this does not modify the rb_block_call_func
type or add a separate function type. The functions of type
rb_block_call_func are called by Ruby with a separate VM frame, and we can
get the keyword flag information from the VM frame flags, so it doesn't need
to be passed as a function argument.
These changes require the following VM functions accept a keyword flag:
* vm_yield_with_cref
* vm_yield
* vm_yield_with_block
GCC emits a lot of false positives for rb_scan_args because:
* `rb_scan_args(argc, argv, "*:", NULL, &opts);` makes `n_mand == 0`,
* `n_mand == argc + 1` implies `argc == -1`, and
* `memcpy(ptr, argv, sizeof(VALUE)*argc);` explodes
However, we know that argc is never so big, thus this is a false
positive. This change suppresses it by adding a condition `n_mand > 0`.
```
In file included from /usr/include/string.h:494,
from ./include/ruby/defines.h:145,
from ./include/ruby/ruby.h:29,
from ./include/ruby/encoding.h:27,
from dir.c:14:
In function 'memcpy',
inlined from 'ruby_nonempty_memcpy.part.0' at ./include/ruby/ruby.h:1763:17,
inlined from 'ruby_nonempty_memcpy' at ./include/ruby/ruby.h:1760:1,
inlined from 'rb_scan_args_set' at ./include/ruby/ruby.h:2594:9,
inlined from 'dir_s_aref' at dir.c:2774:12:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:34:10: warning: '__builtin___memcpy_chk' pointer overflow between offset 0 and size [-8, 9223372036854775807] [-Warray-bounds]
return __builtin___memcpy_chk (__dest, __src, __len, __bos0 (__dest));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:34:10: warning:
'__builtin___memcpy_chk' specified size 18446744073709551608 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
```
Cfuncs that use rb_scan_args with the : entry suffer similar keyword
argument separation issues that Ruby methods suffer if the cfuncs
accept optional or variable arguments.
This makes the following changes to : handling.
* Treats as **kw, prompting keyword argument separation warnings
if called with a positional hash.
* Do not look for an option hash if empty keywords are provided.
For backwards compatibility, treat an empty keyword splat as a empty
mandatory positional hash argument, but emit a a warning, as this
behavior will be removed in Ruby 3. The argument number check
needs to be moved lower so it can correctly handle an empty
positional argument being added.
* If the last argument is nil and it is necessary to treat it as an option
hash in order to make sure all arguments are processed, continue to
treat the last argument as the option hash. Emit a warning in this case,
as this behavior will be removed in Ruby 3.
* If splitting the keyword hash into two hashes, issue a warning, as we
will not be splitting hashes in Ruby 3.
* If the keyword argument is required to fill a mandatory positional
argument, continue to do so, but emit a warning as this behavior will
be going away in Ruby 3.
* If keyword arguments are provided and the last argument is not a hash,
that indicates something wrong. This can happen if a cfunc is calling
rb_scan_args multiple times, and providing arguments that were not
passed to it from Ruby. Callers need to switch to the new
rb_scan_args_kw function, which allows passing of whether keywords
were provided.
This commit fixes all warnings caused by the changes above.
It switches some function calls to *_kw versions with appropriate
kw_splat flags. If delegating arguments, RB_PASS_CALLED_KEYWORDS
is used. If creating new arguments, RB_PASS_KEYWORDS is used if
the last argument is a hash to be treated as keywords.
In open_key_args in io.c, use rb_scan_args_kw.
In this case, the arguments provided come from another C
function, not Ruby. The last argument may or may not be a hash,
so we can't set keyword argument mode. However, if it is a
hash, we don't want to warn when treating it as keywords.
In Ruby files, make sure to appropriately use keyword splats
or literal keywords when calling Cfuncs that now issue keyword
argument separation warnings through rb_scan_args. Also, make
sure not to pass nil in place of an option hash.
Work around Kernel#warn warnings due to problems in the Rubygems
override of the method. There is an open pull request to fix
these issues in Rubygems, but part of the Rubygems tests for
their override fail on ruby-head due to rb_scan_args not
recognizing empty keyword splats, which this commit fixes.
Implementation wise, adding rb_scan_args_kw is kind of a pain,
because rb_scan_args takes a variable number of arguments.
In order to not duplicate all the code, the function internals need
to be split into two functions taking a va_list, and to avoid passing
in a ton of arguments, a single struct argument is used to handle
the variables previously local to the function.
卜部昌平
Kenta Murata
Kenta Murata
Koichi Sasada
Hiroshi SHIBATA
Chris Seaton
Samuel Williams
Nobuyoshi Nakada
Yukihiro "Matz" Matsumoto
Kazuhiro NISHIYAMA
AGSaidi
Marc-Andre Lafortune
Jeremy Evans
Yusuke Endoh
Derek Argueta
K.Takata
Aaron Patterson