Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead. This would significantly
speed up incremental builds.
We take the following inclusion order in this changeset:
1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very
first thing among everything).
2. RUBY_EXTCONF_H if any.
3. Standard C headers, sorted alphabetically.
4. Other system headers, maybe guarded by #ifdef
5. Everything else, sorted alphabetically.
Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
Reduce macros to make them inline functions, as well as mark
MJIT_FUNC_EXPORTED functions explicitly as such.
Definition of ar_hint_t is simplified. This has been the only possible
definition so far.
Akatsuki reported ENV['TZ'] = 'UTC' improved 7x-8x faster on following code.
t = Time.now; 100000.times { Time.new(2019) }; Time.now - t
https://hackerslab.aktsk.jp/2019/12/01/141551
commit 4bc1669127(reduce tzset) dramatically improved this situation. But still,
TZ=UTC is faster than default.
This patch removs unnecessary tzset() call completely.
Performance check
----------------------
test program: t = Time.now; 100000.times { Time.new(2019) }; Time.now - t
before: 0.387sec
before(w/ TZ): 0.197sec
after: 0.162sec
after(w/ TZ): 0.165sec
OK. Now, Time creation 2x faster *and* TZ=UTC doesn't improve anything.
We can forget this hack completely. :)
Side note:
This patch slightly changes Time.new(t) behavior implicitly. Before this patch, it might changes
default timezone implicitly. But after this patch, it doesn't. You need to reset TZ
(I mean ENV['TZ'] = nil) explicitly.
But I don't think this is big impact. Don't try to change /etc/localtime on runtime.
Side note2: following test might be useful for testing "ENV['TZ'] = nil".
-----------------------------------------
% cat <<'End' | sudo sh -s
rm -f /etc/localtime-; cp -a /etc/localtime /etc/localtime-
rm /etc/localtime; ln -s /usr/share/zoneinfo/Asia/Tokyo /etc/localtime
./ruby -e '
p Time.new(2000).zone # JST
File.unlink("/etc/localtime"); File.symlink("/usr/share/zoneinfo/America/Los_Angeles", "/etc/localtime")
p Time.new(2000).zone # JST (ruby does not follow /etc/localtime modification automatically)
ENV["TZ"] = nil
p Time.new(2000).zone # PST (ruby detect /etc/localtime modification)
'
rm /etc/localtime; cp -a /etc/localtime- /etc/localtime; rm /etc/localtime-
End
These functions are used from within a compilation unit so we can
make them static, for better binary size. This changeset reduces
the size of generated ruby binary from 26,590,128 bytes to
26,584,472 bytes on my macihne.
This removes the related tests, and puts the related specs behind
version guards. This affects all code in lib, including some
libraries that may want to support older versions of Ruby.
This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.
This modifies some internal functions that took a safe level argument
to no longer take the argument.
rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.
One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd. We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.
Looking at the list of symbols inside of libruby-static.a, I found
hundreds of functions that are defined, but used from nowhere.
There can be reasons for each of them (e.g. some functions are
specific to some platform, some are useful when debugging, etc).
However it seems the functions deleted here exist for no reason.
This changeset reduces the size of ruby binary from 26,671,456
bytes to 26,592,864 bytes on my machine.
This changes object_id from being based on the objects location in
memory (or a nearby memory location in the case of a conflict) to be
based on an always increasing number.
This number is a Ruby Integer which allows it to overflow the size of a
pointer without issue (very unlikely to happen in real programs
especially on 64-bit, but a nice guarantee).
This changes obj_to_id_tbl and id_to_obj_tbl to both be maps of Ruby
objects to Ruby objects (previously they were Ruby object to C integer)
which simplifies updating them after compaction as we can run them
through gc_update_table_refs.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
This changes object_id from being based on the objects location in
memory (or a nearby memory location in the case of a conflict) to be
based on an always increasing number.
This number is a Ruby Integer which allows it to overflow the size of a
pointer without issue (very unlikely to happen in real programs
especially on 64-bit, but a nice guarantee).
This changes obj_to_id_tbl and id_to_obj_tbl to both be maps of Ruby
objects to Ruby objects (previously they were Ruby object to C integer)
which simplifies updating them after compaction as we can run them
through gc_update_table_refs.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
* Stop making a redundant hash copy in Hash#dup
It was making a copy of the hash without rehashing, then created an
extra copy of the hash to do the rehashing. Since rehashing creates
a new copy already, this change just uses that rehashing to make
the copy.
[Bug #16121]
* Remove redundant Check_Type after to_hash
* Fix freeing and clearing destination hash in Hash#initialize_copy
The code was assuming the state of the destination hash based on the
source hash for clearing any existing table on it. If these don't match,
then that can cause the old table to be leaked. This can be seen by
compiling hash.c with `#define HASH_DEBUG 1` and running the following
script, which will crash from a debug assertion.
```ruby
h = 9.times.map { |i| [i, i] }.to_h
h.send(:initialize_copy, {})
```
* Remove dead code paths in rb_hash_initialize_copy
Given that `RHASH_ST_TABLE_P(h)` is defined as `(!RHASH_AR_TABLE_P(h))`
it shouldn't be possible for a hash to be neither of these, so there
is no need for the removed `else if` blocks.
* Share implementation between Hash#replace and Hash#initialize_copy
This also fixes key rehashing for small hashes backed by an array
table for Hash#replace. This used to be done consistently in ruby
2.5.x, but stopped being done for small arrays in ruby 2.6.x.
This also bring optimization improvements that were done for
Hash#initialize_copy to Hash#replace.
* Add the Hash#dup benchmark
This fixes instance_exec and similar methods. It also fixes
Enumerator::Yielder#yield, rb_yield_block, and a couple of cases
with Proc#{<<,>>}.
This support requires the addition of rb_yield_values_kw, similar to
rb_yield_values2, for passing the keyword flag.
Unlike earlier attempts at this, this does not modify the rb_block_call_func
type or add a separate function type. The functions of type
rb_block_call_func are called by Ruby with a separate VM frame, and we can
get the keyword flag information from the VM frame flags, so it doesn't need
to be passed as a function argument.
These changes require the following VM functions accept a keyword flag:
* vm_yield_with_cref
* vm_yield
* vm_yield_with_block
Previously, calling transform_values would call rb_hash_aset for each
key, needing to rehash it and look up its location.
Instead, we can use rb_hash_stlike_foreach_with_replace to replace the
values as we iterate without rehashing the keys.
Treat the ** syntax as passing a copy of the hash as the last
positional argument. If the hash being double splatted is empty, do
not add a positional argument.
Remove rb_no_keyword_hash, no longer needed.
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct. This commit adds function prototypes
for rb_hash_foreach / st_foreach_safe. Also fixes some prototype
mismatches.
"hash_iter_lev" can be exported by Marshal.dump and it will
introduce inconsistency. To avoid this issue, use internal_id
instead of normal ID. This issue is pointed out by Chikanaga-san.
On ar_table, Do not keep a full-length hash value (FLHV, 8 bytes)
but keep a 1 byte hint from a FLHV (lowest byte of FLHV).
An ar_table only contains at least 8 entries, so hints consumes
8 bytes at most. We can store hints in RHash::ar_hint.
On 32bit CPU, we use 4 entries ar_table.
The advantages:
* We don't need to keep FLHV so ar_table only consumes
16 bytes (VALUEs of key and value) * 8 entries = 128 bytes.
* We don't need to scan ar_table, but only need to check hints
in many cases. Especially we don't need to access ar_table
if there is no match entries (in many cases).
It will increase memory cache locality.
The disadvantages:
* This technique can increase `#eql?` time because hints can
conflicts (in theory, it conflicts once in 256 times).
It can introduce incompatibility if there is a object x where
x.eql? returns true even if hash values are different.
I believe we don't need to care such irregular case.
* We need to re-calculate FLHV if we need to switch from ar_table
to st_table (e.g. exceeds 8 entries).
It also can introduce incompatibility, on mutating key objects.
I believe we don't need to care such irregular case too.
Add new debug counters to measure the performance:
* artable_hint_hit - hint is matched and eql?#=>true
* artable_hint_miss - hint is not matched but eql?#=>false
* artable_hint_notfound - lookup counts
iter_lev is used to detect the hash is iterating or not.
Usually, iter_lev should be very small number (1 or 2) so
`int` is overkill.
This patch introduce iter_lev in flags (7 bits, FL13 to FL19)
and if iter_lev exceeds this range, save it in hidden attribute.
We can get 1 word in RHash.
We can't modify frozen objects. Therefore I added new internal API
`rb_ivar_set_internal()` which allows us to set an attribute
even if the target object is frozen
if the name is hidden ivar (the name without `@` prefix).
The behavior of `Hash[[nil]] #=> {}` was a bug until 1.9.3, but had been
remained with a warning because some programs depended upon it.
Now, six years passed. We can remove the compatibility behavior.
[Bug #7300]
For example when an array containing objects is a hash key, the contents
of the array may move which can cause the hash value for the array to
change. This commit makes the default `hash` value based off the
object id, so the hash value will remain stable.
Fixes test/shell/test_command_processor.rb
For some reason symbols (or classes) are being overridden in trunk
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67598 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit adds the new method `GC.compact` and compacting GC support.
Please see this issue for caveats:
https://bugs.ruby-lang.org/issues/15626
[Feature #15626]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because hard to specify commits related to r67479 only.
So please commit again.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit adds the new method `GC.compact` and compacting GC support.
Please see this issue for caveats:
https://bugs.ruby-lang.org/issues/15626
[Feature #15626]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67479 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (RHASH_AR_TABLE_SIZE_DEC): generally, we need to check all
entries to calculate exact "bound" in ar_table, but if size == 0,
we can clear bound because there are no active entries.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66843 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c: ar_table only supports `objhash` so we can call compare/hash
functions directly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66718 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (hash_stlike_lookup): introduce inline a function and use it
instead of using ar_lookup()/st_lookup() directly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66717 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c: the type of `ar_table_entry::hash` is not a `VALUE`,
but a `st_hash_t`.
Also `st_hash_t` is not a `st_data_t`, but `st_index_t` (same as st.c).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66700 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* internal.h: move ar_table def to hash.c because other files
don't need to know implementation of ar_table.
* hash.c (rb_hash_ar_table_size): added because gc.c needs to know
the size_of(ar_table).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c: separate NULL and EMPTY check functions.
`RHASH_TABLE_EMPTY` function checks NULL table or not,
but it should be named "NULL_P".
Introduce `RHASH_TABLE_EMPTY_P` function to check size == 0.
There are cases that hash has table data even if data is not NULL
(in case removed after inserted elements).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66392 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* internal.h: rename the following names:
* li_table -> ar_table. "li" means linear (from linear search),
but we use the word "array" (from data layout).
* RHASH_ARRAY -> RHASH_AR_TABLE. AR_TABLE is more clear.
* rb_hash_array_* -> rb_hash_ar_table_*.
* RHASH_TABLE_P() -> RHASH_ST_TABLE_P(). more clear.
* RHASH_CLEAR() -> RHASH_ST_CLEAR().
* hash.c: rename "linear_" prefix functions to "ar_" prefix.
* hash.c (linear_init_table): rename to ar_alloc_table.
* debug_counter.h: rename obj_hash_array to obj_hash_ar.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66390 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (linear_update): initialize `bin` just to silence false
warnings by old gcc 4.8. [Bug #15299]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66306 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The reserved hash values in hash.c must be consistend with st.c.
[ruby-core:90356] [Bug #15389]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66274 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* include/ruby/ruby.h: de-transient at
`RARRAY_PTR_USE` and `RARRAY_PTR_USE_START`.
Introduce `RARRAY_PTR_USE_TRANSIENT` and
`RARRAY_PTR_USE_START_TRANSIENT` if you don't want to
de-transient an array. Generally, it is difficult
so C-extension writers should not use them.
* array.c: use `RARRAY_PTR_USE_TRANSIENT` if possible.
* hash.c: ditto.
* enum.c (enum_sort_by): remove `rb_ary_transient_heap_evacuate()`
because `RARRAY_PTR_USE` do de-transient.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66165 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (linear_copy): solve two issues on `Hash#replace`.
(1) fix memory leak
(1-1) don't allocate memory if destination already
has a memory area.
(1-2) free destination memory if src is NULL.
(2) clear transient heap flag if src is NULL. [Bug #15358]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When a negative double is casted into an unsigned type, that operation
is undefined (cf: ISO/IEC 9899:1990 section 6.2.9.3). Recent versions
of C kindly footnotes that "The remaindering operation performed when
a value of integer type is converted to unsigned type need not be
performed when a value of real floating type is converted to unsigned
type" (cf: ISO/IEC 9899:1999 section 6.3.1.4 footnote 50).
So it is a wrong idea to just cast a double to st_data_t.
The intention of the code is commented as "mix the actual float value
in". It seems we should do a reinterpret_cast and rule out
static_cast.
Confirmed this changeset does not affect `make benchmark`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65737 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Before this changeset RHASH_ARRAY_SIZE_DEC() was expaneded to include
an expression like `RHASH_ARRAY_SIZE+(-1)`. RHASH_ARRAY_SIZE is by
definition unsigned int. -1 is signed, of course. Adding a signed
and an unsigned value requires the "usual arithmetic conversions" (cf:
ISO/IEC 9899:1990 section 6.2.1.5). -1 is converted to 0xFFFF by that.
This patch prevents that conversion.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65632 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (RHASH_ARRAY_BOUND_RAW): should be unsigned as well as
RHASH_ARRAY_SIZE_RAW.
* hash.c (find_entry): return unsigned for the consistency with
RHASH_ARRAY_SIZE and RHASH_ARRAY_BOUND.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65513 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (linear_copy): remember a hash object to mark pointing objects.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65479 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c: remove '//' style comments pointed out by the following
build log: https://travis-ci.org/ruby/ruby/jobs/448551951
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65464 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c, internal.h: support theap for small Hash.
Introduce RHASH_ARRAY (li_table) besides st_table and small Hash
(<=8 entries) are managed by an array data structure.
This array data can be managed by theap.
If st_table is needed, then converting array data to st_table data.
For st_table using code, we prepare "stlike" APIs which accepts hash value
and are very similar to st_ APIs.
This work is based on the GSoC achievement
by tacinight <tacingiht@gmail.com> and refined by ko1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65454 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We revisit [Bug #9188] since st.c is much improved since then,
and benchmarks against so_k_nucleotide seem to indicate little
or no performance change compared to before.
[ruby-core:89555] [Feature #15251]
From: Anmol Chopra <chopraanmol1@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65371 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c: [DOC] improve docs for Hash#{merge,merge!,update}:
various rewordings, avoid referring to the receiver as `hsh'
(does not appear in the call-seq of the generated HTML docs),
mention that Hash#update is an alias for Hash#merge!,
use more distinct example values, fix spacing in code.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65069 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c: [DOC] shorten example code for Hash#{size,length}
and mention aliases at the end; also enable links.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65036 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (env_enc_str_new): as no locale/filesystem encoding is
available in miniruby on Windows, fallback the encoding to
ASCII-8BIT so it is valid encoding when the conversion failed.
[ruby-core:89177] [Bug #15164]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64860 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Hash#merge, merge!, and update could merge exactly two hashes.
Now, they accepts zero or more hashes as arguments so that it can merge
hashes more than two.
This patch was created by Koki Ryu <liukoki@gmail.com> at Ruby Hack
Challenge #5. Thank you!
[ruby-core:88970] [Feature #15111] [Fix GH-1951]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64777 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (ruby_setenv): do not check environment block size.
c.f. https://msdn.microsoft.com/en-us/library/windows/desktop/ms682653(v=vs.85).aspx
Starting with Windows Vista and Windows Server 2008, there is no
technical limitation on the size of the environment block.
[ruby-core:88400] [Bug #14979]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64293 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Both methods Hash#length and Hash#size share the same source code in
Ruby, but they also share the same documentation. Now when you look at
the documentation of Hash#size you only see examples for Hash#length,
which is confusing. This commit includes Hash#size in the examples and
also remarks that both methods are equivalent to each other.
Co-authored-by: Alberto Almagro <alberto.almagro@rakuten.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64081 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* time.c (rb_localtime_r): call tzset() only after TZ environment
variable is changed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63994 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* In Enumerable, Enumerator::Lazy, Array, Hash and Set
[Feature #13784] [ruby-core:82285]
* Share specs for the various #select#select! methods and
reuse them for #filter/#filter!.
* Add corresponding filter tests for select tests.
* Update NEWS.
[Fix GH-1824]
From: Alexander Patrick <adp90@case.edu>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62575 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
which has been developed by Takashi Kokubun <takashikkbn@gmail> as
YARV-MJIT. Many of its bugs are fixed by wanabe <s.wanabe@gmail.com>.
This JIT compiler is designed to be a safe migration path to introduce
JIT compiler to MRI. So this commit does not include any bytecode
changes or dynamic instruction modifications, which are done in original
MJIT.
This commit even strips off some aggressive optimizations from
YARV-MJIT, and thus it's slower than YARV-MJIT too. But it's still
fairly faster than Ruby 2.5 in some benchmarks (attached below).
Note that this JIT compiler passes `make test`, `make test-all`, `make
test-spec` without JIT, and even with JIT. Not only it's perfectly safe
with JIT disabled because it does not replace VM instructions unlike
MJIT, but also with JIT enabled it stably runs Ruby applications
including Rails applications.
I'm expecting this version as just "initial" JIT compiler. I have many
optimization ideas which are skipped for initial merging, and you may
easily replace this JIT compiler with a faster one by just replacing
mjit_compile.c. `mjit_compile` interface is designed for the purpose.
common.mk: update dependencies for mjit_compile.c.
internal.h: declare `rb_vm_insn_addr2insn` for MJIT.
vm.c: exclude some definitions if `-DMJIT_HEADER` is provided to
compiler. This avoids to include some functions which take a long time
to compile, e.g. vm_exec_core. Some of the purpose is achieved in
transform_mjit_header.rb (see `IGNORED_FUNCTIONS`) but others are
manually resolved for now. Load mjit_helper.h for MJIT header.
mjit_helper.h: New. This is a file used only by JIT-ed code. I'll
refactor `mjit_call_cfunc` later.
vm_eval.c: add some #ifdef switches to skip compiling some functions
like Init_vm_eval.
win32/mkexports.rb: export thread/ec functions, which are used by MJIT.
include/ruby/defines.h: add MJIT_FUNC_EXPORTED macro alis to clarify
that a function is exported only for MJIT.
array.c: export a function used by MJIT.
bignum.c: ditto.
class.c: ditto.
compile.c: ditto.
error.c: ditto.
gc.c: ditto.
hash.c: ditto.
iseq.c: ditto.
numeric.c: ditto.
object.c: ditto.
proc.c: ditto.
re.c: ditto.
st.c: ditto.
string.c: ditto.
thread.c: ditto.
variable.c: ditto.
vm_backtrace.c: ditto.
vm_insnhelper.c: ditto.
vm_method.c: ditto.
I would like to improve maintainability of function exports, but I
believe this way is acceptable as initial merging if we clarify the
new exports are for MJIT (so that we can use them as TODO list to fix)
and add unit tests to detect unresolved symbols.
I'll add unit tests of JIT compilations in succeeding commits.
Author: Takashi Kokubun <takashikkbn@gmail.com>
Contributor: wanabe <s.wanabe@gmail.com>
Part of [Feature #14235]
---
* Known issues
* Code generated by gcc is faster than clang. The benchmark may be worse
in macOS. Following benchmark result is provided by gcc w/ Linux.
* Performance is decreased when Google Chrome is running
* JIT can work on MinGW, but it doesn't improve performance at least
in short running benchmark.
* Currently it doesn't perform well with Rails. We'll try to fix this
before release.
---
* Benchmark reslts
Benchmarked with:
Intel 4.0GHz i7-4790K with 16GB memory under x86-64 Ubuntu 8 Cores
- 2.0.0-p0: Ruby 2.0.0-p0
- r62186: Ruby trunk (early 2.6.0), before MJIT changes
- JIT off: On this commit, but without `--jit` option
- JIT on: On this commit, and with `--jit` option
** Optcarrot fps
Benchmark: https://github.com/mame/optcarrot
| |2.0.0-p0 |r62186 |JIT off |JIT on |
|:--------|:--------|:--------|:--------|:--------|
|fps |37.32 |51.46 |51.31 |58.88 |
|vs 2.0.0 |1.00x |1.38x |1.37x |1.58x |
** MJIT benchmarks
Benchmark: https://github.com/benchmark-driver/mjit-benchmarks
(Original: https://github.com/vnmakarov/ruby/tree/rtl_mjit_branch/MJIT-benchmarks)
| |2.0.0-p0 |r62186 |JIT off |JIT on |
|:----------|:--------|:--------|:--------|:--------|
|aread |1.00 |1.09 |1.07 |2.19 |
|aref |1.00 |1.13 |1.11 |2.22 |
|aset |1.00 |1.50 |1.45 |2.64 |
|awrite |1.00 |1.17 |1.13 |2.20 |
|call |1.00 |1.29 |1.26 |2.02 |
|const2 |1.00 |1.10 |1.10 |2.19 |
|const |1.00 |1.11 |1.10 |2.19 |
|fannk |1.00 |1.04 |1.02 |1.00 |
|fib |1.00 |1.32 |1.31 |1.84 |
|ivread |1.00 |1.13 |1.12 |2.43 |
|ivwrite |1.00 |1.23 |1.21 |2.40 |
|mandelbrot |1.00 |1.13 |1.16 |1.28 |
|meteor |1.00 |2.97 |2.92 |3.17 |
|nbody |1.00 |1.17 |1.15 |1.49 |
|nest-ntimes|1.00 |1.22 |1.20 |1.39 |
|nest-while |1.00 |1.10 |1.10 |1.37 |
|norm |1.00 |1.18 |1.16 |1.24 |
|nsvb |1.00 |1.16 |1.16 |1.17 |
|red-black |1.00 |1.02 |0.99 |1.12 |
|sieve |1.00 |1.30 |1.28 |1.62 |
|trees |1.00 |1.14 |1.13 |1.19 |
|while |1.00 |1.12 |1.11 |2.41 |
** Discourse's script/bench.rb
Benchmark: https://github.com/discourse/discourse/blob/v1.8.7/script/bench.rb
NOTE: Rails performance was somehow a little degraded with JIT for now.
We should fix this.
(At least I know opt_aref is performing badly in JIT and I have an idea
to fix it. Please wait for the fix.)
*** JIT off
Your Results: (note for timings- percentile is first, duration is second in millisecs)
categories_admin:
50: 17
75: 18
90: 22
99: 29
home_admin:
50: 21
75: 21
90: 27
99: 40
topic_admin:
50: 17
75: 18
90: 22
99: 32
categories:
50: 35
75: 41
90: 43
99: 77
home:
50: 39
75: 46
90: 49
99: 95
topic:
50: 46
75: 52
90: 56
99: 101
*** JIT on
Your Results: (note for timings- percentile is first, duration is second in millisecs)
categories_admin:
50: 19
75: 21
90: 25
99: 33
home_admin:
50: 24
75: 26
90: 30
99: 35
topic_admin:
50: 19
75: 20
90: 25
99: 30
categories:
50: 40
75: 44
90: 48
99: 76
home:
50: 42
75: 48
90: 51
99: 89
topic:
50: 49
75: 55
90: 58
99: 99
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (prime2): turned into a uint32_t prime, as the lower
32bits, non-prime part only was used always.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61473 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Current check for __uint128_t in hash.c is not valid, since it ignores
compilers other than gcc. We hit this on lcc on e2k platform.
Configure script properly checks from 128-bit data types support and
sets HAVE_UINT128_T accordingly. This approach is already used within
ruby at bignum.c, random.c, etc.
Probably hash.c is an overlooked remnant of old days. This patch fixes
this.
[ruby-core:84438] [Bug #14231] [Fix GH-1781]
From: Andrew Savchenko <bircoph@altlinux.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61471 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and specialized Array#any? and Hash#any?
Based on patch by D.E. Akers [#11286]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61098 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Traditionally, method coverage measurement was implemented by inserting
`trace2` instruction to the head of method iseq. So, it just measured
methods defined by `def` keyword.
This commit drastically changes the measuring mechanism of method
coverage; at `RUBY_EVENT_CALL`, it keeps a hash from rb_method_entry_t*
to runs (i.e., it counts the runs per method entry), and at
`Coverage.result`, it creates the result hash by enumerating all
`rb_method_entry_t*` objects (by `ObjectSpace.each_object`).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61023 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c: [DOC] clarify description for Hash#slice and remove
a sentence that might suggest that the receiver is modified;
improve example to also include a case where a hash with
several elements is returned.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60713 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* array.c (rb_to_array_type): make public to share common code
internally.
* hash.c (rb_to_hash_type): make public to share common code
internally.
* symbol.c (rb_to_symbol_type): make public to share common code
internally.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60438 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
hash.c (rb_hash_compare_by_id): avoid unnecessary allocation of st_table.
formerly, st_table created in rb_hash_modify() was not used and replaced immediately.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60337 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The new function rb_yield_assoc_or_values() will reduce branching.
* vm_eval.c: add rb_yield_assoc_or_values()
* internal.h: ditto
* hash.c: use rb_yield_assoc_or_values()
* struct.c: ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60095 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
st.c was improved in r56650 that it permits deletion during iteration.
In this commit, special treatments for previous implementation are
removed.
* hash.c: don't use *_check and *_safe functions in st.c
* internal.h: remove HASH_DELETED flag
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60076 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Sometimes, size of a hash can be calcluated a priori. By providing
such info to the constructor we can avoid unnecessary internal re-
allocations. This can boost for instance creation of hash literals.
[Bug #13861]
Signed-off-by: Urabe, Shyouhei <shyouhei@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59744 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (rb_hash_compact_bang): [DOC] update the case if no
changes were made. [ruby-core:82591] [Bug #13855] [Fix GH-1692]
Author: Lucas Buchala <lucasbuchala@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59719 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (env_enc_str_new): convert to the expected encoding
without intermediate string, and set econv flags if default
internal encoding is set too.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59449 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
These caused numerous CI failures I haven't been able to
reproduce [ruby-core:82102]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59364 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The same hash keys may be loaded from tainted data sources
frequently (e.g. parsing headers from socket or loading
YAML data from a file). If a non-tainted fstring already
exists (because the application expects the hash key),
cache and deduplicate the tainted version in the new
tainted_frozen_strings table.
For non-embedded strings, this also allows sharing with the
underlying malloc-ed data.
* vm_core.h (rb_vm_struct): add tainted_frozen_strings
* vm.c (ruby_vm_destruct): free tainted_frozen_strings
(Init_vm_objects): initialize tainted_frozen_strings
(rb_vm_tfstring_table): accessor for tainted_frozen_strings
* internal.h: declare rb_fstring_existing, rb_vm_tfstring_table
* hash.c (fstring_existing_str): remove (moved to string.c)
(hash_aset_str): use rb_fstring_existing
* string.c (rb_fstring_existing): new, based on fstring_existing_str
(tainted_fstr_update): new
(rb_fstring_existing0): new, based on fstring_existing_str
(rb_tainted_fstring_existing): new, special case for tainted strings
(rb_str_free): delete from tainted_frozen_strings table
* test/ruby/test_optimization.rb (test_hash_reuse_fstring): new test
[ruby-core:82012] [Bug #13737]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Fix up r59328. It is possible that the given block abuses
ObjectSpace.each_object to shrink the temporary array.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59334 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (hash_aset_str): create frozen string for tainted objects.
(should not use fsting table on this case).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59310 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
In typical applications, hash entries are read after being
written to. Blindly writing to hashes which are never read
makes little sense. So, for any hash which is read from, an
fstring entry for the key should already exist for the key.
We no longer blindly create fstrings if the code is blindly
setting random hash keys, preventing the performance regression
in the reverted r43870.
Regarding <https://bugs.ruby-lang.org/issues/9188>, this has a
minimum impact on the bm_so_k_nucleotide where hash keys are set
and not reused, performance is within 1-2% of existing cases.
* hash.c: #include gc.h for rb_objspace_garbage_object_p
(hash_aset_str): do read-only check of fstring table and
reuse fstring if it exists and is still alive (not garbage)
[ruby-core:81942] [Feature #13725]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59304 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
To convert the object implicitly, it has had two parts in convert_type() which are
1. lookink up the method's id
2. calling the method
Seems that strncmp() and strcmp() in convert_type() are slightly heavy to look up
the method's id for type conversion.
This patch will add and use internal APIs (rb_convert_type_with_id, rb_check_convert_type_with_id)
to call the method without looking up the method's id when convert the object.
Array#flatten -> 19 % up
Array#+ -> 3 % up
[ruby-dev:50024] [Bug #13341] [Fix GH-1537]
### Before
Array#flatten 104.119k (± 1.1%) i/s - 525.690k in 5.049517s
Array#+ 1.993M (± 1.8%) i/s - 10.010M in 5.024258s
### After
Array#flatten 124.005k (± 1.0%) i/s - 624.240k in 5.034477s
Array#+ 2.058M (± 4.8%) i/s - 10.302M in 5.019328s
### Test Code
require 'benchmark/ips'
class Foo
def to_ary
[1,2,3]
end
end
Benchmark.ips do |x|
ary = []
100.times { |i| ary << i }
array = [ary]
x.report "Array#flatten" do |i|
i.times { array.flatten }
end
x.report "Array#+" do |i|
obj = Foo.new
i.times { array + obj }
end
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c: [DOC] fix return value in call-seq of Hash#transform_values;
other small fixes.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58888 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Hash#transform_values! returns the receiver rather than a new Hash
object.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58845 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Explicitly says that the methods return a new hash rather than just
stating it return a new something we don't know.
[ci skip]
[Fix GH-1619]
Author: Nicolas Cavigneaux <nico@bounga.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58830 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (rb_hash_merge): use rb_hash_dup() instead of rb_obj_dup() to duplicate
Hash object. rb_hash_dup() is faster duplicating function for Hash object
which got rid of Hash#initialize_dup method calling.
Hash#merge will be faster around 60%.
[ruby-dev:50026] [Bug #13343] [Fix GH-1533]
### Before
user system total real
Hash#merge 0.160000 0.020000 0.180000 ( 0.182357)
### After
user system total real
Hash#merge 0.110000 0.010000 0.120000 ( 0.114404)
### Test code
require 'benchmark'
Benchmark.bmbm do |x|
hash1 = {}
100.times { |i| hash1[i.to_s] = i }
hash2 = {}
100.times { |i| hash2[(i*2).to_s] = i*2 }
x.report "Hash#merge" do
10000.times do
hash1.merge(hash2)
end
end
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58811 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* st.c (rb_hash_bulk_insert): new API to bulk insert entries
into a hash. Given arguments are first inserted into the
table at once, then reindexed. This is faster than inserting
things using rb_hash_aset() one by one.
This arrangement (rb_ prefixed function placed in st.c) is
unavoidable because it both touches table internal and write
barrier at once.
* internal.h: delcare the new function.
* hash.c (rb_hash_s_create): use the new function.
* vm.c (core_hash_merge): ditto.
* insns.def (newhash): ditto.
* test/ruby/test_hash.rb: more coverage on hash creation.
* test/ruby/test_literal.rb: ditto.
-----------------------------------------------------------
benchmark results:
minimum results in each 7 measurements.
Execution time (sec)
name before after
loop_whileloop2 0.136 0.137
vm2_bighash* 1.249 0.623
Speedup ratio: compare with the result of `before' (greater is better)
name after
loop_whileloop2 0.996
vm2_bighash* 2.004
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58492 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
We need to fix GC bug before merging this. Revert revisions
58452, 58435, 58434, 58428, 58427 in this order.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58463 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (rb_hash_new_from_object): same as r58434.
Newly created frozen objects are not referred from any roots/objects.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58452 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (rb_hash_new_from_values_with_klass): before this fix,
only a st table are filled with passed values. However, newly
created frozen strings are not marked correctly only reference
from st table. This patch marks such created frozen strings
by Hash object which refers to the st table.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58434 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Same as rb_ary_tmp_new_from_values(), it reduces vm_exec_core binary
size from 26,176 bytes to 26,080 bytes. But this time, also with a
bit of optimizations:
- Because we are allocating a new hash and no back references are
introduced at all, we can safely skip write barriers.
- Also, the iteration never recurs. We can avoid complicated
function callbacks by using st_insert instead of st_update.
----
* hash.c (rb_hash_new_from_values): refactor
extract the bulk insert into a function.
* hash.c (rb_hash_new_from_object): also refactor.
* hash.c (rb_hash_s_create): use the new functions.
* insns.def (newhash): ditto.
* vm.c (core_hash_from_ary): ditto.
* iternal.h: export the new function.
-----------------------------------------------------------
benchmark results:
minimum results in each 7 measurements.
Execution time (sec)
name before after
loop_whileloop2 0.135 0.134
vm2_bighash* 1.236 0.687
Speedup ratio: compare with the result of `before' (greater is better)
name after
loop_whileloop2 1.008
vm2_bighash* 1.798
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58427 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Patch by Eric Wong [ruby-core:78797].
I don't like the idea of making insns.def any bigger to support
a corner case, and "test_hash_aref_fstring_identity" shows
how contrived this is.
[ruby-core:78783] [Bug #12855]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57278 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (prime1, prime2): split long long literals for platforms
where LL suffix is not supported, e.g., VC6.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (any_hash): should return `long', because ruby assumes
the hash value of the object id of an object is `long'.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57012 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* st.h (struct st_hash_type): Remove strong_hash.
(struct st_table): Remove inside_rebuild_p and curr_hash.
* st.c (do_hash): Use type->hash instead of curr_hash.
(make_tab_empty): Remove setting up curr_hash.
(st_init_table_with_size): Remove setting up inside_rebuild_p.
(rebuild_table): Remove clearing inside_rebuild_p.
(reset_entry_hashes, HIT_THRESHOULD_FOR_STRONG_HASH): Remove code
recognizing a denial attack and switching to strong hash.
* hash.c (rb_dbl_long_hash, rb_objid_hash, rb_ident_hash): Use
rb_hash_start to randomize the hash.
(str_seed): Remove.
(any_hash): Remove strong_p and use always rb_str_hash for
strings.
(any_hash_weak, rb_any_hash_weak): Remove.
(st_hash_type objhash): Remove rb_any_hash_weak.
based on the patch by Vladimir N Makarov <vmakarov@redhat.com> at
[ruby-core:78490]. [Bug #13002]
* test/ruby/test_hash.rb (test_wrapper): objects other than special
constants should be able to be wrapped.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56992 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Always add a space between a comma and the next element. These spaces
were there sometimes, but not always. This keeps to code consistent.
Patch by: Herwin Weststrate <herwin@snt.utwente.nl>
[ruby-core:78297] [Misc #12977] [GH-1492]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56971 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
[Feature #12142]
See header of st.c for improvment details.
You can see all of code history here:
<https://github.com/vnmakarov/ruby/tree/hash_tables_with_open_addressing>
This improvement is discussed at
<https://bugs.ruby-lang.org/issues/12142>
with many people, especially with Yura Sokolov.
* st.c: improve st_table.
* include/ruby/st.h: ditto.
* internal.h, numeric.c, hash.c (rb_dbl_long_hash): extract a function.
* ext/-test-/st/foreach/foreach.c: catch up this change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56650 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (rb_hash_compact_bang): should return nil if no elements
is deleted. [ruby-core:77709] [Bug #12863]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56473 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (rb_hash_compact, rb_hash_compact_bang): Removes nil
values from the original hash, to port Active Support behavior.
[Feature #11818]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56414 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
a hash value of Object might be Bignum, but it causes many troubles
expecially the Object is used as a key of a hash. so I've gave up
to do so.
* array.c (rb_ary_hash): use above macro.
* bignum.c (rb_big_hash): ditto.
* hash.c (rb_obj_hash, rb_hash_hash): ditto.
* numeric.c (rb_dbl_hash): ditto.
* proc.c (proc_hash): ditto.
* re.c (rb_reg_hash, match_hash): ditto.
* string.c (rb_str_hash_m): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56340 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (env_enc_str_new): make string for an environment
variable name or value.
* hash.c (env_name_new): make environment value string with the
encoding for its name.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55819 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (w32_getenv): call rb_w32_getenv and rb_w32_ugetenv via
this pointer without further comparisons.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55813 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (env_assoc): the encoding of the value should be the
locale, as well as other methods, [], fetch, values, etc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55811 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* *.c: rename rb_funcall2 to rb_funcallv, except for extensions
which are/will be/may be gems. [Fix GH-1406]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55773 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Lourens Naudé
Koichi Sasada
Yusuke Endoh
卜部昌平
Nobuyoshi Nakada
BurdetteLamar
KOSAKI Motohiro
Kazuhiro NISHIYAMA
Jeremy Evans
John Hawthorn
Aaron Patterson
Dylan Thacker-Smith
Daniel Radetsky
git
Koichi Sasada
Kenichi Kamiya
k0kubun
tenderlove
ktsj
kazu
nobu
ko1
shyouhei
stomar
normal
mame
eregon
mrkn
usa
marcandre
glass
hsbt
rhe
watson1978
naruse
ngoto
akr