Граф коммитов

1539 Коммитов

Автор SHA1 Сообщение Дата
shyouhei d154bec0d5 setbyte / ungetbyte allow out-of-range integers
* string.c: String#setbyte to accept arbitrary integers [Bug #15460]

* io.c: ditto for IO#ungetbyte

* ext/strringio/stringio.c: ditto for StringIO#ungetbyte



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66824 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-15 06:41:58 +00:00
nobu 50784a0a44 Defer escaping control char in error messages
* eval_error.c (print_errinfo): defer escaping control char in
  error messages until writing to stderr, instead of quoting at
  building the message.  [ruby-core:90853] [Bug #15497]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66753 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-08 09:08:31 +00:00
mame 94bdc4edf0 string.c: remove the deprecation warnings of `String#bytes` with block
And its friends: lines, chars, grapheme_clusters, and codepoints.
[Feature #6670] [ruby-core:90728]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66579 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-26 14:43:25 +00:00
mame 0df1de8b32 Revert "string.c: remove the deprecation warnings of `String#bytes` with block"
Forgot to write the ticket number in the commit log...

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66578 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-26 14:42:07 +00:00
mame 2b21744efa string.c: remove the deprecation warnings of `String#bytes` with block
And its friends: lines, chars, grapheme_clusters, and codepoints.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66575 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-26 08:52:19 +00:00
stomar 31dc65b275 string.c: [DOC] fix typos
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66375 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-12 22:04:48 +00:00
duerst 3628eae2e7 implement special behavior for Georgian for String#capitalize
The modern Georgian script is special in that it has an 'uppercase'
variant called MTAVRULI which can be used for emphasis of whole words,
for screamy headlines, and so on. However, in contrast to all other
bicameral scripts, there is no usage of capitalizing the first letter
in a word or a sentence. Words with mixed capitalization are not used
at all.

We therefore implement special behavior for String#capitalize. Formally,
we define String#capitalize as first applying String#downcase for the
whole string, then using titlecase on the first letter. Because Georgian
defines titlecase as the identity function both for MTAVRULI ('uppercase')
and Mkhedruli (lowercase), this results in String#capitalize being
equivalent to String#downcase for Georgian. This avoids undesirable
mixed case.

* enc/unicode.c: Actual implementation

* string.c: Add mention of this special case for documentation

* test/ruby/enc/test_case_mapping.rb: Add two tests, a general one
  that uses String#capitalize on some (including nonsensical)
  combinations of MTAVRULI and Mkhedruli, and a canary test to
  detect the potential assignment of characters to the currently
  open slots (holes) at U+1CBB and U+1CBC.

* test/ruby/enc/test_case_comprehensive.rb: Tweak generation of
  expectation data.

Together with r65933, this closes issue #14839.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66300 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-09 23:14:29 +00:00
naruse e39a83a150 suppress warning: unused variable 'vbits'
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66245 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-06 10:42:35 +00:00
nobu 98e65d9d92 Prefer rb_check_arity when 0 or 1 arguments
Especially over checking argc then calling rb_scan_args just to
raise an ArgumentError.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66238 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-06 07:49:24 +00:00
shyouhei 37c22bd945 string.c: [DOC] deprecate String#crypt [ci skip] [Feature #14915]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66154 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 05:46:46 +00:00
svn 64148e66b6 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65957 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 12:26:11 +00:00
naruse 1f9731654e fix r65954; Keep tainty
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65956 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 12:26:07 +00:00
naruse 7850586af4 Don't use single byte optimization on grapheme clusters
Unicode Text Segmentation considers CRLF as a character. [Bug #15337]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65954 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 11:53:19 +00:00
shyouhei 953091a4b1 char is not unsigned
It seems that decades ago, ruby was written under assumption that
char is unsigned.  Which is of course a false assumption.  We
need to explicitly store a numeric value into an unsigned char
variable to tell we expect 0..255 value.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65900 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-21 08:51:39 +00:00
shyouhei 7213568733 string.c: setbyte silently ignores upper bits
The behaviour of String#setbyte has been depending on the width
of int, which is not portable.  Must check explicitly.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65804 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-19 09:52:46 +00:00
shyouhei 74fe1cc3d9 string.c: this assumption is false [ci skip]
Looking at the lines right above, it is clear than a blue sky
that we cannot assume `p` to be aligned at all when
UNALIGNED_WORD_ACCESS is true.  It is a wrong idea to use
__builtin_assume_aligned for that situation.

See also: https://travis-ci.org/ruby/ruby/jobs/451710732#L2007


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65592 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-07 05:23:03 +00:00
shyouhei 4a80c0540f adopt sanitizer API
These APIs are much like <valgrind/memcheck.h>. Use them to
fine-grain annotate the usage of our memory.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65573 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-06 10:06:07 +00:00
ko1 870363886f fix type.
* string.c (rb_str_format_m): should pass `int`.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65456 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 22:16:26 +00:00
ko1 312b105d0e introduce TransientHeap. [Bug #14858]
* transient_heap.c, transient_heap.h: implement TransientHeap (theap).
  theap is designed for Ruby's object system. theap is like Eden heap
  on generational GC terminology. theap allocation is very fast because
  it only needs to bump up pointer and deallocation is also fast because
  we don't do anything. However we need to evacuate (Copy GC terminology)
  if theap memory is long-lived. Evacuation logic is needed for each type.

  See [Bug #14858] for details.

* array.c: Now, theap for T_ARRAY is supported.

  ary_heap_alloc() tries to allocate memory area from theap. If this trial
  sccesses, this array has theap ptr and RARRAY_TRANSIENT_FLAG is turned on.
  We don't need to free theap ptr.

* ruby.h: RARRAY_CONST_PTR() returns malloc'ed memory area. It menas that
  if ary is allocated at theap, force evacuation to malloc'ed memory.
  It makes programs slow, but very compatible with current code because
  theap memory can be evacuated (theap memory will be recycled).

  If you want to get transient heap ptr, use RARRAY_CONST_PTR_TRANSIENT()
  instead of RARRAY_CONST_PTR(). If you can't understand when evacuation
  will occur, use RARRAY_CONST_PTR().

(re-commit of r65444)


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65449 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 21:53:56 +00:00
svn 69b8ffcd5b * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65448 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 21:02:12 +00:00
ko1 7d359f9b69 revert r65444 and r65446 because of commit miss
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 21:01:55 +00:00
ko1 90ac549fa6 introduce TransientHeap. [Bug #14858]
* transient_heap.c, transient_heap.h: implement TransientHeap (theap).
  theap is designed for Ruby's object system. theap is like Eden heap
  on generational GC terminology. theap allocation is very fast because
  it only needs to bump up pointer and deallocation is also fast because
  we don't do anything. However we need to evacuate (Copy GC terminology)
  if theap memory is long-lived. Evacuation logic is needed for each type.

  See [Bug #14858] for details.

* array.c: Now, theap for T_ARRAY is supported.

  ary_heap_alloc() tries to allocate memory area from theap. If this trial
  sccesses, this array has theap ptr and RARRAY_TRANSIENT_FLAG is turned on.
  We don't need to free theap ptr.

* ruby.h: RARRAY_CONST_PTR() returns malloc'ed memory area. It menas that
  if ary is allocated at theap, force evacuation to malloc'ed memory.
  It makes programs slow, but very compatible with current code because
  theap memory can be evacuated (theap memory will be recycled).

  If you want to get transient heap ptr, use RARRAY_CONST_PTR_TRANSIENT()
  instead of RARRAY_CONST_PTR(). If you can't understand when evacuation
  will occur, use RARRAY_CONST_PTR().


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65444 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 20:46:24 +00:00
stomar 8f0eb44d93 string.c: improve docs for String#strip and related
* string.c: [DOC] improve docs for String#{strip,lstrip,rstrip}{,!}:
  small clarification, avoid referring to the receiver as `str'
  (does not appear in the call-seq of the generated HTML docs),
  enable links for cross-references, simplify rdoc.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-26 20:30:09 +00:00
stomar af7f9de4b9 array.c, file.c, string.c: [DOC] fix typos
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65185 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-19 21:35:51 +00:00
nobu 569fe2922f string.c: grapheme cluster regexp failure
* string.c (get_reg_grapheme_cluster): show error info and relax
  to rb_fatal from rb_bug.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-16 09:11:12 +00:00
stomar 41a486e966 string.c: [DOC] add example code for String#strip!
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65068 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-13 19:04:02 +00:00
stomar ee49d04540 string.c: small doc improvement
* string.c: [DOC] move unaltered case for String#strip to the end,
  similar to other strip methods.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65067 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-13 19:02:51 +00:00
nobu fa8b08b424 Prefer `rb_fstring_lit` over `rb_fstring_cstr`
The former states explicitly that the argument must be a literal,
and can optimize away `strlen` on all compilers.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65059 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-13 09:59:22 +00:00
nobu 83a01e6f52 Added comments to rb_setup_fake_str and rb_fstring_new [ci skip]
`ptr` for these functions must refer constant string literals.
Otherwise, the result string's content can be modified/discarded
unexpectedly.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65058 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-13 09:23:56 +00:00
marcandre 2521b079fa [DOC] Improve String#strip documentation.
Patch by Josh Goldberg. [Fix GH-1933] [ci skip]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64757 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-16 02:49:44 +00:00
shyouhei 22444ae9b1 move function declarations from insns.def to internal.h
Just avoid being loose.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63755 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-27 00:57:16 +00:00
stomar 7215cecfb5 string.c: [DOC] grammar fixes
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63632 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-11 20:16:27 +00:00
nobu 46d7dc1162 [Docs] Improve documentation of String#lines
* Document about optional getline arguments
* Add examples, especially for the demonstration of `chomp: true`
[Fix GH-1886]

From: Koki Takahashi <hakatasiloving@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63610 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-08 10:45:01 +00:00
normal 256411b47f String#uminus dedupes unconditionally
[Feature #14478] [ruby-core:85669]

Thanks-to: Sam Saffron <sam.saffron@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63566 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-04 23:26:03 +00:00
nobu ce2f4f8526 string.c: trivial optimizations
* string.c (rb_str_aset): prefer BUILTIN_TYPE over TYPE after
  SPECIAL_CONST_P check.

* string.c (rb_str_start_with): prefer RB_TYPE_P over switch by
  TYPE.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63543 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-01 06:53:26 +00:00
nobu 87ccf7e50a string.c: doc for [Feature #13712]
* string.c (rb_str_start_with): [DOC] start_with? example with
  regexp.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63541 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-01 06:37:14 +00:00
normal 2fd1525b0f string.c: MAYBE_UNUSED to suppress warnings for `old`
Building with HAVE_MALLOC_USABLE_SIZE currently makes
SIZED_REALLOC_N ignore the old size arg.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63487 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-22 01:58:47 +00:00
normal 0a4be5beda string.c: size hints for free and realloc calls
Another part of the plan to reduce dependencies on malloc_usable_size:
https://bugs.ruby-lang.org/issues/10238

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63485 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-22 01:13:08 +00:00
nobu 703a5dd3e0 string.c: adjust to rb_str_upto_each
* range.c (range_each_func): adjust the signature of the callback
  function to rb_str_upto_each, and exit the loop if the callback
  returned non-zero.

* string.c (rb_str_upto_endless_each): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-28 11:16:54 +00:00
nobu 2c8f16e6c0 string.c: fix scanned substring with `\K`
* string.c (scan_once): fix the matched substring with `\K`, the
  beginning of that string may differ from the matched position.
  [ruby-core:86663] [Bug #14707]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63252 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-24 12:25:46 +00:00
mame 7f95eed19e Introduce endless range [Feature#12912]
Typical usages:
```
p ary[1..]          # drop the first element; identical to ary[1..-1]
(1..).each {|n|...} # iterate forever from 1; identical to 1.step{...}
```

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63192 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-19 15:18:50 +00:00
nobu 7c35618c53 string.c: suppress warning
* string.c (str_undump): get rid of warning C4129 by VC.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63170 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-17 04:12:57 +00:00
nobu cea438b0ca string.c: fix dumped suffix
* string.c (rb_str_dump): get rid of an error on evaling with
  frozen-string-literal enabled.  [ruby-core:86539] [Bug #14687]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63164 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-16 07:12:06 +00:00
nobu 5b2b1130cf string.c: fix checking order
* string.c (str_undump): check for suffix before if Unicode escape
  conflicts with it.  the message "but used force_encoding" sounds
  strange when it is not used.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63162 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-16 06:37:42 +00:00
stomar 5e99863393 string.c: [DOC] fix typo
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63160 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-14 16:50:42 +00:00
naruse 42f1b58964 Factor out get_reg_grapheme_cluster
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62893 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 07:58:39 +00:00
naruse 41b2ef4685 fix each_grapheme_cluster's size [Bug #14363]
From: Hugo Peixoto <hugo.peixoto@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62892 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 07:58:38 +00:00
naruse 6e0f5b8407 Revert "each_grapheme_cluster shouldn't return size [Bug #14363]"
This reverts commit r62887.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62891 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 07:58:37 +00:00
naruse 613decd088 each_grapheme_cluster shouldn't return size [Bug #14363]
From: Stefan Schüßler <mail@stefanschuessler.de>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62888 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-22 06:59:54 +00:00
nobu 7506fde3e9 Improve documentation for 'text '.split
The documentation didn't mention trailing spaces and the
example only demonstrated the case with leading spaces.
[Fix GH-1845]

From: Rodrigo Rosenfeld Rosas <rr.rosas@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62881 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-21 16:02:26 +00:00
nobu 1bf9dec04c string.c: [DOC] split with block [ci skip]
* string.c (rb_str_split_m): [DOC] about split with block.
  [Feature #4780]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-17 04:13:26 +00:00
nobu 2258a97fe2 string.c: split with block
* string.c (rb_str_split_m): yield each split substrings if the
  block is given, instead of returing the array.  [Feature #4780]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62763 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-15 11:08:04 +00:00
nobu c05fa459bb quote symbols
* sprintf.c (ruby__sfvextra): quote symbols as identifiers.

* string.c (rb_id_quote_unprintable): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62747 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-14 02:35:51 +00:00
k0kubun 288b44328d Export some missing symbols for MJIT
tool/ruby_vm/views/_insn_name_info.erb: on Linux, rb_vm_insn_name_offset
was needed to compile with --jit-debug (Usually --jit-debug requires
more symbols than the situation without --jit-debug because -O2 skips
some functions to compile).

vm.c: when running transform_mjit_header.rb with --jit-wait,
rb_source_location_cstr was repoted to be missing.

string.c: ditto, for rb_str_eql
numeric.c: ditto, for rb_float_eql

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62313 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-08 13:54:37 +00:00
k0kubun ed935aa5be mjit_compile.c: merge initial JIT compiler
which has been developed by Takashi Kokubun <takashikkbn@gmail> as
YARV-MJIT. Many of its bugs are fixed by wanabe <s.wanabe@gmail.com>.

This JIT compiler is designed to be a safe migration path to introduce
JIT compiler to MRI. So this commit does not include any bytecode
changes or dynamic instruction modifications, which are done in original
MJIT.

This commit even strips off some aggressive optimizations from
YARV-MJIT, and thus it's slower than YARV-MJIT too. But it's still
fairly faster than Ruby 2.5 in some benchmarks (attached below).

Note that this JIT compiler passes `make test`, `make test-all`, `make
test-spec` without JIT, and even with JIT. Not only it's perfectly safe
with JIT disabled because it does not replace VM instructions unlike
MJIT, but also with JIT enabled it stably runs Ruby applications
including Rails applications.

I'm expecting this version as just "initial" JIT compiler. I have many
optimization ideas which are skipped for initial merging, and you may
easily replace this JIT compiler with a faster one by just replacing
mjit_compile.c. `mjit_compile` interface is designed for the purpose.

common.mk: update dependencies for mjit_compile.c.

internal.h: declare `rb_vm_insn_addr2insn` for MJIT.

vm.c: exclude some definitions if `-DMJIT_HEADER` is provided to
compiler. This avoids to include some functions which take a long time
to compile, e.g. vm_exec_core. Some of the purpose is achieved in
transform_mjit_header.rb (see `IGNORED_FUNCTIONS`) but others are
manually resolved for now. Load mjit_helper.h for MJIT header.
mjit_helper.h: New. This is a file used only by JIT-ed code. I'll
refactor `mjit_call_cfunc` later.
vm_eval.c: add some #ifdef switches to skip compiling some functions
like Init_vm_eval.

win32/mkexports.rb: export thread/ec functions, which are used by MJIT.

include/ruby/defines.h: add MJIT_FUNC_EXPORTED macro alis to clarify
that a function is exported only for MJIT.

array.c: export a function used by MJIT.
bignum.c: ditto.
class.c: ditto.
compile.c: ditto.
error.c: ditto.
gc.c: ditto.
hash.c: ditto.
iseq.c: ditto.
numeric.c: ditto.
object.c: ditto.
proc.c: ditto.
re.c: ditto.
st.c: ditto.
string.c: ditto.
thread.c: ditto.
variable.c: ditto.
vm_backtrace.c: ditto.
vm_insnhelper.c: ditto.
vm_method.c: ditto.

I would like to improve maintainability of function exports, but I
believe this way is acceptable as initial merging if we clarify the
new exports are for MJIT (so that we can use them as TODO list to fix)
and add unit tests to detect unresolved symbols.
I'll add unit tests of JIT compilations in succeeding commits.

Author: Takashi Kokubun <takashikkbn@gmail.com>
Contributor: wanabe <s.wanabe@gmail.com>

Part of [Feature #14235]

---

* Known issues
  * Code generated by gcc is faster than clang. The benchmark may be worse
    in macOS. Following benchmark result is provided by gcc w/ Linux.
  * Performance is decreased when Google Chrome is running
  * JIT can work on MinGW, but it doesn't improve performance at least
    in short running benchmark.
  * Currently it doesn't perform well with Rails. We'll try to fix this
    before release.

---

* Benchmark reslts

Benchmarked with:
Intel 4.0GHz i7-4790K with 16GB memory under x86-64 Ubuntu 8 Cores

- 2.0.0-p0: Ruby 2.0.0-p0
- r62186: Ruby trunk (early 2.6.0), before MJIT changes
- JIT off: On this commit, but without `--jit` option
- JIT on: On this commit, and with `--jit` option

** Optcarrot fps

Benchmark: https://github.com/mame/optcarrot

|         |2.0.0-p0 |r62186   |JIT off  |JIT on   |
|:--------|:--------|:--------|:--------|:--------|
|fps      |37.32    |51.46    |51.31    |58.88    |
|vs 2.0.0 |1.00x    |1.38x    |1.37x    |1.58x    |

** MJIT benchmarks

Benchmark: https://github.com/benchmark-driver/mjit-benchmarks
(Original: https://github.com/vnmakarov/ruby/tree/rtl_mjit_branch/MJIT-benchmarks)

|           |2.0.0-p0 |r62186   |JIT off  |JIT on   |
|:----------|:--------|:--------|:--------|:--------|
|aread      |1.00     |1.09     |1.07     |2.19     |
|aref       |1.00     |1.13     |1.11     |2.22     |
|aset       |1.00     |1.50     |1.45     |2.64     |
|awrite     |1.00     |1.17     |1.13     |2.20     |
|call       |1.00     |1.29     |1.26     |2.02     |
|const2     |1.00     |1.10     |1.10     |2.19     |
|const      |1.00     |1.11     |1.10     |2.19     |
|fannk      |1.00     |1.04     |1.02     |1.00     |
|fib        |1.00     |1.32     |1.31     |1.84     |
|ivread     |1.00     |1.13     |1.12     |2.43     |
|ivwrite    |1.00     |1.23     |1.21     |2.40     |
|mandelbrot |1.00     |1.13     |1.16     |1.28     |
|meteor     |1.00     |2.97     |2.92     |3.17     |
|nbody      |1.00     |1.17     |1.15     |1.49     |
|nest-ntimes|1.00     |1.22     |1.20     |1.39     |
|nest-while |1.00     |1.10     |1.10     |1.37     |
|norm       |1.00     |1.18     |1.16     |1.24     |
|nsvb       |1.00     |1.16     |1.16     |1.17     |
|red-black  |1.00     |1.02     |0.99     |1.12     |
|sieve      |1.00     |1.30     |1.28     |1.62     |
|trees      |1.00     |1.14     |1.13     |1.19     |
|while      |1.00     |1.12     |1.11     |2.41     |

** Discourse's script/bench.rb

Benchmark: https://github.com/discourse/discourse/blob/v1.8.7/script/bench.rb

NOTE: Rails performance was somehow a little degraded with JIT for now.
We should fix this.
(At least I know opt_aref is performing badly in JIT and I have an idea
 to fix it. Please wait for the fix.)

*** JIT off
Your Results: (note for timings- percentile is first, duration is second in millisecs)

categories_admin:
  50: 17
  75: 18
  90: 22
  99: 29
home_admin:
  50: 21
  75: 21
  90: 27
  99: 40
topic_admin:
  50: 17
  75: 18
  90: 22
  99: 32
categories:
  50: 35
  75: 41
  90: 43
  99: 77
home:
  50: 39
  75: 46
  90: 49
  99: 95
topic:
  50: 46
  75: 52
  90: 56
  99: 101

*** JIT on
Your Results: (note for timings- percentile is first, duration is second in millisecs)

categories_admin:
  50: 19
  75: 21
  90: 25
  99: 33
home_admin:
  50: 24
  75: 26
  90: 30
  99: 35
topic_admin:
  50: 19
  75: 20
  90: 25
  99: 30
categories:
  50: 40
  75: 44
  90: 48
  99: 76
home:
  50: 42
  75: 48
  90: 51
  99: 89
topic:
  50: 49
  75: 55
  90: 58
  99: 99

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 11:22:28 +00:00
mame 552a5a993c string.c (rb_str_format_m): Fix the example code of the doc
Change `%08x` to `%016x` because of two reasons:

* `%016x` demonstrates that we can use two or more digits here.
* Currently, many people uses 64-bit environment.
  (I'm unsure if object_id is a good example here, though...)
I'm unsure if

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-29 08:40:22 +00:00
nobu 9237049efe string.c: clear substring code range
* string.c (str_substr): substring of broken code range string may
  be valid or broken.  patch by tommy (Masahiro Tomita) at
  [ruby-dev:50430] [Bug #14388].

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62040 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-25 13:10:14 +00:00
shyouhei dc1e6f17ba sizeof(uintptr_t) != sizeof(uintptr_t *)
Reported by mame.  Thanks.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61865 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-16 03:09:53 +00:00
shyouhei 39cfa67b4f __builtin_assume_aligned for *(foo *) casts
These casts are guarded. Must be safe to assume alignments.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61829 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-15 02:35:18 +00:00
nobu 6b5e0bd98c exclude flexible array size with old compilers
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61814 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-14 11:19:18 +00:00
mame 982e9e6235 string.c (struct mapping_buffer): Use FLEX_ARY_LEN
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61811 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-13 13:08:05 +00:00
usa b87571100a should cause preprocess error as other cases
* string.c (NONASCII_MASK): should cause preprocess error immediately if the
  compiler does not satisfy our assumptions.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61756 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-10 03:54:02 +00:00
nobu e9cb552ec9 internal.h: remove dependecy on ruby/encoding.h
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61713 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 06:24:11 +00:00
nobu ee85a6e72b internal.h: remove dependecy on ruby/io.h
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61712 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 06:24:10 +00:00
nobu e043ae7348 string.c: out-of-bounds access
* string.c (rb_str_enumerate_lines): fix out-of-bounds access when
  record separator is longer than the last element.  [Bug #14257]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61636 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-06 08:44:17 +00:00
shyouhei beaf2ace87 ULL suffix is a C99ism
Don't assume long long == 8 bytes.

If you can assume C99, there are macros named UINT64_C and
such for appropriate integer literal suffixes.
If you can't, no way but do a bitwise or.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61594 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-04 07:51:17 +00:00
nobu a1bbb2a780 Fix doc typo in Symbol#to_proc [Fix GH-1785]
[ci skip]

From: Dimitris Zorbas <dimitrisplusplus@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61588 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-04 00:44:40 +00:00
nobu 634a48c5c1 string.c: chomp rs at the end
* string.c (rb_str_enumerate_lines): should chomp record separator
  only, but not a newline, at the end of the receiver as well as
  middle, if the separator is given.
  [ruby-core:84552] [Bug #14257]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61513 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-29 12:19:03 +00:00
kazu c9bdee3e0d [DOC] Fix typos in downcase [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-27 00:04:30 +00:00
nobu e2479cc43f encoding.c: rb_enc_find_index2
* string.c (str_undump): use rb_enc_find_index2 to find encoding
  by unterminated string.  check the format before encoding name.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61396 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-22 01:03:17 +00:00
nobu 168c019998 string.c: fix memory leak
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21 07:59:00 +00:00
naruse 05d1d29d1f Don't allow mixed escape
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61381 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21 05:09:17 +00:00
naruse 188d85934b move dump format validation into parsing epilogue
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61380 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21 05:09:16 +00:00
naruse 29c6ca423c fix escapes in undump
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61379 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21 05:08:57 +00:00
nobu 7c18db61a1 string.c: multiple codepoints
* string.c (undump_after_backslash): fix multiple codepoints in
  braces.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-16 00:30:52 +00:00
nobu ae18c8f5b6 string.c: suppress warning
* string.c (str_undump): suppress maybe-uninitialized warning by
  gcc 7 and later.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61289 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-16 00:03:51 +00:00
tadd bbec11d329 Implement String#undump to unescape String#dump-ed string
[Feature #12275] [close GH-1765]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61228 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-14 08:47:13 +00:00
nobu a1692f7fdf string.c: fix rb_external_str_new_with_enc
* string.c (rb_external_str_new_with_enc): do not search non-ascii
  by NULL pointer.  [ruby-core:84055] [Bug #14150]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60979 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-02 07:09:16 +00:00
nobu 73e41247b9 string.c: prefer rb_syserr_fail
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60761 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-14 03:02:58 +00:00
rhe a82aaea719 string.c: fix up r60748
An #ifdef was missing in r60748 and build broke on systems without
crypt_r().

https://rubyci.org/logs/rubyci.s3.amazonaws.com/unstable11s/ruby-trunk/log/20171112T162503Z.fail.html.gz

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60749 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-12 17:10:29 +00:00
rhe 0b845a8458 string.c: fix memory leak in String#crypt
Use ALLOCV to allocate struct crypt_data for slightly cleaner and less
error-prone code. It is currently possible it leaks when an invalid
argument is passed to String#crypt or rb_str_new_cstr() fails to
allocate memory.

SIZEOF_CRYPT_DATA macro in missing/crypt.h is removed since it is not
used any longer.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60748 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-12 15:55:04 +00:00
stomar 8b1c1c55a9 string.c: improve docs for String#{concat,<<}
* string.c: [DOC] remove a misleading call-seq for String#concat,
  which suggests that all arguments must be Integers in this case;
  also clarify in the example that the receiver is modified;
  fix grammar for String#<<; move references to the end.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60712 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-07 20:15:59 +00:00
stomar 3262f4910f string.c: fix typos
* string.c: [DOC] fix typos in doxygen comments.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60707 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-07 20:11:09 +00:00
stomar 51b0230a9b string.c: improve docs
* string.c: [DOC] fix rdoc for cross reference; fix grammar.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60574 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-29 21:43:36 +00:00
watson1978 b03a44c4ac string.c: Improve String#prepend performance if only one argument is given
* string.c (rb_str_prepend_multi): Prepend the string without generating
    temporary String object if only one argument is given.
	This is very similar with https://github.com/ruby/ruby/pull/1634

	String#prepend -> 47.5 % up

    [Fix GH-1670] [ruby-core:82195] [Bug #13773]

* Before
      String#prepend      1.517M (± 1.8%) i/s -      7.614M in   5.019819s

* After
      String#prepend      2.236M (± 3.4%) i/s -     11.234M in   5.029716s

* Test code
require 'benchmark/ips'

Benchmark.ips do |x|
  x.report "String#prepend" do |loop|
    loop.times { "!".prepend("hello") }
  end
end

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60480 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 14:55:03 +00:00
nobu 2050b50d85 string.c: comment layout [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60331 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 00:00:41 +00:00
svn 4b8c94dd84 * remove trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60329 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 23:49:36 +00:00
sonots 84616bf979 * string.c: [DOC] Split rdoc of String#<< and String#concat [ci skip]
Split String#<< and String#concat docs to reflect single and multiple
arguments

patched by MSP-Greg [fix GH-1614]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60328 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 23:49:35 +00:00
sonots d9e11970f8 * string.c: Remove errant "the" in gsub documentation
patched by jlmuir (J. Lewis Muir) [fix GH-1679]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60324 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 23:35:40 +00:00
nobu 80c50308f9 Improve performance of string interpolation
This patch will add pre-allocation in string interpolation.
By this, unecessary capacity resizing is avoided.

For small strings, optimized `rb_str_resurrect` operation is
faster, so pre-allocation is done only when concatenated strings
are large.  `MIN_PRE_ALLOC_SIZE` was decided by experimenting with
local machine (x86_64-apple-darwin 16.5.0, Apple LLVM version
8.1.0 (clang - 802.0.42)).

String interpolation will be faster around 72% when large string is created.

* Before
  ```
  Calculating -------------------------------------
  Large string interpolation
                            1.276M (± 5.9%) i/s -      6.358M in   5.002022s
  Small string interpolation
                            5.156M (± 5.5%) i/s -     25.728M in   5.005731s
  ```

* After
  ```
  Calculating -------------------------------------
  Large string interpolation
                            2.201M (± 5.8%) i/s -     11.063M in   5.043724s
  Small string interpolation
                            5.192M (± 5.7%) i/s -     25.971M in   5.020516s
  ```

* Test code
  ```ruby
  require 'benchmark/ips'

  Benchmark.ips do |x|
    x.report "Large string interpolation" do |t|
      a = "Hellooooooooooooooooooooooooooooooooooooooooooooooooooo"
      b = "Wooooooooooooooooooooooooooooooooooooooooooooooooooorld"

      t.times do
        "#{a}, #{b}!"
      end
    end

    x.report "Small string interpolation" do |t|
      a = "Hello"
      b = "World"

      t.times do
        "#{a}, #{b}!"
      end
    end
  end
  ```

[Fix GH-1626]
From: Nao Minami <south37777@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60320 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 23:21:05 +00:00
hsbt 2c27e52f8e Add documentation for `chomp` option.
https://github.com/ruby/ruby/pull/1717

  Patch by @ksss [fix GH-1717]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 16:11:58 +00:00
sonots ec30bc5930 * string.c (deleted_prefix_length, deleted_suffix_length):
Add doxygen comment.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60254 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 10:33:25 +00:00
naruse 6187b0001b [Feature #13712] String#start_with? supports regexp
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60234 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 06:51:01 +00:00
glass 8320be1007 string.c: avoid unnecessary call of str_strlen()
* string.c (rb_strseq_index): refactor and avoid
  call of str_strlen() when offset == 0.
  it will improve performance of String#index and #include?

* benchmark/bm_string_index.rb: benchmark for this change

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-01 13:44:49 +00:00
nobu 16759238ad string.c: fix ASCII-only on succ
* string.c (str_succ): clear coderange cache when no alpha-numeric
  character case, carried part may become ASCII-only.
  [ruby-core:83062] [Bug #13952]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-30 00:01:23 +00:00
nobu 2d42119903 string.c: ASCII-incompatible is not ASCII only
* string.c (tr_trans): ASCII-incompatible encoding strings cannot
  be ASCII-only even if valid.  [ruby-core:83056] [Bug #13950]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60060 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-29 08:15:50 +00:00
nobu 8c59fdb8d8 dup String#split return value
* string.c (rb_str_split): return duplicated receiver, when no
  splits.  patched by tompng (tomoya ishida) in [ruby-core:82911],
  and the test case by Seiei Miyagi <hanachin@gmail.com>.
  [Bug#13925] [Fix GH-1705]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-23 07:09:07 +00:00
nobu e1be1d0c38 dup String#rpartition return value
* string.c (rb_str_rpartition): return duplicated receiver, when
  no splits.  [ruby-core:82911] [Bug#13925]

Author:    Seiei Miyagi <hanachin@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60001 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-23 07:09:06 +00:00
nobu b0326bce01 dup String#partition return value
* string.c (rb_str_partition): return duplicated receiver, when no
  splits.  [ruby-core:82911] [Bug#13925]

Author:    Seiei Miyagi <hanachin@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60000 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-23 07:09:05 +00:00
nobu b2da3824c5 refinements in string interpolation
* compile.c (iseq_compile_each0): insert to_s method call, so that
  refinements activated at the caller should take place.
  [Feature #13812]

* insns.def (tostring): fix up converted object to a string,
  infect and fallback.

* insns.def (branchiftype): new instruction for conversion.
  branches if TOS is an instance of the given type.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59950 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-18 02:27:13 +00:00