* string.c (rb_str_update): do not use negation of LONG_MIN, which
is negative too.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57800 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_byte_substr): fix another integer overflow which
can happen only when SHARABLE_MIDDLE_SUBSTRING is enabled.
[ruby-core:79951] [Bug #13289]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57799 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_subpos): fix integer overflow which can happen
only when SHARABLE_MIDDLE_SUBSTRING is enabled. incorpolate
https://github.com/mruby/mruby/commit/7db0786abdd243ba031e24683f
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57797 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: [DOC] restore documentation for String#<<
which became undocumented with r56021; fix a typo.
[ruby-core:79865] [Bug #13268]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57758 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This exposes the rb_fstring internal function to return a
deduped and frozen string when a non-frozen string is given.
This is useful for writing all sorts of record processing key
values maybe stored, but certain keys and values are often
duplicated at a high frequency, so memory savings can
noticeable.
Use cases are many:
* email/NNTP header processing
There are some standard header keys everybody uses
(From/To/Cc/Date/Subject/Received/Message-ID/References/In-Reply-To),
as well as common ones specific to a certain lists:
(ruby-core has X-Redmine-* headers)
It is also useful to dedupe values, as most inboxes have
multiple messages from the same sender, or MUA.
* package management systems -
things like RubyGems stores identical strings for licenses,
dependency names, author names/emails, etc
* HTTP headers/trailers -
standard headers (Host/Accept/Accept-Encoding/User-Agent/...)
are common, but there are also uncommon ones.
Values may be deduped, as well, as it is likely a user
agent will make multiple/parallel requests to the same
server.
* version control systems -
this can be useful for deduplicating names of frequent
committers (like "nobu" :)
In linux.git and git.git, there are also common
trailers such as Signed-Off-By/Acked-by/Reviewed-by/Fixes/...
as well as less common ones.
* audio metadata -
There are commonly used tags (Artist/Album/Title/Tracknumber),
but Vorbis comments allows arbitrary key values to be stored.
Music collections contain songs by the same artist or mutiple
songs from the same album, so deduplicating values will be
helpful there, too.
* JSON, YAML, XML, HTML processing
Certain fields, tags and attributes are commonly used
across the same and multiple documents
There is no security concern in this being a DoS vector by
causing immortal strings. The fstring table is not a GC-root
and not walked during the mark phase. GC-able dynamic symbols
since Ruby 2.2 are handled in the same manner, and that
implementation also relies on the non-immortality of fstrings.
[Feature #13077] [ruby-core:79663]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57698 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_shared_replace): use RUBY_ASSERT for
pre-condition.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57628 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_enumerate_lines): hint to suppress a
maybe-uninitialized warning by gcc.
* thread.c (rb_fd_no_init): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57618 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: add example for Symbol#to_s.
The docs for Symbol#to_s only include an example for
Symbol#id2name, but not for #to_s which is an alias;
the docs should include examples for both methods.
From: Marcus Stollsteimer <sto.mar@web.de>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57536 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Since the fstring table encompasses all strings in the
symbol table, we may reuse the fstring table walk to set
the class and eliminate the branch in rb_id2str.
* string.c (Init_String): use rb_cString immediately after definition
* symbol.c (rb_id2str): eliminate branch to set class
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57521 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Handle the embedded case first, since we may have an embedded
duplicate and non-embedded original string.
* string.c (rb_str_tmp_frozen_release): handled embedded strings
* test/ruby/test_io.rb (test_write_no_garbage): new test
[ruby-core:78898] [Bug #13085]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57471 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (STR_IS_SHARED_M): new flag to mark shared mulitple times
(STR_SET_SHARED): set STR_IS_SHARED_M
(rb_str_tmp_frozen_acquire, rb_str_tmp_frozen_release): new functions
(str_new_frozen): set/unset STR_IS_SHARED_M as appropriate
* internal.h: declare new functions
* io.c (fwrite_arg, fwrite_do, fwrite_end): new
(io_fwrite): use new functions
Introduce rb_str_tmp_frozen_acquire and rb_str_tmp_frozen_release
to manage a hidden, frozen string. Reuse one bit of the embed
length for shared strings as STR_IS_SHARED_M to indicate a string
has been shared multiple times. In the common case, the string
is only shared once so the object slot can be reclaimed immediately.
minimum results in each 3 measurements. (time and size)
Execution time (sec)
name trunk built
io_copy_stream_write 0.682 0.254
io_copy_stream_write_socket 1.225 0.751
Speedup ratio: compare with the result of `trunk' (greater is better)
name built
io_copy_stream_write 2.680
io_copy_stream_write_socket 1.630
Memory usage (last size) (B)
name trunk built
io_copy_stream_write 95436800.000 6512640.000
io_copy_stream_write_socket 117628928.000 7127040.000
Memory consuming ratio (size) with the result of `trunk' (greater is better)
name built
io_copy_stream_write 14.654
io_copy_stream_write_socket 16.505
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57469 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This seems a bug introduced by r520 (1.4.0). [ruby-core:79110] [Bug #13135]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57374 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* file.c (rb_get_path_check_convert): refine the error message
when the path name contains null byte.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57336 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_enc_str_scrub): only one of replacement and block
is allowed. [ruby-core:79038] [Bug #13119]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57304 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_enc_str_scrub): yield the invalid part only with
ASCII-incompatible. [ruby-core:79039] [Bug #13120]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57303 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_enumerate_lines): in paragraph mode, do not
include newlines which separate paragraphs, so that it will be
consistent with IO#each_line.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57184 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_casecmp_p): [DOC] use Unicode escape form to
get rid of warning C4819 by Microsoft Visual C++.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57154 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (crypt.h): crypt_r() was added in FreeBSD 12.0 but is
declared in unistd.h. [ruby-core:78664] [Bug #13038]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (chomp_newline): fix chomping newline only line.
rb_enc_prev_char return NULL if no previous character and must
not call rb_enc_ascget on it. a patch by Ary Borenszweig
<asterite AT gmail.com> at [ruby-core:78666]. [Bug #13037]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57088 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_equal): [DOC] fix fallback method name. the
peer's == method will be used, not ===.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57056 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_match_m_p): inverse of Regexp#match?. based on
the patch by Herwin Weststrate <herwin@snt.utwente.nl>.
[Fix GH-1483] [Feature #12898]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57053 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Fix documentation of String#casecmp? (examples didn't have the '?').
Add an example with non-ASCII characters. Clarify that casecmp,
unlike casecmp?, only does case-insensitivity on A-Z/a-z.
[ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56926 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (mapping_buffer): get rid of zero-length array member,
which is not a part of C90.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: Implement String#casecmp? and Symbol#casecmp? by using
String#downcase :fold for Unicode case folding. This does not include
options such as :turkic, because these currently cannot be combined
with the :fold option. This implements feature #12786.
* test/ruby/test_string.rb/test_symbol.rb: Tests for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
a hash value of Object might be Bignum, but it causes many troubles
expecially the Object is used as a key of a hash. so I've gave up
to do so.
* array.c (rb_ary_hash): use above macro.
* bignum.c (rb_big_hash): ditto.
* hash.c (rb_obj_hash, rb_hash_hash): ditto.
* numeric.c (rb_dbl_hash): ditto.
* proc.c (proc_hash): ditto.
* re.c (rb_reg_hash, match_hash): ditto.
* string.c (rb_str_hash_m): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56340 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
size with int, and of course also not guaranteed the value can be
Fixnum.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56320 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (lstrip_offset): add a fast path in the case of single
byte optimizable strings, as well as rstrip_offset.
[ruby-core:77392] [Feature #12788]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56250 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (enc_strlen, rb_enc_strlen_cr): Avoid signed integer
overflow. The result type of a pointer subtraction may have the same
size as long. This fixes String#size returning an negative value on
i686-linux environment:
str = "\x00" * ((1<<31)-2))
str.slice!(-3, 3)
str.force_encoding("UTF-32BE")
str << 1234
p str.size
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56247 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
actually check its availability rather to check GCC's version.
* configure.in (WARN_UNUSED_RESULT): moved to here.
* configure.in (RUBY_FUNC_ATTRIBUTE): change function declaration
to return int rather than void, because it makes no sense for a
warn_unused_result attributed function to return void.
Funny thing however is that it also makes no sense for noreturn
attributed function to return int. So there is a fundamental
conflict between them. While I tested this, I confirmed both
GCC 6 and Clang 3.8 prefers int over void to correctly detect
necessary attributes under this setup. Maybe subject to change
in future.
* internal.h (UNINITIALIZED_VAR): renamed to MAYBE_UNUSED, then
moved to configure.in for the same reason we move
WARN_UNUSED_RESULT.
* configure.in (MAYBE_UNUSED): moved to here.
* internal.h (__has_attribute): deleted, because it has no use now.
* string.c (rb_str_enumerate_lines): refactor macro rename.
* string.c (rb_str_enumerate_bytes): ditto.
* string.c (rb_str_enumerate_chars): ditto.
* string.c (rb_str_enumerate_codepoints): ditto.
* thread.c (do_select): ditto.
* vm_backtrace.c (rb_debug_inspector_open): ditto.
* vsnprintf.c (BSD_vfprintf): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56169 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The behavior on signed integer overflow is undefined. On platform with
sizeof(long)==4, it's fairly easy that 'len + termlen' overflows, where
len is the string length and termlen is the terminator length.
So, prevent the integer overflow by avoiding adding to a string length,
or casting to size_t before adding where the total size is passed to
{RE,}ALLOC*().
* string.c (STR_HEAP_SIZE, RESIZE_CAPA_TERM, str_new0, rb_str_buf_new,
str_shared_replace, rb_str_init, str_make_independent_expand,
rb_str_resize): Avoid overflow by casting the length to size_t. size_t
should be able to represent LONG_MAX+termlen.
* string.c (rb_str_modify_expand): Check that the new length is in the
range of long before resizing. Also refactor to use RESIZE_CAPA_TERM
macro.
* string.c (str_buf_cat): Fix so that it does not create a negative
length String. Also fix the condition for 'string sizes too big', the
total length can be up to LONG_MAX.
* string.c (rb_str_plus): Check the resulting String length does not
exceed LONG_MAX.
* string.c (rb_str_dump): Fix integer overflow. The dump result will be
longer then the original String.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56157 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (STR_EMBEDDABLE_P): Renamed from STR_EMBEDABLE_P(). And use
it in more places.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56155 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (STR_EMBEDABLE_P): extract the predicate macro to tell
if the given length is capable in an embedded string, and fix
possible integer overflow.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56151 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_change_terminator_length): fix integer overflow
in the case growing the terminator length and the string length
is around LONG_MAX.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56149 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_set_len): The buffer overflow check is wrong. The
space for termlen is allocated outside the capacity returned by
rb_str_capacity(). This fixes r41920 ("string.c: multi-byte
terminator", 2013-07-11). [ruby-core:77257] [Bug #12757]
* test/-ext-/string/test_set_len.rb (test_capacity_equals_to_new_size):
Test for this change. Applying only the test will trigger [BUG].
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56148 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* array.c (rb_ary_concat_multi): take multiple arguments. based
on the patch by Satoru Horie. [Feature #12333]
* string.c (rb_str_concat_multi, rb_str_prepend_multi): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56021 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_fs_setter): check and convert $; value at
assignment.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55990 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_split_m): show $; name in error message when it
is a wrong object.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* *.c: rename rb_funcall2 to rb_funcallv, except for extensions
which are/will be/may be gems. [Fix GH-1406]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55773 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
[Bug #12628]
This patch introduce many changes.
* Introduce concept of "Block Handler (BH)" to represent
passed blocks.
* move rb_control_frame_t::flag to ep[0] (as a special local
variable). This flags represents not only frame type, but also
env flags such as escaped.
* rename `rb_block_t` to `struct rb_block`.
* Make Proc, Binding and RubyVM::Env objects wb-protected.
Check [Bug #12628] for more details.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55766 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
UTF-8 to use upper-case four-digit hexadecimal escapes without braces
where possible [Feature #12419].
* test/ruby/test_string.rb (test_dump): Add tests for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55728 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (STR_BUF_MIN_SIZE): reduce from 128 to 127
[ruby-core:76371] [Feature #12025]
* string.c (rb_str_buf_new): adjust for above reduction
From Jeremy Evans <code@jeremyevans.net>:
This changes the minimum buffer size for string buffers from 128 to
127. The underlying C buffer is always 1 more than the ruby buffer,
so this changes the actual amount of memory used for the minimum
string buffer from 129 to 128. This makes it much easier on the
malloc implementation, as evidenced by the following code (note that
time -l is used here, but Linux systems may need time -v).
$ cat bench_mem.rb
i = ARGV.first.to_i
Array.new(1000000){" " * i}
$ /usr/bin/time -l ruby bench_mem.rb 128
3.10 real 2.19 user 0.46 sys
289080 maximum resident set size
72673 minor page faults
13 block output operations
29 voluntary context switches
$ /usr/bin/time -l ruby bench_mem.rb 127
2.64 real 2.09 user 0.27 sys
162720 maximum resident set size
40966 minor page faults
2 block output operations
4 voluntary context switches
To try to ensure a power-of-2 growth, when a ruby string capacity
needs to be increased, after doubling the capacity, add one. This
ensures the ruby capacity will be odd, which means actual amount
of memory used will be even, which is probably better than the
current case of the ruby capacity being even and the actual amount
of memory used being odd.
A very similar patch was proposed 4 years ago in feature #5875. It
ended up being rejected, because no performance increase was shown.
One reason for that is that ruby does not use STR_BUF_MIN_SIZE
unless rb_str_buf_new is called, and that previously did not have
a ruby API, only a C API, so unless you were using a C extension
that called it, there would be no performance increase.
With the recently proposed feature #12024, String.buffer is added,
which is a ruby API for creating string buffers. Using
String.buffer(100) wastes much less memory with this patch, as the
malloc implementation can more easily deal with the power-of-2
sized memory usage. As measured above, memory usage is 44% less,
and performance is 17% better.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55686 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
termlen and resize heap for the terminator. This is split from
rb_str_fill_terminator (str_fill_term) because filling terminator
and changing terminator length are different things. [Bug #12536]
* internal.h: declaration for rb_str_change_terminator_length.
* string.c (str_fill_term): Simplify only to zero-fill the terminator.
For non-shared strings, it assumes that (capa + termlen) bytes of
heap is allocated. This partially reverts r55557.
* encoding.c (rb_enc_associate_index): rb_str_change_terminator_length
is used, and it should be called whenever the termlen is changed.
* string.c (str_capacity): New static function to return capacity
of a string with the given termlen, because the termlen may
sometimes be different from TERM_LEN(str) especially during
changing termlen or filling terminator with specific termlen.
* string.c (rb_str_capacity): Use str_capacity.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55575 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
ChangeLog about the reverted changes are also deleted in this file.
[Bug #12536] [ruby-dev:49699] [ruby-dev:49702]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55559 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
of memory for termlen should always be needed.
In this fix, if possible, decrease capa instead of realloc.
[Bug #12536] [ruby-dev:49699]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55557 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Additional fix for [Bug #12536] [ruby-dev:49699].
* string.c (rb_usascii_str_new, rb_utf8_str_new): Specify termlen
which is apparently 1 for the encodings.
* string.c (str_new0_cstr): New static function to create a String
object from a C string with specifying termlen.
* string.c (rb_usascii_str_new_cstr, rb_utf8_str_new_cstr): Specify
termlen by using new str_new0_cstr().
* string.c (str_new_static): Specify termlen from the given encoding
when creating a new String object is needed.
* string.c (rb_tainted_str_new_with_enc): New function to create a
tainted String object with the given encoding. This means that
the termlen is correctly specified. Curretly static function.
The function name might be renamed to rb_tainted_enc_str_new
or rb_enc_tainted_str_new.
* string.c (rb_external_str_new_with_enc): Use encoding by using the
above rb_tainted_str_new_with_enc().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55555 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
is used, TERM_LEN(str) should be considered with it because
embedded strings are also processed by TERM_FILL.
Additional fix for [Bug #12536] [ruby-dev:49699].
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55552 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
[Bug #12536] [ruby-dev:49699]
* string.c (TERM_LEN_MAX): Macro for the longest TERM_FILL length,
the same as largest value of rb_enc_mbminlen(enc) among encodings.
* string.c (str_new, rb_str_buf_new, str_shared_replace): Allocate
+TERM_LEN_MAX bytes instead of +1. This change may increase memory
usage.
* string.c (rb_str_new_with_class): Use TERM_LEN of the "obj".
* string.c (rb_str_plus, rb_str_justify): Use str_new0 which is aware
of termlen.
* string.c (str_shared_replace): Copy +termlen bytes instead of +1.
* string.c (rb_str_times): termlen should not be included in capa.
* string.c (RESIZE_CAPA_TERM): When using RSTRING_EMBED_LEN_MAX,
termlen should be counted with it because embedded strings are
also processed by TERM_FILL.
* string.c (rb_str_capacity, str_shared_replace, str_buf_cat): ditto.
* string.c (rb_str_drop_bytes, rb_str_setbyte, str_byte_substr): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55547 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_casemap): do not put code with side effects
inside RSTRING_PTR() macro which evaluates the argument multiple
times.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55481 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
:ascii option in rb_str_upcase_bang and rb_str_downcase_bang.
* regenc.c: Fix a bug (wrong use of unnecessary slack at end of string).
* regenc.h -> include/ruby/oniguruma.h: Move declaration of
onigenc_ascii_only_case_map so that it is visible in string.c.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55329 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
for Unicode case mapping.
* test/ruby/enc/test_case_comprehensive.rb: Tests for above
functionality; fixed an encoding issue in assertion error message.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55296 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* missing/crypt.h (struct crypt_data): remove unnecessary member
"initialized".
* missing/crypt.c (des_setkey_r): nothing to be initialized in
crypt_data.
* configure.in (struct crypt_data): check for "initialized" in
struct crypt_data, which may be only in glibc, and isn't on AIX
at least.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55272 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
case mapping methods.
* enc/unicode.c: Check for invalid string and signal with negative
length value.
* test/ruby/enc/test_case_mapping.rb: Add tests for above.
* test/ruby/test_m17n_comb.rb: Add a message to clarify test failure.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55253 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: prefer crypt_r to crypt iff system crypt nor crypt_r
are not provided.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55250 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* configure.in: revert r55237. replace crypt, not crypt_r, and
check if crypt is broken more.
* missing/crypt.c: move crypt_r.c
* string.c (rb_str_crypt): use crypt_r if provided by the system.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55245 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
the protective check for the presence of an option.
Update documentation.
* test/ruby/enc/test_case_comprehensive.rb: Adjust tests for above change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55225 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* transcode.c (str_transcode0): scrub in the given encoding when
the source encoding is given, not in the encoding of the
receiver. [ruby-core:75732] [Bug #12431]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55181 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* include/ruby/ruby.h (RB_INTEGER_TYPE_P): new macro and
underlying inline function to check if the object is an
Integer (Fixnum or Bignum).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55044 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
and define CONSTFUNC and PUREFUNC if available.
Note that I don't add those options as default because
it still shows many false-positive (it seems not to consider
longjmp).
* vm_eval.c (stack_check): get rb_thread_t* as an argument
to avoid duplicate call of GET_THREAD().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54952 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
only if it can use SSE 4.2 POPCNT whose latency is 3 cycle.
* internal.h (rb_popcount64): use __builtin_popcountll because now
it is in fast path.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54894 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_concat): shortcut concatenation to ASCII-8BIT
as well as US-ASCII.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54882 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_concat): [DOC] fix the indefinite article, for
replacement from Fixnum to Integer.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54881 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* configure.in (__builtin_ctz): check.
* configure.in (__builtin_ctzll): check.
* internal.h (rb_popcount32): defined for ntz_int32.
it can use __builtin_popcount but this function is not used on
GCC environment because it uses __builtin_ctz.
When another function uses this, using __builtin_popcount
should be re-considered.
* internal.h (rb_popcount64): ditto.
* internal.h (ntz_int32): defined for ntz_intptr.
* internal.h (ntz_int64): defined for ntz_intptr.
* internal.h (ntz_intptr): defined as ntz for uintptr_t.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54854 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (enc_succ_alnum_char): try to skip an invalid character
gap between GREEK CAPITAL RHO and SIGMA.
[ruby-core:74478] [Bug #12204]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54210 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (sym_match_m): delegate to String#match but not
String#=~. [ruby-core:72864] [Bug #11991]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53866 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_dump): share same string literal instead of a
magic number.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53774 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_external_str_with_enc, rb_str_concat, rb_str_dump):
use encoding index as shortcut without rb_encoding.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53773 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_fstring_enc_new, rb_fstring_enc_cstr): functions to
make fstring with encoding.
* re.c (rb_reg_initialize): make fstring without copying.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53736 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* error.c (rb_assert_failure): assertion with stack dump.
* ruby_assert.h (RUBY_ASSERT): new header for the assertion.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53615 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
upcase/downcase/capitalize/swapcase. :lithuanian can be used for
testing if no specific option is desired.
* test/ruby/enc/test_case_mapping.rb: Adjusted to above.
(with Kimihito Matsui)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53565 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
added hand-coded support for Turkic, fixed logic for swapcase.
* string.c: Made use of new case mapping code possible from upcase,
capitalize, and swapcase (with :lithuanian as a guard).
* test/ruby/enc/test_case_mapping.rb: Adjusted for above.
(with Kimihito Matsui)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53562 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
case mapping. The code path is currently guarded by the :lithuanian
option to avoid accidental problems in daily use.
* test/ruby/enc/test_case_mapping.rb: Test for above.
* string.c: function 'check_case_options': fixed logical errors
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53548 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
argument of `String#split` is not `$;` but `nil`.
When `nil` is passed as first argument, `$;` is used.
[ci skip] [Bug #11729] [ruby-dev:49378]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53260 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_compat_and_valid): as scrub does nothing for dummy
encoding string now, incompatible encoding is not a matter.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53235 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_scrub): the result should be infected by the
original string.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53169 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
instead of raising Encoding::CompatibilityError. [Feature #11801]
* string.c (rb_str_escape): added to dump given string like
rb_str_inspect without quotes and always dump in US-ASCII
like rb_str_dump.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53027 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_init): rb_id_encoding() returns same ID with
caching.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52982 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_obj_as_string): fstring should not be infected.
re-apply r52872 and fix a typo.
TODO: other frozen strings also may not be.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52882 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_obj_as_string): fstring should not be infected.
TODO: other frozen strings also may not be.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52872 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_make_independent_expand): adjust argument
qualifier to get rid of a VC bug.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52844 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_string_value_cstr): should not raise on frozen
string.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52833 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (id_to_s): remove redundant variable
(rb_obj_as_string): trade id_to_s for idTo_s
(rb_str_equal): replace rb_intern(...) with pre-defined ID
(rb_str_cmp_m): ditto
(rb_str_match): ditto
(str_upto_each): ditto
(rb_str_sum): ditto
(Init_String): remove id_to_s initialization
This leads to a minor size reduction on my x86 (32-bit) system:
text data bss dec hex filename
129373 8 32 129413 1f985 string.o-orig
129082 8 8 129098 1f84a string.o
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52479 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This function only accept T_STRING (and T_REGEXP).
This patch improves performance of a tiny_segmenter benchmark
(num=2) 2.54sec -> 2.42sec on my machine.
https://github.com/chezou/TinySegmenter.jl/blob/master/benchmark/benchmark.rb
* encoding.c: add ENC_DEBUG and ENC_ASSERT() macros.
* internal.h: add a decl. of rb_enc_check_str().
* string.c (rb_str_plus): use rb_enc_check_str().
* string.c (rb_str_subpat_set): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52350 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* internal.h (RUBY_DTRACE_CREATE_HOOK): macro to call hook at
object creation.
* vm.c (rb_source_location, rb_source_loc): retrieve source path
and line number at once.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52340 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* file.c (rb_file_expand_path_internal): concatenate converted
string to the result instead of making converted string and
append it.
* string.c (rb_str_cat_conv_enc_opts): from rb_str_conv_enc_opts,
separate function to concatenate with transcoding.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52147 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* proc.c (rb_sym_to_proc): move from string.c and create a Proc
with no environments. [ruby-core:71088] [Bug #11594]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52129 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
https://github.com/ruby/ruby/pull/1050
by Kazuho Oku <kazuho@natadeco.co>.
This pull request has the following commits.
* gc.c: reduce # of args to 6 (max. of register args on x86-64) so
that the `newobj_of_slowpass` can be called via TCO.
* gc.c (newobj_of), string.c (str_duplicate): for performance,
the hot functions must be inlined.
* gc.c: for performance, preceding arguments of `.*newobj_of.*`
must be same, so that the arg registers can be reused in case of
TCO.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_duplicate): move from rb_str_resurrect to short
circuit initialization.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52074 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_resurrect): fix resurrection of short enough to
be embedded but not embedded string.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52073 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_times): optimize for the argument 0 and 1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52071 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_new_frozen): use raw macro for RString object.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52070 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* vm_args.c (args_setup_block_parameter): wrap a symbol in ifunc
by a proc as a block parameter.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52056 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_resurrect): optimize by short circuit to copy
hidden string without checking length, encoding and so on.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52048 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* proc.c (proc_to_s): include the original symbol name in string
form.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51988 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* compile.c (compile_dstr_fragments): fix performance by omitting
the first empty string only for keeping literal encoding if
other literals are too. [ruby-core:70930] [Bug #11556]
* string.c (rb_str_append_literal): append but keep encoding non
US-ASCII.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51970 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_setbyte): separate resetting code range by each
code range, and remove unnecessary branches.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51907 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_setbyte): keep the code range as possible.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51873 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encindex.h: separate encoding index constants from internal.h.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51861 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* node.c (rb_alloc_tmp_buffer): use NODE_ALLOCA to mark locations
like as builtin alloca. [ruby-core:70251] [Bug #11418]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51492 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* re.c (rb_memsearch): should match only char boundaries in wide
character encodings. [ruby-core:70220] [Bug #11413]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51470 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (rb_obj_hash): move in order to share with rb_any_hash.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51430 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* hash.c (rb_sym_hash): return same value as rb_any_hash() of
Symbol. [Bug #9381]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51426 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_replace_shared_without_enc): fill the terminator
of embedded strings in wide char encodings.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51372 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (fstring_cmp, rb_str_hash_cmp): compare lengths first,
then encodings, and contents at last.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51370 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (fstr_update_callback): fstring must not be a shared
string, or the content without RSTRING_FSTR may be freed.
[ruby-dev:49188] [Bug #11386]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51363 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (fstr_update_callback): pool bare strings only.
* string.c (rb_fstring): return the original string with sharing a
fstring if it has extra attributes, not the fstring itself.
[ruby-dev:49188] [Bug #11386]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51360 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (fstr_update_callback): create new string for fake
string, and pool shared target unless substring.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51355 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* include/ruby/ruby.h: add raw FL macros, which assume always the
argument object is not a special constant.
* internal.h (STR_EMBED_P, STR_SHARED_P): valid only for T_STRING.
* string.c: deal with taint flags directly across String instances.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51353 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_reverse): reversed string is not a substring,
and should not set coderange of the original string.
[ruby-dev:49189] [Bug #11387]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51344 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* include/ruby/encoding.h (ENC_CODERANGE_CLEAN_P): predicate that
tells if the coderange is clean, that is 7bit or valid, and no
needs to scrub.
* re.c (rb_reg_expr_str): use ENC_CODERANGE_CLEAN_P.
* string.c (enc_strlen, rb_enc_cr_str_buf_cat, rb_str_scrub):
ditto.
* string.c (rb_str_enumerate_chars): ditto, and suppress a warning
by gcc6.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51278 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_enumerate_lines): do not check if a block is
given twice. RETURN_ENUMERATOR includes rb_block_given_p()
check which is redundant here.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51266 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_enumerate_bytes, rb_str_enumerate_codepoints):
do not check if a block is given twice. RETURN_SIZED_ENUMERATOR
includes rb_block_given_p() check which is redundant here.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51248 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_str_freeze may resize oversized buffers to save memory, so favor
it over rb_obj_freeze. This is useful because IO methods do not
prematurely shrink buffers, as they are likely to be overwritten
with full data.
* string.c (Init_String): use rb_str_freeze for String#freeze
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51170 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
See doc/extension.rdoc for justification using RB_GC_GUARD instead
of volatile.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51011 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_fstring_cstr): new function to make a fstring from
a string literal.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51009 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reduces GC overhead and makes the API more consistent
with IO#read and IO#read_nonblock.
* ext/socket/basicsocket.c (bsock_recv): document outbuf
* ext/socket/unixsocket.c (unix_recvfrom): ditto
* ext/socket/init.c (rsock_strbuf, recvfrom_locktmp): new functions
(rsock_s_recvfrom): support destination buffer as 3rd arg
(rsock_s_recvfrom_nonblock): ditto
* string.c (rb_str_locktmp_ensure): export for internal ext
* test/socket/test_nonblock.rb: test recv_nonblock
* test/socket/test_unix.rb: test recv
[ruby-core:69543] [Feature #11242]
Benchmark results:
user system total real
alloc 0.130000 0.280000 0.410000 ( 0.420656)
extbuf 0.100000 0.220000 0.320000 ( 0.318708)
-------------------8<--------------------
require 'socket'
require 'benchmark'
nr = 100000
msg = ' ' * 16384
size = msg.bytesize
buf = ' ' * size
UNIXSocket.pair(:DGRAM) do |a, b|
Benchmark.bmbm do |x|
x.report('alloc') do
nr.times do
b.send(msg, 0)
a.recv(size, 0)
end
end
x.report('extbuf') do
nr.times do
b.send(msg, 0)
a.recv(size, 0, buf)
end
end
end
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
string passed to String#crypt contains null.
the patch is from jrusnack <jrusnack at redhat.com>.
[Bug #10988] [fix GH-853]
* test/ruby/test_string.rb: test for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50458 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_buf_cat): expand later so that the buffer can be
larger for further use. [Bug #11080] [Bug #11080]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_buf_cat): expand the heap buffer at making
str independent, not only just copying.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50336 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_setbyte): check the argument first not to
discard shared string and code range unnecessarily until
actually changing the contents. pointed out by headius.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50162 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (STR_SET_SHARED): ignore FAKESTR because only Ruby objects
can use write barrier.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49998 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (chompped_length): enable smart chomp for all non-dummy
encoding strings, not only default_rs.
[ruby-core:68258] [Bug #10893]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49706 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (get_pat_quoted): simply raise a RegexpError, TypeError
is never raised.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49696 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_split_m): raise ArgumentError at broken string
not RegexpError, as Regexp is not involved in.
[ruby-core:68229] [Bug #10886]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49695 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* use rb_funcallv() for no arguments call instead of variadic
rb_funcall().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49612 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
[Bug #10853][ruby-core:68110]
* bootstraptest/test_string.rb: test for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49586 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_buf_cat): use local variables instead of repeating
macros.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49407 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_make_independent_expand): terminate String when
moved from heap to embedded. [Fix GH-821].
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49405 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_succ_bang): get rid of making intermediate
string object.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_succ): extract local variables from constant
RSTRING_PTR and RSTRING_LEN.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48780 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_crypt): check arguments more strictly.
* crypt() is not for wide char strings
* salt bytes should not be NUL
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48779 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (chompped_length, chomp_rs): extract from
rb_str_chomp_bang to share with rb_str_chomp.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48756 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (setup_fake_str): fake string does not share another
string, but just should not free.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48695 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* include/ruby/intern.h (SIZED_ENUMERATOR): separate from
RETURN_SIZED_ENUMERATOR.
* string.c (rb_str_enumerate_chars): get rid of calling
rb_block_given_p() twice.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48693 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_setter): preserve encoding of global variable
name in error message.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48549 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_obj_equal() is specially optimized in
opt_eq_func()@vm_insnhelper.c.
This fix is made from this discussion:
https://www.omniref.com/ruby/2.1.4/symbols/Symbol/%3D%3D#line=8361.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48369 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (sym_printable): QUOTE() should not raise an exception
even on invalid byte sequence.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48305 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_{,l,r}strip_bang): rb_str_subseq() will not
NUL-terminate the result string, in the future, so it will not
be needed in other cases.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48280 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_make_independent_expand): drop NOFREE flag after
reallocation, static buffer is not pointed anymore.
[ruby-core:65317] [Bug #10304]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47737 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
All symbols have an fstring entry, and we initialize symbol tables
with 1000 bins. This reduces resizes during startup.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47544 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
key first.
Garbage keys can be swept by lazy sweeping invoked by creating new
fstring. So that simply do:
(1) delete garbage key and return `fstr_update_callback' immediately
(2) try again `fstr_update_callback()' to create a new fstr.
This bug can be cause memory corruption, reported by
http://u64.rubyci.org/~chkbuild/ruby-trunk/log/20140821T220302Z.fail.html.gz
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47252 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_setup_fake_str): setup fake string from C pointer,
length, and encoding.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47198 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
RBASIC_SET_CLASS() because it insert write barriers to fake
(non-RVALUE) structure.
It can cause unexpected behaviour.
Ruby 2.1 also have a same problem (setup_fake_str() in parse.y).
* symbol.c (setup_fake_str): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47190 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_to_symbol): new function to convert an object to a
symbol.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47003 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_count): move code for the first argument
outside loop for the rest, as it is executed only once.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46898 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
exposing IDs from collectable symbols.
[Bug #10014]
Now, rb_check_id() returns 0 if corresponding symbol is
pinned dynamic symbol.
There is remaining intern_cstr_without_pindown(), it can return
IDs from collectable symbols. We must be careful to use it
(only used in parse.y). I think it should be removed if
it does not have impact for performance.
* parse.y:
add:
* STATIC_SYM2ID()
* STATIC_ID2SYM()
rename:
* rb_pin_dynamic_symbol() -> dsymbol_pindown()
* internal.h:
remove:
* rb_check_id_without_pindown()
* rb_sym2id_without_pindown()
add:
* rb_check_symbol()
* rb_check_symbol_cstr()
* load.c: use rb_check_id() or rb_check_id_cstr().
* object.c: ditto.
* struct.c: ditto.
* thread.c: ditto.
* vm_method.c: ditto.
* string.c (sym_find): use only rb_check_symbol().
* sprintf.c (rb_str_format): use rb_check_symbol_cstr().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46764 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (fstr_update_callback): should not access garbage
object contents, copy from the argument instead.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46728 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
any more.
Make new frozen string and replace with garbage frozen string.
* common.mk: use gc.h from string.c.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46727 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_fstring_new): make US-ASCII string. another
function may be used for non-ASCII strings.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46689 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (sym_succ, sym_cmp, sym_casecmp, sym_match, sym_aref):
not need to make copy of interned strings, use directly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46443 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_resize): should consider the capacity instead
of the old length, as pointed out by nagachika.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46413 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_freeze): shrink the buffer before freezing, as
pointed out by Eric Wong at [ruby-core:63119].
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46411 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_resize): shrink the buffer even if new length
is same but it is enough smaller than the capacity.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46408 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_slice_bang): [DOC] update return value against
a fixnum, which has changed because of M17N. [fix GH-631]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46376 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_substr): need to reset code range for shared
string too, not only copied string.
[ruby-core:62842] [Bug #9882]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46243 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_pat_search): advance by byte offset but not by char
offset. [ruby-core:62669] [Bug #9849]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46013 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
ENC_CODERANGE_VALID if the string is ASCII-8BIT and already has a non
ASCII character.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45651 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_update): rename rb_str_splice as exported name
rb_str_update, and make the former an alias macro of the latter.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45647 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (SHARABLE_SUBSTRING_P): predicate if substring can be
shared with the original string. true if just at the end of the
original string, for the time being. all substring will be able to
be shared in the future.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45633 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_buf_cat): revert r45622. broken data can be added
later.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45624 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_buf_cat): revert r45621. this function does not
scan appended data.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45623 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_cat_cstr): check the argument as other `_cstr`
functions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45610 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_cat, rb_str_cat2): make aliases of
rb_str_buf_cat and rb_str_buf_cat2 respectively.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45607 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_cat, rb_str_cat2): reduce duplicated code, they
are same as rb_str_buf_cat and rb_str_buf_cat2 respectively now.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45606 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_enc_cr_str_buf_cat): keep code range of the source
string even if code range of the destination string is unknown.
no reason to the former is affected by the latter.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45581 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (sym_to_proc), proc.c (rb_block_clear_env_self): clear
caller's self which is useless, so that it can get collected.
[Fixes GH-592]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_buf_cat): remove unnecessary terminator space,
since the capacity does not include its length but RESIZE_CAPA()
considers it.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45536 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_buf_cat): should round up the capacity by 4KiB,
but not number of rooms. [ruby-core:61886] [Bug #9709]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45534 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_pat_search): match result should be infected by the
pattern.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45460 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_pat_search): unset $~ if the last match failed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45458 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* re.c (match_regexp): set regexp for MatchData from string.
* re.c (rb_backref_set_string): create MatchData from string and
set backref.
* string.c (rb_pat_search, rb_str_sub, rb_str_sub_bang, str_gsub),
(scan_once, rb_str_scan, rb_str_partition): use rb_str_index
instead of rb_reg_search() when pattern is a String. based on
the patch by Sam Rawlins <sam.rawlins@gmail.com> [Fixes GH-579]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45451 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (mustnot_broken): extract function to reject invalid
byte sequence, from rb_str_split_m().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45450 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_gsub): invert and rename `str_replace` flag as
`need_backref`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45446 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
See this ticket about Symbol GC.
* include/ruby/ruby.h:
Declare few functions.
* rb_sym2id: almost same as old SYM2ID but support dynamic symbols.
* rb_id2sym: almost same as old ID2SYM but support dynamic symbols.
* rb_sym2str: almost same as `rb_id2str(SYM2ID(sym))` but not
pin down a dynamic symbol.
Declare a new struct.
* struct RSymbol: represents a dynamic symbol as object in
Ruby's heaps.
Add few macros.
* STATIC_SYM_P: check a static symbol.
* DYNAMIC_SYM_P: check a dynamic symbol.
* RSYMBOL: cast to RSymbol
* gc.c: declare RSymbol. support T_SYMBOL.
* internal.h: Declare few functions.
* rb_gc_free_dsymbol: free up a dynamic symbol. GC call this
function at a sweep phase.
* rb_str_dynamic_intern: convert a string to a dynamic symbol.
* rb_check_id_without_pindown: not pinning function.
* rb_sym2id_without_pindown: ditto.
* rb_check_id_cstr_without_pindown: ditto.
* string.c (Init_String): String#intern and String#to_sym use
rb_str_dynamic_intern.
* template/id.h.tmpl: use LSB of ID as a flag for determining a
static symbol, so we shift left other ruby_id_types.
* string.c: use rb_sym2str instead `rb_id2str(SYM2ID(sym))` to
avoid pinning.
* load.c: use xx_without_pindown function at creating temporary ID
to avoid pinning.
* object.c: ditto.
* sprintf.c: ditto.
* struct.c: ditto.
* thread.c: ditto.
* variable.c: ditto.
* vm_method.c: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45426 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* internal.h: add prototype for rb_reg_search0
* re.c: rename rb_reg_search to rb_reg_search0, add set_backref_str
argument to allow callers to indicate that they don't require the
backref string to be allocated
* string.c: don't allocate backref str if replacement string is provided
Closes GH-578. [Bug #9676] [ruby-core:61682]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45414 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
add links to `Object#hash` to each #`hash` methods rdocs.
[Fixes GH-567]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45329 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_hash_m): [DOC] hash value depends on the
encoding too.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45328 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_format_m): trade volatile for RB_GC_GUARD
RB_GC_GUARD meaning is clear and has better code generation.
[ruby-core:60688]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44951 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
because str is not embed.
* string.c (str_replace): remove `FL_SET(str, STR_SHARED)' line
because STR_SET_SHARED() set STR_SHARED.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44828 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (STR_UNSET_NOCAPA): removed.
Use FL_UNSET() with STR_SHARED.
* string.c (rb_str_capacity): check STR_SHARED directly
beacuse it is not a embed string.
* string.c (rb_str_modify_expand): ditto.
* string.c (rb_str_shared_replace): use STR_SET_SHARED().
* string.c (str_make_independent_expand): remove STR_UNSET_NOCAPA()
because `str' is not shared string.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44825 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Move code from str_new_frozen_with_klass() (and remove it)
* `aux.shared' should not be 0 for STR_SHARED strings.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44822 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
By r44804, string objects can not have STR_ASSOC flag.
* internal.h: ditto.
* ext/objspace/objspace_dump.c (dump_object): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44805 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* pack.c (str_associate, str_associated): keep associated objects
in an instance variables, instead of in the internal structure.
* string.c (rb_str_associate, rb_str_associated): deprecate.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44804 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_modify_expand): enable capacity and disable
assocation with packed objects when setting capa, so that
pack("p") string fails to unpack properly after modified.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44803 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (get_actual_encoding): get actual encoding according to
the BOM if exists.
* string.c (rb_str_inspect): use according encoding, instead of
pseudo encodings, UTF-{16,32}. [ruby-core:59757] [Bug #8940]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44605 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (register_symid_str): use fstrings in symbol table
[Bug #9171] [ruby-core:58656]
* parse.y (rb_id2str): ditto
* string.c (rb_fstring): create frozen_strings on first usage. this
allows rb_fstring() calls from the parser (before cString is created)
* string.c (fstring_set_class_i): set klass on fstrings generated
before cString was defined
* string.c (Init_String): convert frozen_strings table to String
objects after boot
* ext/-test-/symbol/type.c (bug_sym_id2str): expose rb_id2str()
* test/-ext-/symbol/test_type.rb (module Test_Symbol): verify symbol
table entries are fstrings
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44057 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (fstr_update_callback): move a variable declaration since
ISO C90 forbids mixed declarations and code.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43988 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
based on feedback from @nagachika. In the existing case, we can
return ST_STOP to prevent any hash modification. In the !existing
case, set both key and value to the fstr.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
nobu
kazu
stomar
normal
shugo
rhe
duerst
usa
shyouhei
akr
ko1
ngoto
naruse
yui-knk
hsbt
svn
glass
mame
marcandre
ayumin
ktsj
nari
charliesome
tmm1