Use ALLOCV to allocate struct crypt_data for slightly cleaner and less
error-prone code. It is currently possible it leaks when an invalid
argument is passed to String#crypt or rb_str_new_cstr() fails to
allocate memory.
SIZEOF_CRYPT_DATA macro in missing/crypt.h is removed since it is not
used any longer.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60748 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: [DOC] remove a misleading call-seq for String#concat,
which suggests that all arguments must be Integers in this case;
also clarify in the example that the receiver is modified;
fix grammar for String#<<; move references to the end.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60712 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_prepend_multi): Prepend the string without generating
temporary String object if only one argument is given.
This is very similar with https://github.com/ruby/ruby/pull/1634
String#prepend -> 47.5 % up
[Fix GH-1670] [ruby-core:82195] [Bug #13773]
* Before
String#prepend 1.517M (± 1.8%) i/s - 7.614M in 5.019819s
* After
String#prepend 2.236M (± 3.4%) i/s - 11.234M in 5.029716s
* Test code
require 'benchmark/ips'
Benchmark.ips do |x|
x.report "String#prepend" do |loop|
loop.times { "!".prepend("hello") }
end
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60480 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Split String#<< and String#concat docs to reflect single and multiple
arguments
patched by MSP-Greg [fix GH-1614]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60328 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This patch will add pre-allocation in string interpolation.
By this, unecessary capacity resizing is avoided.
For small strings, optimized `rb_str_resurrect` operation is
faster, so pre-allocation is done only when concatenated strings
are large. `MIN_PRE_ALLOC_SIZE` was decided by experimenting with
local machine (x86_64-apple-darwin 16.5.0, Apple LLVM version
8.1.0 (clang - 802.0.42)).
String interpolation will be faster around 72% when large string is created.
* Before
```
Calculating -------------------------------------
Large string interpolation
1.276M (± 5.9%) i/s - 6.358M in 5.002022s
Small string interpolation
5.156M (± 5.5%) i/s - 25.728M in 5.005731s
```
* After
```
Calculating -------------------------------------
Large string interpolation
2.201M (± 5.8%) i/s - 11.063M in 5.043724s
Small string interpolation
5.192M (± 5.7%) i/s - 25.971M in 5.020516s
```
* Test code
```ruby
require 'benchmark/ips'
Benchmark.ips do |x|
x.report "Large string interpolation" do |t|
a = "Hellooooooooooooooooooooooooooooooooooooooooooooooooooo"
b = "Wooooooooooooooooooooooooooooooooooooooooooooooooooorld"
t.times do
"#{a}, #{b}!"
end
end
x.report "Small string interpolation" do |t|
a = "Hello"
b = "World"
t.times do
"#{a}, #{b}!"
end
end
end
```
[Fix GH-1626]
From: Nao Minami <south37777@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60320 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_strseq_index): refactor and avoid
call of str_strlen() when offset == 0.
it will improve performance of String#index and #include?
* benchmark/bm_string_index.rb: benchmark for this change
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_succ): clear coderange cache when no alpha-numeric
character case, carried part may become ASCII-only.
[ruby-core:83062] [Bug #13952]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_split): return duplicated receiver, when no
splits. patched by tompng (tomoya ishida) in [ruby-core:82911],
and the test case by Seiei Miyagi <hanachin@gmail.com>.
[Bug#13925] [Fix GH-1705]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* compile.c (iseq_compile_each0): insert to_s method call, so that
refinements activated at the caller should take place.
[Feature #13812]
* insns.def (tostring): fix up converted object to a string,
infect and fallback.
* insns.def (branchiftype): new instruction for conversion.
branches if TOS is an instance of the given type.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59950 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_enc_str_scrub): enc can differ from the actual
encoding of the string, the cached coderange is useless then.
[ruby-core:82674] [Bug #13874]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59763 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (WANTARRAY): make array for the result in method
functions and pass it to enumerator functions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59736 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (enumerator_wantarray): show warnings at method
functions for proper method names.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59732 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_enumerate_grapheme_clusters): suppress a
maybe-uninitialized warning by old gcc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59730 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_split_m): fix potential bug when rb_memsearch()
matches a octet in the middle of a multi-byte character sequence.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59673 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_rstrip_bang): improve the performance in 50%
for a string pattern, and in 10% for a regexp pattern. get rid
of making MatchData in middle, which is not used.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59496 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_initialize): new function to (re)initialize a
string with data and encoding. extracted from
rb_external_str_new_with_enc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59448 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
These caused numerous CI failures I haven't been able to
reproduce [ruby-core:82102]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59364 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The same hash keys may be loaded from tainted data sources
frequently (e.g. parsing headers from socket or loading
YAML data from a file). If a non-tainted fstring already
exists (because the application expects the hash key),
cache and deduplicate the tainted version in the new
tainted_frozen_strings table.
For non-embedded strings, this also allows sharing with the
underlying malloc-ed data.
* vm_core.h (rb_vm_struct): add tainted_frozen_strings
* vm.c (ruby_vm_destruct): free tainted_frozen_strings
(Init_vm_objects): initialize tainted_frozen_strings
(rb_vm_tfstring_table): accessor for tainted_frozen_strings
* internal.h: declare rb_fstring_existing, rb_vm_tfstring_table
* hash.c (fstring_existing_str): remove (moved to string.c)
(hash_aset_str): use rb_fstring_existing
* string.c (rb_fstring_existing): new, based on fstring_existing_str
(tainted_fstr_update): new
(rb_fstring_existing0): new, based on fstring_existing_str
(rb_tainted_fstring_existing): new, special case for tainted strings
(rb_str_free): delete from tainted_frozen_strings table
* test/ruby/test_optimization.rb (test_hash_reuse_fstring): new test
[ruby-core:82012] [Bug #13737]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Fix a wrong jump so replacing a byte in an ASCII-only string with an
ASCII character won't clear the coderange.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59272 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to remove leading substr [Feature #12694] [fix GH-1632]
* string.c (rb_str_delete_prefix_bang): add a new method
to remove prefix destuctively.
* string.c (rb_str_delete_prefix): add a new method
to remove prefix non-destuctively.
* test/ruby/test_string.rb: add tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59132 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_chomp_bang): check if modifiable after checking
an argument and just before modification, as it can get frozen
during the argument conversion to String.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59112 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: [DOC] clarify docs for String#split when called
with limit and capture groups.
Reported by Cichol Tsai. [ruby-core:81505] [Bug #13621]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
To convert the object implicitly, it has had two parts in convert_type() which are
1. lookink up the method's id
2. calling the method
Seems that strncmp() and strcmp() in convert_type() are slightly heavy to look up
the method's id for type conversion.
This patch will add and use internal APIs (rb_convert_type_with_id, rb_check_convert_type_with_id)
to call the method without looking up the method's id when convert the object.
Array#flatten -> 19 % up
Array#+ -> 3 % up
[ruby-dev:50024] [Bug #13341] [Fix GH-1537]
### Before
Array#flatten 104.119k (± 1.1%) i/s - 525.690k in 5.049517s
Array#+ 1.993M (± 1.8%) i/s - 10.010M in 5.024258s
### After
Array#flatten 124.005k (± 1.0%) i/s - 624.240k in 5.034477s
Array#+ 2.058M (± 4.8%) i/s - 10.302M in 5.019328s
### Test Code
require 'benchmark/ips'
class Foo
def to_ary
[1,2,3]
end
end
Benchmark.ips do |x|
ary = []
100.times { |i| ary << i }
array = [ary]
x.report "Array#flatten" do |i|
i.times { array.flatten }
end
x.report "Array#+" do |i|
obj = Foo.new
i.times { array + obj }
end
end
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* vm_insnhelper.c (rb_eql_opt): should call #eql? on Float and
String, not #==.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58882 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_crypt): struct crypt_data defined in
missing/crypt.h is small enough.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58866 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* debug_counter.h: add the following counters to measure object types.
obj_free: freed count
obj_str_ptr: freed count of Strings they have extra buff.
obj_str_embed: freed count of Strings they don't have extra buff.
obj_str_shared: freed count of Strings they have shared extra buff.
obj_str_nofree: freed count of Strings they are marked as nofree.
obj_str_fstr: freed count of Strings they are marked as fstr.
obj_ary_ptr: freed count of Arrays they have extra buff.
obj_ary_embed: freed count of Arrays they don't have extra buff.
obj_obj_ptr: freed count of Objects (T_OBJECT) they have extra buff.
obj_obj_embed: freed count of Objects they don't have extra buff.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58865 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
"struct crypt_data" is 131232 bytes on x86-64 GNU/Linux,
making it unsafe to use tiny Fiber stack sizes.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58864 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: make String#{casecmp,casecmp?} return nil for
non-string arguments instead of raising a TypeError.
* test/ruby/test_string.rb: add tests.
Reported by Marcus Stollsteimer. Based on a patch by Shingo Morita.
[ruby-core:80145] [Bug #13312]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58837 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_external_str_new_with_enc): cut down intermediate
string for conversion source, by appending with conversion.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58709 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_external_str_new_with_enc): fix the case of
conversion failure. when conversion failed for some reason,
just ignores the default internal encoding and returns in the
given encoding.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58705 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_external_str_new_with_enc): cut down intermediate
string for conversion source, by appending with conversion.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58703 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_unicode_normalize): remove bare Unicode. do
not assume that all compilers can handle UTF-8.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58688 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: [DOC] add example for String#match with pos argument.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58669 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: [DOC] adopt call-seq's for Symbol#{match,match?} from
String methods; other small improvements for Symbol docs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58668 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (unicode_normalize_common): aggregation type cannot be
initialized with dynamic values, in C89.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58621 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
In string.c, replace hand-written argument count check by call to rb_scan_args.
This allows to use rb_funcallv once, rather than using rb_funcall twice.
Thanks to Hanmac (Hans Mackowiak) for the idea, see
https://bugs.ruby-lang.org/issues/11078#note-7.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58618 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
In string.c, refactor the common parts (requiring of unicode_normalize/normalize.rb,
check of number of arguments) of the unicode normalization functions
(rb_str_unicode_normalize, rb_str_unicode_normalize_bang, rb_str_unicode_normalized_p)
into the new function unicode_normalize_common.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58558 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/unicode_normalize.rb: Remove definition of String#unicode_normalized?
(including documentation). Leave a comment explaining that the file is now empty.
* string.c: Define String#unicode_normalized? in rb_str_unicode_normalized_p in C,
(including documentation)
* lib/unicode_normalize/normalize.rb: Remove (re)definition of
String#unicode_normalized? to avoid warnings (when $VERBOSE==true) and
problems when String is frozen
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58555 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/unicode_normalize.rb: Remove definition of String#unicode_normalize!
(including documentation)
* string.c: Define String#unicode_normalize! in rb_str_unicode_normalize_bang in C,
(including documentation)
* lib/unicode_normalize/normalize.rb: Remove (re)definition of
String#unicode_normalize! to avoid warnings (when $VERBOSE==true) and
problems when String is frozen
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58553 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lib/unicode_normalize.rb: Remove definition of String#unicode_normalize
(including documentation)
* string.c: Define String#unicode_normalize in rb_str_unicode_normalize in C,
(including documentation)
* lib/unicode_normalize/normalize.rb: Remove (re)definition of
String#unicode_normalize to avoid warnings (when $VERBOSE==true) and
problems when String is frozen
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58550 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_change_terminator_length): when called after
the content has been copied, old terminator length no longer
makes sense. use the whole usable size instead of capacity
without terminator. [ruby-core:80257] [Bug #13339]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58042 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Make sure it's clear that the returned values are not Unicode codepoints
for encodings other than UTF-8/UTF-16(BE|LE)/UTF-32(BE|LE).
[ci skip] [Bug #13321]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58000 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Anybody who hits these code paths can hit them again in the
future, so try deduplicating across multiple runs of these
methods to reduce garbage.
* string.c (str_upto_each): fstring on "%.*d"
* strftime.c (rb_strftime_with_timespec): fstring on "%0*d"
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57997 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_casecmp, str_casecmp_p): split to skip argument
check when it is a String certainly.
* string.c (sym_casecmp, sym_casecmp_p): shortcut argument checks.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_cmp_m): use rb_check_string_type for check and
conversion, instead of calling the conversion method directly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57965 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: [DOC] improve docs of Symbol#casecmp and Symbol#casecmp?
according to the similar String methods; fix RDoc markup and typos;
fix call-seq's for Symbol#{upcase,downcase,capitalize,swapcase}.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57963 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (Init_String): $; must be a GC-root, not to be
collected. [ruby-core:79582]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57958 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: [DOC] specify when String#casecmp and String#casecmp?
return nil; modify examples to better show difference to <=>;
fix RDoc markup and typos.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57886 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_update): do not use negation of LONG_MIN, which
is negative too.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57800 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_byte_substr): fix another integer overflow which
can happen only when SHARABLE_MIDDLE_SUBSTRING is enabled.
[ruby-core:79951] [Bug #13289]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57799 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_subpos): fix integer overflow which can happen
only when SHARABLE_MIDDLE_SUBSTRING is enabled. incorpolate
https://github.com/mruby/mruby/commit/7db0786abdd243ba031e24683f
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57797 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: [DOC] restore documentation for String#<<
which became undocumented with r56021; fix a typo.
[ruby-core:79865] [Bug #13268]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57758 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This exposes the rb_fstring internal function to return a
deduped and frozen string when a non-frozen string is given.
This is useful for writing all sorts of record processing key
values maybe stored, but certain keys and values are often
duplicated at a high frequency, so memory savings can
noticeable.
Use cases are many:
* email/NNTP header processing
There are some standard header keys everybody uses
(From/To/Cc/Date/Subject/Received/Message-ID/References/In-Reply-To),
as well as common ones specific to a certain lists:
(ruby-core has X-Redmine-* headers)
It is also useful to dedupe values, as most inboxes have
multiple messages from the same sender, or MUA.
* package management systems -
things like RubyGems stores identical strings for licenses,
dependency names, author names/emails, etc
* HTTP headers/trailers -
standard headers (Host/Accept/Accept-Encoding/User-Agent/...)
are common, but there are also uncommon ones.
Values may be deduped, as well, as it is likely a user
agent will make multiple/parallel requests to the same
server.
* version control systems -
this can be useful for deduplicating names of frequent
committers (like "nobu" :)
In linux.git and git.git, there are also common
trailers such as Signed-Off-By/Acked-by/Reviewed-by/Fixes/...
as well as less common ones.
* audio metadata -
There are commonly used tags (Artist/Album/Title/Tracknumber),
but Vorbis comments allows arbitrary key values to be stored.
Music collections contain songs by the same artist or mutiple
songs from the same album, so deduplicating values will be
helpful there, too.
* JSON, YAML, XML, HTML processing
Certain fields, tags and attributes are commonly used
across the same and multiple documents
There is no security concern in this being a DoS vector by
causing immortal strings. The fstring table is not a GC-root
and not walked during the mark phase. GC-able dynamic symbols
since Ruby 2.2 are handled in the same manner, and that
implementation also relies on the non-immortality of fstrings.
[Feature #13077] [ruby-core:79663]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57698 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (str_shared_replace): use RUBY_ASSERT for
pre-condition.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57628 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
nobu
tadd
rhe
stomar
watson1978
svn
sonots
hsbt
naruse
glass
kazu
normal
k0kubun
ko1
duerst
ktsj
sorah