ruby_encoding_index to avoid linear search in rb_enc_to_index.
* include/ruby/encoding.h (rb_enc_to_index): macro defined to use
ruby_encoding_index.
* encoding.c (rb_enc_to_index): removed.
(enc_register_at): initialize ruby_encoding_index member.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14931 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (coderange_scan): extracted from rb_enc_str_coderange.
(rb_enc_str_coderange): use coderange_scan.
(rb_str_shared_replace): copy encoding and coderange.
(rb_enc_str_buf_cat): new function for linear complexity string
accumulation with encoding.
(rb_str_sub_bang): don't conflict substituted part and replacement.
(str_gsub): use rb_enc_str_buf_cat.
(rb_str_clear): clear coderange.
* re.c (rb_reg_regsub): use rb_enc_str_buf_cat.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14910 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
string, using gcc's __builtin_constant_p and statement expression.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14888 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
avoid "i".to_i(36) cause 0 under tr_TR locale.
This is newly implemented, not a copy of missing/strtoul.c.
* include/ruby/ruby.h (ruby_strtoul): declared.
(STRTOUL): defined to use ruby_strtoul.
* bignum.c, pack.c, ext/socket/socket.c: use STRTOUL.
* configure.in (strtoul): don't check.
* missing/strtoul.c: removed.
* include/ruby/missing.h (strtoul): removed.
* common.mk (strtoul.o): removed.
* LEGAL (missing/strtoul.c): removed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14850 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* sprintf.c (rb_enc_sprintf, rb_enc_vsprintf): new functions to format
arguments with encoding.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14806 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* include/ruby/encoding.h (rb_enc_right_char_head): ditto.
* io.c (appendline): does multibyte RS search in the function.
* io.c (prepare_getline_args): RS may be nil.
* io.c (rb_io_getc): should process character based on external
encoding, when transcoding required.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14619 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* load.c (search_required): returns path too if feature is being
loaded. [ruby-dev:32048] [TODO: refactoring]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14586 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
encoding.
* include/ruby/encoding.h (rb_enc_codepoint): macro is replaced as a
declaration.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14524 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
ruby.c, transcode.c: rename rb_ascii_encoding. to
rb_ascii8bit_encoding. rb_ascii_encoding is ambiguous with
ASCII-8BIT and US-ASCII.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14504 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* include/ruby/ruby.h (rb_block_call_func): function to be called back
as block.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14416 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
(rb_enc_nth): don't check the return value of rb_enc_mbclen.
(rb_enc_strlen): ditto.
(rb_enc_precise_mbclen): return needmore(1) if e <= p.
(rb_enc_get_ascii): new function for extracting ASCII character.
* include/ruby/encoding.h (rb_enc_get_ascii): declared.
* include/ruby/regex.h (ismbchar): removed.
* re.c (rb_reg_expr_str): use rb_enc_get_ascii.
(unescape_escaped_nonascii): use rb_enc_precise_mbclen to determine
the termination of escaped non-ASCII character.
(unescape_nonascii): use rb_enc_precise_mbclen.
(rb_reg_quote): use rb_enc_get_ascii.
(rb_reg_regsub): use rb_enc_get_ascii.
* string.c (rb_str_reverse) don't check the return value of
rb_enc_mbclen.
(rb_str_split_m): don't call rb_enc_mbclen with e <= p.
* parse.y (is_identchar): use ISASCII.
(parser_ismbchar): removed.
(parser_precise_mbclen): new macro.
(parser_isascii): new macro.
(parser_tokadd_mbchar): use parser_precise_mbclen to check invalid
character precisely.
(parser_tokadd_string): use parser_isascii.
(parser_yylex): ditto.
(is_special_global_name): don't call is_identchar with e <= p.
(rb_enc_symname_p): ditto.
[ruby-dev:32455]
* ext/tk/sample/tkextlib/vu/canvSticker2.rb: remove coding cookie
because the encoding is not UTF-8. [ruby-dev:32475]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
validation.
* include/ruby/encoding.h (rb_enc_precise_mbclen): declared.
(MBCLEN_CHARFOUND): new macro.
(MBCLEN_INVALID): new macro.
(MBCLEN_NEEDMORE): new macro.
* include/ruby/oniguruma.h (OnigEncodingTypeST): replace mbc_enc_len
by precise_mbc_enc_len.
(ONIGENC_PRECISE_MBC_ENC_LEN): new macro.
(ONIGENC_CONSTRUCT_MBCLEN_CHARFOUND): new macro.
(ONIGENC_CONSTRUCT_MBCLEN_INVALID): new macro.
(ONIGENC_CONSTRUCT_MBCLEN_NEEDMORE): new macro.
(ONIGENC_MBCLEN_CHARFOUND): new macro.
(ONIGENC_MBCLEN_INVALID): new macro.
(ONIGENC_MBCLEN_NEEDMORE): new macro.
(ONIGENC_MBC_ENC_LEN): use ONIGENC_PRECISE_MBC_ENC_LEN.
* enc/euc_jp.c: validation implemented.
* enc/sjis.c: ditto.
* enc/utf8.c: ditto.
* string.c (rb_str_inspect): use rb_enc_precise_mbclen for invalid
encoding.
(rb_str_valid_encoding_p): new method String#valid_encoding?.
* io.c (rb_io_getc): use rb_enc_precise_mbclen.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14119 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* re.c (rb_reg_preprocess): new function for dynamic regexp with
\u{} such as Regexp.new("\\u{6666}").
(rb_reg_prepare_re): preprocess regexp for recompiling.
(read_escaped_byte): new function.
(unescape_escaped_nonascii): new function.
(append_utf8): new function.
(unescape_unicode_list): new function.
(unescape_unicode_bmp): new function.
(unescape_nonascii): new function.
(rb_reg_initialize): preprocess regexp.
* pack.c (rb_uv_to_utf8): renamed from uv_to_utf8.
* parse.y (STR_NEW3): take func instead of has8 and hasmb.
(parser_str_new): use default coderange mechanism except for regexp.
(parser_tokadd_utf8): copy regexp source as-is.
(parser_read_escape): UTF-8 stuff removed.
(parser_tokadd_escape): has8bit and hasmb removed.
(parser_tokadd_string): fix 8-bit single byte character with \u.
(parser_parse_string): has8bit and hasmb removed.
(parser_here_document): has8bit and hasmb removed.
(parser_yylex): call parser_tokadd_utf8 instead of read_escape for
UTF-8 character.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14072 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
naruse
akr
nobu
matz
shugo
ko1
davidflanagan