of using fixed index value.
* enc/Makefile.in (encsrcdir): make US-ASCII built-in.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15047 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/make_encdb.rb (enc_name_list): constified.
* enc/make_encdb.rb (enc_init_db): moved some functions to encoding.c.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15023 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/sjis.c: move to enc/shift_jis.c, to make encoding name equal to
filename for convinience of loading lib.
* enc/shift_jis.c: moved from enc/sjis.c.
* common.mk: follows enc/shift_jis.c.
* enc/Makefile.in: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15014 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* regenc.h (ENC_REPLICATE, ENC_ALIAS): added for defining replica
encoding and encoding alias.
* encoding.c (rb_enc_init): move alias definitions to enc/*.c.
(rb_enc_find_index): search original of replica and alias when no
encoding library.
(rb_enc_name_list, rb_enc_aliases_enc_i, rb_enc_aliases_str_i,
rb_enc_aliases, Encoding.name_list, Encoding.aliases): added.
(Init_Encoding): init encdb.
* enc/ascii.c, enc/us_ascii.c, enc/euc_jp.c, enc/sjis.c:
add replica encoding and encoding alias difinition.
* common.mk (dist-clean-local): add rule for remvoe encdb.h.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15007 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
ruby_encoding_index to avoid linear search in rb_enc_to_index.
* include/ruby/encoding.h (rb_enc_to_index): macro defined to use
ruby_encoding_index.
* encoding.c (rb_enc_to_index): removed.
(enc_register_at): initialize ruby_encoding_index member.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14931 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
encoding.
* include/ruby/encoding.h (rb_enc_codepoint): macro is replaced as a
declaration.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14524 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
ruby.c, transcode.c: rename rb_ascii_encoding. to
rb_ascii8bit_encoding. rb_ascii_encoding is ambiguous with
ASCII-8BIT and US-ASCII.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14504 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* io.c (rb_io_binmode): specifies encoding to ASCII-8BIT (binary).
* io.c (rb_io_s_read): IO should be in binary mode when offset is
specified.
* encoding.c (rb_to_encoding): returns default encoding if no
corresponding encoding found.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14411 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c (rb_enc_find_index): replace non-alphanumeric chars with
underscores, so that initialize function can be called.
* ruby.c (proc_options, process_options): finds encoding after
load_path is initialized.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14399 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c: use langinfo.h only if available.
* main.c: use locale.h only if available.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c (Init_Encoding): ISO-8859-1 is no longer a replica.
* regenc.h (OnigEncodingDefine): names of extension and encoding can
differ.
* enc/Makefile.in: always shared.
* enc/depend (deffile): should not upcase.
* enc/{ascii,euc_jp,sjis,utf8,iso_8859_{1..16}}.c: fix for Init.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14376 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
receiver) would have higher priority in encoding detection.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14220 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
(rb_enc_nth): don't check the return value of rb_enc_mbclen.
(rb_enc_strlen): ditto.
(rb_enc_precise_mbclen): return needmore(1) if e <= p.
(rb_enc_get_ascii): new function for extracting ASCII character.
* include/ruby/encoding.h (rb_enc_get_ascii): declared.
* include/ruby/regex.h (ismbchar): removed.
* re.c (rb_reg_expr_str): use rb_enc_get_ascii.
(unescape_escaped_nonascii): use rb_enc_precise_mbclen to determine
the termination of escaped non-ASCII character.
(unescape_nonascii): use rb_enc_precise_mbclen.
(rb_reg_quote): use rb_enc_get_ascii.
(rb_reg_regsub): use rb_enc_get_ascii.
* string.c (rb_str_reverse) don't check the return value of
rb_enc_mbclen.
(rb_str_split_m): don't call rb_enc_mbclen with e <= p.
* parse.y (is_identchar): use ISASCII.
(parser_ismbchar): removed.
(parser_precise_mbclen): new macro.
(parser_isascii): new macro.
(parser_tokadd_mbchar): use parser_precise_mbclen to check invalid
character precisely.
(parser_tokadd_string): use parser_isascii.
(parser_yylex): ditto.
(is_special_global_name): don't call is_identchar with e <= p.
(rb_enc_symname_p): ditto.
[ruby-dev:32455]
* ext/tk/sample/tkextlib/vu/canvSticker2.rb: remove coding cookie
because the encoding is not UTF-8. [ruby-dev:32475]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
validation.
* include/ruby/encoding.h (rb_enc_precise_mbclen): declared.
(MBCLEN_CHARFOUND): new macro.
(MBCLEN_INVALID): new macro.
(MBCLEN_NEEDMORE): new macro.
* include/ruby/oniguruma.h (OnigEncodingTypeST): replace mbc_enc_len
by precise_mbc_enc_len.
(ONIGENC_PRECISE_MBC_ENC_LEN): new macro.
(ONIGENC_CONSTRUCT_MBCLEN_CHARFOUND): new macro.
(ONIGENC_CONSTRUCT_MBCLEN_INVALID): new macro.
(ONIGENC_CONSTRUCT_MBCLEN_NEEDMORE): new macro.
(ONIGENC_MBCLEN_CHARFOUND): new macro.
(ONIGENC_MBCLEN_INVALID): new macro.
(ONIGENC_MBCLEN_NEEDMORE): new macro.
(ONIGENC_MBC_ENC_LEN): use ONIGENC_PRECISE_MBC_ENC_LEN.
* enc/euc_jp.c: validation implemented.
* enc/sjis.c: ditto.
* enc/utf8.c: ditto.
* string.c (rb_str_inspect): use rb_enc_precise_mbclen for invalid
encoding.
(rb_str_valid_encoding_p): new method String#valid_encoding?.
* io.c (rb_io_getc): use rb_enc_precise_mbclen.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14119 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rename ENC_CODERANGE_SINGLE to ENC_CODERANGE_7BIT.
rename ENC_CODERANGE_MULTI to ENC_CODERANGE_8BIT.
Because single byte 8bit character, such as Shift_JIS 1byte katakana,
is represented by ENC_CODERANGE_MULTI even if it is not multi byte.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14027 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
behavior. [ruby-dev:32192]
* encoding.c (enc_name): Encoding should not rely on ENCODING in
the FL_USERS flags.
* encoding.c (rb_enc_from_encoding): do not call rb_enc_associate
for encoding itself.
* encoding.c (enc_register_at): ditto.
* marshal.c (r_ivar): do not set real instance variable for
encoding data associated.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13848 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
enc_table_alias to a name-to-index hash.
* encoding.c (rb_enc_init): use upper case names for aliases to use as
constant names.
* encoding.c (enc_find): allow symbols.
* encoding.c (Init_Encoding): define encoding constants.
* st.c (strcasehash): fix wrong code range condition.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13731 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Flanagan <david AT davidflanagan.com>. [ruby-core:12664]
* encoding.c (enc_dump, enc_load): marshaling feature. a patch from
David Flanagan. [ruby-core:12665]
* encoding.c (Init_Encoding): undefine allocator of Encoding.
[ruby-core:12665], [ruby-core:12666]
* test/ruby/test_encoding.rb: tests for Encoding from David Flanagan
[ruby-core:12665]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13697 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ascii.c (OnigEncodingASCII): change the name US-ASCII to ASCII-8BIT.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13670 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c (rb_enc_alias): check if original name is registered.
* encoding.c (rb_enc_init): register in same order as kcode options in
re.c. added new aliases.
* string.c (rb_str_force_encoding): check if valid encoding name.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13643 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* re.c (Init_Regexp): new method Regexp#encoding.
* string.c (str_encoding): moved to encoding.c
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13613 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c (rb_enc_find_index): search the encoding which has the
given name and return its index if found, or -1.
* st.c (type_strcasehash): case-insensitive string hash type.
* string.c (rb_str_force_encoding): force encoding of self. this name
comes from [ruby-dev:31894] by Martin Duerst. [ruby-dev:31744]
* include/ruby/encoding.h (rb_enc_find_index, rb_enc_associate_index):
prototyped.
* include/ruby/encoding.h (rb_enc_isctype): direct interface to ctype.
* include/ruby/st.h (st_init_strcasetable): prototyped.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13556 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
object is encoding capable. [ruby-dev:31780]
* string.c (rb_str_subpat_set): check for if the argument is a String.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
matz
naruse
nobu
akr
usa