github
/
ruby
зеркало из https://github.com/github/ruby.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
45 701 коммит 219 Ветки 1 255 Теги 390 MiB
Граф коммитов

338 Коммитов

Автор SHA1 Сообщение Дата
matz 44cd8e457b * regparse.c (PINC): use optimized enclen() instead of 
ONIGENC_MBC_ENC_LEN().

* regparse.c (PFETCH): ditto.

* regparse.c (PFETCH): small optimization.

* regexec.c (slow_search): single byte encoding optimization.

* regenc.h (enclen): avoid calling function when encoding's
  min_len == max_len.

* re.c (rb_reg_regsub): rb_enc_ascget() optimization for single
  byte encoding.

* re.c (rb_reg_search): avoid allocating new re_registers if we
  already have MatchData.

* re.c (match_init_copy): avoid unnecessary onig_region_free()
  before onig_region_copy. 

* encoding.c (rb_enc_get_index): remove implicit enc_capable check
  each time.

* encoding.c (rb_enc_set_index): ditto.

* encoding.c (enc_compatible_p): small refactoring.

* include/ruby/encoding.h (rb_enc_dummy_p): inline
  rb_enc_dummy_p() and export related code.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16477 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-05-19 08:25:03 +00:00
nobu 1369cfd16e * encoding.c (enc_init_db): moved to enc/encdb.c. 
* transcode.c (init_transcoder_table): moved to enc/trans/transdb.c.

* enc/depend (enc/encdb.o enc/trans/transdb.o): depend on
  corresponding headers.

* common.mk (COMMONOBJS): moved transcode.o from OBJS


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-04-07 06:51:33 +00:00
nobu 555eb7ded4 * encoding.c (enc_check_encoding): should not load autoloaded encoding 
directly, instead use rb_enc_find_index() which deal with alias and
  replica.  [ruby-core:15957]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15798 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-03-19 05:08:49 +00:00
matz 2694b2f937 * encoding.c (rb_enc_associate_index): pass unnecessary enc_capable(). 
* string.c (rb_str_cmp): reduce invocation of rb_enc_compatible().

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15782 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-03-15 09:19:38 +00:00
nobu cd89edc065 * encoding.c (rb_enc_dummy_p): bootstrap encodings can not be dummy. 
* encoding.c (rb_enc_ascget): no needs to call rb_enc_precise_mbclen()
  twice.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15645 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-29 15:29:09 +00:00
naruse 5eafc201d8 * encoding.c (enc_capable): IMMEDIATE_P doesn't include Qnil and Qfalse. 
use SPECIAL_CONST_P.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15632 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-28 13:22:29 +00:00
naruse 6f99342c92 * encoding.c (enc_find): check type of argument and convert to String 
if it is StringValue. [ruby-cvs:22866]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15631 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-28 11:58:11 +00:00
duerst ad3577b59b Mon Feb 25 10:41:41 2008 Martin Duerst <duerst@it.aoyama.ac.jp> 
* encoding.c (Encoding#dummy): minor grammatical fixes
	  in rdoc documentation.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15595 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-25 01:42:47 +00:00
akr 81fc1cf201 * encoding.c (rb_enc_mbclen): return minlen instead of 1 when 
a character is not found properly.

* string.c (rb_enc_strlen): round up string length with fixed
  multibyte encoding such as UTF-32.
  (rb_enc_strlen_cr): ditto.
  (rb_str_substr): fix substring with fixed multibyte encoding.
  (rb_str_justify): check number of characters.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15573 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-22 06:52:54 +00:00
naruse b62df564a6 * string.c (rb_str_times): empty string's coderange is CODERANGE_7BIT. 
* string.c (rb_str_substr): ditto.

* encoding.c (rb_enc_compatible): empty string is compatible with not
  only nonasciicompatible strings. [ruby-dev:33895]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15566 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-21 19:54:48 +00:00
naruse 1e0958098f * encoding.c (rb_enc_associate_index): doesn't clear coderange 
when new encoding equals to old one.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15554 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-20 10:44:28 +00:00
akr 8bcfa698a0 * encoding.c (rb_enc_compatible): check encoding incapable arguments. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15510 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-16 11:43:12 +00:00
akr 7eeba5f440 * encoding.c (rb_enc_compatible): empty strings are always compatible. 
* string.c (rb_enc_cr_str_buf_cat): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15506 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-16 09:02:12 +00:00
akr af75cc01bc * encoding.c (rb_enc_strlen): moved to string.c. 
* string.c (rb_enc_strlen): use search_nonascii.
  (str_strlen): don't use search_nonascii.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15498 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-16 02:55:08 +00:00
akr 0831222a91 * encoding.c (rb_enc_nth): moved to string.c. 
* string.c (rb_enc_nth): moved from string.c.  use search_nonascii
  for ASCII compatible string.
  (str_nth): wrong optimization removed to fix
  "a".force_encoding("EUC-JP").slice!(0,10) returns
  "a\x00\x00\x00\x00\x00\x00\x00\x00\x00"


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15492 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-15 18:48:27 +00:00
matz 8b09f7015a * string.c (str_strlen): use search_nonascii() for performance. 
* string.c (str_nth): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15486 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-02-15 08:14:40 +00:00
nobu f3ed7fa233 * encoding.c (encdb_{replicate,alias,dummy,declare}): define only if 
NO_ENCDB_H is not defined.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15316 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-29 09:05:50 +00:00
naruse 315196bc7a * dmyencoding.c, encoding.c (enc_init_db, NO_ENCDB_H): 
miniruby doesn't use encdb.

* common.mk: encdb.h use miniruby.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15314 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-29 08:48:35 +00:00
akr fc208c1bd5 * include/ruby/oniguruma.h: precise mbclen API redesigned to avoid 
inline functions.
  (onigenc_mbclen_charfound): removed.
  (onigenc_mbclen_needmore): removed.
  (onigenc_mbclen_recover): removed.
  (ONIGENC_MBCLEN_CHARFOUND): removed.
  (ONIGENC_MBCLEN_CHARFOUND_P): defined.
  (ONIGENC_MBCLEN_CHARFOUND_LEN): defined.
  (ONIGENC_MBCLEN_INVALID): removed.
  (ONIGENC_MBCLEN_INVALID_P): defined.
  (ONIGENC_MBCLEN_NEEDMORE): removed.
  (ONIGENC_MBCLEN_NEEDMORE_P): defined.
  (ONIGENC_MBCLEN_NEEDMORE_LEN): defined.
  (ONIGENC_MBC_ENC_LEN): use onigenc_mbclen_approximate.

* regenc.c (onigenc_mbclen_approximate): defined.

* include/ruby/encoding.h (MBCLEN_CHARFOUND): removed.
  (MBCLEN_INVALID): removed.
  (MBCLEN_NEEDMORE): removed.
  (MBCLEN_CHARFOUND_P): defined.
  (MBCLEN_INVALID_P): defined.
  (MBCLEN_NEEDMORE_P): defined.
  (MBCLEN_CHARFOUND_LEN): defined.
  (MBCLEN_NEEDMORE_LEN): defined.

* encoding.c: use new API.

* re.c: ditto.

* string.c: ditto.

* parse.y: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15280 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-27 14:27:07 +00:00
naruse 42dcda08ae * string.c (rb_str_usascii_new{,2}: defined. 
(rb_str_new): set US-ASCII and ENC_CODERANGE_7BIT when empty
  string.

* encoding.c (rb_usascii_encoding, rb_usascii_encindex): defined.
  (rb_enc_inspect, enc_name, rb_locale_charmap, rb_enc_name_list_i):
  use rb_str_ascii_new.

* array.c (recursive_join, inspect_ary): ditto.

* object.c (nil_to_s, nil_inspect, true_to_s, false_to_s,
  rb_mod_to_s): ditto.

* hash.c (inspect_hash, rb_hash_inspect, rb_f_getenv, env_fetch,
  env_clear, env_to_s, env_inspect): ditto.

* numeric.c (flo_to_s, int_chr, rb_fix2str): ditto.

* bignum.c (rb_big2str): ditto.

* file.c (rb_file_ftype, rb_file_s_dirname, rb_file_s_extname,
  file_inspect_join, Init_file): ditto.

* test/ruby/test_ruby_m17n.rb: add checks for encoding of string.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15244 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-25 16:40:02 +00:00
naruse 1ccb0642ed * encoding.{c, h} (rb_usascii_encoding): added. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15194 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-23 16:03:29 +00:00
matz 707f2fd26e * encoding.c (rb_enc_compatible): wrong compatibility condition. 
[ruby-dev:33273]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15168 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-22 06:56:44 +00:00
matz 65a8185eb2 * configure.in (MINIRUBY): remove -I$(EXTOUT)/$(arch) from 
MINIRUBY since miniruby might not be able to load DLL.

* test/ruby/test_m17n.rb: move tests from bootstrap test.

* encoding.c (enc_find): should check name if ASCII compatible.

* string.c (rb_str_end_with): should check character boundary.

* encoding.c (rb_enc_compatible): encoding must be ASCII
  compatible before checking ENC_CODERANGE_7BIT.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15167 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-22 03:59:53 +00:00
akr bbeefa53ca * encoding.c (enc_new): don't free rb_encoding to avoid SEGV by 
`miniruby -e exit' on x86_64 GNU/Linux.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15127 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-19 11:43:24 +00:00
nobu b66eb53650 * encoding.c (rb_enc_name_list_i, rb_enc_aliases_enc_i): freeze 
element strings to be returned.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15110 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-18 06:56:06 +00:00
nobu 845ae33b17 * encoding.c (load_encoding): check if successfully loaded. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15107 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-18 03:25:15 +00:00
nobu 9f9ac39996 * encoding.c (rb_enc_find_index): use original encoding name to 
replicate loaded encoding instead alias.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-18 02:39:12 +00:00
matz f28118eb69 * encoding.c (rb_enc_find_index): use original encoding name to 
load DLL.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15104 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-18 01:37:27 +00:00
matz d8644f632b * encoding.c (rb_enc_aliases_enc_i): exclude non alias names from 
Encoding.aliases.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-18 01:30:41 +00:00
matz 231ab57f6f * parse.y (parser_initialize): explicitly call rb_ascii8bit_encoding(). 
* parse.y (parser_prepare): lex_input may not be have encoding (e.g. IO).

* parse.y (rb_parser_compile_string): set encoding from input string. 

* encoding.c (rb_enc_find_index): use ASCII-8BIT if loading known
  encoding failed.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-18 00:23:18 +00:00
nobu a3776588ef * encoding.c (enc_free): removed since rb_encoding may be used while 
cleanup.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15089 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-17 15:23:33 +00:00
nobu 0052259d5e * common.mk (encdb.h): give output file name to make_encdb.rb. 
* encoding.c (enc_table): simplified.

* encoding.c (enc_register_at): lazy loading.  [ruby-dev:33013]

* regenc.h (ENC_DUMMY): added.

* enc/make_encdb.rb: now emits macros only.

* enc/iso_2022_jp.h: split from encoding.c.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-17 14:56:22 +00:00
nobu d969620ea8 * encoding.c (enc_register_at): make own copy. [ruby-dev:33136] 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15081 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-16 13:45:29 +00:00
naruse edc61cf4c1 * encoding.c (ENC_REGISTER): use &OnigEncoding*. 
(ENCINDEX_UTF_8): renamed from ENCINDEX_UTF8.
  (rb_enc_init): use ENC_REGISTER.

* include/ruby/oniguruma.h (OnigEncodingUTF8, ONIG_ENCODING_UTF8):
  removed.

* enc/*.c: remove use of &encoding_*; use enc argument instead.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15067 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-15 13:36:18 +00:00
nobu 3c3b2fe143 * encoding.c (rb_locale_charmap): use ASCII-8BIT in miniruby. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15057 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-15 05:44:55 +00:00
nobu 2e8965863d * common.mk (OBJS): moved encoding.o from COMMONOBJS. 
* common.mk (dmyencoding.o): added.  [ruby-dev:33099]

* configure.in, {win,bcc}32/Makefile.sub (MINIOBJS): added
  dmyencoding.o.

* dmyencoding.c (rb_locale_charmap): returns nil for miniruby.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15052 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-14 16:19:02 +00:00
matz d9ff499bf3 * re.c (rb_char_to_option_kcode): use rb_enc_find_index() instead 
of using fixed index value.

* enc/Makefile.in (encsrcdir): make US-ASCII built-in.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15047 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-14 13:49:29 +00:00
naruse 0605d15f6a * encoding.c (rb_locale_encoding): return US-ASCII when charmap is nil. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15039 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-14 09:08:45 +00:00
naruse 21671b558c * enc/make_encdb.h: sort encoding names by original name. 
* encoding.c, enc/*.c: define replicas and aliases.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15025 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-13 14:29:12 +00:00
naruse 62c596cb36 * encoding.c: add documents. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15024 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-13 11:24:44 +00:00
nobu bb8ddbe847 * encoding.c (Init_Encoding): moved initialization from encdb.h. 
* enc/make_encdb.rb (enc_name_list): constified.

* enc/make_encdb.rb (enc_init_db): moved some functions to encoding.c.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15023 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-13 09:41:50 +00:00
naruse 513d0ca7f6 * encoding.c (ENCINDEX_EUC_JP, ENCINDEX_SJIS): removed. 
(rb_enc_init): EUC-JP and Shift_JIS are not builtin now.

* enc/Makefile.in: ditto.

* common.mk: ditto.

* ruby.c (proc_options): ditto.

* enc/shift_jis.c, enc/euc_jp.c: fixes for romove from builtin.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15016 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-13 02:51:15 +00:00
nobu 00fb802284 * encoding.c (enc_table): packed all enc_table stuff. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15015 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-13 01:21:42 +00:00
naruse 80a569906d * encoding.c (rb_enc_init): revert removing SJIS. 
* enc/sjis.c: move to enc/shift_jis.c, to make encoding name equal to
  filename for convinience of loading lib.

* enc/shift_jis.c: moved from enc/sjis.c.

* common.mk: follows enc/shift_jis.c.

* enc/Makefile.in: ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15014 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-13 01:15:32 +00:00
naruse 4261dd6ab7 * encoding.c (rb_enc_aliases): don't see enc_table_alias when it wasn't initialized yet. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-12 16:34:08 +00:00
naruse 5b9739a832 * enc/make_encdb.rb: added. search enc/*.c and make encoding database. 
* regenc.h (ENC_REPLICATE, ENC_ALIAS): added for defining replica
  encoding and encoding alias.

* encoding.c (rb_enc_init): move alias definitions to enc/*.c.
  (rb_enc_find_index): search original of replica and alias when no
  encoding library.
  (rb_enc_name_list, rb_enc_aliases_enc_i, rb_enc_aliases_str_i,
   rb_enc_aliases, Encoding.name_list, Encoding.aliases): added.
  (Init_Encoding): init encdb.

* enc/ascii.c, enc/us_ascii.c, enc/euc_jp.c, enc/sjis.c:
  add replica encoding and encoding alias difinition.

* common.mk (dist-clean-local): add rule for remvoe encdb.h.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15007 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-12 16:03:51 +00:00
naruse 9e35ba455d * encoding (set_base_encoding): must use rb_enc_dummy_p. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14982 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-11 00:39:44 +00:00
nobu 78900b98b3 * encoding.c (rb_to_encoding_index, rb_to_encoding): disallow nil. 
[ruby-dev:33003]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14981 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-10 21:13:21 +00:00
naruse ad4fb57a3a * encoding.c (rb_enc_replicate): replica of dummy is a dummy. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14969 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-09 13:06:00 +00:00
naruse 5cd70fa5e9 * encoding.c (rb_enc_init): UTF-{16,32}{BE,LE} are not builtin. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14957 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-08 14:56:50 +00:00
naruse ed540e8bdf * encoding.c, Makefile.in, include/ruby/oniguruma.h, 
enc/Makefile.in: fix rules for UTF-{16,32}{BE,LE}.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14956 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-08 13:35:24 +00:00
akr 4e4d4331ca * include/ruby/oniguruma.h (OnigEncodingType): new member 
ruby_encoding_index to avoid linear search in rb_enc_to_index.

* include/ruby/encoding.h (rb_enc_to_index): macro defined to use
  ruby_encoding_index.

* encoding.c (rb_enc_to_index): removed.
  (enc_register_at): initialize ruby_encoding_index member.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14931 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-07 07:48:24 +00:00
akr 063beac343 * encoding.c (rb_enc_internal_get_index): extracted from 
rb_enc_get_index.
  (rb_enc_internal_set_index): extracted from rb_enc_associate_index

* include/ruby/encoding.h (ENCODING_SET): work over ENCODING_INLINE_MAX.
  (ENCODING_GET): ditto.
  (ENCODING_IS_ASCII8BIT): defined.
  (ENCODING_CODERANGE_SET): defined.

* re.c (rb_reg_fixed_encoding_p): use ENCODING_IS_ASCII8BIT.

* string.c (rb_enc_str_buf_cat): use ENCODING_IS_ASCII8BIT.

* parse.y (reg_fragment_setenc_gen): use ENCODING_IS_ASCII8BIT.

* marshal.c (has_ivars): use ENCODING_IS_ASCII8BIT.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14922 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-07 02:49:01 +00:00
akr 6cdef2dc7e * $Date$ keyword removed to avoid inclusion of locale dependent 
string.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-06 15:49:38 +00:00
akr 8702c4d0d7 update rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14890 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-05 03:35:13 +00:00
akr 9c2204972b * encoding.c (Init_Encoding): alias csWindows31J to Windows-31J. 
IE6 accepts csWindows31J but Windows-31J.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14889 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-05 02:51:24 +00:00
akr 059fe7997e * encoding.c: (rb_tolower, rb_toupper): body was exchanged. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14867 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-03 09:40:00 +00:00
akr 5f237d7903 * encoding.c (rb_isalnum): defined. 
(rb_isalpha): ditto.
  (rb_isblank): ditto.
  (rb_iscntrl): ditto.
  (rb_isdigit): ditto.
  (rb_isgraph): ditto.
  (rb_islower): ditto.
  (rb_isprint): ditto.
  (rb_ispunct): ditto.
  (rb_isspace): ditto.
  (rb_isupper): ditto.
  (rb_isxdigit): ditto.
  (rb_tolower): ditto.
  (rb_toupper): ditto.

* include/ruby/ruby.h: don't include include/ruby/encoding.h.
  (rb_isascii): defined.
  (rb_isalnum): declared.
  (rb_isalpha): ditto.
  (rb_isblank): ditto.
  (rb_iscntrl): ditto.
  (rb_isdigit): ditto.
  (rb_isgraph): ditto.
  (rb_islower): ditto.
  (rb_isprint): ditto.
  (rb_ispunct): ditto.
  (rb_isspace): ditto.
  (rb_isupper): ditto.
  (rb_isxdigit): ditto.
  (rb_tolower): ditto.
  (rb_toupper): ditto.
  (ISASCII): simplified.
  (ISPRINT): ditto.
  (ISSPACE): ditto.
  (ISUPPER): ditto.
  (ISLOWER): ditto.
  (ISALNUM): ditto.
  (ISALPHA): ditto.
  (ISDIGIT): ditto.
  (ISXDIGIT): ditto.
  (TOUPPER): ditto.
  (TOLOWER): ditto.

* include/ruby/encoding.h (rb_isascii): removed.
  (rb_isalnum): ditto.
  (rb_isalpha): ditto.
  (rb_isblank): ditto.
  (rb_iscntrl): ditto.
  (rb_isdigit): ditto.
  (rb_isgraph): ditto.
  (rb_islower): ditto.
  (rb_isprint): ditto.
  (rb_ispunct): ditto.
  (rb_isspace): ditto.
  (rb_isupper): ditto.
  (rb_isxdigit): ditto.
  (rb_tolower): ditto.
  (rb_toupper): ditto.

* common.mk: dependency updated.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14865 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-03 08:44:01 +00:00
akr 041e829127 * include/ruby/encoding.h (rb_isascii): defined. 
(rb_isalnum): ditto.
  (rb_isalpha): ditto.
  (rb_isblank): ditto.
  (rb_iscntrl): ditto.
  (rb_isdigit): ditto.
  (rb_isgraph): ditto.
  (rb_islower): ditto.
  (rb_isprint): ditto.
  (rb_ispunct): ditto.
  (rb_isspace): ditto.
  (rb_isupper): ditto.
  (rb_isxdigit): ditto.
  (rb_tolower): ditto.
  (rb_toupper): ditto.

* include/ruby/st.h (st_strcasecmp): declared.
  (st_strncasecmp): ditto.

* st.c (type_strcasehash): use st_strcasecmp instead of strcasecmp.
  (st_strcasecmp): defined.
  (st_strncasecmp): ditto.

* include/ruby/ruby.h: include include/ruby/encoding.h.
  (ISASCII): use rb_isascii.
  (ISPRINT): use rb_isprint.
  (ISSPACE): use rb_isspace.
  (ISUPPER): use rb_isupper.
  (ISLOWER): use rb_islower.
  (ISALNUM): use rb_isalnum.
  (ISALPHA): use rb_isalpha.
  (ISDIGIT): use rb_isdigit.
  (ISXDIGIT): use rb_isxdigit.
  (TOUPPER): defined.
  (TOLOWER): ditto.
  (STRCASECMP): ditto.
  (STRNCASECMP): ditto.

* dir.c, encoding.c, file.c, hash.c, process.c, ruby.c, time.c,
  transcode.c, ext/readline/readline.c: use locale insensitive
  functions.  [ruby-core:14662]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14829 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-01 12:24:04 +00:00
nobu 1701a536d1 * encoding.c (rb_to_encoding): raises for non-nil, non-encoding, 
non-string object.  [ruby-core:14634]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14822 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-31 16:29:06 +00:00
akr e21907e0f8 fix typos. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-31 05:52:59 +00:00
nobu 1ed2aad958 * encoding.c (Init_Encoding): registered rb_encoding differs from 
ONIG_ENCODINGs.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14794 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-30 08:43:56 +00:00
usa 7c1cab2a4f * encoding.c (Init_encoding): register Windows-31J and its alias. 
[ruby-dev:32843]

	* ruby.c (proc_options): -Ks options means Windows-31J, not Shift_JIS.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-29 21:34:15 +00:00
usa eb1014e4ae * encoding.c (rb_locale_encoding): should check return value from 
rb_locale_charmap().

	* ruby.c (locale_encoding): removed.

	* ruby.c (process_options): use rb_locale_encoding() instead of
	  locale_encoding().

	* ext/readline/readline.c (readline_readline): use locale encoding
	  instead of input IO's encoding. [ruby-dev:32872]



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14770 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-28 10:41:52 +00:00
akr 371977ff3d * encoding.c (rb_locale_encoding): defined. 
* include/ruby/encoding.h (rb_locale_encoding): declared.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14768 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-28 10:12:13 +00:00
nobu c3c58513d7 * encoding.c (set_base_encoding, enc_base_encoding): renamed 
based_encoding as base_encoding.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14756 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-28 06:10:49 +00:00
akr ce2b982cd2 * encoding.c (rb_enc_codelen): show codepoint in error message. 
* include/ruby/encoding.h (rb_enc_codelen): comment it returns
  positive integer.

* string.c (rb_str_concat): rb_enc_codelen doesn't return 0.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14733 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-27 06:27:39 +00:00
akr cd2fd33edf * encoding.c (rb_enc_codelen): error message refined. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14732 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-27 06:19:09 +00:00
akr 8bb84c92d4 * encoding.c (rb_enc_check): show encodings in error message. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14731 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-27 06:12:19 +00:00
nobu 0ee5a49dd4 * encoding.h (rb_enc_mbc_to_codepoint): wrapper for 
ONIGENC_MBC_TO_CODE().

* string.c (rb_str_succ): deal with invalid sequence as binary.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14692 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-25 10:01:06 +00:00
akr ead329fd8e * string.c (rb_str_inspect): don't call rb_enc_codepoint with empty 
string.  fix '#'.inspect.

* encoding.c (rb_enc_codepoint): raise on empty string.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14687 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-25 09:07:32 +00:00
nobu 75e4572768 * encoding.c (rb_enc_register): do not use based_encoding to check if 
dummy encoding.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14674 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-25 06:59:53 +00:00
akr cf36df97fb * encoding.c (rb_enc_codepoint): implemented to raise invalid 
encoding.

* include/ruby/encoding.h (rb_enc_codepoint): macro is replaced as a
  declaration.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14524 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-23 14:06:00 +00:00
akr 057d7369f9 add a comment for eucJP. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14518 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-23 08:55:30 +00:00
akr 090b9a4f7d * encoding.c (rb_enc_init): add eucJP as an alias of EUC-JP. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14517 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-23 08:40:37 +00:00
akr 5b809a28f8 * include/ruby/encoding.h, encoding.c, re.c, io.c, parse.y, numeric.c, 
ruby.c, transcode.c: rename rb_ascii_encoding. to
  rb_ascii8bit_encoding.  rb_ascii_encoding is ambiguous with 
  ASCII-8BIT and US-ASCII.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14504 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-22 23:47:18 +00:00
akr 5b1d121bfa document updated. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-22 15:24:23 +00:00
akr 0fc7dfedd3 add rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14478 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-22 08:11:47 +00:00
akr bcb064eb0f * regenc.c (onigenc_ascii_is_code_ctype): moved from enc/ascii.c. 
* regenc.h (onigenc_ascii_is_code_ctype): declared.

* enc/ascii.c: use onigenc_ascii_is_code_ctype.

* enc/us_ascii.c: new file for US-ASCII.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14463 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-22 05:38:33 +00:00
usa b507930297 * encoding.c (rb_locale_charmap): win32 support. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14456 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-22 02:37:40 +00:00
matz d7cc14d436 * encoding.c (rb_ascii_encoding): renamed from previous 
rb_default_encoding().

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14443 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 18:55:30 +00:00
nobu 5f2e5c07a7 * encoding.c (rb_enc_replicate): now creates first class encoding. 
* encoding.c (rb_define_dummy_encoding): always based on the default
  encoding.

* encoding.c (rb_enc_dummy_p): check if dummy.

* encoding.c (enc_inspect): shows if dummy.

* encoding.c (Init_Encoding): added dummy? method

* include/ruby/encoding.h (ENCODING_INLINE_MAX): increased.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14429 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 11:00:04 +00:00
akr 5bb7c8612f * encoding.c (Init_Encoding): use enc_name as to_s. 
(enc_inspect): renamed from enc_to_s.  add "#" at beginning.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14418 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 07:52:49 +00:00
matz e48ea0a5f3 * io.c (rb_io_s_read): allow specifying encoding explicitly. 
* io.c (rb_io_binmode): specifies encoding to ASCII-8BIT (binary).

* io.c (rb_io_s_read): IO should be in binary mode when offset is
  specified.

* encoding.c (rb_to_encoding): returns default encoding if no
  corresponding encoding found.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14411 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 07:02:55 +00:00
nobu 12df6cf7ce * encoding.c (rb_enc_init): use enc_register_at() directly. 
* encoding.c (rb_utf8_encoding): returns utf-8 encoding.

* include/ruby/encoding.h (rb_utf8_encoding): prototyped.

* parse.y (UTF8_ENC): uses rb_utf8_encoding().


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14410 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 06:59:48 +00:00
nobu 4baea7dd8e * encoding.c (rb_to_encoding_index): should return error instead of 
exception even if type is incorrect.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14405 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 06:16:02 +00:00
nobu e15390d08d * encoding.c (rb_enc_find_index): extension libraries have lower case 
names conventionally.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14400 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 04:47:00 +00:00
nobu ef0706a5bf * encoding.c (rb_enc_register): set encoding constant. 
* encoding.c (rb_enc_find_index): replace non-alphanumeric chars with
  underscores, so that initialize function can be called.

* ruby.c (proc_options, process_options): finds encoding after
  load_path is initialized.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14399 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 04:40:13 +00:00
matz 1b27cfcd6c * io.c (rb_io_external_encoding): new method. 
* encoding.c (rb_enc_from_encoding): returns Qnil for NULL
  encoding.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14397 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 04:26:38 +00:00
akr 8c68d9a152 * encoding.c (rb_locale_charmap): return nil if no locale information. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14394 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 04:09:45 +00:00
akr f2515d9938 * configure.in: check langinfo.h and locale.h. 
* encoding.c: use langinfo.h only if available.

* main.c: use locale.h only if available.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 03:03:29 +00:00
akr 0530cf9ff8 * encoding.c: include locale.h 
(rb_locale_charmap): new method Encoding.locale_charmap for
  nl_langinfo(CODESET).

* include/ruby/encoding.h (rb_locale_charmap): declared.

* main.c (main): call setlocale with LC_CTYPE.

* ruby.c (locale_encoding): use rb_locale_charmap.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14380 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 02:52:23 +00:00
nobu 817a4e3c83 * common.mk (enc.mk): depends on $(RBCONFIG) instead of rbconfig.rb. 
* encoding.c (Init_Encoding): ISO-8859-1 is no longer a replica.

* regenc.h (OnigEncodingDefine): names of extension and encoding can
  differ.

* enc/Makefile.in: always shared.

* enc/depend (deffile): should not upcase.

* enc/{ascii,euc_jp,sjis,utf8,iso_8859_{1..16}}.c: fix for Init.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14376 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-21 02:23:26 +00:00
matz ea1b9d58b9 * string.c (rb_str_index): check if substring is broken. 
* string.c (rb_str_rindex): ditto.

* string.c (rb_str_succ): should carry over.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14268 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-17 09:08:27 +00:00
nobu f2bd108d8d * configure.in (enc/Makefile): add external encoding objects list. 
* common.mk (BUILTIN_ENCOBJS): renamed from ENCOBJS.

* Makefile.in (BUILTIN_ENCOBJS): substitued by autoconf.

* enc/Makefile.in: new file to compile external encoding sources.

* encoding.c (rb_enc_find_index): auto-load external encoding objects
  as "ext/ENCODING_NAME".  [ruby-dev:32606]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14238 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-15 09:56:59 +00:00
nobu 8063edd4c9 * encoding.c (rb_enc_register, rb_enc_replicate, rb_enc_alias): check 
if already registered.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14234 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-15 03:58:02 +00:00
matz e0334fb250 * encoding.c (rb_enc_compatible): 1st argument (typically the 
receiver) would have higher priority in encoding detection.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14220 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-13 17:30:54 +00:00
matz a648fc802b * encoding.c (rb_enc_compatible): encoding should never fall back 
to ASCII-8BIT unless both encodings are ASCII-8BIT.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14217 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-13 13:44:02 +00:00
nobu e18e893846 * encoding.c (enc_get_encoding): removed. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14210 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-13 08:05:00 +00:00
nobu af69c73087 * encoding.c (rb_enc_compatible): should swap encoding indexes too. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14209 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-13 08:03:31 +00:00
matz 02501dbe91 * encoding.c (rb_enc_compatible): should not judge compatibility 
based on rb_enc_asciicompat().

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14208 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-13 07:44:52 +00:00
akr 646f27b822 * encoding.c (rb_enc_ascget): renamed from rb_enc_get_ascii. 
* include/ruby/encoding.h: follow the renaming.

* re.c: ditto. 



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14195 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-11 07:39:16 +00:00
akr 5802768b40 * encoding.c (rb_enc_get_ascii): add an argument to provide the 
length of the returned character.

* include/ruby/encoding.h (rb_enc_get_ascii): add the argument.

* re.c (rb_reg_expr_str): modify rb_enc_get_ascii call.
  (rb_reg_quote): ditto.
  (rb_reg_regsub): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14190 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-11 03:08:50 +00:00
akr 78eae5da62 * encoding.c (rb_enc_precise_mbclen): return needmore if underlying 
implementation returns a length longer than e-p.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14144 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-08 19:10:58 +00:00
akr cde5b2a9cf * encoding.c (rb_enc_mbclen): return 1 if underlying implementation 
returns a length longer than e-p.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-08 18:41:47 +00:00
akr f1b7e60cb9 * encoding.c (rb_enc_mbclen): make it never fail. 
(rb_enc_nth): don't check the return value of rb_enc_mbclen.
  (rb_enc_strlen): ditto.
  (rb_enc_precise_mbclen): return needmore(1) if e <= p.
  (rb_enc_get_ascii): new function for extracting ASCII character.

* include/ruby/encoding.h (rb_enc_get_ascii): declared.

* include/ruby/regex.h (ismbchar): removed.

* re.c (rb_reg_expr_str): use rb_enc_get_ascii.
  (unescape_escaped_nonascii): use rb_enc_precise_mbclen to determine
  the termination of escaped non-ASCII character.
  (unescape_nonascii): use rb_enc_precise_mbclen.
  (rb_reg_quote): use rb_enc_get_ascii.
  (rb_reg_regsub): use rb_enc_get_ascii.

* string.c (rb_str_reverse) don't check the return value of
  rb_enc_mbclen.
  (rb_str_split_m): don't call rb_enc_mbclen with e <= p.

* parse.y (is_identchar): use ISASCII.
  (parser_ismbchar): removed.
  (parser_precise_mbclen): new macro.
  (parser_isascii): new macro.
  (parser_tokadd_mbchar): use parser_precise_mbclen to check invalid
  character precisely.
  (parser_tokadd_string): use parser_isascii.
  (parser_yylex): ditto.
  (is_special_global_name): don't call is_identchar with e <= p.
  (rb_enc_symname_p): ditto.

  [ruby-dev:32455]

* ext/tk/sample/tkextlib/vu/canvSticker2.rb: remove coding cookie
  because the encoding is not UTF-8.  [ruby-dev:32475]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-08 02:50:43 +00:00
akr 69406aad50 * encoding.c (rb_enc_precise_mbclen): new function for mbclen with 
validation.

* include/ruby/encoding.h (rb_enc_precise_mbclen): declared.
  (MBCLEN_CHARFOUND): new macro.
  (MBCLEN_INVALID): new macro.
  (MBCLEN_NEEDMORE): new macro.

* include/ruby/oniguruma.h (OnigEncodingTypeST): replace mbc_enc_len
  by precise_mbc_enc_len.
  (ONIGENC_PRECISE_MBC_ENC_LEN): new macro.
  (ONIGENC_CONSTRUCT_MBCLEN_CHARFOUND): new macro.
  (ONIGENC_CONSTRUCT_MBCLEN_INVALID): new macro.
  (ONIGENC_CONSTRUCT_MBCLEN_NEEDMORE): new macro.
  (ONIGENC_MBCLEN_CHARFOUND): new macro.
  (ONIGENC_MBCLEN_INVALID): new macro.
  (ONIGENC_MBCLEN_NEEDMORE): new macro.
  (ONIGENC_MBC_ENC_LEN): use ONIGENC_PRECISE_MBC_ENC_LEN.

* enc/euc_jp.c: validation implemented.

* enc/sjis.c: ditto.

* enc/utf8.c: ditto.

* string.c (rb_str_inspect): use rb_enc_precise_mbclen for invalid
  encoding.
  (rb_str_valid_encoding_p): new method String#valid_encoding?.

* io.c (rb_io_getc): use rb_enc_precise_mbclen.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14119 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-06 09:28:26 +00:00
matz d5bd9d1593 * encoding.c: rename primary_encoding -> default_external (encoding). 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14069 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-12-01 14:05:53 +00:00
akr f5ee0fd521 * include/ruby/encoding.h, encoding.c, re.c, string.c, parse.y: 
rename ENC_CODERANGE_SINGLE to ENC_CODERANGE_7BIT.
  rename ENC_CODERANGE_MULTI to ENC_CODERANGE_8BIT.
  Because single byte 8bit character, such as Shift_JIS 1byte katakana,
  is represented by ENC_CODERANGE_MULTI even if it is not multi byte.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14027 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-11-27 02:21:17 +00:00
nobu 7bfed9613c * encoding.c (enc_check_encoding, rb_set_primary_encoding): ENCODING 
is no longer in FL_USERS flags.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13852 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-11-09 06:52:02 +00:00
matz 4ab384be2d * test/ruby/test_basicinstructions.rb: updated for new class 
behavior.  [ruby-dev:32192]

* encoding.c (enc_name): Encoding should not rely on ENCODING in
  the FL_USERS flags.

* encoding.c (rb_enc_from_encoding): do not call rb_enc_associate
  for encoding itself.

* encoding.c (enc_register_at): ditto.

* marshal.c (r_ivar): do not set real instance variable for
  encoding data associated.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13848 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-11-09 04:37:36 +00:00
matz 19c4d26c51 * string.c (tr_setup_table): use C array for characters that fit 
in a byte to gain performance.

* string.c (rb_str_delete_bang): ditto.

* string.c (rb_str_squeeze_bang): ditto.

* string.c (rb_str_count): ditto.

* string.c (tr_trans): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13812 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-11-03 19:04:53 +00:00
nobu b088414390 * encoding.c (rb_enc_replicate): new function to replicate encoding. 
* encoding.c (enc_based_encoding): Encoding#base_encoding returns
  based encoding of replica.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-29 08:20:13 +00:00
nobu 3dacfd3fe4 * encoding.c (rb_enc_compatible): ASCII encoding is compatible with 
ASCII-compatible encoding, even for non-string objects.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13789 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-29 08:18:36 +00:00
nobu d0cbfbd995 * encoding.c (rb_to_encoding_index, rb_to_encoding): commit miss. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13752 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-22 02:06:50 +00:00
nobu 3cb10b32bb * encoding.c (enc_check_encoding): returns index now. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13751 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-22 02:03:10 +00:00
nobu 00befb4b3f * encoding.c (rb_enc_compatible): check if two objects have compatible 
encodings.

* encoding.c (enc_compatible_p): added Encoding.compatible?.

* include/ruby/encoding.h (rb_enc_compatible): prototype.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13750 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-22 01:57:08 +00:00
nobu b2c81746ac * encoding.c (rb_enc_default, rb_enc_primary): return pointers to 
rb_encoding of default and primary respectively.  [ruby-core:12795]

* encoding.c (set_primary_encoding): removed primary_encoding setter.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13746 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-21 09:29:19 +00:00
nobu 5d5d0b5c38 * encoding.c (rb_id_encoding): returns ID "encoding". 
* marshal.c (w_encoding): dump encoding name.

* marshal.c (r_ivar): load encoding.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13744 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-19 11:08:16 +00:00
nobu 2d1d6c4705 * encoding.c (rb_enc_from_encoding, rb_enc_register): associate index 
to self.

* encoding.c (enc_capable): Encoding objects are encoding capable.

* re.c (rb_reg_s_union): check if encoding matching by exact encoding
  objects.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13732 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-17 02:30:57 +00:00
nobu 5daecbc0e1 * encoding.c (rb_enc_alias, rb_enc_find_index): changed 
enc_table_alias to a name-to-index hash.

* encoding.c (rb_enc_init): use upper case names for aliases to use as
  constant names.

* encoding.c (enc_find): allow symbols.

* encoding.c (Init_Encoding): define encoding constants.

* st.c (strcasehash): fix wrong code range condition.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13731 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-16 21:18:09 +00:00
nobu de46b903a9 * encoding.c (enc_list): seems a commit miss. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13730 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-16 20:26:29 +00:00
nobu 3cd040b6ef * encoding.c (rb_primary_encoding): added Encoding.primary_encoding. 
* parse.y (rb_parser_encoding): added.

* ruby.c (proc_options): added -E and --encoding options.

* ruby.c (process_options): set primary encoding from command line
  option if set, or source encoding.

* include/ruby/encoding.h (rb_enc_from_encoding,
  rb_get_primary_encoding, rb_set_primary_encoding): prototypes.

* include/ruby/node.h (rb_parser_encoding): prototype.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13729 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-16 20:07:20 +00:00
matz 4ad19a7dee * encoding.c (enc_to_s): rename function. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13715 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-16 04:50:16 +00:00
matz ff3859899d * encoding.c (Init_Encoding): define #to_s to show encoding name 
in to_s representation as well as #inspect.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13712 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-15 16:28:18 +00:00
nobu b314d3e999 * encoding.c (rb_obj_encoding): rdoc update. a patch from David 
Flanagan <david AT davidflanagan.com>.  [ruby-core:12664]

* encoding.c (enc_dump, enc_load): marshaling feature.  a patch from
  David Flanagan.  [ruby-core:12665]

* encoding.c (Init_Encoding): undefine allocator of Encoding.
  [ruby-core:12665], [ruby-core:12666]

* test/ruby/test_encoding.rb: tests for Encoding from David Flanagan
  [ruby-core:12665]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13697 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-14 08:31:51 +00:00
nobu 9c24fed339 * encoding.c (rb_cEncoding): new Encoding class. 
* encoding.c (rb_to_encoding, rb_to_encoding_index): helper functions.

* encoding.c (rb_obj_encoding): return Encoding object now.

* gc.c (garbage_collect): mark Encoding objects.

* string.c (rb_str_force_encoding): accept Encoding object as well as
  encoding name.

* include/ruby/encoding.h (rb_to_encoding_index, rb_to_encoding):
  prototypes.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13692 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-13 16:32:40 +00:00
akr ee624f97ac * encoding.c (rb_enc_init): don't alias iso-8859-1 to ascii. 
* ascii.c (OnigEncodingASCII): change the name US-ASCII to ASCII-8BIT.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13670 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-10 06:50:33 +00:00
nobu 597974c21f * encoding.c (rb_enc_register): returns new index or -1 if failed. 
* encoding.c (rb_enc_alias): check if original name is registered.

* encoding.c (rb_enc_init): register in same order as kcode options in
  re.c.  added new aliases.

* string.c (rb_str_force_encoding): check if valid encoding name.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13643 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-06 05:56:09 +00:00
nobu 19dee8af57 * encoding.c (rb_obj_encoding): returns encoding of the given object. 
* re.c (Init_Regexp): new method Regexp#encoding.

* string.c (str_encoding): moved to encoding.c


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13613 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-10-04 06:57:19 +00:00
nobu c351afc372 * encoding.c (rb_enc_alias): allow encodings multiple aliases. 
* encoding.c (rb_enc_find_index): search the encoding which has the
  given name and return its index if found, or -1.

* st.c (type_strcasehash): case-insensitive string hash type.

* string.c (rb_str_force_encoding): force encoding of self.  this name
  comes from [ruby-dev:31894] by Martin Duerst.  [ruby-dev:31744]

* include/ruby/encoding.h (rb_enc_find_index, rb_enc_associate_index):
  prototyped.

* include/ruby/encoding.h (rb_enc_isctype): direct interface to ctype.

* include/ruby/st.h (st_init_strcasetable): prototyped.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13556 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-09-28 19:27:10 +00:00
matz 335fe1ee7b * string.c (rb_str_comparable): need not to check asciicompat here. 
* encoding.c (rb_enc_check): ditto.

* string.c (rb_enc_str_coderange): tuned a bit; no broken check.

* encoding.c (rb_enc_check): new encoding comparison criteria.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13547 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-09-28 09:07:02 +00:00
nobu e41b84895e * encoding.c (rb_enc_associate_index): deal with ASCII compatible 
flags.

* encoding.c (rb_enc_check): allow ASCII compatible strings.

* parse.y (rb_intern_str): use ASCII encoding for ASCII string.

* string.c (rb_enc_str_coderange): check for code-range.

* string.c (rb_str_modify): clear code-range flags.

* string.c (rb_str_hash, rb_str_eql): ASCII compatible strings are
  comparable.

* include/ruby/encoding.h: added code-range flags.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13529 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-09-26 19:46:58 +00:00
nobu 94a0db11e7 * encoding.c (rb_enc_check): check for ASCII-compatibilities. 
* parse.y (parser_tokadd_string, parser_parse_string,
  parser_here_document, parser_yylex): set encoding to US-ASCII.

* parse.y (rb_enc_symname_p): check if valid with encoding.

* parse.y (rb_intern3): let symbols have encoding.

* string.c (rb_str_hash): add encoding index.

* string.c (rb_str_comparable, rb_str_equal, rb_str_eql): check if
  compatible encoding.

* string.c (sym_inspect): made encoding aware.

* insns.def (opt_eq): compare with encoding.

* include/ruby/encoding.h (rb_enc_asciicompat): check if ASCII
  compatible.

* include/ruby/encoding.h (rb_enc_get_index): added prototype.

* include/ruby/intern.h (rb_str_comparable, rb_str_equal): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13518 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-09-26 09:39:08 +00:00
nobu 26adfc185f * encoding.c (rb_enc_associate_index, rb_enc_get_index): check if 
object is encoding capable.  [ruby-dev:31780]

* string.c (rb_str_subpat_set): check for if the argument is a String.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-09-15 08:04:10 +00:00
matz edd7c787ad * array.c (rb_ary_cycle): typo in rdoc. a patch from Yugui 
<yugui@yugui.sakura.ne.jp>.  [ruby-dev:31748]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13348 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-09-06 12:33:45 +00:00
matz 4461bd2f1f * encoding.c (rb_enc_codelen): raises invalid sequence exception 
if ONIGENC_CODE_TO_MBCLEN() returns zero.  [ruby-dev:31661]

* encoding.c (rb_enc_mbclen): check invalid sequence.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13287 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-08-27 13:48:09 +00:00
matz 5678ab5182 encoding.c: added. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13263 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2007-08-25 04:07:20 +00:00