github
/
ruby
зеркало из https://github.com/github/ruby.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
66 923 коммитов 219 Ветки 1 255 Теги 478 MiB
Граф коммитов

400 Коммитов

Автор SHA1 Сообщение Дата
akr b9e0294eda fix typos. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25635 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-11-03 17:46:28 +00:00
nobu 5d4b63d2fd * transcode.c (rb_transcoding, str_transcoding_resize): fixed 
types.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25289 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-10-10 14:51:25 +00:00
nobu 89154445fb * transcode.c: reverted r25277. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25288 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-10-10 12:11:11 +00:00
nobu c4ff156d91 * transcode.c (rb_transcoding, str_transcoding_resize): fixed 
types.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25277 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-10-10 03:21:33 +00:00
nobu 913fdf25bc * transcode.c (econv_data_type): typed. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-09-09 03:56:58 +00:00
usa a436212282 * transcode.c (str_encode_bang): C99ism. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24335 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-07-31 04:16:01 +00:00
duerst 5e85648b77 * transcode.c: added check for frozen string for encode! (see Bug #1836) 
* test/ruby/test_transcode.rb: added tests for the above


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24322 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-07-30 09:40:06 +00:00
nobu 3448160b24 * transcode.c (enc_arg): default interanl encoding may not be set. 
[ruby-core:23932]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23773 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-06-21 01:46:37 +00:00
ko1 3caa4ee5e5 * array.c (rb_ary_memsize): added. 
* io.c (rb_io_memsize): added.
* regcomp.c (onig_memsize): added.
* string.c (rb_str_memsize): added.
* transcode.c (rb_transcoding_memsize, rb_econv_memsize): added.
* variable.c (rb_geneic_ivar_memsize): added.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23714 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-06-16 22:36:27 +00:00
naruse e54f880643 * transcode.c (transcode_restartable0): refix can't build with VC9. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23703 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-06-16 11:10:10 +00:00
ko1 d3027870ee * transcode.c (transcode_restartable0): revert last commit because 
this change cause SEGV at test-all.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23690 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-06-14 05:50:46 +00:00
naruse 44a8a15037 * transcode.c (transcode_restartable0): can't build with VC9. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23685 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-06-13 16:54:53 +00:00
naruse 6066cfadab Fix rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23682 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-06-13 09:59:46 +00:00
matz 91e5ba1cb8 * encoding.c (rb_enc_codepoint_len): combine rb_enc_codepoint() 
and rb_enc_codelen() in one function to reduce calls.

* encoding.c (rb_enc_codepoint): compatibility function.

* sprintf.c (rb_str_format): use rb_enc_codepoint_len().

* string.c (rb_str_inspect, rb_str_upcase_bang,
  rb_str_downcase_bang, rb_str_capitalize_bang,
  rb_str_swapcase_bang, trnext, tr_trans, rb_str_delete_bang,
  rb_str_squeeze_bang, rb_str_count, rb_str_split_m,
  rb_str_each_line, rb_str_each_codepoint, rb_str_lstrip_bang,
  sym_printable): ditto.

* transcode.c (make_econv_exception): use rb_enc_mbc_to_codepoint()

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-05-19 16:59:22 +00:00
naruse 3abca796f4 Fix: DON'T move in_p because before in_p is replaced by buffered data. 
* transcode.c: NOMAP is now multibyte direct map.

* transcode.c: remove ASIS.

* transcode_data.h: ditto.

* tool/transcode-tb (ActionMap#generate_info): remove :asis.

* tool/transcode-tb (ActionMap#generate_info): add :nomap0.

* enc/trans/utf8_mac.trans: replace :asis by :nomap0.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23344 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-05-05 00:05:11 +00:00
naruse 0881a39acd * transcode.c (ASIS): added for multi byte direct map. 
* transcode.c (transcode_restartable0): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23294 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-04-26 13:03:50 +00:00
nobu d1abc53721 * array.c, bignum.c, dln.c, error.c, gc.c, io.c, marshal.c, 
numeric.c, pack.c, strftime.c, string.c, thread.c, transcode.c,
  transcode_data.h, util.c, variable.c, vm_dump.c,
  include/ruby/encoding.h, missing/crypt.c, missing/vsnprintf.c:
  suppress VC type warnings.  [ruby-core:22726]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-03-12 09:16:15 +00:00
nobu 12d2c8ba41 stripped trailing spaces. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22552 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-02-22 14:23:33 +00:00
akr b916afddc9 * transcode.c (make_econv_exception): refine error message for 
undefined conversion.  [ruby-core:21828]



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22049 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-02-04 16:45:31 +00:00
akr e193fa79d6 * transcode.c (make_econv_exception): show U+XXXX form for undefined 
conversion error from UTF-8.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22039 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-02-04 13:01:12 +00:00
akr 07aeaa0686 * transcode.c (rb_econv_init_by_convpath_i): make it static. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-01-29 17:57:13 +00:00
nobu 882cb2e37d * transcode.c (struct rb_transcoding): added ary member for debug. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21789 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-01-27 01:15:37 +00:00
naruse 69ee4246e5 * transcode.c (str_transcode0): fix: :xml option doesn't 
work on str.encode([options]) form without default_internal.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21649 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-01-18 18:32:24 +00:00
naruse 81bdf8a1f2 Add rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21532 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-01-15 13:47:47 +00:00
duerst 82c673d3a1 * enc/trans/gb18030.trans, gb18030-tbl.rb: 
new Chinese GB18030 transcoding (from Yoshihiro Kambayashi)

* test/ruby/test_transcode.rb: added tests for the above
  (from Yoshihiro Kambayashi)

* transcode_data.h, transcode.c, tool/transcode_tblgen.rb:
  added support for GB18030-specific 4-byte sequences
  (with Yoshihiro Kambayashi)



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21509 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-01-14 11:12:30 +00:00
naruse 7371086f68 Add dummy definition for fake rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-01-13 08:18:16 +00:00
duerst bf6c750c35 * transcode.c: Minor fixes and tweaks in documentation. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-12-30 09:03:04 +00:00
akr 415d59e3e4 * transcode.c (make_econv_exception): show source and destination 
encoding.  [ruby-dev:37285]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21092 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-12-27 06:28:34 +00:00
akr b17c5eba46 * io.c (fptr_finalize): close the IO object even if finish_writeconv or 
flush is failed.
  (finish_writeconv): don't raise.  return errno or exception.
  (finish_writeconv_arg): removed.
  (finish_writeconv_sync): follow finish_writeconv change.

* transcode.c (rb_econv_make_exception): new function.

* include/ruby/encoding.h (rb_econv_make_exception): declared.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21059 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-12-26 05:04:39 +00:00
naruse 20f5c2a3ba * transcode.c (str_transcode0): set encoding when String#encode was 
given explicit but the same destination and source encoding.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21047 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-12-25 11:59:57 +00:00
naruse df4699320d * transcode.c (sym_ignore): remove useless symbol. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21044 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-12-25 10:43:35 +00:00
naruse d42b3e5c88 * transcode.c (decorate_convpath): show type of escaping for 
xml_attr_quote or some conversions.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@20991 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-12-25 08:54:51 +00:00
yugui 5677106ec9 * io.c: rdoc for File::open and 1.9 feature in file modes. 
* transcode.c: rdoc for String#encode

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@20946 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-12-23 11:30:44 +00:00
duerst da1ffe490d * transcode.c: in transcode_search_path, elimintated a warning 
on cygwin about pathlen potentially not being initialized


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19830 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-18 07:41:09 +00:00
matz 5ef37965fc * transcode.c (str_transcode0): String#encode without argument now 
behave as if :undef => :replace, :invalid => :replace specified.

* transcode.c (rb_econv_prepare_opts): should preserve options in
  any case.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19818 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-17 09:13:04 +00:00
akr c2b9d34f62 * transcode.c (transcode_restartable0): in_pos and out_pos never be 
NULL, now.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-16 16:26:53 +00:00
matz 80cdf946f7 * transcode.c (str_transcode0): should not raise error when 
default_internal is not set.  [ruby-core:19309]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-14 14:05:11 +00:00
nobu a09a8ea86d * transcode.c (rb_econv_init_by_convpath): suppressed warnings. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19769 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-13 00:56:40 +00:00
matz 284f4c8431 * transcode.c (str_transcode0): String#encode() with no encoding 
specified transcode the string into Encoding.default_internal.
  inspired by [ruby-core:19298].

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19764 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-12 22:52:01 +00:00
mame fc9c37dfc9 * transcode.c (rb_econv_add_coverter): remove unnecessary NULL check. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19733 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-09 15:12:20 +00:00
akr a3bebd5564 * include/ruby/encoding.h (rb_str_encode): renamed from 
rb_str_transcode.  [ruby-dev:36593]
  (rb_econv_has_convpath_p): renamed from rb_transcode_convertible.

* transcode.c: follow the renaming.

* io.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19589 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-26 10:35:00 +00:00
matz 3deb806a06 * error.c (Init_Exception): introduce EncodingError which is a 
superclass for all encoding related exception classes,
  e.g. Encoding::CompatibilityError.  [ruby-dev:36371]

* transcode.c (Init_transcode): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19570 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-26 03:53:11 +00:00
usa e76b1a11ca * transcode.c, include/ruby/encoding.c (rb_transcode_convertible): 
new function. checking the existance of converter.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19565 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-26 02:07:19 +00:00
akr 2a608fed88 update rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19558 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-25 17:42:52 +00:00
akr d7dbdef4e5 * transcode.c (rb_eUndefinedConversionError): renamed from 
rb_eConversionUndefinedError.
  (rb_eConverterNotFoundError): renamed from rb_eNoConverterError.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19554 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-25 12:47:30 +00:00
akr fee3bb664a update rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-25 11:42:49 +00:00
akr a7dd79c1b4 update rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19532 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-24 16:02:37 +00:00
akr 2cbc7b69b7 * transcode.c (enc_arg): must take pointer argument to avoid GC 
problem.  StringValueCStr modifies the argument and it should be
  preserved while the string StringValueCStr returns is used.
  Since the string is used by caller, the modified argument should be
  hold by caller.  Actually
    GC.stress = true
    def (o=Object.new).to_str()
      "universal"+"_newline"
    end                         
    "\u3042".encode(o, "")' 
  causes curious warning:
    rb_define_const: invalid name `' for constant



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19408 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-18 09:29:01 +00:00
matz d5ec36c486 * transcode.c: add "Error" suffix for Encoding exception classes. 
a patch from Tadashi Saito <shiba at mail2.accsnet.ne.jp> in
  [ruby-dev:36346].

* encoding.c (Init_Encoding): rename EncodingCompatibilityError to
  Encoding::CompatibilityError.  [ruby-dev:36366]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19407 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-18 08:57:33 +00:00
matz 45ee7d2e78 * transcode.c (enc_arg): revert last change. too early exception 
raising.

* transcode.c (enc_arg): need not to take pointer argument.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19406 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-18 08:27:17 +00:00
naruse bad9a9ad0d * transcode.c (enc_arg): raise exception when unknown encoding is 
given.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19388 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-16 16:18:32 +00:00
akr 6eb3843b17 * transcode_data.h (STR1_LENGTH): defined. 
(makeSTR1LEN): defined.

* tool/transcode-tblgen.rb: use makeSTR1LEN.  generate STR1 for 4 to
  259 bytes.

* transcode.c (rb_transcoding): new field: output_index.
  (transcode_restartable0): use STR1_LENGTH.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19366 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-15 15:42:30 +00:00
akr e0f064cf3c * transcode.c (econv_opts): don't use to_sym. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19363 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-15 14:38:12 +00:00
akr c767be3039 * transcode_data.h: return output functions ssize_t. 
* transcode.c (transcode_restartable0): don't need to cast the result
  of output functions.

* enc/trans/newline.trans: follow the type change.

* enc/trans/escape.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/japanese.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19351 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-14 18:35:17 +00:00
akr a3c8c0adec * transcode_data.h: output function takes output buffer size. 
* transcode.c: give output buffer size for output functions.

* enc/trans/newline.trans: follow the type change.

* enc/trans/escape.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/japanese.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19350 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-14 18:06:20 +00:00
nobu 2f67b43aec * transcode.c (str_encode): returns duplicated string if nothing 
changed.  [ruby-core:18578]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19349 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-14 17:37:35 +00:00
akr 0a3fc5be05 * transcode.c (rb_transcoding): align state field. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19326 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-13 09:52:02 +00:00
akr d4c7473841 * transcode.c (DECORATOR_P): renamed from SUPPLEMENTAL_CONVERSION. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19325 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-13 09:21:34 +00:00
akr 8d7e774f5f * io.c (make_writeconv): use rb_enc_name. 
(make_readconv): ditto.

* transcode.c (rb_econv_open_opts): ditto.
  (econv_args): ditto.
  (decorate_convpath): ditto.
  (econv_set_replacement): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19324 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-13 09:06:51 +00:00
akr 2cc9b488a0 * transcode.c (rb_econv_prepare_opts): raise ArgumentError if 
a broken string is specified as a replacement.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19316 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 18:45:44 +00:00
akr 7e58885f0a * transcode.c (rb_econv_prepare_opts): initialize *opt and return 0 
if opthash is nil. 


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19312 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 17:53:26 +00:00
akr 41d3a01486 * enc/trans/escape.trans: transcoder name renamed to use underscore. 
* transcode.c: follow the renaming.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19311 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 17:30:07 +00:00
akr 248acf0025 * transcode.c (rb_econv_init_by_convpath_i): new function. 
(rb_econv_init_by_convpath): new function.
  (econv_init): use rb_econv_init_by_convpath.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19310 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 17:13:04 +00:00
akr cf86176338 * transcode.c (econv_args): take snamev_p and dnamev_p to prevent 
possible GC problem.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 16:13:02 +00:00
akr aa423b9311 * transcode.c (enc_arg): take arg as VALUE * to prevent GC problem. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19307 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 15:39:46 +00:00
akr c0bd38eb94 * transcode.c (rb_econv_add_transcoder_at): fix loop condition. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19306 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 14:20:27 +00:00
akr 2d80b1695d * transcode.c (decorator_names): extracted from rb_econv_open. 
(rb_econv_open): use decorator_names.
  (econv_args): extracted from econv_init.
  (econv_init): use econv_args.
  (decorate_convpath): new function.
  (search_convpath_i): new function.
  (econv_s_search_convpath): new method.
  (Init_transcode): new method defined.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19305 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 13:53:42 +00:00
akr 89a5c998c7 * transcode.c (rb_econv_t): new field: num_allocated. 
(rb_econv_alloc): extracted from
  rb_econv_open_by_transcoder_entries.
  (rb_econv_add_transcoder_at): extracted from rb_econv_decorate_at
  and generalized 
  (rb_econv_open_by_transcoder_entries): use rb_econv_alloc and
  rb_econv_add_transcoder_at.
  (rb_econv_add_converter): extracted from rb_econv_decorate_at.
  (rb_econv_decorate_at): use rb_econv_add_converter.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19304 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 12:58:23 +00:00
akr 19438eee8b * transcode.c (rb_econv_t): new field: started. 
(rb_econv_open_by_transcoder_entries): initialize started field.
  (rb_econv_convert): set started field.
  (rb_econv_insert_output): ditto.
  (rb_econv_decorate_at): check started field instead of num_finished.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19303 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 12:23:25 +00:00
akr 10ddce3833 * transcode.c (make_encoding): new function. 
(make_encobj): new function.
  (econv_s_asciicompat_encoding): use make_encoding.
  (rb_econv_open_exc): use SUPPLEMENTAL_CONVERSION.
  (econv_convpath): use encoding object in the result.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19288 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-10 17:26:49 +00:00
akr 4e93ace24e * transcode.c (econv_convpath): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19286 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-10 17:15:08 +00:00
akr 2d26208a55 * include/ruby/encoding.h (econv_after_output): renamed from 
econv_output_followed_by_input.
  (ECONV_AFTER_OUTPUT): renamed from ECONV_OUTPUT_FOLLOWED_BY_INPUT.

* transcode.c: follow the renaming.

* io.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19278 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-09 16:27:02 +00:00
akr 3cc7c9d0dd * transcode.c (get_replacement_character): don't return ASCII 
incompatible replacements.
  (make_replacement): don't convert the result of
  get_replacement_character.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19277 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-09 16:18:29 +00:00
akr 817a623d13 * enc/trans/newline.trans (rb_universal_newline): swap src_encoding 
and dst_encoding.

* transcode.c (rb_econv_decorate_at): call get_transcoder_entry only
  once.
  (rb_econv_binmode): follow universal_newline change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19276 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-09 16:06:54 +00:00
akr d456f7e357 * include/ruby/encoding.h (ECONV_DECORATOR_MASK): merge 
ECONV_ENCODER_MASK and ECONV_DECORATOR_MASK.
  (ECONV_UNIVERSAL_NEWLINE_DECORATOR): renamed from
  ECONV_UNIVERSAL_NEWLINE_DECODER.
  (ECONV_CRLF_NEWLINE_DECORATOR): renamed from
  ECONV_CRLF_NEWLINE_ENCODER.
  (ECONV_CR_NEWLINE_DECORATOR): renamed from ECONV_CR_NEWLINE_ENCODER.
  (ECONV_XML_TEXT_DECORATOR): renamed from ECONV_XML_TEXT_ENCODER.
  (ECONV_XML_ATTR_CONTENT_DECORATOR): renamed from
  ECONV_XML_ATTR_CONTENT_ENCODER.
  (ECONV_STATEFUL_DECORATOR_MASK): renamed from
  ECONV_STATEFUL_ENCODER_MASK.
  (ECONV_XML_ATTR_QUOTE_DECORATOR): renamed from
  ECONV_XML_ATTR_CONTENT_DECORATOR.

* io.c: follow the renaming.

* transcode.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19271 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-09 12:22:43 +00:00
akr 8c2d4076f0 * transcode.c (rb_econv_open0): make it static. 
(rb_econv_open): place decorators at last.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19269 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-09 11:57:31 +00:00
akr 40c31d5b75 * transcode.c (rb_econv_t): last_trans_index removed. 
(rb_econv_open_by_transcoder_entries): follow the type change.
  (rb_econv_open0): ditto.
  (rb_econv_decorate_at): ditto.
  (rb_econv_binmode): ditto.
  (rb_econv_insert_output): simplified because there are no decorators
  at last.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19267 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-09 11:37:23 +00:00
akr b96c3dfe67 * transcode.c (rb_econv_open): reverse the order of decorders. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19266 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-09 11:22:17 +00:00
akr 4fcb83c06c * transcode.c (rb_econv_open0): num_encoders and num_decoders removed. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19265 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-09 11:18:59 +00:00
akr 795bf7a665 * include/ruby/encoding.h (rb_econv_decorate_at_first): declared. 
(rb_econv_decorate_at_last): declared.

* transcode.c (rb_econv_open_by_transcoder_entries): initialize
  replacement_enc.  allocate outbuf for the last transcoder.
  (rb_econv_open0): extracted from rb_econv_open.
  (rb_econv_open): use rb_econv_open0 and decorate the result using
  rb_econv_decorate_at_first and rb_econv_decorate_at_last.
  (rb_econv_decorate_at): new function.
  (rb_econv_decorate_at_first): ditto.
  (rb_econv_decorate_at_last): ditto.
  (rb_econv_binmode): fix iteration end condition.
  (econv_init): don't set source_encoding_name and
  destination_encoding_name because they are set in rb_econv_open0.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19262 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-08 17:23:37 +00:00
akr 65110d6464 * transcode.c (rb_econv_open): make ec a local variable. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19260 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-08 16:11:32 +00:00
akr 4106a7c067 * transcode.c (rb_econv_asciicompat_encoding): check decoder. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19253 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-08 15:21:05 +00:00
akr dbf0487581 * transcode.c (rb_econv_t): last_error.partial_input removed. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19250 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-08 15:01:16 +00:00
akr 6270ad5b7f * include/ruby/encoding.h (rb_econv_asciicompat_encoding): renamed 
from rb_econv_stateless_encoding to apply stateless ASCII
  incompatible encodings such as UTF-16BE.

* io.c (make_writeconv): use rb_econv_asciicompat_encoding.

* transcode_data.h (rb_transcoder_asciicompat_type_t): renamed from
  rb_transcoder_stateful_type_t.
  (rb_transcoder): use rb_transcoder_asciicompat_type_t.

* transcode.c: follow the type change.
  (asciicompat_encoding_i): renamed from stateless_encoding_i.
  (rb_econv_asciicompat_encoding): renamed from
  rb_econv_stateless_encoding.
  (econv_s_asciicompat_encoding): method renamed.

* tool/transcode-tblgen.rb: follow the type change.

* enc/trans/utf_16_32.trans: follow the type change.
  rb_from_UTF_16BE to UTF-8 is asciicompat_decoder.
  rb_from_UTF_16LE to UTF-8 is asciicompat_decoder.
  rb_from_UTF_32BE to UTF-8 is asciicompat_decoder.
  rb_from_UTF_32LE to UTF-8 is asciicompat_decoder.
  UTF-8 to rb_to_UTF_16BE is asciicompat_encoder.
  UTF-8 to rb_to_UTF_16LE is asciicompat_encoder.
  UTF-8 to rb_to_UTF_32BE is asciicompat_encoder.
  UTF-8 to rb_to_UTF_32LE is asciicompat_encoder.

* enc/trans/newline.trans: follow the type change.  universal newline
  decoder is asciicompat_converter.

* enc/trans/escape.trans: follow the type change.

* enc/trans/iso2022.trans: ditto.

* enc/trans/japanese.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19249 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-08 14:33:17 +00:00
akr a98bb55e01 * transcode.c (rb_econv_insert_output): "readagain" part should be 
after replacement.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19247 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-08 14:08:44 +00:00
akr 6e36c4842e * include/ruby/encoding.h (rb_econv_set_replacement): renamed from 
rb_econv_set_replacemenet.

* transcode.c: follow the renaming.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19227 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 17:08:38 +00:00
akr d4832b7aac rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19223 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 15:05:30 +00:00
akr c0bec2fae1 * transcode_data.h (STR1): defined for a string up to 255 bytes. 
(STR1_BYTEINDEX): defined.
  (makeSTR1): defined.

* tool/transcode-tblgen.rb: generate STR1.

* transcode.c (transcode_restartable0): interpret STR1.

* enc/trans/escape.trans (fun_so_escape_xml_chref): removed.  STR1 is
  used instead.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19214 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 09:13:16 +00:00
akr 020e681eec * include/ruby/encoding.h (ECONV_XML_ATTR_CONTENT_ENCODER): defined. 
(ECONV_STATEFUL_ENCODER_MASK): defined.
  (ECONV_XML_ATTR_QUOTE_ENCODER): defined.
  (ECONV_XML_ATTR_ENCODER): removed.

* enc/trans/escape.trans (rb_escape_xml_attr_content): defined.
  (rb_escape_xml_attr_quote): defined.
  (rb_escape_xml_attr): removed.

* io.c (NEED_WRITECONV): writeconv is required if supplemental
  converter is used.
  (make_writeconv): apply stateful encoder in writeconv.

* transcode.c: follow the constant change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19209 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 03:13:29 +00:00
akr 76b3063022 * include/ruby/encoding.h (ECONV_XML_TEXT_ENCODER): renamed from 
ECONV_HTML_TEXT_ENCODER.
  (ECONV_XML_ATTR_ENCODER): renamed from ECONV_HTML_ATTR_ENCODER.

* enc/trans/escape.trans: follow the renaming.

* transcode.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19191 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 14:05:10 +00:00
akr ca3148059b * transcode.c (econv_description): show ECONV_HTML_TEXT_ENCODER and 
ECONV_HTML_ATTR_ENCODER.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19188 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 11:11:21 +00:00
akr e4b70ffe43 * transcode.c (rb_econv_binmode): check actual transcoders. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 11:07:01 +00:00
akr f88c9f03f7 * transcode.c (rb_econv_open): fix last_tc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19186 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 10:54:59 +00:00
akr f10af82d25 * transcode.c (str_transcode0): check ECONV_HTML_TEXT_ENCODER and 
ECONV_HTML_ATTR_ENCODER.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19184 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 09:48:32 +00:00
akr 987ec6b787 * transcode.c (allocate_converted_string): fix overflow condition. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19183 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 09:39:12 +00:00
akr e490d92548 * transcode.c (sym_html): new variable. 
(sym_text): ditto.
  (sym_attr): ditto.
  (econv_opts): check :html=>:text and :html=>:attr.
  (Init_transcode): initialize the above variables.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19180 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 06:07:34 +00:00
akr 34be4c1ce5 * include/ruby/encoding.h (ECONV_HTML_TEXT_ENCODER): new constant. 
(ECONV_HTML_ATTR_ENCODER): ditto.

* transcode.c (rb_econv_open): check ECONV_HTML_TEXT_ENCODER and
  ECONV_HTML_ATTR_ENCODER.
  (Init_transcode): Encoding::Converter::HTML_TEXT_ENCODER and
  Encoding::Converter::HTML_ATTR_ENCODER defined.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19179 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 05:47:30 +00:00
akr 5117020d9e * transcode.c (struct trans_open_t): defined to pass num_additional. 
(trans_open_i): use struct trans_open_t.
  (rb_econv_open): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19178 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 05:16:40 +00:00
akr 27e9c48ccc * transcode.c (rb_econv_open): needless branch removed. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19176 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 03:44:31 +00:00
akr 073bced367 * transcode.c (rb_econv_open): test condition simplified. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19175 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 03:38:59 +00:00
akr 393f57670f * transcode.c (stateless_encoding_i): ignore supplemental conversions. 
Encoding::Converter.stateless_encoding("html-attr-escaped") should be
  nil.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 03:26:35 +00:00
akr 0be34e2a49 * transcode.c (rb_econv_open): fail for ASCII incompatible with 
newline conversion.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19167 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 22:27:46 +00:00
akr dc98e2528a * transcode.c (output_hex_charref): upcase hexadecimal digits. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19163 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 20:37:36 +00:00
akr c500c37fbc * include/ruby/encoding.h (ECONV_UNDEF_HEX_CHARREF): defined. 
* transcode.c (output_hex_charref): new function.
  (rb_econv_convert): call output_hex_charref if
  ECONV_UNDEF_HEX_CHARREF.
  (Init_transcode): Encoding::Converter::UNDEF_HEX_CHARREF added.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19162 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 20:24:18 +00:00
akr 7d3598ee60 * transcode.c (rb_econv_convert): use ECONV_INVALID_MASK and 
ECONV_UNDEF_MASK.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19161 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 19:03:21 +00:00
akr 80abde2c89 * transcode.c (allocate_converted_string): add arguments for a buffer 
allocated by caller.
  (rb_econv_insert_output): provide caller allocated buffer to
  allocate_converted_string.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19159 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 18:20:20 +00:00
akr f0d9f949e8 * transcode.c (str_transcode_enc_args): local variables renamed. 
(str_transcode0): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19158 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 17:59:32 +00:00
akr e93289c03f update rdoc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19157 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 17:25:56 +00:00
akr 134ca08dd1 * transcode.c (econv_s_stateless_encoding): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19156 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 17:23:49 +00:00
akr f2fd6d3b15 * transcode.c (enc_arg): extracted from str_transcode_enc_args. 
(str_transcode_enc_args): use enc_arg.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19154 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 17:02:33 +00:00
akr f549f1ae32 * transcode_data.h (rb_transcoder): rename fields: 
from_encoding -> src_encoding, to_encoding -> dst_encoding.

* transcode.c: follow the renaming.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 15:46:18 +00:00
akr a50b307900 * transcode.c: variables renamed for consistency. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 15:35:30 +00:00
akr 0406b0f154 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19134 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 13:17:40 +00:00
akr 5da4e660ad * transcode.c (econv_init): accept an integer as 3rd argument as well. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19132 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 12:55:02 +00:00
akr 2494e67f16 * transcode.c (sym_partial_input): new variable. 
(econv_primitive_convert): accept a hash as 5th argument as well.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 12:48:21 +00:00
akr f7607eff9a * transcode.c (sym_universal_newline_decoder): new variable. 
(sym_crlf_newline_encoder): ditto.
  (sym_cr_newline_encoder): ditto.
  (econv_opts): check newline converter options.
  (econv_init): make 3rd argument hash/nil only.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19130 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 12:08:54 +00:00
akr 0031967d20 * transcode_data.h (PType): defined unconditionaly. 
* transcode.c (PType): don't define here.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19127 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 11:31:37 +00:00
akr 45c3707303 * include/ruby/encoding.h (ECONV_INVALID_IGNORE): removed because 
it tend to cause security problem.  If the behaviour is really
  required, ECONV_INVALID_REPLACE with empty string can be used.
  For example, CVE-2006-2313, CVE-2008-1036, [ruby-core:15645]
  (ECONV_UNDEF_IGNORE): ditto.

* transcode.c (rb_econv_convert): follow the above change.
  (econv_opts): ditto.
  (Init_transcode): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19123 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 10:15:34 +00:00
akr 58964e3249 * include/ruby/io.h (rb_io_t): new fields: encs.ecopts and 
writeconv_pre_ecopts.
  (MakeOpenFile): initialize them.

* include/ruby/encoding.h (rb_str_transcode): take ecopts argument.
  (rb_econv_flags): removed.
  (rb_econv_prepare_opts): declared.
  (rb_econv_open_opts): declared.

* io.c (make_writeconv): use rb_econv_open_opts.
  (make_readconv): ditto.
  (io_fwrite): follow rb_str_transcode change.
  (rb_io_extract_modeenc): use rb_econv_prepare_opts.
  (rb_file_open_generic): initialize encs.ecopts.
  (rb_file_open_internal): ditto.
  (rb_io_reopen): ditto.
  (argf_ecopts): defined.
  (argf_next_argv): set encs.ecopts.
  (io_encoding_set): use rb_econv_prepare_opts.
  (argf_set_encoding): set argf_ecopts.

* gc.c (gc_mark_children): mark encs.ecopts and writeconv_pre_ecopts
  in T_FILE.

* transcode.c (transcode_loop): take ecopts argument. use
  rb_econv_open_opts.
  (rb_econv_flags): removed.
  (rb_econv_prepare_opts): defined.
  (rb_econv_open_opts): defined.
  (str_transcode0): take ecopts.
  (str_transcode): use rb_econv_prepare_opts.
  (rb_str_transcode): take ecopts.
  (econv_init): accept hash argument.
  (econv_insert_output): follow rb_str_transcode change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19109 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 18:18:10 +00:00
akr 0ebcad6a7b * include/ruby/encoding.h (rb_econv_set_replacemenet): declared. 
* transcode.c (rb_econv_t): new fields: replacement_str,
  replacement_len, replacement_enc and replacement_allocated.
  (get_replacement_character): make len as size_t.
  (rb_econv_open_by_transcoder_entries): initialize the new fields.
  (rb_econv_close): deallocate replacement_str if it allocated.
  (make_replacement): new function.
  (output_replacement_character): use make_replacement.
  (rb_econv_set_replacemenet): defined.
  (econv_get_replacement): new method.
  (econv_set_replacement): new method.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19108 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 16:34:11 +00:00
usa 9b6608c78d * transcode.c (transcode_restartable0): avoid VC++6's bug. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 16:02:10 +00:00
akr 56343375b8 * include/ruby/encoding.h (rb_econv_option_t): removed. Since 
rb_econv_option_t has only one field, int flags, rb_econv_option_t is
  replaced by int.

* include/ruby/io.h: follow the above change.

* io.c: ditto.

* transcode.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 15:11:46 +00:00
akr 1afc1b7bc8 * transcode.c (rb_transcoding): moved from transcode_data.h. 
(TRANSCODING_READBUF): ditto.
  (TRANSCODING_WRITEBUF): ditto.
  (TRANSCODING_STATE_EMBED_MAX): ditto.
  (TRANSCODING_STATE): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19097 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 14:34:09 +00:00
akr f6441bf61c * transcode_data.h (rb_transcoding): remove stateful field. 
add state field.
  (TRANSCODING_STATE): defined.
  (rb_transcoder): add fields: state_size, state_init_func,
  state_fini_func.
  change rb_transcoding* argument to void*.

* transcode.c (transcode_restartable0): use TRANSCODING_STATE for
  first arguments of transcoder functions.
  (rb_transcoding_open_by_transcoder): initialize state field.
  (rb_transcoding_close): finalize state field.

* tool/transcode-tblgen.rb: provide state size/init/fini.

* enc/trans/newline.trans (universal_newline_init): defined.
  (fun_so_universal_newline): take void* as a state pointer.
  (rb_universal_newline): provide state size/init/fini.
  (rb_crlf_newline): ditto.
  (rb_cr_newline): ditto.

* enc/trans/iso2022.trans (iso2022jp_init): defined.
  (fun_si_iso2022jp_to_eucjp): take void* as a state pointer.
  (fun_so_iso2022jp_to_eucjp): ditto.
  (fun_so_eucjp_to_iso2022jp): ditto.
  (iso2022jp_reset_sequence_size): ditto.
  (finish_eucjp_to_iso2022jp): ditto.
  (rb_ISO_2022_JP_to_EUC_JP): provide state size/init/fini.
  (rb_EUC_JP_to_ISO_2022_JP): ditto.

* enc/trans/utf_16_32.trans (fun_so_from_utf_16be): take void* as a
  state pointer.
  (fun_so_to_utf_16be): ditto.
  (fun_so_from_utf_16le): ditto.
  (fun_so_to_utf_16le): ditto.
  (fun_so_from_utf_32be): ditto.
  (fun_so_to_utf_32be): ditto.
  (fun_so_from_utf_32le): ditto.
  (fun_so_to_utf_32le): ditto.
  (rb_from_UTF_16BE): provide state size/init/fini.
  (rb_to_UTF_16BE): ditto.
  (rb_from_UTF_16LE): ditto.
  (rb_to_UTF_16LE): ditto.
  (rb_from_UTF_32BE): ditto.
  (rb_to_UTF_32BE): ditto.
  (rb_from_UTF_32LE): ditto.
  (rb_to_UTF_32LE): ditto.

* enc/trans/japanese.trans (fun_so_eucjp2sjis): take void* as a state
  pointer.
  (fun_so_sjis2eucjp): ditto.
  (rb_eucjp2sjis): provide state size/init/fini.
  (rb_sjis2eucjp): provide state size/init/fini.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 14:12:06 +00:00
akr 6378201f8b * transcode_data.h (WORDINDEX_SHIFT_BITS): defined. 
(WORDINDEX2INFO): defined.
  (INFO2WORDINDEX): defined.

* tool/transcode-tblgen.rb: use WORDINDEX2INFO.

* transcode.c: use INFO2WORDINDEX.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 12:35:29 +00:00
akr 6fe24d1f13 refactor last change. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 12:09:59 +00:00
akr 069f85f812 * transcode.c (BL_MIN_BYTE): defined. 
(BL_MAX_BYTE): defined.
  (BL_OFFSET): defined.
  (BL_ACTION): defined.
  (transcode_restartable0): use BL_MIN_BYTE, BL_MAX_BYTE and
  BL_ACTION.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19090 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 12:08:27 +00:00
akr ac5631e6f0 * transcode.c (BYTE_ADDR): defined. 
(WORD_ADDR): ditto.
  (BL_BASE): use BYTE_ADDR and WORD_ADDR.
  (BL_INFO): use WORD_ADDR.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19089 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 11:59:50 +00:00
akr 41e1933fd8 * transcode_data.h (base_element): removed. 
(BYTE_LOOKUP): removed.
  (BYTE_LOOKUP_BASE): don't cast.
  (BYTE_LOOKUP_INFO): ditto.
  (PType): unsigned int, instead of uintptr_t.
  (rb_transcoding): change type of next_field, conv_tree_start and
  word_array.

* tool/transcode-tblgen.rb: generate word_array as array of unsigned
  int.

* transcode.c (transcode_restartable0): follow the above type change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19070 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 13:26:02 +00:00
akr 6058eeaf03 * tool/transcode-tblgen.rb: record infos and BYTE_LOOKUPs as index of 
word_array to avoid relocation.

* transcode.c (transcode_restartable0): add word_array to get infos
  and BYTE_LOOKUPs.

* transcode_data.h (BYTE_LOOKUP_INFO): change return type to
  uintptr_t.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19045 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 18:06:28 +00:00
akr 7908180df1 * tool/transcode-tblgen.rb: record offsets array as index of 
byte_array to avoid relocation.

* transcode.c (transcode_restartable0): add byte_array to get offsets
  array.

* transcode_data.h (BYTE_LOOKUP_BASE): change return type to
  uintptr_t.
  (rb_transcoder): add fields: byte_array, word_array and word_size.

* enc/trans/newline.trans: follow rb_transcoder change.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 17:40:32 +00:00
akr 752e053a1d * transcode_data.h (BYTE_LOOKUP): change to uintptr_t array. 
(BYTE_LOOKUP_BASE): follow the type change.
  (BYTE_LOOKUP_INFO): ditto.
  (PType): ditto.
  (rb_transcoding): ditto.

* tool/transcode-tblgen.rb: follow the type change.

* transcode.c: ditto.

* enc/trans/newline.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 16:22:49 +00:00
akr 2ae7b0328e * transcode_data.h (BYTE_LOOKUP_BASE): abstract accessor for 
BYTE_LOOKUP.
  (BYTE_LOOKUP_INFO): ditto.

* transcode.c (transcode_restartable0): use BYTE_LOOKUP_BASE and
  BYTE_LOOKUP_INFO.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 14:33:56 +00:00
akr e7ee4fabd4 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 15:31:14 +00:00
akr a017cecf42 fix typos. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 14:24:51 +00:00
akr c3434e35ac rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18996 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 09:38:12 +00:00
akr 69610b07c3 * transcode.c (transcode.c): set source_encoding and 
destination_encoding as encoding object.
  (ecerr_source_encoding): new method.
  (ecerr_destination_encoding): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18993 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 09:08:31 +00:00
akr 72ee10b19c * transcode.c (make_econv_exception): rename instance variable names 
for storing encoding names.
  (ecerr_source_encoding_name): method renamed.
  (ecerr_destination_encoding_name): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 08:17:48 +00:00
akr 2d71485ec1 * transcode.c (econv_putback): associate encoding to the result. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 07:59:03 +00:00
akr d427cf3af8 * transcode.c (econv_insert_output): raise ArgumentError on failure. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18984 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 07:43:19 +00:00
akr c3f55e6d48 * transcode.c (econv_last_error): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18980 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:27:52 +00:00
akr 9d2accff2b * transcode.c (econv_primitive_convert): make two arguments, 
destination_byteoffset and destination_bytesize, optional.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18979 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:18:29 +00:00
akr f3c43ae0c2 * transcode.c (make_econv_exception): error message simplified. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:12:32 +00:00
akr d9c5f88abc * transcode.c (econv_putback): make max argument optional. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18967 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:39:16 +00:00
akr 97a5808a14 * transcode.c (rb_econv_putback): putback from last byte. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18966 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:36:38 +00:00
akr 998a4ec5af * transcode.c (econv_putback): renamed from econv_primitive_putback. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18965 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:27:35 +00:00
akr 5bbc51ed43 * transcode.c (econv_insert_output): renamed from 
econv_primitive_insert_output.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18964 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:23:23 +00:00
akr 5e222d6945 * transcode.c (econv_primitive_errinfo): 5th element (partial_input) 
removed from result.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18962 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 18:45:48 +00:00
akr c2ba5cf9c5 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-29 16:25:27 +00:00
akr d4afbe49f3 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 18:28:52 +00:00