github
/
ruby
зеркало из https://github.com/github/ruby.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
14 630 коммитов 219 Ветки 1 255 Теги 478 MiB
Граф коммитов

218 Коммитов

Автор SHA1 Сообщение Дата
nobu da6300e8f8 * enc/us_ascii.c (us_ascii_mbc_enc_len): made static. a patch by 
Tadashi Saito <shiba AT mail2.accsnet.ne.jp> at [ruby-dev:36916]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-24 19:00:35 +00:00
duerst b014f1bc02 * enc/trans/single_byte.trans: adding WINDOWS-wwww encodings 
(wwww = 874/1250/1251/1253/1254/1255/1256/1257)
  (contributed by Yoshihiro Kambayashi)

* enc/trans/windows-wwww-tbl.rb: 8 new files
  (contributed by Yoshihiro Kambayashi)

* test/ruby/test_transcode.rb: added test_windows_wwww
  (contributed by Yoshihiro Kambayashi)


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19846 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-19 09:15:37 +00:00
duerst 7adbfbb793 * tool/transcode-tblgen.rb: added set_valid_byte_pattern 
to reduce coupling between table generation script and
  specific encodings.

* enc/trans/single_byte.trans: using set_valid_byte_pattern


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19831 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-18 08:10:57 +00:00
nobu 7485e91f76 * common.mk, enc/depend (enc, trans): targets for sources. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19799 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-16 05:34:25 +00:00
akr b968fa97f6 * enc/trans/single_byte.trans (transcode_tblgen_singlebyte): renamed 
from transcode_tblgen_windows.
  (transcode_tblgen_iso8859): use transcode_tblgen_singlebyte.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19780 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-14 11:33:17 +00:00
duerst 48a303c027 * enc/trans/single_byte.trans: added windows-1252 
* enc/trans/windows-1252-tbl.rb: new file
  (contributed by Yoshihiro Kambayashi)

* tool/transcode-tblgen.rb: listed windows-1252 as '1byte'

* test/ruby/test_transcode.rb: added test_windows_1252
  (contributed by Yoshihiro Kambayashi)


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19778 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-10-14 04:37:10 +00:00
akr 081c802cb9 * grapheme cluster implementation reverted. [ruby-dev:36375] 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19417 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-18 12:53:25 +00:00
akr b3d772643e fix typos. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19390 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-16 16:59:08 +00:00
akr a67d4fa01c * include/ruby/oniguruma.h (OnigEncodingTypeST): add precise_ret 
argument for mbc_to_code.
  (ONIGENC_MBC_TO_CODE): provide NULL for precise_ret.
  (ONIGENC_MBC_PRECISE_CODEPOINT): defined.

* include/ruby/encoding.h (rb_enc_mbc_precise_codepoint): defined.

* regenc.h (onigenc_single_byte_mbc_to_code): precise_ret argument
  added.
  (onigenc_mbn_mbc_to_code): ditto.

* regenc.c (onigenc_single_byte_mbc_to_code): precise_ret argument
  added.
  (onigenc_mbn_mbc_to_code): ditto.

* string.c (count_utf8_lead_bytes_with_word): removed.
  (str_utf8_nth): removed.
  (str_utf8_offset): removed.
  (str_strlen): UTF-8 codepoint oriented optimization removed.
  (rb_str_substr): ditto.
  (enc_succ_char): use rb_enc_mbc_precise_codepoint.
  (enc_pred_char): ditto.
  (rb_str_succ): ditto.

* encoding.c (rb_enc_ascget): check length with
  rb_enc_mbc_precise_codepoint.
  (rb_enc_codepoint): use rb_enc_mbc_precise_codepoint.

* regexec.c (string_cmp_ic): add text_end argument.
  (match_at): check end of character after exact string matches.

* enc/utf_8.c (graphme_table): defined for extended graphme cluster
  boundary.
  (grapheme_cmp): defined.
  (get_grapheme_properties): defined.
  (grapheme_boundary_p): defined.
  (MAX_BYTES_LENGTH): defined.
  (comb_char_enc_len): defined.
  (mbc_to_code0): extracted from mbc_to_code.
  (mbc_to_code): use mbc_to_code0.
  (left_adjust_combchar_head): defined.
  (utf_8): use a extended graphme cluster as a unit.

* enc/unicode.c (onigenc_unicode_mbc_case_fold): use
  ONIGENC_MBC_PRECISE_CODEPOINT to extract codepoints.
  (onigenc_unicode_get_case_fold_codes_by_str): ditto.

* enc/euc_jp.c (mbc_to_code): follow mbc_to_code field change.
  use onigenc_mbn_mbc_to_code.

* enc/shift_jis.c (mbc_to_code): ditto.

* enc/emacs_mule.c (mbc_to_code): ditto.

* enc/gbk.c (gbk_mbc_to_code): follow mbc_to_code field and
  onigenc_mbn_mbc_to_code change.

* enc/cp949.c (cp949_mbc_to_code): ditto.

* enc/big5.c (big5_mbc_to_code): ditto.

* enc/euc_tw.c (euctw_mbc_to_code): ditto.

* enc/euc_kr.c (euckr_mbc_to_code): ditto.

* enc/gb18030.c (gb18030_mbc_to_code): ditto.

* enc/utf_32be.c (utf32be_mbc_to_code): follow mbc_to_code field
  change.

* enc/utf_16be.c (utf16be_mbc_to_code): ditto.

* enc/utf_32le.c (utf32le_mbc_to_code): ditto.

* enc/utf_16le.c (utf16le_mbc_to_code): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-16 16:48:05 +00:00
akr 0675246ba6 * transcode_data.h (rb_transcoder): resetsize_func and resetstate_func 
also returns ssize_t.

* enc/trans/iso2022.trans: follow the type change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-15 02:11:50 +00:00
akr c767be3039 * transcode_data.h: return output functions ssize_t. 
* transcode.c (transcode_restartable0): don't need to cast the result
  of output functions.

* enc/trans/newline.trans: follow the type change.

* enc/trans/escape.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/japanese.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19351 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-14 18:35:17 +00:00
akr a3c8c0adec * transcode_data.h: output function takes output buffer size. 
* transcode.c: give output buffer size for output functions.

* enc/trans/newline.trans: follow the type change.

* enc/trans/escape.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/japanese.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19350 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-14 18:06:20 +00:00
akr 19416601a0 * include/ruby/oniguruma.h (OnigEncodingTypeST): add end argument for 
left_adjust_char_head.
  (ONIGENC_LEFT_ADJUST_CHAR_HEAD): add end argument.
  (onigenc_get_left_adjust_char_head): ditto.

* include/ruby/encoding.h (rb_enc_left_char_head): add end argument.

* regenc.h (onigenc_single_byte_left_adjust_char_head): ditto.

* regenc.c (onigenc_get_right_adjust_char_head): follow the interface
  change.
  (onigenc_get_right_adjust_char_head_with_prev): ditto.
  (onigenc_get_prev_char_head): ditto.
  (onigenc_step_back): ditto.
  (onigenc_get_left_adjust_char_head): ditto.
  (onigenc_single_byte_code_to_mbc): ditto.

* re.c: ditto.

* string.c: ditto.

* io.c: ditto.

* regexec.c: ditto.

* enc/euc_jp.c: ditto.

* enc/cp949.c: ditto.

* enc/shift_jis.c: ditto.

* enc/gbk.c: ditto.

* enc/big5.c: ditto.

* enc/euc_tw.c: ditto.

* enc/euc_kr.c: ditto.

* enc/emacs_mule.c: ditto.

* enc/gb18030.c: ditto.

* enc/utf_8.c: ditto.

* enc/utf_16le.c: ditto.

* enc/utf_16be.c: ditto.

* enc/utf_32le.c: ditto.

* enc/utf_32be.c: ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19334 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-13 19:23:52 +00:00
akr 41d3a01486 * enc/trans/escape.trans: transcoder name renamed to use underscore. 
* transcode.c: follow the renaming.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19311 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-12 17:30:07 +00:00
naruse d51b061565 * include/ruby/oniguruma.h (OnigCodePoint): unsigned long to unsigned int. 
* include/ruby/encoding.h (rb_enc_codepoint): ditto.

* encoding.c (rb_enc_codepoint): signed int to unsigned int.

* encoding.c (rb_enc_ascget): ditto.

* string.c (rb_str_casecmp): ditto.

* string.c (enc_succ_alnum_char): ditto.

* string.c (rb_str_inspect): ditto.

* string.c (rb_str_upcase_bang): ditto.

* string.c (rb_str_downcase_bang): ditto.

* string.c (rb_str_capitalize_bang): ditto.

* string.c (rb_str_swapcase_bang): ditto.

* string.c (struct tr): ditto.

* string.c (trnext): ditto.

* string.c (tr_trans): ditto.

* string.c (tr_setup_table): ditto.

* string.c (tr_find): ditto.

* string.c (rb_str_delete_bang): ditto.

* string.c (rb_str_squeeze_bang): ditto.

* string.c (rb_str_count): ditto.

* string.c (rb_str_split_m): ditto.

* string.c (rb_str_each_line): ditto.

* string.c (rb_str_lstrip_bang): ditto.

* string.c (rb_str_rstrip_bang): ditto.

* string.c (rb_str_intern): ditto.

* dir.c (char_casecmp): ditto.

* sprintf.c (rb_str_format): ditto.

* enc/emacs_mule.c (mbc_to_code): to be 32bit clean.

* enc/emacs_mule.c (code_to_mbc): ditto.

* enc/gb18030.c (mbc_to_code): ditto.

* enc/gb18030.c (code_to_mbc): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19295 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-11 10:34:59 +00:00
akr 817a623d13 * enc/trans/newline.trans (rb_universal_newline): swap src_encoding 
and dst_encoding.

* transcode.c (rb_econv_decorate_at): call get_transcoder_entry only
  once.
  (rb_econv_binmode): follow universal_newline change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19276 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-09 16:06:54 +00:00
akr 6270ad5b7f * include/ruby/encoding.h (rb_econv_asciicompat_encoding): renamed 
from rb_econv_stateless_encoding to apply stateless ASCII
  incompatible encodings such as UTF-16BE.

* io.c (make_writeconv): use rb_econv_asciicompat_encoding.

* transcode_data.h (rb_transcoder_asciicompat_type_t): renamed from
  rb_transcoder_stateful_type_t.
  (rb_transcoder): use rb_transcoder_asciicompat_type_t.

* transcode.c: follow the type change.
  (asciicompat_encoding_i): renamed from stateless_encoding_i.
  (rb_econv_asciicompat_encoding): renamed from
  rb_econv_stateless_encoding.
  (econv_s_asciicompat_encoding): method renamed.

* tool/transcode-tblgen.rb: follow the type change.

* enc/trans/utf_16_32.trans: follow the type change.
  rb_from_UTF_16BE to UTF-8 is asciicompat_decoder.
  rb_from_UTF_16LE to UTF-8 is asciicompat_decoder.
  rb_from_UTF_32BE to UTF-8 is asciicompat_decoder.
  rb_from_UTF_32LE to UTF-8 is asciicompat_decoder.
  UTF-8 to rb_to_UTF_16BE is asciicompat_encoder.
  UTF-8 to rb_to_UTF_16LE is asciicompat_encoder.
  UTF-8 to rb_to_UTF_32BE is asciicompat_encoder.
  UTF-8 to rb_to_UTF_32LE is asciicompat_encoder.

* enc/trans/newline.trans: follow the type change.  universal newline
  decoder is asciicompat_converter.

* enc/trans/escape.trans: follow the type change.

* enc/trans/iso2022.trans: ditto.

* enc/trans/japanese.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19249 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-08 14:33:17 +00:00
akr 7de9b10819 * enc/trans/iso2022.trans: upcase to iso-2022-jp. 
* enc/emacs_mule.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19221 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 15:05:12 +00:00
akr 78350a0cc6 * enc/trans/iso2022.trans: stateless-iso-2022-jp is defined to avoid 
undefined conversion error between iso-2022-jp and the corresponding
  stateless encoding.

* enc/emacs_mule.c: replicate emacs-mule as stateless-iso-2022-jp.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19220 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 14:49:05 +00:00
akr a3cfd3233e * enc/trans/escape.trans (hexstr): renamed from str1. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19218 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 11:03:24 +00:00
akr faa1387484 * enc/trans/escape.trans: use transcode_tblgen. 
* tool/transcode-tblgen.rb: generate an empty line after str1.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19217 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 10:46:30 +00:00
akr c0bec2fae1 * transcode_data.h (STR1): defined for a string up to 255 bytes. 
(STR1_BYTEINDEX): defined.
  (makeSTR1): defined.

* tool/transcode-tblgen.rb: generate STR1.

* transcode.c (transcode_restartable0): interpret STR1.

* enc/trans/escape.trans (fun_so_escape_xml_chref): removed.  STR1 is
  used instead.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19214 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 09:13:16 +00:00
akr 020e681eec * include/ruby/encoding.h (ECONV_XML_ATTR_CONTENT_ENCODER): defined. 
(ECONV_STATEFUL_ENCODER_MASK): defined.
  (ECONV_XML_ATTR_QUOTE_ENCODER): defined.
  (ECONV_XML_ATTR_ENCODER): removed.

* enc/trans/escape.trans (rb_escape_xml_attr_content): defined.
  (rb_escape_xml_attr_quote): defined.
  (rb_escape_xml_attr): removed.

* io.c (NEED_WRITECONV): writeconv is required if supplemental
  converter is used.
  (make_writeconv): apply stateful encoder in writeconv.

* transcode.c: follow the constant change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19209 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-07 03:13:29 +00:00
akr 76b3063022 * include/ruby/encoding.h (ECONV_XML_TEXT_ENCODER): renamed from 
ECONV_HTML_TEXT_ENCODER.
  (ECONV_XML_ATTR_ENCODER): renamed from ECONV_HTML_ATTR_ENCODER.

* enc/trans/escape.trans: follow the renaming.

* transcode.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19191 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 14:05:10 +00:00
akr 9f687d7be9 * enc/trans/escape.trans (fun_so_escape_html_attr): fix return type. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19177 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 04:43:39 +00:00
akr 091171a286 * enc/trans/escape.trans (escape_html_attr_init): new function. 
(fun_so_escape_html_attr): new function.
  (escape_html_attr_finish): new function.
  (rb_escape_html_attr): use them to quote the converted result.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19173 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-06 03:20:51 +00:00
akr a10a5ddaac * enc/trans/escape.trans: new file. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19165 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 21:29:12 +00:00
akr debeb7db77 * enc/trans/newline.trans (universal_newline_finish): new function. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19152 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-05 11:16:28 +00:00
akr 64b633f069 * enc/trans/newline.trans: record newline types met in universal 
newline decoder.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19136 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-04 14:20:14 +00:00
akr f6441bf61c * transcode_data.h (rb_transcoding): remove stateful field. 
add state field.
  (TRANSCODING_STATE): defined.
  (rb_transcoder): add fields: state_size, state_init_func,
  state_fini_func.
  change rb_transcoding* argument to void*.

* transcode.c (transcode_restartable0): use TRANSCODING_STATE for
  first arguments of transcoder functions.
  (rb_transcoding_open_by_transcoder): initialize state field.
  (rb_transcoding_close): finalize state field.

* tool/transcode-tblgen.rb: provide state size/init/fini.

* enc/trans/newline.trans (universal_newline_init): defined.
  (fun_so_universal_newline): take void* as a state pointer.
  (rb_universal_newline): provide state size/init/fini.
  (rb_crlf_newline): ditto.
  (rb_cr_newline): ditto.

* enc/trans/iso2022.trans (iso2022jp_init): defined.
  (fun_si_iso2022jp_to_eucjp): take void* as a state pointer.
  (fun_so_iso2022jp_to_eucjp): ditto.
  (fun_so_eucjp_to_iso2022jp): ditto.
  (iso2022jp_reset_sequence_size): ditto.
  (finish_eucjp_to_iso2022jp): ditto.
  (rb_ISO_2022_JP_to_EUC_JP): provide state size/init/fini.
  (rb_EUC_JP_to_ISO_2022_JP): ditto.

* enc/trans/utf_16_32.trans (fun_so_from_utf_16be): take void* as a
  state pointer.
  (fun_so_to_utf_16be): ditto.
  (fun_so_from_utf_16le): ditto.
  (fun_so_to_utf_16le): ditto.
  (fun_so_from_utf_32be): ditto.
  (fun_so_to_utf_32be): ditto.
  (fun_so_from_utf_32le): ditto.
  (fun_so_to_utf_32le): ditto.
  (rb_from_UTF_16BE): provide state size/init/fini.
  (rb_to_UTF_16BE): ditto.
  (rb_from_UTF_16LE): ditto.
  (rb_to_UTF_16LE): ditto.
  (rb_from_UTF_32BE): ditto.
  (rb_to_UTF_32BE): ditto.
  (rb_from_UTF_32LE): ditto.
  (rb_to_UTF_32LE): ditto.

* enc/trans/japanese.trans (fun_so_eucjp2sjis): take void* as a state
  pointer.
  (fun_so_sjis2eucjp): ditto.
  (rb_eucjp2sjis): provide state size/init/fini.
  (rb_sjis2eucjp): provide state size/init/fini.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 14:12:06 +00:00
akr 4406629bd6 * enc/trans/japanese.trans: new file. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 11:09:25 +00:00
naruse 42a48c1e9d * enc/trans/make_transdb.rb: check $(srcdir)/enc/trans before 
enc/trans.

* enc/trans/make_transdb.rb: keep names_t.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19081 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-03 08:57:18 +00:00
akr 5cea1b07f4 * enc/trans/make_transdb.rb: check foo.c only if foo.trans exists. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19068 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 11:57:49 +00:00
akr 797faf92d9 * enc/trans/make_transdb.rb: error message improved. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19067 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 10:20:11 +00:00
akr 298d66296d revert last commit. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 10:12:14 +00:00
akr 01d7d19077 * enc/trans/make_transdb.rb: error message improved. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 09:35:38 +00:00
usa 7f8e24d43f * enc/trans/utf_16_32.trans (from_UTF_8): rename from to_UTF_16BE 
because it was not collect.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19063 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 05:06:05 +00:00
yugui 2e094d086a * enc/emacs_mule.c (svn:executable): dropped executable bit. 
* enc/make_encdb.rb (svn:executable): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19062 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 05:04:30 +00:00
akr a2ada174a5 * enc/trans/japanese_euc.trans: splitted from japanese.trans to avoid 
compiler limitation.  reported by usa.

* enc/trans/japanese_sjis.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19058 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-02 03:39:11 +00:00
akr c5759bfb5b * tool/transcode-tblgen.rb: define TRANSCODE_TABLE_INFO in generated 
code.  use it in rb_transcoder.

* enc/trans/newline.trans: use TRANSCODE_TABLE_INFO.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19046 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 18:18:50 +00:00
akr 7908180df1 * tool/transcode-tblgen.rb: record offsets array as index of 
byte_array to avoid relocation.

* transcode.c (transcode_restartable0): add byte_array to get offsets
  array.

* transcode_data.h (BYTE_LOOKUP_BASE): change return type to
  uintptr_t.
  (rb_transcoder): add fields: byte_array, word_array and word_size.

* enc/trans/newline.trans: follow rb_transcoder change.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 17:40:32 +00:00
akr 752e053a1d * transcode_data.h (BYTE_LOOKUP): change to uintptr_t array. 
(BYTE_LOOKUP_BASE): follow the type change.
  (BYTE_LOOKUP_INFO): ditto.
  (PType): ditto.
  (rb_transcoding): ditto.

* tool/transcode-tblgen.rb: follow the type change.

* transcode.c: ditto.

* enc/trans/newline.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 16:22:49 +00:00
naruse 3960c2f7a7 * enc/euc_jp.c (euc-jp-ms): euc-jp-ms is not EUC-JP not an alias of 
eucJP-ms.

* enc/trans/japanese.trans (eucJP-ms): eucJP-ms is the correct
  name of the encoding in Ruby.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19021 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 08:24:55 +00:00
naruse a7cdd20204 * enc/trans/japanese.trans: fix mapping priority. 
IBM extended is prior than NEC selected IBM.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19019 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 06:16:42 +00:00
nobu c6c4ce81c2 * enc/depend: transdb.c may not present. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19015 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 05:37:04 +00:00
naruse 7ea3d11e22 * enc/trans/japanese.trans: fix Ruby 1.8 compatibility. 
* enc/trans/japanese.trans: fix mapping priority. [ruby-dev:36068]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19014 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 05:27:48 +00:00
akr 8841969485 * tool/transcode-tblgen.rb (transcode_generated_code): defined for 
generating table at once.
  (transcode_tblgen): returns an empty string.
  (transcode_generate_node): ditto.

* enc/trans/newline.trans: use transcode_generated_code.

* enc/trans/iso2022.trans: ditto.

* enc/trans/single_byte.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.

* enc/trans/japanese.trans: ditto.

* enc/trans/korean.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19006 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 17:35:00 +00:00
naruse 3d8043d062 * enc/trans/eucjp-tbl.rb: replace by previous Citrus maps. 
* enc/trans/sjis-tble.rb: ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19004 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 16:25:38 +00:00
naruse 1893e19865 * tool/transcode-tblgen.rb: add table generator from Citrus maps. 
* enc/trans/japanese.trans: use Citrus maps.

* enc/trans/CP: add maps from Citrus.

* enc/trans/JIS: ditto.

* test/ruby/test_transcode.rb: Shift_JIS and EUC-JP doesn't support
  IBM extended characters.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19003 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 16:23:04 +00:00
akr d1429c3cc6 * enc/trans/single_byte.trans (us_ascii_map): don't define 8bit bytes. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18982 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 07:37:10 +00:00