github
/
ruby
зеркало из https://github.com/github/ruby.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
13 384 коммитов 219 Ветки 1 255 Теги 478 MiB
Граф коммитов

161 Коммитов

Автор SHA1 Сообщение Дата
akr 279e612973 fix variable name. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18558 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-13 05:53:20 +00:00
akr a14d5eb09b * enc/trans/newline.trans (rb_crlf_newline): new transcoder. 
(rb_cr_newline): new transcoder.

* transcode.c (trans_open_i): one more exra room for input newline
  converter.
  (rb_trans_open): crlf newline and cr newline implemented.
  (Init_transcode): Encoding::Converter::CRLF_NEWLINE and
  Encoding::Converter::LF_NEWLINE defined.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18557 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-13 05:48:57 +00:00
akr 74a2a7bdbf * enc/trans/newline.trans: new file. 
* transcode_data.h (rb_trans_t): add last_tc field.

* transcode.c (UNIVERSAL_NEWLINE): defined.
  (CRLF_NEWLINE): defined.
  (CR_NEWLINE): defined.
  (rb_trans_open_by_transcoder_entries): initialize last_tc.
  (trans_open_i): allocate one more room for newline converter.
  (rb_trans_open): universal newline implemented.
  (more_output_buffer): take max_output argument instead ts.
  (output_replacement_character): take tc argument instead of ts.
  (transcode_loop): use last_tc field.
  (econv_init): add flags argument for rb_trans_open.
  (Init_transcode): Encoding::Converter::UNIVERSAL_NEWLINE defined.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18556 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-13 05:30:42 +00:00
akr 79cd32fe3b * enc/trans/make_transdb.rb: *.erb.c is not used anymore. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18532 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-12 09:43:44 +00:00
usa 6d8c1483d6 * enc/depend: (transvpath_prefix): prefix has no extension, so replace 
%s with "".



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18517 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-12 06:19:00 +00:00
nobu 13f1418d54 * enc/Makefile.in (.SUFFIXES): renamed to .trans. 
* enc/make_encmake.rb: added --encs and --no-encs options.

* enc/depend (TRANSVPATH): fix for nmake.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18513 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-12 05:25:14 +00:00
akr 94ca2d94de * transcode_data.h (rb_transcoder): add resetstate_func field for 
resetting a state of stateful encoding.

* enc/trans/iso2022.trans (rb_EUC_JP_to_ISO_2022_JP): specify
  finish_eucjp_to_iso2022jp for resetstate_func.

* tool/transcode-tblgen.rb: specify NULL for resetstate_func.

* transcode.c (output_replacement_character): call resetstate_func
  before appending the replacement character.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18503 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-11 22:44:23 +00:00
akr 2897632f16 * enc/trans/iso2022.trans: renamed from iso2022.erb.c. 
* enc/trans/single_byte.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.

* enc/trans/korean.trans: ditto.

* enc/trans/japanese.trans: ditto.

* enc/depend: follow the renaming.

* tool/build-transcode: ditto.




git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-11 07:39:52 +00:00
nobu e3d9fc76e6 * enc/Makefile.in (make-workdir): use MAKEDIRS. 
* enc/depend: makes target directory before compile/link.

* tool/transcode-tblgen.rb: creates target directory.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18469 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-10 02:45:18 +00:00
akr 139234e1a0 * transcode_data.h (rb_transcoding): add fields for restartable 
transcoding.
  (rb_transcoder): add max_input field.
  from_unit_length field is renamed to input_unit_length.

* tool/transcode-tblgen.rb: generate max_input field.

* enc/trans/iso2022.erb.c: follow rb_transcoder change.

* enc/trans/utf_16_32.erb.c: ditto.

* transcode.c (PARTIAL_INPUT): new constant.
  (transcode_char_start): new function.
  (transcode_result_t): new type.
  (transcode_restartable): new function.
  (more_output_buffer): new function.
  (transcode_loop): use transcode_restartable.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18452 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-09 06:02:01 +00:00
nobu 1727355a22 * enc/make_encdb.rb, enc/trans/make_transdb.rb: skip nonexistent 
directory.  [ruby-dev:35802]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18449 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-08 21:37:43 +00:00
akr 6db9e77e64 * enc/trans/utf_16_32.erb.c (fun_so_from_utf_32le): implemented. 
(fun_so_to_utf_32le): implemented.
  [ruby-dev:35777]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-08 16:08:50 +00:00
akr 2833d9f95d * transcode_data.h (rb_transcoder): from_unit_length field added. 
from_utf8 field removed.

* tool/transcode-tblgen.rb: generate offsets range.
  follow rb_transcoder change.

* transcode.c (transcode_loop): don't use from_utf8.
  make invalid region from_unit_length wise.

* enc/trans/iso2022.erb.c: follow rb_transcoder and 
  transcode_generate_node change.

* enc/trans/utf_16_32.erb.c: follow rb_transcoder and
  transcode_generate_node change.
  explicit :invalid map removed.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18445 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-08 15:48:17 +00:00
nobu a456f022fc * enc/depend (TRANSCSRCS): needs rule_subst to apply. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18444 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-08 14:30:05 +00:00
nobu 2293171034 * common.mk (srcs-enc): renamed from transcodes. 
* enc/Makefile.in (make-workdir): creates object directories.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18437 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-08 07:44:42 +00:00
nobu b917553e68 * common.mk (encdb.h): see both $(srcdir)/enc and enc. 
* enc/make_encdb.rb: ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18435 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-08 07:15:52 +00:00
nobu 2f37e03736 * enc/trans/make_transdb.rb: fix for the case no transdirs are given. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18434 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-08 07:09:03 +00:00
nobu 73d3ff0074 * enc/trans/make_transdb.rb: converts only one transcoders for each 
basename.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18433 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-08 07:03:35 +00:00
naruse b41a687fe6 * common.mk: see both $(srcdir)/enc/trans and enc/trans. 
* enc/trans/make_transdb.rb: ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18422 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-07 19:24:47 +00:00
akr 1504652373 * transcode_data.h (rb_transcoding): new field "stateful". 
(rb_transcoder): preprocessor and postprocessor field removed.
  change arguments of func_ii, func_si, func_io and func_so.
  new field "finish_func".

* tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio
  generatable.

* transcode.c (transcoder_lib_table): removed.
  (transcoder_table): change structure.
  (transcoder_key): removed because the above structure change.
  (make_transcoder_entry): new function.
  (get_transcoder_entry): ditto.
  (rb_register_transcoder): follow the structure change.
  (declare_transcoder): ditto.
  (transcode_search_path): new function for breadth first search to
  find a list of converters.
  (transcode_search_path_i): new function.
  (transcode_dispatch_cb): ditto.
  (transcode_dispatch): use transcode_search_path.
  (transcode_loop): follow the argument change.
  (str_transcode): preprocessor and postprocessor stuff removed.

* enc/trans/iso2022.erb.c: new file.  ISO-2022-JP conversion
  re-implemented.

* enc/trans/japanese.erb.c: ISO-2022-JP stuff removed.

nute(23:52:53)% head -40 ChangeLog
Thu Aug  7 23:43:11 2008  Tanaka Akira  <akr@fsij.org>

* transcode_data.h (rb_transcoding): new field "stateful".
  (rb_transcoder): preprocessor and postprocessor field removed.
  change arguments of func_ii, func_si, func_io and func_so.
  new field "finish_func".

* tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio
  generatable.

* transcode.c (transcoder_lib_table): removed.
  (transcoder_table): change structure.
  (transcoder_key): removed because the above structure change.
  (make_transcoder_entry): new function.
  (get_transcoder_entry): ditto.
  (rb_register_transcoder): follow the structure change.
  (declare_transcoder): ditto.
  (transcode_search_path): new function for breadth first search to
  find a list of converters.
  (transcode_search_path_i): new function.
  (transcode_dispatch_cb): ditto.
  (transcode_dispatch): use transcode_search_path.
  (transcode_loop): follow the argument change.
  (str_transcode): preprocessor and postprocessor stuff removed.

* enc/trans/iso2022.erb.c: new file.  ISO-2022-JP conversion
  re-implemented.

* enc/trans/japanese.erb.c: ISO-2022-JP stuff removed.

* enc/trans/utf_16_32.erb.c: follow argument change of FUNso.

[ruby-dev:35798]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18419 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-07 14:53:30 +00:00
nobu 380e558f33 * enc/depend: add transdb.c. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18412 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-07 08:47:58 +00:00
nobu fa3283c7ba * enc/depend: removed needless explicit commands. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18410 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-07 05:17:34 +00:00
naruse ec85af4955 * enc/depend: enc/*.c is source but enc/trans/*.c is generated. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18400 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-06 20:19:25 +00:00
naruse 508b170dcd * enc/depend: for build in other than srcdir. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18396 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-06 19:44:50 +00:00
akr e22b3b773d * tool/transcode-tblgen.rb (transcode_generate_node): code 
argument removed.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18395 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-06 13:42:21 +00:00
akr a016b30844 * enc/depend: transcode table generation depends on 
tool/transcode-tblgen.rb.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18393 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-06 11:57:32 +00:00
akr fc841ddc66 useless comment removed. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-06 11:44:08 +00:00
nobu 8db17b30d6 * common.mk (transdb.h): requires transcoders. 
* enc/depend (srcs): target for transcoders.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18387 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-06 05:40:13 +00:00
usa ce074dcf42 * enc/depend: replace not only $(<:...) but also $<. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-06 05:04:36 +00:00
usa 19e2cdbd47 * win32/Makefile.sub (config.status): export BASERUBY. 
* enc/depend: avoid GNU make'ism.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18384 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-06 04:55:39 +00:00
akr e90dd02529 * tool/transcode-tblgen.rb: show generating tables in verbose mode. 
(transcode_generate_node): call ActionMap#generate_node with showing
  table name.

* enc/trans/utf_16_32.erb.c: use transcode_generate_node.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-05 22:47:44 +00:00
nobu 0031e09170 * common.mk (transcodes), tool/build-transcode: generates transcode 
sources.

* enc/trans/{japanese,korean,single_byte,utf_16_32}.c: to be
  autogenerated now.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18376 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-05 18:56:42 +00:00
nobu 82e89f1237 * enc/depend: added rules for .c from .erb.c. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18374 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-05 18:15:52 +00:00
akr f694ec83e8 * tool/build-transcode: new file. 
* tool/transcode-tblgen.rb: new file.

* enc/trans/make_transdb.rb: exclude *.erb.c.

* enc/depend: exclude *.erb.c.

* enc/trans/utf_16_32.erb.c: new file.

* enc/trans/single_byte.erb.c: new file.

* enc/trans/japanese.erb.c: new file.

* enc/trans/korean.erb.c: new file.

* enc/trans/iso-8859-2-tbl.rb: new file.

* enc/trans/iso-8859-3-tbl.rb: new file.

* enc/trans/iso-8859-4-tbl.rb: new file.

* enc/trans/iso-8859-5-tbl.rb: new file.

* enc/trans/iso-8859-6-tbl.rb: new file.

* enc/trans/iso-8859-7-tbl.rb: new file.

* enc/trans/iso-8859-8-tbl.rb: new file.

* enc/trans/iso-8859-9-tbl.rb: new file.

* enc/trans/iso-8859-10-tbl.rb: new file.

* enc/trans/iso-8859-11-tbl.rb: new file.

* enc/trans/iso-8859-13-tbl.rb: new file.

* enc/trans/iso-8859-14-tbl.rb: new file.

* enc/trans/iso-8859-15-tbl.rb: new file.

* enc/trans/eucjp-tbl.rb: new file.

* enc/trans/sjis-tbl.rb: new file.

* enc/trans/euckr-tbl.rb: new file.

* enc/trans/utf_16_32.c: regenerated.

* enc/trans/single_byte.c: regenerated.

* enc/trans/japanese.c: regenerated.

* enc/trans/korean.c: regenerated.

[ruby-dev:35730]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18373 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-05 12:32:13 +00:00
nobu b84d31c524 * transcode_data.h (TRANSCODE_ERROR): common transcode failure 
exception, would be changed later.

* enc/trans/japanese.c (UNSUPPORTED_MODE): unsupported mode transition
  exception.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18363 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-05 03:34:52 +00:00
naruse 4de6475690 * enc/trans/japanese.c: add U+FF5E to EUC-JP. 
[ruby-dev:35720] [ruby-dev:35722]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18315 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-01 16:26:23 +00:00
naruse 4fc3ef70e1 * enc/trans/japanese.c: add support for CP51932, 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18310 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-01 14:17:21 +00:00
naruse 35bac2083c * enc/trans/japanese.c: add U+FF0C, 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-01 14:03:14 +00:00
naruse 143b3339e6 * enc/trans/japanese.c (to_SHIFT_JIS_EF_BF_offsets): add U+FFF3, 
U+FFF4, U+FFF5.

* enc/trans/japanese.c (to_SHIFT_JIS_EF_BF_infos): ditto.

* enc/trans/japanese.c (to_EUC_JP_EF_BF_infos): added.

* enc/trans/japanese.c (to_EUC_JP_EF_BF): added.

* enc/trans/japanese.c (to_EUC_JP_EF_infos): change size.
  [ruby-dev:35714]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18305 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-01 12:26:56 +00:00
nobu 94ed51b281 * transcode.c (transcode_loop): constified. 
* transcode.c (str_transcode): rb_str_set_len() sets a delimiter.

* transcode_data.h (rb_transcoder): constified preprocessor and
  postprocessor input.

* enc/trans/japanese.c: ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-07-14 09:47:33 +00:00
naruse 5b7dfb7c13 * enc/shift_jis.c (code_is_ctype): HALF WIDTH KATAKANA is 
a character.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17774 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-07-01 10:05:48 +00:00
shyouhei 0ef21e44e7 forgot to commit 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17773 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-07-01 09:50:30 +00:00
usa 40612c5bb5 * enc/make_encdb.h: always add ';' at the end of line. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17771 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-07-01 08:51:33 +00:00
shyouhei 53cffba8cb * enc/ascii.c: ISO C does not allow extra ';' outside of a 
function

  	* enc/us_ascii.c: ditto.

	* enc/utf_8.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17768 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-07-01 08:39:10 +00:00
mame d7baa40cac * enc/euc_jp.c (property_name_to_ctype): core dumped when sizeof(int) 
differs from sizeof(long).

* enc/shift_jis.c (property_name_to_ctype): ditto.

* enc/unicode.c (onigenc_unicode_property_name_to_ctype): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17381 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-06-17 13:06:34 +00:00
nobu fc0babb4c2 * enc/depend (clean): remove build directories. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17060 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-06-09 12:17:29 +00:00
nobu ce29c17877 * lib/mkmf.rb (configuration): set flags. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17058 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-06-09 12:14:39 +00:00
mame 65670f9400 * enc/iso_8859_5.c: Large omicron should lowercase to small omicron. 
* test/ruby/test_big5.rb, test/ruby/test_cp949.rb,
  test/ruby/test_euc_jp.rb, test/ruby/test_euc_kr.rb,
  test/ruby/test_euc_tw.rb, test/ruby/test_gb18030.rb,
  test/ruby/test_gbk.rb, test/ruby/test_iso_8859.rb,
  test/ruby/test_koi8.rb, test/ruby/test_shift_jis.rb,
  test/ruby/test_windows_1251.rb: new tests for encoding.

* test/ruby/test_utf16.rb, test/ruby/test_utf32.rb,
  test/ruby/test_regexp.rb: add tests.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16759 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-06-02 13:30:38 +00:00
naruse 5397015d2e * enc/gb18030.c (gb18030_code_to_mbc): add 0x80000000 
for 4bytes character.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16739 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-06-01 19:36:28 +00:00
naruse 9c13fc7d89 * enc/gb18030.c (gb18030_mbc_to_code): mask by 0x7FFFFFFF 
because OnigCodePoint will be used as 32bit signed int.
  Masking by 0x7FFFFFFF is ok on GB18030;
  Minumum 4bytes character is 0x81308130.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16737 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-06-01 18:29:08 +00:00