Граф коммитов

417 Коммитов

Автор SHA1 Сообщение Дата
nobu 3e65d11090 * transcode.c (transcode_dispatch): fix for multistep transcode.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14669 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-25 06:21:35 +00:00
nobu b7db9036be * common.mk (COMMONOBJS): transcode_data_*.c moved under enc/trans.
* transcode_data.h (rb_transcoding, rb_transcoder): prefixed.

* transcode.c (rb_register_transcoder, rb_declare_transcoder): split
  declaration and registration.  [ruby-dev:32704]

* transcode.c (transcode_dispatch): autoload pre-declared transcoder.

* transcode.c (str_transcode): use rb_define_dummy_encoding().

* transcode.c (Init_transcode): initialize transcoder tables.

* enc/trans/single_byte.c, enc/trans/japanese.c: moved from top.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14666 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-25 05:57:04 +00:00
duerst e7ac333ba8 Tue Dec 25 12:32:32 2007 Martin Duerst <duerst@it.aoyama.ac.jp>
* transcode.c: Moving a static counter from inside register_transcoder()
	  and register_functional_transcoder() to outside the functions, renaming
	  from n to next_transcoder_position. Fixes 3) in [ruby-dev:32715].



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14651 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-25 03:33:23 +00:00
naruse be86e3de33 * transocode.c: register_functional_transcoder() added.
(init_transcoder_table(: register ISO-2022-JP.
  (str_transcode): add preprocessor and postprocessor.

* transcode_data_japanese.c: add ISO-2022-JP support.

* transcode_data.h: moved transcoder and transcoding difinition from
  transcode.c.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14607 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-24 13:51:19 +00:00
duerst 9c7718ac6b Mon Dec 24 09:45:45 2007 Martin Duerst <duerst@it.aoyama.ac.jp>
* transcode.c, transcode_data_one_byte.c, transcode_data_japanese.c:
	  added rb_ prefix to external data symbols.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14561 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-24 00:47:21 +00:00
akr 5b809a28f8 * include/ruby/encoding.h, encoding.c, re.c, io.c, parse.y, numeric.c,
ruby.c, transcode.c: rename rb_ascii_encoding. to
  rb_ascii8bit_encoding.  rb_ascii_encoding is ambiguous with 
  ASCII-8BIT and US-ASCII.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14504 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-22 23:47:18 +00:00
duerst 5ad8c5566d Sat Dec 22 15:45:45 2007 Martin Duerst <duerst@it.aoyama.ac.jp>
* transcode_data_one_byte: slightly optimized

	* transcode_data_japanese: new data file for EUC-JP and SHIFT_JIS
	  (not yet optimized; tests to follow; data from
	   http://nkf.sourceforge.jp/ucm/{SJIS|eucJP}-nkf.ucm)

	* common.mk, transcode.c: Adjusted for transcode_data_japanese



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14472 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-22 06:45:55 +00:00
matz d7cc14d436 * encoding.c (rb_ascii_encoding): renamed from previous
rb_default_encoding().

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14443 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-21 18:55:30 +00:00
nobu d66a188c4a * transcode.c (rb_str_transcode_bang): returns self if no conversion.
[ruby-dev:32662]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14425 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-21 08:49:08 +00:00
nobu 98da73bc96 * transcode.c (rb_str_transcode_bang, rb_str_transcode): set new
encoding even if no conversion is done because of 7bit only.
  [ruby-dev:32591]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14293 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-18 08:04:26 +00:00
matz 5c4cf9bfdf for undefined conversions.
* transcode_data_iso_8859.c: Changed from character constants
  ('\xC2') to integer contants (0xC2) for shorter files and
  better readability; eliminated duplicated tables; changed
  from -1 offset to actual UNDEF entry (not yet distinguishing
  UNDEF and ILLEGAL correctly).

* test/ruby/test_transcode.rb: added a test for UNDEF conversion.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14251 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-17 01:28:26 +00:00
matz f2b0dba1cf * transcode.c (str_transcode, transcode_dispatch): added two-step
* trancode.c: some minor formatting fixes

* transcode_data.h, transcode_data_iso_8859.c: Shortened
  extremely frequently used macros to shorten file length.

* test/ruby/test_transcode.rb: Fixed name of test class;
  added setup method to ensure all necessary encodings exist;
  split tests into more test methods; added tests; fixed ordering
  of arguments in assert_equal to have expected result first.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14236 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-15 05:42:25 +00:00
nobu a153fdb6ed * transcode.c (transcode_loop): get rid of SEGV at sequence can not be
converted.

* transcode.c (rb_str_transcode_bang): copy encoding.  [ruby-dev:32532]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14191 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-11 04:57:51 +00:00
nobu 6d7999c132 * transcode.c (str_transcode): allow non-registered encodings.
[ruby-dev:32520]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14182 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-10 12:47:55 +00:00
nobu 3a3bda73dd * string.c (rb_str_tmp_new): creates hidden temporary buffer.
* transcode.c (transcoding): added a pointer to function to flush.

* transcode.c (transcode_loop): do not use string internal.
  [ruby-dev:32512]

* transcode.c (str_transcode): allow Encoding objects.

* transcode_data.h (BYTE_LOOKUP): use actual struct name.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14176 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-10 08:46:06 +00:00
nobu 38b92f838f * transcode*.[ch], test/ruby/test_transcode.rb: set properties.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14175 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-10 08:25:01 +00:00
matz 7ded13f54b * transcode.c: new file to provide encoding conversion features.
code contributed by Martin Duerst.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14172 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-10 05:01:47 +00:00