for test), so should permit to run ruby if unicode_normalize.rb is
missing.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48060 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
the methods that are available on String are available without explicit require.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48023 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/unicode.c (init_case_fold_table): no longer need to
initialize tables at runtime.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46273 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/unicode/case-folding.rb (lookup_hash): make perfect hash to
lookup case unfolding table 3.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46272 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/unicode/case-folding.rb (lookup_hash): make perfect hash to
lookup case unfolding table 2.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46271 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/unicode/case-folding.rb (lookup_hash): make perfect hash to
lookup case unfolding table 1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46270 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/unicode/case-folding.rb (lookup_hash): make perfect hash to
lookup case folding table.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46269 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/jis/props.kwd: constify character property tables of JIS
based encodings by perfect hash.
* enc/euc_jp.c, enc/shift_jis.c: use character property functions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46039 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* complex.c, rational.c: remove unused functions, which are warned
by clang 5.1, and also variables only used by removed functions.
* ext/date/date_core.c: ditto.
* enc/utf_16be.c, enc/utf_16le.c: comment out constants only used
by commented out functions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/encdb.c, enc/utf_16_32.h (ENC_DUMMY_UNICODE): Unicode with BOM
must be based on big endian variants, so that actual encodings would
work. [ruby-core:57318] [Bug #8940]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43023 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Previous table is used on Mac OS X 10.1 or prior.
This table is used on 10.2 or later. [ruby-dev:47680]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42789 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
tool/transcode-tblgen.rb: change EUC-JP-2004 to EUC-JIS-2004.
This is follow up to changes in r41024.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41035 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/depend (ENCOBJS, TRANSOBJS): use explicit path to ruby.h for
nmake.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/depend (ENCOBJS, TRANSOBJS): fix header dependency, VPATH has
$(srcdir)/include/ruby but not $(srcdir)/include, so cannot find out
ruby/ruby.h. use ruby.h instead and ../ruby for include/ruby.h.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40186 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/depend (ARFLAGS, RANLIB): these values can be nil. [Bug #7950]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@39490 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/depend (ARFLAGS): VisualC++ linker does not allow spaces between
output option and the output file name. [Bug #7950]
* enc/depend (RANLIB): set default command to do nothing, or make the
entire line a label on Windows.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@39489 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/depend: fix inplace-build condition. enc.mk is generated with
setting $srcdir to enc, but pwd is still top build direcory.
[ruby-core:47236] [Bug #6888]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36725 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/enc-unicode.rb: add comment why it uses Hash#index.
* enc/unicode/{name2ctype.kwd,name2ctype.src,name2ctype.h.blt}:
update to follow the current name2ctype.h.
FYI current Unicode version is 6.1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36070 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
ONIGERR_INVALID_CODE_POINT_VALUE if the code is invalid.
* enc/shift_jis.c (tr_next): increment character until the code
is a valid character. [ruby-dev:45652] [Bug #6450]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35724 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/encinit.c.erb: use %-lines to adjust indent in the generated file.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35670 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Fixes --with-static-linked-ext.
Patch by Google Inc. [ruby-core:45073].
* Makefile.in (ENCOBJS, EXTOBJS): New variables to specify static
linked libraries. Also reintroduces extinit.o, introduces encinit.o
introduces encinit.o
* common.mk: Builds static libraries rather than shared objects if
specified.
* configure.in (LD): new substitution.
Avoids PIE if s
* enc/depend: Supports static linked libraries
(libencs, libenc, libtrans): New target.
* enc/encinit.c.erb: new template to generate the initialization of
statically linked encodings.
* enc/make_encmake.rb (--module): new flag to specify whether static
or dynamic.
* transcode_data.h (TRANS_INIT): New macro to get rid of the name
collision of encoding initializers and transcoder initializers.
* ext/extmk.rb: Fixes the behavior on $extstatic is true.
* lib/mkmf.rb (clean-static): new target to clean up static linked
libraries.
* ruby.c (process_options): New initializes statically linked
encodings here.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
fix 'array of strings' to 'array of symbols'
[ruby-core:44152][Bug #6264]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35244 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Merge Onigmo 27278c12e6674043cc8affca6507e20e119a86ee.
* regparse.c (is_onechar_cclass): [bug] unexpected match occurs when a
char class contains no char
* enc/unicode.c (init_case_fold_table): define the sizes of case
folding tables in casefold.h
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34860 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rest_sweep() instead of it, because some dead objects might be
marked in next the mark phase by false pointers.
[ruby-core:42672]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34719 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
is invalid. [Feature #5855] [Bug #5863] [Bug #5864]
* string.c (rb_str_concat): ditto.
* string.c (rb_str_concat): set encoding as ASCII-8BIT when the string
is US-ASCII and the argument is an integer greater than 127.
* regenc.c (onigenc_mb2_code_to_mbclen): rearrange error code.
* enc/euc_jp.c (code_to_mbclen): ditto.
* enc/shift_jis.c (code_to_mbclen): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34236 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
--
* enc/Makefile.in (ECHO1): Same as the recent fix in common.mk.
":" in a make variable replacement cause a syntax error with
/usr/ccs/bin/make on Solaris. Uses $(NULLCMD) instead.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@32787 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
CP932 UDA. Another reason is emacs-mule: the implementation of
stateless-iso-2022-jp doesn't support beyond 94x94 (0x7fxx);
but CP932 UDA is in 7Fxx-92xx.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31366 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
patched by oCameLo oTnTh [ruby-core:33256]
* enc/big5.c: add alias Big5-HKSCS:2008 to Big5-HKSCS.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29922 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/trans/utf_16_32.trans: add a converter from UTF-16 to UTF-8.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29889 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt,
enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src:
Add Age property to regexp. [ruby-core:33019]
patched by Ammar Ali, tested by Run Paint Run Run
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29717 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt,
enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src:
Update Oniguruma for Unicode 6.
patched by Run Paint Run Run. [ruby-core:32923] #3989
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29620 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
--
* enc/Makefile.in (realclean): has been missing. necessary
for make realclean-enc.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29177 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/trans/single_byte.trans: remove ambiguous maping such as
\xD6 -> U+05F2 and \xD6\xC7 -> U+FB1F in Windows-1255
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
support for new transcoding instruction FUNsio (with Tatsuya Mizuno)
* enc/trans/gb18030.trans: Significantly reduced GB18030 conversion
table footprint using FUNsio and differences (with Tatsuya Mizuno)
* test/ruby/test_transcode.rb: Minor name fix (from Tatsuya Mizuno)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
(from Tatsuya Mizuno)
* test/ruby/test_transcode.rb: Added test for converting full range of
Unicode codepoints from/to GB18030 (from Tatsuya Mizuno)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25980 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
after \r\n detection instead of just after \r.
[ruby-list:45988] [ruby-core:25881] [ruby-core:26788]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
test/ruby/test-transcode.rb: Added Encoding 'Big5-UAO' and transcoding
for it (from Tatsuya Mizuno) (see Bug #1784)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25822 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt,
enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src:
use UTS#18 for POSIX character class.
http://rubyspec.org/issues/show/161
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25338 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/enc-unicode.rb,
enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt,
enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src:
Add DerivedCoreProperties, PropList (Binary Property),
PropertyAlias and PropertyValueAlias.
Now users of tool/enc-unicode.rb should specify
the directory of UCD files.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25324 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd,
enc/unicode/name2ctype.src: Updated to Unicode 5.2.0.
NOTE: when you update these data, download UnicodeData.txt
and Scripts.txt from http://www.unicode.org/Public/UNIDATA/
and run
ruby1.9 tool/enc-unicode.rb UnicodeData.txt Scripts.txt \
> enc/unicode/name2ctype.kwd
* enc/unicode/Scripts.txt: removed.
* enc/unicode/UnicodeData.txt: removed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25190 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/enc-unicode.rb: added for generate name2ctype.kwd.
contributed by Run Paint Run Run [ruby-core:24775]
use like following:
ruby19 tool/enc-unicode.rb enc/unicode/UnicodeData.txt \
enc/unicode/Scripts.txt > enc/unicode/name2ctype.kwd
* enc/unicode.c (CodeRanges): move definitions to name2ctype.h.
* enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd,
enc/unicode/name2ctype.src: updated to v5.1.
* enc/unicode/UnicodeData.txt, enc/unicode/Scripts.txt: added v5.1.
* Makefile.in: add rule to generate name2ctype.kwd from
UnicodeData.txt and Scripts.txt.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24651 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* enc/big5.c (EncLen_Big5): back to original Big5 table.
(EncLen_Big5_HKSCS): for Big5-HKSCS.
(trans): add the lead byte table for Big5-HKSCS.
(big5_mbc_enc_len): abstract function for Big5 series.
(big5_mbc_enc_len): for Big5.
(big5_hkscs_mbc_enc_len): for Big5-HKSCS.
(BIG5_HKSCS_P): added.
(BIG5_ISMB_FIRST): add routine for Big5-HKSCS.
(big5_hkscs): add for Big5-HKSCS.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24384 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c (rb_enc_set_base): Add for setting base encoding
with their names. this is internal function.
* template/encdb.h.tmpl: specify ENC_SET_BASE for second
encodings in each encoding files.
* enc/encdb.c (rb_enc_set_base): add a declaration.
(ENC_SET_BASE): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24383 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
new Chinese BIG5-HKSCS transcoding (with Tatsuya Mizuno)
* test/ruby/test_transcode.rb: added tests for the above
(with Tatsuya Mizuno)
* enc/big5.c: Added BIG5-HKSCS as a replicate encoding of BIG5
(short term solution, needs more work; with Tatsuya Mizuno)
* tool/transcode-tblgen.rb: made 'pat' directly accessible in
class StrSet
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24264 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
new Chinese GBK transcoding (from Yoshihiro Kambayashi)
* test/ruby/test_transcode.rb: added tests for the above
(from Yoshihiro Kambayashi)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21315 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
new Chinese Big5 transcoding (from Yoshihiro Kambayashi)
* test/ruby/test_transcode.rb: added tests for the above
(from Yoshihiro Kambayashi)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21313 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to add more transcodings (with Yoshihiro Kambayashi)
* enc/trans/iso-8859-1-tbl.rb: new file to avoid having to
treat ISO-8859-1 as special
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@20054 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Tadashi Saito <shiba AT mail2.accsnet.ne.jp> at [ruby-dev:36916]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
to reduce coupling between table generation script and
specific encodings.
* enc/trans/single_byte.trans: using set_valid_byte_pattern
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19831 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
from transcode_tblgen_windows.
(transcode_tblgen_iso8859): use transcode_tblgen_singlebyte.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19780 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
also returns ssize_t.
* enc/trans/iso2022.trans: follow the type change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* transcode.c (transcode_restartable0): don't need to cast the result
of output functions.
* enc/trans/newline.trans: follow the type change.
* enc/trans/escape.trans: ditto.
* enc/trans/utf_16_32.trans: ditto.
* enc/trans/iso2022.trans: ditto.
* enc/trans/japanese.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19351 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
from rb_econv_stateless_encoding to apply stateless ASCII
incompatible encodings such as UTF-16BE.
* io.c (make_writeconv): use rb_econv_asciicompat_encoding.
* transcode_data.h (rb_transcoder_asciicompat_type_t): renamed from
rb_transcoder_stateful_type_t.
(rb_transcoder): use rb_transcoder_asciicompat_type_t.
* transcode.c: follow the type change.
(asciicompat_encoding_i): renamed from stateless_encoding_i.
(rb_econv_asciicompat_encoding): renamed from
rb_econv_stateless_encoding.
(econv_s_asciicompat_encoding): method renamed.
* tool/transcode-tblgen.rb: follow the type change.
* enc/trans/utf_16_32.trans: follow the type change.
rb_from_UTF_16BE to UTF-8 is asciicompat_decoder.
rb_from_UTF_16LE to UTF-8 is asciicompat_decoder.
rb_from_UTF_32BE to UTF-8 is asciicompat_decoder.
rb_from_UTF_32LE to UTF-8 is asciicompat_decoder.
UTF-8 to rb_to_UTF_16BE is asciicompat_encoder.
UTF-8 to rb_to_UTF_16LE is asciicompat_encoder.
UTF-8 to rb_to_UTF_32BE is asciicompat_encoder.
UTF-8 to rb_to_UTF_32LE is asciicompat_encoder.
* enc/trans/newline.trans: follow the type change. universal newline
decoder is asciicompat_converter.
* enc/trans/escape.trans: follow the type change.
* enc/trans/iso2022.trans: ditto.
* enc/trans/japanese.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19249 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
undefined conversion error between iso-2022-jp and the corresponding
stateless encoding.
* enc/emacs_mule.c: replicate emacs-mule as stateless-iso-2022-jp.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19220 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/transcode-tblgen.rb: generate an empty line after str1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19217 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
akr
usa
duerst
naruse
nobu
kazu
ktsj
yugui
nari
muraken
matz