github
/
ruby
зеркало из https://github.com/github/ruby.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
48 300 коммитов 217 Ветки 1 255 Теги 471 MiB
Граф коммитов

28 Коммитов

Автор SHA1 Сообщение Дата
duerst 11954049fa Change max byte length of UTF-8 to 4 bytes 
In enc/utf_8.c, change maximum byte length of UTF-8 to 4 bytes (from 6)
to conform to definition of UTF-8. This closes issue #13590.
(This is a retry of r58954, after issue #13590 has been addressed.)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58965 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2017-05-30 05:43:41 +00:00
duerst e07bff3ce3 revert r58954 temporarily 
Revert change to maximum of 4 bytes for UTF-8 characters at r58954 temporarily.
This failed spec at https://travis-ci.org/ruby/ruby/builds/237086017, but it
is totally unclear why.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58955 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2017-05-29 08:59:41 +00:00
duerst a03690ae73 Change max byte length of UTF-8 to 4 bytes 
In enc/utf_8.c, change maximum byte length of UTF-8 to 4 bytes (from 6)
to conform to definition of UTF-8. This closes issue #13590.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58954 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2017-05-29 08:41:23 +00:00
naruse 2873edeafb Merge Onigmo 6.0.0 
* https://github.com/k-takata/Onigmo/blob/Onigmo-6.0.0/HISTORY
* fix for ruby 2.4: https://github.com/k-takata/Onigmo/pull/78
* suppress warning: https://github.com/k-takata/Onigmo/pull/79
* include/ruby/oniguruma.h: include onigmo.h.
* template/encdb.h.tmpl: ignore duplicated definition of EUC-CN in
  enc/euc_kr.c. It is defined in enc/gb2313.c with CRuby macro.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57045 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2016-12-10 17:47:04 +00:00
duerst 6ed393ad89 * regenc.h/c, include/ruby/oniguruma.h, enc/ascii.c, big5.c, cp949.c, 
emacs_mule.c, euc_jp.c, euc_kr.c, euc_tw.c, gb18030.c, gbk.c,
  iso_8859_1|2|3|4|5|6|7|8|9|10|11|13|14|15|16.c, koi8_r.c, koi8_u.c,
  shift_jis.c, unicode.c, us_ascii.c, utf_16|32be|le.c, utf_8.c,
  windows_1250|51|52|53|54|57.c, windows_31j.c, unicode.c:
  Remove conditional compilation macro ONIG_CASE_MAPPING. [Feature #12386].


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55740 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2016-07-24 07:33:15 +00:00
duerst 78c5ca7074 * include/ruby/oniguruma.h: Extend OnigEncodingTypeDefine to define a 
new encoding primitive 'case_map' for case mapping
* enc/utf-8.c, utf_16be/le.c, utf_32be/le.c:
  add onigenc_unicode_case_map as case_map primitive
* enc/ascii.c, big5.c, cp949.c, emacs_mule.c, euc_jp/kr/tw.c, gb18030.c,
  gbk.c, iso_8859_1/2/3/4/5/6/7/8/9/10/11/13/14/15/16.c, koi8_r/u.c,
  shift_jis.c, us_ascii.c, windows_1250/1251/1252.c:
  add onigenc_not_support_case_map as case_map primitive


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55113 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2016-05-22 05:57:44 +00:00
nobu 75a1d33bcf enc: fundamental encindex 
* enc/{ascii,us_ascii,utf_8}.c: set encoding indexes of
  fundamental built-in encodings so that usable as well as
  allocated rb_encoding before rb_enc_init().

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51862 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2015-09-15 05:49:52 +00:00
nobu 859f88f330 enc/utf_8.c: limit UTF-8 
* enc/utf_8.c (code_to_mbclen, code_to_mbc): reject values larger
  than UTF-8 max codepoints.  [Feature #11094]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50392 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2015-04-25 22:36:52 +00:00
naruse d2a5354255 * reg*.c: Merge Onigmo 5.15.0 38a870960aa7370051a3544 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47598 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2014-09-15 16:18:41 +00:00
naruse 9ae509504b Revert r41786 "broken utf-8" 
revert unexpected commit

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2013-07-09 07:38:15 +00:00
naruse ed37bfcc9d broken utf-8 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41786 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2013-07-05 00:54:11 +00:00
naruse 0424e152c6 * Merge Onigmo-5.13.1. [ruby-dev:45057] [Feature #5820] 
https://github.com/k-takata/Onigmo
  cp reg{comp,enc,error,exec,parse,syntax}.c reg{enc,int,parse}.h
  cp oniguruma.h
  cp tool/enc-unicode.rb
  cp -r enc/

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34663 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2012-02-17 07:42:23 +00:00
nobu 3a47cf3395 * remove trailing spaces. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31573 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2011-05-15 11:55:52 +00:00
naruse 90be970018 * enc/utf_8.c: Add new alias UTF-8-HFS for UTF8-MAC. 
http://www.gnu.org/software/emacs/NEWS.23.2

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27736 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2010-05-11 06:15:53 +00:00
nobu a606038c6a * enc/utf_8.c (code_to_mbc): suppressed a warning. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24607 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2009-08-21 06:37:36 +00:00
akr 081c802cb9 * grapheme cluster implementation reverted. [ruby-dev:36375] 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19417 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-18 12:53:25 +00:00
akr b3d772643e fix typos. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19390 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-16 16:59:08 +00:00
akr a67d4fa01c * include/ruby/oniguruma.h (OnigEncodingTypeST): add precise_ret 
argument for mbc_to_code.
  (ONIGENC_MBC_TO_CODE): provide NULL for precise_ret.
  (ONIGENC_MBC_PRECISE_CODEPOINT): defined.

* include/ruby/encoding.h (rb_enc_mbc_precise_codepoint): defined.

* regenc.h (onigenc_single_byte_mbc_to_code): precise_ret argument
  added.
  (onigenc_mbn_mbc_to_code): ditto.

* regenc.c (onigenc_single_byte_mbc_to_code): precise_ret argument
  added.
  (onigenc_mbn_mbc_to_code): ditto.

* string.c (count_utf8_lead_bytes_with_word): removed.
  (str_utf8_nth): removed.
  (str_utf8_offset): removed.
  (str_strlen): UTF-8 codepoint oriented optimization removed.
  (rb_str_substr): ditto.
  (enc_succ_char): use rb_enc_mbc_precise_codepoint.
  (enc_pred_char): ditto.
  (rb_str_succ): ditto.

* encoding.c (rb_enc_ascget): check length with
  rb_enc_mbc_precise_codepoint.
  (rb_enc_codepoint): use rb_enc_mbc_precise_codepoint.

* regexec.c (string_cmp_ic): add text_end argument.
  (match_at): check end of character after exact string matches.

* enc/utf_8.c (graphme_table): defined for extended graphme cluster
  boundary.
  (grapheme_cmp): defined.
  (get_grapheme_properties): defined.
  (grapheme_boundary_p): defined.
  (MAX_BYTES_LENGTH): defined.
  (comb_char_enc_len): defined.
  (mbc_to_code0): extracted from mbc_to_code.
  (mbc_to_code): use mbc_to_code0.
  (left_adjust_combchar_head): defined.
  (utf_8): use a extended graphme cluster as a unit.

* enc/unicode.c (onigenc_unicode_mbc_case_fold): use
  ONIGENC_MBC_PRECISE_CODEPOINT to extract codepoints.
  (onigenc_unicode_get_case_fold_codes_by_str): ditto.

* enc/euc_jp.c (mbc_to_code): follow mbc_to_code field change.
  use onigenc_mbn_mbc_to_code.

* enc/shift_jis.c (mbc_to_code): ditto.

* enc/emacs_mule.c (mbc_to_code): ditto.

* enc/gbk.c (gbk_mbc_to_code): follow mbc_to_code field and
  onigenc_mbn_mbc_to_code change.

* enc/cp949.c (cp949_mbc_to_code): ditto.

* enc/big5.c (big5_mbc_to_code): ditto.

* enc/euc_tw.c (euctw_mbc_to_code): ditto.

* enc/euc_kr.c (euckr_mbc_to_code): ditto.

* enc/gb18030.c (gb18030_mbc_to_code): ditto.

* enc/utf_32be.c (utf32be_mbc_to_code): follow mbc_to_code field
  change.

* enc/utf_16be.c (utf16be_mbc_to_code): ditto.

* enc/utf_32le.c (utf32le_mbc_to_code): ditto.

* enc/utf_16le.c (utf16le_mbc_to_code): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-16 16:48:05 +00:00
akr 19416601a0 * include/ruby/oniguruma.h (OnigEncodingTypeST): add end argument for 
left_adjust_char_head.
  (ONIGENC_LEFT_ADJUST_CHAR_HEAD): add end argument.
  (onigenc_get_left_adjust_char_head): ditto.

* include/ruby/encoding.h (rb_enc_left_char_head): add end argument.

* regenc.h (onigenc_single_byte_left_adjust_char_head): ditto.

* regenc.c (onigenc_get_right_adjust_char_head): follow the interface
  change.
  (onigenc_get_right_adjust_char_head_with_prev): ditto.
  (onigenc_get_prev_char_head): ditto.
  (onigenc_step_back): ditto.
  (onigenc_get_left_adjust_char_head): ditto.
  (onigenc_single_byte_code_to_mbc): ditto.

* re.c: ditto.

* string.c: ditto.

* io.c: ditto.

* regexec.c: ditto.

* enc/euc_jp.c: ditto.

* enc/cp949.c: ditto.

* enc/shift_jis.c: ditto.

* enc/gbk.c: ditto.

* enc/big5.c: ditto.

* enc/euc_tw.c: ditto.

* enc/euc_kr.c: ditto.

* enc/emacs_mule.c: ditto.

* enc/gb18030.c: ditto.

* enc/utf_8.c: ditto.

* enc/utf_16le.c: ditto.

* enc/utf_16be.c: ditto.

* enc/utf_32le.c: ditto.

* enc/utf_32be.c: ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19334 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-13 19:23:52 +00:00
shyouhei 53cffba8cb * enc/ascii.c: ISO C does not allow extra ';' outside of a 
function

  	* enc/us_ascii.c: ditto.

	* enc/utf_8.c: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17768 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-07-01 08:39:10 +00:00
naruse d6025a3be4 * enc/utf_8.c: add UTF8-MAC (UTF-8-MAC). 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16697 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-05-29 22:12:57 +00:00
usa fee57bb8c8 * enc/utf_8.c: add alias CP65001. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-28 08:41:49 +00:00
naruse 6e1c3a0f54 * enc/koi8_u.c: added. 
* regenc.c, enc/utf_8.c, enc/unicode.c, enc/gb18030.c: add ARG_UNUSED.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15130 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-19 15:37:06 +00:00
naruse edc61cf4c1 * encoding.c (ENC_REGISTER): use &OnigEncoding*. 
(ENCINDEX_UTF_8): renamed from ENCINDEX_UTF8.
  (rb_enc_init): use ENC_REGISTER.

* include/ruby/oniguruma.h (OnigEncodingUTF8, ONIG_ENCODING_UTF8):
  removed.

* enc/*.c: remove use of &encoding_*; use enc argument instead.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15067 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-15 13:36:18 +00:00
matz d2a377d747 * enc/utf_8.c: remove use of ONIG_ENCODING_UTF8 altogether; use 
enc argument instead.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-15 09:47:58 +00:00
usa 648c0f7c80 * enc/utf_8.c (ONIG_ENCODING_UTF8): reverted. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-15 09:06:03 +00:00
matz a34288d947 * enc/utf_8.c (OnigEncodingDefine): encoding name should be kept 
unchanged.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15063 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-15 09:00:48 +00:00
nobu ad73c8b348 * enc/utf_8.c: renamed as IANA name. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15061 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-01-15 08:26:54 +00:00