Граф коммитов

23 Коммитов

Автор SHA1 Сообщение Дата
nobu b65b392e96 tables.rb: add
* lib/unicode_normalize/tables.rb: commit not to download and
  convert Unicode data files every time.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-11 17:41:53 +00:00
nobu 9b559f194c normalize.rb: fix syntax error
* lib/unicode_normalize/normalize.rb (normalized): fix syntax
  error, `when` clase allows `*` but not `**`.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48340 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-09 10:01:37 +00:00
duerst 62b511b6aa lib/unicode_normalize/normalize.rb: Replaced if-else by case in self.normalized? in parallel to r48309.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48338 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-09 09:33:36 +00:00
nobu d436c05163 unicode_norm_gen.tmpl: expand kompatible_table
* template/unicode_norm_gen.tmpl: expand kompatible_table so that
  recursive expansion is not needed at runtime.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48311 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-06 15:00:24 +00:00
nobu b8788417f0 normalize.rb: trivial optimizations
* lib/unicode_normalize/normalize.rb (nfc_one, normalize): trivial
  optimizations.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-06 15:00:17 +00:00
nobu 64034372b7 normalize.rb: explicit separator
* lib/unicode_normalize/normalize.rb (canonical_ordering_one):
  use explicit separator, not to depend on $,.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-06 15:00:14 +00:00
duerst 2b7f0289f8 lib/unicode_normalize/normalize.rb: Comment clarification. [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-05 23:49:55 +00:00
duerst 4fda619836 lib/unicode_normalize/normalize.rb: added US_ASCII
as trivially supported encoding (is always normalized,
and may appear mixed in with UTF-8 or other Unicode
encodings).

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48134 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-25 11:09:08 +00:00
nobu 696141dab4 lib/unicode_normalize/tables.rb: remove auto generated file.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48074 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-21 13:48:05 +00:00
duerst 5c27164d59 lib/unicode_normalize/tables.rb: Committing to make version
update easier and more predictable, and reducing compilation
time.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48072 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-21 08:12:20 +00:00
duerst 7415796ca3 lib/unicode_normalize/normalize.rb: Added comment to point to
relevant portion of Unicode standard for Hangul (de)composition
identifiers and algorithm.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48071 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-21 06:56:58 +00:00
nobu e64a3869bc unicode_normalize/normalize.rb: rename variable
* lib/unicode_normalize/normalize.rb (hangul_decomp_one): use more
  descriptive name.  leave [SLVT]BASE and [LVTNS]COUNT as they are
  vague names.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48055 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 11:50:00 +00:00
duerst acaafe2101 lib/unicode_normalize.rb: revert r48046. The s in sIndex
is not hungarian notation. The variable name sIndex is
directly taken from the relevant part of the Unicode
Standard, where it is written SIndex and stands for
'syllable index'. See pp. 144/145 of
http://www.unicode.org/versions/Unicode7.0.0/ch03.pdf.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48052 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 10:06:11 +00:00
nobu 6948188f38 unicode_normalize/normalize.rb: remove prefix
* lib/unicode_normalize/normalize.rb (hangul_decomp_one): remove
  system hungarian prefix, nonsense in ruby.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48046 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 05:01:02 +00:00
nobu 7f652dc6cf unicode_normalize/normalize.rb: simplify
* lib/unicode_normalize/normalize.rb (NF_HASH_{D,C,K}): remove
  first element by Hash#shift.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48045 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 05:00:58 +00:00
nobu 3a2f81cf9a unicode_normalize/normalize.rb: remove unnecessary module names
* lib/unicode_normalize/normalize.rb (UnicodeNormalize): use self
  instead of module name and remove module name if unnecessary.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48044 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 05:00:46 +00:00
nobu 51af3be356 lib/unicode_normalize.rb: remove BOMs
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48028 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 15:29:58 +00:00
duerst 8c722a9a1e lib/unicode_normalize/normalize.rb: Added a missing file extension in require statement.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48022 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 09:35:45 +00:00
duerst 982f0de141 tool/unicode_norm_gen.rb, lib/unicode_normalize.rb:
File name change from lib/unicode_normalize/normalize_tables.rb
to lib/unicode_normalize/tables.rb.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48015 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 02:09:13 +00:00
svn 4bf30d2944 * remove trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48009 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 00:48:55 +00:00
duerst 4c769ce021 lib/unicode_normalize/normalize.rb: Changed module name, adjusted copyright.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 00:48:52 +00:00
svn d64dc54e0c * properties.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48007 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 00:38:53 +00:00
duerst 6017de0314 lib/unicode_normalize/normalize.rb: Importing from
https://github.com/duerst/eprun/blob/master/lib/normalize.rb.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48005 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 00:38:40 +00:00