Граф коммитов

669 Коммитов

Автор SHA1 Сообщение Дата
Nobuyoshi Nakada 612aa5c24a
Use class methods of `File` over `Kernel.open` and `IO.read` 2022-11-21 22:16:49 +09:00
卜部昌平 45741918e1 reserved_word: just use gperf 3.1 declaration
The reason why this was commented out was because of gperf 3.0 vs 3.1
differences (see [Feature #13883]).  Five years passed, I am pretty
confident that we can drop support of old versions here.

Ditto for uniname2ctype_p(), onig_jis_property(), and zonetab().
2022-09-21 11:44:09 +09:00
Nobuyoshi Nakada ca4cbe59ed
Move case-folding.rb to tooldir with enc-prefix 2022-09-17 12:37:48 +09:00
Benoit Daloze 6525b6f760 Remove get_actual_encoding() and the dynamic endian detection for dummy UTF-16/UTF-32
* And simplify callers of get_actual_encoding().
* See [Feature #18949].
* See https://github.com/ruby/ruby/pull/6322#issuecomment-1242758474
2022-09-12 14:02:34 +02:00
Jeremy Evans 6f3857f6a7 Support Encoding::Converter newline: :lf and :lf_newline options
Previously, newline: :lf was accepted but ignored.  Where it
should have been used was commented out code that didn't work,
but unlike all other invalid values, using newline: :lf did
not raise an error.

This adds support for newline: :lf and :lf_newline, for consistency
with newline: :cr and :cr_newline.  This is basically the same as
universal_newline, except that it only affects writing and not
reading due to RUBY_ECONV_NEWLINE_DECORATOR_WRITE_MASK.

Add tests for the File.open :newline option while here.

Fixes [Bug #12436]
2022-08-19 20:23:36 -07:00
Yuta Saito 6d8b9a9d61 Resolve abi symbol references from miniruby to avoid circular deps
Adding `ruby` to `PREP` causes the following circular dependencies
because `PREP` is used as a prerequisite by some targets required to
build `ruby` target itself.
```
make: Circular .rbconfig.time <- ruby dependency dropped.
make: Circular builtin_binary.inc <- ruby dependency dropped.
make: Circular ext/extinit.c <- ruby dependency dropped.
make: Circular ruby <- ruby dependency dropped.
```

Adding a new Make variable like `EXTPREP` only for exts may be also
reasonable, but it would introduce another complexity into our build
system. `-bundle_loader` doesn't care that link-time and run-time
loader executables are different as long as bound symbols are provided,
so it's ok to resolve from miniruby to simplify our build.
2022-08-04 16:29:22 +09:00
Yuta Saito 50d81bfbc1 Link ext bundles with bundle loader option for newer ld64
ld64 shipped with Xcode 14 emits a warning when using `-undefined
dynamic_lookup`.

```
ld: warning: -undefined dynamic_lookup may not work with chained fixups
```

Actually, `-undefined dynamic_lookup` doesn't work when:

1. Link a *shared library* with the option
2. Link it with a program that uses the chained-fixup introduced from
   macOS 12 and iOS 15
because `-undefined dynamic_lookup` uses lazy-bindings and they won't be
bound while dyld fixes-up by traversing chained-fixup info.

However, we build exts as *bundles* and they are loaded only through
`dlopen`, so it's safe to use `-undefined dynamic_lookup` in theory.
So the warning produced by ld64 is false-positive, and it results
failure of option checking in configuration. Therefore, it would be an
option to ignore the warning during our configuration.

On the other hand, `-undefined dynamic_lookup` is already deprecated on
all darwin platforms except for macOS, so it's good time to get rid of
the option. ld64 also provides `-bundle_loader <executable>` option,
which allows to resolve symbols defined in the executable symtab while
linking. It behaves almost the same with `-undefined dynamic_lookup`,
but it makes the following changes:

1. Require that unresolved symbols among input objects must be defined
   in the executable.
2. Lazy symbol binding will lookup only the symtab of the bundle loader
   executable. (`-undefined dynamic_lookup` lookups all symtab as flat
   namespace)

This patch adds `-bundle_loader $(RUBY)` when non-EXTSTATIC
configuration by assuming ruby executable can be linked before building
exts.

See "New Features" subsection under "Linking" section for chained fixup
https://developer.apple.com/documentation/xcode-release-notes/xcode-13-release-notes
2022-08-04 16:29:22 +09:00
Jean Boussier d084585f01 Rename ENCINDEX_ASCII to ENCINDEX_ASCII_8BIT
Otherwise it's way too easy to confuse it with US_ASCII.
2022-07-19 08:48:56 +02:00
Martin Dürst 56d9d78f14 Remove Unicode 13.0.0 related files 2022-03-16 08:30:04 +09:00
Martin Dürst 45e0711f29 update Unicode Version to 14.0.0 and Emoji version to 14.0 2022-03-13 09:19:52 +09:00
Nobuyoshi Nakada db740b7e5c
Revert "enc/depend: fix out-of-src build with --with-static-linked-ext" (#5616)
This reverts commit 32ad8df9d1,
which broke out-of-src build with the pre-generated transcoder sources.
2022-03-02 18:19:01 +09:00
Yuta Saito 32ad8df9d1 enc/depend: fix out-of-src build with --with-static-linked-ext
When out-of-src build, at the beginning of a build, `make -f enc.mk
srcs` generates trans C sources under build dir.

On the other hand, enc/trans/*.o were built from trans C sources
generated under srcdir due to the following auto-generated rules from
enc/depend.

```
encsrcdir = ../src/enc
...
enc/trans/big5.$(OBJEXT): $(encsrcdir)/trans/big5.c
```

Therefore, trans C sources are generated twice under srcdir and build
dir during a build.

Ideally, trans C sources have always been built before compilation of
enc/trans/*.o because the source generation is prereq, so making
enc/trans/*.o doesn't trigger trans C source generation and shouldn't
require MINIRUBY as a make arg for enc.mk. However, the second trans C
source gen is unintentionally triggered by enc/trans/*.o, so `make -f
enc.mk libencs` requires MINIRUBY for now.

When no `--with-static-linked-ext`, `make -f enc.mk libencs` is
triggered from common.mk with MINIRUBY, so there is no problem.
But when `--with-static-linked-ext`, libencs should be statically-linked
to ruby, so `make -f enc.mk libencs` is triggered from exts.mk, and
exts.mk invokes it without MINIRUBY.

Therefore, when out-of-src build and with `--with-static-linked-ext`,
the second trans C source gen fails due to missing MINIRUBY.
This issue is deterministically reproducible without -j because
common.mk's `main` rule also has libencs prerequisite.

This patch supresses the second trans C source gen.
2022-03-02 09:40:58 +09:00
Peter Zhu 2d5ecd60a5 [Feature #18249] Update dependencies 2022-02-22 09:55:21 -05:00
Peter Zhu 638fd8774b [Feature #18249] Include ruby.h in extensions to have ABI version
All shared libraries must have `include/ruby/internal/abi.h` to include
the ABI version. Including `ruby.h` will guarantee that.
2022-02-22 09:55:21 -05:00
Nobuyoshi Nakada ac152b3cac
Update dependencies 2021-11-21 16:21:18 +09:00
卜部昌平 5c167a9778 ruby tool/update-deps --fix 2021-10-05 14:18:23 +09:00
Martin Dürst 6072239121 Remove no longer needed include files (Unicode Version 12.1.0) 2021-07-09 16:22:38 +09:00
Martin Dürst 323ff38c04 Add directory and include files for Unicode version 13.0.0
- Add directory enc/unicode/13.0.0
- Add include files casefold.h and name2ctype.h for Unicode
  version 13.0.0
2021-07-08 14:45:03 +09:00
Nobuyoshi Nakada 1ac228378c
Use $ignore_error defined in mkmf.rb 2021-07-03 12:52:46 +09:00
卜部昌平 6413dc27dc dependency updates 2021-04-13 14:30:21 +09:00
Nobuyoshi Nakada d57c5a7b61 transcode-tblgen.rb: make silent a little when just -v 2020-12-29 17:45:19 +09:00
Lars Kanis d403591b34
Add string encoding IBM720 alias CP720 (#3803)
The mapping table is generated from the ICU project:
  https://github.com/unicode-org/icu/blob/master/icu4c/source/data/mappings/ibm-720_P100-1997.ucm

Fixes bug 16233 : https://bugs.ruby-lang.org/issues/16233
2020-11-22 22:23:40 +09:00
卜部昌平 490010084e sed -i '/rmodule.h/d' 2020-08-27 16:42:06 +09:00
卜部昌平 756403d775 sed -i '/r_cast.h/d' 2020-08-27 15:03:36 +09:00
卜部昌平 0da2a3f1fc sed -i '\,2/extern.h,d' 2020-08-27 14:07:49 +09:00
Kazuhiro NISHIYAMA 946cd6c534
Use https instead of http 2020-07-28 19:51:54 +09:00
Jeremy Evans ddd9704ae9 Encode ' as &apos; when using encode(xml: :attr)
Fixes [Bug #16922]
2020-07-10 09:34:08 -07:00
卜部昌平 9e41a75255 sed -i 's|ruby/impl|ruby/internal|'
To fix build failures.
2020-05-11 09:24:08 +09:00
卜部昌平 d7f4d732c1 sed -i s|ruby/3|ruby/impl|g
This shall fix compile errors.
2020-05-11 09:24:08 +09:00
Nobuyoshi Nakada b7e1eda932
Suppress warnings by gcc 10.1.0-RC-20200430
* Folding results should not be empty.

  If `OnigCodePointCount(to->n)` were 0, `for` loop using `fn`
  wouldn't execute and `ncs` elements are not initialized.

  ```
  enc/unicode.c:557:21: warning: 'ncs[0]' may be used uninitialized in this function [-Wmaybe-uninitialized]
    557 |  for (i = 0; i < ncs[0]; i++) {
        |                  ~~~^~~
  ```

* Cast to `enum yytokentype`

  Additional enums for scanner events by ripper are not included
  in `yytokentype`.

  ```
  ripper.y:7274:28: warning: implicit conversion from 'enum <anonymous>' to 'enum yytokentype' [-Wenum-conversion]
  ```
2020-05-04 12:28:24 +09:00
卜部昌平 9e6e39c351
Merge pull request #2991 from shyouhei/ruby.h
Split ruby.h
2020-04-08 13:28:13 +09:00
Kazuki Tsujimoto b25ef4bf70
Suppress warnings: reserved for numbered parameter 2020-04-05 18:24:59 +09:00
Nobuyoshi Nakada 21d0b40de2 Added tooldir variable 2020-04-05 09:26:57 +09:00
卜部昌平 115fec062c more on NULL versus functions.
Function pointers are not void*.  See also
ce4ea956d2
8427fca49b
2020-02-07 14:24:19 +09:00
卜部昌平 0c2d731ef2 update dependencies 2019-12-26 20:45:12 +09:00
卜部昌平 5e22f873ed decouple internal.h headers
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead.  This would significantly
speed up incremental builds.

We take the following inclusion order in this changeset:

1.  "ruby/config.h", where _GNU_SOURCE is defined (must be the very
    first thing among everything).
2.  RUBY_EXTCONF_H if any.
3.  Standard C headers, sorted alphabetically.
4.  Other system headers, maybe guarded by #ifdef
5.  Everything else, sorted alphabetically.

Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
2019-12-26 20:45:12 +09:00
Nobuyoshi Nakada 992aa2cda5
enc/x_emoji.h: fixed dead-links [ci skip]
English version pages seem no longer provided.
2019-12-24 10:33:32 +09:00
Nobuyoshi Nakada cc87037f1c
Fixed misspellings
Fixed misspellings reported at [Bug #16437], missed and a new
typo.
2019-12-22 22:49:17 +09:00
Nobuyoshi Nakada e1b2341488
Update dependencies 2019-11-18 23:16:22 +09:00
Nobuyoshi Nakada 162cf2879a
Init function is need to link statically 2019-08-10 01:41:50 +09:00
Nobuyoshi Nakada cecae8593a
Removed unnecessary headers 2019-08-10 01:05:09 +09:00
Nobuyoshi Nakada 88db6fa479
Use ENC_REPLICATE to copy an encoding 2019-08-10 01:04:39 +09:00
Yusuke Endoh a8ba22cd32 Revert "Removed unused includes"
This reverts commit c9eb8f82e9.

The change caused "implicit declaration" warning and actual segfault.

```
/tmp/ruby/v2/src/trunk-gc-asserts/enc/gb2312.c: In function ‘Init_gb2312’:
/tmp/ruby/v2/src/trunk-gc-asserts/enc/gb2312.c:6:31: warning: implicit declaration of function ‘rb_enc_find’ [-Wimplicit-function-declaration]
     rb_enc_register("GB2312", rb_enc_find("EUC-KR"));
                               ^~~~~~~~~~~
/tmp/ruby/v2/src/trunk-gc-asserts/enc/gb2312.c:6:31: warning: passing argument 2 of ‘rb_enc_register’ makes pointer from integer without a cast [-Wint-conversion]
<command-line>:0:19: note: expected ‘OnigEncoding {aka const struct OnigEncodingTypeST *}’ but argument is of type ‘int’
/tmp/ruby/v2/src/trunk-gc-asserts/regenc.h:231:12: note: in expansion of macro ‘ONIG_ENC_REGISTER’
 extern int ONIG_ENC_REGISTER(const char *, OnigEncoding);
            ^~~~~~~~~~~~~~~~~
```
2019-08-10 00:01:36 +09:00
Nobuyoshi Nakada c9eb8f82e9
Removed unused includes 2019-08-09 23:08:30 +09:00
Nobuyoshi Nakada 715955ff27
Include ruby/assert.h in ruby/ruby.h so that assertions can be there 2019-07-14 17:58:03 +09:00
Takashi Kokubun 18603e9046
Update dependencies for 369ff79394
Just copy-pasting diff from
https://travis-ci.org/ruby/ruby/jobs/558407687
2019-07-14 12:55:58 +09:00
Martin Dürst 369ff79394 add encoding conversion from/to CESU-8
Add encoding conversion (transcoding) from UTF-8 to CESU-8
and back. CESU-8 is an encoding similar to UTF-8, but encodes
codepoints above U+FFFF as two surrogates, these surrogates
again being encoded as if they were UTF-8 codepoints. This
preserves the same binary sorting order as in UTF-16. It is
also somewhat similar (although not exactly identical) to an
encoding used internally by Java.

This completes issue #15995.

enc/trans/cesu_8.trans: Add encoding conversion from/to CESU-8
test/ruby/test_transcode.rb: Add tests for above
2019-07-14 10:58:50 +09:00
Nobuyoshi Nakada d905ff61e6
Update dependencies 2019-07-09 13:47:07 +09:00
NARUSE, Yui 4275f09015 remove UNREACHABLE 2019-06-24 16:01:46 +09:00
NARUSE, Yui 7f64a0b4db Add new encoding CESU-8 [Feature #15931] 2019-06-24 12:58:33 +09:00