Граф коммитов

692 Коммитов

Автор SHA1 Сообщение Дата
Jeremy Daer e1a034cb96
IBM864 single-byte transcoding (#10518)
Include € euro sign from CCSID 864
2024-07-11 17:03:14 +09:00
Nobuyoshi Nakada 1213623e5c
Use gperf 3.1 to generate ANSI-C code 2024-06-24 23:43:45 +09:00
Nobuyoshi Nakada 5fa6ba9568 [Bug #20500] Search non-default directories for jemalloc
Co-Authored-by: lish82 (Hiroki Katagiri)
2024-05-23 13:16:48 +09:00
卜部昌平 c844968b72 ruby tool/update-deps --fix 2024-04-27 21:55:28 +09:00
Nobuyoshi Nakada 1984db2db8
Preserve old encinit.c if unchanged 2024-04-19 14:12:42 +09:00
Yusuke Endoh 295f2a3305 Prevent a warning: literal string will be frozen in the future 2024-04-04 13:30:27 +09:00
Hiroshi SHIBATA 7630a89a4b Use www.rfc-editor.org for RFC text.
We use the following site for that now:

* https://tools.ietf.org/ or http
* https://datatracker.ietf.org or http

Today, IETF said the official site of RFC is www.rfc-editor.org.

FYI: https://authors.ietf.org/en/references-in-rfcxml

I replaced them to www.rfc-editor.org.
2024-03-28 11:44:45 +09:00
Nobuyoshi Nakada 0fb39ab1b9
Clean intermediate files and debug info for each target
By replacing `ALLOBJS` suffix with intermediate file suffixes instead
of roughly removing by wildcards.  Made `cleanlibs` append `.dSYM`
suffix for each word in `TARGET_SO`, not the end of the entire list.
2024-03-10 22:12:00 +09:00
Nobuyoshi Nakada fbec42fb39
Clean up the incidentals to encoding/transcoding objects [ci skip] 2024-03-07 14:16:22 +09:00
Yuta Saito 0d9a681eff enc: Expand substitution variables in Makefile.in by default
This change makes `make_encmake.rb` expand all substitution variables
when replacing them in `enc/Makefile.in` to avoid propagating all
possibly referenced variables to the generated Makefile. The old
behavior, which don't expand make variables when generating Makefile,
was useful to temporarily override inherited variables like `cflags` in
`CFLAGS` at make-time. However, it's not a common use case and it
requires to propagate all possibly referenced variables properly
considering key name duplication between `enc/Makefile.in` and
`RbConfig::CONFIG`.
2024-03-04 00:57:57 +09:00
Yuta Saito 072761bb3f enc: Define missing `POSTLINK` variable in enc.mk
This is a follow up change to 71d511615b
2024-03-04 00:57:57 +09:00
Nobuyoshi Nakada e9a7801a93
Drop support for old ERB 2024-03-03 00:55:45 +09:00
Yuta Saito 57f014b2fa Use configured `LD` for linking enc and ext libraries
"AR" was well propagated to the enc.mk and mkmf, but "LD" was not. This
caused the dynamic libraries to be linked with a linker found in the PATH,
which could be different from the one used in the Ruby build process.
This is especially important for cross-compilation, where the host
linker may not be compatible with the target system. (e.g. WebAssembly)
2024-03-01 03:16:23 +09:00
Nobuyoshi Nakada 05046b8955
Add `modencs` target to build encodings/transcoders as modules 2023-09-11 11:06:49 +09:00
Janosch Müller 08b3fb1152
[Bug #19728] Auto-generate unicode property docs
https://bugs.ruby-lang.org/issues/19728
2023-07-01 23:22:17 +09:00
Josef Haider 2c8f2871a8
Fix handling of 6-byte codepoints in left_adjust_char_head in CESU-8 encoding 2023-03-18 15:43:54 +09:00
Matt Valentine-House 5e4b80177e Update the depend files 2023-02-28 09:09:00 -08:00
Matt Valentine-House f38c6552f9 Remove intern/gc.h from Make deps 2023-02-27 10:11:56 -08:00
Nobuyoshi Nakada 899ea35035
Extract include/ruby/internal/attr/packed_struct.h
Split `PACKED_STRUCT` and `PACKED_STRUCT_UNALIGNED` macros into the
macros bellow:
* `RBIMPL_ATTR_PACKED_STRUCT_BEGIN`
* `RBIMPL_ATTR_PACKED_STRUCT_END`
* `RBIMPL_ATTR_PACKED_STRUCT_UNALIGNED_BEGIN`
* `RBIMPL_ATTR_PACKED_STRUCT_UNALIGNED_END`
2023-02-08 12:34:13 +09:00
Nobuyoshi Nakada dd0b401716
Ensure newline at EOF [ci skip]
Cf. https://github.com/ruby/ruby/pull/7069
2023-01-06 12:13:15 +09:00
Martin Dürst 368b844874 update Unicode include files to version 15.0.0 2022-12-06 10:11:33 +09:00
Martin Dürst 66bf15636f move Unicode include files to version 15.0.0 directory 2022-12-06 10:11:33 +09:00
Martin Dürst e31d645da5 fix wrong property information in name2ctype.h
Co-authored-by: Nobuyoshi Nakada [nobu@ruby-lang.org](mailto:nobu@ruby-lang.org)

See https://github.com/ruby/ruby/pull/6451 and
https://bugs.ruby-lang.org/issues/19007.

This keeps the Unicode version at 14.0.0, so this commit
is suited for backporting where applicable.

At the time of this commit, the reason for the wrong properties
which we fix here is still not completely known, so issue 19007
should be kept open.
2022-12-06 08:58:30 +09:00
Nobuyoshi Nakada 612aa5c24a
Use class methods of `File` over `Kernel.open` and `IO.read` 2022-11-21 22:16:49 +09:00
卜部昌平 45741918e1 reserved_word: just use gperf 3.1 declaration
The reason why this was commented out was because of gperf 3.0 vs 3.1
differences (see [Feature #13883]).  Five years passed, I am pretty
confident that we can drop support of old versions here.

Ditto for uniname2ctype_p(), onig_jis_property(), and zonetab().
2022-09-21 11:44:09 +09:00
Nobuyoshi Nakada ca4cbe59ed
Move case-folding.rb to tooldir with enc-prefix 2022-09-17 12:37:48 +09:00
Benoit Daloze 6525b6f760 Remove get_actual_encoding() and the dynamic endian detection for dummy UTF-16/UTF-32
* And simplify callers of get_actual_encoding().
* See [Feature #18949].
* See https://github.com/ruby/ruby/pull/6322#issuecomment-1242758474
2022-09-12 14:02:34 +02:00
Jeremy Evans 6f3857f6a7 Support Encoding::Converter newline: :lf and :lf_newline options
Previously, newline: :lf was accepted but ignored.  Where it
should have been used was commented out code that didn't work,
but unlike all other invalid values, using newline: :lf did
not raise an error.

This adds support for newline: :lf and :lf_newline, for consistency
with newline: :cr and :cr_newline.  This is basically the same as
universal_newline, except that it only affects writing and not
reading due to RUBY_ECONV_NEWLINE_DECORATOR_WRITE_MASK.

Add tests for the File.open :newline option while here.

Fixes [Bug #12436]
2022-08-19 20:23:36 -07:00
Yuta Saito 6d8b9a9d61 Resolve abi symbol references from miniruby to avoid circular deps
Adding `ruby` to `PREP` causes the following circular dependencies
because `PREP` is used as a prerequisite by some targets required to
build `ruby` target itself.
```
make: Circular .rbconfig.time <- ruby dependency dropped.
make: Circular builtin_binary.inc <- ruby dependency dropped.
make: Circular ext/extinit.c <- ruby dependency dropped.
make: Circular ruby <- ruby dependency dropped.
```

Adding a new Make variable like `EXTPREP` only for exts may be also
reasonable, but it would introduce another complexity into our build
system. `-bundle_loader` doesn't care that link-time and run-time
loader executables are different as long as bound symbols are provided,
so it's ok to resolve from miniruby to simplify our build.
2022-08-04 16:29:22 +09:00
Yuta Saito 50d81bfbc1 Link ext bundles with bundle loader option for newer ld64
ld64 shipped with Xcode 14 emits a warning when using `-undefined
dynamic_lookup`.

```
ld: warning: -undefined dynamic_lookup may not work with chained fixups
```

Actually, `-undefined dynamic_lookup` doesn't work when:

1. Link a *shared library* with the option
2. Link it with a program that uses the chained-fixup introduced from
   macOS 12 and iOS 15
because `-undefined dynamic_lookup` uses lazy-bindings and they won't be
bound while dyld fixes-up by traversing chained-fixup info.

However, we build exts as *bundles* and they are loaded only through
`dlopen`, so it's safe to use `-undefined dynamic_lookup` in theory.
So the warning produced by ld64 is false-positive, and it results
failure of option checking in configuration. Therefore, it would be an
option to ignore the warning during our configuration.

On the other hand, `-undefined dynamic_lookup` is already deprecated on
all darwin platforms except for macOS, so it's good time to get rid of
the option. ld64 also provides `-bundle_loader <executable>` option,
which allows to resolve symbols defined in the executable symtab while
linking. It behaves almost the same with `-undefined dynamic_lookup`,
but it makes the following changes:

1. Require that unresolved symbols among input objects must be defined
   in the executable.
2. Lazy symbol binding will lookup only the symtab of the bundle loader
   executable. (`-undefined dynamic_lookup` lookups all symtab as flat
   namespace)

This patch adds `-bundle_loader $(RUBY)` when non-EXTSTATIC
configuration by assuming ruby executable can be linked before building
exts.

See "New Features" subsection under "Linking" section for chained fixup
https://developer.apple.com/documentation/xcode-release-notes/xcode-13-release-notes
2022-08-04 16:29:22 +09:00
Jean Boussier d084585f01 Rename ENCINDEX_ASCII to ENCINDEX_ASCII_8BIT
Otherwise it's way too easy to confuse it with US_ASCII.
2022-07-19 08:48:56 +02:00
Martin Dürst 56d9d78f14 Remove Unicode 13.0.0 related files 2022-03-16 08:30:04 +09:00
Martin Dürst 45e0711f29 update Unicode Version to 14.0.0 and Emoji version to 14.0 2022-03-13 09:19:52 +09:00
Nobuyoshi Nakada db740b7e5c
Revert "enc/depend: fix out-of-src build with --with-static-linked-ext" (#5616)
This reverts commit 32ad8df9d1,
which broke out-of-src build with the pre-generated transcoder sources.
2022-03-02 18:19:01 +09:00
Yuta Saito 32ad8df9d1 enc/depend: fix out-of-src build with --with-static-linked-ext
When out-of-src build, at the beginning of a build, `make -f enc.mk
srcs` generates trans C sources under build dir.

On the other hand, enc/trans/*.o were built from trans C sources
generated under srcdir due to the following auto-generated rules from
enc/depend.

```
encsrcdir = ../src/enc
...
enc/trans/big5.$(OBJEXT): $(encsrcdir)/trans/big5.c
```

Therefore, trans C sources are generated twice under srcdir and build
dir during a build.

Ideally, trans C sources have always been built before compilation of
enc/trans/*.o because the source generation is prereq, so making
enc/trans/*.o doesn't trigger trans C source generation and shouldn't
require MINIRUBY as a make arg for enc.mk. However, the second trans C
source gen is unintentionally triggered by enc/trans/*.o, so `make -f
enc.mk libencs` requires MINIRUBY for now.

When no `--with-static-linked-ext`, `make -f enc.mk libencs` is
triggered from common.mk with MINIRUBY, so there is no problem.
But when `--with-static-linked-ext`, libencs should be statically-linked
to ruby, so `make -f enc.mk libencs` is triggered from exts.mk, and
exts.mk invokes it without MINIRUBY.

Therefore, when out-of-src build and with `--with-static-linked-ext`,
the second trans C source gen fails due to missing MINIRUBY.
This issue is deterministically reproducible without -j because
common.mk's `main` rule also has libencs prerequisite.

This patch supresses the second trans C source gen.
2022-03-02 09:40:58 +09:00
Peter Zhu 2d5ecd60a5 [Feature #18249] Update dependencies 2022-02-22 09:55:21 -05:00
Peter Zhu 638fd8774b [Feature #18249] Include ruby.h in extensions to have ABI version
All shared libraries must have `include/ruby/internal/abi.h` to include
the ABI version. Including `ruby.h` will guarantee that.
2022-02-22 09:55:21 -05:00
Nobuyoshi Nakada ac152b3cac
Update dependencies 2021-11-21 16:21:18 +09:00
卜部昌平 5c167a9778 ruby tool/update-deps --fix 2021-10-05 14:18:23 +09:00
Martin Dürst 6072239121 Remove no longer needed include files (Unicode Version 12.1.0) 2021-07-09 16:22:38 +09:00
Martin Dürst 323ff38c04 Add directory and include files for Unicode version 13.0.0
- Add directory enc/unicode/13.0.0
- Add include files casefold.h and name2ctype.h for Unicode
  version 13.0.0
2021-07-08 14:45:03 +09:00
Nobuyoshi Nakada 1ac228378c
Use $ignore_error defined in mkmf.rb 2021-07-03 12:52:46 +09:00
卜部昌平 6413dc27dc dependency updates 2021-04-13 14:30:21 +09:00
Nobuyoshi Nakada d57c5a7b61 transcode-tblgen.rb: make silent a little when just -v 2020-12-29 17:45:19 +09:00
Lars Kanis d403591b34
Add string encoding IBM720 alias CP720 (#3803)
The mapping table is generated from the ICU project:
  https://github.com/unicode-org/icu/blob/master/icu4c/source/data/mappings/ibm-720_P100-1997.ucm

Fixes bug 16233 : https://bugs.ruby-lang.org/issues/16233
2020-11-22 22:23:40 +09:00
卜部昌平 490010084e sed -i '/rmodule.h/d' 2020-08-27 16:42:06 +09:00
卜部昌平 756403d775 sed -i '/r_cast.h/d' 2020-08-27 15:03:36 +09:00
卜部昌平 0da2a3f1fc sed -i '\,2/extern.h,d' 2020-08-27 14:07:49 +09:00
Kazuhiro NISHIYAMA 946cd6c534
Use https instead of http 2020-07-28 19:51:54 +09:00
Jeremy Evans ddd9704ae9 Encode ' as &apos; when using encode(xml: :attr)
Fixes [Bug #16922]
2020-07-10 09:34:08 -07:00