Граф коммитов

408 Коммитов

Автор SHA1 Сообщение Дата
Nobuyoshi Nakada 62c1583353
[DOC] Place a non-US-ASCII document in a document-specific script 2022-02-26 16:23:04 +09:00
Burdette Lamar 26ffda2fd2
[DOC] Enhanced RDoc for some encoding methods (#5598)
In String, treats:

    #b
    #scrub
    #scrub!
    #unicode_normalize
    #unicode_normalize!
    #encode
    #encode!

Also adds a note to IO.new (suggested by @jeremyevans).
2022-02-25 13:12:59 -06:00
S.H af10771045
Add `econv_get_encoding` function 2022-02-02 22:52:55 +09:00
Peter Zhu a5b6598192 [Feature #18239] Implement VWA for strings
This commit adds support for embedded strings with variable capacity and
uses Variable Width Allocation to allocate strings.
2021-10-25 13:26:23 -04:00
S.H dc9112cf10
Using NIL_P macro instead of `== Qnil` 2021-10-03 22:34:45 +09:00
Jeremy Evans 3f7da458a7 Make encoding loading not issue warning
Instead of relying on setting an unsetting ruby_verbose, which is
not thread-safe, restructure require_internal and load_lock to
accept a warn argument for whether to warn, and add
rb_require_internal_silent to require without warnings.  Use
rb_require_internal_silent when loading encoding.

Note this does not modify ruby_debug and errinfo handling, those
remain thread-unsafe.

Also silent requires when loading transcoders.
2021-10-02 05:51:29 -09:00
S-H-GAMELINKS 56065f0686 Using SYMBOL_P macro 2021-09-11 08:48:56 +09:00
S-H-GAMELINKS bdd6d8746f Replace RBOOL macro 2021-09-05 23:01:27 +09:00
Nobuyoshi Nakada 03e7fc895e Extracted repeatedly defined IDs 2021-07-27 15:40:27 +09:00
Nobuyoshi Nakada 537e824561 Don't redefine #rb_intern over and over again 2021-07-27 15:40:27 +09:00
Jeremy Evans e86c1f6fc5
Work around issue transcoding issue with non-ASCII compatible encodings and xml escaping
When using a non-ASCII compatible source and destination encoding
and xml escaping (the :xml option to String#encode), the resulting
string was broken, as it used the correct non-ASCII compatible
encoding, but contained data that was ASCII-compatible instead of
compatible with the string's encoding.

Work around this issue by detecting the case where both the
source and destination encoding are non-ASCII compatible, and
transcoding the source string from the non-ASCII compatible
encoding to UTF-8. The xml escaping code will correctly handle
the UTF-8 source string and the return the correctly encoded
and escaped value.

Fixes [Bug #12052]

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2021-06-26 12:32:39 -07:00
Ryuta Kamizono 33f2ff3bab Fix some typos by spell checker 2021-04-26 10:07:41 +09:00
Nobuhiro IMAI 939729dd82 fix result of example [ci skip] 2021-01-06 21:58:34 +09:00
卜部昌平 c30f03d328 Data: delete
Has been deprecated since 684bdf6171.

Matz says in [ruby-core:83954] that Data should be an alias of Object.
Because rb_cData has not been deprecated, let us deprecate the constant
to make it a C-level synonym of rb_cObject.
2020-12-22 02:51:49 +09:00
Jeremy Evans 9195310168 Update documentation for String#encode{,!} [ci skip]
These methods take keywords, not a hash.

From tjschuck (T.J. Schuck)

Fixes [Bug #17364]
2020-12-03 12:47:34 -08:00
卜部昌平 ff30358d13 RARRAY_AREF: convert into an inline function
RARRAY_AREF has been a macro for reasons.  We might not be able to
change that for public APIs, but why not relax the situation internally
to make it an inline function.
2020-08-15 12:09:26 +09:00
卜部昌平 3d636eccfc make_econv_exception: do not goto into a branch
I'm not necessarily against every goto in general, but jumping into a
branch is definitely a bad idea.  Better refactor.
2020-06-29 11:05:41 +09:00
卜部昌平 9e41a75255 sed -i 's|ruby/impl|ruby/internal|'
To fix build failures.
2020-05-11 09:24:08 +09:00
卜部昌平 d7f4d732c1 sed -i s|ruby/3|ruby/impl|g
This shall fix compile errors.
2020-05-11 09:24:08 +09:00
卜部昌平 9e6e39c351
Merge pull request #2991 from shyouhei/ruby.h
Split ruby.h
2020-04-08 13:28:13 +09:00
Nobuyoshi Nakada 8b6e2685a4
Fixed symbol misused as ID
`rb_funcallv_public` and `rb_respond_to` require an `ID`, not a
`Symbol`.  [Bug #16649]
2020-02-24 15:37:52 +09:00
卜部昌平 115fec062c more on NULL versus functions.
Function pointers are not void*.  See also
ce4ea956d2
8427fca49b
2020-02-07 14:24:19 +09:00
Nobuyoshi Nakada eb737916b1
Warn when :newline precedes other newline options 2020-01-11 10:42:22 +09:00
卜部昌平 5e22f873ed decouple internal.h headers
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead.  This would significantly
speed up incremental builds.

We take the following inclusion order in this changeset:

1.  "ruby/config.h", where _GNU_SOURCE is defined (must be the very
    first thing among everything).
2.  RUBY_EXTCONF_H if any.
3.  Standard C headers, sorted alphabetically.
4.  Other system headers, maybe guarded by #ifdef
5.  Everything else, sorted alphabetically.

Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
2019-12-26 20:45:12 +09:00
Jeremy Evans ffd0820ab3 Deprecate taint/trust and related methods, and make the methods no-ops
This removes the related tests, and puts the related specs behind
version guards.  This affects all code in lib, including some
libraries that may want to support older versions of Ruby.
2019-11-18 01:00:25 +02:00
Jeremy Evans c5c05460ac Warn on access/modify of $SAFE, and remove effects of modifying $SAFE
This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.

This modifies some internal functions that took a safe level argument
to no longer take the argument.

rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.

One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd.  We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.
2019-11-18 01:00:25 +02:00
Yusuke Endoh a4e5690760 transcode.c (rb_econv_open0): remove unused code
Coverity Scan found this.
2019-07-24 04:49:51 +09:00
Yusuke Endoh 0bf829c17b transcode.c (rb_trans_conv): remove unnecessary assignments
This change will suppress Coverity Scan warnings
2019-07-14 23:02:23 +09:00
nobu 98e65d9d92 Prefer rb_check_arity when 0 or 1 arguments
Especially over checking argc then calling rb_scan_args just to
raise an ArgumentError.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66238 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-06 07:49:24 +00:00
kazu 504fc28fc7 Fix call-seq of Encoding::Converter#putback [ci skip]
[Fix GH-1999]

From: Takayuki Matsubara <takayuki.1229@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65487 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-01 06:36:04 +00:00
svn b8c23ce39e * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64881 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-29 06:47:34 +00:00
shirosaki b965781e8f transcode.c: add GC guard on raise
* transcode.c (econv_s_search_convpath): add GC guard to fix SEGV
  on raise.
  [Bug #15162] [ruby-core:89172]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64879 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-29 06:47:30 +00:00
nobu 96db72ce38 [DOC] missing docs at toplevel
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62544 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-23 02:18:52 +00:00
normal 7e669f40be ruby/ruby.h: remove unnecessary exports from C-API
Needlessly exporting can reduce performance locally and increase
binary size.

Increasing the footprint of our C-API larger is also detrimental
to our development as it encourages tighter coupling with our
internals; making it harder for us to preserve compatibility.

If some parts of the core codebase needs access to globals,
internal.h should be used instead of anything in include/ruby/*.

"Urabe, Shyouhei" <shyouhei@ruby-lang.org> wrote:
> On Thu, Jan 18, 2018 at 7:33 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > shyouhei@ruby-lang.org wrote:
> >>   https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=61908
> >>
> >>     export rb_mFConst
> >
> > Why are we exporting all these and making the public C-API bigger?
> > If anything, we should make these static.  Thanks.
>
> No concrete reason, except they have already been externed in 2.5.
> These variables had lacked declarations so far, which resulted in their
> visibility to be that of extern. The commit is just confirming the status quo.
>
> I'm not against to turn them into static.

This reverts changes from r61910, r61909, r61908, r61907, and r61906.

* transcode.c (rb_eUndefinedConversionError): make static
  (rb_eInvalidByteSequenceError): ditto
  (rb_eConverterNotFoundError): ditto
* process.c (rb_mProcGID, rb_mProcUid, rb_mProcID_Syscall): ditto
* file.c (rb_mFConst): ditto
* error.c (rb_mWarning, rb_cWarningBuffer): ditto
* enumerator.c (rb_cLazy): ditto
  [Misc #14381]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62029 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-24 21:07:14 +00:00
nobu e9cb552ec9 internal.h: remove dependecy on ruby/encoding.h
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61713 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 06:24:11 +00:00
ko1 c39bdb798d `$SAFE` as a process global state. [Feature #14250]
* vm_core.h (rb_vm_t): move `rb_execution_context_t::safe_level` to
  `rb_vm_t::safe_level_` because `$SAFE` is a process (VM) global state.

* vm_core.h (rb_proc_t): remove `rb_proc_t::safe_level` because `Proc`
  objects don't need to keep `$SAFE` at the creation.
  Also make `is_from_method` and `is_lambda` as 1 bit fields.

* cont.c (cont_restore_thread): no need to keep `$SAFE` for Continuation.

* eval.c (ruby_cleanup): use `rb_set_safe_level_force()` instead of access
  `vm->safe_level_` directly.

* eval_jump.c: End procs `END{}` doesn't keep `$SAFE`.

* proc.c (proc_dup): removed and introduce `rb_proc_dup` in vm.c.

* safe.c (rb_set_safe_level): don't check `$SAFE` 1 -> 0 changes.

* safe.c (safe_setter): use `rb_set_safe_level()`.

* thread.c (rb_thread_safe_level): `Thread#safe_level` returns `$SAFE`.
  It should be obsolete.

* transcode.c (load_transcoder_entry): `rb_safe_level()` only returns
  0 or 1 so that this check is not needed.

* vm.c (vm_proc_create_from_captured): don't need to keep `$SAFE` for Proc.

* vm.c (rb_proc_create): renamed to `proc_create`.

* vm.c (rb_proc_dup): moved from proc.c.

* vm.c (vm_invoke_proc): do not need to set and restore `$SAFE`
  for `Proc#call`.

* vm_eval.c (rb_eval_cmd): rename a local variable to represent clearer
  meaning.

* lib/drb/drb.rb: restore `$SAFE`.

* lib/erb.rb: restore `$SAFE`, too.

* test/lib/leakchecker.rb: check `$SAFE == 0` at the end of tests.

* test/rubygems/test_gem.rb: do not set `$SAFE = 1`.

* bootstraptest/test_proc.rb: catch up this change.

* spec/ruby/optional/capi/string_spec.rb: ditto.

* test/bigdecimal/test_bigdecimal.rb: ditto.

* test/fiddle/test_func.rb: ditto.

* test/fiddle/test_handle.rb: ditto.

* test/net/imap/test_imap_response_parser.rb: ditto.

* test/pathname/test_pathname.rb: ditto.

* test/readline/test_readline.rb: ditto.

* test/ruby/test_file.rb: ditto.

* test/ruby/test_optimization.rb: ditto.

* test/ruby/test_proc.rb: ditto.

* test/ruby/test_require.rb: ditto.

* test/ruby/test_thread.rb: ditto.

* test/rubygems/test_gem_specification.rb: ditto.

* test/test_tempfile.rb: ditto.

* test/test_tmpdir.rb: ditto.

* test/win32ole/test_win32ole.rb: ditto.

* test/win32ole/test_win32ole_event.rb: ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61510 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-28 20:09:24 +00:00
hsbt 6f6ed7fbba * lib/net/http/header.rb: Fix typo. [ci skip][fix GH-1407]
* transcode.c: ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-08-16 02:18:51 +00:00
nobu 4fad63da01 transcode.c: scrub in the given encoding
* transcode.c (str_transcode0): scrub in the given encoding when
  the source encoding is given, not in the encoding of the
  receiver.  [ruby-core:75732] [Bug #12431]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55181 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-27 08:09:46 +00:00
nobu 6ce18e3a8b transcode.c: GC guards
* transcode.c (enc_arg, str_transcode_enc_args, econv_args):
  remove volatile, and add GC guards in callers.
  [ruby-core:75664] [Bug #12411]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55115 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-22 08:06:57 +00:00
nobu eec8adb168 transcode.c: infection
* transcode.c (rb_econv_substr_append, econv_primitive_convert):
  the result should be infected by the original string.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53168 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-17 04:35:26 +00:00
ko1 2f5b8f0529 * *.c (*_memsize): do not check ptr.
NULL checking is finished Before call of memsize functions.
  See r52979.




git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-09 00:38:32 +00:00
hsbt ccf647f71c * transcode.c: fix a typo
[ci skip][fix GH-1091] Patch by @jwworth

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52534 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-11-11 05:14:36 +00:00
nobu b55217c691 transcode.c: GC guards
* transcode.c (econv_init): add guards to prevent source encoding
  name and destination encoding name from GC.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51489 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-08-05 05:40:17 +00:00
nobu 3969b1861c transcode.c: fix dangling pointers
* transcode.c (rb_econv_open0): rb_econv_t::source_encoding_name
  and rb_econv_t::destination_encoding_name should refer static
  strings always or NULL.  [ruby-core:70247] [Bug #11416]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-08-05 05:37:58 +00:00
nobu e0ec0c7dc4 transcode.c: empty encoding name
* transcode.c (rb_econv_set_replacement): target encoding name can
  be empty now.  [ruby-core:69841] [Bug #11324]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51116 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-07-03 03:25:21 +00:00
nobu 9a7e4bf9ea transcode.c: fix race condition
* transcode.c (load_transcoder_entry): fix transcoder loading race
  condition, by waiting in require.  [ruby-dev:49106] [Bug #11277]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-26 03:48:41 +00:00
nobu e580a631be use 0 for reserved
use 0 for rb_data_type_t::reserved instead of NULL, since its type
may be changed in the future and possibly not a pointer type.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-01 06:38:04 +00:00
akr 7cd76ab0c5 * internal.h: Include ruby.h and ruby/encoding.h to be
includable without prior inclusion.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-15 11:49:06 +00:00
nobu fdfb939d72 transcode.c: use string
* transcode.c (econv_opts): get rid of inadvertent ID creation,
  and preserve argument encoding in error messages.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47041 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-08-03 01:55:32 +00:00
ktsj 3b895e8355 * process.c (rlimit_resource_type, rlimit_resource_value):
get rid of inadvertent dynamic symbol pin-down.

* re.c (match_backref_number): ditto.

* signal.c (esignal_init, rb_f_kill, trap_signm): ditto.

* transcode.c (econv_opts): ditto.

* vm_trace.c (symbol2event_flag): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47005 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-07-30 13:17:45 +00:00