Граф коммитов

478 Коммитов

Автор SHA1 Сообщение Дата
Yusuke Endoh a5ecf7e0a1 dir.c (join_path_from_pattern): check NULL from malloc
Coverity Scan points out that all the return values of GLOB_ALLOC_N are
NULL-checked except this call.
2019-10-13 00:51:50 +09:00
卜部昌平 7e0ae1698d avoid overflow in integer multiplication
This changeset basically replaces `ruby_xmalloc(x * y)` into
`ruby_xmalloc2(x, y)`.  Some convenient functions are also
provided for instance `rb_xmalloc_mul_add(x, y, z)` which allocates
x * y + z byes.
2019-10-09 12:12:28 +09:00
Nobuyoshi Nakada a0a2640b39 Fix for wrong fnmatch patttern
* dir.c (file_s_fnmatch): ensure that pattern does not contain a
  NUL character.  https://hackerone.com/reports/449617
2019-10-01 19:19:56 +09:00
Jeremy Evans 80b5a0ff2a
Make rb_scan_args handle keywords more similar to Ruby methods (#2460)
Cfuncs that use rb_scan_args with the : entry suffer similar keyword
argument separation issues that Ruby methods suffer if the cfuncs
accept optional or variable arguments.

This makes the following changes to : handling.

* Treats as **kw, prompting keyword argument separation warnings
  if called with a positional hash.

* Do not look for an option hash if empty keywords are provided.
  For backwards compatibility, treat an empty keyword splat as a empty
  mandatory positional hash argument, but emit a a warning, as this
  behavior will be removed in Ruby 3.  The argument number check
  needs to be moved lower so it can correctly handle an empty
  positional argument being added.

* If the last argument is nil and it is necessary to treat it as an option
  hash in order to make sure all arguments are processed, continue to
  treat the last argument as the option hash. Emit a warning in this case,
  as this behavior will be removed in Ruby 3.

* If splitting the keyword hash into two hashes, issue a warning, as we
  will not be splitting hashes in Ruby 3.

* If the keyword argument is required to fill a mandatory positional
  argument, continue to do so, but emit a warning as this behavior will
  be going away in Ruby 3.

* If keyword arguments are provided and the last argument is not a hash,
  that indicates something wrong. This can happen if a cfunc is calling
  rb_scan_args multiple times, and providing arguments that were not
  passed to it from Ruby.  Callers need to switch to the new
  rb_scan_args_kw function, which allows passing of whether keywords
  were provided.

This commit fixes all warnings caused by the changes above.

It switches some function calls to *_kw versions with appropriate
kw_splat flags. If delegating arguments, RB_PASS_CALLED_KEYWORDS
is used.  If creating new arguments, RB_PASS_KEYWORDS is used if
the last argument is a hash to be treated as keywords.

In open_key_args in io.c, use rb_scan_args_kw.
In this case, the arguments provided come from another C
function, not Ruby.  The last argument may or may not be a hash,
so we can't set keyword argument mode.  However, if it is a
hash, we don't want to warn when treating it as keywords.

In Ruby files, make sure to appropriately use keyword splats
or literal keywords when calling Cfuncs that now issue keyword
argument separation warnings through rb_scan_args.  Also, make
sure not to pass nil in place of an option hash.

Work around Kernel#warn warnings due to problems in the Rubygems
override of the method.  There is an open pull request to fix
these issues in Rubygems, but part of the Rubygems tests for
their override fail on ruby-head due to rb_scan_args not
recognizing empty keyword splats, which this commit fixes.

Implementation wise, adding rb_scan_args_kw is kind of a pain,
because rb_scan_args takes a variable number of arguments.
In order to not duplicate all the code, the function internals need
to be split into two functions taking a va_list, and to avoid passing
in a ton of arguments, a single struct argument is used to handle
the variables previously local to the function.
2019-09-25 11:18:49 -07:00
Nobuyoshi Nakada 6f206b8ec6
Prohibit nul-separated glob pattern [Feature #14643] (#2419) 2019-09-02 15:08:53 +09:00
卜部昌平 703783324c rb_ensure now free from ANYARGS
After 5e86b005c0, I now think ANYARGS is
dangerous and should be extinct.  This commit deletes ANYARGS from
rb_ensure, which also revealed many arity / type mismatches.
2019-08-27 15:52:26 +09:00
Yusuke Endoh 517c8a5324 dir.c (dir_each_entry): remove unnecessary check
I guess that this check was to detect re-entrance (maybe callcc?).
But currently it does not work as intended.
2019-07-15 00:10:11 +09:00
nobu 56557ec28a [DOC] fix markups [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67337 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-22 11:04:59 +00:00
shirosaki 0f63d96116 dir.c: fix Dir.glob starts with brace
* dir.c (ruby_glob0): expand braces if a glob pattern starts with
  brace.
  [ruby-core:91728] [Bug #15649]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67332 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-21 13:32:03 +00:00
shirosaki d1417ff8fa dir.c: fix Dir.glob with braces and matching dir
* dir.c (join_path_from_pattern): add the last slash for directory
  matching.

* test/ruby/test_dir.rb (test_glob_recursive_directory): add a test
  for above.
  [ruby-core:91110] [Bug #15540]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66838 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-16 14:06:37 +00:00
nobu 78d6e33702 Fix styles [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66762 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-01-09 13:58:49 +00:00
nobu 09ef29a78f Prefer rb_check_arity when 0 or 1 arguments
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66205 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-05 01:09:44 +00:00
ko1 fa3d1ca0f4 fix "initialize miss".
* dir.c (glob_helper): initialize `args.pathtype`.
  missed at r64810.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64849 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-26 02:23:21 +00:00
shirosaki f73d504c6d dir.c: fix glob with recursive and brace
Fixed bug that glob with recursive and braces (**/{a,b}) pattern
fails.
[Feature #13167]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64836 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-25 15:31:02 +00:00
shirosaki b14325443a dir.c: fix memory leak of glob with braces
join_path uses malloc. So free is required.
[Feature #13167]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64835 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-25 15:31:01 +00:00
shirosaki 2a11904214 dir.c: performance fix with braces
Braces were expended before ruby_glob0(). This caused to call
replace_real_basename() for same plain patterns repeatedly.
Move blace expansion into glob_helper() in ruby_glob0() to reduce
replace_real_basename() call.
This fix changes the order of glob results.
[Feature #13167] [Fix GH-1864]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-22 01:11:40 +00:00
nobu 0d95805c2e Suppress more -Wparentheses warnings
[Fix GH-1958]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64808 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-21 10:31:11 +00:00
nobu 2bfc133477 dir.c: fix glob with base when no DT_UNKNOWN
* dir.c (do_stat, do_lstat, do_opendir): need the length of the base
  path for fstatat() when fd is valid.

* dir.c (glob_helper): fix for platforms where DT_UNKNOWN is not
  available, e.g. Solaris.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64067 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-26 12:48:02 +00:00
naruse f1186a3d38 Revert "dir.c: fix glob with base when no DT_UNKNOWN"
This reverts commit r63982.
It breaks build on Solaris 11.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-24 20:24:27 +00:00
nobu 71120953d7 dir.c: fix glob with base when no DT_UNKNOWN
* dir.c (do_stat, do_lstat): need the length of the base path for
  fstatat() when fd is valid.

* dir.c (glob_helper): fix for platforms where DT_UNKNOWN is not
  available, e.g. Solaris.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63982 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-17 00:47:19 +00:00
nobu 58f2e6bc94 dir.c: fix directory glob
* dir.c (glob_helper): fix directory glob which resulted in lacking
  the first byte.  adjust the length of basename to be appended as
  well as removing the heading path, not the length of the joined
  path.  [ruby-dev:50588] [Bug #14899]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63909 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-09 01:20:20 +00:00
normal 45765e94d9 dir.c: define O_CLOEXEC for older systems
SuSE 10 has openat(), but not O_CLOEXEC

Reported-by: wangpeiwen

[ruby-core:87591] [Bug #14864]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63720 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-22 08:47:12 +00:00
nobu c635662d7f dir.c: warning for NUL
* dir.c (rb_push_glob): warn NUL-separated glob patterns.
  [Feature #14643]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63190 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-19 07:05:39 +00:00
kazu 6ce7986668 [DOC] `*` in glob matches `\n` too [ci skip]
```
File.fnmatch("a*b", "a\nb") # => true
/\Aa.*b\z/x.match?("a\nb")  # => false
/\Aa.*b\z/mx.match?("a\nb") # => true
```

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63115 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-04-08 07:31:46 +00:00
nobu 5dfd225c64 dir.c: Dir.glob example
* dir.c (dir_s_glob): [DOC] added an example of Dir.glob using
  pattern list.  [ci skip]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63041 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-30 13:48:16 +00:00
nobu 97e6aaca7c dir.c: do not assume NUL terminator
* dir.c (rb_push_glob): do not assume string is NUL terminated
  always, shared substring may not in the future.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63034 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-29 05:56:04 +00:00
nobu bd5661a3cb dir.c: check NUL bytes
* dir.c (GlobPathValue): should be used in rb_push_glob only.
  other methods should use FilePathValue.
  https://hackerone.com/reports/302338

* dir.c (rb_push_glob): expand GlobPathValue

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62989 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-28 09:58:52 +00:00
nobu 96db72ce38 [DOC] missing docs at toplevel
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62544 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-23 02:18:52 +00:00
nobu 6a3a7e9114 dir.c: Dir#each_child
* dir.c (dir_each_child_m): new instance methods Dir#each_child
  and Dir#children.  [Feature #13969]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62022 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-24 07:15:55 +00:00
kazu b8cc476ce9 use predefined IDs
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-22 13:09:37 +00:00
shyouhei ebe0247c03 dir_closed marked as NORETURN
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-18 09:44:38 +00:00
normal 7c7d47d236 dir.c: pass flags to openat(2) correctly
Flags are 3rd argument of openat(2) while the 4th argument
(mode_t) is unnecessary for our uses.  This bug exists since
Ruby 2.5 from r58860 and was discoverd by strace.

* dir.c (nogvl_opendir_at): use openat correctly
  [Feature #13056] [Feature #14346]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61754 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-10 01:07:27 +00:00
nobu e9cb552ec9 internal.h: remove dependecy on ruby/encoding.h
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61713 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 06:24:11 +00:00
usa 91de3d6539 support 128bit ino on Windows (if available)
* win32/win32.c, include/ruby/win32.h (stati128, rb_{,u,l,ul}stati128): rename
  from stati64ns, change the type of st_ino to 64bit and added st_inohigh.

* dir.c, file.c (stat, lstat): follow above changes.

* file.c (rb_stat_ino): support 128bit ino.

* win32/win32.c (rb_{,u,l,ul}stati128): ditto.
  [Feature #13731]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-10 17:26:58 +00:00
normal dceb897e5a Dir#chdir keeps GVL if passed block
On further examination, Dir.chdir with a block from multiple
threads is thread-unsafe given our use of the chdir_blocking and
chdir_thread global variables.

This bug was only introduced in r60583 so not part of any stable
release.

Dir.chdir without a block can still make senses in a MT context
as only one thread could be cwd-sensitive and other threads do
not care which directory they're in.

* dir.c (dir_chdir): keep GVL here
  (dir_s_chdir): release GVL if no block given

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-09 03:46:39 +00:00
usa ce7c1c0577 support nanosec file timestamp on newer Windows
Support nanosec file timestamp on Windows 8 or later.
Original patches are written by kubo (Kubo Takehiro).
Windows 7 and earlier also supports nanosec file timestamp, but it's too
accurate than system time.  so, this feature is disabled on such versions.
[Feature #13726]

this change also includes [Misc #13702]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61013 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-04 10:48:30 +00:00
nobu cea5a91163 util.h: remove my_getcwd
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60908 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-26 01:36:33 +00:00
kazu 246c986eac Use `const void*` instead of `const char*`
Use cast from `char*` to `void*` instead of union in opendir_without_gvl,
because convert from `void*` to `char*` without union in nogvl_opendir.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60875 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-21 12:29:51 +00:00
usa 4f83ca015d Cannot call rb_thread_call_with{out,}_gvl before running VM
* dir.c (opendir_without_gvl, with_gvl_gc_for_fd, opendir_at): check the VM is
  already initialized before calling rb_thread_call_with{out,}_gvl().
  [Bug #14115]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60837 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-18 08:25:29 +00:00
normal 8840b033fb dir.c: openat calls release GVL, too
openat(2) also performs a path lookup, so it is also subject
to pathological slowdowns like opendir(3) and open(2) syscalls.

* dir.c (struct opendir_at_arg): new struct for callback
  (with_gvl_gc_for_fd): new callback for rb_thread_call_with_gvl
  (gc_for_fd_with_gvl): moved up
  (nogvl_opendir_at): extracted from do_opendir
  (opendir_at): new wrapper to release GVL for opendir_at
  (do_opendir): use new wrappers to release GVL
  (nogvl_dir_empty_p): adjust for gc_for_fd_with_gvl

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60831 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-18 02:01:49 +00:00
normal 089eb0c5ce dir: release GVL on opendir
opendir(3) is subject to the same pathological slowdowns on
slow or unreliable filesystems as open(2), so release the GVL
to avoid stalling the entire VM like we do with IO#open

* dir.c (nogvl_opendir): new function
  (opendir_without_gvl): new function
  (dir_initialize): s/opendir/&_without_gvl/
  (do_opendir): ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60830 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-18 02:01:44 +00:00
normal 0cc57ea018 dir.c: revert r60772, r60770, and r60769
Using readdir(3) without any locking causes thread-safety
problems if directory streams get shared between threads.  On
ancient platforms, readdir(3) may have thread-safety problems
even on different directory streams.

Using readdir_r(3) is not viable, either, as it's deprecated
due to name overflow problems.

So for now, rely on GVL as in previous Rubies and perhaps
consider per-"struct dir_data" mutexes for modern platforms
which allow concurrent calls to readdir(3) on different
directory streams.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60774 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-15 07:24:26 +00:00
usa e9614f9a37 Cannot call rb_thread_call_without_gvl before running VM
* dir.c (readdir_without_gvl): check the VM is already initialized before
  calling rb_thread_call_without_gvl().
  [Bug #14108]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60772 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-15 05:18:37 +00:00
usa 5570dba96f nogvl readdir make SEGV on Windows
* dir.c (readdir_without_gvl): workaround for Windows.  [Bug #14108]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60770 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-15 03:44:58 +00:00
normal ea3211720d dir.c: release GVL around remaining readdir calls
Release GVL around all remaining readdir calls from the Dir
class to prevent pathological stalls on slow filesystems in
multi-threaded applications.

opendir, rewinddir, closedir calls are not affected yet, but
will be changed in future commits.

In the future, further work may be done consolidate multiple GVL
releasing calls to reduce overhead, similar to how changes to
Dir.empty? were made in r60111

* dir.c (nogvl_readdir): new function
  (readdir_without_gvl): ditto
  (dir_read): s/READDIR/readdir_without_gvl/
  (dir_each_entry): ditto
  (glob_helper): ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60769 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-14 23:20:31 +00:00
nobu bfbf5b0b3e dir.c: cast to suppress a warning
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60597 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-01 01:55:45 +00:00
normal f5f6232399 dir: Dir.mkdir and Dir.rmdir release GVL
This avoids blocking the entire VM when operating on slow or
unreliable filesystems.  Instead, only the thread performing
the mkdir or rmdir operation is blocked and other threads
are free to proceed.

* dir.c (nogvl_mkdir): new function
  (nogvl_rmdir): ditto
  (dir_s_mkdir): release GVL
  (dir_s_rmdir): ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60595 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-01 00:17:11 +00:00
normal c9d41f6055 use mode_t where applicable (instead of int)
mode_t is the correct type for these syscalls and it can be
easier-to-understand.  It may also help portability to future
platforms and improve type checking.

* dir.c (dir_s_mkdir): use mode_t for mkdir(2)

* file.c (chmod_internal): use mode_t for chmod(2)
  (rb_file_s_chmod): s/int/mode_t/
  (lchmod_internal): ditto, deref pointer as in chmod_internal
  (rb_file_s_lchmod): pass pointer as in rb_file_s_chmod
  (rb_file_s_rename): use mode_t for umask(2)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60592 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-31 19:10:19 +00:00
normal 0b0e71ba2b dir: Dir.chdir releases GVL
chdir(2) is subject to all the pathological slowdowns and
caveats as open(2) on slow or unreliable filesystems, so ensure
other threads can proceed while this is happening.

* dir.c (nogvl_chdir): new function

* dir.c (dir_chdir): release GVL

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60583 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-30 23:47:27 +00:00
naruse a6488f04de Revert "Dir.glob with FNM_EXTGLOB is optimized [Feature #13873]"
This reverts commit r60341,r60342,r60344,r60345.
Breaking compabitility of the order of result breaks many tests.
To avoid such effort to fix tests, the order should be kept.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60478 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 08:40:40 +00:00