Граф коммитов

86549 Коммитов

Автор SHA1 Сообщение Дата
Jeremy Evans 2c79a7641f Remove splatarray true -> splatarray false peephole optimization
The compiler now uses splatarray false for all cases that would
previously have been optimized, so this is all dead code.
2024-07-18 22:17:21 -07:00
Jeremy Evans 3de20efc30 Avoid unnecessary array allocations for f(arg, *arg, **arg, **arg), f(*arg, a: lvar), and other calls
The `f(arg, *arg, **arg, **arg)` case was previously not optimized.
The optimizer didn't optimize this case because of the multiple
keyword splats, and the compiler didn't optimize it because the
`f(*arg, **arg, **arg)` optimization added in
0ee3960685 didn't apply.

I found it difficult to apply this optimization without changing
the `setup_args_core` API, since by the time you get to the ARGSCAT
case, you don't know whether you were called recursively or directly,
so I'm not sure if it was possible to know at that point whether the
array allocation could be avoided.

This changes the dup_rest argument in `setup_args_core` from an int
to a pointer to int.  This allows us to track whether we have allocated
a caller side array for multiple splats or splat+post across
recursive calls. Check the pointed value (*dup_rest) to determine the
`splatarray` argument. If dup_rest is 1, then use `splatarray true`
(caller-side array allocation), then set *dup_rest back to 0, ensuring
only a single `splatarray true` per method call.

Before calling `setup_args_core`, check whether the array allocation
can be avoided safely using `splatarray false`.  Optimizable cases are:

```
// f(*arg)
SPLAT

// f(1, *arg)
ARGSCAT
 LIST

// f(*arg, **arg)
ARGSPUSH
 SPLAT
 HASH nd_brace=0

// f(1, *arg, **arg)
ARGSPUSH
  ARGSCAT
   LIST
  HASH nd_brace=0
```

If so, dup_rest is set to 0 instead of 1 to avoid the allocation.

After calling `setup_args_core`, check the flag. If the flag
includes `VM_CALL_ARGS_SPLAT`, and the pointed value has changed,
indicating `splatarray true` was used, then also set
`VM_CALL_ARGS_SPLAT_MUT` in the flag.

My initial attempt at this broke the `f(*ary, &ary.pop)` test,
because we were not duplicating the ary in the splat even though
it was modified later (evaluation order issue). The initial attempt
would also break `f(*ary, **ary.pop)` or `f(*ary, kw: ary.pop)` cases
for the same reason. I added test cases for those evaluation
order issues.

Add setup_args_dup_rest_p static function that checks that a given
node is safe.  Call that on the block pass node to determine if
the block pass node is safe.  Also call it on each of the hash
key/value nodes to test that they are safe.  If any are not safe,
then set dup_rest = 1 so that `splatarray true` will be used to
avoid the evaluation order issue.

This new approach has the affect of optimizing most cases of
literal keywords after positional splats.  Previously, only
static keyword hashes after positional splats avoided array
allocation for the splat.  Now, most dynamic keyword hashes
after positional splats also avoid array allocation.

Add allocation tests for dynamic keyword keyword hashes after
positional splats.

setup_args_dup_rest_p is currently fairly conservative. It
could definitely be expanded to handle additional node types
to reduce allocations in additional cases.
2024-07-18 22:17:21 -07:00
KJ Tsanaktsidis ca0dae25ed Don't crash if madvise(MADV_FREE or MADV_DONTNEED) fails
The M:N threading stack cleanup machinery tries to call MADV_FREE on the native
thread's stack, and calls rb_bug if it fails. Unfortunately, there's no way to
distinguish between "You passed bad parameters to madvise" and "MADV_FREE is
not supported on the kernel you are running on"; both cases just return EINVAL.
This means that if you have a Ruby on a system that was built on a system with
MADV_FREE and run it on a system without it, you get a crash in nt_free_stack.

I ran into this because rr actually emulates MADV_FREE by just returning EINVAL
and pretending it's not supported (since it can otherwise introduce
nondeterministic behaviour). So if you run bootstraptest/test_ractor.rb under
rr, you get this crash.

I think we should just get rid of the error handling here; freeing memory like
this is strictly optional anyway.

[Bug #20632]
2024-07-19 13:44:01 +10:00
Hiroshi SHIBATA e5c06005f1 mustermann depends on URI::RFC2396_PARSER behavior
It's part of dependencies for Sinatra. we should fix mustermann before final release of Ruby 3.4
2024-07-19 12:40:56 +09:00
alpha0x00 3222c67262 [rubygems/rubygems] Fix line comment issue for map
https://github.com/rubygems/rubygems/commit/7ca06e139b
2024-07-19 03:03:16 +00:00
Kevin Newton 1c81d1a69d [PRISM] Refactor parser support into its own module 2024-07-18 21:15:54 -04:00
eileencodes 69e65b9b5a Fix interpolated sybmol node instructions
If the symbol node is interpolated like this `:"#{foo}"` the instruction
sequence should be `putstring` followed by `intern`. In this case it was
a `putobject` causing the `test_yjit` tests to fail. Note that yjit is
not required to reproduce - the instructions are `putstring` and
`intern` for yjit and non-yjit with the original parser.

To fix I moved `pm_interpolated_node_compile` out of the else, and
entirely removed the conditional. `pm_interpolated_node_compile` knows
how / when to use `putstring` over `putobject` already. The `intern` is
then added by removing the conditional.

Before:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(1,11)>
0000 putobject                              :foo                      (   1)[Li]
0002 leave
```

After:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(1,11)>
0000 putstring                              "foo"                     (   1)[Li]
0002 intern
0003 leave
```

Fixes the test `TestYJIT#test_compile_dynamic_symbol`. Related to ruby/prism#2935
2024-07-18 21:15:43 -04:00
Hiroshi SHIBATA 8db2325a11 [ruby/uri] Also support URI::PATTERN with switch-back
https://github.com/ruby/uri/commit/823697edb4
2024-07-19 00:50:38 +00:00
Hiroshi SHIBATA 082335494b [ruby/uri] Added test for constant definition and remove URI::REGEXP when using RFC3986_PARSER
https://github.com/ruby/uri/commit/6f616d97fc
2024-07-19 00:50:37 +00:00
Hiroshi SHIBATA 2a56c1841d [ruby/uri] URI.extract needs to pass block
If given block to URI.extract, it returns nil.

https://github.com/ruby/uri/commit/984145c407
2024-07-19 00:50:37 +00:00
Hiroshi SHIBATA 862041d054 [ruby/uri] Rename and switch RFC2396_PARSER test
https://github.com/ruby/uri/commit/2e0f73f05e
2024-07-19 00:50:36 +00:00
Hiroshi SHIBATA ce4da88a57 [ruby/uri] Switch to inspect with default parser
https://github.com/ruby/uri/commit/0ab9abbf08
2024-07-19 00:50:36 +00:00
Hiroshi SHIBATA 6452cf5cb5 [ruby/uri] Added compatibility methods for RFC2396 parser
https://github.com/ruby/uri/commit/bbb8a40eae
2024-07-19 00:50:36 +00:00
Hiroshi SHIBATA 08e449d89b [ruby/uri] Added URI.parser= method for switch back to RFC2396_Parser
https://github.com/ruby/uri/commit/d7dc19ad3f
2024-07-19 00:50:35 +00:00
Takashi Kokubun 8df74deab1 YJIT: Tweak a comment a little [ci skip] 2024-07-18 13:03:17 -07:00
Takashi Kokubun 2de8b5b805
YJIT: Allow dev_nodebug to disasm release-mode code (#11198)
* YJIT: Allow dev_nodebug to disasm release-mode code

* Revert "YJIT: Squash canary before falling back"

This reverts commit f05ad373d8.
The stray canary issue should have been solved by
def7023ee4, alleviating this codegen
accommodation.

* s/runtime_assertions/runtime_checks/

---------

Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
2024-07-18 13:01:47 -07:00
Kevin Newton 059535bd65 [ruby/prism] Mark local variable writes in value positions as being read
https://github.com/ruby/prism/commit/f5149870ab
2024-07-18 19:38:53 +00:00
Kevin Newton 1fd1fb2aa5 [PRISM] Use KW_SPLAT_MUT when possible for method calls 2024-07-18 15:30:45 -04:00
Kevin Newton 53710be557 [PRISM] Use concattoarray instead of splatarray+concatarray 2024-07-18 15:30:45 -04:00
David Rodríguez 50e7c8f051 [rubygems/rubygems] Fix unused variable warning when running RubyGems tests
https://github.com/rubygems/rubygems/commit/155d8fd051
2024-07-18 19:23:06 +00:00
David Rodríguez 104dad3dd0 [rubygems/rubygems] Small tweak to avoid making the same mistake again
We checking completeness of a SpecSet, we should always ignore
dependencies not relevant for the current platform, since the resolver
and the lockfile ignore those too.

https://github.com/rubygems/rubygems/commit/c4b0c6d84e
2024-07-18 18:08:37 +00:00
David Rodríguez d62af8e513 [rubygems/rubygems] Fix another removal issue
I failed to ignore (again) specs only considered for resolution under
some platforms that are not the current one.

https://github.com/rubygems/rubygems/commit/b72deec57e
2024-07-18 18:08:37 +00:00
David Rodríguez b07c77730b [rubygems/rubygems] Simplify spec assertion
All that we expect here is no changes.

https://github.com/rubygems/rubygems/commit/ff984b6133
2024-07-18 18:08:36 +00:00
David Rodríguez c9d2343f5c [rubygems/rubygems] Fix incorrect standalone script when default gems with extensions are used
https://github.com/rubygems/rubygems/commit/55649cd09b
2024-07-18 18:07:09 +00:00
David Rodríguez bb9a9f31ca [rubygems/rubygems] Remove unnecessary `artifice` parameter
It's automatically detected from Gemfile.

https://github.com/rubygems/rubygems/commit/72301a2e3b
2024-07-18 18:07:08 +00:00
David Rodríguez ba6ffaf2e0 [rubygems/rubygems] Use latest shellwords for standalone test
https://github.com/rubygems/rubygems/commit/fcd04daf68
2024-07-18 18:07:08 +00:00
David Rodríguez 99bf4021fb [rubygems/rubygems] Ext is generally not in `require_paths`
https://github.com/rubygems/rubygems/commit/83b417a166
2024-07-18 18:07:07 +00:00
Peter Zhu d6ef74407b Use rb_obj_hide instead of setting klass to 0 2024-07-18 13:47:00 -04:00
Kevin Newton b1608fc6bc [PRISM] Do not respect xflag when eflag is set 2024-07-18 13:03:33 -04:00
Kevin Newton 8e5ac5a87d [PRISM] Ensure not opening directories 2024-07-18 13:03:25 -04:00
Kevin Newton 76ea5cde2a Refactor RUBY_DESCRIPTION assertions in test_rubyoptions 2024-07-18 13:03:17 -04:00
eileencodes aa3030ac24 Fix empty hash instruction
When we have an empty hash the iseq should have a `newhash` but instead
had a `duphash`. To fix, check if the node's elements are equal to `0`.
If so we want a `newhash`, otherwise use the original `duphash`
instructions.

Before:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(1,2)>
0000 duphash                                {}                        (   1)[Li]
0002 leave
```

After:

```
== disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(1,2)>
0000 newhash                                0                         (   1)[Li]
0002 leave
```

Fixes the test `TestYJIT#test_compile_newhash`. Related to ruby/prism#2935
2024-07-18 10:12:20 -04:00
tomoya ishida c304bf13b5 [ruby/irb] Clear ENV["XDG_CONFIG_HOME"] to avoid loading
user-defined irbrc in TestIRB::ConfigValidationTest
(https://github.com/ruby/irb/pull/982)

https://github.com/ruby/irb/commit/632da0ff29
2024-07-18 10:56:19 +00:00
Nobuyoshi Nakada b61e3a6218 Write rbinc files at once
Unexpected error can make empty files which result in unclear
compilation errors.
2024-07-18 19:34:10 +09:00
yui-knk 231a9acc15 Free `data` of `struct rb_parser_ary` in `rb_parser_ary_free`
For example:

    10.times do
      100_000.times do
        RubyVM::AbstractSyntaxTree.parse("x = 1 + 2 +", keep_tokens: true)
      rescue SyntaxError
      end

      puts `ps -o rss= -p #{$$}`
    end

Before:

    28944
    44816
    60720
    76496
    92336
   108160
   123968
   139808
   155648
   171408

After:

    11984
    12704
    12816
    12832
    13072
    13088
    13088
    13136
    13136
    13152
2024-07-18 19:19:27 +09:00
David Rodríguez 86c99a8d14 [rubygems/rubygems] Fix gemspec `require_paths` type validation
It was not properly being detected as an Array attribute, and thus not
properly validated.

Fixing this allows us to remove a strange `rescue` clause in Bundler.

https://github.com/rubygems/rubygems/commit/4121a32408
2024-07-18 09:25:17 +00:00
David Rodríguez 95728a8b42 [rubygems/rubygems] Warn non flattened require paths in old RubyGems versions too
https://github.com/rubygems/rubygems/commit/b3cdccc6fb
2024-07-18 09:25:16 +00:00
David Rodríguez f78a8761c4 [rubygems/rubygems] Remove unnecessary Windows test skip
https://github.com/rubygems/rubygems/commit/946180f5c1
2024-07-18 09:25:16 +00:00
Nobuyoshi Nakada c032e2c225 [rubygems/rubygems] Use `caller_locations` instead of splitting `caller`
Also limit caller ranges

https://github.com/rubygems/rubygems/commit/a274b1af78
2024-07-18 04:20:07 +00:00
David Rodríguez c639bacd45 [rubygems/rubygems] Fix detection of `gem_repo1` being updated
https://github.com/rubygems/rubygems/commit/9f9493c77c
2024-07-18 04:20:06 +00:00
Hiroshi SHIBATA e0f40dc9b0
Split URI::Parser examples with RFC2396 and RFC3986
Prepare for https://github.com/ruby/uri/pull/107
2024-07-18 12:42:36 +09:00
Naoto Ono 509f1b50c2 Lanunchable: Add missing condition statement
Addresses https://github.com/ruby/ruby/pull/11183/files#r1680617485.
2024-07-18 12:22:47 +09:00
Nobuyoshi Nakada d11d615ba6
Fix `utimesat` availability condition
As `__has_attribute` macro is always defined in internal/compilers.h,
gcc warns `-Wunguarded-availability-new` as unknown option.  Check if
the warning option is usable instead.
2024-07-18 11:20:17 +09:00
Hartley McGuire d0c17cbd09
Require space between hash/content in ATX heading (#1140)
While writing some Markdown documentation for Rails, I came across an
interesting case where trying to link to an instance method at the start
of a line would instead parse as an H1 heading:

```markdown
#response_body=
```

Expected:

```html
<a href=""><code>#response_body=</code></a>
```

Actual:

```html
<h1>response_body=</h1>
```

According to the CommonMark spec:

> At least one space or tab is required between the # characters and the
> heading’s contents, unless the heading is empty. Note that many
> implementations currently do not require the space. However, the space
> was required by the original ATX implementation, and it helps prevent
> things like the following from being parsed as headings:
>
> Example 64

So while some implementations do not follow this requirement, I believe
RDoc should because it makes it easy to write text similar to Example 64
(which was used in the new test) and it also enables automatically
linking to instance methods at the start of a line.
2024-07-18 09:40:01 +09:00
Stan Lo 239d54dfbc [ruby/rdoc] Improve rubocop setup
(https://github.com/ruby/rdoc/pull/1139)

* Rename rake rubocop to rake format_generated_files

* Add rubocop rules to ensure spaces are applied consistently

* Improve rubocop related CI workflows

https://github.com/ruby/rdoc/commit/27932d001c
2024-07-17 20:43:08 +00:00
Peter Zhu 573c2893dc Don't disable GC in rb_gc_impl_object_id
Disabling GC when creating the object ID was introduced in commit
67b2c21, but we shouldn't need to disable the GC.
2024-07-17 15:46:41 -04:00
Kevin Newton 2cc20c06e0
[PRISM] Use RSTRING_PTR for Ruby parsing with fgets 2024-07-17 15:45:07 -04:00
Kevin Newton e77e4aa608 [ruby/prism] Have parse_stream handle NUL bytes
https://github.com/ruby/prism/commit/4a41d298c8
2024-07-17 19:44:32 +00:00
Koichi ITO 0fe816f380 [ruby/prism] [Doc] Tweak example of `Prism::Dispatcher`
This PR tweaked the documentation to correct an error encountered
when running the example code of `Prism::Dispatcher`.
This aims to make understanding the example smoother.

https://github.com/ruby/prism/commit/165a1a0e78
2024-07-17 18:17:59 +00:00
Kevin Newton 7de2c06352
[PRISM] Use RSTRING_LEN for Prism stream parsing 2024-07-17 14:13:16 -04:00