github
/
ruby
зеркало из https://github.com/github/ruby.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
83 750 коммитов 219 Ветки 1 255 Теги 478 MiB
Граф коммитов

2471 Коммитов

Автор SHA1 Сообщение Дата
yui-knk 91cb303531 Remove not used universal parser macros and functions 2024-02-21 13:36:45 +09:00
yui-knk 5fb574ae4e `rb_parser_warn_duplicate_keys` doesn't need to take care of NODE_LIT anymore 
NODE_LIT is created only for `shareable_constant_value`.
This means hash key node is never NODE_LIT.
 2024-02-21 10:57:32 +09:00
yui-knk 776dbbba72 Remove hack for ripper.y generation 
Before Rearchitect Ripper (89cfc15), parser and ripper used different
semantic value data type for same symbols.
"ext/ripper/tools/preproc.rb" replaced these types when it generated
ripper.y. Starting the line with other than `%token` suppressed
the type replacement.
However, after Rearchitect Ripper, both parser and ripper use
same semantic value data type. Therefore these comments are not needed
anymore.
 2024-02-21 09:16:09 +09:00
yui-knk e7ab5d891c Introduce NODE_REGX to manage regexp literal 2024-02-21 08:06:48 +09:00
yui-knk 7cb8fd7800 Move ripper_validate_object to ripper_init.c.tmpl 2024-02-20 19:19:31 +09:00
yui-knk 474f41880f Suppress unused function warning for UNIVERSAL_PARSER build 
Suppress the warning:

```
parse.y:2221:1: warning: unused function 'rb_parser_str_hash' [-Wunused-function]
 2221 | rb_parser_str_hash(rb_parser_string_t *str)
      | ^~~~~~~~~~~~~~~~~~
```
 2024-02-20 19:07:40 +09:00
yui-knk 2a4b6ed37c Workaround for `Prism::ParseTest#test_filepath` for "unparser/corpus/literal/def.txt" 
See the discussion on https://github.com/ruby/ruby/pull/9923
 2024-02-20 17:33:58 +09:00
yui-knk 89cfc15207 [Feature #20257] Rearchitect Ripper 
Introduce another semantic value stack for Ripper so that
Ripper can manage both Node and Ruby Object separately.
This rearchitectutre of Ripper solves these issues.
Therefore adding test cases for them.

* [Bug 10436] https://bugs.ruby-lang.org/issues/10436
* [Bug 18988] https://bugs.ruby-lang.org/issues/18988
* [Bug 20055] https://bugs.ruby-lang.org/issues/20055

Checked the differences of `Ripper.sexp` for files under `/test/ruby`
are only on test_pattern_matching.rb.
The differences comes from the differences between
`new_hash_pattern_tail` functions between parser and Ripper.
Ripper `new_hash_pattern_tail` didn’t call `assignable` then
`kw_rest_arg` wasn’t marked as local variable.
This is also fixed by this commit.

```
--- a/./tmp/before/test_pattern_matching.rb
+++ b/./tmp/after/test_pattern_matching.rb
@@ -3607,7 +3607,7 @@
                  [:in,
                   [:hshptn, nil, [], [:var_field, [:@ident, “a”, [984, 13]]]],
                   [[:binary,
-                    [:vcall, [:@ident, “a”, [985, 10]]],
+                    [:var_ref, [:@ident, “a”, [985, 10]]],
                     :==,
                     [:hash, nil]]],
                   nil]]],
@@ -3662,7 +3662,7 @@
                  [:in,
                   [:hshptn, nil, [], [:var_field, [:@ident, “a”, [993, 13]]]],
                   [[:binary,
-                    [:vcall, [:@ident, “a”, [994, 10]]],
+                    [:var_ref, [:@ident, “a”, [994, 10]]],
                     :==,
                     [:hash,
                      [:assoclist_from_args,
@@ -3813,7 +3813,7 @@
                    [:command,
                     [:@ident, “raise”, [1022, 10]],
                     [:args_add_block,
-                     [[:vcall, [:@ident, “b”, [1022, 16]]]],
+                     [[:var_ref, [:@ident, “b”, [1022, 16]]]],
                      false]]],
                   [:else, [[:var_ref, [:@kw, “true”, [1024, 10]]]]]]]],
                nil,
@@ -3876,7 +3876,7 @@
                      [:@int, “0”, [1033, 15]]],
                     :“&&“,
                     [:binary,
-                     [:vcall, [:@ident, “b”, [1033, 20]]],
+                     [:var_ref, [:@ident, “b”, [1033, 20]]],
                      :==,
                      [:hash, nil]]]],
                   nil]]],
@@ -3946,7 +3946,7 @@
                      [:@int, “0”, [1042, 15]]],
                     :“&&“,
                     [:binary,
-                     [:vcall, [:@ident, “b”, [1042, 20]]],
+                     [:var_ref, [:@ident, “b”, [1042, 20]]],
                      :==,
                      [:hash,
                       [:assoclist_from_args,
@@ -5206,7 +5206,7 @@
                      [[:assoc_new,
                        [:@label, “c:“, [1352, 22]],
                        [:@int, “0”, [1352, 25]]]]]],
-                   [:vcall, [:@ident, “r”, [1352, 29]]]],
+                   [:var_ref, [:@ident, “r”, [1352, 29]]]],
                   false]]],
                [:binary,
                 [:call,
@@ -5299,7 +5299,7 @@
                       [:assoc_new,
                        [:@label, “c:“, [1367, 34]],
                        [:@int, “0”, [1367, 37]]]]]],
-                   [:vcall, [:@ident, “r”, [1367, 41]]]],
+                   [:var_ref, [:@ident, “r”, [1367, 41]]]],
                   false]]],
                [:binary,
                 [:call,
@@ -5931,7 +5931,7 @@
              [:in,
               [:hshptn, nil, [], [:var_field, [:@ident, “r”, [1533, 11]]]],
               [[:binary,
-                [:vcall, [:@ident, “r”, [1534, 8]]],
+                [:var_ref, [:@ident, “r”, [1534, 8]]],
                 :==,
                 [:hash,
                  [:assoclist_from_args,
```
 2024-02-20 17:33:58 +09:00
Nobuyoshi Nakada b1d70e4264
[Bug #20280] Check by `rb_parser_enc_str_coderange` 
Co-authored-by: Yuichiro Kaneko <spiketeika@gmail.com>
 2024-02-19 16:33:26 +09:00
Nobuyoshi Nakada fcc55dc226
[Bug #20280] Raise SyntaxError on invalid encoding symbol 2024-02-19 16:33:26 +09:00
Yusuke Endoh 25d74b9527 Do not include a backtick in error messages and backtraces 
[Feature #16495]
 2024-02-15 18:42:31 +09:00
Nobuyoshi Nakada c57880e68d
Show the invalid source encoding in messages 2024-02-15 13:39:33 +09:00
Peter Zhu a71d1ed838 Fix memory leak when parsing invalid hash symbol 
For example:

    10.times do
      100_000.times do
        eval('{"\xC3": 1}')
      rescue EncodingError
      end

      puts `ps -o rss= -p #{$$}`
    end

Before:

    32032
    48464
    66112
    84192
    100592
    117520
    134096
    150656
    167168
    183760

After:

    17120
    17120
    17120
    17120
    18560
    18560
    18560
    18560
    18560
    18560
 2024-02-13 11:05:56 -05:00
yui-knk 038189b61f Use dedicated parser_string hash function 
Define and use `rb_parser_str_hash` for `rb_parser_string_t`
instead of `rb_str_hash` to remove dependency on `rb_str_hash`.
 2024-02-13 10:56:32 +09:00
yui-knk 8a345860d3 Warn duplication of `__ENCODING__` on the hash 
```
$ ruby -e 'h = { __ENCODING__ => 1, __ENCODING__ => 2 }'
-e:1: warning: key #<Encoding:UTF-8> is duplicated and overwritten on line 1
```
 2024-02-13 08:40:14 +09:00
yui-knk 7fc89a9262 Use Node for `warn_duplicate_keys` st_table keys 2024-02-12 17:46:22 +09:00
yui-knk fdd92c2d61 Fix the variable to be checked 
It should check the result of `rb_parser_search_nonascii`.
 2024-02-10 18:58:42 +09:00
yui-knk 33c1e082d0 Remove ruby object from string nodes 
String nodes holds ruby string object on `VALUE nd_lit`.
This commit changes it to `struct rb_parser_string *string`
to reduce dependency on ruby object.
Sometimes these strings are concatenated with other string
therefore string concatenate functions are needed.
 2024-02-09 14:20:17 +09:00
Peter Zhu b2392c6be4 Fix memory leak when parsing invalid pattern matching 
If the pattern matching is invalid, then the pvtbl would get leaked. For
example:

    10.times do
      100_000.times do
        eval(<<~RUBY)
          case {a: 1}
          in {"a" => 1}
          end
        RUBY
      rescue SyntaxError
      end

      puts `ps -o rss= -p #{$$}`
    end

Before:

    28096
    44768
    61472
    78512
    94992
    111504
    128096
    144528
    161008
    177472

After:

    14096
    14112
    14112
    14176
    14208
    14240
    14240
    14240
    14240
    14240
 2024-02-07 12:15:33 -05:00
yui-knk 68b57ceb46 Use bool to check ascii only in parse_ident 
No need to use ENC_CODERANGE to record ascii only or not.
 2024-02-03 09:15:41 +09:00
S.H f3df218f48
Introduced `rb_node_const_decl_val` function 
Introduce `rb_node_const_decl_val` function to allow `rb_ary_join` and
`rb_ary_reverse` functions to be removed from Universal Parser.
 2024-01-31 13:31:38 +09:00
Nobuyoshi Nakada 23b8337cd1
[Bug #20219] `gettable` returns NULL on error 2024-01-28 19:15:31 +09:00
Nobuyoshi Nakada 5f733a1ae7
[Bug #20217] `rescue` block is void only if all children are void 2024-01-28 18:44:09 +09:00
Nobuyoshi Nakada fed877c791
[Bug #20217] `return` with `ensure` is a void value expression 2024-01-28 18:44:09 +09:00
Nobuyoshi Nakada e018036d89
Rename `nd_head` in `RNode_RESBODY` as `nd_next` 2024-01-28 11:12:22 +09:00
Nobuyoshi Nakada 0f98d284f3
Remove unused `nd_resq` from `RNode_ENSURE` 2024-01-28 11:11:13 +09:00
S.H 9b40f42c22
Introduce `NODE_ENCODING` 
`__ENCODING__ `was managed by `NODE_LIT` with Encoding object. 

Introduce `NODE_ENCODING` for
1. `__ENCODING__` is detectable from AST Node.
2. Reduce dependency Ruby object for parse.y
 2024-01-27 08:11:10 +00:00
yui-knk 68b9a32a62 bvar is not NODE but ID 
Before this commit `ruby -y -e 'tap {|;x, y|}'` failed with SEGV.
This change fixes it.
 2024-01-27 16:50:35 +09:00
Nobuyoshi Nakada ff55d6b8e1
Use `token_seen` and simplify `comment_at_top` 
Instead of scanning before the current comment.
 2024-01-25 15:06:14 +09:00
Jeremy Evans 4f77d8d328 Do not use ruby2_keywords for ... argument forwarding 
This allows ... argument forwarding to benefit from Allocationless
Anonymous Splat Forwarding, allowing the `f` call below to not
allocate an array or a hash.

```ruby
a = [1]
kw = {b: 2}

def c(a, b:)
end

def f(...)
  c(...)
end

f(*a, **kw)
```

This temporarily skips prism locals tests until prism is changed
to use * and ** for ..., instead of using ruby2_keywords.

Ignore failures in rbs bundled gems tests, since they fail due
to this change.
 2024-01-24 18:25:55 -08:00
yui-knk ee7f63ebba Make lastline and nextline to be rb_parser_string 
This commit changes `struct parser_params` lastline and nextline
from `VALUE` (String object) to `rb_parser_string_t *` so that
dependency on Ruby Object is reduced.
`parser_string_buffer_t string_buffer` is added to `struct parser_params`
to manage `rb_parser_string_t` pointers of each line. All allocated line
strings are freed in `rb_ruby_parser_free`.
 2024-01-23 08:58:16 +09:00
yui-knk 3d19409637 Use index for referring to symbols in `args` rule instead of named references 
In `args: args ',' arg_splat`, `args` is not unique name.
Currently the associated rule is interpreted as
`$$ = rest_arg_append(p, $$, $3, &@$);`.
The action works as expected because `$$` is initialized with
`$1` before each action is executed.
However it's misleading then change to use index.
 2024-01-22 16:05:43 +09:00
Nobuyoshi Nakada 0610f555ea
Constify `rb_global_parser_config` 2024-01-14 17:55:11 +09:00
yui-knk ccd45a1399 Stop using Array to manage dummy `end` token locations 
Before this commit, Array is used to store token locations
which expect `end` token, e.g. `class` and `module`.
This commit introduces dedicated struct to manage them
so that dependency on Ruby Object is reduced.
 2024-01-13 20:41:22 +09:00
S-H-GAMELINKS 524770d3dc Suppress warnings in parser_set_encode function 2024-01-12 22:46:07 +09:00
yui-knk b35e21b388 Remove reference counter from rb_parser_config 
It's allocated outside of parser then no need to track
reference count in rb_parser_config.
 2024-01-12 21:17:41 +09:00
yui-knk 52d9e55903 Statically allocate parser config 2024-01-12 21:17:41 +09:00
yui-knk c3b2436154 `set_yylval_literal` is not used 2024-01-12 21:11:00 +09:00
Nobuyoshi Nakada 3d3bc029c5
Reject encodings determined at runtime as source code encodings 
The encodings determined at runtime are affected by the runtime
environment, such as the OS and locale, while the file contents are
not.
 2024-01-11 18:46:51 +09:00
Nobuyoshi Nakada 7cc8d58cc9
Remove duplicate function `nd_st_key_val` 2024-01-11 17:46:09 +09:00
S-H-GAMELINKS a971229462 Fixed return values for some node types in nd_st_key function 2024-01-11 12:26:30 +09:00
Nobuyoshi Nakada e59a730477
`st_index_t` is not `VALUE` 2024-01-10 14:06:33 +09:00
Peter Zhu 02d8bad6e1 Fix memory leak in parser for invalid syntax 
The strterm is leaked when there is invalid syntax.

For example:

    10.times do
      100_000.times do
        begin
          RubyVM::InstructionSequence.compile('private def foo = puts "Hello"')
        rescue SyntaxError
        end
      end

      puts `ps -o rss= -p #{$$}`
    end

Before:

    20384
    26256
    32592
    36720
    42016
    47888
    53248
    57456
    62928
    65936

After:

    16720
    17488
    17616
    17616
    17616
    17616
    17616
    17616
    17616
    16032

Co-Authored-By: Kevin Newton <kddnewton@gmail.com>
 2024-01-09 09:41:02 -05:00
Nobuyoshi Nakada 38bc107f0b
Convert a series of `else if` lines to a `switch` 2024-01-09 18:47:15 +09:00
yui-knk db476cc71c Introduce NODE_SYM to manage symbol literal 
`:sym` was managed by `NODE_LIT` with `Symbol` object.
This commit introduces `NODE_SYM` so that

1. Symbol literal is detectable from AST Node
2. Reduce dependency on ruby object
 2024-01-09 16:07:19 +09:00
Nobuyoshi Nakada 4b01983bf8
Simplify empty hahs with DSTAR 2024-01-09 13:05:34 +09:00
Nobuyoshi Nakada a4406bc89a
Extract repeating NODE references as a local variable 2024-01-09 13:04:26 +09:00
yui-knk 5ecf2d2880 Use `strcmp` to compare strings 2024-01-09 07:42:44 +09:00
yui-knk 41e2d180a3 Do not convert NODE_STR to NODE_LIT when the string is hash key 
parse.y converted NODE_STR when the string is hash key like

```
h1 = {"str1" => 1}
m1("str2" => 2)
m2({"str3" => 3})
```

This commit stop the conversion.
`static_literal_node_p` needs to know the node is for hash key or not
for the optimization.
 2024-01-08 18:48:24 +09:00
yui-knk 7ffff3e043 Change numeric node value functions argument to `NODE *` 
Change the argument to align with other node value functions
like `rb_node_line_lineno_val`.
 2024-01-08 14:02:48 +09:00