This change partially reverts r61724 and take another approach:
exploiting struct local_vars to backup the cond_stack state.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61727 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (primary): save/restore COND and CMDARG stacks at method
definition, to distinguish do_block and do_cond properly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61724 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because the name "code_range" is ambiguous with encoding's.
Abbreviations ("crange", and "cr") are also renamed to "loc".
The traditional "code_location" (a pair of lineno and column) is
renamed to "code_position". Abbreviations are also renamed
(first_loc to beg_pos, and last_loc to end_pos).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61721 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because the purpose is now unsure (maybe, to support very old bison?).
If an issue occurs, it should be resurrected with explicit comment.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61709 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because the part of the code is already within `#ifndef RIPPER`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61707 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: These nodes are created with `@$` locations.
Start position of `@$` is same as start position of `@1`.
And NEW_XXX macros set first_loc.lineno of a passed
code range to nd_line. So these nd_set_line are not needed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61697 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
"loc" was ambiguous; it might refer both a location and a code range.
This change uses "loc" for a location, and "crange" or "cr" for a code
range.
A location (abbr. loc) is a point in a program and consists of line
number and column number. A code range (abbr. crange and cr) is a range
within a program and consists of a pair of locations which is the first
and the last.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61690 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (block_append_gen): Use nd_loc of
head(tail) when create NEW_BLOCK of head(tail).
e.g. The locations of the NODE_BLOCK is fixed:
```
BEGIN {
:a
}
BEGIN {
:b
}
```
* Before
```
NODE_BLOCK (line: 5, code_range: (5,0)-(7,1))
```
* After
```
NODE_BLOCK (line: 1, code_range: (1,0)-(7,1))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61686 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Follow up of r61676. r61676 removed
ruby_sourceline from some actions. When stop to
use lineno of "n th" symbol, it's better to
use last location's lineno of "n-1 th" symbol.
e.g.
```
primary : k_begin {} bodystmt k_end
```
Before r61676 we use lineno of `@2` (ruby_sourceline).
In this case, last location's lineno of `k_begin` (`@1`)
is suitable.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61685 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I think that recycling the delimiter string objects doesn't pay its
complexity.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61682 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (NULL_LOC): We will create NODEs only
inside of parse.y, so make NULL_LOC to be internal.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61669 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This change initializes nd_line, lineno of each node, by default, by
using the first line number of code range that bison tracks, instead of
extracting from lexer state.
The lexer state basically provides only the last line number of code
range, so many hacks are used to approximate the first line number. The
hacks have been introduced on demand, or very ad-hocly. I think this
change will make it possible to remove most of the hacks.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61667 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_qcall): set nd_line to the method name location.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61651 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Also, use method_add_block instead of direct modification to nd_iter.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61621 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
NODE_PRELUDE contains a `BEGIN` node, a main node, and compile_option.
This node is assumed that it must be located immediately under the root
NODE_SCOPE, but this strange assumption is not so good, IMO.
This change removes the assumtion; it integrates the former two nodes by
block_append, and moves compile_option into rb_ast_body_t.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61610 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yyerror): use the given location as the end of
erred code, instead of the current position.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61538 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yyerror): consider the case first_loc and
last_loc point different lines.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61528 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yyerror): utilize the location given by bison.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61522 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (gettable_gen): warn for __FILE__/__LINE__ when eval
with binding only. promote use of Binding#source_location
instead.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61483 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This change (ad-hocly) adjusts the code range of NODE_SCOPE in
class/module definition because the same adjust is already done in
method definition. I intend to just remove inconsistency between
class/module definition and method definition, but this kind of adjust
is dirty, so it should be fixed later (maybe in 2.6).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61383 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yylex): deal with end of script chars just after
ignored newline as other places. [ruby-core:84349] [Bug #14206]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61346 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Because top_stmts is generated from none (not "/* none */"),
@0 is not set by YYLLOC_DEFAULT. So @0 is a meaningless location.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61198 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Change the last location of none to be
equal to the first location of none.
Sometimes none has length (`parser->tokp` does not
match `lex_p` when none is generated).
This leads to invalid code_ranges.
e.g. The locations of the NODE_CALL (:sort) is fixed:
```
x.sort.join(" ")
```
* Before
```
NODE_CALL (line: 1, code_range: (1,0)-(1,7))
```
* After
```
NODE_CALL (line: 1, code_range: (1,0)-(1,6))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61171 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Currently the location of do_body is set
by new_do_body. Sometimes the last part of do_body is
none, because bodystmt ends with opt_ensure.
Token keyword_end has been looked ahead when a tokenizer
generates none, so the last location of opt_ensure matches
the last location of `end`. But this relation will be
broken when we change the last location of none to be equal to
the first location of none. So set locations of nd_body in
NODE_ITER explicitly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61168 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of mlhs_item
(exclude ',' form range).
e.g. The locations of the NODE_ARRAY is fixed:
```
(a,) = 1,2
```
* Before
```
NODE_ARRAY (line: 1, code_range: (1,1)-(1,3))
```
* After
```
NODE_ARRAY (line: 1, code_range: (1,1)-(1,2))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61133 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (match_op_gen): Fix to only include a range of node2.
e.g. The locations of the NODE_ARRAY is fixed:
```
re =~ s1
```
* Before
```
NODE_ARRAY (line: 1, code_range: (1,0)-(1,8))
```
* After
```
NODE_ARRAY (line: 1, code_range: (1,6)-(1,8))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61130 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of assocs
(exclude ',' form range).
e.g. The locations of the NODE_ARRAY is fixed:
```
m1(str: "bar",)
```
* Before
```
NODE_ARRAY (line: 1, code_range: (1,3)-(1,14))
```
* After
```
NODE_ARRAY (line: 1, code_range: (1,3)-(1,13))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61128 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of assocs.
e.g. The locations of the NODE_ARRAY is fixed:
```
m1(str: "bar", &blk)
```
* Before
```
NODE_ARRAY (line: 1, code_range: (1,3)-(1,19))
```
* After
```
NODE_ARRAY (line: 1, code_range: (1,3)-(1,13))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61126 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Change the last location of NODE_DVAR to
be equal to the first location of NODE_DVAR.
NODE_DVAR of NODE_MASGN (nd_value) is an internal variable,
so it has no length.
e.g. The locations of the NODE_DVAR is changed:
```
a.b {|(c,d)| e}
```
* Before
```
NODE_DVAR (line: 1, code_range: (1,7)-(1,10))
```
* After
```
NODE_DVAR (line: 1, code_range: (1,7)-(1,7))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61124 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (evstr2dstr_gen): Fix to only include a range of node.
e.g. The locations of the NODE_DSTR is fixed:
```
%W[a #{b} c]
```
* Before
```
NODE_DSTR (line: 1, code_range: (1,3)-(1,9))
```
* After
```
NODE_DSTR (line: 1, code_range: (1,5)-(1,9))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61106 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of bodystmt.
e.g. The locations of the NODE_SCOPE is fixed:
```
module M
def m
end
end
```
* Before
```
NODE_SCOPE (line: 4, code_range: (1,0)-(4,3))
```
* After
```
NODE_SCOPE (line: 4, code_range: (1,8)-(4,3))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61102 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range from primary_value to tCONSTANT.
e.g. The locations of the NODE_COLON2 is fixed:
```
A::B ||= 1
```
* Before
```
NODE_COLON2 (line: 1, code_range: (1,0)-(1,10))
```
* After
```
NODE_COLON2 (line: 1, code_range: (1,0)-(1,4))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61101 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (list_append_gen): Change the first location to
start with the location of item if new list is generated.
e.g. The locations of the NODE_ARRAY is changed:
```
"#{a}.#{b}"
```
* Before
```
NODE_ARRAY (line: 1, code_range: (1,0)-(1,6))
```
* After
```
NODE_ARRAY (line: 1, code_range: (1,5)-(1,6))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (arg_append_gen): Fix to only include a range of
node2 if new list is generated.
e.g. The locations of the NODE_ARRAY is fixed:
```
x.default = 5
```
* Before
```
NODE_ARRAY (line: 1, code_range: (1,0)-(1,13))
```
* After
```
NODE_ARRAY (line: 1, code_range: (1,12)-(1,13))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61095 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Change to only include a range from exc_var to compstmt.
e.g. The locations of the NODE_BLOCK is changed:
```
begin
:a
rescue E => e
:b
end
```
* Before
```
NODE_BLOCK (line: 3, code_range: (3,0)-(5,3))
```
* After
```
NODE_BLOCK (line: 3, code_range: (3,9)-(4,4))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of bodystmt.
e.g. The locations of the NODE_SCOPE is fixed:
```
class << []
def m; end
end
```
* Before
```
NODE_SCOPE (line: 3, code_range: (1,0)-(3,3))
```
* After
```
NODE_SCOPE (line: 3, code_range: (2,2)-(3,3))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61092 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of arg1.
e.g. The locations of the NODE_ARRAY is fixed:
```
1 + 2
```
* Before
```
NODE_ARRAY (line: 1, code_range: (1,0)-(1,5))
```
* After
```
NODE_ARRAY (line: 1, code_range: (1,4)-(1,5))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61090 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of bodystmt.
e.g. The locations of the NODE_SCOPE is fixed:
```
class A
def a; end
end
```
* Before
```
NODE_SCOPE (line: 3, code_range: (1,0)-(3,3))
```
* After
```
NODE_SCOPE (line: 3, code_range: (1,7)-(3,3))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61089 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range from modifier_rescue
to stmt (or arg).
e.g. The locations of the NODE_RESBODY is fixed:
```
a rescue 1
```
* Before
```
NODE_RESBODY (line: 1, code_range: (1,0)-(1,10))
```
* After
```
NODE_RESBODY (line: 1, code_range: (1,2)-(1,10))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61074 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to include locations of tSTRING_BEG and tSTRING_END.
e.g. The locations of the NODE_STR is fixed:
```
"a"
```
* Before
```
NODE_STR (line: 1, code_range: (1,1)-(1,2))
```
* After
```
NODE_STR (line: 1, code_range: (1,0)-(1,3))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61072 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (make_array): Set locations of ary to
include locations of start token (tLBRACK, tWORDS_BEG, ...)
and end token (']', tSTRING_END, ...) of array.
e.g. The locations of the NODE_ARRAY is fixed:
```
[1, 2, 3]
```
* Before
```
NODE_ARRAY (line: 1, code_range: (1,1)-(1,8))
```
* After
```
NODE_ARRAY (line: 1, code_range: (1,0)-(1,9))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61070 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (arg_value): initialization of aggregation type with
non-constant values is not allowed in C89.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61068 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range from tSTRING_BEG to tLABEL_END.
e.g. The locations of the NODE_LIT is fixed:
```
{ "a": 10 }
```
* Before
```
NODE_LIT (line: 1, code_range: (1,2)-(1,9))
```
* After
```
NODE_LIT (line: 1, code_range: (1,2)-(1,6))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (dsym_node_gen): Always set locations
to include locations of tSYMBEG and tSTRING_END.
e.g. The locations of the NODE_LIT is fixed:
```
:"a"
```
* Before
```
NODE_LIT (line: 1, code_range: (1,2)-(1,3))
```
* After
```
NODE_LIT (line: 1, code_range: (1,0)-(1,4))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_regexp_gen): Always set locations
to include locations of tREGEXP_BEG and tREGEXP_END.
e.g. The locations of the NODE_LIT is fixed:
```
/a/
```
* Before
```
NODE_LIT (line: 1, code_range: (1,1)-(1,2))
```
* After
```
NODE_LIT (line: 1, code_range: (1,0)-(1,3))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61064 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_xstring_gen): Always set locations
to include locations of tXSTRING_BEG and tSTRING_END.
e.g. The locations of the NODE_XSTR is fixed:
```
`a`
```
* Before
```
NODE_XSTR (line: 1, code_range: (1,1)-(1,2))
```
* After
```
NODE_XSTR (line: 1, code_range: (1,0)-(1,3))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61063 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_heredoc_identifier):
Put length of term at the head of rb_strterm_heredoc_struct.term.
* parse.y (rb_parser_set_location_from_strterm_heredoc):
Use length of term to calculate first_loc.column.
e.g. The locations of the NODE_DSTR is fixed:
```
a <<STR
123
#{:a}
STR
```
* Before
```
NODE_DSTR (line: 3, code_range: (1,3)-(1,7))
```
* After
```
NODE_DSTR (line: 3, code_range: (1,2)-(1,7))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61039 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (rb_parser_set_location_from_strterm_heredoc):
Set locations based on rb_strterm_heredoc_t.
* parse.y (yylex): Set yylloc based on rb_strterm_heredoc_t
when parsing heredoc.
e.g. The locations of the NODE_DSTR is changed:
```
a <<STR
123
#{:a}
STR
```
* Before
```
NODE_DSTR (line: 3, code_range: (3,0)-(1,7))
```
* After
```
NODE_DSTR (line: 3, code_range: (1,3)-(1,7))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (nd_set_loc): set nd_line of the newly created node to
the first location.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61034 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (call_uni_op): set nd_line to the unary operator
location, same as non-operator method calls.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61033 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (logop): set nd_line to the logical operator location,
same as non-operator method calls.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61032 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (match_op): set nd_line to the match operator location,
same as non-operator method calls.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61031 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (call_bin_op): set nd_line to the binary operator
location, same as non-operator method calls.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61030 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of exc_var.
e.g. The locations of the NODE_DASGN_CURR and NODE_ERRINFO are fixed:
```
begin
1
rescue => e
2
end
```
* Before
```
NODE_DASGN_CURR (line: 3, first_lineno: 3, first_column: 0, last_lineno: 5, last_column: 3)
NODE_ERRINFO (line: 5, first_lineno: 3, first_column: 0, last_lineno: 5, last_column: 3)
```
* After
```
NODE_DASGN_CURR (line: 3, first_lineno: 3, first_column: 7, last_lineno: 3, last_column: 11)
NODE_ERRINFO (line: 5, first_lineno: 3, first_column: 7, last_lineno: 3, last_column: 11)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61016 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (arg_append_gen): Update the last location of
NODE_BLOCK_PASS when NODE is appended to nd_head.
e.g. The locations of the NODE_BLOCK_PASS is fixed:
```
o[1, &bl] = :c
```
* Before
```
NODE_BLOCK_PASS (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 8)
```
* After
```
NODE_BLOCK_PASS (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 14)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61011 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (arg_append_gen): Update the last location of
NODE_ARGSCAT when NODE is appended to nd_body.
e.g. The locations of the NODE_ARGSCAT is fixed:
```
m(*a, :b, :c)
```
* Before
```
NODE_ARGSCAT (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 8)
```
* After
```
NODE_ARGSCAT (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 12)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61010 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix the first location to be equal to the location
of the first element of NODE_ARRAY.
e.g. The locations of the NODE_ARRAY is fixed:
```
m(*a, :b, :c)
```
* Before
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 12)
```
* After
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 12)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of opt_call_args.
e.g. The locations of the NODE_ZARRAY is fixed:
```
a[] ||= 1
```
* Before
```
NODE_ZARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 9)
```
* After
```
NODE_ZARRAY (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 3)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61000 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_args_tail_gen): Set only a location of NODE_DVAR.
e.g. The locations of the NODE_DVAR is fixed:
```
def a(k: 1, **kws) end
```
* Before
```
NODE_DVAR (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 18)
```
* After
```
NODE_DVAR (line: 1, first_lineno: 1, first_column: 12, last_lineno: 1, last_column: 17)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60996 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (top_stmt): wrap BEGIN statement to store the whole
location for each block.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60988 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of the first argument of cond.
e.g. The locations of the NODE_MATCH2 and NODE_GVAR are fixed:
```
1 while /#{:a}/
```
* Before
```
NODE_MATCH2 (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 15)
NODE_GVAR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 15)
```
* After
```
NODE_MATCH2 (line: 1, first_lineno: 1, first_column: 8, last_lineno: 1, last_column: 15)
NODE_GVAR (line: 1, first_lineno: 1, first_column: 8, last_lineno: 1, last_column: 15)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of for_var.
e.g. The locations of the NODE_ARGS and NODE_DVAR are fixed:
```
for a in m do n end
```
* Before
```
NODE_ARGS (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 19)
NODE_DVAR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 19)
```
* After
```
NODE_ARGS (line: 1, first_lineno: 1, first_column: 4, last_lineno: 1, last_column: 5)
NODE_DVAR (line: 1, first_lineno: 1, first_column: 4, last_lineno: 1, last_column: 5)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60965 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Fix to only include a range of assocs.
e.g. The locations of the NODE_HASH is fixed:
```
a(1, b: 10, &block)
```
* Before
```
NODE_HASH (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 18)
```
* After
```
NODE_HASH (line: 1, first_lineno: 1, first_column: 5, last_lineno: 1, last_column: 10)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60960 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Use @1 to only include a range of tLABEL.
e.g. The locations of the NODE_LIT(:b) is fixed:
```
a(1, b: 10)
```
* Before
```
NODE_LIT (line: 1, first_lineno: 1, first_column: 5, last_lineno: 1, last_column: 10)
```
* After
```
NODE_LIT (line: 1, first_lineno: 1, first_column: 5, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60958 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (kwd_append): Update the last location of
NODE_KW_ARG when NODE is appended to the last.
e.g. The locations of the first NODE_KW_ARG is fixed:
```
def a(b: 1, c: 2); end
```
* Before
```
NODE_KW_ARG (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 10)
```
* After
```
NODE_KW_ARG (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 16)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (opt_arg_append): Update the last location of
NODE_OPT_ARG when NODE is appended to the last.
e.g. The locations of the first NODE_OPT_ARG is fixed:
```
def a(b = 1, c = 2); end
```
* Before
```
NODE_OPT_ARG (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 11)
```
* After
```
NODE_OPT_ARG (line: 1, first_lineno: 1, first_column: 6, last_lineno: 1, last_column: 18)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_op_assign_gen): Update the location of
lhs when NODE_OP_ASGN_OR/NODE_OP_ASGN_AND are generated.
When NODE_OP_ASGN_OR/NODE_OP_ASGN_AND are generated
a nd_value of lhs is set, so it is needed to update
a location of lhs to include a location of rhs (same as
node_assign_gen).
e.g. The locations of NODE_DASGN_CURR is fixed:
```
a ||= 1
```
* Before
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1)
```
* After
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60903 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (words, symbols, qwords, qsymbols): unify empty list and
non-empty list.
* parse.y (parser_parse_string): always dispatch a word separator
at the beginning of list literals.
[ruby-core:83871] [Bug #14126]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60892 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (logop_gen): Update the last location of
NODE_AND/NODE_OR when NODE is appended to the last.
e.g. The locations of NODE_AND is fixed:
```
a && b && c
```
* Before
```
NODE_AND (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 6)
```
* After
```
NODE_AND (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 11)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60887 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_skip_words_sep): QWORDS_BEG should not include
the first separators in ripper.
* parse.y (parser_parse_string): WORDS_SEP should not include
the closing parentheses of a word list in ripper, should include
spaces at beginning of lines. [ruby-core:83864] [Bug #14126]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Use @2 to only include a range of tSTRING_CONTENT.
e.g. The locations of NODE_STR is fixed:
```
%w[a]
```
* Before
```
NODE_STR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
```
* After
```
NODE_STR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 4)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60868 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: list_append uses the locations
of the second argument. So we should set the
locations of $2 before pass it to list_append.
e.g. The locations of NODE_ARRAY is fixed:
```
%w[a b]
```
* Before
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 0, last_column: -1)
```
* After
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60867 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Use @2 to not include a range of ' '.
e.g. The locations of NODE_LIT is fixed:
```
%i[a]
```
* Before
```
NODE_LIT (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
```
* After
```
NODE_LIT (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 4)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60866 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: list_append uses the locations
of the second argument. So we should set the
locations of $2 before pass it to list_append.
e.g. The locations of NODE_ARRAY is fixed:
```
%i[a b]
```
* Before
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 0, last_column: -1)
```
* After
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60864 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Update the locations of NODE_FCALL
when nd_args is determined.
e.g. The locations of NODE_FCALL is fixed:
```
a 1
```
* Before
```
NODE_FCALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1)
```
* After
```
NODE_FCALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 3)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60862 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (block_append_gen): Update the last
location of NODE_BLOCK when a tail is appended.
e.g. The locations of NODE_BLOCK is fixed:
```
a; b; c
```
* Before
```
NODE_BLOCK (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 4)
```
* After
```
NODE_BLOCK (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Update the locations of NODE_ITER
when nd_iter is determined.
e.g. The locations of NODE_ITER is fixed:
```
a {b}
```
* Before
```
NODE_ITER (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 5)
```
* After
```
NODE_ITER (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60840 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (list_append_gen, list_concat): Update
the last location of NODE_ARRAY when an item is
appended or concatenated with another NODE_ARRAY.
e.g. The locations of NODE_ARRAY is fixed:
```
a(1,2,3)
```
* Before
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 3)
```
* After
```
NODE_ARRAY (line: 1, first_lineno: 1, first_column: 2, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60829 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (node_assign_gen): NODE_CALL is not passed to
node_assign_gen. NODE_CALL is not assignable. Assignable
method call (array set and attr set) is represented by NODE_ATTRASGN.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60823 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (node_assign_gen): NODE_ATTRASGN is created
before rhs is created. It is needed to set location
after rhs is created to rhs range be included to the
location of NODE_ATTRASGN.
e.g. The locations of NODE_ATTRASGN is fixed:
```
a[1] = 2
```
* Before
```
NODE_ATTRASGN (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 4)
```
* After
```
NODE_ATTRASGN (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 8)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60821 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (node_assign_gen): In some case assignable nodes
are created before rhs is created. In this case it is
needed to set location after rhs is shifted to
rhs range be included to assignable nodes.
e.g. The locations of NODE_DASGN_CURR is fixed:
```
a = 10
```
* Before
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1)
```
* After
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 6)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60816 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (k_return): prohibit return in class/module body except
for singleton class.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_op_assign_gen): Fix location of asgn node.
Assignable node (e.g. NODE_LASGN) is generated before rhs
is generated, so we reset the location when nd_value of
asgn is fixed.
e.g. :
```
a -= 10
```
* Before
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1)
```
* After
```
NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7)
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60781 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_params): Add prevline to store previous line.
* parse.y (yycompile0): Initialize prevline with 0.
* parse.y (parser_nextline): Store previous line on prevline.
* parse.y (parser_nextc): Check parser is on EOF or has nextline.
Now parser_yylex does not always set lex_p as lex_pend, we should check
EOF flag and nextline is set.
* parse.y (parser_yylex): Restore previous line, set lex_p and tokp
on '\n'. Before this commit, tokp is on the head of next line of '\n'
and lex_p is on the tail of next line when next token is '\n'.
By this behavior, in some case the last column of NODE_CALL (or NODE_QCALL) is
set to the last column of next line. NODE_CALL can be generated
via `primary_value call_op operation2 {} opt_paren_args` and opt_paren_args
can be none. If none is generated with next token '\n', the last column of
none is set to the last column of next line.
e.g. :
```
a.b
cd.ef
```
The location of NODE_CALL of first line is set to,
* Before
```
NODE_CALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 6)
```
* After
```
NODE_CALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 3)
```
* parse.y (parser_mark): GC mark prevline.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60780 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (node_newnode): Initialize last column of nodes
with -1 and lineno with 0 to make it easy to detect nodes
which we forget to set a column number or lineno.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60751 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* node.c (rb_node_init): Initialize last location with 0.
* node.h (struct rb_code_range_struct): Define a structure
which contains first location and last location of a node.
* node.h (struct RNode): Use rb_code_range_t to store last
location of a node.
* node.h (nd_column, nd_set_column, nd_lineno, nd_set_lineno):
Follow-up the change of struct RNode.
* node.h (nd_last_column, nd_set_last_column, nd_last_lineno, nd_set_last_lineno):
Define getter/setter macros for last location of RNode.
* parse.y : Set last location of tokens.
Thanks to Yusuke Endoh (mame) for design of data structures.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60750 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`rb_ast_dispose` calls `rb_gc_writebarrier_remember`. Unless we call
it, the marked objects may not be GC'ed until `rb_ast_t` is GC'ed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60722 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (rb_strterm_heredoc_t): should not define sourceline as
union, to fix wrong read on big-endian platforms.
as rb_imemo_new stores all members as VALUEs, should read it as
VALUE too.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60720 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_params): removed in_single, which is referenced
with in_def always.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60716 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* node.h (NEW_CASE2): Omit first argument of NEW_CASE2,
bacause the first argument of NEW_CASE2 is always 0.
* parse.y: Ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60714 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
r60655 changed the usage of bison by defining a customized YYLTYPE,
which seemed to cause a random stall at rb_thread_terminate_all.
Kazuki Tsujimoto investigated the issue and pointed out the commit that
caused it. Thanks!
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60660 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
rb_code_range_t has two t_code_location_t, i.e., the first and last
locations.
This is used for YYLTYPE, tracked locations of bison, and will be also
used for representing the "range", the first and the last locations of
each NODE. Currently, each NODE keeps only the first location, though.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60655 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (rb_strterm_mark): do not define in ripper to get rid of
duplicate symbol with static-linked-ext.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This magic number has two meanings depending upon the context:
* "required keyword argument (no name)" on NODE_LASGN (`def foo(x:)`)
* "rest argument (no name)" on NODE_MASGN and NODE_POSTARG
('a, b, * = ary` or `a, b, *, z = ary`)
To show this intention explicitly, two macros are introduced:
NODE_SPECIAL_REQUIRED_KEYWORD and NODE_SPECIAL_NO_NAME_REST.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60650 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`new_args_tail_gen` returned imemo, but the value was later accessed as
`NODE*`. This prevented change of NODE structure.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60647 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (rb_discard_node_gen): rb_discard_node() is not used in
ripper right now.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60645 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_set_line): removed no longer used function.
* parse.y (rb_strterm_heredoc_t): adjust type of sourceline to
ruby_sourceline.
* parse.y (rb_strterm_t): get rid of redefinition.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60644 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This is a follow-up of r60488. Not only the Ruby parser but also Ripper
now use NODE buffer instead of NODE objects managed by GC.
Now we can change the struct of NODEs so that it can keep detailed
location information, perhaps (not tried yet). Note that, however, the
first word of each NODE must be still compatible with RBasic structure.
This is because Ripper handles NODEs and other objects uniformly;
especially, it still uses `RB_TYPE_P(obj, T_NODE)` for distinguishing
between NODEs and other objects.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Just refactoring.
NODE_STRTERM and NODE_HEREDOC are not an internal node of AST, but a
temporary storage for managing termination of string literals and
heredocs. Instead of NODE abuse, I want to use imemo for the storage
in order to avoid (my) confusion.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60634 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
When NODE_WHEN is compiled by iseq_compile_each0,
the node passed to compile_when is NODE_WHEN (not NODE_CASE).
So we can not handle the location of NODE_CASE of
case statements which don't have case expressions.
e.g. :
```
case; when 1; foo; when 2; bar; else baz; end
```
This commit adds NODE_CASE2, and compiles it by
iseq_compile_each0.
* compile.c (compile_case): Does not call COMPILE_ when
NODE_CASE does not have case expressions.
* compile.c (compile_case2): Compile NODE_CASE2 by compile_case2.
* compile.c (compile_when): Delete an obsoleted function.
* compile.c (iseq_compile_each0): Compile NODE_CASE2.
* ext/objspace/objspace.c (count_nodes): Add NODE_CASE2 case.
* node.c (dump_node, rb_gc_mark_node): Add NODE_CASE2 case.
* node.h (node_type): Add NODE_CASE2.
* node.h (NEW_CASE2): Add a macro which generates NODE_CASE2.
* parse.y: Generate NODE_CASE2 if case expressions don't exist.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60585 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* node.c (A_NODE_HEADER): Print lineno.
* node.h (nd_column, nd_set_column): Store column
number with 16-bit.
* node.h (nd_lineno, nd_set_lineno): Define
getter/setter macros for lineno of RNode.
* parse.y : Set first lineno of tokens.
Thanks to takeshinoda for review.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60580 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y : Column of resq is set by new_resbody, so
this line is needless.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This re-introduces r60485.
This reverts commit 5a176b75b1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
NODEs in AST are no longer objects managed by GC. This change will
remove the restriction imposed by the GC. For example, a NODE can use
more than five words (this is my primary purpose; we want to store the
position data for each NODE, for coverage library), or even a NODE can
have variable length (some kinds of NODEs have unused fields).
To do this, however, we need more work, since Ripper still uses T_NODE
objects managed by the GC.
The life time of NODEs is more obvious than other kinds of objects; they
are created at parsing, and they become disused immediately after
compilation. This change releases all NODEs by a few `xfree`s after
compilation, so performance will be improved a bit. In extreme example,
`eval("x=1;" * 10000000)` runs much faster (40 sec. -> 7.8 sec. on my
machine).
The most important part of this change is `ast_t` struct, which has
three contents: (1) NODE buffer (malloc'ed memory), (2) a reference to
the root NODE, and (3) an array that contains objects that must be
marked during parsing (such as literal objects). Some functions that
had received `NODE*` arguments, must now receive `ast_t*`.
* node.c, node.h: defines `ast_t` struct and related operations.
* gc.c, internal.h: defines `imemo_ast`.
* parse.y: makes `parser_params` struct have a reference to `ast_t`.
Instead of `rb_node_newnode`, use `rb_ast_newnode` to create a NODE.
* iseq.c, load.c, ruby.c, template/prelude.c.tmpl: modifies some
functions to handle `ast_t*` instead of `NODE*`.
* test/ruby/test_gc.rb: ad-hoc fix for a failed test. The test assumes
GC eden is increased at startup by NODE object creation. However,
this change now create no NODE object, so GC eden is not necessarily
increased.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60485 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This is just a preparation to manage AST NODEs out of GC.
Currently `add_mark_object` does nothing.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60481 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
imemo_alloc provides a memory buffer whose contents are marked by GC.
C code can access imemo_alloc buffer freely, so imemo_alloc must be
considered writebarrier-unprotected. But T_IMEMO is writebarrier-
protected by default, which caused a GC bug.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60427 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The idiom `rb_gc_force_recycle((VALUE)...)` was used heavily.
This change defines `rb_discard_nod(NODE*)` and allows C compiler to
typecheck.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60391 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The magic number was used to mark NODE_ATTRASGN when its receiver is
self. But the hack was refactored at r46366. So the remaining code
fragments are no longer needed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_op_assign_gen): new_op_assign_gen for
ripper does not need column information.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60289 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_op_assign): Define the macro separately
for ruby parser and for ripper.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60286 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* internal.h (rb_imemo_alloc_struct), gc.c (gc_mark_imemo): turned
next into the pointer to chain.
* parse.y (NEWHEAP): needs a cast.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60244 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parse_numeric): Do not literal_flush if
float is read correctly. This will fix the first
column of float.
before:
```
1.2i1.1
^
```
after:
```
1.2i1.1
^~~
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60225 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_cdecl_gen): Rename local variable name
from cdecl to nd_cdecl. We can not use cdecl on Windows.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60223 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* node.c (A_NODE_HEADER): Print column number.
* node.h (nd_column, nd_set_column): Define getter/setter
macros for column number of RNode.
* parse.y: Set first column number of tokens.
Notes:
* Use `@n` to get the location of token in parse.y.
* When we use `@n`, arguments of yyerror and yylex are
changed.
* Initialize column of nodes with -1 to make it easy
to detect nodes which we forget to set a column number.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60220 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_here_document): update indent at '#', which is
not a space. [ruby-core:83368] [Bug #14032]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60213 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (rel_expr): warn sequence of comparisons, which would be
probably unintentional. [EXPERIMENTAL]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60207 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_prepare): set token_info_enabled flag first,
before returning at BOM. [ruby-dev:50288] [Bug #13998]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60160 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (KWD2EID): should respect the previous callback result
for keywords. [ruby-core:83106] [Bug #13971]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60120 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (mlhs_node): dispatch var_field as well as lhs.
* parse.y (lhs, var_hs): dispatch var_field before assignable
check so it is inside assign_error at a wrong assignment, as
well as backref_assign_error.
* parse.y (var_field_gen): wrap var_field value with the variable
ID.
* parse.y (assignable_gen, const_decl_gen): return the callback
result on a syntax error.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60116 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (reswords): declare reserved words as <id> to remove
extra cast.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60115 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (op): do not set event ID here to dispatch on_op.
[ruby-core:83089] [Bug #13967]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60113 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: stripping wrapping NODEs from ripper callback results,
when storing in Array objects. NODEs must not appear in Ruby
level.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parse_ident): leave identifier type decision (local or
const) to rb_enc_symname_type, and set the token type by ID
type.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parse_ident): disable tLPAREN_ARG state by local
variable. [ruby-list:50578]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59957 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (f_arglist, parser_yylex): set lex_state via
SET_LEX_STATE macro for yydebug messages.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59920 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (ripper_state): add states of scanner to tokens from
Ripper.lex and Ripper::Filter#on_*. based on the patch by
aycabta (Code Ahss) at [ruby-core:81789]. [Feature #13686]
* ext/ripper/tools/preproc.rb (prelude, usercode): generate EXPR_*
constants from enums.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59896 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`unless` statement was a syntactic sugar for `if` statement,
which made the result of branch coverage hard to understand.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59889 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
To prepare new measuring targets: branch and method coverages.
So far, iseq->coverage was an array of counts executed for line coverage.
Now, it is a three-element array for each measuring target,
whose first element is an array for line coverage.
The second element is planned for branch coverage, and the third will be
for method coverage.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59738 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (primary): should not be 0, since it can be a receiver.
[ruby-core:82447] [Bug #13836]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59644 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_number_literal_suffix): keep token after numeric
suffix to show the value in an error message.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59611 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (singleton): literal nodes are always value expressions,
no check is necessary.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59601 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (dvar_defined_ref, dvar_defined): rename macros. only
gettable uses the former. assignable should not set LVAR_USED
flag.
* parse.y (gettable_gen): set used flag on local/dynamic variables
instead of setting in lexer. [ruby-core:82368] [Bug #13809]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59585 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yylex): array is another literal. set state to
END too not only ENDARG and after a literal, so that a left
brace after it should be a primary block bound to the literal,
which causes syntax error. [ruby-core:81037] [Bug #13547]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59420 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_tokadd_codepoint): limit Unicode codepoint
length. too long codepoint has been split unexpectedly since
r57050.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59417 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (f_kwrest): Dispatch kwrest_param event. This is especially
useful for unnamed kwrest parameter for which we expose the internal
ID currently. [ruby-core:75528] [Feature #12387]
* test/ripper/dummyparser.rb (on_kwrest_param): Add handler for
kwrest_param parser event.
* test/ripper/test_parser_events.rb (test_params): Adapt to the change
in DummyParser.
(test_kwrest_param): Test that kwrest_param event handler is called.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (command): NODE_ARRAY with NULL is invalid. traversal
in defined_expr0 is simplified than iseq_compile_each0.
[ruby-core:82113] [Bug #13756]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59375 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yyerror): strip newline in error source which
will not be printed, when lex_p has passed it.
* parse.y (parser_tokadd_utf8): unterminated Unicode escape even
after invalid Unicode escape.
* parse.y (parser_parse_string): show the error source line when
unterminated literal at end of file.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59349 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (literal_flush): rename from numeric_literal_flush, as
it is not just for numerics now.
* parse.y (parser_tokadd_codepoint): show invalid character
position, but not the start of Unicode escape.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59348 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_tokadd_utf8): skip spaces in the current line,
without advancing the line, to get rid of dangling pointer.
[ruby-core:82029] [Bug #13742]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59344 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_read_escape, parser_tok_hex): refine error
messages. point from the backslash up to the invalid char.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59341 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yyerror): add ellipsis properly when error line
is truncated.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59194 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_number_literal_suffix, parse_numeric): flush
numeric token so following unexpected token part is marked.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59191 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (call_bin_op_gen): adjust line number to the receiver,
instead of the rhs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59179 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yyerror): mark the whole last token which caused
the error, not only the end.
before:
```
if end
^
```
after:
```
if end
^~~
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59178 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_precise_mbclen): check invalid multibyte char at
skipping strings following `?x` literal string, not to stuck in
a infinite loop. [ruby-core:81746] [Bug #13672]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59161 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (symbol, dsym, parser_set_number_literal, parser_yylex):
set state to END too not only ENDARG and after a literal, so
that a left brace after it should be a primary block bound to
the literal, which causes syntax error.
[ruby-core:81037] [Bug #13547]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58641 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yylex): set state to ENDARG after a string
literal, so that `do` after a literal should be `do_block` and
bound to the outer method, as well as a numeric/symbol literal.
[ruby-core:72482] [Bug #11873]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58640 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (dedent_string): ensure that the string is modifiable,
not to set the length of shared string.
[ruby-core:80987] [Bug #13540]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58561 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (set_line_body, primary): fix line number of bodystmt as
the beginning of the block. [ruby-core:79388] [Bug #13181]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
mame
kazu
nobu
yui-knk
ko1
svn
rhe