* parse.y (here_document): a continuing line is not the
terminator. [ruby-core:86283] [Bug #14621]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62873 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (tokadd_string): stop at continued line in dedented here
documents, to dedent for each lines before removing escaped
newlines. [ruby-core:86236] [Bug #14621]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62872 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (yycompile): in some cases (warning, error, dtrace,...),
ruby_sourcefile is expected to be NUL-terminated, so ensure it.
* template/prelude.c.tmpl (prelude_name): NUL-terminate to get rid
of copying static data.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62841 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (k_rescue, k_ensure): ignore indentations of `do`, it
is not at the beginning of line usually.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62838 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (f_kwrest): add the variable name as an argument, as
well as an internal variable.
* parse.y (new_args_tail): now keyword rest argument variable is
always placed between keyword arguments and block argument, so
so just reorder required and optional keyword arguments. no
longer kwrest duplicates.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62832 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: use tSP same as ripper instead of tSPACE.
[ruby-core:86080] [Bug #14597]
* ext/ripper/eventids2.c: tSP is defined in ripper.c now.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62727 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (heredoc_dedent): fix interpolated string literal dedent,
remove indentations from only nodes with the newline flag.
[ruby-core:85983] [Bug #14584]
* parse.y (here_document): set the newline flag on literal string
nodes starting at the beginning of line.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62724 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (tSPACE): define a separate token for escaped space, to
fix `redefining user token number of ' '` error at word list
separator on bison 2.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62675 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: named escaped whitespaces to show unexpected character.
bare whitespaces should not appear outside of word_list.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62663 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (assign_in_cond): refine a warning message for
assignment of a literal in conditinal expression.
[ruby-core:85872] [Bug #14562]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62620 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yylex): as well as `tLBRACE_ARG` (expr block),
`tLBRACE` (primary block) also does not accept a label. only
hash brace accepts a label.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62000 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Currently, parser.y includes core parser and Ripper parser obscurity.
In addition, *some* Ripper code uses the result of the core parser,
which make it difficult to separate the two parsers.
I want to simplify this, not by separating the two parsers, but by
making *all* Ripper actions follows the core parser actions.
In other words, all the core parser actions run always even in Ripper,
and after that, Ripper-specific actions run.
For tha sake, in principle, I want to put `/*% ripper: ... %*/` in the
end of actions.
Anyway, it is too dirty to put it within expressions, IMO.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61990 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I thought this trick was needed because the result of var_field was
passed to different arguments, as follows:
```
$1 = var_field(p, $1);
$$ = backref_assign_error(p, $1, $1, &@$);
```
Currently the DSL supports that one result is passed to one argument.
However, after the refactoring, I found that `backref_assign_error`
uses only one `$1`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61979 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (dot_or_colon): made type of `dot_or_colon` and EXPR_DOT
tokens to <id> and set those IDs at tokenization. type cast at
primary in ripper is no longer needed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (block_param): embed excessed_comma event result in
place of rest argument, instead of dispatching with the whole
parameters.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61977 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
`foo!(...)` means Ripper event, and `foo(...)` means C function/macro
call. This is for fail-safe; if I forget `!` accidentally, it would
fail to compile, instead of wrongly adding a new Ripper event.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61962 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (command): assign to $$. should not rely upon
undocumented behavior.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61960 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Instead of `_foo`. This makes it useful to do word boundary search of
the editor.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61957 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Currently, parse.y actions are hard to read and write because the code
has double meaning (for core parser and for ripper). I think that, if
it is easy to write ripper's code shortly and simply, the double meaning
trick is not needed.
For the sake, this change adds a simple DSL for ripper's code. For
example, in parse.y, we can write:
/*% ripper: stmts_add(stmts_new, void_stmt) %*/
instead of:
$$ = dispatch2(stmts_add, dispatch0(stmts_new),
dispatch0(void_stmt));
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61952 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (assignable): no longer needs to undef since r61899.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61901 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (REQUIRED_KEYWORD): special argument for required
keyword argument, for core and ripper.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61899 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
NODE_FOR was used both for "for"-statement itself and for
multi-assignment of for-statement (for x, y, in...end).
This change separates the two purposes, NODE_FOR for the former, and
newly introduced NODE_FOR_MASGN for the latter.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61871 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (tokadd_mbchar): renamed and expand callers with p.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61866 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Mixing other encoding string literals in one Regexp caused SEGV.
This bug was found by CoverityScan.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61862 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There were four cases that uses new_command_qcall and then
method_add_block. This change factors out the four rules.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61856 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Nowadays, there are less rules whose return value is NULL.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61849 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There are many usages assuming that bodystmt always returns non-null.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61848 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The code fragments were commented out in YARV merge era.
I believe that it will be never needed in near future.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61837 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (yydebug): define to disable a global variable and get
rid of linker error when static linked ext.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61822 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I don't know what it was, but seems that it has been already fixed since
r12117.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61821 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The current parse.y won't compile with yacc since it depends on many
bison's extensions. Also, configure.ac does not have a check for yacc,
so the macro OLD_YACC is no longer used.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61819 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
In parse.y many functions were suffixed "_gen" and had companion macros
to pass struct parser_params implicitly, which made parse.c bigger and
more obscure.
This change expands and removes almost all "*_gen" macros. This
requires explicit passing of struct parser_params, i.e., we need to
write "foo(p, ..)" instead of "foo(..)". However, it is just extra
three letters. I believe that this is easier to understand.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61818 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
For example, `lex_strterm` is expanded to `p->lex.strterm`.
I believe that this expansion make the code straightforward.
They look not so annoying because "parser" was renamed to "p".
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61817 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because I want to use the name "p" for struct parser_params through
parse.c. This change renames "p" to "ptr", "paren", etc. depending upon
the context.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61815 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because the part of the code is already within `#ifndef RIPPER`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61797 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* remove unused argument len
* factor out initialization code of token_info
* make the condition of "mismatched indentations" warning easy to understand
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61796 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
cmdarg_stack became inconsistent state due to look-ahead, and LEXPOP
hack smoothed over the inconsistency.
This commit fixes the inconsisitent state itself, and removes LEXPOP
hack.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61772 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This change partially reverts r61724 and take another approach:
exploiting struct local_vars to backup the cond_stack state.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61727 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (primary): save/restore COND and CMDARG stacks at method
definition, to distinguish do_block and do_cond properly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61724 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because the name "code_range" is ambiguous with encoding's.
Abbreviations ("crange", and "cr") are also renamed to "loc".
The traditional "code_location" (a pair of lineno and column) is
renamed to "code_position". Abbreviations are also renamed
(first_loc to beg_pos, and last_loc to end_pos).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61721 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because the purpose is now unsure (maybe, to support very old bison?).
If an issue occurs, it should be resurrected with explicit comment.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61709 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Because the part of the code is already within `#ifndef RIPPER`.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61707 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: These nodes are created with `@$` locations.
Start position of `@$` is same as start position of `@1`.
And NEW_XXX macros set first_loc.lineno of a passed
code range to nd_line. So these nd_set_line are not needed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61697 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
"loc" was ambiguous; it might refer both a location and a code range.
This change uses "loc" for a location, and "crange" or "cr" for a code
range.
A location (abbr. loc) is a point in a program and consists of line
number and column number. A code range (abbr. crange and cr) is a range
within a program and consists of a pair of locations which is the first
and the last.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61690 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (block_append_gen): Use nd_loc of
head(tail) when create NEW_BLOCK of head(tail).
e.g. The locations of the NODE_BLOCK is fixed:
```
BEGIN {
:a
}
BEGIN {
:b
}
```
* Before
```
NODE_BLOCK (line: 5, code_range: (5,0)-(7,1))
```
* After
```
NODE_BLOCK (line: 1, code_range: (1,0)-(7,1))
```
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61686 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Follow up of r61676. r61676 removed
ruby_sourceline from some actions. When stop to
use lineno of "n th" symbol, it's better to
use last location's lineno of "n-1 th" symbol.
e.g.
```
primary : k_begin {} bodystmt k_end
```
Before r61676 we use lineno of `@2` (ruby_sourceline).
In this case, last location's lineno of `k_begin` (`@1`)
is suitable.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61685 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
I think that recycling the delimiter string objects doesn't pay its
complexity.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61682 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (NULL_LOC): We will create NODEs only
inside of parse.y, so make NULL_LOC to be internal.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61669 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This change initializes nd_line, lineno of each node, by default, by
using the first line number of code range that bison tracks, instead of
extracting from lexer state.
The lexer state basically provides only the last line number of code
range, so many hacks are used to approximate the first line number. The
hacks have been introduced on demand, or very ad-hocly. I think this
change will make it possible to remove most of the hacks.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61667 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_qcall): set nd_line to the method name location.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61651 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Also, use method_add_block instead of direct modification to nd_iter.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61621 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
NODE_PRELUDE contains a `BEGIN` node, a main node, and compile_option.
This node is assumed that it must be located immediately under the root
NODE_SCOPE, but this strange assumption is not so good, IMO.
This change removes the assumtion; it integrates the former two nodes by
block_append, and moves compile_option into rb_ast_body_t.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61610 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yyerror): use the given location as the end of
erred code, instead of the current position.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61538 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (parser_yyerror): consider the case first_loc and
last_loc point different lines.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61528 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
nobu
mame
shyouhei
kazu
yui-knk