Граф коммитов

31 Коммитов

Автор SHA1 Сообщение Дата
Takashi Kokubun 23f4b78ad8 merge revision(s) 27321290: [Backport #20521]
[Bug #20521] ripper: Clean up strterm
2024-06-11 11:17:32 -07:00
Takashi Kokubun 1ff55bb09d merge revision(s) 05553cf22d43dd78b8f30cc4591230b5c000c538: [Backport #20517]
[Bug #20517] Make a multibyte character one token at meta escape
2024-06-04 14:58:37 -07:00
Nobuyoshi Nakada e5e1f9813e [Bug #19838] Flush delayed token nonconsecutive with the next token 2023-12-02 02:41:39 +09:00
Nobuyoshi Nakada 6be402e172
[Bug #19736] Recover after unterminated interpolation 2023-06-20 20:10:46 +09:00
Nobuyoshi Nakada fad48fefe1 [Bug #19399] Parsing invalid heredoc inside block parameter
Although this is of course invalid as Ruby code, allow to just parse
and tokenize.
2023-02-02 12:20:10 +09:00
Nobuyoshi Nakada ace2eee544
[Bug #18963] Separate string contents by here document terminator 2022-08-28 09:29:24 +09:00
Nobuyoshi Nakada 844a0edbae [Bug #18962] Do not read again once reached EOF
`Ripper::Lexer#parse` re-parses the source code with syntax errors
when `raise_errors: false`.

Co-Authored-By: tompng <tomoyapenguin@gmail.com>
2022-08-12 15:58:18 +09:00
Shugo Maeda a319c04818
Fix typo [ci skip] 2021-02-19 17:27:53 +09:00
Shugo Maeda 5de38c41ae
ripper: fix a bug of Ripper::Lexer with syntax error and heredoc [Bug #17644] 2021-02-19 16:40:29 +09:00
manga_osyo b84b253a69 Fix Ripper with heredoc. 2021-01-17 12:58:13 +09:00
Nobuhiro IMAI 7ff0e93f96
parse.y: handle "duplicated argument name" appropriately on ripper.y
refs: 733ed1e184
2021-01-09 13:33:33 +09:00
Nobuyoshi Nakada 733ed1e184 ripper: fix bad label parameter handling [Bug #17425] 2020-12-23 09:56:35 +09:00
Nobuhiro IMAI e33eb09b76 ripper: fix `#tok` on some error events [Bug 17345]
sorting alias target by event arity, and setup suitable `Elem` for error.
2020-12-19 17:32:39 +09:00
Nobuyoshi Nakada e0bdd54348 Ripper: Refined error callbacks [Bug #17345] 2020-12-15 21:36:23 +09:00
Nobuyoshi Nakada f5ca3ff4db
Store all kinds of syntax errors [Bug #17345] 2020-11-26 20:14:34 +09:00
Nobuyoshi Nakada 1df3896382
Removed a duplicate assertion 2020-11-26 20:03:38 +09:00
Nobuhiro IMAI 1800f3fa5c
Ripper.{lex,tokenize} return full tokens even if syntax error
yet another implements [Feature #17276]
2020-11-20 11:44:57 +09:00
Jeremy Evans cd0877a93e
Support raise_errors keyword for Ripper.{lex,tokenize,sexp,sexp_raw}
Implements [Feature #17276]
2020-11-17 21:15:50 -08:00
Takashi Kokubun 4a7dccf44f
Add a Ripper.lex test of :on_embexpr_end
This is a weird use case of Ripper.lex which I'm not sure is supposed to
be maintained, so I'm adding this test so that we can easily notice such
changes.

If we change the behavior, this will break the behavior of hamlit.gem v1
and code like https://github.com/haml/haml/pull/1043.
2020-10-19 20:56:09 -07:00
Nobuyoshi Nakada ea68bb914a Changed numbered parameter prefix 2019-09-24 21:57:54 +09:00
Nobuyoshi Nakada 5331932cd2
Removed unused variables 2019-06-28 17:08:21 +09:00
Nobuyoshi Nakada 92ecf58b1e
parse.y: adjust here-doc error token
* parse.y (here_document): adjust token to the here-doc identifier
  in compile_error when a here-document misses the closing
  identifier.
2019-05-30 22:03:33 +09:00
Nobuyoshi Nakada b0e2b7a5ff
Include stack elements left after errors 2019-05-30 21:54:38 +09:00
Nobuyoshi Nakada 1da5c73932
parse.y: fix state after ivar/cvar
* parse.y (parse_atmark): return EXPR_END or EXPR_ENDFN, depending
  on the previous state, even incomplete names consistently.
2019-05-29 21:42:58 +09:00
k0kubun 0e60fdc511 test/ripper/test_lexer.rb: add test for r62743
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62744 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-13 13:03:12 +00:00
nobu 8b501f6d7f parse.y: indent at '#'
* parse.y (parser_here_document): update indent at '#', which is
  not a space.  [ruby-core:83368] [Bug #14032]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60213 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-19 05:10:38 +00:00
nobu 0e7936f896 lexer.rb: no dedent strings in middle
* ext/ripper/lib/ripper/lexer.rb (on_heredoc_dedent): dedent only
  strings at the beginning, not strings in middle.
  [ruby-core:83343] [Bug #14027]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60212 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-19 04:25:30 +00:00
yui-knk a7452a8f4f test_lexer.rb: require ripper directly
* test/ripper/test_lexer.rb: This file does not use
  DummyParser, so requiring ripper directly instead of
  requiring dummyparser.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59340 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-07-15 07:24:12 +00:00
nobu d9d8ca0b8f lexer.rb: ignore dedented space
* ext/ripper/lib/ripper/lexer.rb (on_heredoc_dedent): replace an
  empty string content because of dedentation with :on_ignored_sp.
  an empty token makes the sorted order unstable.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59269 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-07-06 04:12:51 +00:00
nobu 4671737b0f ripper/lexer.rb: nested indented heredoc
* ext/ripper/lib/ripper/lexer.rb (on_heredoc_dedent): insert
  stripped leading spaces as `on_ignored_sp` elements, so that the
  original source can be reconsructed.
  [ruby-core:80977] [Bug #13536]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58584 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-06 15:14:58 +00:00
nobu d24c4683f7 ripper/lexer.rb: nested indented heredoc
* ext/ripper/lib/ripper/lexer.rb (on_heredoc_dedent): fix for
  nested indedented here documents, where `Elem`s are nested too.
  [ruby-core:80977] [Bug #13536]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58545 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-02 17:48:10 +00:00