зеркало из https://github.com/github/ruby.git
d8601621ed
Implementation for Language Server Protocol (LSP) sometimes needs token information. For example both `m(1)` and `m(1, )` has same AST structure other than node locations then it's impossible to check the existence of `,` from AST. However in later case, it might be better to suggest variables list for the second argument. Token information is important for such case. This commit adds these methods. * Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of` * Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes. * Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node. [Feature #19070] Impacts on memory usage and performance are below: Memory usage: ``` $ cat test.rb root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true) $ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] 11408kb # keep_tokens :false $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 17508kb # keep_tokens :true $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 30960kb ``` Performance: ``` $ cat ../ast_keep_tokens.yml prelude: | src = <<~SRC module M class C def m1(a, b) 1 + a + b end end end SRC benchmark: without_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false) with_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true) $ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml /home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \ --executables="compare-ruby::./ruby -I.ext/common --disable-gem" \ --executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \ --output=markdown --output-compare -v ../ast_keep_tokens.yml compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] warming up.. | |compare-ruby|built-ruby| |:--------------------|-----------:|---------:| |without_keep_tokens | 21.659k| 21.303k| | | 1.02x| -| |with_keep_tokens | 6.220k| 5.691k| | | 1.09x| -| ``` |
||
|---|---|---|
| .github | ||
| basictest | ||
| benchmark | ||
| bin | ||
| bootstraptest | ||
| ccan | ||
| coroutine | ||
| coverage | ||
| cygwin | ||
| defs | ||
| doc | ||
| enc | ||
| ext | ||
| gems | ||
| include | ||
| internal | ||
| lib | ||
| libexec | ||
| man | ||
| misc | ||
| missing | ||
| sample | ||
| spec | ||
| template | ||
| test | ||
| tool | ||
| wasm | ||
| win32 | ||
| yjit | ||
| .appveyor.yml | ||
| .cirrus.yml | ||
| .dir-locals.el | ||
| .document | ||
| .editorconfig | ||
| .gdbinit | ||
| .git-blame-ignore-revs | ||
| .gitattributes | ||
| .gitignore | ||
| .indent.pro | ||
| .rdoc_options | ||
| .rspec_parallel | ||
| .travis.yml | ||
| BSDL | ||
| CONTRIBUTING.md | ||
| COPYING | ||
| COPYING.ja | ||
| GPL | ||
| KNOWNBUGS.rb | ||
| LEGAL | ||
| NEWS.md | ||
| README.EXT | ||
| README.EXT.ja | ||
| README.ja.md | ||
| README.md | ||
| aclocal.m4 | ||
| addr2line.c | ||
| addr2line.h | ||
| array.c | ||
| array.rb | ||
| ast.c | ||
| ast.rb | ||
| autogen.sh | ||
| bignum.c | ||
| builtin.c | ||
| builtin.h | ||
| class.c | ||
| common.mk | ||
| compar.c | ||
| compile.c | ||
| complex.c | ||
| configure.ac | ||
| constant.h | ||
| cont.c | ||
| darray.h | ||
| debug.c | ||
| debug_counter.c | ||
| debug_counter.h | ||
| dir.c | ||
| dir.rb | ||
| dln.c | ||
| dln.h | ||
| dln_find.c | ||
| dmydln.c | ||
| dmyenc.c | ||
| dmyext.c | ||
| encindex.h | ||
| encoding.c | ||
| enum.c | ||
| enumerator.c | ||
| error.c | ||
| eval.c | ||
| eval_error.c | ||
| eval_intern.h | ||
| eval_jump.c | ||
| file.c | ||
| gc.c | ||
| gc.h | ||
| gc.rb | ||
| gem_prelude.rb | ||
| golf_prelude.rb | ||
| goruby.c | ||
| hash.c | ||
| hrtime.h | ||
| id_table.c | ||
| id_table.h | ||
| inits.c | ||
| insns.def | ||
| internal.h | ||
| io.c | ||
| io.rb | ||
| io_buffer.c | ||
| iseq.c | ||
| iseq.h | ||
| kernel.rb | ||
| lex.c.blt | ||
| load.c | ||
| loadpath.c | ||
| localeinit.c | ||
| main.c | ||
| marshal.c | ||
| marshal.rb | ||
| math.c | ||
| memory_view.c | ||
| method.h | ||
| mini_builtin.c | ||
| miniinit.c | ||
| mjit.c | ||
| mjit.h | ||
| mjit.rb | ||
| mjit_c.rb | ||
| mjit_compiler.c | ||
| mjit_compiler.h | ||
| mjit_compiler.rb | ||
| mjit_unit.h | ||
| nilclass.rb | ||
| node.c | ||
| node.h | ||
| numeric.c | ||
| numeric.rb | ||
| object.c | ||
| pack.c | ||
| pack.rb | ||
| parse.y | ||
| prelude.rb | ||
| probes.d | ||
| probes_helper.h | ||
| proc.c | ||
| process.c | ||
| ractor.c | ||
| ractor.rb | ||
| ractor_core.h | ||
| random.c | ||
| range.c | ||
| rational.c | ||
| re.c | ||
| regcomp.c | ||
| regenc.c | ||
| regenc.h | ||
| regerror.c | ||
| regexec.c | ||
| regint.h | ||
| regparse.c | ||
| regparse.h | ||
| regsyntax.c | ||
| ruby-runner.c | ||
| ruby.c | ||
| ruby_assert.h | ||
| ruby_atomic.h | ||
| rubystub.c | ||
| scheduler.c | ||
| shape.c | ||
| shape.h | ||
| signal.c | ||
| siphash.c | ||
| siphash.h | ||
| sparc.c | ||
| sprintf.c | ||
| st.c | ||
| strftime.c | ||
| string.c | ||
| string.rb | ||
| struct.c | ||
| symbol.c | ||
| symbol.h | ||
| symbol.rb | ||
| thread.c | ||
| thread_none.c | ||
| thread_none.h | ||
| thread_pthread.c | ||
| thread_pthread.h | ||
| thread_sync.c | ||
| thread_sync.rb | ||
| thread_win32.c | ||
| thread_win32.h | ||
| time.c | ||
| timev.h | ||
| timev.rb | ||
| trace_point.rb | ||
| transcode.c | ||
| transcode_data.h | ||
| transient_heap.c | ||
| transient_heap.h | ||
| util.c | ||
| variable.c | ||
| variable.h | ||
| version.c | ||
| version.h | ||
| vm.c | ||
| vm_args.c | ||
| vm_backtrace.c | ||
| vm_callinfo.h | ||
| vm_core.h | ||
| vm_debug.h | ||
| vm_dump.c | ||
| vm_eval.c | ||
| vm_exec.c | ||
| vm_exec.h | ||
| vm_insnhelper.c | ||
| vm_insnhelper.h | ||
| vm_method.c | ||
| vm_opts.h | ||
| vm_sync.c | ||
| vm_sync.h | ||
| vm_trace.c | ||
| vsnprintf.c | ||
| warning.rb | ||
| yjit.c | ||
| yjit.h | ||
| yjit.rb | ||
README.md
What is Ruby?
Ruby is an interpreted object-oriented programming language often used for web development. It also offers many scripting features to process plain text and serialized files, or manage system tasks. It is simple, straightforward, and extensible.
Features of Ruby
- Simple Syntax
- Normal Object-oriented Features (e.g. class, method calls)
- Advanced Object-oriented Features (e.g. mix-in, singleton-method)
- Operator Overloading
- Exception Handling
- Iterators and Closures
- Garbage Collection
- Dynamic Loading of Object Files (on some architectures)
- Highly Portable (works on many Unix-like/POSIX compatible platforms as well as Windows, macOS, etc.) cf. https://github.com/ruby/ruby/blob/master/doc/maintainers.rdoc#label-Platform+Maintainers
How to get Ruby with Git
For a complete list of ways to install Ruby, including using third-party tools like rvm, see:
https://www.ruby-lang.org/en/downloads/
The mirror of the Ruby source tree can be checked out with the following command:
$ git clone https://github.com/ruby/ruby.git
There are some other branches under development. Try the following command to see the list of branches:
$ git ls-remote https://github.com/ruby/ruby.git
You may also want to use https://git.ruby-lang.org/ruby.git (actual master of Ruby source) if you are a committer.
How to build
see Building Ruby
Ruby home page
Documentation
Mailing list
There is a mailing list to discuss Ruby. To subscribe to this list, please send the following phrase:
subscribe
in the mail body (not subject) to the address ruby-talk-request@ruby-lang.org.
Copying
See the file COPYING.
Feedback
Questions about the Ruby language can be asked on the Ruby-Talk mailing list or on websites like https://stackoverflow.com.
Bugs should be reported at https://bugs.ruby-lang.org. Read "Reporting Issues" for more information.
Contributing
See "Contributing to Ruby", which includes setup and build instructions.
The Author
Ruby was originally designed and developed by Yukihiro Matsumoto (Matz) in 1995.