Similarly to how the other ignored files are intended for local
development and not for production, the Gemfile and Gemfile.lock files
for a gem only relate to local development and aren't useful to people
installing the gem.
https://github.com/rubygems/rubygems/commit/59049c04be
* Unify length field for embedded and heap strings
The length field is of the same type and position in RString for both
embedded and heap allocated strings, so we can unify it.
* Remove RSTRING_EMBED_LEN
It's the only part that needs "root folder resultion" to figure out the
folder for the cache, but it's only needed for some things, so run that
logic lazily when needed.
https://github.com/rubygems/rubygems/commit/c7b9eae0bc
If we're in inline mode, Bundler first resolves using only local gems,
and if some gems are missing, then it re-resolves using remote gems.
However, "source resolution" from the initial "local" try was being
memoized, resulting in Bundler not looking for some gems remotely in the
second resolution.
This commit forces a proper re-resolve in this case.
https://github.com/rubygems/rubygems/commit/fdc631075e
(https://github.com/ruby/irb/pull/598)
* Rename conf to irb_context
* Drop Nop#irb method because it's only used by irb/ext/loader.rb
We don't need to expose this method to all command classes, especially
when it's just an alias of `irb_context.irb`.
(https://github.com/ruby/irb/pull/599)
I received a `RegexpError` when I typed `::Array[`.
::Array[/Users/mi/ghq/github.com/ruby/irb/lib/irb/completion.rb:236:in `retrieve_completion_data': premature end of char-class: /^Array[/ (RegexpError)
I cannot find where this is defined. I'm guessing neither this
nor the else branch is ever hit. I only found out when testing
VM changes, which had a bug that exposed this.
https://github.com/ruby/racc/commit/8983d0c134
This commit is to add an extra option to enable verbose mode (V=1) in the
generated `Makefile` at runtime of the Ruby to print compiler command lines by
the commands below when building native extensions. It's possible to enable the
verbose mode by setting the environment variable `MAKEFLAGS="V=1"`[1]
implemented in GNU make. However, I wanted to make a consistent user-interface
not depending on the specific make's implementation.
```
$ ruby /path/to/extconf.rb -- --with-verbose
```
You can also add the extra option via rake-compiler gem.
```
$ rake compiler -- --with-verbose
```
If the extra option is not given, the value of the
`RbConfig::CONFIG["MKMF_VERBOSE"]` enabled by the configure option below is
used.
```
$ ./configure --enable-mkmf-verbose
```
For the unit tests, updated the following files.
* The `test/mkmf/test_configuration.rb` was created to test the cases with the
`configuration` method and this implementation.
* Updated the `TestMkmf#assert_separately` to set the extra
arguments in `test/mkmf/base.rb`. Updated tests using the `assert_separately`.
* Added tests for `MakeMakefile#with_config` in the `test/mkmf/test_config.rb`.
[1] https://www.gnu.org/software/make/manual/html_node/Variables_002fRecursion.html
Fixes [Bug #19695]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
We can install RubyGems plugin by "gem install XXX". The installed
plugin is used from the NEXT "gem ...".
For example, "gem install gem-src kaminari" doesn't use gem-src plugin
for kaminari. "gem install gem-src && gem install kaminari" uses
gem-src plugin for kaminari.
How about loading a plugin immediately when the plugin is installed?
If this proposal is implemented, "gem install gem-src kaminari" works
like "gem install gem-src && gem install kaminari".
https://github.com/rubygems/rubygems/commit/4917d96f4c
They have version.rb files with same directory.
But version.rb have been removed at https://github.com/ruby/ruby/pull/3375
There is no reason to locate under the library name of directory.
(https://github.com/ruby/csv/pull/250)
Since PR #159, the minimum Ruby version is 2.5.0, a version which no
longer requires refinements for String#delete_suffix?, String#match? and
Regexp#match?.
(https://github.com/ruby/irb/pull/589)
* Allow `show_source` for private methods
Fix https://github.com/ruby/irb/pull/577
* Pend tests on TruffleRuby
It seems `eval(..., __LINE__ + 1)` does not work.
Other similar tests are also pended on TruffleRuby, so I think it
is acceptable.
* Use `private_method_defined?` instead of `defined?`
https://github.com/ruby/syntax_suggest/pull/187 Handle if/else with
empty/comment
line
Reported in #187 this code:
```
class Foo
def foo
if cond?
foo
else
# comment
end
end
# ...
def bar
if @recv
end_is_missing_here
end
end
```
Triggers an incorrect suggestion:
```
Unmatched keyword, missing `end' ?
1 class Foo
2 def foo
> 3 if cond?
> 5 else
8 end
16 end
```
Part of the issue is that while scanning we're using newlines to determine when to stop and pause. This is useful for determining logically smaller chunks to evaluate but in this case it causes us to pause before grabbing the "end" that is right below the newline. This problem is similar to https://github.com/ruby/syntax_suggest/pull/179.
However in the case of expanding same indentation "neighbors" I want to always grab all empty values at the end of the scan. I don't want to do that when changing indentation levels as it affects scan results.
There may be some way to normalize this behavior between the two, but the tests really don't like that change.
To fix this issue for expanding against different indentation I needed a way to first try and grab any additional newlines with the ability to rollback that guess. In #192 I experimented with decoupling scanning from the AroundBlockScan logic. It also added the ability to take a snapshot of the current scanner and rollback to prior changes.
With this ability in place now we:
- Grab extra empties before looking at the above/below line for the matching keyword/end statement
- If there's a match, grab it
- If there's no match, discard the newlines we picked up in the evaluation
That solves the issue.
https://github.com/ruby/syntax_suggest/commit/513646b912
AroundBlockScan started as a utility class that was meant to be used as a DSL for scanning and making new blocks. As logic got added to this class it became hard to reason about what exactly is being mutated when. I pulled the scanning logic out into it's own class which gives us a clean separation of concerns. This allowed me to remove a lot of accessors that aren't core to the logic provided by AroundBlockScan.
In addition to this refactor the ScanHistory class can snapshot a scan. This allows us to be more aggressive with scans in the future as we can now snapshot and rollback if it didn't turn out the way we wanted.
The change comes with a minor perf impact:
before: 5.092678 0.104299 5.196977 ( 5.226494)
after: 5.128536 0.099871 5.228407 ( 5.249542)
This represents a 0.996x change in speed (where 1x would be no change and 2x would be twice as fast). This is a 0.38% decrease in performance which is negligible. It's likely coming from the extra blocks being created while scanning. This is negligible and if the history feature works well we might be able to make better block decisions which is means fewer calls to ripper which is the biggest bottleneck.
While this doesn't fix https://github.com/ruby/syntax_suggest/issues/187 it's a good intermediate step that will hopefully make working on that issue easier.
https://github.com/ruby/syntax_suggest/commit/ad8487d8aa
The env var DEBUG does not work to produce detailed output. It is SYNTAX_SUGGEST_DEBUG. It was changed as part of the dead_end to syntax_suggest migration for Ruby 3.2
https://github.com/ruby/syntax_suggest/commit/c41da7aab7
(https://github.com/ruby/irb/pull/576)
* Simplify each_top_level_statement, reduce instance vars
* Update lib/irb/ruby-lex.rb
Co-authored-by: Stan Lo <stan001212@gmail.com>
* Remove unused ltype from TestRubyLex#check_state response
* Remove unnecessary const path of TerminateLineInput
* Combine duplicated code state check into method
---------
https://github.com/ruby/irb/commit/172453cec4
Co-authored-by: Stan Lo <stan001212@gmail.com>
(https://github.com/ruby/irb/pull/583)
1. Make `RubyLex#set_input` simply assign the input block. This matches
the behavior of `RubyLex#set_prompt`.
2. Merge `RubyLex#set_input`'s IO configuration logic with `#set_auto_indent`
into `#configure_io`.
(https://github.com/ruby/reline/pull/492)
* Rewrite dialog rendering
* Fix failing test of dialog with small screen
* Add multiple-dialog rendering test
* Add description comments for each part of render_dialog_changes
- Follow up RegexpLiteral at 9264d834215aa7ce14b0273032a7686c20141db9.
- Split the code to be generated so that `REGEXP` does not need
escapes.
- Use `REGEXP.match?` since support for Ruby 2.3 or earlier has been
dropped.
(https://github.com/ruby/irb/pull/574)
`MagicFile` was introduced around v0.9.6, which was like 14~15 years ago.
It was needed because back then we needed to read a file's magic comment
to determine the encoding of it, and read it with that encoding.
Commit: 3ee79e89
But now we expect files to be encoded in UTF-8 and don't specify encoding
through magic comments anymore, `MagicFile` can be retired.
(https://github.com/ruby/irb/pull/571)
* Simplify Locale#load
Instead of loading file content with `MagicFile` and then evaluting it,
we can just use `Kernel.load` to load the file.
* Remove unused optional argument
* Remove unused Locale#require and #toplevel_load
(https://github.com/ruby/irb/pull/573)
`MagicFile` was introduced around v0.9.6, which was like 14~15 years ago.
It was needed because back then we needed to read a file's magic comment
to determine the encoding of it, and read it with that encoding.
Commit: 3ee79e89ad
But now both EN and JA's help-message file are UTF-8 and have removed the
encoding comment, we don't need to open them with `MagicFile` anymore.
(https://github.com/ruby/irb/pull/569)
We don't have to load another file to define the legacy encoding aliases
map because there's only one definition of it. We can define it in
locale.rb directly.
(https://github.com/ruby/irb/pull/568)
In 3ee79e89ad,
`encoding_aliases.rb` was introduced to return the correct encoding object for
`ujis` and `euc` encodings.
However, the return value of `@@legacy_encoding_alias_map[@encoding_name]`
is always overridden by a second look up with `Encoding.find(@encoding_name)`.
So the logic didn't work as expected.
This commit fixes the problem.
I previously left a comment stating I didn't know why a certain method existed. In investigating the code in `CaptureCodeContext#capture_before_after_kws` I found that it was added as to give a slightly less noisy output.
The docs for AroundBlockScan#capture_neighbor_context only describe keywords as being a primary concern. I modified that code to only include lines that are keywords or ends. This reduces the output noise even more.
This allows me to remove that `start_at_next_line` method.
One weird side effect of the prior logic is it would cause this code to produce this output:
```
class OH
def hello
def hai
end
end
```
```
1 class OH
> 2 def hello
4 def hai
5 end
6 end
```
But this code to produce this output:
```
class OH
def hello
def hai
end
end
```
```
1 class OH
> 2 def hello
4 end
5 end
```
Note the missing `def hai`. The only difference between them is that space.
With this change, they're now both consistent.
https://github.com/ruby/syntax_suggest/commit/4a54767a3e
(https://github.com/ruby/reline/pull/535)
This PR was an effort to address #321 (ed_quoted_insert doesn't work
properly) but per the reporter it did not work correctly.
Moreover, it introduced a major regression: Shell job control stopped
working in all applications that use reline, notably IRB.
Bash and other shells send SIGTSTP in response to C-z to implement job
suspension. Handling SIGSTP opts out of this functionality. For a
line oriented terminal program this should be avoided (not to mention,
this behavior diverges from readline's)
https://github.com/ruby/reline/commit/26383d25b8
Co-authored-by: Carl Brasic <cbrasic@drwholdings.com>
(https://github.com/ruby/irb/pull/564)
The current method-redefining approach brings little benefit, makes it
harder to understand the code, and causes warnings like:
> warning: method redefined; discarding old execute
This patch simplifies it while displaying more helpful message when rdoc
couldn't be loaded.
<Class/Module>`
(https://github.com/ruby/irb/pull/562)
Instead of always printing methods inherited from Class or Module, IRB by
default should filter them out unless `<Class/Module>` is specified to be
either of those.
OpenSSL's man page previously stated that "the application is
responsible for seeding the PRNG by calling RAND_add" (see [1]).
So we had this code. However things changed. They no longer
say so, instead "manual (re-)seeding of the default OpenSSL
random generator is not necessary" now (see [2]). It seems all
OpenSSL versions that we support now already behaves like this.
Let's follow that.
[1]: https://www.openssl.org/docs/man1.0.2/man3/RAND_add.html
[2]: https://www.openssl.org/docs/manmaster/man3/RAND_add.html
(https://github.com/ruby/irb/pull/559)
* Remove unnecessary command argument generation code
Now that all the supported Ruby versions handle splat args and splat kwargs,
we don't need that args generation code anymore.
* Remove unused command definition code
If we look at `@EXTEND_COMMANDS`, all commands are defined in a file, which
means the `if load_file` branch is always executed. Therefore we can drop
the else branch of that condition.
* Avoid defining unnecessary command methods
There's no need to define another command method just to call `Command.execute`.
If Gemfile has a lot of dependencies, we have an optimization that uses
the full index in that case, assuming it's going to be faster.
I think this is an old optimization that predates compact index API
times, I believe we no longer need it these days.
Also, since a few releases ago we check for circular dependencies when
resolving by looping through all versions of each name and removing
those that have circular dependencies that would trip up the resolver.
This loop becomes actually very slow when full indexes are used because
to find dependencies of a gemspec, we need to explicitly fetch the
marshaled gemspec (`gemspec.rz` endpoint) for it, so the optimization
has the opposite effect of making things very slow.
https://github.com/rubygems/rubygems/commit/2f46289bd3
Fix a CI error and add a test to ensure we're testing the current version:
```
Run bundle exec rake test
bundler: failed to load command: rake (/home/runner/work/syntax_suggest/syntax_suggest/vendor/bundle/ruby/3.2.0/bin/rake)
/opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/runtime.rb:309:in `check_for_activated_spec!': You have already activated syntax_suggest 1.0.2, but your Gemfile requires syntax_suggest 1.0.3. Since syntax_suggest is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports syntax_suggest as a default gem. (Gem::LoadError)
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/runtime.rb:25:in `block in setup'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/spec_set.rb:138:in `each'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/spec_set.rb:138:in `each'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/runtime.rb:24:in `map'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/runtime.rb:24:in `setup'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler.rb:151:in `setup'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/setup.rb:20:in `block in <top (required)>'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/ui/shell.rb:136:in `with_level'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/ui/shell.rb:88:in `silence'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/setup.rb:20:in `<top (required)>'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/cli/exec.rb:56:in `require_relative'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/cli/exec.rb:56:in `kernel_load'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/cli/exec.rb:23:in `run'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/cli.rb:483:in `exec'
from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
```
syntax_suggest did not work great when there is no new line at the end
of the input file.
Input:
```
def foo
end
end # No newline at end of file
```
Previous output:
```
$ ruby test.rb
test.rb: --> test.rb
Unmatched `end', missing keyword (`do', `def`, `if`, etc.) ?
> 1 def foo
> 2 end
> 3 end # No newline at end of filetest.rb:3: syntax error, unexpected `end' (SyntaxError)
end # No newline at end of file
^~~
```
Note that "test.rb:3: ..." is appended to the last line of the
annotation.
This change makes sure that the annotation ends with a new line.
New output:
```
$ ruby test.rb
test.rb: --> test.rb
Unmatched `end', missing keyword (`do', `def`, `if`, etc.) ?
> 1 def foo
> 2 end
> 3 end # No newline at end of file
test.rb:3: syntax error, unexpected `end' (SyntaxError)
end # No newline at end of file
^~~
```
https://github.com/ruby/syntax_suggest/commit/db4cf9147d
Originally I fixed https://github.com/ruby/syntax_suggest/pull/177 by making the process of comment removal indentation aware. The next commit is the more general fix and means we don't need to carry that additional logic/overhead.
Also: Update syntax via linter
While #177 is reported as being caused by a comment, the underlying behavior is a problem due to the newline that we generated (from a comment). The prior commit fixed that problem by preserving whitespace before the comment. That guarantees that a block will form there from the frontier before it will be expanded there via a "neighbors" method. Since empty lines are valid ruby code, it will be hidden and be safe.
## Problem setup
This failure mode is not fixed by the prior commit, because the indentation is 0. To provide good results, we must make the algorithm less greedy. One heuristic/signal to follow is developer added newlines. If a developer puts a newline between code, it's more likely they're unrelated. For example:
```
port = rand(1000...9999)
stub_request(:any, "localhost:#{port}")
query = Cutlass::FunctionQuery.new(
port: port
).call
expect(WebMock).to have_requested(:post, "localhost:#{port}").
with(body: "{}")
```
This code is split into three chunks by the developer. Each are likely (but not guaranteed) to be intended to stand on their own (in terms of syntax). This behavior is good for scanning neighbors (same indent or higher) within a method, but bad for parsing neighbors across methods.
## Problem
Code is expanded to capture all neighbors, and then it decreases indent level which allows it to capture surrounding scope (think moving from within the method to also capturing the `def/end` definition. Once the indentation level has been increased, we go back to scanning neighbors, but now neighbors also contain keywords.
For example:
```
1 def bark
2
3 end
4
5 def sit
6 end
```
In this case if lines 4, 5, and 6 are in a block when it tries to expand neighbors it will expand up. If it stops after line 2 or 3 it may cause problems since there's a valid kw/end pair, but the block will be checked without it.
TLDR; It's good to stop scanning code after hitting a newline when you're in a method...it causes a problem scanning code between methods when everything inside of one of the methods is an empty line.
In this case it grabs the end on line 3 and since the problem was an extra end, the program now compiles correctly. It incorrectly assumes that the block it captured was causing the problem.
## Extra bit of context
One other technical detail is that after we've decided to stop scanning code for a new neighbor block expansion, we look around the block and grab any empty newlines. Basically adding empty newlines before of after a code block do not affect the parsing of that block.
## The fix
Since we know that this problem only happens when there's a newline inside of a method and we know this particular failure mode is due to having an invalid block (capturing an extra end, but not it's keyword) we have all the metadata we need to detect this scenario and correct it.
We know that the next line above our block must be code or empty (since we grabbed extra newlines). Same for code below it. We can count all the keywords and ends in the block. If they are balanced, it's likely (but not guaranteed) we formed the block correctly. If they're imbalanced, look above or below (depending on the nature of the imbalance), check to see if adding that line would balance the count.
This concept of balance and "leaning" comes from work in https://github.com/ruby/syntax_suggest/pull/152 and has proven useful, but not been formally introduced into the main branch.
## Outcome
Adding this extra check introduced no regressions and fixed the test case. It might be possible there's a mirror or similar problem that we're not handling. That will come out in time. It might also be possible that this causes a worse case in some code not under test. That too would come out in time.
One other possible concern to adding logic in this area (which is a hot codepath), is performance. This extra count check will be performed for every block. In general the two most helpful performance strategies I've found are reducing total number of blocks (therefore reducing overall N internal iterations) and making better matches (the parser to determine if a close block is valid or not is a major bottleneck. If we can split valid code into valid blocks, then it's only evaluated by the parser once, where as invalid code must be continuously re-checked by the parser until it becomes valid, or is determined to be the cause of the core problem.
This extra logic should very rarely result in a change, but when it does it should tend to produce slightly larger blocks (by one line) and more accurate blocks.
Informally it seems to have no impact on performance:
``
This branch:
DEBUG_DISPLAY=1 bundle exec rspec spec/ --format=failures 3.01s user 1.62s system 113% cpu 4.076 total
```
```
On main:
DEBUG_DISPLAY=1 bundle exec rspec spec/ --format=failures 3.02s user 1.64s system 113% cpu 4.098 total
```
https://github.com/ruby/syntax_suggest/commit/13739c6946
When removing comments I previously replaced them with a newline. This loses some context and may affect the order of the indent search which in turn affects the final result. By preserving whitespace in front of the comment, we preserve the "natural" indentation order of the line while also allowing the parser/lexer to see and join naturally consecutive (method chain) lines.
close https://github.com/ruby/syntax_suggest/pull/177
When `tmp/alias` already exists, I'm now getting phantom folders in the directory pointing at older aliases which is distracting/confusing. By checking and removing that alias before symlinking we can prevent this strange behavior (possibly caused by newer Mac OS?).
While working on locking multiple platforms by default, I got an
infinite resolution loop in one of our resolver specs.
The culprit ended up being that when dealing with lockfile specs with
incomplete dependencies (spec appears in lockfile, but its dependencies
don't), those specs were not being properly expired and that tripped up
resolution.
The issue for some reason only manifests when dealing with multiple
lockfile platforms, that's why it only manifested when working on
locking multiple platforms by default.
https://github.com/rubygems/rubygems/commit/4ca72913bb
* Test existing behavior
Typing Ctrl-D ends editing but typing <Del> does not.
Also renamed a test that is not testing ed_delete_next_char but
key_delete.
* Check if line empty first in em_delete
By distributivity of AND over OR, we can factor out this condition. This
will make the next commit simpler.
* Use em_delete in key_delete
When the editing mode is emacs, use `em_delete` in `key_delete`. We need
to add a condition though to `em_delete`, because it implements both
`delete-char` and `end-of-file`. We only want the `end-of-file` behavior
is the key is really Ctrl-D.
This matches the behavior of the <Del> key with readline, i.e. deleting
the next character if there is one, but not moving the cursor, while not
finishing the editing if there are no characters.
Remove !USE_RVARGC code
[Feature #19579]
The Variable Width Allocation feature was turned on by default in Ruby
3.2. Since then, we haven't received bug reports or backports to the
non-Variable Width Allocation code paths, so we assume that nobody is
using it. We also don't plan on maintaining the non-Variable Width
Allocation code, so we are going to remove it.
David Rodríguez
Hiroshi SHIBATA
tomoya ishida
Samuel Williams
Nobuyoshi Nakada
Stan Lo
Gareth Adams
Peter Zhu
ima1zumi
Daniel Pepper
Vinicius Stock
Jeremy Evans
Benoit Daloze
Jun Aruga
Masafumi Koba
Sutou Kouhei
Mau Magnaguagno
Jenny Shen
TSUYUSATO Kitsune
schneems
Adam Daniels
Samuel Giddins
Takumasa Ochi
zzak
Takashi Kokubun
Akinori MUSHA
Carl Brasic
Yusuf Daniju
Aaron Patterson
卜部昌平
John Hawthorn
Marcin Rozmus
Ashley Ellis Pierce
Eric Herscovich
Kasumi Hanazuki
Yusuke Endoh
Vít Ondruch
Étienne Barrié