Граф коммитов

170 Коммитов

Автор SHA1 Сообщение Дата
Nobuyoshi Nakada 54f0e63a8c Remove `NODE_DASGN_CURR` [Feature #18406]
This `NODE` type was used in pre-YARV implementation, to improve
the performance of assignment to dynamic local variable defined at
the innermost scope.  It has no longer any actual difference with
`NODE_DASGN`, except for the node dump.
2021-12-13 12:53:03 +09:00
S.H ec7f14d9fa
Add `nd_type_p` macro 2021-12-04 00:01:24 +09:00
Yusuke Endoh feda058531 Refactor hacky ID tables to struct rb_ast_id_table_t
The implementation of a local variable tables was represented as `ID*`,
but it was very hacky: the first element is not an ID but the size of
the table, and, the last element is (sometimes) a link to the next local
table only when the id tables are a linked list.

This change converts the hacky implementation to a normal struct.
2021-11-21 08:59:24 +09:00
Yusuke Endoh 753cfbdbf3 node.c (dump_node): update format explanation for NODE_ARGS 2021-11-17 23:38:52 +09:00
Yusuke Endoh 5a7b4dba26 node.c (dump_node): trivial refactoring 2021-11-17 23:38:19 +09:00
Nobuyoshi Nakada 6504ca006b
Show node IDs in dump 2021-07-12 12:10:16 +09:00
Yusuke Endoh acae5f363d ast.rb: RubyVM::AST.parse and .of accepts `save_script_lines: true`
This option makes the parser keep the original source as an array of
the original code lines. This feature exploits the mechanism of
`SCRIPT_LINES__` but records only the specified code that is passed to
RubyVM::AST.of or .parse, instead of recording all parsed program texts.
2021-06-18 02:34:27 +09:00
Yusuke Endoh e48109d86f Partially revert 2c7d3b3a72
to make imemo_ast WB-protected again. Only the test is kept.
2021-04-27 17:05:19 +09:00
Yusuke Endoh 2c7d3b3a72 node.c (rb_ast_new): imemo_ast is WB-unprotected
Previously imemo_ast was handled as WB-protected which caused a segfault
of the following code:

    # shareable_constant_value: literal
    M0 = {}
    M1 = {}
    ...
    M100000 = {}

My analysis is here: `shareable_constant_value: literal` creates many
Hash instances during parsing, and add them to node_buffer of imemo_ast.
However, the contents are missed because imemo_ast is incorrectly
WB-protected.

This changeset makes imemo_ast as WB-unprotected.
2021-04-26 22:46:51 +09:00
Nobuyoshi Nakada c060bdc2b4
NODE markability should not change by nd_set_type 2021-01-14 16:12:02 +09:00
Kazuki Tsujimoto e03e1982bd
Change NODE layout for pattern matching
I prefer pconst to be the first element of NODE.

  Before:

       | ARYPTN | FNDPTN | HSHPTN
    ---+--------+--------+-----------
    u1 | imemo  | imemo  | pkwargs
    u2 | pconst | pconst | pconst
    u3 | apinfo | fpinfo | pkwrestarg

  After:

       | ARYPTN | FNDPTN | HSHPTN
    ---+--------+--------+-----------
    u1 | pconst | pconst | pconst
    u2 | imemo  | imemo  | pkwargs
    u3 | apinfo | fpinfo | pkwrestarg
2020-11-01 16:19:07 +09:00
Nobuyoshi Nakada 081cc4eb28
Dump FrozenCore specially 2020-10-20 23:52:19 +09:00
Nobuyoshi Nakada 7b2bea42a2
Unfreeze string-literal-only interpolated string-literal
[Feature #17104]
2020-09-30 22:15:28 +09:00
Kazuki Tsujimoto fcdbdff631
rb_{ary,fnd}_pattern_info: Remove imemo member to reduce memory usage
This is a partial revert commit of 8f096226e1.

NODE layout:

  Before:

       | ARYPTN | FNDPTN | HSHPTN
    ---+--------+--------+-----------
    u1 | pconst | pconst | pconst
    u2 | unused | unused | pkwargs
    u3 | apinfo | fpinfo | pkwrestarg

  After:

       | ARYPTN | FNDPTN | HSHPTN
    ---+--------+--------+-----------
    u1 | imemo  | imemo  | pkwargs
    u2 | pconst | pconst | pconst
    u3 | apinfo | fpinfo | pkwrestarg
2020-08-02 01:04:06 +09:00
Aaron Patterson 7533519990
NODE_MATCH needs reference updating 2020-07-30 11:11:13 -07:00
Aaron Patterson 35ba2783fe Use a linked list to eliminate imemo tmp bufs for managing local tables
This patch changes local table memory to be managed by a linked list
rather than via the garbage collector.  It reduces allocations from the
GC and also fixes a use-after-free bug in the concurrent-with-sweep
compactor I'm working on.
2020-07-27 12:40:01 -07:00
Koichi Sasada a0f12a0258
Use ID instead of GENTRY for gvars. (#3278)
Use ID instead of GENTRY for gvars.

Global variables are compiled into GENTRY (a pointer to struct
rb_global_entry). This patch replace this GENTRY to ID and
make the code simple.

We need to search GENTRY from ID every time (st_lookup), so
additional overhead will be introduced.
However, the performance of accessing global variables is not
important now a day and this simplicity helps Ractor development.
2020-07-03 16:56:44 +09:00
Kazuki Tsujimoto ddded1157a
Introduce find pattern [Feature #16828] 2020-06-14 09:24:36 +09:00
卜部昌平 5e22f873ed decouple internal.h headers
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead.  This would significantly
speed up incremental builds.

We take the following inclusion order in this changeset:

1.  "ruby/config.h", where _GNU_SOURCE is defined (must be the very
    first thing among everything).
2.  RUBY_EXTCONF_H if any.
3.  Standard C headers, sorted alphabetically.
4.  Other system headers, maybe guarded by #ifdef
5.  Everything else, sorted alphabetically.

Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
2019-12-26 20:45:12 +09:00
Nobuyoshi Nakada fb6a489af2
Revert "Method reference operator"
This reverts commit 67c5747369.
[Feature #16275]
2019-11-12 17:24:48 +09:00
Aaron Patterson 7460c884fb
Use an identity hash for pinning Ripper objects
Ripper reuses parse.y for its implementation.  Ripper changes the
grammar productions to sometimes return Ruby objects.  This Ruby objects
are put in to the parser's stack, so they must be kept alive.  This is
where the "mark_ary" comes in.  The mark array ensures that Ruby objects
created and pushed on the stack during the course of parsing will stay
alive for the life of the parsing functions.

Unfortunately, Arrays do not prevent their contents from moving.  If the
compactor runs, objects on the parser stack could move because the array
won't prevent them from moving.  But the GC doesn't know about the
parser stack, so it can't update references in that stack (it will
update them in the array).

This commit changes the mark array to be an identity hash.  Since the
identity hash relies on memory addresses for the definition of identity,
the GC will not allow keys in an identity hash to move.  We can prevent
movement of objects in the parser stack by sticking them in an identity
hash.
2019-11-05 08:24:14 -08:00
卜部昌平 7e0ae1698d avoid overflow in integer multiplication
This changeset basically replaces `ruby_xmalloc(x * y)` into
`ruby_xmalloc2(x, y)`.  Some convenient functions are also
provided for instance `rb_xmalloc_mul_add(x, y, z)` which allocates
x * y + z byes.
2019-10-09 12:12:28 +09:00
Nobuyoshi Nakada 0c6f36668a
Adjusted spaces [ci skip] 2019-09-27 10:20:56 +09:00
Aaron Patterson 293c6c8cc3
Add compaction support to `rb_ast_t`
This commit adds compaction support to `rb_ast_t`.
2019-09-26 15:41:46 -07:00
Aaron Patterson 414a80d242
`NODE_MATCH` needs to be marked / allocated from marking bucket
Fixes a test in RubySpec
2019-09-10 10:44:49 -07:00
Aaron Patterson 4524780d17
Revert "Reverting node marking until I can fix GC problem."
This reverts commit 092f31e7e2.
2019-09-09 14:26:51 -07:00
Yusuke Endoh 99c9431ea1 Rename NODE_ARRAY to NODE_LIST to reflect its actual use cases
and NODE_ZARRAY to NODE_ZLIST.

NODE_ARRAY is used not only by an Array literal, but also the contents
of Hash literals, method call arguments, dynamic string literals, etc.
In addition, the structure of NODE_ARRAY is a linked list, not an array.

This is very confusing, so I believe `NODE_LIST` is a better name.
2019-09-07 13:56:29 +09:00
Aaron Patterson 092f31e7e2
Reverting node marking until I can fix GC problem.
Looks like we're getting WB misses during stressful GC on startup.  I am
investigating.
2019-09-05 12:44:23 -07:00
Aaron Patterson f211ab2015
I forgot to add `break` in my case statements
Give me a break.
2019-09-05 11:37:03 -07:00
Aaron Patterson 8f096226e1
Stash tmpbuffer inside internal structs
I guess those AST node were actually used for something, so we'd better
not touch them.  Instead this commit just puts the tmpbuffer inside a
different internal struct so that we can mark them.
2019-09-05 11:04:43 -07:00
Aaron Patterson 8cd845aa5b
add debugging code to the mark function 2019-09-05 10:13:50 -07:00
Aaron Patterson 01aa2462b5
lazily allocate the mark array 2019-09-05 10:13:50 -07:00
Aaron Patterson 545b6db3fb
Create two buckets for allocating NODE structs
This commit adds two buckets for allocating NODE structs, then allocates
"markable" NODE objects from one bucket.  The reason to do this is so
when the AST mark function scans nodes for VALUE objects to mark, we
only scan NODE objects that we know to reference VALUE objects.  If we
*did not* divide the objects, then the mark function spends too much
time scanning objects that don't contain any references.
2019-09-05 10:13:50 -07:00
Aaron Patterson f0fd1c0cd8
Stash the imemo buf at the end of the ID list
Now we can reach the ID table buffer from the id table itself, so when
SCOPE nodes are marked we can keep the buffers alive.  This eliminates
the need for the "mark array" during normal parse / compile (IOW *not*
Ripper).
2019-09-05 10:13:50 -07:00
Aaron Patterson 64817a7cfd
Mark some tmpbufs via node objects
This way we don't need to add the tmpbufs to a Ruby array for marking
2019-09-05 10:13:50 -07:00
Aaron Patterson 581fcde088
Directly mark node objects instead of using a mark array
This patch changes the AST mark function so that it will walk through
nodes in the NODE buffer marking Ruby objects rather than using a mark
array to guarantee liveness.  The reason I want to do this is so that
when compaction happens on major GCs, node objects will have their
references pinned (or possibly we can update them correctly).
2019-09-05 10:13:49 -07:00
Kazuki Tsujimoto 94d6ec1d90
Make pattern matching support **nil syntax 2019-09-01 16:39:34 +09:00
Aaron Patterson 932a471d38
Directly mark compile options from the AST object
`rb_ast_t` holds a reference to this object, so it should mark the
object.  Currently it is relying on the `mark_ary` on `node_buffer` to
ensure that the object stays alive.  But since the array internals can
move, this could cause a segv if compaction impacts the array.
2019-08-27 11:43:18 -07:00
Lourens Naudé 90c4bd2d2b
Let memory sizes of the various IMEMO object types be reflected correctly
[Feature #15805]

Closes: https://github.com/ruby/ruby/pull/2140
2019-07-23 16:22:34 +09:00
Martin Dürst f258137083 Fix grammar of macro name: ECCESSED -> ECCESSIVE
Fix the name of the macro variable introduced in 0872ea5330
from NODE_SPECIAL_EXCESSED_COMMA to NODE_SPECIAL_EXCESSIVE_COMMA.
2019-06-05 14:03:50 +09:00
git b31e1b4a7c * expand tabs. 2019-06-04 23:17:38 +09:00
Yusuke Endoh 0872ea5330 node.h: Avoid a magic number to represent excessed comma
`(ID)1` was assigned to NODE_ARGS#rest_arg for `{|x,| }`.
This change removes the magic number by introducing an explicit macro
variable for it: NODE_SPECIAL_EXCESSED_COMMA.
2019-06-04 23:17:18 +09:00
git b0cb4bdb4e * expand tabs. 2019-06-04 11:40:21 +09:00
Yusuke Endoh 7866ed850c node.c: Show the ID of internal variable 2019-06-04 11:39:27 +09:00
Kazuki Tsujimoto bc01f7b721
Fix description of NODE_IN 2019-04-27 12:55:32 +09:00
ktsj 243842f68a Avoid usage of the dummy empty BEGIN node
Use NODE_SPECIAL_NO_NAME_REST instead.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67629 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20 03:37:22 +00:00
yui-knk bd698312fa Fix the format of NODE_IN node
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67592 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-17 07:15:08 +00:00
ktsj 9738f96fcf Introduce pattern matching [EXPERIMENTAL]
[ruby-core:87945] [Feature #14912]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67586 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-17 06:48:03 +00:00
svn d4f7fac6b8 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67267 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-15 05:19:58 +00:00
mame c36a6f97f2 node.h: introduce nd_brace to determine if a hash literal is a keyword
NODE_HASH#nd_brace is a flag that is 1 for `foo({ k: 1 })` and 0 for
`foo(k: 1)`.
nd_alen had been abused for the flag (and the implementation is
completely the same), but an explicit name is better to read.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67266 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-15 05:19:54 +00:00