Граф коммитов

18 Коммитов

Автор SHA1 Сообщение Дата
Ryuta Kamizono 9d24ddbb53 Fix `make benchmark` example
`make benchmark ARGS=../benchmark/erb_render.yml` does not work.

```
% make benchmark ARGS=../benchmark/erb_render.yml
/Users/kamipo/.rbenv/shims/ruby --disable=gems -rrubygems -I./benchmark/lib ./benchmark/benchmark-driver/exe/benchmark-driver \
	            --executables="compare-ruby::/Users/kamipo/.rbenv/shims/ruby --disable=gems -I.ext/common --disable-gem" \
	            --executables="built-ruby::./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems --disable-gem" \
	            ../benchmark/erb_render.yml 
Traceback (most recent call last):
	6: from ./benchmark/benchmark-driver/exe/benchmark-driver:112:in `<main>'
	5: from ./benchmark/benchmark-driver/exe/benchmark-driver:112:in `flat_map'
	4: from ./benchmark/benchmark-driver/exe/benchmark-driver:112:in `each'
	3: from ./benchmark/benchmark-driver/exe/benchmark-driver:122:in `block in <main>'
	2: from /Users/kamipo/.rbenv/versions/2.6.6/lib/ruby/2.6.0/psych.rb:577:in `load_file'
	1: from /Users/kamipo/.rbenv/versions/2.6.6/lib/ruby/2.6.0/psych.rb:577:in `open'
/Users/kamipo/.rbenv/versions/2.6.6/lib/ruby/2.6.0/psych.rb:577:in `initialize': No such file or directory @ rb_sysopen - ../benchmark/erb_render.yml (Errno::ENOENT)
make: *** [benchmark] Error 1

% make benchmark ARGS=benchmark/erb_render.yml
/Users/kamipo/.rbenv/shims/ruby --disable=gems -rrubygems -I./benchmark/lib ./benchmark/benchmark-driver/exe/benchmark-driver \
	            --executables="compare-ruby::/Users/kamipo/.rbenv/shims/ruby --disable=gems -I.ext/common --disable-gem" \
	            --executables="built-ruby::./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems --disable-gem" \
	            benchmark/erb_render.yml 
Calculating -------------------------------------
                     compare-ruby  built-ruby 
          erb_render     825.454k    783.664k i/s -      1.500M times in 1.817181s 1.914086s

Comparison:
                       erb_render
        compare-ruby:    825454.4 i/s 
          built-ruby:    783663.8 i/s - 1.05x  slower

```
2020-06-07 10:33:14 +09:00
Takashi Kokubun f883d6219e
Unify vm benchmark prefixes to vm_ (#3028)
The vm1_ prefix and vm2_ had had special meaning until
820ad9cb1d and
12068aa4e9. AFAIK there's no special
meaning in vm3_ prefix.

As they have confused people (like "In `benchmark` what is difference
between `vm1_`, `vm2_` and `vm3_`"), I'd like to remove the obsoleted
prefix as we obviated that two years ago.
2020-04-13 21:37:42 -07:00
k0kubun 706d266496 benchmark/README.md: skip showing --timeout option [ci skip]
because it's only available for limited platforms for now.
I'll make it portable and show it later.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63954 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-11 22:49:48 +00:00
k0kubun 3e93bf3b46 common.mk: upgrade benchmark_driver
benchmark/README.md: fix help output, which is changed on v0.14.6.
Especially `e1::path1,arg1,...; e2::path2,arg2` part was wrong since `,`
can't be used to split arguments anymore.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63945 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-11 00:34:32 +00:00
k0kubun 1053c3d4bf benchmark/README.md: some are defined with YAML [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63944 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 23:56:57 +00:00
k0kubun fdd7f57b97 benchmark/README.md: fix typo [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63942 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 16:03:51 +00:00
k0kubun 3efcb74036 benchmark: revise ERB benchmarks using YAML syntax
to improve the accuracy of measurement by stop using block.

benchmark/app_erb.rb -> benchmark/app_erb.yml: renamed and revised
benchmark/erb_render.rb -> benchmark/erb_render.yml: ditto

benchmark/README.md: follow renames

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63941 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 15:58:52 +00:00
k0kubun 498487ec2a benchmark/README.md: erb_render is no longer yml but rb [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63940 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 15:51:29 +00:00
k0kubun 301cb9ea7c benchmark/README.md: wording consistency in one sentence [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63939 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 15:49:42 +00:00
k0kubun f328fe40b3 benchmark_driver/runner: add runners for metrics
supported by legacy benchmark/driver.rb.

benchmark/README.md: document them

common.mk: update benchmark_driver to correct 0.0 output and to fix
spacing format of `-o simple` and `-o markdown`.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63933 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 15:01:27 +00:00
k0kubun 34eed0cd47 benchmark/README.md: add help output [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63931 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 13:41:13 +00:00
k0kubun 59ec265e33 benchmark/README.md: don't recommend alias installation [ci skip]
benchmark_driver is the official way to install benchmark_driver.
benchmark-driver is just an alias for it.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63930 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 13:36:44 +00:00
k0kubun 8068d34cfe Revert "benchmark/*.yml: convert from benchmark/bm_*.rb"
This reverts r63900.

Having single-execution benchmark as a normal Ruby script is preferred
by ko1. I'm not a big fan of having inconsistent benchmark formats, but
I can understand some benefits of it.

common.mk: remove obsolsted benchmark-each PHONY declaration, support
running Ruby scripts added by this commit.

README.md: follow ARGS change

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63926 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 13:05:29 +00:00
k0kubun 12a38cbac0 benchmark: resurrect peak / size metrics
by adding runner plugins for them.

benchmark/lib/benchmark_driver/runner/peak.rb: added peak runner plugin
benchmark/lib/benchmark_driver/runner/size.rb: added size runner plugin
common.mk: allow using them

benchmark/memory_wrapper.rb: deleted in favor of those runner plugins

benchmark/README.md: document them

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63924 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-10 12:14:04 +00:00
k0kubun 443773836e benchmark/README.md: fix -e interface for v0.14 [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63919 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-09 16:43:42 +00:00
k0kubun 7bed459a26 benchmark/driver.rb: fully obsolete this
in favor of just using benchmark_driver.gem.

common.mk: The new `make benchmark` covers the both usages for old `make
benchmark` and old `make benchmark-each`. So `make benchmark-each` is
dropped now.

benchmark/README.md: Explain its details

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63918 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-09 16:31:35 +00:00
k0kubun 2cab2fc8b1 benchmark/README.md: [ci skip] change syntax highlight
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63905 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08 18:10:14 +00:00
k0kubun 8b0158671b benchmark/README.md: [ci skip] write documentation
about this directory

benchmark/driver.rb: ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63903 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-08 18:08:26 +00:00