Граф коммитов

1845 Коммитов

Автор SHA1 Сообщение Дата
Adam Sitnik d4249f7e42 Disassembly diagnoser should be kept in a separate directory to avoid dependency conflicts, fixes #1059 2019-02-08 17:12:31 +01:00
Adam Sitnik 9dd9e77ddc Ctlr+C: we should kill the entire process tree, not only for the benchmarks but also for the helper processes like dotnet cli 2019-02-08 15:40:47 +01:00
Michal Strehovský 190b9be0f4 Allow reflecting on DebuggableAttribute on CoreRT (#1055)
* Allow reflecting on DebuggableAttribute on CoreRT

See https://github.com/dotnet/corert/pull/6955#discussion_r254766270.

BenchmakeDotNet reflects on this attribute here:

a76f438a81/src/BenchmarkDotNet/Extensions/AssemblyExtensions.cs (L28-L38)

* Comment
2019-02-08 07:20:48 +01:00
Adam Sitnik 20744eaa4f allow the user to choose the strategy from console line arguments 2019-02-07 11:44:40 +01:00
Adam Sitnik 41a367bd6d
update Travis Ubuntu image from 14.04 to 16.04 2019-02-07 11:05:13 +01:00
Adam Sitnik 1c431a9fb7
read the process output in a thread safe way, fixes #1051 (#1053)
* read the process output in a thread safe way, fixes #1051

* this test started to fail, most probably due to an AppVeyor change

* this test takes a LOT of time and since we have new InProcessEmitToolchain we can disable it

* disable UserCanSpecifyCustomNuGetPackageDependency which is for some reason this test is unstable on Ubuntu for both AzureDevOps and Travis CI
2019-02-07 06:44:13 +01:00
Adam Sitnik 84cc3e0dad
dotnet cli version update + reducing the number of long running tests (#1052), #1051
* update dotnet cli version hoping that it's going to help with unstable CI..

* don't set the capacity for StringBuilder hoping that it's going to help..

* remove duplicated logging for MemoryDiagnoserTests

* AccurateAllocations.Nothing duplicates EngineShouldNotInterfereAllocationResults, removing it to save CI time

* CoreRT support is verified by CoreRtIsSupported, we don;t need to run ArgumentsTests for CoreRT (each of them takes 0.5 minute)
2019-02-06 05:39:25 +01:00
Adam Sitnik 5215eeb39f Display the number of benchmarks to run, fixes #1048 2019-02-04 13:43:10 +01:00
ig-sinicyn cbdf7b41e7 Use invariant culture for csc messages (#1042)
* Use invariant culture for csc messages

* Update tests/BenchmarkDotNet.IntegrationTests/RoslynToolchainTest.cs

Co-Authored-By: ig-sinicyn <ig-sinicyn@users.noreply.github.com>

* Update tests/BenchmarkDotNet.IntegrationTests/RoslynToolchainTest.cs

Co-Authored-By: ig-sinicyn <ig-sinicyn@users.noreply.github.com>
2019-02-04 12:52:55 +01:00
Adam Sitnik 904dddc9bd tests that ensure that when a benchmark throws the runner does not throw, fixes #1046 2019-02-04 07:58:50 +01:00
Adam Sitnik 82170f86bf if global cleanup throws, we should report the problem but don't rethrow because it could eat the exception from benchmark, fixes #1045 2019-02-04 07:43:09 +01:00
Adam Sitnik da4128f252
Merge pull request #1041 from dotnet/newInProcess
InProcessEmitToolchain
2019-02-03 20:05:09 +01:00
ig-sinicyn e935b5866c minor InProcess fix: diff now checks for implementation flags, (#1043)
fix CustomTypeThatAlsoExistsInTheSystemNamespaceAsArgument test
2019-02-03 18:38:34 +01:00
Adam Sitnik 11d8cb73d8
Making the new InProcessEmitToolchain work after my recent refactor and .NET Standard 2.0 port (#1040)
* make it work after my recent refactor

* Roslyn Toolchain does not work on .NET Core becasue we don't know which assemblies to reference

* static benchmarks are not supported for out-of-proc toolchains

* use reflection to allow for assembly saving for tests purposes only

* some pedantic renaming

* remove the .dlls after the checks pass in order to avoid file in use exception

* we don't need to mimic DirtyAssemblyResolveHelper in the in-proc toolchain

* make the new InProcessEmitToolchain a default one for -i --inprocess console line arguments

* We can't use Roslyn toolchain for .NET Core because we don't know which assemblies to reference and .NET Core does not support dynamic assembly saving
2019-02-03 12:40:14 +01:00
Wojciech Nagórski f6dbe4bdcd Change the name of the global tool (#1038) 2019-02-02 04:22:06 +01:00
Wojciech Nagórski c01bc57151 Remove InternalsVisibleTo for Samples application (#1036) 2019-02-01 19:05:51 +01:00
Wojciech Nagórski aac7a69c97 Improve global tool (#1035) 2019-02-01 14:27:15 +01:00
Adam Sitnik a1df2783a1
Merge pull request #921 from ig-sinicyn/feature-inprocessemit
InProcessEmitToolchain
2019-02-01 11:58:03 +01:00
Adam Sitnik 33eaebbcc0
Merge branch 'master' into feature-inprocessemit 2019-02-01 11:26:56 +01:00
Adam Sitnik 66c2640e63 reverting things that should not introduce problems but did... 2019-02-01 09:49:59 +01:00
Adam Sitnik 3bd18fc1fd introduce ConfigOptions - an enum flag which make it easier to introduce new boolean flags to the IConfig in the future
Allow for disabling JitOptimizationsValidator on demand (fixes #988)
2019-02-01 09:34:00 +01:00
Adam Sitnik 1b9f9f579a almost no warnings ;) 2019-02-01 09:34:00 +01:00
Adam Sitnik ccee3e8a1f
Merge pull request #1033 from dotnet/tools
BenchmarkDotNet as global tool (#1006), fixes #213
2019-02-01 07:35:07 +01:00
Adam Sitnik aa1ded5b80 Merge branch 'master' into tools
# Conflicts:
#	BenchmarkDotNet.sln
2019-02-01 06:51:31 +01:00
Adam Sitnik f9c8cc5b83
Target .NET Standard 2.0 (#1032), apply a workaround for assembly binding redirect issues, fixes #942, fixes #896, fixes #895, fixes #667
* target .NET Standard 2.0 only

* don't box ;)

* use DirtyAssemblyResolveHelper in the auto-generated code because it's simply very common case that VS generates wrong redirects..

* disable two tests which fail due to CommandLineParser issue...

* bring Full .NET Framework tests back to life!

* make it possible to build BDN from source on Mac

* there is no way to do tell dotnet restore which configuration to use, we need to handle that for custom build configurations

* perform full build when user provided custom build configuration

* workaround bad assembly binding redirects on our side to keep the users happy ;)
2019-02-01 06:44:15 +01:00
Adam Sitnik f10752ace1
Duplicates handling for IConfig = big refactor, fixes #360, closes #464 and fixes #938 (#912)
* Duplicates handling for IConfig = big refactor, fixes #360

* handle the case when IExporterDependencies were added by user in a wrong order

* every benchmark case has it's own Config, try to avoid global config which can be incomplete in some scenarios

* print the test output to the console

* refactor

* update dependencies
2019-02-01 05:40:08 +01:00
Adam Sitnik 2aec75f0e9 remove CustomCoreClrToolchain, it was causing too much trouble. We can run the benchmrks with --corerun or --cli and keep it simple, fixes #928 2019-01-30 04:39:48 +01:00
Adam Sitnik 20a0110e06 C# keywords are prohibited for benchmark names, print nice error message, fixes #849 2019-01-29 21:49:27 +01:00
Adam Sitnik 935ead0dd0 use .NET 4.6.1 everywhere, we don't support 4.6 anymore. Cleanup after #992 2019-01-29 21:09:31 +01:00
Adam Sitnik c69934f50b Give a warning when the [Benchmark] method is static: handle some edge-cases, closes #983 2019-01-29 20:53:05 +01:00
Adam Sitnik ef090df489 if we fail to do the full build, we try with --no-dependencies (for the publishing toolchains like CoreRT) 2019-01-22 09:57:54 -08:00
Wojciech Nagórski 6ccf453aa1 Improve diff disassembly (#1022)
Currently when you generate diff two disassembly, the result file looks like:

## BenchmarkDotNet.Samples.IntroDisassemblyCode
**Diff for LoweringTESTtoBT method between:**
.NET Core 2.0.9 (CoreCLR 4.6.26614.01, CoreFX 4.6.26614.01), 64bit RyuJIT
.NET Core 2.1.6 (CoreCLR 4.6.27019.06, CoreFX 4.6.27019.05), 64bit RyuJIT
```diff
        xor     edx,edx
        jmp     M00_L01
 M00_L00:
-       mov     ecx,eax
-       and     ecx,1Fh
-       mov     edx,1
-       shl     edx,cl
-       mov     ecx,dword ptr [rsp+4]
-       test    edx,ecx
+       bt      ecx,eax
        mov     edx,ecx
-       jne     M00_L01
+       jb      M00_L01
        inc     eax
 M00_L01:
        lea     ecx,[rdx+1]
-       mov     dword ptr [rsp+4],ecx
        cmp     edx,989680h
        jl      M00_L00
-       add     rsp,8
-; Total bytes of code 49
+       ret
+; Total bytes of code 27
 


```

After changes: 
## BenchmarkDotNet.Samples.IntroDisassemblyCode
**Diff for LoweringTESTtoBT method between:**
.NET Core 2.0.9 (CoreCLR 4.6.26614.01, CoreFX 4.6.26614.01), 64bit RyuJIT
.NET Core 2.1.6 (CoreCLR 4.6.27019.06, CoreFX 4.6.27019.05), 64bit RyuJIT
```diff
; BenchmarkDotNet.Samples.IntroDisassemblyCode.LoweringTESTtoBT()
        xor     eax,eax
        xor     edx,edx
        jmp     M00_L01
 M00_L00:
-       mov     ecx,eax
-       and     ecx,1Fh
-       mov     edx,1
-       shl     edx,cl
-       mov     ecx,dword ptr [rsp+4]
-       test    edx,ecx
+       bt      ecx,eax
        mov     edx,ecx
-       jne     M00_L01
+       jb      M00_L01
        inc     eax
 M00_L01:
        lea     ecx,[rdx+1]
-       mov     dword ptr [rsp+4],ecx
        cmp     edx,989680h
        jl      M00_L00
-       add     rsp,8
-; Total bytes of code 49
+       ret
+; Total bytes of code 27
```
2019-01-21 20:13:35 +03:00
Adam Sitnik 1577ba1ffe MonoAotToolchain refactoring, post #940 2019-01-20 10:28:41 -08:00
ig-sinicyn fc530fc568 Fix build after rebase 2019-01-20 14:31:59 +03:00
ig-sinicyn 351ca51f27 Ignore NOPs for ldarg too 2019-01-20 14:28:19 +03:00
ig-sinicyn aa9ff82e41 + emit correct IL 2019-01-20 14:28:19 +03:00
ig-sinicyn bf362a8810 Diff now compares nops 2019-01-20 14:28:18 +03:00
ig-sinicyn 57acd6a4a4 No NOPs (thanks to @Warpten!) 2019-01-20 14:28:18 +03:00
ig-sinicyn 7ec2f3f3c4 Check if there's something wrong with RoslynToolchain on .Net Core 2019-01-20 14:28:18 +03:00
ig-sinicyn 252d7a700a Fix StringCanBePassedToBenchmarkAsReadOnlySpan 2019-01-20 14:28:18 +03:00
ig-sinicyn 862e6e3907 InProcessEmitToolchain cleanup 2019-01-20 14:28:18 +03:00
ig-sinicyn 011c794f00 + InProcessEmitToolchain 2019-01-20 14:28:17 +03:00
Alexander Kyte 573566b70f Add support for mono AOT pass (#940)
* Add support for mono AOT pass

* Add documentation for using mono aot args
2019-01-17 10:43:58 -08:00
Andrey Akinshin 8d971412a8
Support machines without .NET DevPack, fix #1018 2019-01-17 11:35:07 +03:00
Adam Sitnik 6ee21bb635 if we fail to do the full build, we try with --no-dependencies 2019-01-15 17:03:34 -08:00
Glenn Watson 5c519fb19c Use .NET Standard 2.0 CommandLineParser, Update to net461 for NS2.0 support (#992)
* Use .NET Standard 2.0 CommandLineParser, Update to net461 for NS2.0 support

* change to the newly released CommandLineParser 2.4.3 with NS 2.0 support
2019-01-13 16:10:56 +01:00
Adam Sitnik 5e44289541 make SortedValues internal property, don't export it! 2019-01-13 15:09:53 +01:00
Andrey Akinshin 7ef5f659ea
Introduce StoppingCriteria (#984) 2019-01-13 14:52:45 +03:00
Code Therapist 923b23cbd3 BenchmarkDotNet as global tool (#1006), fixes #213
* BenchmarkDotNet as global tool

This resolves #213.

* Set TargetFramework to "netcoreapp2.1"

* Add import for common build properties.

* Docs: BenchmarkDotNet as global dotnet tool
2019-01-12 18:30:33 +01:00
Wojciech Nagórski 2869966129 Improve restore, build and publish projects - Fix for #1002 (#1013)
* Comment dotnet publish with --no-build

* Fix restore, build and publish projects

* Remove LaunchSettings.json
2019-01-11 15:41:24 +01:00