dotnet
/
performance
зеркало из https://github.com/dotnet/performance.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Update markdown for 6.0 (#1713)

This commit is contained in:
Dan Moseley 2021-03-07 10:44:13 -08:00 коммит произвёл GitHub
Родитель e2a9af60b3
Коммит f5747259c7
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
7 изменённых файлов: 83 добавлений и 83 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

36
docs/basic-scenarios.md
Просмотреть файл

@ -17,9 +17,9 @@ Same instruction of [Scenario Tests Guide - Step 1](./scenarios-workflow.md#step
For **Self-Contained Empty Console App Size On Disk** scenario, run precommand to create an empty console template and publish it:
```
cd emptyconsoletemplate
python3 pre.py publish -f net5.0 -c Release -r win-x64
python3 pre.py publish -f net6.0 -c Release -r win-x64
```
`-f net5.0` sets the new template project targeting `net5.0` framework; `-c Release` configures the publish to be in release; `-r win-x64` takes an [RID](https://docs.microsoft.com/en-us/dotnet/core/rid-catalog)(Runtime Identifier) and specifies which runtime it supports.
`-f net6.0` sets the new template project targeting `net6.0` framework; `-c Release` configures the publish to be in release; `-r win-x64` takes an [RID](https://docs.microsoft.com/en-us/dotnet/core/rid-catalog)(Runtime Identifier) and specifies which runtime it supports.
**Note that by specifying RID option `-r <RID>`, it defaults to publish the app into a [SCD](https://docs.microsoft.com/en-us/dotnet/core/deploying/#publish-self-contained)(Self-contained Deployment) app; without it, a [FDD](https://docs.microsoft.com/en-us/dotnet/core/deploying/#publish-framework-dependent)(Framework Dependent Deployment) app will be published.**
@ -55,30 +55,30 @@ Same instruction of [Scenario Tests Guide - Step 4](./scenarios-workflow.md#step
## Command Matrix
- \<tfm> values:
- netcoreapp2.1
- netcoreapp3.0
- netcoreapp3.1
- net5.0
- net6.0
- \<-r RID> values:
- ""(WITHOUT `-r <RID>` --> FDD app)
- `"-r <RID>"` (WITH `-r` --> SCD app, [list of RID](https://docs.microsoft.com/en-us/dotnet/core/rid-catalog))
| Scenario | Asset Directory | Precommand | Testcommand | Postcommand | Supported Framework | Supported Platform |
|-----------------------------------------------|-------------------------|-----------------------------------------------|-----------------|-------------|--------------------------------------------------|--------------------|
| Static Console Template Publish Startup | staticconsoletemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows |
| Static Console Template Publish SizeOnDisk | staticconsoletemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| Static Console Template Build SizeOnDisk | staticconsoletemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| Static VB Console Template Publish Startup | staticvbconsoletemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows |
| Static VB Console Template Publish SizeOnDisk | staticvbconsoletemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| Static VB Console Template Build SizeOnDisk | staticvbconsoletemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| Static Winforms Template Publish Startup | staticwinformstemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1 | Windows |
| Static Winforms Template Publish SizeOnDisk | staticwinformstemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1 | Windows;Linux |
| Static Winforms Template Build SizeOnDisk | staticwinformstemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1 | Windows;Linux |
| New Console Template Publish Startup | emptyconsoletemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows |
| New Console Template Publish SizeOnDisk | emptyconsoletemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| New Console Template Build SizeOnDisk | emptyconsoletemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| New VB Console Template Publish Startup | emptyvbconsoletemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows |
| New VB Console Template Publish SizeOnDisk | emptyvbconsoletemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| New VB Console Template Build SizeOnDisk | emptyvbconsoletemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| Static Console Template Publish Startup | staticconsoletemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows |
| Static Console Template Publish SizeOnDisk | staticconsoletemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| Static Console Template Build SizeOnDisk | staticconsoletemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| Static VB Console Template Publish Startup | staticvbconsoletemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows |
| Static VB Console Template Publish SizeOnDisk | staticvbconsoletemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| Static VB Console Template Build SizeOnDisk | staticvbconsoletemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| Static Winforms Template Publish Startup | staticwinformstemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.1 | Windows |
| Static Winforms Template Publish SizeOnDisk | staticwinformstemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1 | Windows;Linux |
| Static Winforms Template Build SizeOnDisk | staticwinformstemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1 | Windows;Linux |
| New Console Template Publish Startup | emptyconsoletemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows |
| New Console Template Publish SizeOnDisk | emptyconsoletemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| New Console Template Build SizeOnDisk | emptyconsoletemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| New VB Console Template Publish Startup | emptyvbconsoletemplate | pre.py publish -f /<tfm> -c Release | test.py startup | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows |
| New VB Console Template Publish SizeOnDisk | emptyvbconsoletemplate | pre.py publish -f /<tfm> -c Release /<-r RID> | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| New VB Console Template Build SizeOnDisk | emptyvbconsoletemplate | pre.py build -f /<tfm> -c Release | test.py sod | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
## Relevant Links
- [SCD App](https://docs.microsoft.com/en-us/dotnet/core/deploying/#publish-self-contained)
- [FDD App](https://docs.microsoft.com/en-us/dotnet/core/deploying/#publish-framework-dependent)

50
docs/benchmarkdotnet.md
Просмотреть файл

@ -59,7 +59,7 @@ In order to build or run the benchmarks you will need the **.NET Core command-li
### Using .NET Cli
To build the benchmarks you need to have the right `dotnet cli`. This repository allows to benchmark .NET Core 2.1, 3.1 and 5.0 so you need to install all of them.
To build the benchmarks you need to have the right `dotnet cli`. This repository allows to benchmark .NET Core 2.1, 3.1, 5.0 and 6.0 so you need to install all of them.
All you need to do is run the following command:
@ -70,8 +70,8 @@ dotnet build -c Release
If you don't want to install all of them and just run the benchmarks for selected runtime(s), you need to manually edit the [MicroBenchmarks.csproj](../src/benchmarks/micro/MicroBenchmarks.csproj) file.
```diff
-<TargetFrameworks>netcoreapp2.1;netcoreapp3.1;net5.0</TargetFrameworks>
+<TargetFrameworks>net5.0</TargetFrameworks>
-<TargetFrameworks>netcoreapp2.1;netcoreapp3.1;net5.0;net6.0</TargetFrameworks>
+<TargetFrameworks>net6.0</TargetFrameworks>
```
The alternative is to set `PERFLAB_TARGET_FRAMEWORKS` environment variable to selected Target Framework Moniker.
@ -81,7 +81,7 @@ The alternative is to set `PERFLAB_TARGET_FRAMEWORKS` environment variable to se
If you don't want to install `dotnet cli` manually, we have a Python 3 script which can do that for you. All you need to do is to provide the frameworks:
```cmd
py .\scripts\benchmarks_ci.py --frameworks netcoreapp3.1
py .\scripts\benchmarks_ci.py --frameworks net6.0
```
## Running the Benchmarks
@ -91,7 +91,7 @@ py .\scripts\benchmarks_ci.py --frameworks netcoreapp3.1
To run the benchmarks in interactive mode you have to execute `dotnet run -c Release -f $targetFrameworkMoniker` in the folder with benchmarks project.
```cmd
C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f netcoreapp3.1
C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net6.0
Available Benchmarks:
#0 Burgers
#1 ByteMark
@ -117,42 +117,42 @@ And select one of the benchmarks from the list by either entering its number or
You can filter the benchmarks using `--filter $globPattern` console line argument. The filter is **case insensitive**.
The glob patterns are applied to full benchmark name: namespace.typeName.methodName. Examples:
The glob patterns are applied to full benchmark name: namespace.typeName.methodName. Examples (all in the `src\benchmarks\micro` folder):
- Run all the benchmarks from BenchmarksGame namespace:
```cmd
dotnet run -c Release -f netcoreapp3.1 --filter BenchmarksGame*
dotnet run -c Release -f net6.0 --filter BenchmarksGame*
```
- Run all the benchmarks with type name Richards:
```cmd
dotnet run -c Release -f netcoreapp3.1 --filter *.Richards.*
dotnet run -c Release -f net6.0 --filter *.Richards.*
```
- Run all the benchmarks with method name ToStream:
```cmd
dotnet run -c Release -f netcoreapp3.1 --filter *.ToStream
dotnet run -c Release -f net6.0 --filter *.ToStream
```
- Run ALL benchmarks:
```cmd
dotnet run -c Release -f netcoreapp3.1 --filter *
dotnet run -c Release -f net6.0 --filter *
```
- You can provide many filters (logical disjunction):
```cmd
dotnet run -c Release -f netcoreapp3.1 --filter System.Collections*.Dictionary* *.Perf_Dictionary.*
dotnet run -c Release -f net6.0 --filter System.Collections*.Dictionary* *.Perf_Dictionary.*
```
- To print a **joined summary** for all of the benchmarks (by default printed per type), use `--join`:
```cmd
dotnet run -c Release -f netcoreapp2.1 --filter BenchmarksGame* --join
dotnet run -c Release -f net6.0 --filter BenchmarksGame* --join
```
Please remember that on **Unix** systems `*` is resolved to all files in current directory, so you need to escape it `'*'`.
@ -161,10 +161,10 @@ Please remember that on **Unix** systems `*` is resolved to all files in current
To print the list of all available benchmarks you need to pass `--list [tree/flat]` argument. It can also be combined with `--filter` option.
Example: Show the tree of all the benchmarks from System.Threading namespace that can be run for .NET Core 2.0:
Example: Show the tree of all the benchmarks from System.Threading namespace that can be run for .NET 6.0:
```cmd
dotnet run -c Release -f netcoreapp2.1 --list tree --filter System.Threading*
dotnet run -c Release -f net6.0 --list tree --filter System.Threading*
```
```log
@ -259,7 +259,7 @@ If you want to disassemble the benchmarked code, you need to use the [Disassembl
You can do that by passing `--disassm` to the app or by using `[DisassemblyDiagnoser(printAsm: true, printSource: true)]` attribute or by adding it to your config with `config.With(DisassemblyDiagnoser.Create(new DisassemblyDiagnoserConfig(printAsm: true, recursiveDepth: 1))`.
Example: `dotnet run -c Release -f netcoreapp2.1 -- --filter System.Memory.Span<Int32>.Reverse -d`
Example: `dotnet run -c Release -f net6.0 -- --filter System.Memory.Span<Int32>.Reverse -d`
```assembly
; System.Runtime.InteropServices.MemoryMarshal.GetReference[[System.Byte, System.Private.CoreLib]](System.Span`1<Byte>)
@ -285,30 +285,30 @@ M00_L00:
The `--runtimes` or just `-r` allows you to run the benchmarks for **multiple Runtimes**.
Available options are: Mono, CoreRT, net461, net462, net47, net471, net472, netcoreapp2.1, netcoreapp3.0, netcoreapp3.1 and net5.0.
Available options are: Mono, CoreRT, net461, net462, net47, net471, net472, netcoreapp2.1, netcoreapp3.1, net5.0 and net6.0.
Example: run the benchmarks for .NET Core 3.1 and 5.0:
Example: run the benchmarks for .NET 5.0 and 6.0:
```cmd
dotnet run -c Release -f netcoreapp3.1 --runtimes netcoreapp3.1 net5.0
dotnet run -c Release -f net5.0 --runtimes net5.0 net6.0
```
**Important: The host process needs to be the lowest common API denominator of the runtimes you want to compare!** In this case, it was`netcoreapp3.1`.
**Important: The host process needs to be the lowest common API denominator of the runtimes you want to compare!** In this case, it was `net5.0`.
## Regressions
To perform a Mann–Whitney U Test and display the results in a dedicated column you need to provide the Threshold for Statistical Test via `--statisticalTest` argument. The value can be relative (5%) or absolute (10ms, 100ns, 1s)
Example: run Mann–Whitney U test with relative ratio of 5% for `BinaryTrees_2` for .NET Core 3.1 (base) vs .NET Core 5.0 (diff). .NET Core 3.1 will be baseline because it was first.
Example: run Mann–Whitney U test with relative ratio of 5% for `BinaryTrees_2` for .NET 5.0 (base) vs .NET 6.0 (diff). .NET Core 5.0 will be baseline because it was first.
```cmd
dotnet run -c Release -f netcoreapp3.1 --filter *BinaryTrees_2* --runtimes netcoreapp3.1 net5.0 --statisticalTest 5%
dotnet run -c Release -f net6.0 --filter *BinaryTrees_2* --runtimes net5.0 net6.0 --statisticalTest 5%
```
| Method | Toolchain | Mean | MannWhitney(5%) |
|-------------- |-------------- |---------:|---------------- |
| BinaryTrees_2 | netcoreapp3.1 | 124.4 ms | Base |
| BinaryTrees_2 | net5.0 | 153.7 ms | Slower |
| BinaryTrees_2 | net5.0 | 124.4 ms | Base |
| BinaryTrees_2 | net6.0 | 153.7 ms | Slower |
**Note:** to compare the historical results you need to use [Results Comparer](../src/tools/ResultsComparer/README.md)
@ -329,7 +329,7 @@ Please use this option only when you are sure that the benchmarks you want to ru
It's possible to benchmark private builds of [dotnet/runtime](https://github.com/dotnet/runtime) using CoreRun.
```cmd
dotnet run -c Release -f netcoreapp3.1 --coreRun $thePath
dotnet run -c Release -f net6.0 --coreRun $thePath
```
**Note:** You can provide more than 1 path to CoreRun. In such case, the first path will be the baseline and all the benchmarks are going to be executed for all CoreRuns you have specified.
@ -355,7 +355,7 @@ public void PrintInfo()
You can also use any dotnet cli to build and run the benchmarks.
```cmd
dotnet run -c Release -f netcoreapp3.1 --cli "C:\Projects\performance\.dotnet\dotnet.exe"
dotnet run -c Release -f net6.0 --cli "C:\Projects\performance\.dotnet\dotnet.exe"
```
This is very useful when you want to compare different builds of .NET Core SDK.

24
docs/benchmarking-workflow-dotnet-runtime.md
Просмотреть файл

@ -95,7 +95,7 @@ During the port from xunit-performance to BenchmarkDotNet, the namespaces, type
Please remember that you can filter the benchmarks using a glob pattern applied to namespace.typeName.methodName ([read more](./benchmarkdotnet.md#Filtering-the-Benchmarks)):
```cmd
dotnet run -c Release -f net5.0 --filter System.Memory*
dotnet run -c Release -f net6.0 --filter System.Memory*
```
(Run the above command on `src/benchmarks/micro/MicroBenchmarks.csproj`.)
@ -115,8 +115,8 @@ C:\Projects\runtime> build -c Release
Every time you want to run the benchmarks against local build of [dotnet/runtime](https://github.com/dotnet/runtime) you need to provide the path to CoreRun:
```cmd
dotnet run -c Release -f net5.0 --filter $someFilter \
--coreRun C:\Projects\runtime\artifacts\bin\testhost\net5.0-Windows_NT-Release-x64\shared\Microsoft.NETCore.App\5.0.0\CoreRun.exe
dotnet run -c Release -f net6.0 --filter $someFilter \
--coreRun C:\Projects\runtime\artifacts\bin\testhost\net6.0-windows-Release-x64\shared\Microsoft.NETCore.App\6.0.0\CoreRun.exe
```
**Note:** BenchmarkDotNet expects a path to `CoreRun.exe` file (`corerun` on Unix), not to `Core_Root` folder.
@ -130,7 +130,7 @@ C:\Projects\runtime\src\libraries\System.Text.RegularExpressions\src> dotnet msb
**Note:** the exception to this rule are libraries that **are not part of the shared SDK**. The `build` script of the runtime repo does not copy them to the CoreRun folder so you need to do it on your own:
```cmd
cp artifacts\bin\runtime\net6.0-Windows_NT-Release-x64\Microsoft.Extensions.Caching.Memory.dll artifacts\bin\testhost\net6.0-Windows_NT-Release-x64\shared\Microsoft.NETCore.App\6.0.0\
cp artifacts\bin\runtime\net6.0-Windows_NT-Release-x64\Microsoft.Extensions.Caching.Memory.dll artifacts\bin\testhost\net6.0-windows-Release-x64\shared\Microsoft.NETCore.App\6.0.0\
```
Of course only if you want to benchmark these specific libraries. If you don't, the default versions defined in [MicroBenchmarks.csproj](../src/benchmarks/micro/MicroBenchmarks.csproj) project file are going to get used.
@ -142,9 +142,9 @@ Preventing regressions is a fundamental part of our performance culture. The che
**Before introducing any changes that may impact performance**, you should run the benchmarks that test the performance of the feature that you are going to work on and store the results in a **dedicated** folder.
```cmd
C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net5.0 \
C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net6.0 \
--artifacts "C:\results\before" \
--coreRun "C:\Projects\runtime\artifacts\bin\testhost\net5.0-Windows_NT-Release-x64\shared\Microsoft.NETCore.App\5.0.0\CoreRun.exe" \
--coreRun "C:\Projects\runtime\artifacts\bin\testhost\net6.0-windows-Release-x64\shared\Microsoft.NETCore.App\6.0.0\CoreRun.exe" \
--filter System.IO.Pipes*
```
@ -157,9 +157,9 @@ After you introduce the changes and rebuild the part of [dotnet/runtime](https:/
```cmd
C:\Projects\runtime\src\libraries\System.IO.Pipes\src> dotnet msbuild /p:Configuration=Release
C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net5.0 \
C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net6.0 \
--artifacts "C:\results\after" \
--coreRun "C:\Projects\runtime\artifacts\bin\testhost\net5.0-Windows_NT-Release-x64\shared\Microsoft.NETCore.App\5.0.0\CoreRun.exe" \
--coreRun "C:\Projects\runtime\artifacts\bin\testhost\net6.0-windows-Release-x64\shared\Microsoft.NETCore.App\6.0.0\CoreRun.exe" \
--filter System.IO.Pipes*
```
@ -184,7 +184,7 @@ No Slower results for the provided threshold = 2% and noise filter = 0.3ns.
To run the benchmarks against the latest .NET Core SDK you can use the [benchmarks_ci.py](../scripts/benchmarks_ci.py) script. It's going to download the latest .NET Core SDK(s) for the provided framework(s) and run the benchmarks for you. Please see [Prerequisites](./prerequisites.md#python) for more.
```cmd
C:\Projects\performance> py scripts\benchmarks_ci.py -f net5.0 \
C:\Projects\performance> py scripts\benchmarks_ci.py -f net6.0 \
--bdn-arguments="--artifacts "C:\results\latest_sdk"" \
--filter System.IO.Pipes*
```
@ -205,7 +205,7 @@ The real performance investigation starts with profiling. We have a comprehensiv
To profile the benchmarked code and produce an ETW Trace file ([read more](./benchmarkdotnet.md#Profiling)):
```cmd
dotnet run -c Release -f net5.0 --profiler ETW --filter $YourFilter
dotnet run -c Release -f net6.0 --profiler ETW --filter $YourFilter
```
The benchmarking tool is going to print the path to the `.etl` trace file. You should open it with PerfView or Windows Performance Analyzer and start the analysis from there. If you are not familiar with PerfView, you should watch [PerfView Tutorial](https://channel9.msdn.com/Series/PerfView-Tutorial) by @vancem first. It's an investment that is going to pay off very quickly.
@ -222,7 +222,7 @@ If profiling using the `--profiler ETW` is not enough, you should use a differen
BenchmarkDotNet has some extra features that might be useful when doing performance investigation:
- You can run the benchmarks against [multiple Runtimes](./benchmarkdotnet.md#Multiple-Runtimes). It can be very useful when the regression has been introduced between .NET Core releases, for example: between netcoreapp3.1 and net5.0.
- You can run the benchmarks against [multiple Runtimes](./benchmarkdotnet.md#Multiple-Runtimes). It can be very useful when the regression has been introduced between .NET Core releases, for example: between net5.0 and net6.0.
- You can run the benchmarks using provided [dotnet cli](./benchmarkdotnet.md#dotnet-cli). You can download few dotnet SDKs, unzip them and just run the benchmarks to spot the version that has introduced the regression to narrow down your investigation.
- You can run the benchmarks using few [CoreRuns](./benchmarkdotnet.md#CoreRun). You can build the latest [dotnet/runtime](https://github.com/dotnet/runtime) in Release, create a copy of the folder with CoreRun and use git to checkout an older commit. Then rebuild [dotnet/runtime](https://github.com/dotnet/runtime) and run the benchmarks against the old and new builds. This can narrow down your investigation to the commit that has introduced the bug.
@ -274,7 +274,7 @@ Because the benchmarks are not in the [dotnet/runtime](https://github.com/dotnet
The first thing you need to do is send a PR with the new API to the [dotnet/runtime](https://github.com/dotnet/runtime) repository. Once your PR gets merged and a new NuGet package is published to the [dotnet/runtime](https://github.com/dotnet/runtime) NuGet feed, you should remove the Reference to a `.dll` and install/update the package consumed by [MicroBenchmarks](../src/benchmarks/micro/MicroBenchmarks.csproj). You can do this by running the following script locally:
```cmd
/home/adsitnik/projects/performance>python3 ./scripts/benchmarks_ci.py --filter $YourFilter -f net5.0
/home/adsitnik/projects/performance>python3 ./scripts/benchmarks_ci.py --filter $YourFilter -f net6.0
```
This script will try to pull the latest .NET Core SDK from [dotnet/runtime](https://github.com/dotnet/runtime) nightly build, which should contain the new API that you just merged in your first PR, and use that to build MicroBenchmarks project and then run the benchmarks that satisfy the filter you provided.

2
docs/blazor-scenarios.md
Просмотреть файл

@ -63,7 +63,7 @@ Same instruction of [Step 4 in Scenario Tests Guide](scenarios-workflow.md#step-
For the purpose of quick reference, the commands can be summarized into the following matrix:
| Scenario | Asset Directory | Precommand | Testcommand | Postcommand | Supported Framework | Supported Platform |
|-------------------------------------|-----------------|-------------------------------------------------------------|-------------------------------------------------------------------|-------------|---------------------|--------------------|
| SOD - New Blazor Template - Publish | blazor | pre.py publish --msbuild "/p:_TrimmerDumpDependencies=true" | test.py sod --scenario-name "SOD - New Blazor Template - Publish" | post.py | net5.0 | Windows;Linux |
| SOD - New Blazor Template - Publish | blazor | pre.py publish --msbuild "/p:_TrimmerDumpDependencies=true" | test.py sod --scenario-name "SOD - New Blazor Template - Publish" | post.py | net6.0 | Windows;Linux |
## Relevant Links

8
docs/profiling-workflow-dotnet-runtime.md
Просмотреть файл

@ -64,13 +64,13 @@ The build produces two things that we care about:
* `dotnet` and all `System.XYZ.dlls` used internally to run Libraries unit tests. It can be used by Visual Studio Profiler to run the code that you want to profile. Example:
```log
C:\Projects\runtime\artifacts\bin\testhost\net5.0-Windows_NT-Release-x64\dotnet.exe
C:\Projects\runtime\artifacts\bin\testhost\net6.0-windows-Release-x64\dotnet.exe
```
* `CoreRun` and all `System.XYZ.dlls` that can be used to run the code that you want to profile. Example:
```log
C:\Projects\runtime\artifacts\bin\testhost\net5.0-Windows_NT-Release-x64\shared\Microsoft.NETCore.App\5.0.0\CoreRun.exe
C:\Projects\runtime\artifacts\bin\testhost\net6.0-windows-Release-x64\shared\Microsoft.NETCore.App\6.0.0\CoreRun.exe
```
Once you rebuild the part of [dotnet/runtime](https://github.com/dotnet/runtime) you are working on, the appropriate `.dll` gets updated and the next time you run profiler, dotnet|CoreRun is going to use the updated library.
@ -131,7 +131,7 @@ It's recommended to disable Tiered JIT (to avoid the need of warmup) and emit fu
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net5.0</TargetFramework>
<TargetFramework>net6.0</TargetFramework>
<DebugType>pdbonly</DebugType>
<DebugSymbols>true</DebugSymbols>
@ -175,7 +175,7 @@ start %sln%
You can just save it as `startvs.cmd` file and run providing path to the `testhost` folder produced by [dotnet/runtime](https://github.com/dotnet/runtime) build and a VS solution with repo project:
```cmd
startvs.cmd "C:\Projects\runtime\artifacts\bin\testhost\net5.0-Windows_NT-Release-x64\" "C:\Projects\repro\ProfilingDocs.sln"
startvs.cmd "C:\Projects\runtime\artifacts\bin\testhost\net6.0-windows-Release-x64\" "C:\Projects\repro\ProfilingDocs.sln"
```
## CPU Usage

4
docs/scenarios-workflow.md
Просмотреть файл

@ -55,9 +55,9 @@ python3 pre.py <command> <options> # run precommand
In our **startup time of an empty console template** example, we can run
```
cd emptyconsoletemplate
python3 pre.py publish -f net5.0 -c Release
python3 pre.py publish -f net6.0 -c Release
```
The above command creates a new dotnet console template in `emptyconsoletemplate\app\` folder, builds the project targeting net5.0 in Release and publishs it to `emptyconsoletemplate\pub\` folder.
The above command creates a new dotnet console template in `emptyconsoletemplate\app\` folder, builds the project targeting net6.0 in Release and publishs it to `emptyconsoletemplate\pub\` folder.
Run `python3 pre.py --help` for more command options and their meanings.

42
docs/sdk-scenarios.md
Просмотреть файл

@ -16,7 +16,7 @@ There are 2 types of SDK build --- *Clean Build* and *Build No Change*.
### Prerequisites
- make sure the test directory is clean of artifacts (you can run `post.py` to remove existing artifacts folders from the last run )
- python3 or newer
- dotnet runtime 3.0 or newer
- dotnet runtime 3.1 or newer
- terminal/command prompt **in Admin Mode** (for collecting kernel traces)
- clean state of the test machine (anti-virus scan is off and no other user program's running -- to minimize the influence of environment on the test)
@ -31,18 +31,18 @@ python3 post.py
Run precommand to create a new console template.
```
cd emptyconsoletemplate
python3 pre.py default -f net5.0
python3 pre.py default -f net6.0
```
The `default` command prepares the asset (creating a new project if the asset is a template and copying it to `app\`). Now there should be source code of a console template project under `app\`.
Note that it is important to be aware SDK takes different paths for different TFMs, and you can configure which TFM your SDK tests against. Howeverm your SDK version should be >= the TFM version because SDK cannot build a project that has a newer runtime. Here's a matrix of valid SDK vs. TFM combinations:
| | netcoreapp2.1 | netcoreapp3.0 | netcoreapp3.1 | net5.0 |
|--------------|---------------|---------------|---------------|--------|
| .NET 2.1 SDK | x | | | |
| .NET 3 SDK | x | x | | |
| .NET 3.1 SDK | x | x | x | |
| .NET 5 SDK | x | x | x | x |
| | netcoreapp2.1 | netcoreapp3.1 | net5.0 | net6.0 |
|--------------|---------------|---------------|--------|--------|
| .NET 2.1 SDK | x | | | |
| .NET 3.1 SDK | x | x | | |
| .NET 5 SDK | x | x | x | |
| .NET 6 SDK | x | x | x | x |
You can change TFM of the project by specifying `-f <tfm>`, which allows to replace the `<TargetFramework></TargetFramework>` property in the project file to be the custom TFM value (make sure it's a valid TFM value) you specified.
@ -76,24 +76,24 @@ Same instruction of [Step 4 in Scenario Tests Guide](scenarios-workflow.md#step-
## Command Matrix
- \<tfm> values:
- netcoreapp2.1
- netcoreapp3.0
- netcoreapp3.1
- net5.0
- net6.0
- \<build option> values:
- clean_build
- build_no_change
| Scenario | Asset Directory | Precommand | Testcommand | Postcommand | Supported Framework | Supported Platform |
|-------------------------------|----------------------|--------------------------|-----------------------------|-------------|--------------------------------------------------|---------------------|
| SDK Console Template | emptyconsoletemplate | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| SDK .NET 2.0 Library Template | netstandard2.0 | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| SDK ASP.NET MVC App Template | mvcapptemplate | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| SDK Web Large 2.0 | weblarge2.0 | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.0;netcoreapp3.1 | Windows;Linux |
| SDK Web Large 3.0 | weblarge3.0 | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.0;netcoreapp3.1 | Windows;Linux |
| SDK Windows Forms Large | windowsformslarge | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.0;netcoreapp3.1 | Windows |
| SDK WPF Large | wpflarge | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.0;netcoreapp3.1 | Windows |
| SDK Windows Forms Template | windowsforms | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.0;netcoreapp3.1 | Windows |
| SDK WPF Template | wpf | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.0;netcoreapp3.1 | Windows |
| SDK New Console | emptyconsoletemplate | N/A | test.py sdk new_console | post.py | netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0 | Windows;Linux |
| Scenario | Asset Directory | Precommand | Testcommand | Postcommand | Supported Framework | Supported Platform |
|:------------------------------|:---------------------|:-------------------------|:----------------------------|:------------|:------------------------------------------|:-------------------|
| SDK Console Template | emptyconsoletemplate | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| SDK .NET 2.0 Library Template | netstandard2.0 | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| SDK ASP.NET MVC App Template | mvcapptemplate | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.1;net5.0;net6.0 | Windows;Linux |
| SDK Web Large 2.0 | weblarge2.0 | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.1 | Windows;Linux |
| SDK Web Large 3.0 | weblarge3.0 | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.1 | Windows;Linux |
| SDK Windows Forms Large | windowsformslarge | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.1 | Windows |
| SDK WPF Large | wpflarge | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.1 | Windows |
| SDK Windows Forms Template | windowsforms | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.1 | Windows |
| SDK WPF Template | wpf | pre.py default -f \<tfm> | test.py sdk \<build option> | post.py | netcoreapp3.1 | Windows |
| SDK New Console | emptyconsoletemplate | N/A | test.py sdk new_console | post.py | netcoreapp2.1;netcoreapp3.1;net5.0;net6.0 | Windows;Linux |