This commit is contained in:
Olivia Chen 2020-09-22 14:45:46 -07:00
Родитель 611b37740d
Коммит 3406a83bcd
2 изменённых файлов: 83 добавлений и 21 удалений

Просмотреть файл

@ -1,6 +1,6 @@
# Crossgen Scenarios
An introduction of how to run scenario tests can be found in [Scenarios Tests Guide](link). The current document has specific instruction to run:
An introduction of how to run scenario tests can be found in [Scenarios Tests Guide](./scenarios-workflow.md). The current document has specific instruction to run:
- [Crossgen Throughput](#crossgen-throughput)
- [Crossgen2 Throughput](#crossgen2-throughput)
@ -11,7 +11,7 @@ An introduction of how to run scenario tests can be found in [Scenarios Tests Gu
### Step 0 Generate Core Root
Build CoreCLR in dotnet/runtime repo, which creates Core Root. Instruction can be found [here](https://github.com/dotnet/runtime/blob/master/docs/workflow/building/coreclr/README.md). A detailed introduction of Core Root can be found [here](https://github.com/dotnet/runtime/blob/master/docs/workflow/testing/using-corerun.md).
### Step 1 Initialize Environment
Same instruction of [Step 1 in Scenario Tests Guide](Link).
Same instruction of [Step 1 in Scenario Tests Guide](./scenarios-workflow#step-1-initialize-environment).
### Step 2 Run Precommand
```
cd crossgen
@ -23,7 +23,7 @@ py -3 test.py crossgen --core-root <path to Core Root directory> --test-name <as
```
The above command runs the test. Note `--test-name <assembly to compile>` option refers to the relative path of an assembly that's under Core Root directory. For example, the option can be `--test-name System.Private.Xml.dll` so the test measures the throughput of crossgen compiling `System.Private.Xml.dll`.
### Step 4 Run Postcommand
Same instruction of [Step 4 of Scenario Tests Guide](Link).
Same instruction of [Step 4 of Scenario Tests Guide](./scenarios-workflow#step-4-run-postcommand).
## Crossgen2 Throughput
Refer to [Crossgen Throughput](crossgen-throughput) for Step 0,1 and 4

Просмотреть файл

@ -1,90 +1,152 @@
# Scenario Tests Guide
## Overview
Our existing scenario tests are under `src\scenarios` in this repo, where each subdirectory contains a test asset that can be combined with a specific set of commands to do measurements. Currently we have scenario tests for [SDK](link), [Crossgen](link), [Blazor](blazor_scenarios.md) and [other scenarios]().
Our existing scenario tests are under `src\scenarios` in this repo, where each subdirectory contains a test asset that can be combined with a specific set of commands to do measurements. Currently we have scenario tests for [SDK](link), [Crossgen](link), [Blazor](./blazor-scenarios.md) and [other scenarios]().
## Running scenario tests
This is a general guideline on how the scenario tests are arranged in this repo. We will walk through it by measuring the **startup time of an empty console template** as a sample scenario. For other scenarios, refer to the following links:
- [How to run SDK scenario tests](link)
- [How to run Crossgen scenario tests](link)
- [How to run Blazor tests](blazor_scenarios.md)
- [How to run Crossgen scenario tests](./crossgen-scenarios.md)
- [How to run Blazor tests](./blazor-scenarios.md)
- [How to run other Scenario tests](link)
### Prerequisites:
- python3 or newer
- dotnet runtime 3.0 or newer
### Step 1 Initialize Environment
Go to `src\scenarios` and run the following command:
On Windows, start a new PowerShell environment ***in Admin Mode*** and run:
```
cd src\scenarios
.\init.ps1
```
On Linux, run in the terminal:
```
cd src/scenarios
. init.sh
```
This script sets up `$PYTHONPATH`. Without other options specified it will use the dotnet in `$PATH`. To download a new dotnet or specify a custom dotnet, type `--help` for more command line options.
Command options for `init.ps1` and `init.sh` scripts:
```
# Either
.\init.ps1 -DotnetDirectory <custom dotnet directory> # specify a custom dotnet location
# Or
.\init.ps1 -Channel <Channel to download new dotnet> # channel to download a new dotnet from, which will be downloaded into tools folder
```
Same mearning for `init.sh` on Linux but with `-dotnetdir` and `-channel` options
### Step 2 Run Precommand
For some scenarios, `pre.py` runs a defined precommand before the test run, which can but not limited to set up the asset by either creating a new template or using a static template.
For some scenarios, `pre.py` runs a defined precommand before the test run, which can but not limited to set up the asset by either creating a new template or using a static template.
```
cd emptyconsoletemplate
# format: <Python> pre.py <command> -f <target framework> -c <configuration> -r <runtime>
py -3 pre.py publish -f netcoreapp5.0 -c Release
```
Run `<Python> pre.py --help` for more command options.
The above command creates a new dotnet console template in `emptyconsoletemplate\app\` folder and publish it to `emptyconsoletemplate\pub\` folder.
### Step 3 Run Test
`test.py` runs the test, which defines a set of attributes for each asset. In the same directory as in the previous step, run:
```
# format: <Python> test.py <command> <test-specific options>
py -3 test.py startup
```
Run `<Python> test.py --help` for more command options.
The above command runs the published app under `emptyconsoletemplate\pub\` with specified iterations and measures its startup time. Traces are saved into `emptyconsoleatemplate\traces\` folder.
The above command runs the published app under `emptyconsoletemplate\pub\` with specified iterations and measures its startup time. Traces are saved into `emptyconsoleatemplate\traces\` folder.
### Step 4 Run Postcommand
`post.py` should be optionally executed to clean up the artifacts.
```
# format: <Python> post.py
py -3 post.py
```
The above command removes `app`, `bin`, `traces`, `pub`, `tmp` directories if generated.