coyote/Scripts/run-tests.ps1

62 строки
2.3 KiB
PowerShell
Исходник Обычный вид История

2020-02-29 03:01:21 +03:00
param(
[string]$dotnet = "dotnet",
[ValidateSet("all", "netcoreapp3.1", "net47", "net48")]
[string]$framework = "all",
[ValidateSet("all", "production", "rewriting", "testing", "standalone")]
[string]$test = "all",
[string]$filter = "",
[string]$logger = "",
[ValidateSet("quiet", "minimal", "normal", "detailed", "diagnostic")]
[string]$v = "normal"
2020-02-29 03:01:21 +03:00
)
Import-Module $PSScriptRoot/powershell/common.psm1 -Force
2020-02-29 03:01:21 +03:00
Merged PR 2211: initial prototype for task binary rewriting **How to run:** ``` ..\coyote\bin\netcoreapp3.1\coyote.exe rewrite .\bin\netcoreapp3.1\example.coyote.json ``` Basically the `rewrite` tool option uses `PATH` to declare a JSON configuration file. Add `-d` to get some useful debugging info. Then just do regular `coyote test` on the rewritten binary. **JSON:** ```json { "AssembliesPath": "../../bin/netcoreapp3.1", "OutputPath": "../../bin/netcoreapp3.1/RewrittenBinaries", "Assemblies": [ "Example.dll" ], "Dependencies": [ "Microsoft.Coyote.dll", "Microsoft.Coyote.Test.dll", "Library.dll", "Example.runtimeconfig.json" ] } ``` If `OutputPath` is the same as `AssembliesPath`, then the original assemblies get overwritten. **Discussion:** The main components are an `AssemblyRewriter` that takes a JSON configuration file and uses that info to rewrite a set of assemblies (and their dependencies) and put them in some location (or overwrite original ones). That happens via "coyote rewrite". Once that is done, someone can just run "coyote test" and it should work as if someone wrote controlled tasks manually (ofc we need to push corner cases now, some APIs still I need to do, and support for build in mocks like TCS). We also have CI now, our existing run-test.ps1 script first rewrites the xUnit test project, that is only native tasks, and then it runs it (works only for .NET Core 3.1 via xUnit right now for some reason). The cool thing is that we do not use a custom `Coyote.Task` type anymore. The reason we needed a custom task type was purely to make the manual approach usable. A custom type gives us an awaitable object (because we cannot take control of `GetAwaiter` from native Task, when someone calls await on it), as well as allows you to have it as return type of an async method (for the state machine to be generated). Instead now we focus instrumentation on two aspects: (1) every generated async state machine is replaced with a custom `AsyncStateMachineBuilder` (this is similar to what we had before, but this time we can just replace it automatically as the state machine is already there in the IL , we don't need a custom type to do that for us), and (2) replace some Task methods with custom ones (e.g. `Task.Run` with `ControlledTask.Run`, `Delay`, `Yield`, and importantly `Task.GetAwaiter` with `ControlledTask.GetAwaiter`, to get a callback on an await point). You can think of `ControlledTask` as an "extension" method, not a custom type method (its a purely static class) so its much easier to replace (we don't need to replace `Task` with `Coyote.Task` in every single IL instruction ... as `ControlledTask` APIs just return native tasks, they are just a way to get hooks). **Known issues:** - Figure out how to make xUnit recognize rewritten assemblies with .NET Framework (.NET Core works fine). Related work items: #3911, #4378
2020-07-07 01:48:04 +03:00
$frameworks = Get-ChildItem -Path "$PSScriptRoot/../Tests/bin" | Where-Object Name -CNotIn "netstandard2.0", "netstandard2.1" | Select-Object -expand Name
2020-02-29 03:01:21 +03:00
$targets = [ordered]@{
"production" = "Production.Tests"
"rewriting" = "BinaryRewriting.Tests"
"testing" = "SystematicTesting.Tests"
"standalone" = "Standalone.Tests"
2020-02-29 03:01:21 +03:00
}
$key_file = "$PSScriptRoot/../Common/Key.snk"
[System.Environment]::SetEnvironmentVariable('COYOTE_CLI_TELEMETRY_OPTOUT', '1')
2020-02-29 03:01:21 +03:00
Write-Comment -prefix "." -text "Running the Coyote tests" -color "yellow"
foreach ($kvp in $targets.GetEnumerator()) {
if (($test -ne "all") -and ($test -ne $($kvp.Name))) {
continue
}
foreach ($f in $frameworks) {
if (($framework -ne "all") -and ($f -ne $framework)) {
continue
}
$rewriting_target = ""
if ($($kvp.Name) -eq "rewriting") {
$rewriting_target = "$PSScriptRoot/../Tests/$($kvp.Value)/bin/$f/BinaryRewritingTests.coyote.json"
}
elseif ($($kvp.Name) -eq "standalone") {
$rewriting_target = "$PSScriptRoot/../Tests/bin/$f/Microsoft.Coyote.Standalone.Tests.dll"
}
if ($rewriting_target -ne "") {
# Rewrite the test.
Invoke-CoyoteTool -cmd "rewrite" -dotnet $dotnet -framework $f -target $rewriting_target -key $key_file
# Try rewrite again to make sure we can skip a rewritten assembly and do no damage!
Invoke-CoyoteTool -cmd "rewrite" -dotnet $dotnet -framework $f -target $rewriting_target -key $key_file
}
# Run the (rewritten) test.
$target = "$PSScriptRoot/../Tests/$($kvp.Value)/$($kvp.Value).csproj"
Invoke-DotnetTest -dotnet $dotnet -project $($kvp.Name) -target $target -filter $filter -logger $logger -framework $f -verbosity $v
2020-02-29 03:01:21 +03:00
}
}
Write-Comment -prefix "." -text "Done" -color "green"