Caching the constructed types cannot be generalized, especially for generics. Many types we need to reconstruct under the specific context. This is most likely because the result is not immutable. Instead, we rely on method caching to optimize. We can revisit this in the future.
Related work items: #4891
The previous implementation was blocking, not asynchronous, so we fix that, as well as exception logic in these APIs. Also adding new tests and fixing previous ones.
Related work items: #4873, #4876
We don't need dependency paths, and they are cumbersome to write, lowering usability. We instead mirror the output directory before we do the rewriting (in the case that we are not overwriting).
Related work items: #4495
The following pattern blows up our stack:
```C#
[Test]
public static async Task Test()
{
while (true)
{
await Task.Delay(1);
}
}
```
This PR fixes this and does some major simplifications in the corresponding logic. Also new tests, and adding tests for rewriting. Also some cleanup (not all relevant for this fix, but I was on it already...).
Related work items: #4796
**How to run:**
```
..\coyote\bin\netcoreapp3.1\coyote.exe rewrite .\bin\netcoreapp3.1\example.coyote.json
```
Basically the `rewrite` tool option uses `PATH` to declare a JSON configuration file. Add `-d` to get some useful debugging info.
Then just do regular `coyote test` on the rewritten binary.
**JSON:**
```json
{
"AssembliesPath": "../../bin/netcoreapp3.1",
"OutputPath": "../../bin/netcoreapp3.1/RewrittenBinaries",
"Assemblies": [
"Example.dll"
],
"Dependencies": [
"Microsoft.Coyote.dll",
"Microsoft.Coyote.Test.dll",
"Library.dll",
"Example.runtimeconfig.json"
]
}
```
If `OutputPath` is the same as `AssembliesPath`, then the original assemblies get overwritten.
**Discussion:**
The main components are an `AssemblyRewriter` that takes a JSON configuration file and uses that info to rewrite a set of assemblies (and their dependencies) and put them in some location (or overwrite original ones). That happens via "coyote rewrite". Once that is done, someone can just run "coyote test" and it should work as if someone wrote controlled tasks manually (ofc we need to push corner cases now, some APIs still I need to do, and support for build in mocks like TCS). We also have CI now, our existing run-test.ps1 script first rewrites the xUnit test project, that is only native tasks, and then it runs it (works only for .NET Core 3.1 via xUnit right now for some reason).
The cool thing is that we do not use a custom `Coyote.Task` type anymore. The reason we needed a custom task type was purely to make the manual approach usable. A custom type gives us an awaitable object (because we cannot take control of `GetAwaiter` from native Task, when someone calls await on it), as well as allows you to have it as return type of an async method (for the state machine to be generated). Instead now we focus instrumentation on two aspects: (1) every generated async state machine is replaced with a custom `AsyncStateMachineBuilder` (this is similar to what we had before, but this time we can just replace it automatically as the state machine is already there in the IL , we don't need a custom type to do that for us), and (2) replace some Task methods with custom ones (e.g. `Task.Run` with `ControlledTask.Run`, `Delay`, `Yield`, and importantly `Task.GetAwaiter` with `ControlledTask.GetAwaiter`, to get a callback on an await point). You can think of `ControlledTask` as an "extension" method, not a custom type method (its a purely static class) so its much easier to replace (we don't need to replace `Task` with `Coyote.Task` in every single IL instruction ... as `ControlledTask` APIs just return native tasks, they are just a way to get hooks).
**Known issues:**
- Figure out how to make xUnit recognize rewritten assemblies with .NET Framework (.NET Core works fine).
Related work items: #3911, #4378
Add a test that shows that all possible interleavings in a simple example are actually explored. Also fix RunAsync so it honors the requested # iterations when running in production tests.
Added support for timeout and cancellation APIs on semaphore.
Note: these are only for the production version, we do not support timeouts/cancellation APIs in systematic testing yet (this is not specific to semaphores, but in general). That are separate low-priority work items #729 and #717.
There were two bugs in the scheduler:
- We checked for external uncontrolled concurrency by checking that the current operation is always the operation calling the check, but that was too aggressive, and can genuinely not be the case because of some internal concurrency in the runtime (e.g. we spawn a task continuation inside the TaskController to perform a `ScheduleNext`, which is racy but benign since the scheduler is thread safe). As long as the operation is not null (which means an uncontrolled task is calling) then we are fine. Fixing this removes the uncontrolled task exception in the CoffeMachineTask sample.
However, fixing the above bug, made a new bug manifest (that was causing the scheduler to hang due to a premature completion of an operation before it was supposed to get completed):
- There was a rare internal race condition that could cause the `ScheduleNextOperation` to return without waiting because the current operation was not having `IsHandlerRunning` set as true yet (that flag would be set later on). I fixed this by simplifying the logic and removing lots of complexity. We dont need a special `IsHandlerRunning` flag anymore, we just use the Operation status making things simpler. Now, returning without waiting in `ScheduleNextOperation` happens only if the operation has completed or canceled, which was the expected behavior anyway.
Capability to specify to the scheduler that the program wants to yield the execution of the currently executing operation. This allows strategies to implement different techniques to deal with this. In this PR we implement de-prioritization with PCT. This is very useful for dealing with infinite (polling) task loops and (periodic) timers, but this PR only changes `Task.Yield`, we will deal with timers in a later PR.
In customer code, this reduced the exploration steps to reach cold state in a liveness monitor (and thus the end of a infinite test) from 10-20k down to 1k steps.
Related work items: #4218
Removing old logic with internal interface which is not required, makes the logic more complex (as it requires some ugly casting) and worse perf due to having to go via the interface. The base class is all we need really.
Turns out XUnit does **not** work properly with Microsoft.Coyote.Tasks.Task. It hides unhandled exceptions, so a bunch of failing tests were not reporting those failures.
We don't always need to dump `trx` files, I found that it was dumping tons and tons of them on my laptop from repeated runs, so I made it optional.
You can now pass a `-logger FORMAT` option to the script, where `FORMAT` is a supported format like `trx`, like this:
```
.\Scripts\run-tests.ps1 -logger trx
```
And this will dump files like this:
```
Results File: ...\coyote\Tests\Production.Tests\TestResults\pdeligia_2020-06-10_07_14_18.trx
```
If we ever need this in CI, we can enable it as an option. But I noticed that this does not really do much, besides dumping the results on disk. Perhaps DevOps has to be given the `TestResults` directory?
Unify production and systematic CreateActorIdFromNameTests and CreateActorTests
Add Configuration.WithProductionMonitorEnabled
Fix a flakey test by adding a timeout.
In getting familiar with OperationGrouping I did some cleanup work on the tests.
Promote OperationGroupingTests to be "Actor" level tests instead of being StateMachine tests. Also made them shared code for SystematicTest so we get more test coverage also, and fixed issues in BaseTest to make this work. Also fixed long standing problem with BaseTest swallowing important error messages that where happening in actors on other threads that XUnit was not reporting.
This is a first attempt at sharing code between Production and SystematicTesting for Actor unit tests. (We already have 100% sharing for Task based tests). This one is focused just on CustomActorRuntimeLogTests. The trick was to make WaitAsync work across both production and systematic tests.
Related work items: #762
Fix dispose on CoyoteRuntime so it resets the Current task and fix dispose on ControlledRuntime so it resets IsExecutionControlled. Add test to make sure it works.
This allows mixing of production tests in with systematic tests in the same test run which is important onramp for customers adopting Microsoft.Coyote.Task where they want to be able to run new Coyote tests and their old traditional unit tests in the same test run.
Related work items: #4065
This PR simplifies the logic of bookkeeping the currently executing operation in `OperationScheduler` and make it more robust.
We remove reliance on a dictionary of task ids to operations, which not only simplifies the logic, but also makes it more robust because we cannot rely on task ids. This is because the .NET runtime does not guarantee that task ids will be unique. From the official [docs](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.id?view=netcore-3.1):
> Task IDs are assigned on-demand and do not necessarily represent the order in which task instances are created. Note that although collisions are very rare, task identifiers are not guaranteed to be unique.
Chris Lovett
Pantazis Deligiannis
timbo