Граф коммитов

13 Коммитов

Автор SHA1 Сообщение Дата
Sebastian Burckhardt a69551ab27
Overhaul internal layers and single-host emulation (#194)
* temp commit

* fix stuff

* update retry logic

* update tests

* rename providers to layers

* add display names for pipeline stages

* minor updates to comments and remove unnecessary usings

* temporarily disable the size checking

* edit and reorder CI pipeline for troubleshooting

* do not run replay checker on the default host fixture

* more tinkering with order of tasks in CI pipeline

* update test projects to latest runtime and packages

* remove replay checker from in-memory tests since it does not actually run anyway

* temporarily remove ConcurrentTestsFaster.EachScenarioOnce since it has demonstrated hanging

* update pipeline durations

* reenable EachScenarioOnce with fixes

* remove excessive tests
2022-09-28 13:14:01 -07:00
Varshitha Bachu fa6d980c0e
Add Release Pipeline (#150)
* Set up CI with Azure Pipelines

[skip ci]

* adding separate release pipeline yaml

* update csproj to include manifest in package

* Update VM

* update AzureFunctions csproj

* update comments and spacing

* minor fix

* adding IncludeReferenceProjects to .csproj

* remove include reference projects flag

* updated order of tasks based on dependencies

* updated restore task
2022-04-22 15:37:43 -07:00
Sebastian Burckhardt 344f9ff94f
Retain outbox to handle lost events (#130)
* draft commit

* revise so we can distinguish types of recovery and avoid sending too much stuff

* temp commit

* finish implementation of EH recovery

* update tracing for perf, clients, and load monitor

* fix eh checkpointing, and fix replay behavior

* update timeout handling for queries and requests

* fix timeouts for queries

* fix restart on lost events

* fix accidentally committed stuff

* make blob logger resilient against MemoryStream size overflow

* use storage account instead of storage emulator for CI
2022-04-05 08:03:13 -07:00
Sebastian Burckhardt 177cf161e0
Implement CacheSizeTracker and use FASTER v2 (#128)
* update to FASTER v2 and implement MemoryTracker

* update history size calculation in tests

* fix renamed variable

* work around incorrect status on read completion callback

* minor updates to tests

* adjust timeout for latest bits

* Recover fewer pages.

* handle one more transient timeout on storage calls.

* fix compaction

(cherry picked from commit acec2f0ac06c25e39bda3edcb4ab6a7a8b71d5bb)

* fix client issues causing hangs

(cherry picked from commit 6434c56033ee9fac995659764ab2aedd75fb0d94)

* update to latest prerelease

(cherry picked from commit 0ad283aaa9956022d1e5bf14b616961a59f071d6)

* add EH overlap to work around bug

* add tolerance to MemoryReduction test

* overhaul exception handling in query events

* remove deadlock risks in cancellation token handlers

* compaction must be categorized like scan

* load fewer pages on recovery

* trace event id string when receiving packet

* add condition to size check to account for concurrent modifications

* keep termination continuation direct for tracing and for debugging

* retry slow reads to work around FASTER hang issues

* update tracing

* instrument hang detection

* update timeout limits

* refactor cancellation

* make the faultinjector back off of startup after 2 failures

* overhaul shutdown sequence

* add hang detection to azure storage device

* add CompactThenFail test

* simplify checkpoint state machine as compaction now already moves BeginAddress

* fix bug in PartitionErrorHandler

* Arm assertions in release build also, and run EH CI tests on release

* extend hang detection limit to 90s and remove the slowest ScaleSmallScenarios test

* update compaction progress reporting

* recognize yet another type of transient storage error

* use release build and remove concurrent test from EH tests on CI

* implement thread count watcher

* fix bug in AzureStorageDevice.RemoveSegmentsAsync and add to tracking

* remove .ConfigureAwait(false)

* update PerformanceTests http bindings to support arbitrary prefixes with start/purge/query/count/await

* fix CI

* fix nullref in dispose
2022-03-09 16:40:18 -08:00
Sebastian Burckhardt 0499ba5c18
Add fault injection test, fix bugs (#101)
* implement fault injection tests

* fix tests

* update fault injection tests

* revise MemoryTransport failure handling

* fix serialization of RecoveryCompleted

* refactor concurrent tests into separate file

* revise tracing and timeouts in tests

* fix fault injection tests

* change timeout behavior of wait for orchestration to bring it in line with other backends

* fix bug that causes test to hang when recovering

* revise the recovery mechanism in MemoryTransport

* connect Partition.Assert to TestHooks so it can fail unit tests

* fix missing check for replaying when processing RecoveryCompleted event

* revise fault injector: add timeout to startup and do not inject lease renewals

* annotate tests with whether they support any transport

* adjust timeouts and update pipeline

* add labels to all the assertions and hook them up so they trip unit tests

* remove non-fixture query test from "AnyTransport" test category
2022-01-06 12:43:08 -08:00
Sebastian Burckhardt 6f8ba021ed
Fix test configuration (#40)
* fix implementation of local file storage for FASTER

* fix test configuration in DurableTask.Netherite.AzureFunctions

* generalize the CI pipeline so it runs all tests again.
2021-05-07 09:39:51 -07:00
Sebastian Burckhardt 2e2bf1fde6 Update azure-pipelines.yml for Azure Pipelines 2021-03-31 11:53:16 -07:00
Sebastian Burckhardt d19e8feb91
update azure pipeline to test both simulation and actual event hubs 2021-03-10 16:46:03 -08:00
Sebastian Burckhardt 4445554753 Update azure-pipelines.yml for Azure Pipelines 2021-03-10 14:33:25 -08:00
Sebastian Burckhardt 64b64649e0 Update azure-pipelines.yml for Azure Pipelines 2021-03-10 14:24:15 -08:00
Sebastian Burckhardt 20adec6d1a Update azure-pipelines.yml for Azure Pipelines 2021-03-10 14:22:29 -08:00
Sebastian Burckhardt f7a3dcb7b3 Update azure-pipelines.yml for Azure Pipelines 2021-03-10 14:10:54 -08:00
Sebastian Burckhardt a0703fa660
Initial draft Azure Pipeline (#2) 2020-12-04 08:23:06 -08:00