Граф коммитов

406 Коммитов

Автор SHA1 Сообщение Дата
Sebastian Burckhardt 48e3a955bd
Support DF clients/connections paradigm (#126)
* support client-only NetheriteProvider and use staged startup that can expose exceptions to client calls

* cache NetheriteProviders in static dictionary

* remove providers from the static dictionary when they are stopped.

* trace EH offset provider
2022-02-23 09:24:51 -08:00
Sebastian Burckhardt 404d6e0496
Fix bug in persisting custom status (#127)
* fix custom status persistence

* fix missing assignment
2022-02-23 09:24:36 -08:00
Sebastian Burckhardt 2391111762
Update to 2.6.1 (#125)
* update to version 2.6.1 of the extension

* update DurableTask.Core to 2.7.0
2022-02-15 19:50:37 -08:00
dependabot[bot] e0432ac111
Bump follow-redirects in /samples/Hello_Netherite_with_TypeScript (#124)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.5 to 1.14.8.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.14.5...v1.14.8)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-14 11:09:44 -08:00
Sebastian Burckhardt 30abd4d516
Implement FASTER compaction and checkpoint testing mechanism (#121)
* implement FASTER log compaction state machine.

* fix test

* fix bug in ActivitiesState, must not overwrite ReportedLoad field

* improve tracing of outbox events

* improve event detail tracing

* improve tracing of storage operations

* improve checkpoint removal tracing

* fix state machine logic for removing obsolete checkpoints

* backport fix to EmitCurrentState

* fix batch.SendingEventId assignment: should not be gated by replay
2022-02-07 15:25:54 -08:00
Sebastian Burckhardt 08961df130
fix concurrency bug in outbox (#123) 2022-02-07 13:50:05 -08:00
Sebastian Burckhardt 2499572f7f
Periodic removal of checkpoints (#119)
* use path prefix to solve atomicity of taskhub creation/deletion

* implement periodic removal of obsolete checkpoints
2022-01-25 15:27:33 -08:00
Sebastian Burckhardt ba56fa4b23
use path prefix to solve atomicity of taskhub creation/deletion (#118) 2022-01-25 13:20:28 -08:00
Sebastian Burckhardt a9e6dbca23
Minor tracing changes and refactoring (#116)
* use explicit interface for IOrchestrationService, and add some eventhubs tracing

* remove configureawait and make tracing for received events more visible

* update offset provider

* two more tracing changes
2022-01-25 07:27:20 -08:00
Sebastian Burckhardt 079eceb208
Maintain a count of instances in each partition (#113)
* Add a stats object that tracks the number of instances present in a partition

* add more detail to replay checker
2022-01-24 09:26:17 -08:00
Sebastian Burckhardt d7ef1e1e71
update to durable-functions 1.5.4 (#115) 2022-01-24 09:14:54 -08:00
Sebastian Burckhardt 94129ab595
fix missing await for task in ApplyToStore (#112)
* fix missing await for task in ApplyToStore

* increase timeout for test to fix CI test fail
2022-01-21 14:54:33 -08:00
Sebastian Burckhardt ab89349e3e
encapsulate orchestration service query API in interface, and fix query test that ignored continuation token (#114) 2022-01-21 09:55:42 -08:00
Sebastian Burckhardt ceaea73ef4 update samples 2022-01-12 13:36:41 -08:00
Sebastian Burckhardt 463ab1439d update version to 0.7.0-beta 2022-01-12 12:47:25 -08:00
Sebastian Burckhardt 2f4bce0e2c
Fix autoscaler (#108)
* fix bug in scaling logic

* catch and trace more errors

* overhaul the tracing and logging to use uniform prefix and include more detail
2022-01-12 12:40:46 -08:00
Sebastian Burckhardt 9ec62a771d
fix deduplication of fragmented partition and client events (#106) 2022-01-12 07:33:37 -08:00
Sebastian Burckhardt 4e2b24fdc4
Fix hanging storage (#103)
* undo semantics change on WaitForOrchestration

* improve ping so it is useful for detecting hangs

* catch timeouts

* more tracing in EventHubsProcessor

* fix details in storage format error

* support error injection and replay checking in DF deployments

* reduce tracing detail of exceptions for warnings

* add replay condition to tracing of discarded work items

* fix replay checker to handle serialized tracked objects

* avoid propagation of hangs by adding termination wrapper on all awaits in FasterStorage

* remove instanceprotected assertions since they may not be local to a partition

* add check for termination condition to EH checkpoint saving, and add tracing to event processor host registration

* fix race condition in MemoryTransport
2022-01-11 04:53:10 -08:00
Sebastian Burckhardt 7644f2e9f1
fix bug in client connection establishment, introduced in #82, that caused slow startup (#102) 2022-01-07 07:58:14 -08:00
Sebastian Burckhardt 0499ba5c18
Add fault injection test, fix bugs (#101)
* implement fault injection tests

* fix tests

* update fault injection tests

* revise MemoryTransport failure handling

* fix serialization of RecoveryCompleted

* refactor concurrent tests into separate file

* revise tracing and timeouts in tests

* fix fault injection tests

* change timeout behavior of wait for orchestration to bring it in line with other backends

* fix bug that causes test to hang when recovering

* revise the recovery mechanism in MemoryTransport

* connect Partition.Assert to TestHooks so it can fail unit tests

* fix missing check for replaying when processing RecoveryCompleted event

* revise fault injector: add timeout to startup and do not inject lease renewals

* annotate tests with whether they support any transport

* adjust timeouts and update pipeline

* add labels to all the assertions and hook them up so they trip unit tests

* remove non-fixture query test from "AnyTransport" test category
2022-01-06 12:43:08 -08:00
Sebastian Burckhardt e8f4ab7a87
implement dequeue count for sessions and activities (#92) 2022-01-04 13:11:44 -08:00
Sebastian Burckhardt 885f7b9760
Implement replay checker (#99)
* fix aliasing bug

* implement replay checker

* undo unnecessary change

* add missing teeth: must fail unit tests on test hook errors

* fix bug, remove unnecessary field

* fix replay discrepancy on fields of outgoing messages

* fix race condition when updating creation timestamp during prefetch

* fix last commit

* fix false errors caused by json property order
2022-01-04 09:25:32 -08:00
Sebastian Burckhardt f542c93a01
fix warnings (#100) 2022-01-03 12:57:00 -08:00
Sebastian Burckhardt beaeeb7c07
remove OrchestrationState from the persisted BatchProcessed to save space in commit log (#94) 2021-12-22 16:37:17 -08:00
Sebastian Burckhardt 266c31dcef
fix culling of waiters in instance state (#93) 2021-12-22 15:41:43 -08:00
Sebastian Burckhardt 3989fb03d6
Fix client responses to wait for persistence (#98)
* fixes the response for create, delete, purge, and wait request so it does not send the response until the state is persisted

* fix logic mistake in OutboxState
2021-12-22 12:29:02 -08:00
Sebastian Burckhardt 195cf34c66
Replace double-dynamic invocation (#95)
* implement visitor pattern instead of using C# double-dynamic

* fix missing cases and remove all occurrences of dynamic.
2021-12-22 10:43:25 -08:00
Sebastian Burckhardt 9767b7e610 fix across.ps1 series 2021-12-21 14:18:03 -08:00
Sebastian Burckhardt 822e90381d
separate storage of singleton states from storage of instance and history states. (#30) 2021-12-21 08:04:44 -08:00
Sebastian Burckhardt 3be91de49c
fix nullref in scale monitor (#97) 2021-12-21 08:03:15 -08:00
Sebastian Burckhardt b74f132516
latest version is 0.6.1-alpha, add link to video 2021-12-09 14:54:10 -08:00
Sebastian Burckhardt 0ecbf31c27 update version to alpha-0.6.1 2021-12-09 14:16:26 -08:00
Sebastian Burckhardt 1c50034783
support netstandard2.0 (#91) 2021-12-09 06:55:24 -08:00
Sebastian Burckhardt 30c2a6b8e9
Fix samples to use latest dependencies (#90)
* fix typescript sample to correctly use latest version

* fix python sample to correctly use latest version
2021-11-23 09:58:07 -08:00
Sebastian Burckhardt 5276f50408 update samples to use latest version 0.6.0-alpha 2021-11-17 10:36:41 -08:00
Sebastian Burckhardt 55d8b3fb35 Merge branch 'dev' 2021-11-17 09:56:56 -08:00
Sebastian Burckhardt 6207c0c67b Merge branch 'dev' of https://github.com/microsoft/durabletask-netherite into dev 2021-11-17 09:55:51 -08:00
Sebastian Burckhardt 7a05a25712 update to 0.6.0-alpha 2021-11-17 09:55:28 -08:00
Sebastian Burckhardt 4dd79140ee
Update to latest versions of DF and DT (#89)
* update to latest published versions

* revert update of LoggingAbstractions
2021-11-17 09:49:29 -08:00
Sebastian Burckhardt 012b50cd35
Update performance tests (#88)
* fix small errors and add some tracing in performance tests

* edit perftests

* update concurrency default

* change throughput scale for word hash

* update series, and use public access blob for gutenberg

* update bank and fanoutfanin series

* limit number of words counted in each book to allow better load balancing

* fix error and select EP2 for alt

* update wordcount definitions for EP2 and EP3

* update wordcount definition for EP1

* add load generator app
2021-11-15 12:39:53 -08:00
Sebastian Burckhardt 45e22a23f5
Fix NullReferenceException during recovery, add unit test for recovery (#87)
* temp commit

* add simple recovery test

* remove unnecessary setting

* fix FASTER NullReferenceException during recovery caused by missing delta log
2021-11-12 13:37:35 -08:00
Sebastian Burckhardt fae606f47d
various fixes to tracing (#84) 2021-10-20 08:11:51 -07:00
Sebastian Burckhardt 55e7b2feba
ensure client receiver connections are established when transport layer is started; add more tracing (#82) 2021-10-14 09:27:49 -07:00
Sebastian Burckhardt cb07152e0a
Add tracing to observe taskhub operations (#83) 2021-10-14 09:26:28 -07:00
Sebastian Burckhardt 4a9ffbec31
Update to FASTER 1.9.6 (#80)
* update to FASTER 1.9.3

* update to 1.9.6
2021-10-14 09:26:02 -07:00
Sebastian Burckhardt 4ae87f2eee
fix incorrect uses of the resolved connection string where the connection name should be used. (#77) 2021-10-11 14:59:06 -07:00
Sebastian Burckhardt 8b1d315ae4
fix recovery parameter for FASTER (#79) 2021-10-08 15:22:53 -07:00
Sebastian Burckhardt 0c7aa42853
CacheOrchestrationCursors must be set to false when running out-of-proc (#71) 2021-10-05 11:29:44 -07:00
Christian Lechner 0390605bd9
New Code Samples for Netherite (#72)
* new samples for Netherite

* corrections after review
2021-09-20 11:48:15 -07:00
Sebastian Burckhardt bcb5f67c51
Improve performance of query results (#70)
* launch query and prefetch events earlier to improve latency

* add nicer query endpoint to perf tests

(cherry picked from commit 9851a1de91)

* add tracing for when a sender is starting to send and event.

* support use of multiple client channels; fix range of partitions; support pipelined return of query results

* use custom serialization to compress query results

* fix bug in test
2021-09-13 12:28:22 -07:00