Граф коммитов

387 Коммитов

Автор SHA1 Сообщение Дата
Sebastian Burckhardt db50ba91a7
Update src/DurableTask.Netherite/TransportLayer/EventHubs/BlobBatchReceiver.cs
Co-authored-by: David Justo <david.justo.1996@gmail.com>
2024-04-16 11:24:13 -07:00
sebastianburckhardt cd2850a69f fix handling of missing blobs 2024-04-16 09:29:44 -07:00
sebastianburckhardt 58eabedddb address PR feedback (add comments) 2024-04-15 13:49:33 -07:00
sebastianburckhardt 87af75df97 do not process any more events if already shutting down 2024-04-15 09:23:48 -07:00
sebastianburckhardt 48a1fcc334 add more tracing to blob download 2024-04-15 09:23:18 -07:00
sebastianburckhardt 854c9897a4 fix handling of blob deleted 2024-04-15 08:38:26 -07:00
sebastianburckhardt 3fa1769705 fix broken trace statement 2024-04-15 08:36:18 -07:00
sebastianburckhardt 29b2eb6fd1 fix check that removes confirmed events and add detail-level tracing 2024-04-12 15:21:31 -07:00
sebastianburckhardt 0ec23d9e54 fix incorrect handling of skipped events 2024-04-11 14:41:02 -07:00
sebastianburckhardt 3449be2801 address PR feedback 2024-04-11 09:22:32 -07:00
sebastianburckhardt 5ec7039da3 fix handling of batchposition, and tolerate race condition for deleted blob 2024-04-04 12:43:58 -07:00
sebastianburckhardt 59ed3946bb draft 2024-03-19 14:24:29 -07:00
Sebastian Burckhardt 3399888400
update version to 1.5.0 (#368) 2024-03-18 16:26:11 -07:00
Sebastian Burckhardt 0c1ccbec4a
Add variable-sized generic entity benchmark (#366)
* add a store entity to the performance benchmarks, for testing

* update http triggers
2024-03-18 15:41:46 -07:00
dependabot[bot] a72a1abdba
Bump follow-redirects in /samples/Hello_Netherite_with_TypeScript (#367)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.5 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.5...v1.15.6)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 09:54:38 -07:00
Sebastian Burckhardt 84b45a89b2
add README.md and some historic results to the performance tests folder (#362) 2024-03-18 09:49:40 -07:00
Sebastian Burckhardt 7e16a84d12
Improve error handling for FASTER IO completion callbacks (#349)
* wrap FASTER callbacks so exceptions are logged, and release semaphores prior to callback.

* address PR feedback, add two more lines of tracing, and use simpler trace message

* refactor control path for FASTER callbacks. Eliminate callbacks with nonzero error code.

* immediately cancel all requests waiting for the semaphore when partition is terminated

* cancel faster callbacks prior to disposing FasterKV and FasterLog (to prevent hangs)

* revert reorganization of cancellation since it appears to lead to issues with hanging dispose calls
2024-03-18 09:49:13 -07:00
Sebastian Burckhardt a74178bc6b
loosen azure storage semaphores (#365) 2024-03-18 09:48:09 -07:00
Sebastian Burckhardt d71a5a1ba8
Fix hanging blob creation (#351) (#364)
* fix hanging blob creation

* remove unnecessary try-catch
2024-03-15 13:21:42 -07:00
Sebastian Burckhardt a6924c01dd
handle long or infinite timeout argument in WaitForOrchestrationAsync (#357) 2024-03-15 12:59:34 -07:00
Sebastian Burckhardt 37f49adde5
Add RecoveryTester feature (#341)
* refactor partition management options, remove Scripted partition manager, and add RecoveryTester feature

* prevent creation of fresh taskhub in partition recovery mode
2024-03-15 12:57:01 -07:00
dependabot[bot] 230ada9c11
Bump the npm_and_yarn group group (#360)
Bumps the npm_and_yarn group group in /samples/Hello_Netherite_with_TypeScript with 2 updates: [durable-functions](https://github.com/Azure/azure-functions-durable-js) and [axios](https://github.com/axios/axios).


Updates `durable-functions` from 1.5.4 to 3.1.0
- [Release notes](https://github.com/Azure/azure-functions-durable-js/releases)
- [Changelog](https://github.com/Azure/azure-functions-durable-js/blob/v3.x/azure-pipelines-release.yml)
- [Commits](https://github.com/Azure/azure-functions-durable-js/compare/1.5.4...v3.1.0)

Updates `axios` from 0.21.4 to 1.6.7
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.21.4...v1.6.7)

---
updated-dependencies:
- dependency-name: durable-functions
  dependency-type: direct:production
  dependency-group: npm_and_yarn-security-group
- dependency-name: axios
  dependency-type: indirect
  dependency-group: npm_and_yarn-security-group
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-05 10:00:54 -08:00
dependabot[bot] 2215708925
Bump follow-redirects in /samples/Hello_Netherite_with_TypeScript (#340)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.9 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.14.9...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-05 09:35:24 -08:00
Sebastian Burckhardt e6487b9cec
update README and samples to latest veersion (1.4.2) of Netherite, update other dependencies in the samples, and use consistent taskhub names for all samples (#355) 2024-03-05 09:26:40 -08:00
David Justo 7921c1e4e5
Merge pull request #353 from microsoft/dev
Promote dev to main for 1.4.2 release
2024-02-15 18:54:15 -08:00
Sebastian Burckhardt 60c7d39e03
revise how dispose tasks are organized so that they can all use the same timeout check and error reporting mechanism (#330) 2024-02-12 16:20:10 -08:00
David Justo 0abc16d3e8
Add new logs to recovery path (#337)
* add simple log

* add detail column to FasterAzureStorageAccessCompleted

* refactor read range log

* pass just the position parameter instead of printing full details for FasterStorageAccessCompleted

* put position and id information into details

---------

Co-authored-by: sebastianburckhardt <sburckha@microsoft.com>
2024-02-12 16:19:40 -08:00
David Justo 66762a381d
rev version to 1.4.2 (#346) 2024-02-12 14:20:17 -08:00
Sebastian Burckhardt 2d3e5ea8e3
fix EH batch load cancellation issue (#347)
* do not cancel EH batch loading when partition is terminated, but only when EH processor shuts down

* Update src/DurableTask.Netherite/TransportLayer/EventHubs/EventHubsProcessor.cs

Co-authored-by: David Justo <david.justo.1996@gmail.com>

---------

Co-authored-by: David Justo <david.justo.1996@gmail.com>
2024-02-12 13:53:57 -08:00
Sebastian Burckhardt b2431e62f7
Enhance validation of deserialization and reporting of errors (#345) 2024-02-12 13:09:18 -08:00
Sebastian Burckhardt b9ed29ada7
update to FASTER v2.0.23 (#264)
* draft commit

* update to 2.0.23

---------

Co-authored-by: David Justo <david.justo.1996@gmail.com>
2024-01-16 10:23:30 -08:00
Sebastian Burckhardt b47eae7f40
Always release partition lease when a partition shuts down (#335)
* always release lease when a partition shuts down

* address PR feedback (add comment)

* recognize another empiric case of transient storage exception

* change default of partition checkpointing

* do not release lease if lease was already lost.
2024-01-03 09:48:52 -08:00
Sebastian Burckhardt fc777fed0a
Revise partition startup (#332)
* add "waiting" transition and timeout to partition startup

* address PR feedback
2023-11-03 11:03:18 -07:00
sebastianburckhardt f24f6de01a Merge remote-tracking branch 'origin/main' into dev
# Conflicts:
#	src/DurableTask.Netherite/StorageLayer/Faster/StoreWorker.cs
2023-10-24 13:33:05 -07:00
Sebastian Burckhardt bbbabbaeb1
update version to 1.4.1 (#328) 2023-10-24 13:22:45 -07:00
Sebastian Burckhardt 96b832eb5c
remove ? to prevent nullable warnings (#327) 2023-10-24 10:57:34 -07:00
Sebastian Burckhardt 7a6e760b59
Fix fencing of checkpoints (#326)
* fix fencing of checkpoints that caused occasional concurrent modification exceptions and can cause history corruption

* add more comments to explain sessions and fencing
2023-10-23 13:56:27 -07:00
Varshitha Bachu 76d01adabc
Copy ParentTraceContext when cloning ExecutionStartedEvent (#309)
* Update CreationRequestReceived.cs

* remove the faster-alternate data store and fix broken deserialization. (#308)

* fix and instrument commitlog serialization and deserialization. (#305)

* fix discard of packets so the next packet to expect is updated correctly (#302)

* New EventHubs performance tests (#178)

* rebase, and remove changes to scale monitor

* add hello cities test that prints the history of a nested orchestration

* implement a watchdog that terminates CompletePending quickly if it hangs (#318)

* add unique id to scale monitor constructor (#316)

* fix bugs from PR #302 (#315)

* fix cache size reporting (#321)

* added a comment

* update durable task package references to 2.15.1 and 2.12.0 (#317)

* update DT and DF package references

* update to latest

* sync dev w/ main (#324)

* Update GH automation (#303)

* initial commit (#290)

* Revert "initial commit (#290)" (#314)

This reverts commit 3a2d193e57.

* Bump Azure.Identity from 1.7.0 to 1.10.2 in /samples/TokenCredentialDTFx (#323)

Bumps [Azure.Identity](https://github.com/Azure/azure-sdk-for-net) from 1.7.0 to 1.10.2.
- [Release notes](https://github.com/Azure/azure-sdk-for-net/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.Identity_1.7.0...Azure.Identity_1.10.2)

---
updated-dependencies:
- dependency-name: Azure.Identity
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Terminate partition when FASTER refuses to checkpoint for over a minute (#301)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Varshitha Bachu <vabachu@microsoft.com>
Co-authored-by: Sebastian Burckhardt <sburckha@microsoft.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Sebastian Burckhardt <sburckha@microsoft.com>
Co-authored-by: David Justo <david.justo.1996@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-23 13:55:55 -07:00
David Justo fb4aa617ff
sync dev w/ main (#324)
* Update GH automation (#303)

* initial commit (#290)

* Revert "initial commit (#290)" (#314)

This reverts commit 3a2d193e57.

* Bump Azure.Identity from 1.7.0 to 1.10.2 in /samples/TokenCredentialDTFx (#323)

Bumps [Azure.Identity](https://github.com/Azure/azure-sdk-for-net) from 1.7.0 to 1.10.2.
- [Release notes](https://github.com/Azure/azure-sdk-for-net/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.Identity_1.7.0...Azure.Identity_1.10.2)

---
updated-dependencies:
- dependency-name: Azure.Identity
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Terminate partition when FASTER refuses to checkpoint for over a minute (#301)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Varshitha Bachu <vabachu@microsoft.com>
Co-authored-by: Sebastian Burckhardt <sburckha@microsoft.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-23 11:20:13 -07:00
Sebastian Burckhardt 08e760aeaf
update durable task package references to 2.15.1 and 2.12.0 (#317)
* update DT and DF package references

* update to latest
2023-10-23 11:19:37 -07:00
David Justo ba15eed262
Terminate partition when FASTER refuses to checkpoint for over a minute (#301) 2023-10-19 13:55:54 -07:00
dependabot[bot] 6effbf572c
Bump Azure.Identity from 1.7.0 to 1.10.2 in /samples/TokenCredentialDTFx (#323)
Bumps [Azure.Identity](https://github.com/Azure/azure-sdk-for-net) from 1.7.0 to 1.10.2.
- [Release notes](https://github.com/Azure/azure-sdk-for-net/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.Identity_1.7.0...Azure.Identity_1.10.2)

---
updated-dependencies:
- dependency-name: Azure.Identity
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-18 14:46:37 -07:00
Sebastian Burckhardt 76d7830f08
fix cache size reporting (#321) 2023-10-17 14:59:19 -07:00
Sebastian Burckhardt bdf943738b
fix bugs from PR #302 (#315) 2023-10-17 11:18:01 -07:00
Sebastian Burckhardt 2e86c3a8a5
add unique id to scale monitor constructor (#316) 2023-10-17 11:16:53 -07:00
Sebastian Burckhardt 4cc4ca72e3
implement a watchdog that terminates CompletePending quickly if it hangs (#318) 2023-10-17 11:16:18 -07:00
Sebastian Burckhardt 83d16d2bd9
New EventHubs performance tests (#178)
* rebase, and remove changes to scale monitor

* add hello cities test that prints the history of a nested orchestration
2023-10-12 14:22:33 -07:00
Sebastian Burckhardt eacb42fb0b
Revert "initial commit (#290)" (#314)
This reverts commit 3a2d193e57.
2023-10-11 10:58:20 -07:00
Varshitha Bachu 3a2d193e57
initial commit (#290) 2023-10-10 10:38:53 -07:00
Sebastian Burckhardt 2ae01f892b
fix discard of packets so the next packet to expect is updated correctly (#302) 2023-10-10 10:37:18 -07:00