* Initial code changes to throw 503 on 429/3092.
* Updated client retry policy. Added more tests to cover 429/3092.
* Code changes to update direct package version. Updating the tests.
* Code changes to refactor client retry policy.
* Minor code cleanup.
* Reverting the direct version bump up change.
* Code changes to address some of the review comments.
* Code changes to move failover logic in client retry policy.
* Minor code clean up.
* Code changes to clean up some cosmetic items.
* Further clean up.
* Code changes to address review comments.
* Minor refactor to address cosmetic update.
* Code changes to address cosmetic review comment.
* test improvements
* fixed other multiregion test
* requested changes
* test fix
* adds util function for multiregion setup
* fixed util method/class
* parallelize tasks
* checkin in
* support for both STJ and NSJ
* update contracts.
* name change PreviousLsn
* STJ TypeConverter support for ChangeFeedMetadata
* adding bacl StringEnumConverter
* test for Writes ChangeFeedMetadata
* removing DateTimeOffset as results are inconsistent.
* trying to get GMT, not local
* static UnixEpoch
* static qualifier in tests
* PropertyNameCaseInsensitive = false tests. copy of True tests.
* setting PropertyNameCaseInsensitive correctly for tests
* removed duplication for propertyNameCaseInsensitive tests
* remove JsonStringEnumConverter(), from tests
---------
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* changed hedge region to only show sucessfull region
* added and updated tests
* discussion changes
* suggestions and test improvements
* test fix
* updated hedge context
* fix test
* no string conversion
* requested changes
* suggested changes
* remove and sort usings
* updated nesting
* made trace more resillient
* trace as input
* added beginning of test
* update test
* added wait for global replication
* using for FI client
* Code changes to make STJ serializer public for preview.
* Code changes to make STJ serializer public for GA.
* Code changes to hide STJ serializer implementation behind a boolean flag.
* Code changes to fix tests.
* Revert "Code changes to fix tests."
This reverts commit c1ee171c22.
* Code changes to fix baseline test.
* Code changes to fix root cause.
* Code changes to update tests.
* Code changes to address review comments.
* Code changes to add serializer options as contract.
* Code changes to add serializer options as public contract in builder.
* Code changes to update contract to remove the STJ serializer boolean flag.
* Code changes to update summary.
* Code changes for minor cosmetic update.
* Code changes to fix xml comment.
* Code changes to move remarks to summary. Updated test.
* Code changes to move the validation logic in cosmos client options.
* Code changes to update contract.
* Code changes to remove client context core logic in cosmos client options.
* Code changes to update some tests.
* Code changes to update contract. Made cosmetic changes.
* Unit test coverage to validate Cosmos .NET SDK versioning strategy
* Updating logic to fix the build version comparision
* Renamed a method
* Great catch on exception catching, fixing it
* Using Tests Logger
* init
* Added tests
* Preliminary code
* baseline
* add ordering ignore to test cases
* upate baseline
* address code review 1
* Changed the way comparison between Anonymoustype object worked. Also added handling of multivalue case to call directly to leaf layer visitors, instead of going through the top level scalar expression visitor to avoid changing binding and context scope
* addressed code review
* address code review
* addressed missing field
---------
Co-authored-by: Minh Le (from Dev Box) <leminh@microsoft.com>
* Add a DistributedQuery gateway mode to the v3 sdk
Fix up bug in the continuation token and plumb the page size. Add an integration test
Make distributed query pipeline kick in only for document queries. Make it backwards compatible with older continuation tokens. Add integration tests.
Revert change in TransactionBatchOperationResult
Fix up missing include in OptimisticDirectExecutionQueryBaselineTests
* Incorporate code review feedback
Hack together a parity test for queries that require distribution
Add more partitioned parity tests
A tiny bit of code clean up
rename PaginationOptions to ExecutionOptions
Rename PaginationOptions to ExecutionOptions
rename pagination options to execution options
remove warning disable pragmas
Commit missed changes
* Fix up build issues due to bad merge
* Add another overload for NetworkAttachedDocumentContainer that does not take a distributed query client
* Incorporate code review feedback
* Fix up baseline to account for the class rename
* remove ExecutionEnvironment and the Compute flavor of the query pipeline stages
* Add changes that were missed in the last commit
* Remove GetCosmosElementContinuationToken from FeedIterator
* added MakeList and MakeSet aggregators
* Added MakeList and MakeSet to AggregateQueryTests.cs
* Adjust Aggregators
* Add Array Aggregate Continuation Token Test
* Added group by coverage for MakeList and MakeSet
* address comments
* cleaning
* Refactored test to better detect when to ignore result order
* cleaning
* cleaning, update baseline
* cleaning
* removed old comment
* cleaning/refactoring
* cleaning
* Add explicit cases to hit continuation token limit.
* Added additional case to GroupBy tests
* cleaning
* cleaning
* cleaning, updated baseline test
* cleaning, updated baseline test
* Added coverage to QueryPlanBaselineTests.cs
* refactored
* longrunning cfp avad test for ttl
* add more to comment
* internal set
* other tests fail with higher ms
* run updatecontracts
* Update Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/ChangeFeed/GetChangeFeedProcessorBuilderWithAllVersionsAndDeletesTests.cs
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* Update Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/ChangeFeed/GetChangeFeedProcessorBuilderWithAllVersionsAndDeletesTests.cs
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* Update Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/ChangeFeed/GetChangeFeedProcessorBuilderWithAllVersionsAndDeletesTests.cs
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* Update Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/ChangeFeed/GetChangeFeedProcessorBuilderWithAllVersionsAndDeletesTests.cs
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* using Logger.LogLine
* change back over to ManualResetEvent. fixed assert to look at Previous. keeping stopwatch just for logging. timeout at 5 minutes.
* Update GetChangeFeedProcessorBuilderWithAllVersionsAndDeletesTests.cs
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* try/finally
* removing ctor. making all set internal to address serialization issue. later PRs to test STJ de/serialization.
* fixing de/serialziation issue
* Update GetChangeFeedProcessorBuilderWithAllVersionsAndDeletesTests.cs
change timeout
* Update ReEncryption.csproj
1.11.4
* internal set change to CanWrite:True;
---------
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* Code changes to add default STJ serializer.
* Code changes to add tests for STJ serializer.
* Code changes to rename files.
* Code changes to clean up files.
* Code changes to add more tests.
* Code changes to update respective contracts.
* Code changes to add more documentation.
* Code changes to add LINQ tests. Addressing few review comments.
* Code changes to fix build failures.
* Code changes to update contract files.
* Code changes to add more LINQ specific test cases.
* Code changes to address review comments.
* Code changes to port some of the optimizations from Maya.
* Code changes to update the remarks since the changes from maya will cover the property naming policy.
* Code changes to address review comments.
* Code changes to update preview and GA contracts.
* Plumb the collection VectorEmbeddingPolicy to ServiceInterop
* Add query plan baseline tests for vector search
* Correct typo in the query for baseline test
* Fix build errors
* fix runtime issue in mock setup due to the extra argument for vector embedding policy
* simplify OrderByCrossPartitionQueryPipelineStage to remove state and be a static class.
* Add more test coverage for splits/merge and request charge
* Add more test coverage for splits
* incorporate code review feedback
* Code changes to mark internal vector interfaces as public.
* Code changes to update the public and preview modifiers.
* Code changes to update preview and ga contracts.
* Code changes to mark the policy definitions as public interface.
* Code changes to update GA contract and use #if all over.
* Code changes to fix some of the broken tests in preview.
* Code changes to update the preview contract too reflect the JSON Ignore.
* Code changes to upgrade direct version to 3.34.2
* Introduce an environment config to suppress sending NonStreamingOrderBy in the list of query features sent to the gateway
* Make sure that we dont leak state from the integration test. Add the suppression environment variable for the test pipeline since we run against an old emulator
* Incorporate code review feedback
* Add StoreResponseStatistics startTime to diagnostics
* Add latency in ms to make it easier to read.
* Converted to DurationInMs
* Remove start time
* Update baseline file.
---------
Co-authored-by: jakewilley_microsoft <--global>
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* Fix response message not parsing the index advice as text in latest sdk
* add test coverage
* address code review
---------
Co-authored-by: Minh Le (from Dev Box) <leminh@microsoft.com>
* SDK 3.40.0-preview.0: Adds version bump and changelog.
* SDK 3.40.0-preview.0: Adds version bump and changelog.
* SDK 3.40.0-preview.0: Adds version bump and changelog.
* change version for release; create diffs
* added query PRs to changelog
* add note for Rename refactoring of ChangeFeedItemChange<T> to changelog
* removing 4281 since it was mislabeled. it should have been treated as Internal
* grammar error
* formatting
* case change
* Update changelog.md
Co-authored-by: Justine Cocchi <jucocchi@microsoft.com>
* removing 4317 from change log. should be internal
* add new regions to changelog. also Direct Package version update
* fixed linked to DP PR
* more changes based on review
---------
Co-authored-by: Justine Cocchi <jucocchi@microsoft.com>
* Initial code changes to add vector embedding policy and index type in container.
* Code changes to add container builder for vector index type.
* Code changes to add and fix tests.
* Revert back client create and initialize test.
* Skipping some of the V2 tests for vector indexes.
* Code changes to fix GA and preview contracts.
* Code changes to address review comments part 1.
* Removed unnecessary JSON argument for vector index.
* Code changes to update contract changes.
* Code changes to update preview contract changes in tests.
* Code changes to address few review comments.
* Code changes to add few more tests to validate serialization and deserialization.
* Code changes to address some review comments for best practices.
* Code changes to fix test failures.
* sketch out improved ParallelPrefetcher; focus is on reducing allocations, but we also can't be substantially slower to start all tasks
* little more cleanup to further reduce allocations, and save a tiny amount of CPU
* start on testing
* some tweaks and testing for buffer management
* test exception handling; fix a bug in high concurrency case that would swallow exceptions
* test cancellation
* more testing, a little bit of cleanup
* test the case where the enumerator faults; fixes a couple leaks of Tasks and buffers that could occur in that some places
* tiny bit of cleanup
* cleanup and expand comments; code is tricky, it needs documentation
* address a whole bunch of style nits, just to keep compiler Message counts down
* address some feedback on comment clarity
* don't rely on finalizers for testing, it's too brittle; hold up was not allocating more in the non-test cases, but found a field to reuse; needs benchmarking
* complete ITrace proxy for testing
* style nits and a bit more commentary
* explicit test for concurrent access to the inner IEnumerator
* explicit test that IEnumerator is disposed
* explicitly implement all ITrace members
* address feedback: internal class members should be public or private
* address feedback: break test-only bits of ParallelPrefetch out into a partial
* address feedback: move const above type declarations
* address feedback: naming nits
* address feedback: use the existing NoOpTrace
* address feedback: remove pointless using
* update baseline trace text for QueryAsync test
---------
Co-authored-by: neildsh <35383880+neildsh@users.noreply.github.com>
* preliminary change
* Add some more boiler plate code
* move all linq test to the same folder; add some groupBy test
* fix references error in test refactoring
add code for group by substitution. Still need to adjust binding post groupby
* preliminary for the groupby functions with key and value selector
* trying to change collection inputs for group by
* WIP bookmark
* Successfully ignore "key"
* clean up code
* Sucessfully bind the case of group by with only key selector and no value selector followed by an optional select clause
* enable one group by test
* add support for aggregate value selector
* added baseline
* working on adding support for multivalue value selector and key selector
* code clean up
* more clean up
* more clean up
* update test
* Move test to separate file
* code clean up
* remove baseline file that got moved
* fix merge issue
* Changes test infrastructure to reflect changes from Master
* address code review part 1
* Address code review 2 and adds code coverage
* Addressed code review and added tests. Still a couple of bugs to iron out
* Fix group by translation issue and add more test
* update comments
* address pr comment
---------
Co-authored-by: Minh Le <leminh@microsoft.com>
Co-authored-by: Aditya <adityasa@users.noreply.github.com>
* Initial prep for adding a non streaming order by query pipline stage
* Change TracingAsyncEnumerator to be an adapter class between ITracingAsyncEnumerator and IAsyncEnumerator
* Move TracingAsyncEnumerator to product code
* Change ITracingAsyncEnumerator.MoveNext to take a CancellationToken, thereby removing the awkward method SetCancellationToken from IQueryPipelineStage
* draft implementation of non streaming order by pipeline stage
* Fix bug that drops first record on an initialized page
* Add a test class for non streaming order by unit tests
* remove unnecessary usings
* Add an emulator test for non streaming order by. Also lay groundwork for an alternative implementation for non streaming order by
* Remove the ResponseLengthInBytes property from QueryPage. This was being calculated incorrectly, and seems to be unused to boot.
* Add an ItemCount property to the Page class
* Add a multi level heap implementation for non streaming order by
* Add infrastructure for writing parity tests
* Fix a bug that caused the page enumerator to be dropped when we reach flat heap sizze limit
* When cloning OrderByQueryPartitionRangePageAsyncEnumerator as a fully buffered enumerator, make sure that we set the page size to maximum for the backend requests
* Simplify the non streaming pipeline stage, and add performance test
* Revert the changes for Headers.ItemCount Keep it as a string
* Avoid an allocation each time the OrderByItems property of OrderByQueryResult is touched
* Fix up the OrderByPipelineSatgeBenchmark to use fully materialized CosmosElements in the oages returned from MockContainer
* Add a few more unit tests for non streaming order by
* Add emulator tests for non streaming order by
* Add a few more integration test cases
* Fix up broken unit test
* Add more test coverage for the non streaming order by
* If there is no continuationtoken, assume that the response is streaming
* Add stronger validation to the non streaming order by unit tests
* Fix up broken unit tests to account for ItemCount
* fix up plumbing for index utilization, and incorporate code review feedback
* Minor clean up
* revert bug introduced in pursuit of more elegant code :)
* fix up broken unit test
* Fix up broken perf test
* Fix up broken IndexMetricsParserBaselineTest
* Minor bug fixes for OrderByCrossPartitionEnumerator
* preview cfp ffcf
* ran updatecontracts
* including this in Encryption
* fixing name onChangesDelegate
* sdkproject on encryptioncontainer
* try this again
* try, try, try again
* with impl
* Fixes Multi-Region CI pipeline to include env variable
* fixed space
* added validation test
* string conversion
* temp
* temp2
* Update Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/ClientTests.cs
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* Update Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/ClientTests.cs
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* changed env location
* temp
* test
* space
* remove env
* possible fix
* test fix
* clean up
* changed enviroment variable get
* test
* test fix
* ensure multiregion tests do not run with regular emulator tests
---------
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* test to prove no exception, no documents when change feed more is changed.
* renaming
* throwing exception when changeFeedMode change on CFP with same lease container
* removed PREVIEW tags under advisement. Well create a PR later
* removed usings
* revert to old summary
* defaulting change feed mode to latestversion
* change feed mode exception on switching and tests based on last recommendation.
* more changes based on discussions
* some refactoring
* fixed some tests
* more changes based on recommendations
* another refactor
* change from CosmosException to ArgumentException
* removed CosmosException from comments
* fixin tests to account for extra GetAllLeasesAsync call
* some recommendation changes
* adding a test to repro issue 4285
* fix plus e2e test
* not all request have cosmos exception
* add a quick unit test for ChangeFeedEstimatorIterator when Gone exception happens
* added a test description
* ignoring long running test. meant to be ran manually
* move long running tests to its own folder and category; check response and header for status/substatus
* one test category only, removed using, and exluding LongRunning category from build pipeline
* Update templates/build-test.yml
I am committing your suggestion. Also, the goal of this PR is not to executed long running tests. I will create a new PR for this. Because there are other unknowns that I need to answer. So for now, I just don't want it to be executed with the existing pipelines.
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
* change name of connectionString ENV to something more intention revealing
---------
Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com>
Co-authored-by: Sourabh Jain <sourabhjain@microsoft.com>
* disable network level tracing in sdk
* Revert "disable network level tracing in sdk"
This reverts commit 627cba6404.
* Including Client Configuration in request Diagnostics for Sync calls also
* initial change to support sending continuation as resume value in query spec
* serialize resume value in continuation token.
* fix unit test that was failing. other minor refactors
* additional refactoring
update exception messages
* few minor changes
* additional tests
* code refactoring
* Create separate top level object to represent SqlQueryResumeValue. Different types of resume values are private in this object
Consolidate all the visitor and serialization of resume value to SqlQueryResumeValue
Other code review comments
* handle different continuation token for target partition
* update test for continuation from array and object values. other minor code refactors
* save resume value as CosmosElement instead of defining a separate derived objects.
update cosmosundefinedquerytests. other code review comments.
* support on CosmosNumber64. Other number types are not supported for resume value. unit tests for SqlQueryResumeValue.
* update cosmosundefinedquerytests to include arrays and object in order by and group by tests. reducing the document count to reduce execution time.
* revert change to have separate SqlQueryResumeValue objects as it is better symmetry for comparing arrays and objects.
* group resume value into 3 types: Undefined, Primitive and Complex
* minor fix
---------
Co-authored-by: Bala Perumalswamy <balaperu@ntdev.microsoft.com>
Co-authored-by: Aditya <adityasa@users.noreply.github.com>
* Marking Distributed tracing feature as stable and changed an attribute name
* remove it from emulator test
* marked stable as false
* fix tests
* updated changefeedxml
* remove statuscode and subsstatus from staic list
* regenerated xml files
* remove event counters events
* Code changes to add regional endpoints for account metadata calls.
* Code changes to refactor some codes.
* Code changes to add unit tests.
* Code changes to make minor code clean-up.
* Code changes to fix tests. Refactored API.
* Code changes to refactor the enumeration logic inside global endpoint manager.
* Code changes to address review comments.
* Code changes to fix minor API parameter.
* Code changes to update the API naming.
* Code changes to update some attribute names.
* Code changes to refactor service endpoint creation logic.
* Code changes to address review comments.
* Code changes to address review comments.
* Code changes to update the API contract.
* Cosmetic code changes.
* Code changes to address review comments.
* porting Ian changes to new branch
* changing DynamicTests back to its original state. Create a new test class for AllVersionsAndDeletes
* including tests for AllVersionsAndDeletes DynamicTests
* get master version of DynamicTests
* testing changefeedprocessor with allversionsanddeletes
* tests for changefeedprocessor with allversionsanddeletes
* need to change name here also
* adding example documentation to the GetChangeFeedProcessorBuilderWithAllVersionsAndDeletes
* removed a line from example that was not necessary
* removed ChangeFeedMode from Create signature and using what is in ChangeFeedProcessorOptions
* undoing file
* default to LatestVersion, not Incremental
* change Incremental to LatestVersion on tests
* change access modifier and remove PREVIEW gate for internal classes
* more removal of unnecessary mode on Create for PartitionController and PartitionSupervisor
* change access modifiers and moving methods around due to PREVIEW
* adding GetChangeFeedProcessorBuilderWithAllVersionsAndDeletes to EncryptionContainer since it inherits Container
* running UpdateContract powershell
* override instead of abstract
* fixing override, but not sure about how this works
* taking guessing on this
* lets try this again
* once more
* once more
* internal on new change feed builder
* making it public to test
* internalizing the GetChangeFeedProcessorBuilderWithAllVersionsAndDeletes
* removed DecryptChangeFeedDocumentAsync
* undoing this file
* removing commented code
* moved to ContainerCore.Items.cs
* removing and undoing some files created or updated by UpdateContracts.ps1
* removed space and undoing a file
Debdatta Kunda
Nalu Tripician
Kiran Kumar Kolli
Philip Thomas
Matias Quaranta
Sourabh Jain
neildsh
Aditya
leminh98
Joris Eijmberts
Ezra Haleva
Maya Painter
hdornadula
Asier Isayas
Jake
joegoldman2
Taylor
kevin-montrose
jiajunpeng-msft
Bala Perumalswamy