* Increment package version after release of azure-storage-common
* Increment package version after release of azure-storage-blobs
* Increment package version after release of azure-storage-files-datalake
* Increment package version after release of azure-storage-files-shares
* Increment package version after release of azure-storage-queues
* more compatibility with expanding/contracting packages. add ability for packages to have DependentPackages that must be included in the set of packages that should be built given a changeset
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Storage/STG95 Files OAuth - full data plane support (#5759)
* File OAuth
* Remove comment
* update test record
* [Storage STG95 Branch] Merge Main to fix cmake build error (#5796)
* Update vcpkg SHA (#5757)
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
* Sync eng/common directory with azure-sdk-tools for PR 8528 (#5755)
* Fix default value for env vars in build-test-resource-config template
* Add empty pool condition
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Sync eng/common directory with azure-sdk-tools for PR 8516 (#5752)
* Ensure subConfigFiles is not an empty string
* Skip instances where $file is an empty string
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Sync eng/common directory with azure-sdk-tools for PR 8549 (#5762)
* commit the file changes so that we can see them running
* use standalone tool
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
* Disable codeql as it is causing build issues (#5761)
All the other languages have disabled codeql and we are now hitting blocking issues when running codeql on macos (https://dev.azure.com/twcdot/Data/_workitems/edit/138580) and even on the other OS's the configuration causes the builds to run for a long time.
* Delete eng/common/InterdependencyGraph.html which is unused. (#5763)
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Add missing include (#5766)
* Update GitHubEventProcessor version to 1.0.0-dev.20240708.1 (#5768)
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* Provide a walkthrough for how to install the Azure SDK packages in a VS (#5751)
MSBuild project via vcpkg.
* Sync eng/common directory with azure-sdk-tools for PR 8558 (#5764)
* Support storage network access and worm removal in remove test resources script
* Move storage network access script to common resource helpers file
* Improve storage container deletion resilience
* Plumb through pool variable to live test cleanup template
* Add sleep for network rule application
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* add the ability to override default succeeded() conditioning by parameter (#5780)
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Support regex/negative regex filters for stress test discovery. Add storage env defaults (#5779)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Update vcpkg SHA (#5772)
* Show-FailureLogs to include vcpkg build failure logs (#5776)
* Show-FailureLogs to include vcpkg build failure logs
* Add comment
* Add proper array syntax
* Use proper syntax to create an array even if there's only a single element
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
---------
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
* Turn federated auth on for Identity tests. (#5785)
* Turn federated auth on for Identity tests.
* Update test resources json.
* Move tests to use azure pipeline credentials (#5754)
* test1
* hgdfchg
* remove the remnants of azure client secret
* test KV with federated auth
* UseFederatedAuth
* fdsa
* kv template with managed
* try try again
* retry permissions
* add net acls
* blunt force replace the resource json
* put back stuff
* trey again with new method
* attempt
* missed something
* flip if else
* Temporarily use empty sub config file path for preview cloud
* remove client secret
* try to fix the identity tests
* live skip failing tests and return in samples
* samples for identity fix
* disable failing samples in identity
* fix winhttp failing test
* comment out code
* remove managed identity
* restore version from main
* revert readme changes
* PR comments
* test 2
* clang
* attempt default creds with pipeline chanined
* clangs
* identity test and clangs
* oops
* live
* cleanup
* reter
* test
* revert the DAC change
* missed one
* taking the samples to a farm upstate
* PR comments
* Fix bad merge
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
* Update the RetryPolicy for the GA release, keeping ShouldRetry extension point hidden. (#5771)
* Update the RetryPolicy for the GA release, keeping ShouldRetry extension
point hidden.
* Mark test helper virtual functions private, so they aren't
accessible/callable by callers.
* Update the changelog.
* Update CL.
* Azure Core July GA Release (#5792)
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
* Revert commits related to the new RetryPolicy method (#5793)
* Revert "Update the RetryPolicy for the GA release, keeping ShouldRetry extension point hidden. (#5771)"
This reverts commit 9ccd206ff8.
* Revert "Update the RetryPolicy and ShouldRetry customization logic to allow loosening the retry condition. (#5656)"
This reverts commit f1d95520d1.
* Do not remove changelog entry from a previous beta release
* Revert "Add a virtual ShouldRetry method to the RetryPolicy for customization. (#5584)"
This reverts commit ab90ef68b0.
---------
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
* Sync eng/common directory with azure-sdk-tools for PR 8598 (#5777)
* Set storage account test resources to disable blob public access
* Skip adding network rules to storage accounts that don't need them during cleanup
* Add succeeded check to set pipeline subnet info step
* Disable network firewall by default in resource creation/removal
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Increment package version after release of azure-core (#5794)
* Acknowledge community contribution in the changelog (#5797)
* Mention community contribution in the changelog
* cspell
* Remove double spaces
---------
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Robert Schulze <robert@clickhouse.com>
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
Co-authored-by: Ahson Khan <ahson_ahmedk@yahoo.com>
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
Co-authored-by: George Arama <50641385+gearama@users.noreply.github.com>
* Storage/STG95 Premium Files Paid Burst (#5795)
* paid burst
* update record
* Storeage/STG95 Files Binary ACE (#5824)
* feature code
* update test case to playback only
* update records
* fix cspell
* update record
* fix cspell
* fix cspell
* fix comments
* Remove copy opreation rest code.
* Storage/STG95 Support Sas StringToSign for debugging (#5830)
* StringToSign
* Update deprecate
* Update Func name
* ignore deprecated warning
* Add missing ignore
* fix clang format
* Storage/STG95 Resolve Archboard Review Comments (#5856)
* fix comments
* fix comments
* Add Create dir and upload from support (#5866)
* merge test recordings
* Remove other changes
* Remove deprecated (#5873)
* Remove others
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Robert Schulze <robert@clickhouse.com>
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
Co-authored-by: Ahson Khan <ahson_ahmedk@yahoo.com>
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
Co-authored-by: George Arama <50641385+gearama@users.noreply.github.com>
* Do not mark values as secret that are from git-hosted sub config files
* Simplify sub config build yaml
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Revert "Add support for providing a Resource ID to ManagedIdentityCredential. (#5832)"
This reverts commit 193bbe6963.
* Revert "Add support for customizing the IMDS endpoint within ManagedIdentityCredential using an env var. (#5834)"
This reverts commit 00304a0556.
* Use ClientAssertionCredential within AzurePipelinesCredential.
* Use ClientAssertionCredential in WorkloadIdentityCredential.
* Fix DefaultAzureCredentia.LogMessages test since an extra log got added.
* Disable tests that dont correctly simulate the token request and return
the test response.
* Address PR feedback and make sure base options are passed in to underlying
client assertion credential.
* Address PR feedback - move credential ctor into validation checks.
* Address PR feedback, add const.
* Add a ClientAssertionCredentialImpl to make sure logs use the calling
credential name.
* first build pass
* move models
Refactor KeyVault backup/restore client
This commit represents a significant refactoring of the Azure Security KeyVault Administration client library, transitioning from a generic `KeyVaultClient` to a more specialized `BackupRestoreClient`. The refactoring includes:
- Renaming `KeyVaultClient` to `BackupRestoreClient` in both `backup_restore_client.hpp` and `backup_restore_client.cpp` to better reflect the focused functionality on backup and restore operations.
- Moving model definitions from `backup_restore_client.hpp` to `rest_client_models.hpp`, including structures for errors, full backup, restore, and selective key restore operations, along with their options. This change centralizes model definitions for improved maintainability and reuse across the client library.
- Updating method implementations in `backup_restore_client.cpp` to align with the `BackupRestoreClient` class name and its specialized backup and restore operations.
These changes aim to provide clearer separation of concerns within the library, making it more manageable and focused on backup and restore functionalities for Azure KeyVault.
* Add BackupRestoreClient with tests and options
This commit introduces the BackupRestoreClient for Azure Key Vault backup and restore operations, along with BackupRestoreClientOptions for configuration. Significant refactoring within the Azure::Security::KeyVault::Administration namespace simplifies namespace usage by directly using `using namespace Models;`. The BackupRestoreClient has been enhanced with a new constructor accepting `vaultUrl`, a `TokenCredential` shared pointer, and optionally `BackupRestoreClientOptions`. Additionally, a copy constructor and a default destructor have been added, alongside a private member `m_apiVersion` for API version management.
The BackupRestoreClientOptions struct extends Azure::Core::_internal::ClientOptions with an `ApiVersion` member initialized to "7.5". Changes in the implementation file include additional headers for internal use and authentication, and updates to the constructor for initializing member variables and setting up the HTTP pipeline.
Test infrastructure has been enhanced with modifications in `CMakeLists.txt` to include test source files, the introduction of a base test class `BackupRestoreClientTest` for setup, and a test case `CreateClient1` demonstrating client creation.
These changes collectively enhance the Azure Key Vault Administration library by adding a new client for backup and restore operations, improving code organization, and setting up a testing framework.
* Enhance BackupRestoreClient and tests
This commit significantly updates the BackupRestoreClient and its associated models, introducing new features for backup and restore operations, and improving error handling with a new Error model. The `BackupRestoreClient` class has been marked as `final` to prevent inheritance, and several new methods such as `FullBackup`, `FullBackupStatus`, `FullRestore`, `RestoreStatus`, and `SelectiveKeyRestore` have been added, along with corresponding data models in `rest_client_models.hpp`. These changes enhance the client's functionality and API clarity. Additionally, the test infrastructure has seen substantial improvements, including refactoring in `backup_restore_client_base_test.hpp` for better encapsulation and updated utility methods for more effective testing. The practical application of these enhancements is demonstrated in `backup_restore_client_test.cpp` through the `CreateClient1` test case, showcasing the use of the `FullBackup` method. Overall, these updates bolster the Azure Key Vault Backup and Restore client library's capabilities and test support.
* Enhance BackupRestoreClient for Azure backups
- Updated `BackupRestoreClient` to support full backups and restores using Azure blob storage SAS tokens, including changes to method signatures for SAS token and job identifier parameters.
- Introduced nullable types for `StatusDetails` and `Error` in operation models to handle optional response fields.
- Removed `FullBackupOptions`, `FullRestoreOperationOptions`, and `RestoreStatusOptions` structures in favor of direct parameter usage in methods.
- Refined HTTP request construction to include necessary headers and dynamically set API version.
- Updated JSON parsing logic to handle nullable fields and adapt to changed response JSON structure.
- Enhanced test infrastructure with methods for creating test-configured `BackupRestoreClient` and generating SAS tokens for backups. Added new tests for full backup, backup status, full restore, and restore status functionalities.
- Adjusted URL path construction and query parameter setting in HTTP requests for updated API endpoints and parameters compatibility.
* Refine SelectiveKeyRestore API and tests
This commit overhauls the SelectiveKeyRestore function in the Azure Key Vault Backup and Restore client library. The function signature in `backup_restore_client.hpp` has been updated to replace the `SelectiveKeyRestoreOperationOptions` parameter with two distinct parameters: `keyName` (std::string) and `restoreBlobDetails` (SelectiveKeyRestoreOperationParameters). This change enhances the API's clarity and usability by explicitly requiring the key name and restore blob details for the operation.
Additionally, the `SelectiveKeyRestoreOperationOptions` structure has been removed from `rest_client_models.hpp`, aligning with the shift towards using direct parameters for the selective key restore operation. Corresponding implementation adjustments in `backup_restore_client.cpp` include changes to request URL and JSON body construction, as well as improved error handling in response parsing.
The test suite in `backup_restore_client_test.cpp` has also been updated. The `CreateClient1` test case was removed, and a new test case, `RestoreSelectiveStatus`, was added to demonstrate the updated API's functionality through a full backup and selective key restore operation scenario.
These changes collectively improve the API's intuitiveness and demonstrate its practical application through updated test cases.
* oops
* fds
* negative tests
Enhance backup/restore tests and cleanups
- Added a new test case `BackupFullErrorStatus` to the `BackupRestoreClientTest` suite to simulate a full backup operation with an invalid URI and verify the transition of backup status from "InProgress" to "Failed" with detailed error status.
- Refined the `RestoreFull` test case by removing unnecessary line breaks, improving code formatting without affecting functionality.
- Adjusted formatting in the `RestoreFullStatus` test by removing unnecessary braces and line breaks, streamlining the code.
- Improved readability in the `RestoreSelectiveStatus` test by adding a space after a comma.
- Introduced a new test case `RestoreSelectiveInvalidKeyStatus` to test the system's handling of invalid key restore requests, ensuring error reporting is accurate when keys or their versions are not found.
* recordings and pipeline
* linux build issues
* comment
* parse the error type
* fix paths
* ewrwe
* fdss
* reencode ansi
* fdsfsd
* Enhance samples and refine project structure
- Adjusted `CMakeLists.txt` for better project clarity, moving `BUILD_SAMPLES_HSM` from `test/samples` to `samples` and adding new sample directories for backup and restore functionalities.
- Updated `sample1_administration.cpp` to use `Azure::Core::_internal::Environment::GetVariable` for environment variable access and removed settings management code to focus on specific functionalities.
- Added new sample applications `sample2_full_backup_restore.cpp` and `sample3_backup_selective_restore.cpp` demonstrating full and selective backup and restore operations using the Azure Key Vault administration client library, including detailed steps and error handling.
- Included `CMakeLists.txt` for each new sample, specifying C++ standards and linking necessary libraries, with a focus on ease of use and configuration through the inclusion of `get-env-helper`.
* restore sample 1
* attempt reformat doc
* gfddgdf
* attempt 2
* are you serious ?
* treter
* docs
* typo
* changelog
* doc
* KHlAAAAAAAAAAANg said Kirk
* error check
* Updates to API
* clangs
* test new macro use in cmake file
* clang again
* missing include
* update1
* LROs
* clang 11
* attempt2
* fhkwerjhfkj
* documentation
* attempt++
* separating the op in its own files
* clang
* docs ++
* docc +++
* clang
* typo
* slkjlkdsjflks
* rename to backupclient
* hoping it's building
* PR comments
* address magic param check
* assets update
* Fix exception when an int32 value comes from the json
* Update serializers.cpp
Fix Pipeline errors
* Format using clang
* add more tests
---------
Co-authored-by: George Arama <gearama@microsoft.com>
* Add support for providing a Resource ID to ManagedIdentityCredential.
* Fix typo in param name.
* Fix clang format and undo removing of private methods (that are unused).
* Address PR feedback.
* Add back Azure:: to fix doxygen docs issue.
* Add support for customizing the IMDS endpoint within
ManagedIdentityCredential using an env var.
* Clean up the impl.
* Add imds as an opt-out for spell checl.
* Address PR feedback.
* missed a condition passthrough
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Ahson Khan
Azure SDK Bot
George Arama
microzchang
Anton Kolesnyk
Robert Schulze
arwell
Wes Haggard
JinmingHu
Larry Osterman
Scott Addie