* Upgrade Kusto SDK
Kusto SDK released a new major version in november 2022 with some breaking changes:
- The minimum JDK version for the SDK is now up to jdk 11
- Move to Blob Storage v12
- Remove org.json library to support secure systems that don't allow it. Some of the exception signatures have been changed to custom exception.
The latter change is the one having the most impacts on the connector: org.json is removed in favor of Jackson.
The KustoSinkConfig and KustoSinkTask were updated to take advantage of Jackson capabilities, in particular the mapping configuration is now deserialized to a POJO rather than doing operations on raw JSON objects.
* - leaner instantiation of the object mapper
- disable table access validation by default
- config exception on invalid mapping
- tests
- bump kafka clients version
* Update the documentation
* Cherry-pick changes to upgrade dependencies and fix deprecation warnings
* Format
* Rename KustoTableMapping to TopicToTableMapping
* * Fix review comments in CheckMarx
* * Fix code review comments
* * Update POM / Release notes
* Remove cases in tests where stackTrace calls were used
* * Fix minor observations.Handle exceptions when setting permissions and ensure that they are warn logged
* * Read test files from stream using resource API as opposed to path based access
* * Fix review comments
* * Fix code formatting
* Feature: Add Managed Identity support
azure-kusto-java added support for Managed Identities from version 3.0.0. This commit adds support of managed identities in the Kusto Sink.
There are now two ways to authenticate against Azure Active Directory:
- using an Azure AD Enterprise Application
- using a Managed Identity
Both are service principals, the only different is that Managed Identity used the Azure Instance MetaData Service (IMDS) to get OAuth2 tokens, without the need to use a password.
* Use an enumeration instead of a boolean to setup the way authentication should be done. This makes the configuration more trivial and the code is now less complex. This also allows to extend the authentication to other methods if needed.
Make the Tenant ID / Client ID optional for managed identities, only require it when validation of table access is enabled.
Update the README with some hints on managed identities.
* Fix issue#76. Make mapping optional
* Update README.md as per comments
* Add more tests
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
Co-authored-by: Tanmaya Panda <tanmayapanda@Tanmayas-MacBook-Pro.local>
Ramachandran A G
Ramachandran G
Tanmaya Panda
Tanmaya Panda
Abhishek Saharn
Adrien Bestel
Preston M. Price