* Kusto now queries the provided cluster for metadata. This fixes incompatibility issues with other clouds, or services like Synapse. The results are cached and don't require any changes to the user's code or behavior.
* new API to get primitive as objects to handle null values
* nulls
* better
* comments
* comments
Co-authored-by: KustoIbizaExtension Build <kustodev@microsoft.com>
* Add streaming query API
* - Add some resource closing enhancements
- Wrap the CloseParentResourcesStream in a EofSensorInputStream, as opposed to the converse
* Add TODO to reuse HttpClient
* Use existing Java exception formatting, which is an improvement over the .NET exception
* Add new API to query so that the result is a json. When the caller doesn't need the Java object, this saves time converting to a Java object in the SDK and from the caller converting from the Java object.
* - Split input variables validation
- trim() command
* Rename from executeForJsonResult to executeToJsonResult
* Rename from executeForJsonResult to executeToJsonResult
* Streaming endpoint is always v1
* - Optimize and upgrade dependency versions
- Remove dependency duplicates/conflicts
- Resolve mvn warnings
* - Use range to ensure specific minor version of jackson dependencies
- Upgrade flatten-maven-plugin
* added classes for generating pojos from KustoResultSetTable instances
This commit includes three new classes:
1. KustoType which is a generic class for representing different Kusto
column types and bind them to different Java types
2. KustoColumn which represents a Kusto column - it has a name,
nullability information, and a type (KustoType)
3. KustoResultMapper which can be set up to parse KustoResultSetTable
instances and return a list of pojos. The setup phase is done using a
builder that takes the pojo constructor, as well as zero or more
instances of KustoColumn along with setter functions to set pojo's
fields based on the column values.
This mechanism can potentially be extended to use reflection based on
the target pojo class instead of the manual KustoResultMapper setup
* Update data/src/main/java/com/microsoft/azure/kusto/data/KustoType.java
Changed documentation link to international version
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
* Use String.format() for exception messages
* Introduced ObjectPopulator class, ordinal column mapping
- Unified KustoColumn and QueryResultColumnInfo into the new
ObjectPopulator class
- Added support for manually specified column ordinals
- extractList finds column indexes once per result set to save on column
lookups
* KustoResultMapper extractSingle - don't create empty objects
If the KustoResultSetTable is null, the proper response is to return a
null object, not an unpopulated one
* ObjectPopulator - always use column index, formatting
populateFrom used to get objects from the ResultSet by the column name
unless an ordinal was set. Now it always uses the ordinal (looking it up
if necessary).
* KustoType formatting - don't align field declarations
* KustoResultMapperTest
The test uses a mocked KustoResultMapper to test the pojo mapping
functionality of the KustoResultMapper
* Update KustoType.java
* Removed addNonNullableColumn/addNonNullableColumn, added javadoc
Instead of individual addNonNullableColumn/addNullableColumn methods,
exposed the more generic addColumn methods. Additionally, added javadoc
to document the addColumn parameters
* Renamed ObjectPopulator to KustoResultColumnPopulator
* KustoResultColumnPopulator::populateFrom use passed ordinal
populateFrom used to always lookup the column ordinal by name instead of
using the passed ordinal. Now it's always using the ordinal, moving the
responsibility of correctly determining that ordinal to the caller.
* renamed variable in KustoResultColumnPopulator::populateFrom
* KustoResultMapper::extractList renamed variable
* added "single" to names of tests of KustoResultMapper::extractSingle
* Javadoc fixes in KustoType and KustoResultMapper
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
* Rearchitect the AAD authentication code to be object oriented and follow best-practices (Java and otherwise)
* Rearchitect the AAD authentication code to be object oriented and follow best-practices (Java and otherwise)
* Code review comments:
- Rename to CallbackTokenProvider
- Remove 2 unnecessarily-passed authorityIds
* Add another layer of abstraction (MsalTokenProvider)
* Add another layer of abstraction (MsalTokenProvider)
* Add Override annotation per code review comment
* Convert from ADAL to MSAL (include a few questions to be resolved during code review)
* Resolve some TODOs
* Some minor changes and some changes from code review
* Incorporate most review comments
* Add '/' to authority to resolve issue.
* Per Yochai's request, first try to access the token silently, and if that fails authenticate from scratch
* Make the method determineAadAuthorityUrl() less confusing
* Fix error
* Fix test
* remove federated url path if set
* ignore case
* better
* test
* fix test
* raw format
* comments
* comments
Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
* Streaming client wrong endpoint detection
* Fix invalid character for the file's encoding
* Add parent class to both Queued and Streaming IngestClients for the local purpose of removing significant code duplication in the wrong-endpoint-detection feature.
This change represents a better architecture that will yield further benefits.
* Rename AbstractIngestClient to IngestClientBase
Adds ManagedStreamingIngestClient to support multiple streaming retries and queue fallback.
commits:
* First commit, create file and basic members
* basic copy-paste argument validation, close streams on close.
no logic yet
* Copy ctors for all the needed types
* Filled out other methods (Like in the c# manager, all of them redirect to stream, except blob which goes directly to the queued ingestor)
* Simplest support for actual streaming
* Fixed with new name for the queued client
* Removed unsued stuff for now, added doc comment.
* Some small fixes from testing -
- Log exception
- Add a direct c-tor
- Catch all exception types, since they may be unpredictable
* Reset the stream after an exception
* Adding tests for the class
* cleanup imports
* Added annotations as an explicit dependency(like in data) to appease CI.
* Update ingest/src/main/java/com/microsoft/azure/kusto/ingest/ManagedStreamingIngestClient.java
grammar
Co-authored-by: boazsha <15858748+boazsha@users.noreply.github.com>
* Update ingest/src/main/java/com/microsoft/azure/kusto/ingest/ManagedStreamingIngestClient.java
grammar
Co-authored-by: boazsha <15858748+boazsha@users.noreply.github.com>
* Change retry count to 3
* Made the json file tests different from each other, plus they now check the stream.
The gz file was changed since the old one had CRLF line endings, while the uncompressed one had LF.
* Documentation - added a note to explain ingestFromBlob's behaviour, and some tiny grammar fixes.
* Removed default compile annotation
* Update ingest/src/main/java/com/microsoft/azure/kusto/ingest/ManagedStreamingIngestClient.java
Co-authored-by: Boaz Shalom <15858748+boazsha@users.noreply.github.com>
* Removed todo comments from tests, since we decided to keep them this way
* Pass the to the StreamSourceInfo ctor whenever applicable
* Make comments fit the style
* Added doc for ingestFromBlob function
* Added factory for managed streaming ingest client
* Fixed managedingest from files - since FileInputStream isn't resettable, we can't use managed with it, so I added a class for it and a test.
* Reset the mock properly
* Add license
* Parsing permanent exceptions from the api (marked as @permanent=true in the returning json) and stopping if it is one.
* Reset the client in all tests
* Made compressedJson test work despite the fact that the stream closes in the end
* Adding debug prints to the comparison test since I don't know how else to check tests that run on my machine but fail on the testing machine
* Try using alternative method of reading string to avoid newline issue
* Also fixed the other one to make the tests pass, but we'll still have to look into why the file changes on the test machine
* Add UncloseableStream class
* Fixed typo
* Solve the newlines problem by hardcoding the string result into the test file.
* Use UncloseableStream in utils (still needs tests)
* Update ingest/src/main/java/com/microsoft/azure/kusto/ingest/ManagedStreamingIngestClient.java
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
* Use proper reset function in tests
* Unit test UncloseableStream
* Correct format for web error
Co-authored-by: boazsha <15858748+boazsha@users.noreply.github.com>
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
* Add detection of wrong endpoint with correct endpoint URI suggestion
* Fix code review comments
* Remove extraneous comment from test
* Queued Ingestion's wrong endpoint detection is more generic to reduce code duplication if the same feature is added for Streaming Ingestion.
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
* reduced wait time on Err
* reduced wait time on Err
* test reduced wait time on Err
* reduced wait time on Err
Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
AsafMah
KustoIbizaExtension Build
ohad bitton
Yihezkel Schoenbrun
maltalex
Ohad Bitton
ohbitton
vladikbr
Michael Spector
Yochai Gilad
amshalev