Граф коммитов

155 Коммитов

Автор SHA1 Сообщение Дата
George Banasios d1db64a1be
Removes Apache HTTP URIBuilder (#397)
* removes apache http URIBuilder

* reverts to ParseException
2024-11-14 11:43:11 +02:00
Yihezkel Schoenbrun ab2910233c
Remove unused timeout from HttpClientProperties and ResourceManager (#395)
* Miscellaneous minor improvements

* Remove timeout from ResourceManager and separate timeout from HttpClientProperties

---------

Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
2024-11-12 17:27:43 +02:00
ohad bitton 84218c18c1
Managed streaming - better estimation for decision for streaming vs queuing (#368)
* stream source info size

* some good code, unfished

* manaed streaming queuing policy per format

* ser raw size for stream

* remove todo

* format

* fix test

* print exception message on last retry

* used undeclared jackson dep

* format

* add comment

* good comments

* better

* changelog

* format

* comments - E2E

* changelog

* comments

* comment

* fix

* last comments

---------

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2024-11-10 14:23:52 +02:00
ohad bitton b7d2477f08
New client missing feats (#393)
* remvoed maxConnectionsPerRoute
some missing pieces

* timeouts

* timeouts

* format + changelog

* better

* fix test

* revert

* revert

* clean

* format
2024-11-07 18:08:22 +02:00
Cole d73eeb3663
Implements azure core http client (#388)
* Introduces KustoQuery wrapper object

* working on async client

* updating client with KustoQuery object

* updating client with KustoQuery object

* updating client with KustoQuery object

* will simplify later.

* will simplify later.

* will simplify later.

* ran formatter, fixed test, fixed bad merge

* AtomicBooleans in already modified test. Save some lines, might as well.

* hide KustoQuery object

* rename and hide KustoRequest

* adding E2ETests for async queries

* ran formatter

* no complete in sink

* move blocking operations off the io thread

* run formatter

* replace execute utilization

* fixes teardown. async still doesn't work yet.

* fixes msal4j issue

* build.yaml

* Introduces KustoQuery wrapper object

* working on async client

* updating client with KustoQuery object

* updating client with KustoQuery object

* updating client with KustoQuery object

* will simplify later.

* will simplify later.

* will simplify later.

* ran formatter, fixed test, fixed bad merge

* AtomicBooleans in already modified test. Save some lines, might as well.

* hide KustoQuery object

* rename and hide KustoRequest

* adding E2ETests for async queries

* ran formatter

* no complete in sink

* move blocking operations off the io thread

* run formatter

* replace execute utilization

* fixes teardown. async still doesn't work yet.

* fixes msal4j issue

* rebased

* rework azure branch

* remove unused dependencies

* fix some more issues with imports

* add default timeout 10 min

* re-add readme

* modified changelog

* modified changelog

* removes krL

* remove async test file

* remove closeable from query clients

* fixes pom version change and removes unused reactor property

* CI_EXECUTION

* storageaccountset guard clause

* close clients in E2ETest

* remove extra assertions from teardown

* remove reactor step verifiers for now

* readd deflate to cloudinfo

* renoved reactor test library for now

* run formatter

* added fixme

* move URIBuilder to further facilitate removal of Apache

* add more detail to assertion failure

* Made InvalidConnectionString extend DataClientException so users can optionally catch the more specific exception

* commenting everything async until implementation works

* commenting everything async until implementation works

* ran formatter

* remove unused Apache classes

* removed commented code

* move message to appropriate assertion

* removed last of the commented code and ran formatter

* readd the auto-delete policy

* make request object usage private

* put back more Apache things.

* renove trailing space

* Update BaseClient.java

---------

Co-authored-by: cole <cole.snyder@gmork.tech>
Co-authored-by: asafmahlev <asafmahlev@microsoft.com>
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
2024-10-27 15:11:44 +02:00
ohad bitton 9ff0fb77b5
HttpClient - Proxy planner support (#370)
* working

* comments - E2E

* comments - E2E

* move SimpleProxyPlanner to test

* fix proxy test

* fix build

---------

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2024-08-26 17:12:24 +03:00
Yihezkel Schoenbrun 723e09e7e3
Stop tests from depending on table row store being sealed, to shorten E2E runtime, and prevent timeouts even with the new HTTP client. (#380) 2024-08-07 10:11:40 +03:00
Yihezkel Schoenbrun 374d6fbe8c
Fix client app and version tracing (#371) 2024-07-25 11:27:44 +03:00
AsafMah f54178f5df
Revert new HTTP client (#367) 2024-06-25 11:19:34 +03:00
Yihezkel Schoenbrun 736113800d
Prevent (theoretical) NPE in reportIngestionResult (#361) 2024-05-21 22:48:24 +03:00
Yihezkel Schoenbrun e87f012ee0
Retry loading ResourceManager resources in all cases (#349)
* Retry loading ResourceManager resources in all cases

* Fix build

* Fix tests

---------

Co-authored-by: AsafMah <asafmahlev@microsoft.com>
2024-05-20 13:05:04 +03:00
Cole 626edb2707
bumps dependencies, removes explicit reactor and msal4j dependency (#357)
Co-authored-by: cole <cole.snyder@gmork.tech>
Co-authored-by: AsafMah <asafmahlev@microsoft.com>
2024-05-05 14:06:30 +03:00
asafmahlev 02925fa998 robust check 2024-04-14 11:35:25 +03:00
AsafMah 1eef873847
secretless (#360) 2024-04-11 10:18:20 +03:00
t-ronmoneta 2b49b63feb
App and user for tracing (#359)
* Added kcsb's app for tracing and user for tracing to queued ingest client and ingestion properties

* Changed setter\getters permissions in ingestionProperties

* Removed tracing from ingestionProperties

* Pr comments and created a clientdetails to handle deafults
2024-04-08 13:21:20 +03:00
ohad bitton d112a99e41
remove unused netty dependency (#356)
Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2024-03-24 14:27:37 +02:00
Cole 0057e8f061
Removes apache http libraries and replace with azure core httpclient (#335)
* initial commit removing apache http libraries

* updated MSAL wrapper class to include headers in response

* Added TestHttpResponse class to replace BasicHttpResponse in Tests

* Added KustoParseException to replace ParseException from old Apache Client.

* removed todo regarding ParseException

* removed EofSensorInputStream

* added fixme to replace EofSensorInputStream

* Return of the Apache

* Fixed UtilitiesTest
Removed cast to Closeable in CloudInfo

* return UriBuilder to UriUtils

* add proxy options back to HttpClientFactory

* ran formatter

* closeable and fix build

* fix test

* format

* removed extra throws

* Per Ohad review comments:
-Adds custom HTTP status codes
-Integrates status codes
-Uses Context.NONE static factory

* removed caught exceptions that were rethrown with no changes/side effects in QueuedIngestClientImpl per review comments

* removes block on body by using BinaryData
removes Entity references

* ran formatter

* added Tracing object and builder to shorten method signatures
added HttpRequestBuilder to separate request building logic from execute methods in the client, providing better testability
Created BaseClient class to host the inner httpClient
Put post methods on BaseClient so no need for utils class
removed HttpPostUtils.java
updated impacted tests
left notes on Keep-Alive header in ClientFactory
ran formatter

* puts old URIBuilder back in place

* update readme with ProxyOptions

* aligns with PR 342

* fix issue with Closeable

* ran formatter

* Fix sync HttpWrapper implementation

* fix build

* format

* build

* Few bug fixes
Build works also with OKHttp providers

* readd the exclusion of commons-codec for security posture

* adds response timeout and client provider to the client properties wrapper

* removes references to Netty

* removes extra comment

* adds Keep-Alive back to prevent breaking change for users of Keep-Alive. updates CHANGELOG.md
ran formatter.

* removes deflate algorithm from encoding header

* revert removing Apache core (URIBuilder depends on utilities in core)

* remove netty from client factory

* fix

* removes CloseParentResourcesStream.java

* reusable Gzip logic

* factory now uses azure-core default http client options

* I always forget the formatter

* fix test

---------

Co-authored-by: cole <cole.snyder@gmork.tech>
Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
2024-03-24 14:04:17 +02:00
Yihezkel Schoenbrun 4d66a3c760
Fix ConcurrentModificationException (#353)
* Fix ConcurrentModificationException

* Black

* Asaf was right
2024-02-27 02:32:43 +02:00
Yihezkel Schoenbrun 433bf6e293
Fix integration tests that should be unit tests (#348)
* Fix integration tests that should be unit tests

* Fix integration tests that should be unit tests

* Black

* Add unit-ish test for autoCorrectEndpoint boolean

* Black
2024-02-05 18:34:59 +02:00
ohad bitton 1a34fd40da
Resource manager can return an empty list (#340)
* test fail

* test good

* format

* fix resource manager

* fix resource manager

* ResourceManagerTest fix

* ResourceManagerTest fix

* ResourceManagerTest fix

* tyr not to hang the tests

* format

---------

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2024-01-23 12:43:40 +02:00
Yihezkel Schoenbrun 612e9930fc
Fix excessive test runtime (#344)
* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test without these classes

* Test Ohad's proposal

* Test Ohad's proposal
2024-01-23 08:46:07 +02:00
t-ronmoneta 09b65d03be
T ronmoneta/compression auto detect fix (#337)
* Changed DataFormats to not compress if needed

* Changed CHANGELOG.md
2024-01-21 14:12:34 +02:00
ohad bitton ae1889f44e
better reading streaming sample (#338) 2024-01-16 11:16:13 +02:00
ohad bitton 88ddc35cde
Transient IOError (#331)
* transient IOError

* transient IOError

* move getObjectMapper back to utils

* revert unrelated

* revert unrelated

* fix warning

* revert HttpClientProperties move to http folder

* try again

* format

* disable managed streamign test

* timeout test

* t

* dis q t

* redo test

* redo test

* no retry in cloud test

* no retry in cloud test

* fix vavr not used

* stop it !

* stop it !

* stop it !

* queuedClient test again?

* no managed , queuedClienttest minus ingestFromBlob_IngestionReportMethodIsNotTable_EmptyIngestionStatus

* format

* add again ingestFromBlob_IngestionReportMethodIsNotTable_EmptyIngestionStatus

* now with retries?

* retries not vavr?

* f

* nicer

* nicer

* nicer

---------

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2023-11-27 15:25:23 +02:00
AsafMah af012a65a0
Fixed bug in ranked buckets (#329) 2023-11-02 09:30:41 +02:00
Yihezkel Schoenbrun 061818ffb9
Fix method name capitalization and other non-functional changes (#328)
* Fix capitalization of method names, and other non-functional improvements

* Fix capitalization of method names, and other non-functional improvements

* Fix black

* Revert breaking change

* Additional change required after merge
2023-10-29 14:44:11 +02:00
Tanmaya Panda 1398520aa6
add connection string auth (#319)
* add connection string auth

* removed keywords

* manual execution false

* addressed review comments

* replaced with string utils blank

* fixed javadoc error
2023-10-19 11:29:33 +05:30
AsafMah bfa2b6aa21
Added ranked buckets (#323) 2023-10-17 08:30:34 +03:00
ronmonetaMicro eff584c119
Added autocrrectendpoint usage (#296)
* Added autocrrectendpoint usage

* Added autocrrectendpoint usage

* Added isreserved hostname support and tests

* Maven reforamtting

* fixed clusterurl fix

* Following PR notes - added more tests, revamped isReservedHostName and fixed documentation

* With Formatting

* Changed from var to actual types

* Fixed IPV6 scenario

* Fixed formatting

* Fixed blocking on non local addresses

* Moved tests from e2e to unittests files + Other PR notes anout deprecated functions

* Added more test cases
2023-08-13 12:47:56 +03:00
Cole 3b63ae4915
more dependency bumps! more! (#316) 2023-08-13 12:32:23 +03:00
ohad bitton 5a89292d5e
Dont use timer if closed (#312)
* dont use timer if null

* dont use timer if null

* UserPromptTokenProvider public

* synchronized PublicAppTokenProviderBase

* synchronized PublicAppTokenProviderBase

* format

* add comment

* add comment

---------

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2023-07-26 17:35:30 +03:00
AsafMah e2b288d13a
Queued Ingestion Client now supports retries (#308)
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
2023-07-24 10:20:43 +03:00
Tanmaya Panda 79cc872763
fixed jackson transient serialization bug and handled HTTP status 204 in HTTPResponseWrapper (#310) 2023-07-17 07:51:12 +03:00
Eitan Moed fae7704426
User/t eitanmoed/distributed tracing (#298) 2023-06-26 15:02:47 +03:00
ohad bitton e908bff85f
Guess who's back - Java 8 is back again - tell a friend (#307)
* java 8 and new bom

* changelog and version

* fix

* fix

* comments

---------

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2023-06-15 18:08:00 +03:00
ronmonetaMicro 471d47b3f2
ignore first record support (#304)
* Added ignore first record support + flag whether to test on managed streaming ingest or not

* Update ingest/src/test/java/com/microsoft/azure/kusto/ingest/E2ETest.java

Co-authored-by: AsafMah <asafmahlev@microsoft.com>

* Update ingest/src/test/java/com/microsoft/azure/kusto/ingest/E2ETest.java

Co-authored-by: AsafMah <asafmahlev@microsoft.com>

---------

Co-authored-by: AsafMah <asafmahlev@microsoft.com>
2023-05-15 10:22:20 +03:00
ohad bitton 7c2a1416da
Nicer code for managed streaming (#303)
* nicer code

* f

* fix test

* forat

---------

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2023-05-08 09:45:48 +03:00
ohad bitton 82c27101c8
Add the new Storage exceptions to methods signatures (#299)
* The new Storage implementation hides the exceptions thrown in annotations. such as         @UnexpectedResponseExceptionType(TableServiceErrorException.class)

* fix test

* better log

---------

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2023-05-03 14:02:44 +03:00
ohad bitton 01b608e6e7
Streaming ingest from blob storage (#288)
* working and nice

* fix tests

* format

* revert samples

* test streaming from blob

* revert revert test

* test

* format

* fix test

* revert

---------

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
2023-04-17 13:38:38 +03:00
AsafMah e478801857
Replace non-ascii characters in headers to be in line with the service. (#292)
* Replace non-ascii characters in headers to be in line with the service.

* Use a more concise method
2023-04-16 13:35:10 +03:00
AsafMah 9540e93da3
No redirects by default (#291) 2023-03-30 11:50:16 +03:00
AsafMah 31c603082f
Added props (#281)
* Revert "Revert "Added props""

This reverts commit f40ad30d39.

* Fixed NPE

* Added tests

* Formatting

* Tests again

* Fixed NRE

* Upgrade mockito to fix tests

* Added according to spec

* Fallback to env

* Align name

* format

* format

* format

* Fixed tests

* Some comments

* Moved header logic away.

* Formatter

* Fixed null

* Removed append method

* FFormat

* remove unrelated test

* Optimize imports

* Align param order with .net

* format

* Naming

* Use real url

* PR fixes

* public

* format

* Update Utils.java
2023-02-12 14:01:05 +02:00
asaharn 4e588debb6
IngestionStatusInTableDescription to support serialization (#287)
* TableClient as transient to support serialization
*Rebuilding TableClient in case of deserialization
* version change to 4.0.2
2023-02-10 14:06:16 +05:30
AsafMah c521890cb9
Fixed java wrapper for http props (#283) 2023-01-16 11:40:11 +02:00
ohad bitton 6147035e83
Move to Blob Storage v12 and azure bom versioning (#240)
* storage v12

* storage v12

* complete - working

* STORAGE v12

* STORAGE v12 works and tested

* tests revert

* tests revert

* try pom

* try bom 1.0.6 - fails for reactor netty http

* bom 1.0.5 fails for netty for unknown reason - version match!

* bom 1.0.5 fails for netty for unknown reason - version match!

* revert me

* better

* merge it better

* merge better

* cleaner

* should work

* fix bom

* bom 1.2.4

* closeable clients

* merge with master, format and clean

* merge with master, format and clean

* test

* fix tests

* fix tests

* fix not working from stream

* work with older bom as well

* format

* fix verify

* fix ingest

* fix comments

* merge

* merge

* Fix ingestion status

* Fix e2e

* Fix ResourceManagerTest bug

* fix unused ependency

* revert

* format

* comments

* format

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
Co-authored-by: AsafMah <asafmahlev@microsoft.com>
2022-11-03 18:18:18 +02:00
Tanmaya Panda c3bc04358d
Switch to jackson library from org.json (#272)
* switch to jackson for json

* added formatter

* fixed failing test cases

* fixed testcases

* ran formatter

* added jackson-core to pom

* fixed json string

* fixed string conversion

* bumped release version

* bumped up release version

* addressed Ram's review comments

* formatter changes

* refactored object mapper

* addressed Ram's review comments

* formatter changes

* changes done as per Ohad's comments

* added exception testcase for ResultSetTest

* added bigdecimal for precision in decimal values

* added comment for bigdecimal deserilaization

* made options and parameters mandatory in tojson

* fixed test case

* accept cla

* added formatter

* commit for cla

Co-authored-by: AsafMah <asafmahlev@microsoft.com>
2022-10-27 12:56:38 +05:30
AsafMah e2059e7578
Automatic conversion of query/ingestion urls in ingest clients (#268) 2022-09-13 09:56:54 +03:00
AsafMah b99a30180b
Revert "Automatic conversion of query/endpoint urls"
This reverts commit 6880ed9a0f.
2022-08-18 12:39:57 +03:00
AsafMah 6880ed9a0f
Automatic conversion of query/endpoint urls 2022-08-18 12:39:40 +03:00
ohad bitton a66e8bef26
Verify endpoints (#263)
* start

* matcher

* clean

* matcher

* dragt

* not work

* Tests and all works well

* Tests and all works well

* clean all dirt

* fix format

* fix formatClientImpl

* databind conflict

* Formatting

* fix NPE
comments

* comment

* revert e2e

* fix NPEs !!

* format

Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
Co-authored-by: AsafMah <asafmahlev@microsoft.com>
2022-08-11 15:42:33 +03:00