* Azure Databricks python notebook pre-processing
Azure Databricks python notebook Data pre-processing for Azure Data Explorer
* Azure Synapse pre-processing example
Azure Synapse python notebook
Data pre-processing for Azure Data Explorer
* Update samples/src/main/synapse/pre-process-json-2-ADX.ipynb
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
* Update samples/src/main/synapse/pre-process-json-2-ADX.ipynb
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
* Update samples/src/main/synapse/pre-process-json-2-ADX.ipynb
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
* Update pre-process-json-2-ADX.ipynb
spark conf settings are not necessary, added a comment, that this is not needed for anonymous access, commented out the line (if customers want to access there data they need it, and notebook should show how to do so)
* Operationalize your Spark ML model into Data Explorer pool for scoring
Azure Synapse Analytics - Operationalize your Spark ML model into Data Explorer pool for scoring
Example notebook with ML model training on Spark and scoring on Data Explorer pool
Co-authored-by: ohad bitton <32278684+ohadbitt@users.noreply.github.com>
* Add support for Materialized Views by using new SetNewIngestionTime feature in such cases
* Fix command (SetNewIngestionTime is part of 'with')
* Verbiage fixes, per code review
* Per formatting request from Asaf
* Trivial change to trigger new CI build
* Revert "Revert "New option for the auto delete policy time - apart from the timeout with default of 7 days Make sure we use the same client request properties in all the flow Small bug fix - not allowed url to contain "ingest-"""
This reverts commit 9bccb17a
* better
* commetns
* move extents by batches
* move extents by batches
* debug
* debug
* writer clean
* better
* writer clean
* better count per yoni suggestion
* fix
Co-authored-by: KustoIbizaExtension Build <kustodev@microsoft.com>
* sample
* docs with strings
* Change authentication naming to app id and secret
* Change authentication naming to app id and secret
* Change authentication naming to app id and secret
* Change authentication naming to app id and secret
Co-authored-by: Ohad Bitton <ohbitton@microsoft.com>
* This commit has many changes.
Entered a breaking change that make the user specify SaveMode.Append as before we logged the specified save mode and ignored, docs were updated accordingly.
A few bugs were fixed :
timeout bug - was waiting 1 second for finish ?!
authentication failed when application only had reading permissions due to creation of ingestClient without needing it
ClientRequestProperties bug
Upgrades:
Reading parquet.snappy format as spark utilize it better
Ingestion is done once file is ready and not when all files are ready
Sas is saved and not created from the container object
Logging:
used to log only the exception message - now the full stacktrace is logged
* clientRequestProperties from new java version
* return the result!
* fix PR comments
* fix PR comments
* ClientRequestPropertiesAndIngestionProperties
* ClientRequestProperties And IngestionProperties
* split KustoOptions to KustoSourceOPtions and KustoSinkOptions
* fix maven compilation failure
* move kustoOptions to common package
Ramachandran A G
ohad bitton
Hauke Mallow
Manoj Raheja
Yihezkel Schoenbrun
Ohad Bitton
michazag
Nava Vaisman Levy
Nava Vaisman Levy