From 90d70928d9c738923afe5d08557e0a61c9c7188d Mon Sep 17 00:00:00 2001 From: SJ Date: Fri, 7 Aug 2020 12:25:11 -0700 Subject: [PATCH] Update documentation for the current release (2.3.17) (#530) --- .github/CONTRIBUTING.md | 2 +- README.md | 24 ++++++++++--------- docs/PySpark/structured-streaming-pyspark.md | 8 +++---- docs/spark-streaming-eventhubs-integration.md | 4 ++-- ...uctured-streaming-eventhubs-integration.md | 8 +++---- 5 files changed, 24 insertions(+), 22 deletions(-) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 753f00c9..83a5c070 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -18,7 +18,7 @@ run all unit/integration tests and build a JAR. ### SBT Dependency - libraryDependencies += "com.microsoft.azure" %% "azure-eventhubs-spark" %% "2.3.16" + libraryDependencies += "com.microsoft.azure" %% "azure-eventhubs-spark" %% "2.3.17" ## Filing Issues diff --git a/README.md b/README.md index 2a9fef06..e81fb5ae 100644 --- a/README.md +++ b/README.md @@ -30,21 +30,23 @@ By making Event Hubs and Spark easier to use together, we hope this connector ma #### Spark |Spark Version|Package Name|Package Version| |-------------|------------|----------------| -|Spark 2.4|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.16-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.16%7Cjar)| -|Spark 2.4|azure-eventhubs-spark_2.12|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.16-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.12%7C2.3.16%7Cjar)| -|Spark 2.3|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.16-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.16%7Cjar)| +|Spark 3.0|azure-eventhubs-spark_2.12|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.12%7C2.3.17%7Cjar)| +|Spark 2.4|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.17%7Cjar)| +|Spark 2.4|azure-eventhubs-spark_2.12|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.12%7C2.3.17%7Cjar)| +|Spark 2.3|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.17%7Cjar)| |Spark 2.2|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.2.10-blue.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.2.10%7Cjar)| |Spark 2.1|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.2.10-blue.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.2.10%7Cjar)| #### Databricks |Databricks Runtime Version|Artifact Id|Package Version| |-------------|------------|----------------| -|Databricks Runtime 6.X|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.16-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.16%7Cjar)| -|Databricks Runtime 6.X|azure-eventhubs-spark_2.12|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.16-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.12%7C2.3.16%7Cjar)| -|Databricks Runtime 5.X|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.16-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.16%7Cjar)| -|Databricks Runtime 5.X|azure-eventhubs-spark_2.12|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.16-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.12%7C2.3.16%7Cjar)| -|Databricks Runtime 4.X|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.16-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.16%7Cjar)| -|Databricks Runtime 3.5|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.16-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.16%7Cjar)| +|Databricks Runtime 7.X|azure-eventhubs-spark_2.12|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.12%7C2.3.17%7Cjar)| +|Databricks Runtime 6.X|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.17%7Cjar)| +|Databricks Runtime 6.X|azure-eventhubs-spark_2.12|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.12%7C2.3.17%7Cjar)| +|Databricks Runtime 5.X|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.17%7Cjar)| +|Databricks Runtime 5.X|azure-eventhubs-spark_2.12|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.12%7C2.3.17%7Cjar)| +|Databricks Runtime 4.X|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.17%7Cjar)| +|Databricks Runtime 3.5|azure-eventhubs-spark_2.11|[![Maven Central](https://img.shields.io/badge/maven%20central-2.3.17-brightgreen.svg)](https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-eventhubs-spark_2.11%7C2.3.17%7Cjar)| #### Roadmap @@ -59,13 +61,13 @@ For Scala/Java applications using SBT/Maven project definitions, link your appli groupId = com.microsoft.azure artifactId = azure-eventhubs-spark_2.11 - version = 2.3.16 + version = 2.3.17 or groupId = com.microsoft.azure artifactId = azure-eventhubs-spark_2.12 - version = 2.3.16 + version = 2.3.17 ### Documentation diff --git a/docs/PySpark/structured-streaming-pyspark.md b/docs/PySpark/structured-streaming-pyspark.md index 022356d8..070d13c2 100644 --- a/docs/PySpark/structured-streaming-pyspark.md +++ b/docs/PySpark/structured-streaming-pyspark.md @@ -23,13 +23,13 @@ Structured streaming integration for Azure Event Hubs is ultimately run on the J ``` groupId = com.microsoft.azure artifactId = azure-eventhubs-spark_2.11 - version = 2.3.16 + version = 2.3.17 or groupId = com.microsoft.azure artifactId = azure-eventhubs-spark_2.12 - version = 2.3.16 + version = 2.3.17 ``` For Python applications, you need to add this above library and its dependencies when deploying your application. @@ -397,11 +397,11 @@ AMQP types need to be handled explicitly by the connector. Below we list the AMQ As with any Spark applications, `spark-submit` is used to launch your application. `azure-eventhubs-spark_2.11` and its dependencies can be directly added to `spark-submit` using `--packages`, such as, - ./bin/spark-submit --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.16 ... + ./bin/spark-submit --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.17 ... For experimenting on `spark-shell`, you can also use `--packages` to add `azure-eventhubs-spark_2.11` and its dependencies directly, - ./bin/spark-shell --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.16 ... + ./bin/spark-shell --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.17 ... See [Application Submission Guide](https://spark.apache.org/docs/latest/submitting-applications.html) for more details about submitting applications with external dependencies. diff --git a/docs/spark-streaming-eventhubs-integration.md b/docs/spark-streaming-eventhubs-integration.md index 81646fbd..3988459c 100644 --- a/docs/spark-streaming-eventhubs-integration.md +++ b/docs/spark-streaming-eventhubs-integration.md @@ -23,13 +23,13 @@ For Scala/Java applications using SBT/Maven project definitions, link your appli ``` groupId = com.microsoft.azure artifactId = azure-eventhubs-spark_2.11 - version = 2.3.16 + version = 2.3.17 or groupId = com.microsoft.azure artifactId = azure-eventhubs-spark_2.12 - version = 2.3.16 + version = 2.3.17 ``` For Python applications, you need to add this above library and its dependencies when deploying your application. diff --git a/docs/structured-streaming-eventhubs-integration.md b/docs/structured-streaming-eventhubs-integration.md index e490fb92..8b8d56c9 100644 --- a/docs/structured-streaming-eventhubs-integration.md +++ b/docs/structured-streaming-eventhubs-integration.md @@ -23,13 +23,13 @@ For Scala/Java applications using SBT/Maven project definitions, link your appli ``` groupId = com.microsoft.azure artifactId = azure-eventhubs-spark_2.11 - version = 2.3.16 + version = 2.3.17 or groupId = com.microsoft.azure artifactId = azure-eventhubs-spark_2.12 - version = 2.3.16 + version = 2.3.17 ``` For Python applications, you need to add this above library and its dependencies when deploying your application. @@ -397,11 +397,11 @@ AMQP types need to be handled explicitly by the connector. Below we list the AMQ As with any Spark applications, `spark-submit` is used to launch your application. `azure-eventhubs-spark_2.11` and its dependencies can be directly added to `spark-submit` using `--packages`, such as, - ./bin/spark-submit --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.16 ... + ./bin/spark-submit --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.17 ... For experimenting on `spark-shell`, you can also use `--packages` to add `azure-eventhubs-spark_2.11` and its dependencies directly, - ./bin/spark-shell --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.16 ... + ./bin/spark-shell --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.17 ... See [Application Submission Guide](https://spark.apache.org/docs/latest/submitting-applications.html) for more details about submitting applications with external dependencies.