зеркало из https://github.com/dotnet/spark.git
Update developer guide to support new patch version of Spark (#109)
This commit is contained in:
Родитель
574be7bdca
Коммит
cd91a1360e
|
@ -1,8 +1,12 @@
|
|||
# Developer Guide
|
||||
|
||||
## How to debug
|
||||
## Table of Contents
|
||||
- [How to Do Local Debugging](#how-to-do-local-debugging)
|
||||
- [How to Support New Spark Releases](#how-to-support-new-spark-releases)
|
||||
|
||||
### Debugging .NET application
|
||||
## How to Do Local Debugging
|
||||
|
||||
### Debugging Spark .NET Application
|
||||
|
||||
Open a new command prompt window, run the following:
|
||||
```shell
|
||||
|
@ -34,3 +38,26 @@ spark-submit \
|
|||
<path-to-microsoft-spark-jar> \
|
||||
<path-to-your-app-exe> <argument(s)-to-your-app>
|
||||
```
|
||||
|
||||
## How to Support New Spark Releases
|
||||
|
||||
We encourage developers to first read Apache Spark's [Versioning Policy](https://spark.apache.org/versioning-policy.html) and [Semantic Versioning](https://semver.org/) to gain the most out of the instructions below.
|
||||
|
||||
At a high-level, Spark's versions are: **[MAJOR].[FEATURE].[MAINTENANCE]**. We will cover the upgrade path for each type of version separately below (in increasing order of effort required).
|
||||
|
||||
### [MAINTENANCE]: Upgrading for a Patch Release Version
|
||||
Since Apache Spark's [MAINTENANCE] releases involve only internal changes (e.g., bug fixes etc.), it is straightforward to upgrade the code base to support a [MAINTENANCE] release. The steps to do this are below:
|
||||
|
||||
1. In the corresponding `pom.xml`, update the `spark.version` value to the newly released version.
|
||||
* For example, if a new patch release is 2.4.3, you will update [src/scala/microsoft-spark-2.4.x/pom.xml](/src/scala/microsoft-spark-2.4.x/pom.xml) to have `<spark.version>2.4.3</spark.version>`.
|
||||
2. Update `DotnetRunner.supportedSparkVersions` to include the newly released version.
|
||||
* For example, if a new patch release is 2.4.3, you will update [src/scala/microsoft-spark-2.4.x/src/main/scala/org/apache/spark/deploy/DotnetRunner.scala](/src/scala/microsoft-spark-2.4.x/src/main/scala/org/apache/spark/deploy/DotnetRunner.scala).
|
||||
3. Update the [azure-pipelines.yml](/azure-pipelines.yml) to include E2E testing for the newly released version.
|
||||
|
||||
Refer to [this commit](https://github.com/dotnet/spark/commit/eb26baa46200bfcbe3e1080e650f335853d9990e) for an example.
|
||||
|
||||
### [FEATURE]: Upgrading for a Minor Release Version
|
||||
*WIP*
|
||||
|
||||
### [MAJOR]: Upgrading for a Major Release Version
|
||||
*WIP*
|
||||
|
|
Загрузка…
Ссылка в новой задаче