adding instructions for Linux
This commit is contained in:
Kaarthik Sivashanmugam
Родитель
513915fbfc
Коммит
18668c3c58
|
|
@ -112,7 +112,7 @@ Refer to the [docs folder](docs) for design overview and other info on Mobius
|
|||
|Build & run unit tests |[Build in Windows](notes/windows-instructions.md#building-mobius) |[Build in Linux](notes/linux-instructions.md#building-mobius) |
|
||||
|Run samples (functional tests) in local mode |[Samples in Windows](notes/windows-instructions.md#running-samples) |[Samples in Linux](notes/linux-instructions.md#running-samples) |
|
||||
|Run examples in local mode |[Examples in Windows](/notes/running-mobius-app.md#running-mobius-examples-in-local-mode) |[Examples in Linux](notes/running-mobius-app.md#linux-instructions) |
|
||||
|Run Mobius app |<ul><li>[Standalone cluster](notes/running-mobius-app.md#standalone-cluster)</li><li>[YARN cluster](notes/running-mobius-app.md#yarn-cluster)</li></ul> |[Mobius app in Linux](notes/running-mobius-app.md#linux-instructions) |
|
||||
|Run Mobius app |<ul><li>[Standalone cluster](notes/running-mobius-app.md#standalone-cluster)</li><li>[YARN cluster](notes/running-mobius-app.md#yarn-cluster)</li></ul> |<ul><li>[Linux](notes/running-mobius-app.md#linux-instructions)</li><li>[Azure HDInsight Spark Cluster](/notes/running-mobius-app.md#mobius-in-azure-hdinsight-spark-cluster)</li><li>[AWS EMR Spark Cluster](/notes/running-mobius-app.md#mobius-in-amazon-web-services-emr-spark-cluster)</li> |
|
||||
|
||||
### Useful Links
|
||||
* [Configuration parameters in Mobius](/notes/configuration-mobius.md)
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ The following software need to be installed and appropriate environment variable
|
|||
|JDK |7u85 or 8u60 ([OpenJDK](http://www.azul.com/downloads/zulu/zulu-windows/) or [Oracle JDK](http://www.oracle.com/technetwork/java/javase/downloads/index.html)) |JAVA_HOME | After setting JAVA_HOME, run `set PATH=%PATH%;%JAVA_HOME%\bin` to add java to PATH |
|
||||
|Spark |[1.5.2 or 1.6.*](http://spark.apache.org/downloads.html) | SPARK_HOME |Spark can be downloaded from Spark download website. Alternatively, if you used [`RunSamples.cmd`](../csharp/Samples/Microsoft.Spark.CSharp/samplesusage.md) to run Mobius samples, you can find `toos\spark*` directory (under [`build`](../build) directory) that can be used as SPARK_HOME |
|
||||
|winutils.exe | see [Running Hadoop on Windows](https://wiki.apache.org/hadoop/WindowsProblems) for details |HADOOP_HOME |Spark in Windows needs this utility in `%HADOOP_HOME%\bin` directory. It can be copied over from any Hadoop distribution. Alternative, if you used [`RunSamples.cmd`](../csharp/Samples/Microsoft.Spark.CSharp/samplesusage.md) to run Mobius samples, you can find `toos\winutils` directory (under [`build`](../build) directory) that can be used as HADOOP_HOME |
|
||||
|Mobius |[v1.5.200](https://github.com/Microsoft/Mobius/releases) or v1.6.100-PREVIEW-1 | SPARKCLR_HOME |If you downloaded a [Mobius release](https://github.com/Microsoft/Mobius/releases), SPARKCLR_HOME should be set to the directory named `runtime` (for example, `D:\downloads\spark-clr_2.10-1.5.200\runtime`). Alternatively, if you used [`RunSamples.cmd`](../csharp/Samples/Microsoft.Spark.CSharp/samplesusage.md) to run Mobius samples, you can find `runtime` directory (under [`build`](../build) directory) that can be used as SPARKCLR_HOME. **Note** - setting SPARKCLR_HOME is _optional_ and it is set by sparkclr-submit.cmd if not set. |
|
||||
|Mobius | [v1.6.100](https://github.com/Microsoft/Mobius/releases/tag/v1.6.100) or a valid dev build or [previous release](https://github.com/Microsoft/Mobius/releases) of Mobius | SPARKCLR_HOME |If you downloaded a [Mobius release](https://github.com/Microsoft/Mobius/releases), SPARKCLR_HOME should be set to the directory named `runtime` (for example, `D:\downloads\spark-clr_2.10-1.5.200\runtime`). Alternatively, if you used [`RunSamples.cmd`](../csharp/Samples/Microsoft.Spark.CSharp/samplesusage.md) to run Mobius samples, you can find `runtime` directory (under [`build`](../build) directory) that can be used as SPARKCLR_HOME. **Note** - setting SPARKCLR_HOME is _optional_ and it is set by sparkclr-submit.cmd if not set. |
|
||||
|
||||
## Dependencies
|
||||
Some features in Mobius depend on classes outside of Spark and Mobius. A selected set of jar files that Mobius depends on are available in Mobius release under "runtime\dependencies" folder. These jar files are used with "--jars" parameter in Mobius (that is sparkclr-submit.cmd) and they get passed to Spark (spark-submit.cmd).
|
||||
|
|
@ -120,10 +120,31 @@ sparkclr-submit.cmd ^
|
|||
````
|
||||
|
||||
## Linux Instructions
|
||||
The instructions above cover running Mobius applications in Windows. With the following tweaks, the same instructions can be used to run Mobius applications in Linux.
|
||||
|
||||
### Requirements
|
||||
* Mono 4.2 or above
|
||||
* Follow installation instructions for [Ubuntu and similiar Linux distributions](http://www.mono-project.com/docs/getting-started/install/linux/#debian-ubuntu-and-derivatives) or [CentOS, OSX and similar Linux distributions](http://www.mono-project.com/docs/getting-started/install/linux/#centos-7-fedora-19-and-later-and-derivatives).
|
||||
* If you are using CentOS, OSX and similar Linux distributions, make sure Mono is added to the path using the following script
|
||||
```bash
|
||||
# update mono path if needed
|
||||
export PATH=$PATH:/opt/mono/bin
|
||||
export PKG_CONFIG_PATH=/opt/mono/lib/pkgconfig
|
||||
```
|
||||
* [Mobius release 1.6.101-PREVIEW1](https://github.com/Microsoft/Mobius/releases/tag/v1.6.101-PREVIEW-1) or above
|
||||
|
||||
The [instructions](./running-mobius-app.md#windows-instructions) above cover running Mobius applications in Windows. With the following tweaks, the same instructions can be used to run Mobius applications in Linux.
|
||||
* Instead of `RunSamples.cmd`, use `run-samples.sh`
|
||||
* Instead of `sparkclr-submit.cmd`, use `sparkclr-submit.sh`
|
||||
|
||||
**Note** - only client mode is support in Mobius on YARN in Linux. Support for [cluster mode](https://github.com/Microsoft/Mobius/issues/467) will be added soon.
|
||||
|
||||
### Mobius in Azure HDInsight Spark Cluster
|
||||
* Mono version available in HDInsight cluster is 3.x. Mobius [requires](/notes/linux-instructions.md#prerequisites) 4.2 or above. So, Mono has to be upgraded in HDInsight cluster to use Mobius.
|
||||
* Follow [instructions](./running-mobius-app.md#requirements) for Ubuntu
|
||||
|
||||
### Mobius in Amazon Web Services EMR Spark Cluster
|
||||
* Follow [instructions](./running-mobius-app.md#requirements) for CentOS
|
||||
|
||||
## Running Mobius Examples in Local Mode
|
||||
| Type | Examples |
|
||||
| ------------- |--------------|
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче