Merge pull request #515 from Myasuka/fix-md-files
fix broken links in markdown files
This commit is contained in:
Tao Wang
GitHub
Коммит
95afbf3f65
24
README.md
24
README.md
|
|
@ -25,7 +25,7 @@ metricDataFrame.RegisterTempTable("metrics");
|
|||
var joinDataFrame = GetSqlContext().Sql(
|
||||
"SELECT joinedtable.datacenter" +
|
||||
", MAX(joinedtable.latency) maxlatency" +
|
||||
", AVG(joinedtable.latency) avglatency " +
|
||||
", AVG(joinedtable.latency) avglatency " +
|
||||
"FROM (" +
|
||||
"SELECT a.C1 as datacenter, b.C6 as latency " +
|
||||
"FROM requests a JOIN metrics b ON a.C0 = b.C3) joinedtable " +
|
||||
|
|
@ -79,21 +79,21 @@ StreamingContext sparkStreamingContext = StreamingContext.GetOrCreate(checkpoint
|
|||
sparkStreamingContext.Start();
|
||||
sparkStreamingContext.AwaitTermination();
|
||||
```
|
||||
Refer to [Mobius\csharp\Samples](csharp/Samples) directory and [sample usage](csharp/Samples/Microsoft.Spark.CSharp/samplesusage.md) for complete samples.
|
||||
Refer to [Mobius\csharp\Samples](./csharp/Samples) directory and [sample usage](./csharp/Samples/Microsoft.Spark.CSharp/samplesusage.md) for complete samples.
|
||||
|
||||
## API Documentation
|
||||
|
||||
Refer to [Mobius C# API documentation](csharp/Adapter/documentation/Mobius_API_Documentation.md) for the list of Spark's data processing operations supported in Mobius.
|
||||
Refer to [Mobius C# API documentation](./csharp/Adapter/documentation/Mobius_API_Documentation.md) for the list of Spark's data processing operations supported in Mobius.
|
||||
|
||||
## API Usage
|
||||
|
||||
Mobius API usage samples are available at:
|
||||
|
||||
* [Examples folder](./examples) which contains standalone [C# projects](/notes/running-mobius-app.md#running-mobius-examples-in-local-mode) that can be used as templates to start developing Mobius applications
|
||||
* [Examples folder](./examples) which contains standalone [C# projects](./notes/running-mobius-app.md#running-mobius-examples-in-local-mode) that can be used as templates to start developing Mobius applications
|
||||
|
||||
* [Samples project](csharp/Samples/Microsoft.Spark.CSharp/) which uses a comprehensive set of Mobius APIs to implement samples that are also used for functional validation of APIs
|
||||
|
||||
* Mobius performance test scenarios implemented in [C#](csharp/Perf/Microsoft.Spark.CSharp) and [Scala](scala/perf) for side by side comparison of Spark driver code
|
||||
* [Samples project](./csharp/Samples/Microsoft.Spark.CSharp/) which uses a comprehensive set of Mobius APIs to implement samples that are also used for functional validation of APIs
|
||||
|
||||
* Mobius performance test scenarios implemented in [C#](./csharp/Perf/Microsoft.Spark.CSharp) and [Scala](./scala/perf) for side by side comparison of Spark driver code
|
||||
|
||||
## Documents
|
||||
|
||||
|
|
@ -116,9 +116,9 @@ Refer to the [docs folder](docs) for design overview and other info on Mobius
|
|||
|Run Mobius Shell |<ul><li>[Local](notes/mobius-shell.md#run-shell)</li><li>[YARN](notes/mobius-shell.md#run-shell)</li></ul> | Not supported yet |
|
||||
|
||||
### Useful Links
|
||||
* [Configuration parameters in Mobius](/notes/configuration-mobius.md)
|
||||
* [Troubleshoot errors in Mobius](/notes/troubleshooting-mobius.md)
|
||||
* [Debug Mobius apps](/notes/running-mobius-app.md#debug-mode)
|
||||
* [Configuration parameters in Mobius](./notes/configuration-mobius.md)
|
||||
* [Troubleshoot errors in Mobius](./notes/troubleshooting-mobius.md)
|
||||
* [Debug Mobius apps](./notes/running-mobius-app.md#debug-mode)
|
||||
|
||||
## Supported Spark Versions
|
||||
|
||||
|
|
@ -130,7 +130,7 @@ Mobius releases are available at https://github.com/Microsoft/Mobius/releases. R
|
|||
|
||||
[](https://www.nuget.org/packages/Microsoft.SparkCLR)
|
||||
|
||||
Refer to [mobius-release-info.md](notes/mobius-release-info.md) for the details on versioning policy and the contents of the release.
|
||||
Refer to [mobius-release-info.md](./notes/mobius-release-info.md) for the details on versioning policy and the contents of the release.
|
||||
|
||||
## License
|
||||
|
||||
|
|
@ -146,7 +146,7 @@ Mobius is licensed under the MIT license. See [LICENSE](LICENSE) file for full l
|
|||
[](https://gitter.im/Microsoft/Mobius?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
|
||||
[](https://twitter.com/intent/tweet?text=@MobiusForSpark [your tweet] via @GitHub)
|
||||
|
||||
* Mobius project welcomes contributions. To contribute, follow the instructions in [CONTRIBUTING.md](notes/CONTRIBUTING.md)
|
||||
* Mobius project welcomes contributions. To contribute, follow the instructions in [CONTRIBUTING.md](./notes/CONTRIBUTING.md)
|
||||
|
||||
* Options to ask your question to the Mobius community
|
||||
* create issue on [GitHub](https://github.com/Microsoft/Mobius)
|
||||
|
|
|
|||
|
|
@ -1,17 +1,17 @@
|
|||
Contributing to Mobius
|
||||
========================
|
||||
This page outlines contribution to Mobius
|
||||
|
||||
|
||||
Ways to contribute
|
||||
------------------
|
||||
If you would like to become involved in the development of Mobius, there are many different ways in which you can contribute. We strongly value your feedback, questions, bug reports, and feature requests.
|
||||
If you would like to become involved in the development of Mobius, there are many different ways in which you can contribute. We strongly value your feedback, questions, bug reports, and feature requests.
|
||||
|
||||
Consider these options:
|
||||
Consider these options:
|
||||
* Use Mobius bits
|
||||
* Submit a GitHub issue (see [Issue Guide](docs/project-docs/issue-guide.md)).
|
||||
* Submit a GitHub issue (see [Issue Guide](../docs/project-docs/issue-guide.md)).
|
||||
* Verify fixes for bugs.
|
||||
* Submit a code fix for a bug.
|
||||
* Submit a new feature request (as a GitHub issue in [Issue Guide](docs/project-docs/issue-guide.md)).
|
||||
* Submit a new feature request (as a GitHub issue in [Issue Guide](../docs/project-docs/issue-guide.md)).
|
||||
* Help answer questions on Mobius mailing lists (*[sparkclr-user](https://groups.google.com/d/forum/sparkclr-user)* or *[sparkclr-dev](https://groups.google.com/d/forum/sparkclr-dev)*).
|
||||
* Submit a unit test.
|
||||
* Code review pending pull requests and bug fixes.
|
||||
|
|
@ -22,7 +22,7 @@ Contributing Code Changes
|
|||
|
||||
If you are looking for something to work on, the list of [up-for-grabs issues](https://github.com/Microsoft/Mobius/labels/up%20for%20grabs) is a good starting point.
|
||||
|
||||
Before opening a *pull request*, review [Contributing Code Changes](/docs/project-docs/CONTRIBUTING.md).
|
||||
Before opening a *pull request*, review [Contributing Code Changes](../docs/project-docs/CONTRIBUTING.md).
|
||||
It lists steps that are required before creating a PR. In particular, consider:
|
||||
|
||||
- Is the change important and ready enough to ask the community to spend time reviewing?
|
||||
|
|
@ -31,10 +31,10 @@ It lists steps that are required before creating a PR. In particular, consider:
|
|||
|
||||
Contribution License Agreement
|
||||
------------------------------
|
||||
We appreciate community contributions to code repositories open sourced by Microsoft. By signing a
|
||||
[contribution license agreement](https://cla.microsoft.com/cladoc/microsoft-contribution-license-agreement.pdf), we ensure that the community is free to use your contributions.
|
||||
We appreciate community contributions to code repositories open sourced by Microsoft. By signing a
|
||||
[contribution license agreement](https://cla.microsoft.com/cladoc/microsoft-contribution-license-agreement.pdf), we ensure that the community is free to use your contributions.
|
||||
|
||||
When a contributor makes a pull request, the Microsoft Pull Request BOT (MSBOT) checks whether the change requires a CLA; for example, trivial typo fixes usually don’t require a CLA. If no CLA is required, the pull request is labeled as *cla-not-required* and the contributor is done. If the change requires a CLA, MSBOT checks whether the contributor has already signed one; if you have, the pull request is labeled as *cla-signed* and the contributor is done. If the contributor needs to sign a CLA, MSBOT will label the request as *cla-required* and post a comment pointing you to sign in on the appropriate website to sign the CLA (fully electronic, no faxing involved); once the contributor has signed a CLA, the pull request is labeled as *cla-signed*. You are done.
|
||||
When a contributor makes a pull request, the Microsoft Pull Request BOT (MSBOT) checks whether the change requires a CLA; for example, trivial typo fixes usually don’t require a CLA. If no CLA is required, the pull request is labeled as *cla-not-required* and the contributor is done. If the change requires a CLA, MSBOT checks whether the contributor has already signed one; if you have, the pull request is labeled as *cla-signed* and the contributor is done. If the contributor needs to sign a CLA, MSBOT will label the request as *cla-required* and post a comment pointing you to sign in on the appropriate website to sign the CLA (fully electronic, no faxing involved); once the contributor has signed a CLA, the pull request is labeled as *cla-signed*. You are done.
|
||||
|
||||
We accept only pull requests that are labeled as either *cla-not-required*, *cla-signed* or *cla-already-signed*.
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ The following environment variables should be set properly:
|
|||
|
||||
## Instructions
|
||||
|
||||
Instructions to build Mobius in Linux are same as [instructions for Windows](windows-instructions.md#instructions). The only change required is to use the following script files instead of .cmd files:
|
||||
Instructions to build Mobius in Linux are same as [instructions for Windows](./windows-instructions.md#instructions). The only change required is to use the following script files instead of .cmd files:
|
||||
* build.sh
|
||||
* clean.sh
|
||||
|
||||
|
|
@ -25,9 +25,9 @@ Instructions to build Mobius in Linux are same as [instructions for Windows](win
|
|||
```
|
||||
./test.sh
|
||||
```
|
||||
|
||||
|
||||
# Running Mobius Samples in Linux
|
||||
Same as [instructions for Windows](windows-instructions.md#running-samples) but using the following scripts instead of .cmd files:
|
||||
Same as [instructions for Windows](./windows-instructions.md#running-samples) but using the following scripts instead of .cmd files:
|
||||
* run-samples.sh
|
||||
* sparkclr-submit.sh
|
||||
|
||||
|
|
@ -52,7 +52,7 @@ If you are using CentOS, Fedora, or similar Linux distributions or OS X, follow
|
|||
* Update CSharpWorkerPath setting in Mobius application config (refer to the config files used in Mobius examples like the [config for with Pi example](https://github.com/skaarthik/Mobius/blob/linux/examples/Batch/pi/App.config#L61)) to point to [CSharpWorker.sh.exe](./linux-csharpworker-prefix-script.md) (make sure to set the correct value appropriate for the Spark mode to be used)
|
||||
|
||||
### Mobius in Azure HDInsight Spark Cluster
|
||||
* Mono version available in HDInsight cluster is 3.x. Mobius [requires](/notes/linux-instructions.md#prerequisites) 4.2 or above. So, Mono has to be upgraded in HDInsight cluster to use Mobius.
|
||||
* Mono version available in HDInsight cluster is 3.x. Mobius [requires](./linux-instructions.md#prerequisites) 4.2 or above. So, Mono has to be upgraded in HDInsight cluster to use Mobius.
|
||||
* Follow [instructions](./linux-instructions.md#requirements) for Ubuntu
|
||||
|
||||
### Mobius in Amazon Web Services EMR Spark Cluster
|
||||
|
|
@ -66,6 +66,6 @@ $ sudo wget http://download.mono-project.com/sources/mono/mono-4.4.1.0.tar.bz2
|
|||
$ sudo tar jxf mono-4.4.1.0.tar.bz2
|
||||
$ cd mono-4.4.1.0
|
||||
$ sudo ./configure --prefix=/opt/mono
|
||||
$ sudo make
|
||||
$ sudo make
|
||||
$ sudo make install
|
||||
```
|
||||
|
|
|
|||
|
|
@ -2,19 +2,19 @@
|
|||
|
||||
## Prerequisites
|
||||
|
||||
* All prerequisites listed in [Build in Windows](notes/windows-instructions.md#prerequisites)
|
||||
* All prerequisites listed in [Build in Windows](./windows-instructions.md#prerequisites)
|
||||
* .NET Framework **4.6.1** or above.
|
||||
|
||||
**To Be Noticed**:
|
||||
**To Be Noticed**:
|
||||
Currently Mobius Shell only supports **Windows**, Linux support is not supported yet.
|
||||
|
||||
## Building
|
||||
|
||||
Mobius Shell is included in Mobius release, please refer to [Build in Windows](notes/windows-instructions.md) to build Mobius.
|
||||
Mobius Shell is included in Mobius release, please refer to [Build in Windows](./windows-instructions.md) to build Mobius.
|
||||
|
||||
## Running the Shell
|
||||
|
||||
Please first refer to [Running Mobius App](notes/running-mobius-app.md) to make sure all required softwares and enviroment variables are placed or set properly.
|
||||
Please first refer to [Running Mobius App](./running-mobius-app.md) to make sure all required softwares and enviroment variables are placed or set properly.
|
||||
|
||||
* Local
|
||||
|
||||
|
|
@ -61,7 +61,7 @@ This is from test lib.
|
|||
> var rdd = sc.Parallelize(Enumerable.Range(0, 100), 2).Map(d => d+1);
|
||||
> rdd.Count()
|
||||
100
|
||||
>
|
||||
>
|
||||
> // dataframe example
|
||||
> var df = sqlContext.Read().Json(@"D:\SparkWorkspace\SparkCLR_REPL\build\runtime\data\people.json");
|
||||
> df.Count()
|
||||
|
|
@ -76,4 +76,3 @@ This is from test lib.
|
|||
+--------------------+---+---+-----+
|
||||
> :quit
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -10,20 +10,20 @@ The following environment variables should be set properly in the Developer Comm
|
|||
|
||||
* `JAVA_HOME`
|
||||
|
||||
**To Be Noticed**:
|
||||
**To Be Noticed**:
|
||||
Mobius on Windows includes a C++ component - RIOSock.dll. If your environment does not have VC++ Build Toolset installed, the C++ component will be skipped to compile. Offically, the C++ component is always compiled on AppVeyor.
|
||||
Please enable VC++ component from Visual Studio, or you can download [Visual C++ Build Tools](http://landinghub.visualstudio.com/visual-cpp-build-tools), if you want to build C++ components.
|
||||
|
||||
## Instructions
|
||||
|
||||
* In the Developer Command Prompt for Visual Studio where `JAVA_HOME` is set properly, navigate to [Mobius\build](../build/) directory:
|
||||
* In the Developer Command Prompt for Visual Studio where `JAVA_HOME` is set properly, navigate to [Mobius\build](../build/) directory:
|
||||
|
||||
```
|
||||
Build.cmd
|
||||
```
|
||||
|
||||
* Optional:
|
||||
- Under [Mobius\scala](../scala) directory, run the following command to clean spark-clr*.jar built above:
|
||||
* Optional:
|
||||
- Under [Mobius\scala](../scala) directory, run the following command to clean spark-clr*.jar built above:
|
||||
|
||||
```
|
||||
mvn clean
|
||||
|
|
@ -34,12 +34,12 @@ Please enable VC++ component from Visual Studio, or you can download [Visual C++
|
|||
```
|
||||
Clean.cmd
|
||||
```
|
||||
|
||||
|
||||
[Build.cmd](../build/Build.cmd) downloads necessary build tools; after the build is done, it prepares the folowing directories under `Mobius\build\runtime`:
|
||||
|
||||
* **lib** ( `spark-clr*.jar` )
|
||||
* **bin** ( `Microsoft.Spark.CSharp.Adapter.dll`, `CSharpWorker.exe`)
|
||||
* **samples** ( The contents of `Mobius\csharp\Samples\Microsoft.Spark.CSharp\bin\Release\*`, including `Microsoft.Spark.CSharp.Adapter.dll`, `CSharpWorker.exe`, `SparkCLRSamples.exe`, `SparkCLRSamples.exe.Config` etc. )
|
||||
* **samples** ( The contents of `Mobius\csharp\Samples\Microsoft.Spark.CSharp\bin\Release\*`, including `Microsoft.Spark.CSharp.Adapter.dll`, `CSharpWorker.exe`, `SparkCLRSamples.exe`, `SparkCLRSamples.exe.Config` etc. )
|
||||
* **scripts** ( `sparkclr-submit.cmd` )
|
||||
* **data** ( `Mobius\csharp\Samples\Microsoft.Spark.CSharp\data\*` )
|
||||
|
||||
|
|
@ -47,7 +47,7 @@ Please enable VC++ component from Visual Studio, or you can download [Visual C++
|
|||
|
||||
* In Visual Studio: Install NUnit3 Test Adapter. Run the tests through "Test" -> "Run" -> "All Tests"
|
||||
|
||||
* Install NUnit Runner 3.0 or above using NuGet (see [https://www.nuget.org/packages/NUnit.Runners/](https://www.nuget.org/packages/NUnit.Runners/)). In Developer Command Prompt for VS, set `NUNITCONSOLE` to the path to nunit console, and navigate to `Mobius\csharp` and run the following command:
|
||||
* Install NUnit Runner 3.0 or above using NuGet (see [https://www.nuget.org/packages/NUnit.Runners/](https://www.nuget.org/packages/NUnit.Runners/)). In Developer Command Prompt for VS, set `NUNITCONSOLE` to the path to nunit console, and navigate to `Mobius\csharp` and run the following command:
|
||||
```
|
||||
Test.cmd
|
||||
```
|
||||
|
|
@ -59,7 +59,7 @@ Samples demonstrate comprehesive usage of Mobius API and also serve as functiona
|
|||
* [YARN cluster](#running-in-yarn-mode)
|
||||
* [Local mode dev environment](#running-in-local-mode-dev-environment) (using artifacts built in the local Git repo)
|
||||
|
||||
The prerequisites for running Mobius samples are same as the ones for running any other Mobius applications. Refer to [instructions](.\running-mobius-app.md#pre-requisites) for details on that. [Local mode dev environment](#running-in-local-mode-dev-environment) makes it easier to run samples in dev environment by downloading Spark.
|
||||
The prerequisites for running Mobius samples are same as the ones for running any other Mobius applications. Refer to [instructions](./running-mobius-app.md#pre-requisites) for details on that. [Local mode dev environment](#running-in-local-mode-dev-environment) makes it easier to run samples in dev environment by downloading Spark.
|
||||
|
||||
## Running in Local mode
|
||||
```
|
||||
|
|
@ -91,7 +91,7 @@ It is **required** to run [Build.cmd](../build/Build.cmd) prior to running [RunS
|
|||
[RunSamples.cmd](../build/localmode/RunSamples.cmd) downloads the version of Apache Spark referenced in the current branch, sets up `SPARK_HOME` environment variable, points `SPARKCLR_HOME` to `Mobius\build\runtime` directory created by [Build.cmd](../build/Build.cmd), and invokes [sparkclr-submit.cmd](../scripts/sparkclr-submit.cmd), with `spark.local.dir` set to `Mobius\build\runtime\Temp`.
|
||||
|
||||
A few more [RunSamples.cmd](../build/localmode/RunSamples.cmd) examples:
|
||||
- To display all options supported by [RunSamples.cmd](../build/localmode/RunSamples.cmd):
|
||||
- To display all options supported by [RunSamples.cmd](../build/localmode/RunSamples.cmd):
|
||||
|
||||
```
|
||||
RunSamples.cmd --help
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче