Modify HiveDataFrame example
This commit is contained in:
Родитель
a1fca0733e
Коммит
3ac63040d7
|
@ -24,7 +24,10 @@ namespace Microsoft.Spark.CSharp.Examples
|
||||||
var sparkConf = new SparkConf();
|
var sparkConf = new SparkConf();
|
||||||
var sparkContext = new SparkContext(sparkConf);
|
var sparkContext = new SparkContext(sparkConf);
|
||||||
var hiveContext = new HiveContext(sparkContext);
|
var hiveContext = new HiveContext(sparkContext);
|
||||||
var peopleDataFrame = hiveContext.Read().Json(Path.Combine(Environment.CurrentDirectory, @"data\people.json"));
|
|
||||||
|
// please give the path to input json file
|
||||||
|
var jsonFilePath = args[0];
|
||||||
|
var peopleDataFrame = hiveContext.Read().Json(jsonFilePath);
|
||||||
|
|
||||||
const string dbName = "SampleHiveDataBaseForMobius";
|
const string dbName = "SampleHiveDataBaseForMobius";
|
||||||
const string tableName = "people";
|
const string tableName = "people";
|
||||||
|
|
|
@ -173,9 +173,9 @@ Displays the number of XML elements in the input XML file provided as the first
|
||||||
|
|
||||||
### Hive Example (Sql)
|
### Hive Example (Sql)
|
||||||
*
|
*
|
||||||
`sparkclr-submit.cmd --jars <jar files used for using Hive in Spark> --exe HiveDataFrame.exe C:\Git\Mobius\examples\Sql\HiveDataFrame\bin\Debug`
|
`sparkclr-submit.cmd --exe HiveDataFrame.exe C:\Git\Mobius\examples\Sql\HiveDataFrame\bin\Debug <input json file path>`
|
||||||
|
|
||||||
Reads data from a csv file, creates a Hive table and reads data from it
|
Reads data from a csv file, creates a Hive table and reads data from it. The `<input json file path>` could look like `C:\Git\Mobius\examples\Sql\HiveDataFrame\data\people.json` or `file:///C:/Git/Mobius/examples/Sql/HiveDataFrame/data/people.json` in Windows environment. While in local Linux environment, the `<input json file path>` could look look like `file:///path/to/people.json`.
|
||||||
|
|
||||||
### Cassandra Example (Sql)
|
### Cassandra Example (Sql)
|
||||||
* Download following jars that are needed to use Spark with Cassandra. **Note** that you need to get the right version of the jar files depending on the versions of Spark and Cassandra. Refer to [version compatibility](https://github.com/datastax/spark-cassandra-connector#version-compatibility) table for details.
|
* Download following jars that are needed to use Spark with Cassandra. **Note** that you need to get the right version of the jar files depending on the versions of Spark and Cassandra. Refer to [version compatibility](https://github.com/datastax/spark-cassandra-connector#version-compatibility) table for details.
|
||||||
|
|
Загрузка…
Ссылка в новой задаче