3456772473
updating HiveExample to use SparkSession
2016-10-07 17:40:49 -07:00
edd7b7fd8b
fixing issue with HiveContext introduced during Spark 2.0 upgrade
2016-10-07 17:22:08 -07:00
7a4689f173
use forward slash to work in linux/osx
2016-10-04 14:01:10 -07:00
c63db1834a
use forward slash to work in linux/osx
2016-10-04 13:16:59 -07:00
af71a329bf
Use copy task instead of OS commands so it builds on OSX
2016-09-29 13:41:32 -07:00
3c767bddd4
Add Python version performance benchmark test; Add usage + example and RIOSocket option support for C# version benchmark; Update Scala version benchmark; Update csv package version. ( #565 )
...
Besides normal spark-submit , there's a convenient way using https://github.com/qualiu/testMobius
1.Cluster mode examples as following:
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-python-perf-test.bat d:\msgit\lqmMobius\python\perf 10 hdfs:///perf/data/deletions/*
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-scala-perf-test.bat d:\msgit\lqmMobius\scala\perf 10
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-mobius-perf-test.bat d:\msgit\lqmMobius\csharp\perf 10
2.Local mode examples :
(1) First set SparkOptions : just copy the "Local Mode set SparkOptions=" after running the bat without arguments.
(2) Then submit local test : (you should have downloaded or copied the test data, like d:\mobius\deletions)
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-python-perf-test.bat d:\msgit\lqmMobius\python\perf 3 d:\mobius\deletions\deletions.csv-00000-of-00020
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-scala-perf-test.bat d:\msgit\lqmMobius\scala\perf 3 d:\mobius\deletions\deletions.csv-00000-of-00020
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-mobius-perf-test.bat d:\msgit\lqmMobius\csharp\perf 3 d:\mobius\deletions\deletions.csv-00000-of-00020
2016-09-20 13:41:29 +08:00
a9c2d84a03
Store external dependencies in build before copying to runtime folder
2016-09-16 16:14:59 -07:00
3da5beb91e
Store external dependencies in build before copying to runtime folder
2016-09-16 18:28:42 -04:00
f40cf468ae
Windows path separators used in run-samples.sh
2016-09-16 13:52:01 -07:00
427ce52e8d
Download external dependencies from the build.sh shell script
2016-09-16 16:32:06 -04:00
67046cef3e
Update run-samples.sh to use unix path separators
2016-09-16 14:13:48 -04:00
9a37ec84fd
Use logInfo and logError method instead of println ( #556 )
...
* Use logInfo and logError method insted of println
Finish TODO "logError does not work now"
* remove redundant printStackTrace method
2016-09-09 14:20:34 -07:00
da54d509ed
added support for SparkSession, Catalog and Dataset
2016-09-06 13:11:59 -07:00
7a6d6bfa22
Improve Worker logging to make CSharpWorkerFunc stacktrace clear ( #559 )
2016-09-02 14:06:40 -07:00
9f4276e338
[Mobius-102] Add missing license text in batch and powershell scripts ( #557 )
2016-08-31 11:33:41 -07:00
09d8a2f215
updating examples to use Mobius 2.0 preview release and minor updates
2016-08-25 09:04:19 -07:00
013278bf0e
updating version in Mobius release package name
2016-08-24 14:57:58 -07:00
23d917fab0
version update for preview release
2016-08-24 14:19:39 -07:00
cec7737060
upgrade to Spark 2.0
2016-08-24 12:55:19 -07:00
ad7cecf316
updating dependencies in examples to latest Mobius release
2016-08-23 16:58:12 -07:00
1215ae66c6
Merge pull request #537 from hebinhuang/SocketOptim
...
Fix error 10022 in Worker.WriteOutput() caused by RioSocket
2016-08-23 13:43:17 -07:00
cb8269fe8a
Fix error 10022 in Worker.WriteOutput() caused by RioSocket
2016-08-23 13:24:28 -07:00
3ac63040d7
Modify HiveDataFrame example
2016-08-16 10:05:30 -07:00
a1fca0733e
updating examples to use latest NuGet package
2016-08-15 20:06:24 -07:00
0d16a57739
doc updates for M3 release
2016-08-15 18:09:43 -07:00
a26d7d3098
expose HadoopConfiguration in SparkContext
2016-08-12 13:38:59 -07:00
4944319a55
Add read and write buffer for CSharpWorker ( #521 )
...
* Add read and write buffer for CSharpWorker
* Fix test coverage decrease issue
* Adjust read buffer
* Fix failed unit test
* use --conf option to pass buffer size
* Fix flaky unit test
* Address review comments
* Fix option name renaming issue
2016-08-12 14:21:31 +08:00
3afa007e58
Merge pull request #531 from hebinhuang/SocketOptim
...
Fix SocketStream.ReadByte() to release unreadable cache
2016-08-11 22:35:47 -07:00
2df8109272
Merge pull request #528 from tawan0109/PriorityQueue
...
Improve the implementation of RDD TakeOrdered and Top operations
2016-08-12 13:35:05 +08:00
5c57df9320
Merge pull request #532 from tawan0109/build_fix
...
Escaped special characters in build.cmd
2016-08-12 12:56:00 +08:00
42a0c3561d
Escaped^Cpecial characters in build.cmd
2016-08-12 12:31:59 +08:00
61fef193d3
Address review comments
2016-08-12 11:32:04 +08:00
04f642eed8
documentation updates
2016-08-11 18:18:38 -07:00
9197b265ff
Fix SocketStream.ReadByte() to release unreadable cache
2016-08-11 17:00:57 -07:00
a0c72f5fa1
Improve the implementation of RDD TakeOrdered and Top operations
2016-08-11 21:39:45 +08:00
da03e58dbe
added new F# example
...
* moved existing F# example to have all F# examples under a single folder
* added copyright notice to existing F# examples
* updated documentation
2016-08-10 17:54:09 -07:00
a2fe4f5dee
improving debug experience
...
* Added the option of using default port (5567) when debugging Mobius driver app. It is possible to provide a specific port number for CSharpBackend and also continue using the old debug behavior by specifying port 0
* Deleted App.Config from examples and updated projects to share App.Config
* Updated documentation
* Added/updated tests
2016-08-09 14:52:25 -07:00
1ab980cb20
Update .gitignore
2016-08-09 10:11:40 -07:00
e46e4109be
Update .gitignore
...
Since the .pptx and .docx file name has changed to `Mobius`, we should update .gitignore file.
BTW, I also add the Riosock.sdf file which generated when I open this cpp project in VS.
2016-08-08 16:37:10 +08:00
95afbf3f65
Merge pull request #515 from Myasuka/fix-md-files
...
fix broken links in markdown files
2016-08-01 08:48:16 +08:00
109c3792ff
Merge pull request #513 from xiongrenyi/directstream
...
refactoring to change low level enumerator back to normal
2016-07-30 21:22:02 -07:00
6f94bc7aaa
fix broken links in markdown files
2016-07-31 00:43:13 +08:00
e38b0c2b98
refactoring to change low level enumerator back to normal
2016-07-29 10:06:02 -07:00
c1dd6d1962
Merge pull request #508 from tawan0109/REPL
...
Mobius REPL
2016-07-29 10:04:02 +08:00
030625ea6e
Merge pull request #512 from cosminBoaca/patch-1
...
Fixed broken RunSamples.cmd link
2016-07-28 17:47:10 -07:00
7f618571af
Fixed broken doc link
2016-07-28 16:03:36 -07:00
d20d2812b6
Merge pull request #511 from xiongrenyi/directstream
...
add try finally block to parallel job since driver does not exit if the job fails
2016-07-28 13:19:49 -07:00
5eed2bf959
Add empty line check in RoslynScriptEngine & code refactoring
2016-07-28 16:53:23 +08:00
2c4ea6e07e
Code clean up & add document
2016-07-27 16:31:22 +08:00
fe89bf75e7
add try finally block to parallel job since drive does not exit if the job fails
2016-07-26 15:30:44 -07:00
Kaarthik Sivashanmugam
Shakib Bawa
shakdoesgithub
Quanmao LIU
dwnichols
Yun Tang
Hebin Huang
Tao Wang
Renyi Xiong
cosminBoaca