syncfusion
/
Mobius
зеркало из https://github.com/syncfusion/Mobius.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
1 019 коммитов 10 Ветки 13 Теги 6,9 MiB
Граф коммитов

1019 Коммитов

Автор SHA1 Сообщение Дата
Jacek Laskowski 313dbf308b Update linux-instructions.md 
Add a note about `wget` (Im on macOS and had to install it using homebrew) + formatting
 2017-01-05 21:23:40 +01:00
Attila Szucs 88f76af038 Fix ReduceByKey failure when the value type is a reference type 2016-12-03 22:49:00 -08:00
Kaarthik Sivashanmugam 12c46c8ed1 fixing Travis build issue 
* using openjdk7
 2016-12-03 22:28:59 -08:00
Jorgen Thelin 6e2b820524 Fix Spark version build problems 
* Check that Download-File function got a real file.
- Check in the `Download-File` function that the current downloaded copy of the file is not empty [zero length].
* archive.apache.org
- Use `archive.apache.org` as the default server location for downloading Spark [and Maven] binary distributions.
- Previous location was `www.us.apache.org` but that only supports the most recent distributions for each major release
  [ie currently `2.0.1` but not `2.0.0` which Mobius is using at the moment.]

* The Apache distribution server is used during both build and run phases.
 2016-11-01 22:41:17 -07:00
Kai Zeng 5311e3027a add RunJob method to SparkContext; add SparkContext property to RDD (#590) 
* add RunJob method to SparkContext; add SparkContext property to RDD

* test RDD.SparkContext
 2016-10-25 14:06:42 -07:00
Kaarthik Sivashanmugam 142701109b documentation for implementing Spark Apps in F# using Mobius 2016-10-17 11:58:54 -07:00
Kaarthik Sivashanmugam dae6f9a55d updating HiveExample to use SparkSession 2016-10-11 16:45:49 -07:00
Kaarthik Sivashanmugam 8597df2d0f updating examples to use latest preview release 2016-10-11 16:20:41 -07:00
Kaarthik Sivashanmugam 9a8fbe811d updating AssemblyInfo versions 2016-10-11 12:38:22 -07:00
Kaarthik Sivashanmugam e64e342b33 adding instructions for HDI 2016-10-11 12:38:06 -07:00
Kaarthik Sivashanmugam fbf0f8d841 fixing the issue in YARN clusters introduced during Spark 2.0 upgrade 2016-10-10 23:42:07 -07:00
skaarthik 7d920d1206 fixing the issue in YARN clusters introduced during Spark 2.0 upgrade 2016-10-10 19:11:55 -07:00
Kaarthik Sivashanmugam 3456772473 updating HiveExample to use SparkSession 2016-10-07 17:40:49 -07:00
Kaarthik Sivashanmugam edd7b7fd8b fixing issue with HiveContext introduced during Spark 2.0 upgrade 2016-10-07 17:22:08 -07:00
Kaarthik Sivashanmugam 7a4689f173 use forward slash to work in linux/osx 2016-10-04 14:01:10 -07:00
Shakib Bawa c63db1834a use forward slash to work in linux/osx 2016-10-04 13:16:59 -07:00
shakdoesgithub af71a329bf Use copy task instead of OS commands so it builds on OSX 2016-09-29 13:41:32 -07:00
Quanmao LIU 3c767bddd4 Add Python version performance benchmark test; Add usage + example and RIOSocket option support for C# version benchmark; Update Scala version benchmark; Update csv package version. (#565) 
Besides normal spark-submit , there's a convenient way using https://github.com/qualiu/testMobius
1.Cluster mode examples as following:
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-python-perf-test.bat d:\msgit\lqmMobius\python\perf 10 hdfs:///perf/data/deletions/*
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-scala-perf-test.bat  d:\msgit\lqmMobius\scala\perf  10
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-mobius-perf-test.bat d:\msgit\lqmMobius\csharp\perf 10

2.Local mode examples :
(1) First set SparkOptions : just copy the "Local Mode set SparkOptions=" after running the bat without arguments.
(2) Then submit local test : (you should have downloaded or copied the test data, like d:\mobius\deletions)
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-python-perf-test.bat d:\msgit\lqmMobius\python\perf 3 d:\mobius\deletions\deletions.csv-00000-of-00020
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-scala-perf-test.bat  d:\msgit\lqmMobius\scala\perf  3 d:\mobius\deletions\deletions.csv-00000-of-00020
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-mobius-perf-test.bat d:\msgit\lqmMobius\csharp\perf 3 d:\mobius\deletions\deletions.csv-00000-of-00020
 2016-09-20 13:41:29 +08:00
Kaarthik Sivashanmugam a9c2d84a03 Store external dependencies in build before copying to runtime folder 2016-09-16 16:14:59 -07:00
dwnichols 3da5beb91e Store external dependencies in build before copying to runtime folder 2016-09-16 18:28:42 -04:00
Kaarthik Sivashanmugam f40cf468ae Windows path separators used in run-samples.sh 2016-09-16 13:52:01 -07:00
dwnichols 427ce52e8d Download external dependencies from the build.sh shell script 2016-09-16 16:32:06 -04:00
dwnichols 67046cef3e Update run-samples.sh to use unix path separators 2016-09-16 14:13:48 -04:00
Yun Tang 9a37ec84fd Use logInfo and logError method instead of println (#556) 
* Use logInfo and logError method insted of println

Finish TODO "logError does not work now"

* remove redundant printStackTrace method
 2016-09-09 14:20:34 -07:00
Kaarthik Sivashanmugam da54d509ed added support for SparkSession, Catalog and Dataset 2016-09-06 13:11:59 -07:00
Hebin Huang 7a6d6bfa22 Improve Worker logging to make CSharpWorkerFunc stacktrace clear (#559) 2016-09-02 14:06:40 -07:00
Yun Tang 9f4276e338 [Mobius-102] Add missing license text in batch and powershell scripts (#557) 2016-08-31 11:33:41 -07:00
Kaarthik Sivashanmugam 09d8a2f215 updating examples to use Mobius 2.0 preview release and minor updates 2016-08-25 09:04:19 -07:00
Kaarthik Sivashanmugam cec7737060 upgrade to Spark 2.0 2016-08-24 12:55:19 -07:00
Kaarthik Sivashanmugam ad7cecf316 updating dependencies in examples to latest Mobius release 2016-08-23 16:58:12 -07:00
Hebin Huang 1215ae66c6 Merge pull request #537 from hebinhuang/SocketOptim 
Fix error 10022 in Worker.WriteOutput() caused by RioSocket
 2016-08-23 13:43:17 -07:00
Hebin Huang cb8269fe8a Fix error 10022 in Worker.WriteOutput() caused by RioSocket 2016-08-23 13:24:28 -07:00
Yun Tang 3ac63040d7 Modify HiveDataFrame example 2016-08-16 10:05:30 -07:00
Kaarthik Sivashanmugam a1fca0733e updating examples to use latest NuGet package 2016-08-15 20:06:24 -07:00
Kaarthik Sivashanmugam 0d16a57739 doc updates for M3 release 2016-08-15 18:09:43 -07:00
Kaarthik Sivashanmugam a26d7d3098 expose HadoopConfiguration in SparkContext 2016-08-12 13:38:59 -07:00
Tao Wang 4944319a55 Add read and write buffer for CSharpWorker (#521) 
* Add read and write buffer for CSharpWorker

* Fix test coverage decrease issue

* Adjust read buffer

* Fix failed unit test

* use --conf option to pass buffer size

* Fix flaky unit test

* Address review comments

* Fix option name renaming issue
 2016-08-12 14:21:31 +08:00
Hebin Huang 3afa007e58 Merge pull request #531 from hebinhuang/SocketOptim 
Fix SocketStream.ReadByte() to release unreadable cache
 2016-08-11 22:35:47 -07:00
Tao Wang 2df8109272 Merge pull request #528 from tawan0109/PriorityQueue 
Improve the implementation of RDD TakeOrdered and Top operations
 2016-08-12 13:35:05 +08:00
Tao Wang 5c57df9320 Merge pull request #532 from tawan0109/build_fix 
Escaped special characters in build.cmd
 2016-08-12 12:56:00 +08:00
tawan0109 42a0c3561d Escaped^Cpecial characters in build.cmd 2016-08-12 12:31:59 +08:00
tawan0109 61fef193d3 Address review comments 2016-08-12 11:32:04 +08:00
Kaarthik Sivashanmugam 04f642eed8 documentation updates 2016-08-11 18:18:38 -07:00
Hebin Huang 9197b265ff Fix SocketStream.ReadByte() to release unreadable cache 2016-08-11 17:00:57 -07:00
tawan0109 a0c72f5fa1 Improve the implementation of RDD TakeOrdered and Top operations 2016-08-11 21:39:45 +08:00
Kaarthik Sivashanmugam da03e58dbe added new F# example 
* moved existing F# example to have all F# examples under a single folder
* added copyright notice to existing F# examples
* updated documentation
 2016-08-10 17:54:09 -07:00
Kaarthik Sivashanmugam a2fe4f5dee improving debug experience 
* Added the option of using default port (5567) when debugging Mobius driver app. It is possible to provide a specific port number for CSharpBackend and also continue using the old debug behavior by specifying port 0
* Deleted App.Config from examples and updated projects to share App.Config
* Updated documentation
* Added/updated tests
 2016-08-09 14:52:25 -07:00
Kaarthik Sivashanmugam 1ab980cb20 Update .gitignore 2016-08-09 10:11:40 -07:00
Yun Tang e46e4109be Update .gitignore 
Since the .pptx and .docx file name has changed to `Mobius`, we should update .gitignore file.
BTW, I also add the Riosock.sdf file which generated when I open this cpp project in VS.
 2016-08-08 16:37:10 +08:00
Tao Wang 95afbf3f65 Merge pull request #515 from Myasuka/fix-md-files 
fix broken links in markdown files
 2016-08-01 08:48:16 +08:00