syncfusion
/
Mobius
зеркало из https://github.com/syncfusion/Mobius.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
1 019 коммитов 10 Ветки 13 Теги 6,9 MiB
Граф коммитов

65 Коммитов

Автор SHA1 Сообщение Дата
Jorgen Thelin 6e2b820524 Fix Spark version build problems 
* Check that Download-File function got a real file.
- Check in the `Download-File` function that the current downloaded copy of the file is not empty [zero length].
* archive.apache.org
- Use `archive.apache.org` as the default server location for downloading Spark [and Maven] binary distributions.
- Previous location was `www.us.apache.org` but that only supports the most recent distributions for each major release
  [ie currently `2.0.1` but not `2.0.0` which Mobius is using at the moment.]

* The Apache distribution server is used during both build and run phases.
 2016-11-01 22:41:17 -07:00
Quanmao LIU 3c767bddd4 Add Python version performance benchmark test; Add usage + example and RIOSocket option support for C# version benchmark; Update Scala version benchmark; Update csv package version. (#565) 
Besides normal spark-submit , there's a convenient way using https://github.com/qualiu/testMobius
1.Cluster mode examples as following:
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-python-perf-test.bat d:\msgit\lqmMobius\python\perf 10 hdfs:///perf/data/deletions/*
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-scala-perf-test.bat  d:\msgit\lqmMobius\scala\perf  10
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-mobius-perf-test.bat d:\msgit\lqmMobius\csharp\perf 10

2.Local mode examples :
(1) First set SparkOptions : just copy the "Local Mode set SparkOptions=" after running the bat without arguments.
(2) Then submit local test : (you should have downloaded or copied the test data, like d:\mobius\deletions)
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-python-perf-test.bat d:\msgit\lqmMobius\python\perf 3 d:\mobius\deletions\deletions.csv-00000-of-00020
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-scala-perf-test.bat  d:\msgit\lqmMobius\scala\perf  3 d:\mobius\deletions\deletions.csv-00000-of-00020
D:\msgit\lqmMobius\testMobius\scripts\perf\submit-mobius-perf-test.bat d:\msgit\lqmMobius\csharp\perf 3 d:\mobius\deletions\deletions.csv-00000-of-00020
 2016-09-20 13:41:29 +08:00
dwnichols 3da5beb91e Store external dependencies in build before copying to runtime folder 2016-09-16 18:28:42 -04:00
dwnichols 427ce52e8d Download external dependencies from the build.sh shell script 2016-09-16 16:32:06 -04:00
dwnichols 67046cef3e Update run-samples.sh to use unix path separators 2016-09-16 14:13:48 -04:00
Yun Tang 9f4276e338 [Mobius-102] Add missing license text in batch and powershell scripts (#557) 2016-08-31 11:33:41 -07:00
Kaarthik Sivashanmugam 09d8a2f215 updating examples to use Mobius 2.0 preview release and minor updates 2016-08-25 09:04:19 -07:00
Kaarthik Sivashanmugam cec7737060 upgrade to Spark 2.0 2016-08-24 12:55:19 -07:00
tawan0109 42a0c3561d Escaped^Cpecial characters in build.cmd 2016-08-12 12:31:59 +08:00
Kaarthik Sivashanmugam 04f642eed8 documentation updates 2016-08-11 18:18:38 -07:00
tawan0109 2c4ea6e07e Code clean up & add document 2016-07-27 16:31:22 +08:00
tawan0109 45ef6a5291 Merge branch 'master' of https://github.com/microsoft/SparkCLR into REPL 2016-07-25 20:03:58 +08:00
tawan0109 1eb7d98158 Rename sparkclr-repl.cmd to sparkclr-shell.cmd 2016-07-25 15:00:02 +08:00
Hebin Huang 38a9eb2069 Implement automatic code coverage reports for c++ code 2016-07-19 12:56:55 -07:00
tawan0109 622242c5b5 Merge branch 'master' of https://github.com/microsoft/SparkCLR into REPL 2016-07-13 18:05:36 +08:00
Tao Wang 46c08e45a5 Merge pull request #492 from tawan0109/local_checkpoint 
Rebuild state for DStream UpdateStateByKey operation by replaying batches
 2016-07-12 11:30:25 +08:00
Hebin Huang b6ec24119e Add RIOSocketWrapper and SaeaSocketWrapper 2016-07-09 22:00:10 -07:00
Tao Wang 470bb411fe Merge branch 'master' of https://github.com/microsoft/Mobius into REPL 2016-07-09 18:08:17 +08:00
tawan0109 3b67b2a6a7 Avoid RDD checkpoint for DynamicPartitionKafkaInputDStream by replaying last N batches 2016-07-08 17:52:47 +08:00
Quanmao LIU e75eb189e0 Download big file to temp one to avoid possible download failure and use corrupted file. (#490) 2016-07-08 09:51:16 +08:00
Kaarthik Sivashanmugam db95eb5de8 Updating maven version 2016-07-05 12:27:00 -07:00
Kaarthik Sivashanmugam 754a530961 updating maven version 2016-07-05 11:31:56 -07:00
Kaarthik Sivashanmugam 2337e7cee2 upgrading to Spark 1.6.2 2016-07-01 09:59:26 -07:00
jthelin 38f42d2554 Fix comment line prefix in run_samples.cmd script. 2016-06-20 06:46:08 -07:00
tawan0109 b97ed0a37d Fixing issue that build did not stop when mvn build failed 2016-05-19 12:16:06 +08:00
qualiu 0c91852baa Update download path of winutils.exe to fix running localmode example problem (such as: permission of /tmp/hive) 2016-05-10 10:58:01 +08:00
Isaac Abraham 07b704f8aa Sample word count project in F# 2016-05-07 11:07:19 -07:00
Kaarthik Sivashanmugam f8dea1f0e7 limiting executionpolicy change to one script instead of current user 2016-05-02 16:06:26 -07:00
skaarthik 1268b20906 limiting executionpolicy change to one script instead of current user 2016-05-01 22:43:15 -07:00
skaarthik 2fbd2b0cfe merge from branch-1.6 and version bump 2016-05-01 14:47:30 -07:00
Kaarthik Sivashanmugam 7982c8e4a6 updated doc and packaging step 2016-04-26 21:37:36 -07:00
Kaarthik Sivashanmugam d7f528bc22 fix bug in copyjar that happens in tagged releases due to multiple files that match pattern (#424) 2016-04-26 00:38:22 -07:00
Hebin Huang cc27f3a31a Copies the dependencies of Examples from runtime/bin to use latest Mobius bits 
* Copies the dependencies of Examples from runtime/bin to use latest Mobius bits

* Change the copy order - copy Mobius bits first
 2016-04-25 17:27:11 -07:00
Kaarthik Sivashanmugam 80832ba9f0 doc updates and using Mobius name in scripts 2016-04-25 13:06:18 -07:00
skaarthik cd4cfec562 updating scripts to download & use spark-csv and its dependency 2016-04-24 01:56:58 -07:00
skaarthik 1d1a31873c adding powershell script to copy non-uber jar to runtime folder 2016-04-22 15:16:15 -07:00
Hebin Huang e2eaa6faa3 Fix build.sh to check whether Examples build failed 2016-04-22 07:42:01 -07:00
Hebin Huang 9342171946 Fix Build.cmd to stop building when Examples compiling failed 2016-04-21 10:08:35 -07:00
Hebin Huang 7cb61f0b5d Fix Build.cmd to include examples binaries in release zip file 2016-04-20 20:34:30 -07:00
guiling 0a855ce640 Implement a Daemon process to take charge of launching Worker processes 2016-03-24 16:24:02 +08:00
skaarthik 3a4c938619 upgrade to Spark 1.6.1 2016-03-18 21:55:02 -07:00
hebinhuang c0ba9c59b6 Fix license 2016-03-16 11:53:26 -07:00
skaarthik 601ff40a41 including examples in build 2016-02-04 16:44:50 -08:00
tawan0109 04ac532e81 Rename class name and Assembly name to 'Repl' 2016-02-04 21:22:24 +08:00
tawan0109 1f5b6dddc2 Rename project from shell to Repl 2016-02-04 16:51:18 +08:00
tawan0109 56b00b1e46 Merge branch 'master' of https://github.com/microsoft/SparkCLR into REPL 
Conflicts:
	build/Build.cmd
	build/localmode/RunSamples.cmd
	csharp/Worker/Microsoft.Spark.CSharp/Worker.cs
	scala/src/main/org/apache/spark/deploy/csharp/CSharpRunner.scala
 2016-02-04 15:45:02 +08:00
tqin 0879de6d9e Add input parameter to Build.cmd to enable building CSharp code only 2016-01-26 10:42:43 +08:00
Jorgen Thelin eb25e2ac24 Only copy the shaded spark .jar files to lib. 2016-01-21 11:47:20 -08:00
jthelin c3eaa3afe3 Mvn-Quiet 
- Run Maven in --quiet mode to avoid all the download progress messages in the build server logs.
 2016-01-19 18:52:08 -08:00
jthelin d4310ce4bd If mvn not already installed, then set M2 env variables from download 2016-01-19 18:52:07 -08:00