3b95e7043a
Unit test for CSharpWorker
2016-07-26 11:07:11 +08:00
621a82c35e
Fix ReplTest not included in coverage report issue
2016-07-25 21:45:09 +08:00
45ef6a5291
Merge branch 'master' of https://github.com/microsoft/SparkCLR into REPL
2016-07-25 20:03:58 +08:00
25bb151cdb
Code refactoring: construct SqlContext outside of RolsynScriptEngine
2016-07-25 15:31:53 +08:00
1eb7d98158
Rename sparkclr-repl.cmd to sparkclr-shell.cmd
2016-07-25 15:00:02 +08:00
c753dbb75c
Merge pull request #509 from hebinhuang/FixBuild
...
Merge two code coverage uploads to one
2016-07-22 12:11:53 -07:00
5092917204
Merge two code coverage uploads to one
2016-07-22 11:01:39 -07:00
a5e6fa6eba
Minor fix for directive support
2016-07-22 18:22:18 +08:00
13a27e3eb6
Code clean up
2016-07-22 17:27:33 +08:00
7e879e1180
Support :help directive & code refactoring
2016-07-22 16:46:04 +08:00
eef32e0d7e
Support :load directive
2016-07-22 15:52:09 +08:00
c2766fd479
Remove DisplayDiagnostics() from RoslynScriptEngine.cs
2016-07-21 20:53:12 +08:00
fc968fbb1e
Merge branch 'REPL' of https://github.com/tawan0109/SparkCLR into REPL_DEV2
2016-07-21 20:28:28 +08:00
038c561d60
Add ReplTest project
2016-07-21 20:22:45 +08:00
867e7e0a5d
Merge pull request #3 from kai-zeng/repl
...
code clean up
2016-07-21 11:57:10 +08:00
a33613b5b5
code clean up
2016-07-20 20:54:19 -07:00
9dc4674e93
Merge pull request #507 from xiongrenyi/DirectStream
...
unifying KafkaUtils.CreateDirectStream API
2016-07-20 18:58:53 -07:00
36e8fbbf2c
Update configuration-mobius.md
2016-07-20 14:59:44 -07:00
df00f417d7
Update configuration-mobius.md
2016-07-20 14:58:13 -07:00
2bceabb2a0
Update configuration-mobius.md
...
add spark.mobius.streaming.kafka.numPartitions.<topicName>[.<clusterId>]
2016-07-20 14:51:27 -07:00
66f6f60571
unifying KafkaUtils.CreateDirectStream API
2016-07-20 14:05:06 -07:00
12de1a7129
Code clean && add missing comments
2016-07-20 13:38:02 +08:00
3d541e942e
Change time unit of streaming back to millisecond ( #504 )
...
* Change time unit of streaming back to millisecond
* Update Microsoft.Spark.CSharp.Adapter.Doc.XML and Mobius_API_Documentation.md
* Change according to review comments
2016-07-19 19:16:40 -07:00
ae1494e547
Merge pull request #505 from hebinhuang/NativeCodeCov
...
Implement automatic code coverage reports for c++ code
2016-07-19 13:59:30 -07:00
38a9eb2069
Implement automatic code coverage reports for c++ code
2016-07-19 12:56:55 -07:00
9a0648b397
Code refactoring
2016-07-19 16:58:45 +08:00
0b94b5a23d
Use sparkContext.AddFile to ship generated compilation binary
2016-07-18 19:39:18 +08:00
8cbdf30ed1
Merge branch 'master' of https://github.com/microsoft/SparkCLR into REPL
2016-07-18 14:26:01 +08:00
0bf617ae4a
Create DLL dump diretory under the directory specified by option spark.local.dir & delete DLL dump directory on exit
2016-07-18 11:25:36 +08:00
bf210f47f0
Add CSharpInputDStream to enable control of data injection by C# code ( #500 )
...
* Add CSharpInputDStream
* Add Unit test for CSharpInputDStream
2016-07-17 18:46:26 -07:00
bf84818d46
Use log4net as default logger service as current default logger cannot suppress undesired lower level messages. ( #501 )
2016-07-16 22:24:48 +08:00
94420c1879
Merge pull request #502 from hebinhuang/RioSockBug
...
Fix bugs in RioSocketWrapper and SocketStream that cause vstest failed
2016-07-15 10:30:08 -07:00
41a91b03b0
Fix bugs in RioSocketWrapper and SocketStream that cause vstest failed
2016-07-14 21:08:52 -07:00
622242c5b5
Merge branch 'master' of https://github.com/microsoft/SparkCLR into REPL
2016-07-13 18:05:36 +08:00
f31c92f7f0
Downgrade the log level of WeakObjectManager.cs to reduce logs. ( #499 )
2016-07-13 14:23:20 +08:00
46c08e45a5
Merge pull request #492 from tawan0109/local_checkpoint
...
Rebuild state for DStream UpdateStateByKey operation by replaying batches
2016-07-12 11:30:25 +08:00
ffe223c2f9
Merge pull request #483 from hebinhuang/ManagedRioSock
...
Add RIOSocketWrapper and SaeaSocketWrapper
2016-07-10 07:32:20 -07:00
b6ec24119e
Add RIOSocketWrapper and SaeaSocketWrapper
2016-07-09 22:00:10 -07:00
470bb411fe
Merge branch 'master' of https://github.com/microsoft/Mobius into REPL
2016-07-09 18:08:17 +08:00
3b67b2a6a7
Avoid RDD checkpoint for DynamicPartitionKafkaInputDStream by replaying last N batches
2016-07-08 17:52:47 +08:00
d3999ca43d
Add option to turn on/off local checkpoint for CSharpStateDStream
2016-07-08 17:51:13 +08:00
e75eb189e0
Download big file to temp one to avoid possible download failure and use corrupted file. ( #490 )
2016-07-08 09:51:16 +08:00
db95eb5de8
Updating maven version
2016-07-05 12:27:00 -07:00
dd376179ac
updating maven version
2016-07-05 11:33:22 -07:00
d78493bc42
updating build version to reflect Spark version
2016-07-05 11:32:44 -07:00
754a530961
updating maven version
2016-07-05 11:31:56 -07:00
b15163657a
When check and release unused JVMObjectTracker objects, change the speed adaptively according to current objects count. ( #488 )
2016-07-04 00:12:10 -07:00
33b87b4b0f
Merge pull request #487 from xiongrenyi/DirectReceiver
...
add validation to tqin's for ReduceByKeyAndWindow
2016-07-02 08:46:56 -07:00
9515b44f26
in addition to tqin's fix of a typo and removing prevFunc in ReduceByKeyAndWindow, add numPartition explicitly and add validation to his sample
2016-07-01 16:49:24 -07:00
4004001732
fixes to run Mobius in Linux
...
* allowing to specify worker path for all modes - this will help specifying prefix script for worker exe in CentOS
* fixing issue 467 - with this change Mobius can run in YARN cluster mode
* updating notes since YARN cluster mode is supported with the fix to issue #467
* added check to run OS-specific path tests
2016-07-01 11:21:33 -07:00
tawan0109
Hebin Huang
Kai Zeng
Renyi Xiong
Tao Qin
Quanmao LIU
Kaarthik Sivashanmugam