microsoft
/
spark
зеркало из https://github.com/microsoft/spark.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
3 903 коммитов 13 Ветки 28 Теги 178 MiB
Граф коммитов

230 Коммитов

Автор SHA1 Сообщение Дата
Patrick Wendell 0e375a3cc2 Add assmebly plug in links 2013-09-01 09:43:42 -07:00
Patrick Wendell 6371febe18 Better docs 2013-08-31 19:09:06 -07:00
Matei Zaharia 9ddad0dcb4 Fixes suggested by Patrick 2013-08-31 17:40:33 -07:00
Matei Zaharia 4819baa658 More updates, describing changes to recommended use of environment vars 
and new Python stuff
 2013-08-31 14:21:10 -07:00
Matei Zaharia 4293533032 Update docs about HDFS versions 2013-08-30 15:04:43 -07:00
Matei Zaharia f3a964848d More doc improvements + better warnings when you haven't built Spark 2013-08-30 12:41:25 -07:00
Matei Zaharia 23762efda2 New hardware provisioning doc, and updates to menus 2013-08-30 10:16:26 -07:00
Matei Zaharia 1b0f69c623 Change docs color theme for 0.8 2013-08-30 10:15:58 -07:00
Matei Zaharia e11bc18294 Update Maven docs 2013-08-29 21:19:07 -07:00
Matei Zaharia 2de756ff19 Update some build instructions because only sbt assembly and mvn package 
are now needed
 2013-08-29 21:19:06 -07:00
Matei Zaharia 53cd50c069 Change build and run instructions to use assemblies 
This commit makes Spark invocation saner by using an assembly JAR to
find all of Spark's dependencies instead of adding all the JARs in
lib_managed. It also packages the examples into an assembly and uses
that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script
with two better-named scripts: "run-examples" for examples, and
"spark-class" for Spark internal classes (e.g. REPL, master, etc). This
is also designed to minimize the confusion people have in trying to use
"run" to run their own classes; it's not meant to do that, but now at
least if they look at it, they can modify run-examples to do a decent
job for them.

As part of this, Bagel's examples are also now properly moved to the
examples package instead of bagel.
 2013-08-29 21:19:04 -07:00
Matei Zaharia baa84e7e4c Merge pull request #865 from tgravescs/fixtmpdir 
Spark on Yarn should use yarn approved directories for spark.local.dir and tmp
 2013-08-28 12:44:46 -07:00
Y.CORP.YAHOO.COM\tgraves 63dc635de6 fix typos 2013-08-26 17:06:20 -05:00
Y.CORP.YAHOO.COM\tgraves c9464c74a1 Add ability for user to specify environment variables 2013-08-26 16:44:27 -05:00
Y.CORP.YAHOO.COM\tgraves 6dd64e8bb2 Update docs and remove old reference to --user option 2013-08-26 14:29:24 -05:00
Patrick Wendell 2cfe52ef55 Version bump for ec2 docs 2013-08-24 15:16:53 -07:00
Patrick Wendell 4879685910 Merge remote-tracking branch 'mesos/master' into ec2-updates 2013-08-24 14:50:58 -07:00
Matei Zaharia 5a6ac12840 Merge pull request #701 from ScrapCodes/documentation-suggestions 
Documentation suggestions for spark streaming.
 2013-08-22 22:08:03 -07:00
Prashant Sharma 2bc348e92c Linking custom receiver guide 2013-08-23 09:44:02 +05:30
Prashant Sharma 39a1d58da4 Improved documentation for spark custom receiver 2013-08-23 09:38:50 +05:30
Jey Kottalam f9cc1fbf27 Remove references to unsupported Hadoop versions 2013-08-21 17:14:36 -07:00
Patrick Wendell 6be6b71c8c Merge branch 'master' into ec2-updates 
Conflicts:
	ec2/spark_ec2.py
 2013-08-21 15:34:31 -07:00
Jey Kottalam 6585f49841 Update build docs 2013-08-21 14:51:56 -07:00
Jey Kottalam 9c6f8df30f Update jekyll plugin to match docs/README.md 2013-08-21 12:57:56 -07:00
Matei Zaharia 53b1c30607 Update docs for Spark UI port 2013-08-20 22:57:11 -07:00
Matei Zaharia aa2b89d98d Merge remote-tracking branch 'jey/hadoop-agnostic' 
Conflicts:
	core/src/main/scala/spark/PairRDDFunctions.scala
 2013-08-20 10:14:15 -07:00
Matei Zaharia 2a4ed10210 Address some review comments: 
- When a resourceOffers() call has multiple offers, force the TaskSets
  to consider them in increasing order of locality levels so that they
  get a chance to launch stuff locally across all offers

- Simplify ClusterScheduler.prioritizeContainers

- Add docs on the new configuration options
 2013-08-18 19:51:07 -07:00
Jey Kottalam 14b6bcdf93 update YARN docs 2013-08-15 16:50:37 -07:00
Evan Sparks 4346f0a1e9 Merge pull request #809 from shivaram/sgd-cleanup 
Clean up scaladoc in ML Lib.
 2013-08-12 12:12:12 -07:00
Shivaram Venkataraman 8b5e3e2eb5 Add ML Lib scaladoc to API dropdown 2013-08-11 23:52:43 -07:00
Patrick Wendell 9244524146 Removing dead docs 2013-08-11 20:33:58 -07:00
Shivaram Venkataraman 4935a2558b Clean up scaladoc in ML Lib. 
Also build and copy ML Lib scaladoc in Spark docs build.
Some more minor cleanup with respect to naming, test locations etc.
 2013-08-11 19:02:43 -07:00
Matei Zaharia de6c4c995a Merge pull request #787 from ash211/master 
Update spark-standalone.md
 2013-08-06 17:09:50 -07:00
Andrew Ash afc2c80fdb Update spark-standalone.md 2013-08-07 00:44:43 +01:00
Patrick Wendell 5cc725a0e3 Merge branch 'master' into ec2-updates 
Conflicts:
	ec2/deploy.generic/root/mesos-ec2/ec2-variables.sh
 2013-07-31 21:35:12 -07:00
Patrick Wendell b7b627d5bb Updating relevant documentation 2013-07-31 21:28:27 -07:00
Matei Zaharia 3097d75d6f Merge remote-tracking branch 'dlyubimov/SPARK-827' 
Conflicts:
	docs/configuration.md
 2013-07-31 18:36:43 -07:00
Reynold Xin 5227043f84 Documentation update for compression codec. 2013-07-30 17:12:16 -07:00
Matei Zaharia 497f55755f Add docs about ipython 2013-07-29 02:51:43 -04:00
Dmitriy Lyubimov 0862494d44 typo 2013-07-27 23:16:20 -07:00
Dmitriy Lyubimov f5067abe85 changes per comments. 2013-07-27 23:08:00 -07:00
Ubuntu 88a0823c58 Consistently invoke bash with /usr/bin/env bash in scripts to make code more portable (JIRA Ticket SPARK-817) 2013-07-18 00:51:18 +00:00
Matei Zaharia af3c9d5042 Add Apache license headers and LICENSE and NOTICE files 2013-07-16 17:21:33 -07:00
Matei Zaharia d47c16f78d Add an option to disable reference tracking in Kryo 2013-07-15 01:55:54 +00:00
Andy Konwinski cd7259b4b8 Fixes typos in Spark Streaming Programming Guide 
These typos were reported on the spark-users mailing list, see: https://groups.google.com/d/msg/spark-users/SyLGgJlKCrI/LpeBypOkSMUJ
 2013-07-12 11:51:14 -07:00
Matei Zaharia 1ffadb2d9e Merge remote-tracking branch 'pwendell/ui-updates' 
Conflicts:
	core/src/main/scala/spark/scheduler/DAGScheduler.scala
	core/src/main/scala/spark/util/AkkaUtils.scala
	pom.xml
 2013-07-06 15:51:41 -07:00
root 7cd490ef5b Clarify that PySpark is not supported on Windows 2013-07-01 06:26:43 +00:00
Matei Zaharia 5bbd0eec84 Update docs on SCALA_LIBRARY_PATH 2013-06-30 17:00:40 -07:00
Matei Zaharia 03d0b858c8 Made use of spark.executor.memory setting consistent and documented it 
Conflicts:

	core/src/main/scala/spark/SparkContext.scala
 2013-06-30 15:46:46 -07:00
Matei Zaharia aea727f68d Simplify Python docs a little to do substring search 2013-06-26 21:15:09 -07:00