microsoft
/
spark
зеркало из https://github.com/microsoft/spark.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
2 531 коммит 13 Ветки 28 Теги 178 MiB
Граф коммитов

48 Коммитов

Автор SHA1 Сообщение Дата
Mikhail Bautin 7fd2708eda Add a log4j compile dependency to fix build in IntelliJ 
Also rename parent project to spark-parent (otherwise it shows up as
"parent" in IntelliJ, which is very confusing).
 2013-03-15 11:41:51 -07:00
Mark Hamstra 8b06b359da bump version to 0.7.1-SNAPSHOT in the subproject poms to keep the maven build building. 2013-02-28 23:34:34 -08:00
Matei Zaharia db9b90fdbd Change version to 0.7.1-SNAPSHOT for development branch 2013-02-27 09:15:26 -08:00
Mikhail Bautin fe3eceab57 Remove activation of profiles by default 
See the discussion at https://github.com/mesos/spark/pull/355 for why
default profile activation is a problem.
 2013-01-31 13:30:41 -08:00
Stephen Haberman 7dfb82a992 Replace old 'master' term with 'driver'. 2013-01-25 11:03:00 -06:00
Mikhail Bautin 325297e5c3 Add an Avro dependency to REPL to make it compile with Hadoop 2 2013-01-22 18:11:51 -08:00
Matei Zaharia 6e3754bf47 Add Maven build file for streaming, and fix some issues in SBT file 
As part of this, changed our Scala 2.9.2 Kafka library to be available
as a local Maven repository, following the example in
(http://blog.dub.podval.org/2010/01/maven-in-project-repository.html)
 2013-01-20 19:22:24 -08:00
Tathagata Das cd1521cfdb Merge branch 'master' into streaming 
Conflicts:
	core/src/main/scala/spark/rdd/CoGroupedRDD.scala
	core/src/main/scala/spark/rdd/FilteredRDD.scala
	docs/_layouts/global.html
	docs/index.md
	run
 2013-01-15 12:08:51 -08:00
Shivaram Venkataraman bbc56d85ed Rename environment variable for hadoop profiles to hadoopVersion 2013-01-12 15:24:13 -08:00
Shivaram Venkataraman 9262522306 Activate hadoop2 profile in pom.xml with -Dhadoop=2 2013-01-10 22:07:34 -08:00
Shivaram Venkataraman f7adb382ac Activate hadoop1 if property hadoop is missing. hadoop2 can be activated now 
by using -Dhadoop -Phadoop2.
 2013-01-08 03:19:43 -08:00
Shivaram Venkataraman 4bbe07e5ec Activate hadoop1 profile by default for maven builds 2013-01-07 17:46:22 -08:00
Tathagata Das 4719e6d8fe Changed locations for unit test logs. 2013-01-07 16:06:07 -08:00
Thomas Dudziak 4af6cad37a Fixed repl maven build to produce artifacts with the appropriate hadoop classifier and extracted repl fat-jar and debian packaging into a separate project to make Maven happy 2012-12-18 12:08:19 -08:00
Thomas Dudziak c1d15ae3d5 Shaded repl jar for hadoop1 profile needs to include hadoop classes 2012-12-10 15:06:28 -08:00
Matei Zaharia a9ea14d6e7 Merge pull request #318 from tomdz/master 
Minor tweaks to the debian build
 2012-12-10 10:59:41 -08:00
Matei Zaharia ccff0a089a Use the same output directories that SBT had in subprojects 
This will make it easier to make the "run" script work with a Maven build
 2012-12-10 10:58:56 -08:00
Thomas Dudziak 0e5b1f7981 Minor tweaks to the debian build 2012-12-10 10:30:30 -08:00
Thomas Dudziak 3b643e86bc Updated versions in the pom.xml files to match current master 2012-11-27 17:50:42 -08:00
Thomas Dudziak 69297c64be Addressed code review comments 2012-11-27 15:45:16 -08:00
Thomas Dudziak 24e1e425cd Include the configuration templates in the debian package 2012-11-20 16:19:56 -08:00
Thomas Dudziak 811a32257b Added maven and debian build files 2012-11-20 16:19:51 -08:00
Matei Zaharia 0967e71a00 Bump up version to 0.7.0-SNAPSHOT for master branch 2012-10-22 11:49:42 -07:00
Matei Zaharia 902a608187 Update version to 0.6.1-SNAPSHOT to show this is in development 2012-10-22 11:43:57 -07:00
Matei Zaharia ce6b5a3ee5 Uncomment Maven publishing stuff and set version to 0.6.0 2012-10-13 15:55:39 -07:00
Matei Zaharia eca570f66a Removed the need to sleep in tests due to waiting for Akka to shut down 2012-10-07 00:17:59 -07:00
Matei Zaharia 74a9244255 Write all unit test output to a file 2012-10-01 15:07:42 -07:00
Matei Zaharia 83143f9a5f Fixed several bugs that caused weird behavior with files in spark-shell: 
- SizeEstimator was following through a ClassLoader field of Hadoop
  JobConfs, which referenced the whole interpreter, Scala compiler, etc.
  Chaos ensued, giving an estimated size in the tens of gigabytes.
- Broadcast variables in local mode were only stored as MEMORY_ONLY and
  never made accessible over a server, so they fell out of the cache when
  they were deemed too large and couldn't be reloaded.
 2012-09-30 21:19:39 -07:00
Matei Zaharia 2c16ae36d7 Set log level in tests to WARN 2012-08-23 20:38:14 -07:00
Matei Zaharia e72afdb817 Some refactoring to make cluster scheduler pluggable. 2012-07-06 15:23:26 -07:00
Matei Zaharia 08579ffa11 Update version number for dev branch 2012-06-15 23:55:43 -07:00
Matei Zaharia 4449eb9783 Changed version in master branch to 0.5.1-SNAPSHOT for further 
development.
 2012-06-13 22:26:14 -04:00
Matei Zaharia 4971e0f547 Updated version number to 0.5.0 2012-06-12 13:41:57 -04:00
Matei Zaharia dbc3c86ae3 Merge branch 'master' into mesos-0.9 
Conflicts:
	core/src/main/scala/spark/Executor.scala
 2012-06-03 17:44:04 -07:00
Reynold Xin d176422586 Make spark.repl.Main.interp_ publicly accessible (so Shark can get rid 
of a weird file dediated to accessing this variable).
 2012-05-30 18:40:10 -07:00
Matei Zaharia 08cda89e8a Further fixes to how Mesos is found and used 2012-03-17 13:39:14 -07:00
Matei Zaharia 63da22c025 Update REPL code to use our own version of JLineReader, which fixes #89. 
I'm not entirely sure why this broke in the jump from Scala 2.9.0.1 to
2.9.1 -- maybe something about name resolution changed?
 2011-11-07 20:16:25 -08:00
Ismael Juma 483f724d62 Upgrade to Scala 2.9.1. 
Interestingly, the version in Maven is 2.9.1, but SBT outputs file to the 2.9.1.final
directory inside target.

A couple of small changes in SparkIMain were also required.

All tests pass and ./spark-shell launches successfully.
 2011-08-31 10:43:05 +01:00
Ismael Juma 065043a14f Use process instead of main as the latter is deprecated. 2011-08-02 10:26:03 +01:00
Matei Zaharia cf8f5de61b Merge branch 'master' into scala-2.9 
Conflicts:
	project/build.properties
	repl/src/main/scala/spark/repl/SparkInterpreterLoop.scala
 2011-07-14 17:48:56 -04:00
Matei Zaharia 02678724a4 Update version number to 0.4-SNAPSHOT 2011-07-14 17:47:39 -04:00
Matei Zaharia b187675b68 Print version number 0.3 in REPL 2011-06-26 18:27:01 -07:00
Matei Zaharia b49d1be65b Ensure logging is initialized before any Spark threads run in the REPL 2011-05-31 23:54:48 -07:00
Matei Zaharia 90f924202b Another fix ported forward for the REPL 2011-05-31 23:11:49 -07:00
Matei Zaharia 73975d7491 Further fixes to interpreter (adding in some code generation changes I 
missed before and setting SparkEnv properly on the threads that execute
each line in the 2.9 interpreter).
 2011-05-31 22:05:24 -07:00
Matei Zaharia d52660c969 Ported code generation changes from 2.8 interpreter (to use a class for 
each line's object rather than a singleton object so that we can ship
these classes to worker nodes). This is pretty hairy stuff, which would
be nice to avoid in the future by integrating with the interpreter some
other way.
 2011-05-31 19:23:15 -07:00
Matei Zaharia bcce6e8d01 Various work to use the 2.9 interpreter 2011-05-31 17:31:51 -07:00
Ismael Juma 1396678baa Move REPL classes to separate module. 2011-05-27 11:22:50 +01:00