microsoft
/
spark
зеркало из https://github.com/microsoft/spark.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
1 124 коммитов 13 Ветки 28 Теги 178 MiB
Граф коммитов

1124 Коммитов

Автор SHA1 Сообщение Дата
Matei Zaharia 1473987fb7 Fixed classpath for tests 2010-06-10 22:36:45 -07:00
Matei Zaharia 359e84c585 Use new Nexus API 2010-06-10 22:09:13 -07:00
Matei Zaharia 92246c843b Initial work on 2.8 port 2010-06-10 21:50:55 -07:00
Matei Zaharia c177a546a5 Ignore .DS_Store 2010-06-10 18:08:59 -07:00
Mosharaf Chowdhury 7ab703117a Added timers around BroadcastTest and sendBroadcast. 
Turned OFF saving to HDFS for now for stress tests.
pqOfSources is ordered by least leechers again.
 2010-05-18 20:49:59 -07:00
Mosharaf Chowdhury 2d381c974e Added flush calls after all writeObject calls as well as after creating every 
ObjectOutputStream object.
 2010-05-16 17:06:05 -07:00
Mosharaf Chowdhury e85bb3f04d All ObjectOutputStream objects created before OInputS objects. 2010-05-16 16:24:48 -07:00
Mosharaf Chowdhury 520b594bdf "if (!local) { sendBroadcast }" must be called after all the variables have 
been created /initialized
 2010-05-16 16:20:00 -07:00
Mosharaf Chowdhury 4f0b7eb02d SplitStream was not working in EC2. 
We have turned OFF SSB for now.
 2010-05-16 13:00:47 -07:00
Mosharaf Chowdhury 53a2367c9c - SplitStream working in local machine for single variable broadcast 
- Removed delays before publishing and receiving.
- Commented out some prints.
 2010-05-04 15:32:30 -07:00
root 1c90a32621 Fix native build to use build directory 2010-04-30 22:41:21 +00:00
Mosharaf Chowdhury d0a92571dd - Should work, but not tested yet. 
- Right now, each variable has to come one after another. Within a single
  variable, blocks can come out-of-order.
 2010-04-21 19:48:25 -07:00
Mosharaf Chowdhury c0117f9473 Added flesh to publish/deliver functions. 2010-04-21 02:01:34 -07:00
Mosharaf Chowdhury e2f21279be Moved SplitStreamClient inside the BroadcastSS object with the decision that 
there should be only a single SSClient for the whole Spark program instead of
one for each broadcasted variable.

It's still working well though.
 2010-04-20 19:16:27 -07:00
Mosharaf Chowdhury d2f1d0151a SplitStream integration in progress. 2010-04-20 02:08:48 -07:00
Mosharaf Chowdhury 1c1ac3161d More porting of SplitStream code. 2010-04-19 20:32:17 -07:00
Mosharaf Chowdhury dc2c69e659 SplitStream implementation in progress. 2010-04-19 00:14:53 -07:00
Mosharaf Chowdhury bb0178d1e4 - Receiving retry now starts from where the last try left off, not from the very 
beginning.
- Some refactoring.
 2010-04-15 22:41:44 -07:00
Mosharaf Chowdhury ee6c524fdf Fixed some bugs in speed-based PQ-ing. 2010-04-15 21:39:57 -07:00
Mosharaf Chowdhury c6962f516e Several things, but the most important one is that now we are using node speed 
to select source instead of leecher count.
 2010-04-14 21:19:32 -07:00
Mosharaf Chowdhury e0db4e0482 - HDFS storing is in separate thread. 
- Receivers now ask for a range instead of expecting the whole variable. But,
  they are still asking for the whole range from a single source.
- Next step: make receivers ask for different parts from different sources.
  Also, make sure that Master sends back a list of sources instead of a single
  one.
 2010-04-04 13:50:25 -07:00
Matei Zaharia 10cf3828ad Imported Mosharaf's multi-tracker branch 2010-04-03 23:50:04 -07:00
Matei Zaharia 06aac8a889 Imported changes from old repository (mostly Mosharaf's work, 
plus some fault tolerance code).
 2010-04-03 23:44:55 -07:00
Matei Zaharia df29d0ea4c Initial commit 2010-03-29 16:17:55 -07:00