mozilla
/
jydoop
зеркало из https://github.com/mozilla/jydoop.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
185 коммитов 1 ветка 0 Теги 306 KiB
Граф коммитов

35 Коммитов

Автор SHA1 Сообщение Дата
Benjamin Smedberg 8077126499 Don't hard-code the list of directories that make up driver.jar when calculating dependencies: use find to build that list. This allows symlinking -toolbox libraries into scripts/ and having them work correctly. 2014-02-21 15:46:59 +00:00
Benjamin Smedberg de61ba9ecc Add a makefile target to launch a python shell with a jydoop-like environment. 2013-12-12 20:06:22 +00:00
Mark Reid 4be925ef09 Minor Makefile cleanup 2013-07-10 14:21:19 -03:00
Mark Reid 7d06f24fdb Update Makefile for compatibility with CDH4 2013-07-10 11:42:18 -03:00
Mark Reid 6600e15f82 Refactored to use HadoopDriver.java 2013-06-17 17:10:52 -03:00
Mark Reid 64b16db3d9 Make Makefile work on mango-gw 2013-06-17 14:10:26 -03:00
Mark Reid 45a5544e6e First stab 2013-06-17 11:54:55 -03:00
Mark Reid 7b606548d9 Update makefile to work on the 'peach' cluster 2013-06-17 10:14:33 -03:00
Taras Glek 13e9e29df7 a possibly-working json.dumps 2013-05-01 18:03:08 -07:00
Taras Glek 62ae5c759f run tests via our wrapper 2013-05-01 14:35:40 -07:00
Gregory Szorc d0ff9d698e Move count_fhr_facets.py into fhr script subdir 2013-04-23 16:25:41 -07:00
Gregory Szorc 35a0136f21 Move common modules to jydoop directory 2013-04-22 13:17:15 -07:00
Benjamin Smedberg 6a632f7bb6 Separate the type and value classes so that values can contain lists and dicts (and are not sortable) while keys cannot contain those and are sortable. 2013-04-19 09:35:57 -07:00
Taras Glek aa0afd15c6 added a way to pull resources out of driver.jar 2013-04-18 16:28:35 -07:00
Benjamin Smedberg 2fdda52ddd Rename pydoop->jydoop because there is already a pretty well-known project called pydoop. 2013-04-08 16:07:14 -07:00
Benjamin Smedberg 3ffed3736e Use Jackson instead of Jyson for speed. See also TODO for future speed improvements in JacksonWrapper.loads. 2013-04-06 09:35:28 -07:00
Taras Glek 56fa7e30ce removed from leftovers from proof-of-concept days 2013-04-04 11:33:14 -07:00
Benjamin Smedberg 086147ca31 Ship all the scripts in driver.jar and select which one to run at runtime (preparation for a standard library of pydoop python scripts). 2013-04-04 09:27:20 -07:00
Benjamin Smedberg c2a23923a8 Implement tests for TypeWritable and its implementation of RawComparator. 2013-04-03 15:18:55 -07:00
Benjamin Smedberg 83c5860c41 Use jyson again. The builtin json module that comes with jython is dog-slow: 9 seconds to parse a single telemetry record! 2013-03-29 11:47:35 -04:00
Benjamin Smedberg a798248a5f Fix makefile to use the new package names and Text is no longer correct. 2013-03-25 15:38:35 -04:00
Benjamin Smedberg 4aeb87af81 Move the Java into the correct directory structure expected by a java compiler, and give it a more correct package name. Add a Writeable class which is used to efficiently serialize just the data we might actually want in a python key/value, so that we can now emit tuples and integers. 2013-03-22 03:27:37 +01:00
Benjamin Smedberg 39475c74ef Use wget -c per taras 2013-03-21 10:38:37 +01:00
Benjamin Smedberg c07e0a22e1 Fix misspelling. 2013-03-21 09:59:45 +01:00
Benjamin Smedberg 13d8b684c6 Actually remove the HDFS driver which we don't need. 2013-03-21 09:57:42 +01:00
Benjamin Smedberg 6a24e3c8b0 Use curl and > to avoid redownload issues. 2013-03-21 09:57:05 +01:00
Benjamin Smedberg 4ecb4b8a90 Use the proper jython-standalone.jar, which makes Jyson unnecessary. Also pull akela. 2013-03-21 09:57:05 +01:00
Benjamin Smedberg 62f16e8d75 Make the path of hbase configurable, and make it easier to build. 2013-03-21 09:50:31 +01:00
Taras Glek cd6fc4b416 Some more docs 2013-03-18 08:45:52 -07:00
Taras Glek fdb36db1ed anr script 2013-03-13 17:45:35 -07:00
Taras Glek aa524c8206 Makefile fix, use compression for hbase map output 2013-03-13 16:21:29 -07:00
Taras Glek 237b2b5e21 some cleanup 2013-03-13 12:12:53 -07:00
Taras Glek c9f56fa339 got jython running against hbase too 2013-03-12 23:33:49 -07:00
Taras Glek b162584b2e first successful map/reduce with json 2013-03-06 21:12:37 -08:00
Taras Glek cb18d8598d got wordcount to run partially in python 2013-03-06 12:02:18 -08:00