mozilla
/
taar_loader
зеркало из https://github.com/mozilla/taar_loader.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Граф коммитов

  • e41df7e1f1 pile-o logging and increased the sample size by 100x master Victor Ng 2018-02-06 09:41:03 -0500
  • 9dbcecb462 added extra JSON serialization safeties Victor Ng 2018-02-06 09:27:51 -0500
  • 4648c6cead Added extra check to filter out empty string client_id values. Drop k/v pairs in the JSON blob where keys have empty values. Victor Ng 2018-02-02 00:37:42 -0500
  • 4eac02a6ca added a Makefile so I don't have to remember how to upload to PyPI Victor Ng 2018-02-01 21:18:07 -0500
  • c5a8842901 removed the call to push boto3 to spark workers Victor Ng 2018-02-01 21:16:46 -0500
  • 477846f9c9 added more metadata to support PyPI releases/1.0 Victor Ng 2018-02-01 21:14:13 -0500
  • 82215f1b64 added readme Victor Ng 2018-02-01 20:57:27 -0500
  • c9fee0acf7 dropped dead code module Victor Ng 2018-02-01 20:55:58 -0500
  • ab5a82c427 Added lots of docstirngs to make it clear what is going on. Victor Ng 2018-02-01 20:53:52 -0500
  • 4669884b51 Added the dynamo_reducer to the last stage of processing of the RDD. Victor Ng 2018-02-01 20:53:38 -0500
  • 7a9948923b Removed the unnecessary `load_parquet` closure function and just inlined the relevant template code into the etl function. Victor Ng 2018-02-01 20:52:04 -0500
  • 07a9772d8a Dropped reducer function in taar_dynamo as it's been moved into the filters submodule. Victor Ng 2018-02-01 20:40:18 -0500
  • 9967f9a96f Added a force_write optional argument to the dynamo_reducer. Pulled out some in-function import statements to the module top level. Victor Ng 2018-02-01 20:37:23 -0500
  • 88fb87d6d1 updated airflow_job.py from atmo testing Victor Ng 2018-02-01 20:36:56 -0500
  • 0b644246c0 more minor patches Victor Ng 2018-02-01 17:06:45 -0500
  • 4f79867117 More refactoring of the dynamo loaders. Victor Ng 2018-02-01 11:39:23 -0500
  • e0d6e329fb Refactored code so that loading package into spark nodes is possible. Victor Ng 2018-01-30 14:38:33 -0500
  • 0d4d34cd6a added wheel dependencies for argparse and boto3 so that we can safely load this code on spark nodes Victor Ng 2018-01-30 14:24:22 -0500
  • 35dbeacf6e init commit. Victor Ng 2018-01-30 12:38:39 -0500