Azure
/
aztk
зеркало из https://github.com/Azure/aztk.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
129 коммитов 22 Ветки 17 Теги 9,2 MiB
Граф коммитов

129 Коммитов

Автор SHA1 Сообщение Дата
JS cf6ceff6f7 removed deprecated wasb-dsvm custom script & removed depcredated wasb docker image (#49) 2017-08-23 11:29:27 -04:00
JS fdd87c0a99 Feature/samples (#50) 
* added snapshot of examples from spark-2.2.0

* jupyter samples

* fixed bug

* rename 'pyspark' to 'jupyter'

* curated samples

* Readme update for samples
 2017-08-22 18:36:25 -04:00
Timothee Guerin 5149eacb22 Fix showing password in plain text (#52) 2017-08-22 09:54:18 -07:00
Pablo Selem ae26453ba5 Feature/docs (#51) 
* update configuration.cfg to secrets.cfg

* fix node count pretty print

* add more unit tests

* configure pytest to only run tests from the 'tests' directory

* add missing space

* remove static call to loading secrets file

* initial docs layout

* rename files and add images to getting started

* fix typos

* Add next steps to clusters docs

* Add custom scripts doc

* Initial text for spark submit

* fix merge conflict

* finish spark submit docs

* cloud storage docs

* fix typo in storage docs

* refactor some docs

* pr feedback

* add link to Getting Started in creating account details in main readme
 2017-08-22 09:13:10 -07:00
JS 577ff296b5 Feature/wasb (#47) 
* update configuration.cfg to secrets.cfg

* fix node count pretty print

* add more unit tests

* configure pytest to only run tests from the 'tests' directory

* add missing space

* remove static call to loading secrets file

* initial commit for automatic wasb support

* wasb native support

* fix for wasb

* bug fix in azure_api

* clean up setup_wasb

* README for wasb

* README updates for wasb

* configure wasb creds in start task environment variables

* add missing merged files

* revert back to working state

* added storage account suffix

* removed zip to secrets
 2017-08-22 11:57:45 -04:00
Pablo Selem 4822172b6a Feature/secrets.cfg (#43) 
* update configuration.cfg to secrets.cfg

* fix node count pretty print

* add more unit tests

* configure pytest to only run tests from the 'tests' directory

* add missing space

* remove static call to loading secrets file
 2017-08-21 12:36:45 -07:00
JS 213f9226b0 removed dsvm code (#48) 2017-08-21 02:08:40 -04:00
Timothee Guerin b21cae0cc2 Fix: windows path given in the command line (#41) 
* Fix windows path

* Upload custom-script

* Custom script works with windows path

* Fix util

* 24 Hours
 2017-08-19 13:47:44 -07:00
Timothee Guerin d552624705 Feature: read/tail spark submit logs (#40) 
* Tail log command

* tail log is working

* Cleanup code

* fix cr

* Fix cr

* Submit

* Comment

* fix tail

* Sleep
 2017-08-18 12:35:04 -07:00
JS d94c43ed82 Feature: docker 
Feature: docker
 2017-08-17 09:21:00 -07:00
JS fbfdaf489d remove deprecated instructions (#39) 
* remove deprecated instructions

* Update README.md
 2017-08-10 11:00:43 -07:00
Timothee Guerin ca1517acfc Feature: logging instead of prints and more sdk like (#34) 
* Getting started on logging

* new logger

* Cluster list and get api like

* SSH sdk like

* Job submit log

* util

* Cluster create log

* fix cr
 2017-08-02 08:43:26 -07:00
Timothee Guerin aea5e36690 Feature: Setup user with ssh public key instead of password (#32) 
* Login with ssh public key

* Update to have ssh-key command

* Fix readme

* Fix Cr

* remove unused files

* Change head to master
 2017-07-27 10:45:31 -07:00
Timothee Guerin 9236bc883b Fix software circular (#35) 2017-07-24 13:02:46 -07:00
Timothee Guerin 4fe6c66190 Only list spark clusters (#31) 
* Only list spark clusters

* reformat

* reformat

* reformat

* fix cr

* Spark use enum

* Version

* Fix
 2017-07-24 09:17:42 -07:00
Timothee Guerin 17d09ee659 Force convert to bool (#33) 2017-07-21 08:52:28 -07:00
Timothee Guerin 9e45f78600 Simplify the commands to all be under azb and make windows experience better (#29) 
* Added some commands

* More

* Missing file

* Refactor

* update

* Constant exe name

* List and get command

* Added user

* update cluster create, and job submit

* Remove legacy scripts

* Fix cr comments

* remove leftover print
 2017-07-19 10:48:23 -07:00
Timothee Guerin d92df3c0be Initial structure for test framework and CI (#30) 
* Command builder tests

* Setup travis

* Update to travis

* Update to travis

* Update to travis

* Error only

* builder to array
 2017-07-18 14:07:11 -07:00
Timothee Guerin 4a1d9adf25 Revert "Revert "Feature: Setup the cluster with the start task." (#26)" (#28) 
This reverts commit dc1618ae94.
 2017-07-14 12:00:47 -04:00
JS dc1618ae94 Revert "Feature: Setup the cluster with the start task." (#26) 2017-07-13 20:57:04 -07:00
Timothee Guerin d943237c8e Merge pull request #24 from Azure/feature/start-task-master 
Feature: Setup the cluster with the start task.
 2017-07-13 17:16:59 -07:00
Timothee Guerin 6a2a54c2f1 Typo 2017-07-13 15:23:49 -07:00
Timothee Guerin 9a4eccda1f fix spark master ui port 2017-07-13 15:13:34 -07:00
Timothee Guerin 8f5757cea3 Fix broken code after CR 2017-07-13 13:56:52 -07:00
Timothee Guerin 27b14e6adc Finish reviewing 2017-07-13 13:23:19 -07:00
Timothee Guerin aeecacc194 Some cr fixes 2017-07-13 12:44:10 -07:00
Timothee Guerin b72a9bc935 Move juypter wd to be empty 2017-07-12 15:12:25 -07:00
Timothee Guerin f7b22be1dc fix 2017-07-12 13:57:24 -07:00
Timothee Guerin 3404579b98 Pick master 2017-07-12 13:24:40 -07:00
Timothee Guerin 9dc24bd7f6 Pep8 config file 2017-07-12 13:23:16 -07:00
Timothee Guerin 583eefec79 Command builder 2017-07-12 13:10:13 -07:00
Timothee Guerin de32bf6590 Added editorconfig and settings 2017-07-12 12:41:15 -07:00
Timothee Guerin f334de60c7 Cleanup cluster lib 2017-07-12 12:33:18 -07:00
Timothee Guerin 1ee0225f67 Merge master 2017-07-12 12:25:29 -07:00
Timothee Guerin 1ef325056a Starting master as a worker too 2017-07-12 12:22:08 -07:00
Timothee Guerin 96f94e7714 job submit is working 2017-07-12 11:33:42 -07:00
JS 8e764f9c4d Merge pull request #22 from Azure/fix/sdk-helper 
Remove blob client arguments leftover
 2017-07-12 14:31:05 -04:00
Timothee Guerin 8b45359ff6 Jupyter working 2017-07-12 10:52:16 -07:00
Timothee Guerin 4358042e8c Remove blob client 2017-07-12 10:38:53 -07:00
JS 7f23ede1da Merge pull request #20 from Azure/sample/wasb 
custom-script for wasb connection
 2017-07-12 10:12:04 -04:00
jiata 818bb041c5 updates to custom wasb template 2017-07-11 18:36:00 -04:00
Timothee Guerin dc8f397494 Fix juypiter? 2017-07-11 14:52:40 -07:00
Timothee Guerin e9c7f74e79 more 2017-07-11 14:25:17 -07:00
Timothee Guerin 222821cdb2 Use master node id 2017-07-11 09:05:22 -07:00
Timothee Guerin d60f647540 more fixes 2017-07-10 17:03:04 -07:00
Timothee Guerin f9e7db4f41 Fix 2017-07-10 16:34:54 -07:00
Timothee Guerin ef3d84853e Use popen 2017-07-10 16:25:14 -07:00
Timothee Guerin a4026fa702 Setup spark from the start task 2017-07-10 16:04:17 -07:00
Timothee Guerin 0385df991f Working 2017-07-10 14:20:50 -07:00
Timothee Guerin 58201d1b0d Refactor 2017-07-10 12:33:49 -07:00