Azure
/
aztk
зеркало из https://github.com/Azure/aztk.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
334 коммитов 22 Ветки 17 Теги 9,2 MiB
Граф коммитов

334 Коммитов

Автор SHA1 Сообщение Дата
Pablo b13cffcfee add docs for resource_monitor plugin 2018-04-18 15:32:24 -07:00
Pablo ade9bf9a3a remove bad characters from previous checking 2018-04-16 12:42:11 -07:00
Pablo 9ff68c6a63 updates to make plugins work in container again 2018-04-13 15:52:55 -07:00
Pablo 7490c3a693 Merge branch 'feature/pluginv2' into feature/metrics 2018-04-13 10:40:17 -07:00
Timothee Guerin f8c0d74e39 fix merge issue 2018-04-13 10:12:54 -07:00
Pablo f1ee6d4759 Merge branch 'feature/pluginv2' into feature/metrics 2018-04-13 10:08:27 -07:00
Timothee Guerin 90aceaefb5 merge master 2018-04-13 10:05:31 -07:00
Pablo 16467c4f20 remove sudo calls and update default dashboard 2018-04-13 09:06:37 -07:00
Pablo 43b5307784 make it work with multiple nodes 2018-04-12 16:27:34 -07:00
Pablo abd50d246a remove unused plugins and add required files to resource_mon 2018-04-12 11:22:41 -07:00
Pablo 9f5ce63810 start refactor into single plugin 2018-04-11 16:39:03 -07:00
Jacob Freck 7ef721f0c1
Feature: getting started script (#475) 
* initial changes for getting started scripts

* add temp error handling

* rename file - fix typo

* add debug strings

* add handling for existing user

* WIP: wait for subprocess to complete to get exit code

* WIP: handle existing user and refactor code

* WIP: add missing return statements

* WIP: fix typo

* start sdk refactor

* mostly working create

* working happy create path

* handle errors for vnet, aad application

* make account setup interactive

* add prompt

* add docs

* rename account_setup_refac to account_setup

* add some logging

* pip install msrest, azure-cli-core, import issues

* remove in script pip, add shell wrapper program

* ellipsis to period

* update branch name for account_setup.sh

* docstring

* retry resource group creation

* fix typo, update retry

* explicitly set output location

* wget overwrite flag, docs update

* add prompt for multi tenants

* fix bug with batch account creation

* add spinner, print statements, fix formatting bug

* fix param bug
 2018-04-11 13:27:55 -07:00
Pablo 091293b4c4 changes to find .env files 2018-04-11 09:54:07 -07:00
Pablo bf642a4cf1 initial commit with grafana and influxdb 2018-04-11 08:09:45 -07:00
Jacob Freck 44a07654aa
Feature: spark debug tool (#455) 
* start implementation of cluster debug utility

* update debug program

* update debug

* fix output directory structure

* cleanup output, add error checking

* sort imports

* start untar

* extract tar

* add debug.py to pylintc ignore, line too long

* crlf->lf

* add app logs

* call get_spark_app_logs, typos

* add docs

* remove debug.py from pylintrc ignore

* added debug.py back to pylint ignore

* change pylint ignore

* remove commented log

* update cluster_run

* refactor cluster_copy

* update debug, add spinner for run and copy

* make new sdk cluster_download endpoint
 2018-04-09 15:02:43 -07:00
Jacob Freck 61e7c591cd
Feature: Spark vnet custom dns hostname fix (#490) 
* add hostname to /etc/hosts

* conditionally set hostname in /etc/hosts
 2018-04-09 10:22:32 -07:00
Jacob Freck 013f6e402f
Bug: Spark shuffle service worker registration fail (#492) 
* stop calling start-shuffle-service.sh script

* whitespace

* remove unused method
 2018-04-09 10:10:48 -07:00
Jacob Freck 1eaa1b6e42
Feature: add internal flag to node commands (#482) 
* add internal ssh flag

* add --internal flag to cluster get

* cluster run internal flag

* fix add command back

* cluster copy internal

* fix method params

* fix method params

* add debug statement

* fix params

* remove debug statement

* fixes

* add debug statement

* remove debug statement

* add hostname to /etc/hosts

* remove hostname from /etc/hosts

* add sdk docs for internal switch in cluster run and copy
 2018-04-06 15:59:13 -07:00
Jacob Freck be8cd2a490
Bug: Remove unused ssh plugin flags (#488) 2018-04-06 14:55:47 -07:00
Jacob Freck a33bdbc5a9
Bug: fix broken spark init command (#486) 2018-04-06 14:10:40 -07:00
Jacob Freck 4ef3dd09df
Bug: add spark.history.fs.logDirectory to required keys (#456) 
* add spark.history.fs.logDirectory to requried keys

* add spark_event_log_enabled_key to required_keys

* docs, add history server config to spark-defaults.conf

* fix bad logic

* crlf->lf
 2018-04-05 14:11:35 -07:00
Timothee Guerin a6c08c4229 Fix jupyterlab 2018-04-05 12:59:32 -07:00
Jacob Freck 32de752d53
Feature: Spark add output logs flag (#468) 
* add output flag to cluster submit

* add output flag to cluster app-logs

* add output flag to job get-app-logs

* sort imports

* make spinner context
 2018-04-05 12:21:56 -07:00
Jacob Freck 8889059aad
Feature: match cluster submit exit code in cli (#478) 2018-04-05 11:54:25 -07:00
Jacob Freck a59fe8b959
Bug: throw error if submitting before master elected (#479) 2018-04-05 11:51:57 -07:00
Timothee Guerin 7f1e77ec15 Added types 2018-04-05 09:38:31 -07:00
Timothee Guerin 0f06d69397 Define plugin docs 2018-04-05 09:28:22 -07:00
Timothee Guerin 35ff863456 more 2018-04-05 09:00:17 -07:00
Jacob Freck 82ad0296af
Bug: add gitattributes file (#470) 
Bug: line endings, add gitattributes file
 2018-04-04 13:44:26 -07:00
Jacob Freck ee1e61bb9d
Bug: fix spark job submit path (#474) 
* fix job submit path, fix raise error, remove print

* source bashrc before executing
 2018-04-03 11:19:35 -07:00
Pablo Selem da61337bfe
Feature: JupyterLab plugin (#459) 
* initial commit

* enable jupyter lab as a default plugin

* remove hack text and add more logging

* remove docker compose code. it is not used yet

* remove unused code and comment
 2018-03-29 09:10:11 -07:00
Jacob Freck c1f43c73c1
Bug: fix aztk cluster submit paths, imports (#464) 
* fix cluster submit

* add export pythonpath to docker_main
 2018-03-27 16:05:54 -07:00
Timothee Guerin a2c5d0db6f Fix: tests 2018-03-27 10:10:45 -07:00
Timothee Guerin 6f44148f59 Merge master 2018-03-26 16:42:06 -07:00
Timothee Guerin 2ba79fb05a tweak 2018-03-26 16:23:12 -07:00
Timothee Guerin ef60f2c8a1 Running plugins on the host works 2018-03-26 14:35:34 -07:00
Timothee Guerin 3ecd48caae More fixes 2018-03-26 13:22:57 -07:00
Timothee Guerin fdaafca3f6 Fix 2018-03-26 12:29:21 -07:00
Jacob Freck 2dd7891499
Bug: add support for jars, pyfiles, files in Jobs (#408) 
* add support for jars, pyfiles, files, refactor JobConfig

* set encoding explicitly

* fix typerror bug in mixed_mode()
 2018-03-26 11:38:05 -07:00
Timothee Guerin d1e738204b Works 2018-03-26 11:27:15 -07:00
Timothee Guerin ff7d3106f0 Starting spark again 2018-03-26 11:02:58 -07:00
Timothee Guerin 7eece21f78 Starts 2018-03-26 10:22:55 -07:00
Timothee Guerin c77724b128 More fix 2018-03-23 16:48:37 -07:00
Timothee Guerin 58e06db489 fix issues 2018-03-23 16:37:56 -07:00
Timothee Guerin 8cee57b8a2 Wip docker run on node 2018-03-23 15:58:53 -07:00
Timothee Guerin 578657ca99 Docker runs now 2018-03-23 15:19:02 -07:00
Jacob Freck 5761a3663a
Bug: set explicit file open encoding (#448) 
* explicit file encoding

* crlf->lf
 2018-03-23 13:42:30 -07:00
Timothee Guerin 77c82eeaea Start 2018-03-23 11:30:40 -07:00
Timothee Guerin 373c04a218 More env renaming 2018-03-23 09:39:05 -07:00
Timothee Guerin ee307e0a01 FIx more 2018-03-22 18:23:30 -07:00