8fea9ce092
Feature: Disable scheduling on group of nodes ( #540 )
2018-05-30 13:02:48 -07:00
02f336b0a0
Feature: New Models design with auto validation, default and merging ( #543 )
2018-05-30 09:07:09 -07:00
4d4916e349
Release: v0.7.0 ( #535 )
...
* changelog and migration guide
* update version, links, and doc
* update links
* add docs version
* update changelog
2018-05-01 18:44:26 -07:00
7a7e63c54f
Feature: New Toolkit configuration ( #507 )
2018-05-01 16:36:44 -07:00
9bc76396bc
Docs: Added worker on master docs ( #531 )
2018-05-01 14:40:31 -07:00
779bffb2da
Feature: refactor docker images ( #510 )
...
* add spark2.3.0 hadoop2.8.3 dockerfile
* start update to docker image
* add SPARK_DIST_CLASSPATH to bashrc, source .bashrc in docker run
* add maven install for jars
* docker image update and code fix
* add libthrift (still broken)
* start image refactor, build from source,
* add refactor to r base image
* finish refactor r image
* add storage jars and deps
* exclude netty to get rid of dependency conflict
* add miniconda image
* update 2.2.0 base, anaconda image
* remove unused cuda-8.0 image
* start pipenv implementation
* miniconda version arg
* update anaconda and miniconda image
* style
* pivot to virtualenv
* remove virtualenv from path when submitting apps
* flatten layers
* explicit calls to aztk python instead of activating virtualenv
* update base, miniconda, anaconda
* add compatibility version for base aztk images
* typo fix
* update pom
* update environment variable name
* update environment variables
* add anaconda images base & gpu
* update gpu and miniconda base images
* create venv in cluster create
* update base docker files, remove virtualenv
* fix path
* add exclusion to base images
* update r images
* delete python images (in favor of anaconda and miniconda)
* add miniconda gpu images
* update comment
* update aztk_version_compatibility to dokcer image version
* add a build script
* virutalenv->pipenv, add pipfile & pipfile.lock remove secretstorage
* aztk/staging->aztk/spark
* remove jars, add .null to keep directory
* update pipfile, update jupyter and jupyterlab
* update default images
* update base images to fix hdfs
* update build script with correct path
* add spark1.6.3 anaconda, miniconda, r base and gpu images
* update build script to include spark1.6.3
* mkdir out
* exclude commons lang and slf4j dependencies
* mkdir out
* no fail if dir exists
* update node_scripts
* update env var name
* update env var name
* fix the docker_repo docs
* master->0.7.0
2018-04-30 17:19:01 -07:00
e361c3b0b3
Feature: Readthedocs support ( #497 )
2018-04-26 14:03:45 -07:00
2e995b4899
Feature: spark ui proxy plugin ( #467 )
...
* initial commit
* add args
* add docs
* change default plugins
* update ssh cli ui, remove plugin name
* change conditional
* update docs to include jupyterlab
* remove spark_ui_proxy as default plugin
2018-04-23 12:12:31 -07:00
4ef3dd09df
Bug: add spark.history.fs.logDirectory to required keys ( #456 )
...
* add spark.history.fs.logDirectory to requried keys
* add spark_event_log_enabled_key to required_keys
* docs, add history server config to spark-defaults.conf
* fix bad logic
* crlf->lf
2018-04-05 14:11:35 -07:00
6091b1d390
Docs: update ( #263 )
...
* Update README.md
streamline and update main readme.md
* Update README.md
* Update README.md
* Update 13-configuration.md
* Update 12-docker-image.md
* Update 12-docker-image.md
* Update README.md
* Create README.md
* Update README.md
* Update 10-clusters.md
2017-12-11 17:02:51 -08:00
6c26943819
Feature: update docker image doc ( #251 )
...
* update docker-image readme with new images
* update docs
2017-12-06 14:04:07 -08:00
dbc1adddcd
Feature: Azure Data Lake Store support ( #170 )
...
* initial adls work
* initial native support for ADLS connector
* add core-site.xml and azure storage/adls jars to project
* enable custom jars and core-site.xml changes for custom storage connectors
* remove unused ADL crendentials from environment
* documentation feedback updates
* documentation feedback
* PR feedback
2017-10-19 09:16:14 -07:00
f15d18fd5d
Update docs ( #148 )
...
* Update 11-custom-scripts.md
* Update 11-custom-scripts.md
* Update 12-docker-image.md
* Update 13-configuration.md
* Update 20-spark-submit.md
2017-10-05 10:13:26 -07:00
57cfa9dd1d
Docs/update ( #106 )
...
* Updates to README.md
* typos
* Update 00-getting-started.md
* Update README.md
* Update 30-cloud-storage.md
* faq in readme
* Update 13-configuration.md
* switch to use underscore for consistency
* Update 10-clusters.md
* Update README.md
* Update README.md
* fixes
* fix broken link
* Update 20-spark-submit.md
* update to aztk on README
* aztk fix
* Update 00-getting-started.md
* Update 10-clusters.md
* Update 12-docker-image.md
* Update 13-configuration.md
* Update 20-spark-submit.md
* Update 30-cloud-storage.md
* Update README.md
* Update 10-clusters.md
* vm size link
* dummy commit1
* dummy commit 2
* Update 00-getting-started.md
* Update 13-configuration.md
2017-09-29 22:23:14 -04:00
c2172b5f64
Bug: add spark conf defaults ( #115 )
...
* added default spark conf files to config/
* refactored loading and cleanup of spark conf files
* curated spark conf files and added docs
2017-09-29 13:03:11 -07:00
5f12ff66dc
Feature: prompt for password ( #116 )
...
* prompt for password on cluster create, add secrets validation, add custom ssh priv key
* whitespace
* fixed error
* remove ssh_priv_key from secrets template, add docs
* updated error message, fixed typo
* fixed innacurate message
2017-09-29 11:16:54 -07:00
859c6fa6c8
Feature: ssh configuration file support ( #77 )
...
* added ssh.yaml configuration file
* added jupyter support, better format for ssh.yaml instructions
* refactored read_conf_file method
* rename master ui to web ui
* fixed typo in comments
* added documentation for ssh.yaml
* refactored merge and _merge_dict methods
* changed default ssh experience to require --id cli parameter
* renamed conflicting documentation
* improved docs, changed default port forwarding to standard ports
* fix typo
2017-09-21 17:45:57 -07:00
Timothee Guerin
Jacob Freck
JS
Pablo Selem