603a413d12
Feature: nvBLAS and OpenBLAS plugin ( #539 )
...
* add openblas plugin, update gpu docker images with netlib-lgpl
* update images and plugins
* add nvblas plugin
* revert gpu docker image change, add -Pnetlib-lgpl to base images
* change configuraitons to functions, add pugins to cluster.yaml
2018-05-15 17:47:41 -07:00
779bffb2da
Feature: refactor docker images ( #510 )
...
* add spark2.3.0 hadoop2.8.3 dockerfile
* start update to docker image
* add SPARK_DIST_CLASSPATH to bashrc, source .bashrc in docker run
* add maven install for jars
* docker image update and code fix
* add libthrift (still broken)
* start image refactor, build from source,
* add refactor to r base image
* finish refactor r image
* add storage jars and deps
* exclude netty to get rid of dependency conflict
* add miniconda image
* update 2.2.0 base, anaconda image
* remove unused cuda-8.0 image
* start pipenv implementation
* miniconda version arg
* update anaconda and miniconda image
* style
* pivot to virtualenv
* remove virtualenv from path when submitting apps
* flatten layers
* explicit calls to aztk python instead of activating virtualenv
* update base, miniconda, anaconda
* add compatibility version for base aztk images
* typo fix
* update pom
* update environment variable name
* update environment variables
* add anaconda images base & gpu
* update gpu and miniconda base images
* create venv in cluster create
* update base docker files, remove virtualenv
* fix path
* add exclusion to base images
* update r images
* delete python images (in favor of anaconda and miniconda)
* add miniconda gpu images
* update comment
* update aztk_version_compatibility to dokcer image version
* add a build script
* virutalenv->pipenv, add pipfile & pipfile.lock remove secretstorage
* aztk/staging->aztk/spark
* remove jars, add .null to keep directory
* update pipfile, update jupyter and jupyterlab
* update default images
* update base images to fix hdfs
* update build script with correct path
* add spark1.6.3 anaconda, miniconda, r base and gpu images
* update build script to include spark1.6.3
* mkdir out
* exclude commons lang and slf4j dependencies
* mkdir out
* no fail if dir exists
* update node_scripts
* update env var name
* update env var name
* fix the docker_repo docs
* master->0.7.0
2018-04-30 17:19:01 -07:00
5ceec67346
Bug: Fixed R Dockerfiles ( #357 )
...
* Fixed consistency in format
* Give user permissions to reinstall R packages
2018-01-25 08:25:39 -08:00
89948de905
R GPU Dockerfiles ( #341 )
...
* Added dockerfiles for aztk r gpu
* Removed tensorflow/cntk cpu versions
* Enabled configure option for shared lib for python
* Removed temp paths
* Replaced python version with aztk version
* Cleaning up args
* Added variable definition for tensorflow, shorten lines
2018-01-22 16:36:36 -08:00
6091b1d390
Docs: update ( #263 )
...
* Update README.md
streamline and update main readme.md
* Update README.md
* Update README.md
* Update 13-configuration.md
* Update 12-docker-image.md
* Update 12-docker-image.md
* Update README.md
* Create README.md
* Update README.md
* Update 10-clusters.md
2017-12-11 17:02:51 -08:00
40bd2d62f3
Bug: fix wrong path for global secrets ( #265 )
...
* fix wrong path for global secrets
* load spark_conf files correctly
* docker-image docs fix
* docker-image docs fix
* move load_aztk_spark_config function to config.py
2017-12-11 15:06:08 -05:00
0efadefb98
Feature: Sparklyr ( #243 )
...
* Added rstudio server script
* Added rstudio server port to aztk sdk
* Added R dockerfiles
* Added new line on dockerfiles
* Pointing dockerfiles to new aztk-base
* allow any user or application in the server to write to the history server log directory
2017-12-08 11:17:27 -08:00
6c26943819
Feature: update docker image doc ( #251 )
...
* update docker-image readme with new images
* update docs
2017-12-06 14:04:07 -08:00
8a060a2f78
Feature: Spark GPU ( #206 )
...
* conditionally install and use nvidia-docker
* status statements, and -y flag for install
* add example, remove unnecessary ppa
* rename custom script, remove print statement, update example
* add Dockerfile
* fix path in Dockerfile
* update Docker images to use service account
* updated docs, changed default docker repo for gpu skus
* make timing statements more verbose
* remove unnecessary script
* added gpu docs
* fix up docs and numba example
2017-12-04 13:28:05 -08:00
65a56dd657
Feature/python container ( #210 )
...
* added python container, jupyter install script, vanilla container
* Update README.md
* Create README.md
* Create README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update jupyter.sh
* Update README.md
* Update README.md
* python
* readme update
* docker updates
* Update README.md
* Update 12-docker-image.md
* Update constants.py
* add image files for wiki
* update imageS
* .
* dockerfile typo
* dockerfiles
* Removed r
* readme
* update constants.py
* update readme
* readme updates
* readme updates
2017-11-09 00:43:32 -08:00
6ce3f048ba
Bug: fix pyenv init ( #201 )
...
* fix spark-submit command file paths for extra files
* fix files flag typo
* remove too long lines
* fix command
2017-11-02 15:55:40 -07:00
afd2561e14
Bug: fix error with pyenv export on container ( #182 )
2017-10-23 11:36:28 -07:00
d3a4f9c559
Feature/update docker repo ( #130 )
...
* Update cluster.yaml
* update dockerfile, node_scripts, and constants
* update docker-image/README.md
2017-09-30 22:29:53 -07:00
f158ceafc5
Docs/update ( #128 )
...
* Update README.md
* Update 10-clusters.md
* Update README.md
* Update README.md
* Update README.md
2017-09-30 05:02:25 -04:00
03baff6a26
Feature: custom docker images ( #68 )
...
* symlink /home/spark-version to /home/spark-current
* working checkpoint
* working checkpoint
* docker readme update
* removed deprecated dockerfile
* docker cli + docs for docker
* docs update
* docs update
* remove unneeded comments
* docker version and docs updates, misc fixes
2017-09-15 20:57:16 -07:00
Jacob Freck
Brian
JS