История

Jacob Freck 8a060a2f78 Feature: Spark GPU (#206 ) * conditionally install and use nvidia-docker * status statements, and -y flag for install * add example, remove unnecessary ppa * rename custom script, remove print statement, update example * add Dockerfile * fix path in Dockerfile * update Docker images to use service account * updated docs, changed default docker repo for gpu skus * make timing statements more verbose * remove unnecessary script * added gpu docs * fix up docs and numba example		2017-12-04 13:28:05 -08:00
..
spark1.6.3	Feature: Spark GPU (#206 )	2017-12-04 13:28:05 -08:00
spark2.1.0	Feature: Spark GPU (#206 )	2017-12-04 13:28:05 -08:00
spark2.2.0	Feature: Spark GPU (#206 )	2017-12-04 13:28:05 -08:00
README.md	Feature/python container (#210 )	2017-11-09 00:43:32 -08:00

README.md

Python

This Dockerfile is used to build the aztk-python Docker image used by this toolkit. This image uses Anaconda, providing access to a wide range of popular python packages.

You can modify these Dockerfiles to build your own image. However, in mose cases, building on top of the aztk-base image is recommended.

NOTE: If you plan to use Jupyter Notebooks with your Spark cluster, we recommend using this image as Jupyter Notebook comes pre-installed with Anaconda.

How to build this image

This Dockerfile takes in a variable at build time that allow you to specify your desired Anaconda versions: ANACONDA_VERSION

By default, we set ANACONDA_VERSION=anaconda3-5.0.0.

For example, if I wanted to use Anaconda3 v5.0.0 with Spark v2.1.0, I would select the appropriate Dockerfile and build the image as follows:

# spark2.1.0/Dockerfile
docker build \
    --build-arg ANACONDA_VERSION=anaconda3-5.0.0 \
    -t <my_image_tag> .

ANACONDA_VERSION is used to set the version of Anaconda for your cluster.

NOTE: Most versions of Python will work. However, when selecting your Python version, please make sure that the it is compatible with your selected version of Spark.