2016-08-31 20:44:38 +03:00
|
|
|
[](https://travis-ci.org/Azure/batch-shipyard)
|
|
|
|
|
[](https://hub.docker.com/r/alfpark/batch-shipyard)
|
|
|
|
|
[](http://microbadger.com/images/alfpark/batch-shipyard)
|
|
|
|
|
|
2016-09-01 07:43:03 +03:00
|
|
|
# Batch Shipyard
|
2016-09-01 06:57:10 +03:00
|
|
|
[Batch Shipyard](https://github.com/Azure/batch-shipyard) is a tool to help
|
2016-09-01 19:09:43 +03:00
|
|
|
provision and execute batch-style Docker workloads on
|
2016-09-01 06:57:10 +03:00
|
|
|
[Azure Batch](https://azure.microsoft.com/en-us/services/batch/) compute
|
2016-09-01 07:43:03 +03:00
|
|
|
pools. No experience with the
|
|
|
|
|
[Azure Batch SDK](https://github.com/Azure/azure-batch-samples) is needed; run
|
2016-09-01 19:09:43 +03:00
|
|
|
your Dockerized tasks with easy-to-understand configuration files!
|
2016-08-31 19:38:33 +03:00
|
|
|
|
2016-09-01 07:43:03 +03:00
|
|
|
## Major Features
|
2016-09-02 19:39:35 +03:00
|
|
|
* Automated [Docker Host Engine](https://www.docker.com) installation tuned
|
|
|
|
|
for Azure Batch compute nodes
|
2016-09-01 01:35:33 +03:00
|
|
|
* Automated deployment of required Docker images to compute nodes
|
2016-08-31 20:44:38 +03:00
|
|
|
* Accelerated Docker image deployment at scale to compute pools consisting of
|
2016-09-01 07:43:03 +03:00
|
|
|
a large number of VMs via private peer-to-peer distribution of Docker images
|
|
|
|
|
among the compute nodes
|
|
|
|
|
* Automated Docker Private Registry instance creation on compute nodes with
|
|
|
|
|
Docker images backed to Azure Storage if specified
|
2016-09-15 22:47:43 +03:00
|
|
|
* Automatic shared data volume support for:
|
|
|
|
|
* [Azure File Docker Volume Driver](https://github.com/Azure/azurefile-dockervolumedriver)
|
|
|
|
|
installation and share setup for SMB/CIFS backed to Azure Storage if
|
|
|
|
|
specified
|
|
|
|
|
* [GlusterFS](https://www.gluster.org/) distributed network file system
|
|
|
|
|
installation and setup if specified
|
2016-08-31 20:44:38 +03:00
|
|
|
* Seamless integration with Azure Batch job, task and file concepts along with
|
2016-09-01 07:43:03 +03:00
|
|
|
full pass-through of the
|
|
|
|
|
[Azure Batch API](https://azure.microsoft.com/en-us/documentation/articles/batch-api-basics/)
|
|
|
|
|
to containers executed on compute nodes
|
2016-09-19 23:13:27 +03:00
|
|
|
* Support for
|
|
|
|
|
[Azure Batch task dependencies](https://azure.microsoft.com/en-us/documentation/articles/batch-task-dependencies/)
|
|
|
|
|
allowing complex processing pipelines and graphs with Docker containers
|
2016-09-02 19:39:35 +03:00
|
|
|
* Transparent support for
|
|
|
|
|
[GPU accelerated Docker applications](https://github.com/NVIDIA/nvidia-docker)
|
|
|
|
|
on [Azure N-Series VM instances](https://azure.microsoft.com/en-us/blog/azure-n-series-preview-availability/)
|
|
|
|
|
([Preview](http://gpu.azure.com/))
|
|
|
|
|
* Support for multi-instance tasks to accomodate Dockerized MPI and multi-node
|
2016-09-06 20:32:42 +03:00
|
|
|
cluster applications on compute pools with automatic job cleanup
|
2016-09-01 08:16:22 +03:00
|
|
|
* Transparent assist for running Docker containers utilizing Infiniband/RDMA
|
2016-09-01 01:35:33 +03:00
|
|
|
for MPI on
|
|
|
|
|
[HPC low-latency Azure VM instances](https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-a8-a9-a10-a11-specs/)
|
2016-09-01 07:43:03 +03:00
|
|
|
(i.e., STANDARD\_A8 and STANDARD\_A9)
|
2016-09-01 01:35:33 +03:00
|
|
|
* Automatic set up of SSH tunneling to Docker Hosts on compute nodes if
|
|
|
|
|
specified
|
2016-08-31 20:44:38 +03:00
|
|
|
|
2016-09-01 07:43:03 +03:00
|
|
|
## Installation
|
2016-08-31 21:09:08 +03:00
|
|
|
Simply clone the repository:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
git clone https://github.com/Azure/batch-shipyard.git
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
or [download the latest release](https://github.com/Azure/batch-shipyard/releases).
|
|
|
|
|
|
2016-09-10 00:06:49 +03:00
|
|
|
Please see [this page](docs/01-batch-shipyard-installation.md) for more
|
2016-09-16 23:00:05 +03:00
|
|
|
information regarding installation and requirements.
|
2016-08-31 20:44:38 +03:00
|
|
|
|
2016-09-01 07:43:03 +03:00
|
|
|
## Batch Shipyard Compute Node OS Support
|
2016-09-08 21:03:40 +03:00
|
|
|
Batch Shipyard is currently only compatible with
|
|
|
|
|
[Azure Batch supported Marketplace Linux VMs](https://azure.microsoft.com/en-us/documentation/articles/batch-linux-nodes/#list-of-virtual-machine-images).
|
2016-08-31 19:38:33 +03:00
|
|
|
|
2016-09-01 07:43:03 +03:00
|
|
|
## Documentation
|
2016-08-31 21:09:08 +03:00
|
|
|
Please refer to
|
2016-09-01 07:43:03 +03:00
|
|
|
[this guide](https://github.com/Azure/batch-shipyard/blob/master/docs)
|
2016-09-15 22:47:43 +03:00
|
|
|
for a complete primer on concepts, usage and a quickstart guide.
|
2016-08-31 19:38:33 +03:00
|
|
|
|
2016-09-01 07:43:03 +03:00
|
|
|
Please visit the
|
2016-09-01 19:07:05 +03:00
|
|
|
[recipes directory](https://github.com/Azure/batch-shipyard/blob/master/recipes)
|
2016-09-01 07:43:03 +03:00
|
|
|
for different sample Docker workloads using Azure Batch and Batch Shipyard
|
|
|
|
|
after you have completed the primer.
|
|
|
|
|
|
|
|
|
|
## ChangeLog
|
2016-08-31 19:38:33 +03:00
|
|
|
See the [CHANGELOG.md](https://github.com/Azure/batch-shipyard/blob/master/CHANGELOG.md)
|
|
|
|
|
file.
|
2016-07-18 18:15:56 +03:00
|
|
|
|
|
|
|
|
* * *
|
2016-08-31 17:59:45 +03:00
|
|
|
This project has adopted the
|
|
|
|
|
[Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
|
|
|
|
|
For more information see the
|
|
|
|
|
[Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
|
|
|
|
|
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any
|
|
|
|
|
additional questions or comments.
|
