5f160e8938
- CLI fixes - Add more convenience subcommands |
||
|---|---|---|
| cascade | ||
| config_templates | ||
| convoy | ||
| docs | ||
| recipes | ||
| resources | ||
| scripts | ||
| tfm | ||
| .gitattributes | ||
| .gitignore | ||
| .travis.yml | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| LICENSE | ||
| README.md | ||
| requirements.txt | ||
| shipyard.py | ||
README.md
Batch Shipyard
Batch Shipyard is a tool to help provision and execute batch-style Docker workloads on Azure Batch compute pools. No experience with the Azure Batch SDK is needed; run your Dockerized tasks with easy-to-understand configuration files!
Major Features
- Automated Docker Host Engine installation tuned for Azure Batch compute nodes
- Automated deployment of required Docker images to compute nodes
- Accelerated Docker image deployment at scale to compute pools consisting of a large number of VMs via private peer-to-peer distribution of Docker images among the compute nodes
- Comprehensive data movement support: move data easily between locally accessible storage systems, Azure Blob or File Storage, and compute nodes
- Automated Docker Private Registry instance creation on compute nodes with Docker images backed to Azure Storage if specified
- Automatic shared data volume support:
- Azure File Docker Volume Driver installation and share setup for SMB/CIFS backed to Azure Storage if specified
- GlusterFS distributed network file system installation and setup if specified
- Seamless integration with Azure Batch job, task and file concepts along with full pass-through of the Azure Batch API to containers executed on compute nodes
- Support for Azure Batch task dependencies allowing complex processing pipelines and DAGs with Docker containers
- Transparent support for GPU accelerated Docker applications on Azure N-Series VM instances (Preview)
- Support for multi-instance tasks to accommodate Dockerized MPI and multi-node cluster applications on compute pools with automatic job cleanup
- Transparent assist for running Docker containers utilizing Infiniband/RDMA for MPI on HPC low-latency Azure VM instances:
- Automatic setup of SSH tunneling to Docker Hosts on compute nodes if specified
Installation
Installation can be performed in any of the following methods:
- Clone the repository:
git clone https://github.com/Azure/batch-shipyard.git - Download the latest release and extract to your local machine
- Run the alfpark/batch-shipyard:cli-latest Docker image
Please see this page for more information regarding installation and requirements.
Batch Shipyard Compute Node OS Support
Batch Shipyard is currently only compatible with Azure Batch supported Marketplace Linux VMs.
Documentation
Please refer to this guide for a complete primer on concepts, usage and a quickstart guide.
Please visit the recipes directory for different sample Docker workloads using Azure Batch and Batch Shipyard after you have completed the primer.
ChangeLog
See the CHANGELOG.md file.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.