Azure
/
batch-shipyard
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
batch-shipyard/recipes
История
Fred Park d14b8aa8b4
Post-release fixes 
- Fix merge task with autogen task id
- Recipe updates
- Other minor fixes
 		2019-11-17 22:11:32 +00:00
..
BLAST-CPU Post-release fixes 2019-11-17 22:11:32 +00:00
CNTK-CPU-Infiniband-IntelMPI Fixes and update of recipes (#290) 2019-07-17 18:57:06 -07:00
CNTK-CPU-OpenMPI Fixes and update of recipes (#290) 2019-07-17 18:57:06 -07:00
CNTK-GPU-Infiniband-IntelMPI Update GPU support 2019-08-08 20:36:41 +00:00
CNTK-GPU-OpenMPI Update GPU support 2019-08-08 20:36:41 +00:00
Caffe-CPU Update recipes SSH username 2017-11-13 09:25:20 -08:00
Caffe-GPU Update GPU support 2019-08-08 20:36:41 +00:00
Caffe2-CPU Update recipes SSH username 2017-11-13 09:25:20 -08:00
Caffe2-GPU Update recipes to refer to platform image docs 2018-04-18 12:35:26 -07:00
Chainer-CPU Update recipes SSH username 2017-11-13 09:25:20 -08:00
Chainer-GPU Update GPU support 2019-08-08 20:36:41 +00:00
DiskSpd-Windows Add Windows recipes 2018-12-10 10:58:47 -08:00
DotNet-Windows Add doc note regarding Windows env vars 2019-09-09 15:26:21 +00:00
FFmpeg-GPU Update GPU support 2019-08-08 20:36:41 +00:00
HPCG-Infiniband-IntelMPI Update docs/recipes for Singularity schema change 2019-11-13 03:15:13 +00:00
HPLinpack-Infiniband-IntelMPI Post-release fixes 2019-11-17 22:11:32 +00:00
HPMLA-CPU-OpenMPI Rename MADL recipe to HPMLA 2018-10-04 09:59:19 -07:00
Keras+Theano-CPU Update recipes SSH username 2017-11-13 09:25:20 -08:00
Keras+Theano-GPU Update GPU support 2019-08-08 20:36:41 +00:00
MXNet-CPU Update recipes SSH username 2017-11-13 09:25:20 -08:00
MXNet-GPU Update GPU support 2019-08-08 20:36:41 +00:00
NAMD-GPU Update GPU support 2019-08-08 20:36:41 +00:00
NAMD-Infiniband-IntelMPI Fixes and update of recipes (#290) 2019-07-17 18:57:06 -07:00
NAMD-TCP Update NAMD-TCP recipe 2019-07-09 21:46:34 +00:00
OSUMicroBenchmarks-Infiniband-MVAPICH Update docs/recipes for Singularity schema change 2019-11-13 03:15:13 +00:00
OpenFOAM-Infiniband-IntelMPI Fixes and update of recipes (#290) 2019-07-17 18:57:06 -07:00
OpenFOAM-Infiniband-OpenMPI Add MVAPICH support 2019-08-12 01:58:39 +00:00
OpenFOAM-TCP-OpenMPI Add Infiniband support with Open MPI and MPICH (#297) 2019-08-05 10:39:08 -04:00
PyTorch-CPU Add PyTorch recipes 2018-11-29 08:57:56 -08:00
PyTorch-GPU Various updates 2019-03-22 14:50:26 -07:00
RNASeq-CPU Add Genomics recipes 2018-11-29 08:58:01 -08:00
RemoteFS-GlusterFS Slurm on Batch feature 2019-02-28 12:11:10 -08:00
RemoteFS-GlusterFS+BatchPool Slurm on Batch feature 2019-02-28 12:11:10 -08:00
RemoteFS-NFS Fix possible null node agent info on list nodes 2019-08-30 15:30:49 +00:00
Slurm+NFS Fix remotefs bootstrap 2019-10-09 15:48:40 +00:00
TensorFlow-CPU Update TensorFlow Docker image ref 2018-06-11 13:37:24 -07:00
TensorFlow-Distributed Update TensorFlow Docker image ref 2018-06-11 13:37:24 -07:00
TensorFlow-GPU Update docs/recipes for Singularity schema change 2019-11-13 03:15:13 +00:00
Torch-CPU Update recipes SSH username 2017-11-13 09:25:20 -08:00
Torch-GPU Update GPU support 2019-08-08 20:36:41 +00:00
mpiBench-Infiniband-MPICH Add MVAPICH support 2019-08-12 01:58:39 +00:00
mpiBench-Infiniband-OpenMPI Add MVAPICH support 2019-08-12 01:58:39 +00:00
mpiBench-IntelMPI Update docs/recipes for Singularity schema change 2019-11-13 03:15:13 +00:00
mpiBench-MPICH Update docs/recipes for Singularity schema change 2019-11-13 03:15:13 +00:00
mpiBench-OpenMPI Update docs/recipes for Singularity schema change 2019-11-13 03:15:13 +00:00
README.md Add MVAPICH support 2019-08-12 01:58:39 +00:00

README.md

Batch Shipyard Recipes

Batch Shipyard can accommodate most containerized Batch and HPC workloads. This area contains recipes and sample container workloads that may relate to your own anticipated scenario. Batch Shipyard is not limited to these programs or types of workloads represented by these recipes, but can run arbitrary containers to execute your Batch and HPC scenario.

Please note that all recipes have sample configurations that you can use to quickly get started, however, some of the config files cannot be used as-is as they may need to be modified for specific values (such as the credentials.yaml file which needs to be populated with your Batch and Storage account credentials). Please review each config file you are planning on using and modify accordingly. As these recipe configuration are intended to show example usage, you will need to modify and configure them for your actual workloads.

If you would like to contribute your recipe, please take a look at this guide before submitting a pull request.

Recipe Collections

Use the following links to quickly navigate to recipe collections:

  1. Benchmarks
  2. Computational Fluid Dynamics (CFD)
  3. Deep Learning
  4. Genomics and Bioinformatics
  5. Molecular Dynamics (MD)
  6. RemoteFS
  7. Slurm on Batch
  8. Video Processing
  9. Windows

Benchmarks

HPCG-Infiniband-IntelMPI

This HPCG-Infiniband-IntelMPI recipe contains information on how to containerize HPCG across Infiniband/RDMA Azure VMs with Intel MPI.

HPLinpack-Infiniband-IntelMPI

This HPLinpack-Infiniband-IntelMPI recipe contains information on how to containerize HPLinpack (HPL) across Infiniband/RDMA Azure VMs with Intel MPI.

mpiBench-Infiniband-MPICH

This mpiBench-Infinibnad-MPICH recipe contains information on how to containerize mpiBench across Infiniband/RDMA Azure VMs with MPICH.

mpiBench-Infiniband-OpenMPI

This mpiBench-Infinibnad-MPICH recipe contains information on how to containerize mpiBench across Infiniband/RDMA Azure VMs with MPICH.

mpiBench-IntelMPI

This mpiBench-IntelMPI recipe contains information on how to containerize mpiBench across Azure VMs with Intel MPI.

mpiBench-MPICH

This mpiBench-MPICH recipe contains information on how to containerize mpiBench across Azure VMs with MPICH.

mpiBench-OpenMPI

This mpiBench-OpenMPI recipe contains information on how to containerize mpiBench across Azure VMs with Open MPI.

OSUMicroBenchmarks-Infiniband-MVAPICH

This OSUMicroBenchmarks-Infiniband-MVAPICH recipe contains information on how to containers the OSU Micro-Benchmarks across Infiniband/RDMA Azure VMs with MVAPICH.

Computational Fluid Dynamics (CFD)

OpenFOAM-Infiniband-IntelMPI

This OpenFOAM-Infiniband-IntelMPI recipe contains information on how to containerize distributed OpenFOAM across Infiniband/RDMA Azure VMs with Intel MPI.

OpenFOAM-TCP-OpenMPI

This OpenFOAM-TCP-OpenMPI recipe contains information on how to containerize distributed OpenFOAM across multiple Azure Batch compute nodes.

Deep Learning

Caffe-CPU

This Caffe-CPU recipe contains information on how to containerize Caffe for use on Azure Batch compute nodes.

Caffe-GPU

This Caffe-GPU recipe contains information on how to containerize Caffe on GPUs for use with N-Series Azure VMs.

Caffe2-CPU

This Caffe2-CPU recipe contains information on how to containerize Caffe2 for use on Azure Batch compute nodes.

Caffe2-GPU

This Caffe2-GPU recipe contains information on how to containerize Caffe2 on GPUs for use with N-Series Azure VMs.

Chainer-CPU

This Chainer-CPU recipe contains information on how to containerize Chainer for use on Azure Batch compute nodes.

Chainer-GPU

This Chainer-GPU recipe contains information on how to containerize Chainer on GPUs for use with N-Series Azure VMs.

CNTK-CPU-Infiniband-IntelMPI

This CNTK-CPU-Infiniband-IntelMPI recipe contains information on how to containerize CNTK for CPUs, including execution across multiple Infiniband/RDMA Azure VMs with multi-instance tasks.

CNTK-CPU-OpenMPI

This CNTK-CPU-OpenMPI recipe contains information on how to containerize CNTK for CPUs, including execution across multiple compute nodes with multi-instance tasks.

CNTK-GPU-Infiniband-IntelMPI

This CNTK-GPU-Infiniband-IntelMPI recipe contains information on how to containerize CNTK on GPUs for use with N-Series Azure VMs, including execution across multiple Infiniband/RDMA Azure VMs with multi-instance tasks.

CNTK-GPU-OpenMPI

This CNTK-GPU-OpenMPI recipe contains information on how to containerize CNTK on GPUs for use with N-Series Azure VMs, including execution across multiple compute nodes and multiple GPUs with multi-instance tasks.

HPMLA-CPU-OpenMPI

This recipe contains information on how to containerize the Microsoft High Performance ML Algorithms (HPMLA) for use across multiple compute nodes.

Keras+Theano-CPU

This Keras+Theano-CPU recipe contains information on how to containerize Keras with the Theano backend for use on Azure Batch compute nodes.

Keras+Theano-GPU

This Keras+Theano-GPU recipe contains information on how to containerize Keras with the Theano backend for use with N-Series Azure VMs.

MXNet-CPU

This MXNet-CPU recipe contains information on how to containerize MXNet for use on Azure Batch compute nodes, including execution across multiple compute nodes with multi-instance tasks.

MXNet-GPU

This MXNet-GPU recipe contains information on how to containerize MXNet on GPUs for use with N-Series Azure VMs, including execution across multiple compute nodes and multiple GPUs with multi-instance tasks.

PyTorch-CPU

This PyTorch-CPU recipe contains information on how to containerize PyTorch for use on Azure Batch compute nodes.

PyTorch-GPU

This PyTorch-GPU recipe contains information on how to containerize PyTorch on GPUs for use with N-series Azure VMs.

TensorFlow-CPU

This TensorFlow-CPU recipe contains information on how to containerize TensorFlow for use on Azure Batch compute nodes.

TensorFlow-Distributed

This TensorFlow-Distributed recipe contains information on how to containerize TensorFlow on GPUs for use with N-series Azure VMs or across multiple CPU nodes.

TensorFlow-GPU

This TensorFlow-GPU recipe contains information on how to containerize TensorFlow on GPUs for use with N-series Azure VMs.

Torch-CPU

This Torch-CPU recipe contains information on how to containerize Torch for use on Azure Batch compute nodes.

Torch-GPU

This Torch-GPU recipe contains information on how to containerize Torch on GPUs for use with N-series Azure VMs.

Genomics and Bioinformatics

BLAST-CPU

This BLAST-CPU recipes contains information on how to containerize a BLAST pipeline with parallelization using Azure Batch compute nodes.

RNASeq-CPU

This RNASeq-CPU recipes contains information on how to containerize a proof-of-concept RNA-Seq pipeline using Azure Batch compute nodes.

Molecular Dynamics (MD)

NAMD-GPU

This NAMD-GPU recipe contains information on how to containerize NAMD on GPUs for use with N-Series Azure VMs.

NAMD-Infiniband-IntelMPI

This NAMD-Infiniband-IntelMPI recipe contains information on how to containerize distributed NAMD across Infiniband/RDMA Azure VMs with Intel MPI.

NAMD-TCP

This NAMD-TCP recipe contains information on how to containerize distributed NAMD across multiple Azure Batch compute nodes using TCP.

RemoteFS Provisioning

RemoteFS-GlusterFS

This RemoteFS-GlusterFS recipe contains information on how to provision a sample multi-VM GlusterFS storage cluster.

RemoteFS-GlusterFS+BatchPool

This RemoteFS-GlusterFS+BatchPool recipe contains information on how to provision a Batch pool and automatically link it against a provisioned GlusterFS storage cluster.

RemoteFS-NFS

This RemoteFS-NFS recipe contains information on how to provision a sample single VM NFS server.

Slurm on Batch

Slurm+NFS

This recipe contains information on how to orchestrate a Slurm cluster with an NFS shared file system.

Video Processing

FFmpeg-GPU

This recipe contains information on how to containerize FFmpeg on GPUs for use with the N-series Azure VMs.

Windows

DiskSpd-Windows

This recipe contains information on how to run a containerized version of the DiskSpd tool with Windows Server Containers on Azure VMs.

DotNet-Windows

This recipe contains information on how to run a sample DotNet application with Windows Server Containers on Azure VMs.