azure-hpc/Templates/hpc-cluster

Deploy a Slurm or PBS OSS HPC Cluster in Azure

Deploys a Slurm or PBS OSS cluster with a head node and one or more worker nodes.

1. Fill in the various parameters.

2. Select an existing resource group or enter the name of a new resource group to create.

3. Select the resource group location.

4. Accept the terms and agreements.

5. Click Purchase.

Accessing the cluster

Simply SSH to the master node using the public IP address (PIP).

# ssh azureuser@10.0.0.1

You can log into the head node using the admin user and password specified. Once on the head node you can switch to the HPC user. For security reasons the HPC user cannot login to the head node directly.

Running workloads

HPC User

After SSHing to the head node you can switch to the HPC user specified on creation, the default username is 'hpc'.

This is a special user that should be used to run work and/or SLURM jobs. The HPC user has public key authentication configured across the cluster and can login to any node without a password. The HPC users home directory is a NFS share on the master and shared by all nodes.

To switch to the HPC user.

azureuser@master:~> sudo su hpc
azureuser's password:
hpc@master:/home/azureuser>

Shares

The master node doubles as a NFS server for the worker nodes and exports two shares, one for the HPC user home directory and one for a data disk.

The HPC users home directory is located in /share/home and is shared by all nodes across the cluster.

The master also exports a generic data share under /share/data. This share is mounted under the same location on all worker nodes. This share is backed by 16 disks configured as a RAID-0 volume. You can expect much better IO from this share and it should be used for any shared data across the cluster.

Running a SLURM job

First you'll need to set the worker nodes to IDLE as they were rebooted during the creation process. To do this execute:

hpc@master:~> sudo scontrol update NodeName=worker[0-9] State=IDLE

To verify that SLURM is configured and running as expected you can execute the following.

hpc@master:~> srun -N6 hostname
worker4
worker0
worker2
worker3
worker5
worker1
hpc@master:~>

Replace '6' above with the number of worker nodes your cluster was configured with. The output of the command should print out the hostname of each node.

VM Sizes

The master/head node and worker nodes require VM sizes that support at least 4 data disks being attached, hence A2 and above.

MPI

To run MPI applications you'll need to use the A8/A9 ir H-series 'r' instances which include InfiniBand and RDMA support.

Currently RDMA only supports Intel MPI. You can download the Intel pieces and get an evaluation license from https://software.intel.com/en-us/intel-mpi-library.