This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.
The latest Azure HPC SKUs series - HB and HC, unleash the true power of Azure's performant and scalable HPC network interconnect by virtue of Single Root IO Virtualization (SR-IOV). These high performance VM series offer true InfiniBand/RDMA capabilities and enables all InfiniBand based MPI libraries to run on Azure.
To help HPC users seamlessly take advantage of all the platform features, the Azure HPC team offers a new CentOS based HPC image in Azure Marketplace (published by Rogue Wave Software). This image includes optimizations and recommended configurations aimed at best performance, as well as all popular InfiniBand/RDMA based MPI libraries.
Below are the full list of configuration updates and software installations (on top of a vanilla Azure CentOS VM image):
Configuration Updates:
The included configuration updates are based on optimization recommendations from vendors and partners, as well as learnings from common HPC workloads and usage practices in traditional HPC systems.
- Azure Linux Agent (WAAgent)
- Limit waagent's (VM agent running on every Azure Linux VM) usage of CPU/memory resources.
- Optionally consider disabling waagent at the beginning of your job script, and enabling it back at the end, for CPU sensitive workloads as follows:
sudo systemctl stop waagent <HPC job> sudo systemctl restart waagent
- Higher Memory Limits
- Set max-locked-memory limit to unlimited
- Set number of open files limit to 65535
- Zone Reclaim mode
- Set zone_reclaim_mode to 1
- Disable firewall daemon to help MPI job launchers
- Disable NFS GSS proxy to improve NFS performance
Software Installations:
The CentOS HPC image also comes with a set of pre-installed software packages:
- Mellanox OFED
- Pre-configured IPoIB (IP-over-InfiniBand)
- Development Tools, including GCC (recommended version by platform vendors)
- MPI Libraries and Communication Runtimes
The MPI libraries are configured as environment modules. Users can select any preferred MPI flavor as follows:
module load mpi/<mpi-flavor>
Azure HPC team aims to keep the CentOS HPC images up-to-date with latest software versions.
Deploying the CentOS HPC Image
The CentOS HPC images are available from Azure Marketplace, and it can be deployed through a variety of deployment vehicles (CycleCloud, Batch, ARM templates, etc).
Go ahead and give this image a try for your HPC workloads! We are all ears for your feedback; any additional recommendations for optimizations, including any other software frameworks as part of this image, or perhaps another OS image.
Note: This image is intended *only* for the SR-IOV enabled Azure instances (such as Azure HB, HC, etc.).