CentOS-HPC VM Image for SR-IOV enabled Azure HPC VMs

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.

The latest Azure HPC SKUs series - HB and HC, unleash the true power of Azure's performant and scalable HPC network interconnect by virtue of Single Root IO Virtualization (SR-IOV). These high performance VM series offer true InfiniBand/RDMA capabilities and enables all InfiniBand based MPI libraries to run on Azure.

To help HPC users seamlessly take advantage of all the platform features, the Azure HPC team offers a new CentOS based HPC image in Azure Marketplace (published by Rogue Wave Software). This image includes optimizations and recommended configurations aimed at best performance, as well as all popular InfiniBand/RDMA based MPI libraries.

 

Below are the full list of configuration updates and software installations (on top of a vanilla Azure CentOS VM image):

Configuration Updates:
The included configuration updates are based on optimization recommendations from vendors and partners, as well as learnings from common HPC workloads and usage practices in traditional HPC systems.

  • Azure Linux Agent (WAAgent)
    • Limit waagent's (VM agent running on every Azure Linux VM) usage of CPU/memory resources.
    • Optionally consider disabling waagent at the beginning of your job script, and enabling it back at the end, for CPU sensitive workloads as follows:
      sudo systemctl stop waagent
      <HPC job>
      sudo systemctl restart waagent
  • Higher Memory Limits
    • Set max-locked-memory limit to unlimited
    • Set number of open files limit to 65535
  • Zone Reclaim mode
    • Set zone_reclaim_mode to 1
  • Disable firewall daemon to help MPI job launchers
  • Disable NFS GSS proxy to improve NFS performance

 

Software Installations:

The CentOS HPC image also comes with a set of pre-installed software packages:

 

The MPI libraries are configured as environment modules. Users can select any preferred MPI flavor as follows:

module load mpi/<mpi-flavor>

Azure HPC team aims to keep the CentOS HPC images up-to-date with latest software versions. 

 

Deploying the CentOS HPC Image

The CentOS HPC images are available from Azure Marketplace, and it can be deployed through a variety of deployment vehicles (CycleCloud, Batch, ARM templates, etc).

 

Go ahead and give this image a try for your HPC workloads! We are all ears for your feedback; any additional recommendations for optimizations, including any other software frameworks as part of this image, or perhaps another OS image.

 

Note: This image is intended *only* for the SR-IOV enabled Azure instances (such as Azure HB, HC, etc.). 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.