Ultra Disk Storage for HPC and GPU VMs

Posted by

This post has been republished via RSS; it originally appeared at: Azure Compute articles.

Ultra Disk Storage is now generally available for the HPC and GPU VM sizes.


Ultra Disks

Ultra Disks are the highest performance tier of Azure managed disks for data-intensive workloads. They deliver high throughput and IOPS, and consistent low latency disk storage. Customers can dynamically change the performance of the disks without the need to restart your virtual machines (VM).

For HPC/AI workloads, the Ultra Disks can be used as a higher performance tier to Premium Disks for remote storage scenarios (NFS pools, parallel file systems, etc.)


Considerations for Ultra Disks vis-a-vis Premium Disks

Note that the availability of Ultra Disks by region/AZ/SKU is different (and restrictive today) from that of Premium Disks. Even though a VM size is supported, the support is different per region/zone.


HPC and GPU VM SKUs with Ultra Disks

In addition to the support for Premium Disks, the Ultra Disks can now also be attached to the following H* and N* VM sizes (depending on region/zone as outlined above):

  • HPC: HBv2, HB, HC
    • Currently HBv3 is not co-located with Ultra Disk. So as an example, Ultra Disk cannot be attached to an HBv3 VM in any region where HBv3 is live today.
  • GPU: NDv4, NDv2, ND, NC_T4_v3, NCv3, NCv2, NVv4, NVv3



The table below demonstrated averaged bandwidth, IOPS and latency numbers on some of the HPC and GPU SKUs. These are obtained obtained using fio.

  • The biggest improvement from Ultra Disks on the H*/N* VMs is to latencies that are now sub-millisecond (10x lower than Premium Disks).
  • For most scenarios, the VM limits are the bottleneck with Ultra Disks and the disk limits becomes the bottleneck with Premium Disks. The suspected VM limits are anticipated to be increased in the coming weeks.
    • VM limits are pre-assigned limits for managed disk performance to each VM size.
  • HBv2 and HC obtain max write bandwidth with Ultra Disk. For many other SKUs, However the bandwidth obtained with an Ultra Disk is similar to what is obtained with Premium Disk.
  • A single Ultra Disk can hit the , hence removing the need for striping multiple disks and making disk management easier.
    • The IOPS obtained with a single Ultra Disk is 2.5-4x of a single Premium Disk.
    • Note the results obtained below are using both the Disks at the highest performance tiers
VM SKU Premium (20000 GB) Ultra (1024 GB)
Spec: BW 900 MBps, 20K IOPS Spec: BW 2000 MBps, 160K IOPS
BW (MBps) IOPS (K) Latency (ms) BW (MBps) IOPS (K) Latency (ms)
HC44 876 (R, W) 20 2.495 700 (R), 1944 (W) 64 0.215
HB120_v2 876 (R, W) 18 2.422 840 (R), 1946 (W) 30 0.235
NV48_v3 880 (R, W) 20 1.51 1172 (R, W) 80 0.339
NV32_v4 703 (R, W) 20 2.258 703 (R, W) 49 0.242
NDv2 876 (R, W) 20 2.207 710 (R), 1173 (W) 78 0.216
NC64_T4_v3 704 (R, W) 20 2.149 704 (R, W) 49 0.254


Impact of Accelerated Networking

None. AccelNet does not improve Managed Disks performance. Exceptions maybe corner cases of the VMs doing heavy VM-VM network traffic over the Ethernet interface while doing VM-Managed Disk traffic at the same time.



Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.