Azure Container Storage in Public Preview

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.

Organizations everywhere are rapidly modernizing applications to leverage the agility and scale of containers. To accelerate your journey to run production-scale stateful workloads on Azure Kubernetes Service (AKS), we are excited to announce the Public Preview of Azure Container Storage, a fully managed container-native cloud storage solution. Azure Container Storage is a software defined storage solution and tightly integrated with containers — providing operational simplicity for customers using persistent storage to run workloads.  

Customers running stateful container applications today face multiple challenges including scaling bottlenecks, complexity of configuration or management, and high cost due to retrofitted storage solutions or the need to custom deploy and self-manage open-source options. With Azure Container Storage, you can now leverage different block storage options via a consistent, unified, and fully managed experience optimized for your workload.

Overview of Azure Container Storage

As stateful applications running on Kubernetes grow, users need to navigate the complexity of identifying a storage offering, pinpointing the CSI driver, deciphering the differences in volume handling and configuring it in their environment repeatedly for every cluster, application and workload. Azure Container Storage simplifies this experience by providing a consistent volume management experience, extending CSI constructs, for multiple block storage offerings in Azure. Additionally, Azure Container Storage reduces ongoing operational overhead by providing rapid scale out and failover of storage alongside container lifecycle — offering the same uptime benefits to stateful applications which stateless applications enjoy with Kubernetes.

Azure Container Storage introduces — a grouping of storage resources presented as a single, unified storage resource for your AKS cluster. Storage Pool provides an abstracted storage layer on multiple options including ephemeral storage, Azure Disk or Elastic SAN, enabling you leverage the storage that best aligns with your workload needs (see architectural diagram below). Within a Storage Pool, you can create Persistent Volumes (PV) to store your application data. PVs are closely tied to the container pod lifecycle but offer high durability which can persist data even if your is terminated or recreated. Currently, users running on AKS deploy an Azure Disk for each PV provisioning and managing IOPS, MBps, and capacity per PV. Azure Container Storage architecture enables users to specify properties for a Storage Pool dynamically allocating and sharing to be across PVs in a pool, optimizing total cost of ownership (TCO) and maximizing performance across application containers.

As business-critical stateful workloads move to Kubernetes, data protection and recoverability are top of mind. Azure Container Storage offers volume snapshots integrated with CSI to enable you to capture the current state of a volume, including its data, metadata, and configuration. Snapshots can then be used to create new volumes or restore an existing data to a previous state – allowing you to rapidly recover from planned or unplanned errors. You can also leverage volume creation from snapshots, to create and mount volumes at scale using a baseline volume image for across development, staging and production environments.

Maximize your benefits of running stateful workloads on Azure

Azure Container Storage provides you with the ability to fully leverage the advantages of modernizing your applications, utilizing the flexibility and scalability of cloud-native containers. Below are some key benefits.

Minimize your total cost of ownership (TCO). With Azure Container Storage, PVs dynamically share the provisioned resources on a Storage Pool. This is useful when you have small working sets, but each requires high IOPS and MBps. You can share the IOPS and MBps provisioned on the pool across PVs, improving storage utilization and price-performance. We project a TCO saving of 20% when running a stateful Kubernetes cluster on Azure with AKS Azure Container Storage versus other cloud providers. Specifically, Kubernetes clusters on Azure will see 15% savings on storage in . You will yield higher storage savings with Azure Container Storage for data intensive workloads of larger storage footprints.

Scale easily to tailor to your needs. With optimized latencies for PV creation and increased scalability limits, you can rapidly scale out your storage as your application needs evolve. Compared to the current limits, Azure Container Storage allows for a larger number of PVs that can be attached to a pod. This gives you more flexibility in how you design your application architecture, without having to worry about storage limitations. For example, with Azure Container Storage, pods hosted on small AKS nodes are no longer constrained on the number of PVs per core. For more information, refer to scalability targets below. For more information, refer to scalability targets .

Unify your management experience for all your storage needs. This one-stop solution allows you to easily deploy and manage volumes for You can choose the block storage solution that best suits your requirements in terms of price, performance, and capabilities. Currently, for public preview, Azure Container Storage three backing storage options: Azure Elastic SAN, Ephemeral disks, and Azure Disks. New to Kubernetes line up in Azure Storage are Azure Elastic SAN and Ephemeral disk, giving you more options to optimize storage for your workloads. The three options are detailed as follows:

• Elastic SAN (new): By leveraging Azure Elastic SAN, Azure Container Storage simplifies the process of provisioning and managing storage resources for users. Azure Elastic SAN is a new block storage option that provides optimized price-performance through dynamic resource sharing. With native remote block storage support through iSCSI, it also enables for fast attach and detach of volumes. It is an ideal choice for general purpose database workloads, streaming and messaging services, as well as CI/CD environments.
• Ephemeral disks (new): You can now add a local storage device like NVMe Disks available on the AKS nodes (Lsv3 & Lsv3 VMs) to host your PVs. Ephemeral disks are well suited for latency-critical workloads with application-level replication support, such as Cassandra. Data stored on ephemeral disks is not persisted, you can either setup replication in the application to ensure data availability and durability or use it for temporary storage.
• Azure Disks: Azure Disks are our persistent block storage solutions offering best-in-class availability, durability, and price-performance. If you are using Disks for AKS today via CSI drivers and prefer the granular control of the disk SKU selection and configuration, choosing Azure Container Storage with Disks gives you precisely that.

With either Azure Disks or Elastic SAN options, your data stays persistent even if the cluster goes down.

Work seamlessly with Kubernetes. If kubectl commands are your comfort zone, your transition to Azure Container Storage will be seamless. Deployment, management, and automation of volume management flows are all done through familiar kubectl commands through Kubernetes APIs. We will also offer Azure native user experience support with Azure Portal, CLI, and PowerShell by general availability.

Simple pricing to enhance the Azure Container Storage Experience

The pricing for Azure Container Storage is simply comprised of two components: the cost of the underlying storage you use, and a service fee for Azure Container Storage orchestration. During the preview period, Azure Container Storage orchestration is offered for free for container storage pools deployed of any size, no service fee, and you only pay for the underlying storage resources consumed.

At general availability, if you deploy a storage pool with capacity under 5 TiB, there is no service fee. For deployments over 5 TiB, there is a $0.006 fee charged per GiB per month (price based on East) for the additional provisioned capacity beyond 5 TiB. For more information on pricing, please visit the Azure Container Storage Pricing.

Here are a few examples to help you understand the pricing for Azure Container Storage used with different backing storage options.

Managed

For this scenario, the underlying storage resource is Elastic SAN. A 9 TiB storage pool capacity is requested with Elastic SAN as the backing storage option in the East US region with LRS.

Azure Disk

Here are two use case scenarios with disk as the underlying storage resource.

Let’s say you want to deploy Azure Container Storage with a storage pool capacity of 4 TiB with one 4 TiB P50 Premium SSD managed disk as the backing storage option in the East US region through locally redundant storage (LRS). The price breakdown would be as follows:

Now, let’s say you want to deploy Azure Container Storage with a storage pool capacity of 9 TiB.

Ephemeral disk

If you are using ephemeral disk, you will not be charged for any underlying storage, as this is already included as part of your AKS VM costs. You will only be charged the Azure Container Storage service fee if you deploy storage pools greater than 5 TiB in capacity.

Getting Started

If you are interested in participating in the preview, sign up! Azure Container Storage is currently available in preview in selected regions. You can deploy it to your new or existing AKS clusters as a Kubernetes cluster extension through Azure Arc. Once enrolled in the preview program, you will be able to create persistent storage through Azure Container Storage for your Azure Kubernetes Service (AKS) cluster through Kubernetes command line tool (kubectl). You can refer to the Azure Container Storage documentation to learn more about the service.

We look forward to hearing your feedback. Please email us at azcontainerstorage@microsoft.com with any questions.