This post has been republished via RSS; it originally appeared at: AI - Customer Engineering Team Blog articles.
We are excited to announce the preview of AMD-based confidential virtual machines (VMs) for cluster nodes on Azure Databricks. Confidential VMs are part of the Azure Confidential Computing (ACC) portfolio and provide a trusted execution environment (TEE) for Azure Databricks clusters, protecting data while in use in memory. This provides an additional layer of security for sensitive data, helping organizations meet compliance requirements and protect their most valuable data assets.
To use confidential VMs on Azure Databricks, customers need to select one of the confidential VM types when creating a cluster. This type of cluster can then be used for any workload that requires the protection of highly sensitive data in memory.
For compute-optimized needs, DCasv5 confidential VMs are available, and for memory-optimized needs, ECasv5 confidential VMs can be used. These VMs will be made available in the following regions over the next few days: East US, West US, North EU, and West EU.
Databricks partnership in Confidential Computing:
"We are thrilled to have collaborated with Microsoft to introduce Azure Databricks support for Azure confidential computing,” said David Meyer, Senior Vice President of Product, Databricks. “With the Databricks Lakehouse, customers can build an end-to-end data platform with increased confidentiality and privacy by encrypting data in use thanks to AMD-based confidential virtual machines.”
Learn more:
Watch the demo: Azure Databricks on Confidential VMs Overview and Demo
Read the documentation: https://aka.ms/CVM-ADB-docs.
Read the Databricks announcement blog.