This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.
Welcome to Azure Data Factory’s February monthly update! Here we’ll share the latest updates on what’s new in Azure Data Factory. You can find all our updates here.
We’ll also be hosting our monthly livestream next week on March 14h, 2023! Join us to see some live demos and to ask us your ADF questions!
Join the livestream here.
Table of Contents
Data movement
- Anonymous authentication type supported for Azure Blob storage [Learn more]
- Updated SAP template to easily move SAP data to ADLSGen2 in Delta format [Learn more]
Monitoring
Orchestration
- Set pipeline output value (Public preview) [Learn more]
- Managed Airflow (Public preview) [Learn more]
Developer Productivity
Data movement
Anonymous authentication type supported for Azure Blob storage
Now, users can use the Service Principal authentication type for Azure Blob Storage! This allows you to use Azure role-based access control (Azure RBAC) to grant permissions to a security principal, which may be a user, group, or application service principal, who can then authenticate with Azure AD to authorize a request.
To learn more, read Copy and transform data in Azure Blob Storage - Azure Data Factory & Azure Synapse | Microsoft Learn.
Updated SAP template to easily move SAP data to ADLSGen2 in Delta format
A new SAP template has been added to the Azure Data Factory Template Gallery. This template helps users quickly get started to replicate multiple tables from SAP to ADLSGen2 in Delta format using a key partition.
For more information, read SAP Templates - Azure Data Factory | Microsoft Learn.
Monitoring
Container monitoring view available in default ADF studio
When monitoring your pipeline run, you now have the option to enable the Container view, providing a consolidated view of the activities that ran in the output of your pipeline debug run and in the detailed monitoring view found in the monitoring tab. This view, previously accessible through the ADF Preview Experience, is now generally available in the ADF studio.
To learn more, read Managing Azure Data Factory studio preview experience - Azure Data Factory | Microsoft Learn.
Orchestration
Set pipeline output value (Public Preview)
When building complex workflows in the cloud with Azure Data Factory and Azure Synapse Pipelines, a very common pattern is to separate different workflow branches into child pipelines. Now, we have expanded the Set Variable activity to allow users to set a new system variable, called Pipeline Return Value. This allows you to customize your pipeline return value when communicating between child and parent pipelines.
To learn more about this new system variable, read Setting a pipeline return value with UI.
Managed Airflow
The capability to run Apache Airflow DAGs (Directed Acyclic Graph) within ADF is now in public preview, adding a key Open-Source integration that provides extensibility for orchestrating python-based workflows at scale on Azure. ADF Managed Airflow provides a managed orchestration service for Apache Airflow, simplifying the creation and management of Airflow environments while providing a native integration of Apache Airflow with Azure Active Directory for a single sign-on (SSO) and a more secure solution.
To learn more about Managed Airflow, read Introducing Managed Airflow in Azure Data Factory (microsoft.com).
Developer Productivity
Dark theme support added (Public preview)
As we spend more time on our screens, it has become increasingly important to make accommodations for light sensitivity and help reduce eye strain. With the availability of themes, ADF now offers a new dark color scheme for the Data Factory studio to help improve productivity.
To learn more about the dark theme in ADF, read Introducing Dark Mode for ADF Studio (microsoft.com).
We hope that you found this helpful! Let us know in the comments if there's anything else you'd like to see in our blogs or livestreams. We love hearing your feedback!