ADF Data Flow Updates: CDM and Delta Lake Connectors Generally Available

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.

ADF Data Flow connectors for Common Data Model (CDM) and Delta Lake are both now generally available (GA).

 

This is the documentation link for CDM to learn more about how to read model.json and manifest style of CDM models into ADF. With data flows, you can build powerful ETL processes using CDM formats and then also generate updated manifest files that point to your new, transformed data using CDM as a sink. ADF can use your CDM entity definitions to build ETL projections for transformation and mapping.

 

Here is the documentation link for Delta Lake, which is a Spark-based data lake format that makes working with data in your lake for analytics solution super-easy. With ADF data flows, you can read from Delta Lake folders, transform data, and even update, upsert, insert, delete, and generate new Delta Lake folders using the Delta Lake sink format. You don't need to bring your own Spark cluster, either. ADF provides the Spark compute to create Delta Lake databases.

 

Both of these Data Lake formats are available now in GA under the data flow "Inline Dataset" feature.

REMEMBER: these articles are REPUBLISHED. Your best bet to get a reply is to follow the link at the top of the post to the ORIGINAL post! BUT you're more than welcome to start discussions here:

This site uses Akismet to reduce spam. Learn how your comment data is processed.