Introducing the Flowlets preview for ADF and Synapse

This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .

Next week, we will roll out the public preview for "Flowlets", a new feature of Mapping Data Flows found in Azure Data Factory and Azure Synapse Analytics. Flowlets will enable data engineers to design portions of new data flow logic, or to extract portions of an existing data flow, and save them as separate "Flowlet" artifacts inside your factory or workspace. Flowlets can then be re-used inside other data flows. 

 

The design experience to add new sources to your data flows will now allow for optional Flowlets as a source. Clicking on the Add Source directly inside the new source tile will provide the same 1-click action as today for adding a dataset source.

 

flowlet7.png

Flowlets can also be added inline in the data flow as a custom transformation.

 

flowlet3.png

 

Your saved Flowlets will appear in the Data Flows category in the ADF and Synapse object explorers. You will see an option for both Data Flow and Flowlet now from the new Data Flow menu.

 

flowlet1.png

MarkKromer_0-1638948410134.png

To create a new Flowlet from an existing data flow, you can shift-click or turn on the selector button then highlight the area of your graph which you choose to turn into a flowlet.

 

flowlets-create.png

 

The design surface for Flowlets is the same as the Data Flow canvas with a few exceptions. There are 2 new transformations: Input and Output. This is how you will define the contract or the argument for your Flowlet as inputs and return values. While Output replaces Sink from Data Flows, the Input also include the option to point to an inline dataset which you can use as a Source Flowlet in your data flows.

 

flowlet2.png

 

flowlet6.png

 

We'll post instructional videos with samples next week, so stay tuned for more!

REMEMBER: these articles are REPUBLISHED. Your best bet to get a reply is to follow the link at the top of the post to the ORIGINAL post! BUT you're more than welcome to start discussions here:

This site uses Akismet to reduce spam. Learn how your comment data is processed.