Pipeline Logic 2: OR (at least 1 activity succeeded or failed)

This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .

Series Overview

Orchestration allows conditional logic and enables users to take different paths based upon outcomes of a previous activity. Building upon the concepts of conditional paths, ADF and Synapse Pipelines allows users to build versatile and resilient work flows that can handle unexpected errors that work smoothly in auto-pilot mode.

 

This is an ongoing series that gradually levels up and helps you build even more complicated logic to handle more scenarios. We will walk through examples for some common use cases, and help you to build functional and useful work flows.

 

Please review the first installment in the series: Pipeline Logic 1: Error Handling and Best Effort Step.

 

Or

In more advanced use cases, it is sometimes required to express the notion of OR: if any of the previous activity succeeded/failed, we should invoke a follow up activity. There are many variations of the requirement, and we will cover some common patterns in this category.

 

We will achieve this by combining conditional paths and the conditional if activity. With this implementation, an evaluation will be made after all dependency activities have completed (either succeeded or failed) and the pipeline will decide if the follow up activity should be invoked or not.

#1 Shared error handling logging step

It is often the case that you would have a shared error handling/logging step in your workflow. This will be invoked if any of previous activities failed. You can build your pipeline like this:

  1. Run multiple activities in parallel
  2. Add an If condition to contain the error handling steps, in a True branch
  3. Connect activities to the condition activity using an Upon Completion path
  4. Logical expression for condition activity reads
    (equals(activity('ActivityFailed').Status, 'Failed'), equals(activity('ActivitySucceeded').Status, 'Failed'))
    Note: You'll need concatenated or if you have more than two dependency activities, for instance:
    (or(equals(activity('ActivityFailed').Status, 'Failed'), equals(activity('ActivitySucceeded1').Status, 'Failed')),equals(activity('ActivitySucceeded1').Status, 'Failed'))

conditional-or-1.png

 

#2 Greenlight if any activity succeeded

When all your activities are best effort, you may want to proceed to the next step if any of the previous activities succeeded. You can build your pipeline like this:
  1. Run multiple activities in parallel
  2. Add an If condition to contain the error handling steps, in a True branch
  3. Connect activities to the condition activity using Upon Completion path
  4. Logical expression for condition activity reads:
    (equals(activity('ActivityFailed').Status, 'Succeeded'), equals(activity('ActivitySucceeded').Status, 'Succeeded'))
    Note: The graph will look exactly like the previous scenario. The only difference is the expression language used.

conditional-or-2.png

 

#3 All activities need to succeed to proceed

This pattern combines the best of the two: conditional and + error handling. The pipeline proceeds to next steps if all proceeding activities succeed, or else it runs a shared error logging step. You can build your pipeline like this:

  1. Run multiple activities in parallel
  2. Add an If condition. Add next steps in a True branch, and add error handling code in a False branch
  3. Connect activities to the condition activity using Upon Completion path
  4. Logical expression for condition activity reads:
    @and(equals(activity('ActivityFailed').Status, 'Succeeded'), equals(activity('ActivitySucceeded').Status, 'Succeeded'))

     

 

 

 

conditional-complex-1.png

 

 

We hope that you have found this blog to be helpful! If you have any questions or feedback, please post them in the comments below. 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.