Unlocking the Power of Open AI – Azure DevOps Backlogs from Images/PDFs

This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .

 

VinodSoni_0-1677708081290.png

 

 In today's digital world, the need to convert images and PDFs to text is becoming increasingly important. However, the process of manually transcribing images and PDFs can be time-consuming and error-prone. Fortunately, there is a better way. With the Azure Open AI service, you can easily and quickly convert images and PDFs to text.

 

The Azure Open AI service is a cloud-based platform that provides powerful machine learning and artificial intelligence (AI) capabilities. This platform makes it easy to quickly convert images and PDFs to text using pre-trained models. All you need to do is upload your image or PDF file and the AI service will automatically extract the text from it.

 

Recently I have faced almost similar business use case where customer wants to create Azure DevOps backlogs (e.g. user story, task) from PDF collection they receive from their customers regularly. Sometime they receive input in image form as well. It was very challenging and time-consuming process to create backlogs manually. They were looking some kind of automation process which can quickly convert images and PDFs to text and create backlogs into system.

Sample Business Requirement in image/PDF

 

VinodSoni_1-1677708081297.png

 

I was thinking about Logic Apps or power automate is a services that helps you create automated workflows between your favorite apps and services to synchronize files, get notifications, collect data, and more and integrate AI capabilities to achieve my goal.

 

I did some research and found that power automate comes with prebuilt AI models and can be used for this business case. Text recognition is one of the model I used to implement the following solution. The text recognition prebuilt model extracts words from documents and images into machine-readable character streams. It uses state-of-the-art optical character recognition (OCR) to detect printed and handwritten text in images.

 

I followed below steps to implement the solution –

 

Step 1: Sign in to Power Automate.

 

Step 2: Select Create in the left pane, and then select New flow > Automate Cloud Flow.

 

VinodSoni_2-1677708081300.png

 

Step 3: Name your flow, select Manually trigger a flow under Choose how to trigger this flow, and then select Create.

 

Step 4: Expand Manually trigger a flow, and then select +Add an input > File as the input type.

 

VinodSoni_3-1677708081301.png

 

Step 5: Select +New step > AI Builder, and then select Recognize text in an image or a PDF document in the list of actions.

 

Step 6: Select the Image input, and then select File Content from the Dynamic content list

 

VinodSoni_4-1677708081301.png

 

Step 7: To process results, select +New step > Control, and then select Apply to each.

 

Step 8: Select the input, and then select lines from the Dynamic content list. This will add results to the input and automatically create another Apply to each action as shown in below screenshot.

 

VinodSoni_5-1677708081303.png

 

Step 9: In the successive actions, you can use any columns extracted by the AI Builder model. For example, you can extract Text into a variable, and then use Azure DevOps action called “Create Work Item” and assign Text to description.

 

VinodSoni_6-1677708081304.png

 

Step 10: Congratulations! You've created a flow that uses a text recognition model. Now you can Select  Save on the top right, and then select Test to try out your flow. You final workflow should be look like this.

 

VinodSoni_7-1677708081306.png

 

At the end, you can see user story successfully created in my ADO backlogs from image/PDF.

 

VinodSoni_8-1677708081309.png

 

 

Note: I am using manual trigger to upload image or PDF but you can find filesystem related trigger or actions to kick off the workflow e.g. if image and PDF uploaded to some cloud location e.g. blob storage or disk drive etc. A PDF or image may contains more than one backlog description and divided into multiple paragraphs, in this case we may need to extend our workflow and perform some additional text manipulation.

 

 Feel free to share feedback and suggestions to improve the solution. 

 

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.