Optimizing Data Extraction Accuracy with Custom Models in Azure AI Document Intelligence

This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .



Many users of Azure AI Document Intelligence experience challenges with the accuracy of data extraction. This article addresses a key reason for these challenges: the reliance on a singular model approach.


As businesses turn to AI for extracting data from documents, Azure AI Document Intelligence stands out as a preferred solution. It boasts an impressive mix of user-friendly features and sophisticated AI capabilities, tailored to accommodate a variety of data processing demands. However, the platform's versatility sometimes leads users to a common misunderstanding—that a single, all-encompassing model can meet all their needs.


In this article, we highlight the limitations of this approach and suggest an alternative within Azure AI Document Intelligence. Specifically, we'll advocate the advantages of developing custom models and the significance of tailoring these models to the specific types of documents being processed. We will also cover the role of classification models in improving the precision of data extraction.


The Singular Model Problem


Although it is possible in Azure AI Document Intelligence to create a single extraction model based on all of your varying document types, this poses a major challenge in producing accurate results. For example, extracting valuable information from contracts that vary in format, structure, content, and legal jargon would be equivalent to fitting a square peg into a round hole. Here is a potential conceptual diagram to illustrate the point.
The challenges that we frequently see include:
  • Multiple Document Structures: Usually there are a multitude of document structures. For example, invoices are not the same as receipts, and contracts are not the same as resumes. When trying to use the same model to extract information from different document structures, a singular model struggles to handle this diversity resulting in lower accuracy.
  • Single document with multiple structures: In many cases, a single document can have multiple pages, each with a different structure. Some of these pages for one document may span multiple pages, while for another, they only span one. Treating each section of the document as one can also result in lower accuracy.

The challenges posed by the singular model problem are significant. When dealing with diverse document types, such as contracts, invoices, and resumes, a one-size-fits-all approach struggles to maintain accuracy. In the next section, we’ll explore a practical design to address these issues effectively.


A practical design for improving data extraction accuracy in Azure AI Document Intelligence


In this section, we outline an approach to enhance document processing using Azure AI Document Intelligence. Highlighted by the diagram below, our method employs a system of multiple models, each designed for a specific type of document structure. This section details the steps and key factors to consider, helping users improve their accuracy in data extraction. Consider it as a practical design for document processing using Azure AI Document Intelligence using the following diagram.


Our recommended design involves a shift towards a more composed approach, utilizing multiple custom extraction models and a classification model to enhance the accuracy and efficiency of document processing. Based on this, consider the following:


  • Use multiple extraction models for differing document structures: Train and deploy separate custom extraction models, each specifically designed for a particular type of document structure. This ensures that each model is highly optimized for its intended document type, improving accuracy in data extraction.

  • Create a classification model to determine extraction model route: Implement a custom classification model to serve as the initial layer of processing. This model categorizes documents based on their structure and content, ensuring that each document is routed to the most appropriate model for further processing.

  • Perform pre-processing techniques to optimize accuracy for multi-page documents: For documents consisting of multiple pages with diverse structures, divide them into smaller segments, each representing a distinct document type. Process each segment with the model best suited to its structure. This approach is particularly effective for composite documents that act more like collections of various document types rather than a single, uniform entity.




It is important to recognize that each of your document types require their own techniques for accurate data extraction. Azure AI Document Intelligence is a powerful AI tool to provide this capability, but it our experiences, alongside various ISVs and Digital Natives, revealed a critical insight: the pursuit of a singular, catch-all model falls short of expectations.


The key takeaway is to move beyond the one-model myth and embrace custom models tailored to the unique structures of your documents. By applying a practice of aligning document intelligence capabilities with the specific needs of your documents, this change can significantly enhance the accuracy of your data extraction with Azure AI Document Intelligence. 


Further Reading

Leave a Reply

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.