Intelligent Document Processing (IDP) software technology enables organizations to automate the processing of unstructured data – documents, forms, text, images, video and more. Learn how to apply IDP artificial intelligence to unstructured data workflows with context, accuracy, and nuance – without massive costs.
Among the list of intelligent document processing topics covered on this page:
Intuitive point and click interface for unstructured data classification, extraction, and workflows; no data scientists required.
Award-winning AI Explainability and an intuitive document validation user interface to deliver unmatched output accuracy.
Quickly build custom machine learning models with just 200 examples, tailored precisely to your document understanding challenges.
Intelligent document processing is gaining traction because many organizations have reached the limit of what they can do with robotic process automation (RPA) and are looking to take the next step in their automation journey.
While RPA and optical character recognition (OCR) templated approaches to document automation work well with highly structured data, where the expected data is in the same place every time, they cannot handle unstructured data – such as email, Word documents, images, PDFs and more. It’s an important distinction because in most organizations, at least 80% of their data is unstructured, making it difficult for first-generation automation solutions to deal with.
A sound unstructured data automation solution, however, enables organizations to automate processes that involve unstructured data, and without huge data sets that are normally required to accurately train automation models – and that are out of reach for the vast majority of enterprises.
Instead, the IDP platform is based on a massive, pre-built database of labeled data points. It then incorporates an artificial intelligence technology known as transfer learning, which enables a model trained on one task to be used for a related task. Transfer learning obviates the need for a model to be trained on thousands of documents in order to achieve accuracy.
Indico, for example, has a base model consisting of some 500 million labeled data points. But thanks to transfer learning, it takes only about 200 documents and a few hours to train a business document processing model with about 95% accuracy. That reduces the underlying data required by a factor of 100x to 1000x as compared to traditional approaches.
That dramatic reduction in data also means the Indico platform doesn’t require massive amounts of computing power, like many AI document processing solutions do. Rather, it can run effectively on just one or two GPUs, and scale from there using low-cost CPU.
It’s important to delve deep into how any given automated document processing solution for business works in order to understand its true capabilities. Today, many vendors are playing fast and loose with all kinds of AI-related terms, including IDP. You may come across RPA vendors who say they incorporate AI document processing technology to deliver what sounds like IDP. Under the covers, you will likely find the solution is merely a rules engine at heart that can’t handle unstructured data, especially at scale. In practice, “RPA + AI” solutions will likely solve only for structured and, at best, semi-structured use cases such as invoices.
A true IDP solution should be able to handle any type of data: structured, semi-structured and unstructured. Given the vast majority of data in your company is likely unstructured, only a solution that can effectively handle it will be able to accelerate your business document processing and deliver transformative change. Don’t settle for less.
Indico’s approach to intelligent document processing, known as Intelligent Process Automation, makes it simple and effective to build models that automate document-intensive processes normally performed by humans.
Most processes require humans to read documents, find appropriate data and enter it into a downstream system. With Indico, the business subject matter experts who understand the processes best build models to automate such processes.
Using Indico’s intuitive tools, process experts label the data points they want to extract from documents. As they do so, the model updates in real time to show predictions on how well the model will perform the task at hand. When the prediction hits an acceptable level, you’re done. Typically, it takes a few dozen to maybe 200 documents to properly train a model.
Having the people who understand the business process and the desired results actually build the models is a crucial component of the Indico approach. It turns these business people into “citizen data scientists” – even though they don’t need any data science expertise. It’s a far more rapid and accurate approach than having business people try to explain a process to a data scientist, who then goes off and builds a model. With Indico, there’s no risk of requirements being lost in translation.
All Indico tools are in plain English and are simple to use. Fully working models can be built in as little as an hour.
While Indico’s platform is simple to use, it’s built on some sophisticated cognitive AI technology that we keep behind the scenes.
One example is deep learning. Normally, users have to program a model such that the computer can understand what it needs to do. Deep learning turns that notion around and says, “Show me examples of what you want to achieve and I’ll figure out how to do it.”
Natural language processing (NLP) is likewise a crucial element. NLP enables the Indico platform to understand context in a given piece of data– whether structured or unstructured. It enables the model to “read” a document and understand it just as a human would. But it functions strictly behind the scenes; there’s no need for business users to understand what NLP is or how it works.
That goes for machine learning (ML) as well. The Indico platform is built on sophisticated ML models, but users never have to interact with them. They simply focus on delivering business benefits by building models that remove repetition and complexity from document-intensive processes, while improving accuracy.
You may find other products that incorporate deep learning, NLP and ML to address processes that involve unstructured data, but you will likely find them to be far more complex to implement. Typically they do require data science expertise, along with millions of dollars to implement and maintain.
The mortgage underwriting process typically involves humans looking over lots of documents to assess an applicant’s creditworthiness. Applying intelligent document processing to mortgage underwriting automates the process, with the IDP platform instead “reading” the documents and extracting relevant data for input into the bank’s credit evaluation system.
Another common use case for commercial banking automation is meeting regulatory requirements around anti-money laundering (AML). In the U.S., that means complying with the Bank Secrecy Act and related regulations meant to deter money laundering by terrorist networks and drug cartels.
IDP can take the various documents required to onboard a new customer and automatically classify them, extract relevant data and input it into the bank’s digital management system. Customers are onboarded more quickly, with increased accuracy, resulting in faster time to revenue for the bank and improved customer satisfaction.
Closely related to AML requirements is “know your customer” regulations, and they present similar challenges. As part of the commercial banking client onboarding process, these laws require banks to make an effort to verify the identity of customers as well as the risks involved in any business relationship with them.
The LIBOR interest rate benchmark is due to be phased out at the end of 2021. Banks and financial institutions worldwide are left poring through documents looking for references to LIBOR in order to determine what their exposure is and take steps to address it – a task that screams for IDP.
Applying intelligent document processing to life insurance underwriting can help companies dramatically improve the process by largely taking humans out of the equation. With IDP, companies can create models to quickly categorize and extract data from reams of documents.
For insurance claims processing, intelligent document processing can be used to automate the classification and annotation of new claims, and route them to the appropriate subject matter expert for processing. It can also help extract pertinent information from documents, including unstructured data such as images and free-form notes from insurance adjusters.
Investment firms with wealth management divisions can take advantage of financial services automation by using it to analyze financial documents. Normally, humans read financial statements and pore over investment data, manually extracting relevant data. IDP enables financial firms to automate the process, pulling out relevant data and normalizing it for insertion into data processing tools. The result is a dramatic improvement in speed, efficiency and accuracy.
Investment firms often receive trade processing documents via email and in PDFs. They can use intelligent data processing tools to automate trade processing by extracting relevant unstructured data from these documents, and normalizing it for input into the firm’s digital management system, eliminating hours and hours of manual data processing.
Use cases like those above make it easy to see how IDP saves companies time and money. From our experience with customers, here’s the kind of gains you can expect from Indico:
Realize faster time to market for new initiatives and improve customer satisfaction
Create dramatic cost efficiencies for back-office functions and scale critical processes without increasing expenses
Free up employees from tedious, low-value tasks and repurpose them for higher value, more strategic projects
With no data science expertise required, turn your business process experts into citizen data scientists
Build effective models with a fraction of the data traditional artificial intelligence solutions require
Automate your most complex document-based workflows
Cushman & Wakefield has chosen Indico for intelligent document process automation based on four differentiators: the user experience and intuitive interface; process experts can own and modify models as needed without assistance from IT or data scientists; the analytics capabilities identify relevant terms in a document (even if they don’t appear in the same place); the Indico platform applies to numerous use cases and document types to expand across business units.
You can view the full case study here.
“The Indico Platform lets our senior product and business leaders get directly involved in building models and driving automation at Chatham. With Indico’s expertise and just a few hundred documents, we’ve successfully delivered cutting edge models in a way previously considered impossible.”
Andrew Thornfeldt, Chatham Financial
You can view the full customer story here.
Increase efficiency, reduce costs, transform the enterprise.
Perhaps figures like 85% reduction in cycle times and 4x increase in process capacity sound almost too good to be true. We can assure you they are most certainly real and add up to a rapid return-on-investment (ROI).
Consider a document-intensive process that involves 10 employees who each earn $100,000 per year, or $1 million total. Let’s say the team performs 500,000 tasks per year for a given process; that comes to $2 per task. If an IDP solution can automate 75% of those tasks – a perfectly reachable goal – the cost per task falls to just 50 cents, so the company saves $750,000 per year.
Looking at it another way, you now have $750,000-worth of employee hours to dedicate to other areas – a dramatic increase in overall capacity.
At the same time, by freeing up employees from tedious tasks, you gain soft benefits including increased employee satisfaction and productivity. Meanwhile, the IDP solution will perform the newly automated tasks with increased accuracy and consistency – because computers don’t get tired or make typos.
Change is a constant that’s all around us – and it can doom document process automation projects.
Whether it’s changes in people, process, or technology, companies need to have a way to stay on top of changes and think through the affect any change will have on automated business processes. If you fail to recognize the importance of change management, your process automation project will fail. “This is just a fact,” says Vishesh Bhatia, a process automation expert with Cognizant.
Without a proper change management process in place, a small change may result in a process failure or, perhaps worse, results that are not quite accurate.
Picture a process that goes through a loop, processing hundreds of transactions. For each transaction, the automated system performs the same steps, such as picking up data from system A and feeding it to system B.
Now imagine the business makes a subtle change upstream but fails to inform those responsible for the process automation model. As a result, data that’s supposed to be in system A is no longer there. The process may still run, but without data from system A, it will continually produce a series of errors. It may take a few process cycles before anyone notices the data in system B is incomplete and erroneous.
“With a human workforce, organizations can paper over the cracks because humans can adapt and react to change with a degree of flexibility that bots don’t have,” Bhatia says. “The bot continues to perform the old steps. It will either terminate or perform an undesired activity that might have a negative impact, kind of like a train going off the track.”
It’s a similar story with changes to technology. When an application that’s involved in an automated process is updated or some login dimensions change, that has to be communicated to business process owners to assess the effect on the automated process.
How that change management process is implemented may well be a function of the organization’s automation CoE, according to Nidal Nasr, who worked at AIG for more than 20 years. It’s critical to have a quality assurance review process, where you look at a certain percentage of results from an automated process to ensure accuracy.
“Some failures will be obvious: no results come out. Others are more subtle, where results do come out and are almost right, but there’s something a little off,” Nasr says. “Before that’s detected, company could be running on bad information and bad data.”
Companies also need to consider changes in the regulatory environment, Nasr notes. California has recently implemented stringent data privacy laws, similar to what Europe has with the General Data Protection Regulation (GDPR). Given the focus of document process automation is to extract data from documents, such laws need to be taken into account.
Ensuring proper change management starts with outlining the business objectives you want the automated document process to achieve, Cognizant’s Bhatia says. Then you can define the key performance indicators (KPIs) and the metrics you will measure to determine whether you’re meeting your objectives. Such measurements will allow you to catch issues that may indicate a change management problem.
Another key is properly defining ownership of the automated process. It’s a mistake to simply hand ownership to IT, Bhatia says, because they don’t understand the process as well as the business side does. As a result, IT will always be reactive in responding to change.
If the business owns and runs the automation routines, they are more likely to know about changes to templates or documents involved in the process and alert IT about any changes that need to be made to automation models.
That really gets to the concept of citizen data scientists, where the business people who perform the process are the ones who actually use the intelligent document processing tools involved in automating processes.
That’s the philosophy behind Indico’s Intelligent Document Processing platform. Using our simple tools, it often takes just few hours to label 200 documents and get a working model up and running. Should any changes in the process occur down the line, those same business people can adjust the model accordingly.