Gartner intelligent document processing guide gets the market (mostly) right

I recently had cause to revisit the guide to Intelligent Document Processing Gartner published in early 2022 in which Indico Data was featured and was struck by how well it holds up in its description of the market and the various vendors that play in it. All-in-all, it’s a good handbook for companies trying to decide among intelligent document processing vendors.

The Gartner IDP guide’s “key findings” section nicely lays out some of the most important considerations, each of which I’ll examine in more detail:

• Whether an intelligent document processing vendor can handle structured, semi-structured and unstructured data
• Point solutions vs. platforms
• Commoditized technologies that are lowering the barrier to entry

Related content: Data science and business groups need to speak the same language on intelligent automation

For IDP, intelligence is required

Gartner correctly points out that use cases for intelligent document processing include structured, semi-structured and unstructured data. “However, the majority of vendors are able to efficiently support only the use cases that deal with structured and semi-structured data,” the report says.

Gartner describes the “core,” “differentiating” and “emerging” capabilities of IDP solutions. The first core item is: “Ingest any type of structured and/or semi-structured documents,” while the first differentiating item is, “Process unstructured content, including handwritten content and signatures.”

This is one area where I would take exception to the Gartner IDP analysis. I don’t see how you can claim to play in the intelligent document processing market if you can’t process unstructured data.

Robotic process automation (RPA) and templated tools have for years helped companies process structured and (to a lesser extent) semi-structured data. The “intelligent” in IDP to me connotes a higher level of capability, specifically the use of artificial intelligence technologies to support the processing of unstructured data. So, I would consider the ability to process unstructured data a core capability of any IDP product; without that, sorry, but whatever you’re doing is not intelligent document processing.

Related content: Johnson Controls unlocks its unstructured data with intelligent document processing

IDP point solutions vs. platforms

Another key Gartner finding reads: “Organizations often approach their IDP use cases as point solutions within specific departments. Increased awareness of vendor offerings and consolidation of product features will lead to a greater demand for platforms that can serve multiple use cases.”

This is one of the findings that is proving true in the market. As we’ve covered previously, if you go the point solution route you run the danger of winding up with numerous disparate tools. Each one costs you money up front, but perhaps more importantly in the long run, they each require some level of expertise to configure, support and operate.

A better approach is to employ a single intelligent document processing platform that can handle multiple use cases. Your IT folks only have to learn a single tool and can apply it to any use case you may have, so you can far more quickly scale your automation program.

A bonus with the Indico Unstructured Data Platform is it’s intended to be used by the employees who actually perform the processes you’re trying to automate. With a modicum of training, they can build and operate their own models, with no help from IT or data scientists, so you can scale even more quickly.

Lower barrier to entry

Gartner’s other key finding correctly points out the intelligent document processing market has lots of players. “New and commoditized natural language technology (NLT) techniques have lowered the barrier of entry. However, domain knowledge (such as industry-specific document training sets) and emerging technology capabilities differentiate vendor solutions,” the guide says.

On this point, I agree the ready availability of (often open-source) natural language processing (NLP) technology does lower the barrier for entry. Indico Data, for example, has its own open source repositories, including Finetune, a library of pretrained NLP models for various tasks.

Domain knowledge and emerging technology capabilities are likewise important. I’d argue the entire IDP space is emerging, at least to the extent it’s defined by companies that can actually process unstructured data. Doing that requires not just NLP, but AI technologies including deep learning and transfer learning.

How IDP promotes data analytics

One final takeaway from the Gartner Intelligent Document Processing market guide has to do with one of its “market direction” drivers. Gartner points out that companies must be able to realize insights on “concepts, objects and relationships in language” and to use these insights for “non-IDP use cases.”

One example of what I think Gartner is getting at here is data analytics. Some 85% or more of all data in any given company is unstructured. And unstructured data generally can’t be fed to most business intelligence and data analytics engines for analysis. That means only a small fraction of the data in any given company is available for analysis.

Intelligent document processing can change that equation. Its ability to “read” unstructured data and convert it to a structured format means you can now glean actionable intelligence from all of your corporate data, not just the 15% that’s already structured.

Platforms are best for intelligent document processing

That’s my assessment of the key findings from the Gartner Market Guide for Intelligent Document Processing Solutions. While I can (and do) quibble with how Gartner defines the IDP market, in general they paint an accurate picture of what’s happening in the market.

From where we sit, it’s simple: companies need a solution to deal with their unstructured data pain and are finding point solutions are too expensive over time. The better investment is a platform approach that can take on the use cases you have today as well as whatever may come your way tomorrow.

But I invite you to check out the guide for yourself and draw your own conclusions.

Subscribe to our LinkedIn newsletter.