An issue we’ve seen crop up repeatedly in intelligent document processing projects is data science teams and line of business folks don’t seem to speak the same language with respect to what they expect out of an automation project. Too often, the discussion focuses on “accuracy,” but the term doesn’t mean the same thing to both sides.
This was the topic of a recent episode of our “Unstructured Explained” video series, featuring Indico Data Founder CTO and co-founder Slater Victoroff, VP of Business Development Brandi Corbello, and VP of Research & Development Chris Wells, PhD. They all agreed companies are focusing on the wrong measurements and, in some cases, holding unrealistic expectations.
Too often the business side of the house doesn’t really grasp what the target of an automation project should be, in terms of how to define success. Maybe the data science team comes up with a set of test data and the business basically says, “If you can accurately process that, we’ll be good,” Wells said.
But doing well with test data should not be the target. The only real measurement that matters at the end of the day is how well the intelligent automation model works in a production environment, with real data, he said.
Related content: How intelligent document processing helps address the ambiguity in automation efforts
Setting expectations around accuracy
Another issue is the focus on “accuracy.” “It’s really problematic because the business often comes with totally unrealistic expectations,” Victoroff said. There may be a demand for 90% accuracy, or heads will roll.
“And you [as a data scientist] have a very straightforward way of getting to 90%,” even though you know that doesn’t reflect the reality of the production environment. “What are you going to do? Are you going to keep your job by doing the very, very easy thing? And then in a year maybe people will realize it didn’t quite work out.”
Another issue with such a laser focus on accuracy is that it’s out of line with what we expect of our employees who are currently doing the same job we’re trying to automate.
“How do you know the people who have been performing this task for the last 5 years have been 99% accurate?” Corbello asked. “You’re setting a high expectation that you actually don’t currently even [meet].”
She raises a good point, Wells noted. Today few companies have controls or processes in place to even measure how well their employees are currently doing performing a given process; they just assume everyone is doing a great job.
Related content: How to measure results of intelligent document processing projects
Shoot for improvement, not perfection
The reality is no process is 100% perfect. So, a better measurement of the value of an automation project is improvement, meaning how much you can reduce the time it takes to perform a process. Even if you get a 40% or 50% improvement, that frees up significant resources to dedicate to other, likely more rewarding and strategic endeavors.
The Indico Unstructured Data Platform offers a viable way to automate processes involving even complex, unstructured documents and data. And we can help you set realistic goals around what kind of results to expect.
To learn more subscribe to our LinkedIn newsletter.
