Indico Data releases industry-first large language model benchmark for document understanding tasks
Learn More
  Everest Group IDP
             PEAK Matrix® 2022  
Indico Named as Major Contender and Star Performer in Everest Group's PEAK Matrix® for Intelligent Document Processing (IDP)
Access the Report


Given a scenario, how can I identify which machine learning algorithm is best suited for this scenario?

May 26, 2018 | Ask Slater

Back to Blog

 You try them and see how well they work. Anything other than experimentation is guesswork.

You can make very broad assertions like, “convolutional neural networks work well for image recognition”, but that doesn’t really help you determine which algorithm you should use, and even in this situation there’s no possible way to know ahead of time how well a given CNN implementation will work in your particular scenario.

However, in practice this isn’t really an issue. Why? Well it’s because you don’t need the best machine learning algorithm. You need one that’s good enough. So your goal is first to define what “good enough” is. What kind of accuracy do you need for this to be effective? After you do that you should do a literature review to find which classes of algorithms are currently state of the art (if there is literature to reference, which is usually a safe bet).

Once you’ve tried the approach advocated by the existing literature there are three outcomes:

  1. You’re nowhere close to the accuracy you need. In this case your problem is likely currently intractable or you looked at the wrong reference literature. Sometimes when you see this you’ve done something wrong in your problem framing and you need to approach your problem differently.
  2. You’re close to the accuracy that you need, but you’re not there yet. In this case you should actually experiment with other algorithms, architecture variants, etc… in order to get the additional accuracy points that you need.
  3. You’re above the accuracy that you need. In this case just stop. You’re already good enough and additional time produces diminishing returns.

In very rare cases you’re working in a scenario where every additional accuracy point adds significant value. In this situation, the traditional approach is to continuously try new techniques, pay attention to new literature, and spend years tweaking and tuning the algorithm to squeeze out every additional point of accuracy you can.

View original question on Quora >

Follow Slater on Quora >>

Increase intake capacity. Drive top line revenue growth.


Get started with Indico

1-1 Demo



Gain insights from experts in automation, data, machine learning, and digital transformation.

Unstructured Unlocked

Enterprise leaders discuss how to unlock value from unstructured data.

YouTube Channel

Check out our YouTube channel to see clips from our podcast and more.
Subscribe to our blog

Get our best content on intelligent automation sent to your inbox weekly!