Indico Data releases industry-first large language model benchmark for document understanding tasks
Learn More
  Everest Group IDP
             PEAK Matrix® 2022  
Indico Named as Major Contender and Star Performer in Everest Group's PEAK Matrix® for Intelligent Document Processing (IDP)
Access the Report


Making Deep Learning Practical with Smaller Datasets

October 25, 2017 | Business, Machine Learning

Back to Blog

At the recent VB Summit in Berkeley, Jeff Dean, Head of Google Brain discussed a popular challenge in making Deep Learning a practical solution inside the enterprise: “I would say pretty much any business that has tens or hundreds of thousands of customer interactions has enough scale to start thinking about using these sorts of things”. “If you only have 10 examples of something, it’s going to be hard to make deep learning work. If you have 100,000 things you care about, records or whatever, that’s the kind of scale where you should really start thinking about these kinds of techniques.”
Fact: Training a deep learning model from scratch requires large datasets of at least tens to hundreds of thousands of examples.This is a large barrier to entry for integrating deep learning into a business’ workflow, either due to inability to access such a large amount of data, or, if using supervised learning techniques, the monetary and temporal cost of labeling such a huge dataset.

What is Transfer Learning?

What if we didn’t have to start from scratch? What if instead of starting from zero every time you wanted to create a deep learning model, you could instead start with a model that already understood the basics of language? That’s the promise of an area of machine learning known as transfer learning. By looking at a massive corpus of language up front (typically hundreds of millions of labeled examples), transfer learning can create a basic understanding of language. Using this starting point, enterprises can use deep learning to their advantage even if their training datasets are orders of magnitude smaller than what is typically required.

At a low level, transfer learning “teaches” the model basic concepts of language. Things like synonyms, grammar, and basic syntactic structures. The model still needs help to understand specific documents: be it news reports, legal documents, or social media; but because the model already understands the basic structure of language, it can learn specific concepts much more quickly. You’re not going to get a lawyer model off the shelf, but you can at least get a model that speaks english.

Applying Transfer Learning

If you’re working with unstructured text and image data, Indico’s Custom Collection API enables you to build custom models with just a few hundred (or at most, a few thousand) labeled examples by taking advantage of our high quality feature embeddings. For ideas on how to use Custom Collection for the task you’re trying to solve, explore some of our use case tutorials:

And before you ask…yes, we needed hundreds of millions of examples to build our transfer learning model that enables indico customers to make use of Deep Learning with only a few dozen examples for their use case!


Increase intake capacity. Drive top line revenue growth.


Get started with Indico

1-1 Demo



Gain insights from experts in automation, data, machine learning, and digital transformation.

Unstructured Unlocked

Enterprise leaders discuss how to unlock value from unstructured data.

YouTube Channel

Check out our YouTube channel to see clips from our podcast and more.
Subscribe to our blog

Get our best content on intelligent automation sent to your inbox weekly!