Everest Group IDP
             PEAK Matrix® 2022  
Indico Named as Major Contender and Star Performer in Everest Group's PEAK Matrix® for Intelligent Document Processing (IDP)
Access the Report

BLOG

Everything You Wanted to Know About Optimization

May 17, 2018 | Data Science, Machine Learning

Back to Blog

Last week our co-founder and machine learning architect, Madison May, presented at the Boston Machine Learning Meetup on various optimization methods based on trends from ICLR 2018. Video and slides below.
 

 

Overview

In recent years the use of adaptive momentum methods like Adam and RMSProp has become popular in reducing the sensitivity of machine learning models to optimization hyperparameters and increasing the rate of convergence for complex models. However, past research has shown when properly tuned, using simple SGD + momentum produces better generalization properties and better validation losses at the later stages of training. In a wave of papers submitted in early 2018, researchers have suggested justifications for this unexpected behavior and proposed practical solutions to the problem. This talk will first provide a primer on optimization for machine learning, then summarize the results of these papers and propose practical approaches to applying these findings.

Suggested Posts

Exploiting Text Embeddings for Industry Contexts

Create a Customized RSS Feed with indico Text Tags, Python, & HTML

Deep Advances in Generative Modeling

Automate your most complex unstructured document workflows

Get started with Indico

Interactive demo

Transform your own unstructured documents with our OOTB models

Live Demo

Explore firsthand the value the Indico Platform delivers

Talk with us

Discuss how the Indico Platform can help you tackle your unstructured data problems

Resources

Blog

Gain insights from experts in automation, data, machine learning, and digital transformation.

Unstructured Data Explained

Answers to the most complex questions in unstructured data.

CTO Corner

An accumulation of content straight from our co-founder and CTO.

Unstructured Unlocked

Enterprise leaders discuss how to unlock value from unstructured data.
Subscribe to our blog

Get our best content on intelligent automation sent to your inbox weekly!