Indico Data receives top position in Everest Group's Intelligent Document Processing (IDP) Insurance PEAK Matrix® 2024
Read More
  Everest Group IDP
             PEAK Matrix® 2022  
Indico Named as Major Contender and Star Performer in Everest Group's PEAK Matrix® for Intelligent Document Processing (IDP)
Access the Report


Is sentiment analysis in Twitter old or active research?

May 25, 2018 | Ask Slater

Back to Blog

 There was a particular surge in Twitter sentiment analysis in the early days of Twitter (pre-2015). It was a somewhat narrow window where this kind of research was extremely exciting because all of the traditional approaches to sentiment analysis relied on more formal corpora and these approaches broke down dramatically when applied to Twitter (as in: our predictions were no longer any better than flipping a coin).

However, things have changed. One of the biggest things that have changed is our approach to sentiment analysis. It has become clear that sentiment analysis is a less consistent and more tractable problem than it used to be with the advent of deep learning techniques. These techniques are very capable of recognizing sentiment with comparable accuracy to a human, meaning that the strict sentiment analysis benchmark of old is less cutting edge and exciting than it once was.

That’s not to say that this is a solved problem, only that the traditional formulation of sentiment analysis (predict a 1 or 0 for positive or negative) is no longer particularly interesting because we’ve collectively gotten too good at the task. When we’re at accuracies close to human accuracy, we start looking for other problems because we can no longer objectively advance.

That’s why when we look at modern problems we both focus on more nuanced understanding and more nuanced tasks. We’re either focusing on things like more nuanced emotional analysis, or more commonly more sophisticated parsing of sentiment involving extracting structured sentiment information from text.

The short is that there is a lot of very interesting research happening in the space of natural language processing, but epsilon research on improving Twitter sentiment analysis is not particularly interesting compared to the problems that we’re trying to solve today. It will be hard to orient a thesis or dissertation around Twitter sentiment analysis unless you’ve got a very interesting new view on it, but generally speaking you have to pick a harder problem to tackle if you want to show serious improvement over state of the art methods.

View original question on Quora >

Follow Slater on Quora >>

Increase intake capacity. Drive top line revenue growth.


Unstructured Unlocked podcast

April 10, 2024 | E44

Unstructured Unlocked episode 44 with Tom Wilde, Indico Data CEO, and Robin Merttens, Executive Chairman of InsTech

podcast episode artwork
March 27, 2024 | E43

Unstructured Unlocked episode 43 with Sunil Rao, Chief Executive Officer at Tribble

podcast episode artwork
March 13, 2024 | E42

Unstructured Unlocked episode 42 with Arthur Borden, VP of Digital Business Systems & Architecture for Everest and Alex Taylor, Global Head of Emerging Technology for QBE Ventures

podcast episode artwork

Get started with Indico

1-1 Demo



Gain insights from experts in automation, data, machine learning, and digital transformation.

Unstructured Unlocked

Enterprise leaders discuss how to unlock value from unstructured data.

YouTube Channel

Check out our YouTube channel to see clips from our podcast and more.
Subscribe to our blog

Get our best content on intelligent automation sent to your inbox weekly!