Papperoni

2011

Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions

Richard Socher, Jeffrey Pennington, Eric H. Huang, Andrew Y. Ng, Christopher D. Manning

Open PDF Google Scholar

citations

Cite Score

51

AI summary

This paper introduces a novel machine learning framework based on semi-supervised recursive autoencoders (RAE) that learn vector space representations for multi-word phrases, outperforming state-of-the-art on common sentiment datasets and a new experience project (EP) dataset for predicting multidimensional sentiment distributions without predefined sentiment lexica or polarity shifting rules.

Main Contributions

Introduced a novel machine learning framework based on recursive autoencoders (RAE) for sentence-level sentiment prediction.
The model learns vector space representations for multi-word phrases and exploits hierarchical structure and compositional semantics.
Evaluated on standard datasets (movie reviews, MPQA) and a new Experience Project (EP) dataset, achieving state-of-the-art performance.
The model predicts a multidimensional distribution over several complex, interconnected sentiments, rather than limiting to a positive/negative scale.
The system can be trained on both unlabeled domain data and supervised sentiment data, without requiring language-specific sentiment lexica or polarity shifting rules.

Abstract

We introduce a novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions. Our method learns vector space representations for multi-word phrases. In sentiment prediction tasks these representations outperform other state-of-the-art approaches on commonly used datasets, such as movie reviews, without using any pre-defined sentiment lexica or polarity shifting rules. We also evaluate the model's ability to predict sentiment distributions on a new dataset based on confessions from the experience project. The dataset consists of personal user stories annotated with multiple labels which, when aggregated, form a multinomial distribution that captures emotional reactions. Our algorithm can more accurately predict distributions over such labels compared to several competitive baselines.

Citation Graph

Loading graph...

References [37]

Sort:

Filter:

[1]A Neural Probabilistic Language Model

Yoshua Bengio, R. Ducharme, Pascal Vincent - 2001

62 papers in library cite

What started it all. Very simple and elegant.

[2]A Unified Architecture for Natural Language Processing: Deep Neural Networks With Multitask Learning

Ronan Collobert, Jason Weston - 2008

32 papers in library cite

Really did not add much to the game. I think this was more of a small perf. improvement over other existing things and set a few methodological standards. Maybe main contribution is Multitask Learning + Deep learning

[3]Word Representations: A Simple and General Method for Semi-Supervised Learning

J. Turian, L. Ratinov, Yoshua Bengio - 2010

17 papers in library cite

This basically introduced the concept of embeddings. Very nice.

[4]Parsing Natural Scenes and Natural Language With Recursive Neural Networks

Richard Socher, C. C. Lin, C. Manning, Andrew Y. Ng - 2011

10 papers in library cite

Good idea and nice results, but not my thing right now.

Reference title contains 'et al'

[5]Latent Dirichlet Allocation

D. M. Blei, Andrew Y. Ng, Michael I. Jordan - 2003

10 papers in library cite

30 pages; LDA

[6]Recursive Distributed Representations

J. B. Pollack - 1990

7 papers in library cite

[7]Seeing Stars: Exploiting Class Relationships for Sentiment Categorization With Respect to Rating Scales

Bo Pang, L. Lee - 2005

13 papers in library cite

[8]A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts

Bo Pang, L. A. Lee, L. Lillian - 2004

8 papers in library cite

[9]Annotating Expressions of Opinions and Emotions in Language

J. Wiebe, T. Wilson, T. Theresa, C. A. Cardie, C. Claire - 2005

7 papers in library cite

[10]Distributed Representations, Simple Recurrent Networks, and Grammatical Structure

Jeffrey L. Elman - 1991

5 papers in library cite

[11]Multiple Aspect Ranking Using the Good Grief algorithm

B. Snyder, R. Barzilay - 2007

5 papers in library cite

[12]Learning Accurate, Compact, and Interpretable Tree Annotation

Slav Petrov, L. Barrett, R. Thibaux, Dan Klein - 2006

4 papers in library cite

[13]Learning Continuous Phrase Representations and Syntactic Parsing With Recursive Neural Networks

Richard Socher, Christopher D. Manning, Andrew Y. Ng - 2010

4 papers in library cite

[14]Learning Task-Dependent Distributed Representations by Backpropagation Through Structure

C. Goller, A. Kuchler - 1996

4 papers in library cite

[15]Opinion Mining and Sentiment Analysis

Bo Pang, L. Lee - 2008

4 papers in library cite

[16]Thumbs Up? Sentiment Classification Using Machine Learning Techniques

Bo Pang, L. Lee, S. Vaithyanathan - 2002

4 papers in library cite

[17]Dependency Tree-Based Sentiment Classification Using CRFs With Hidden Variables

T. Nakagawa, K. Inui, S. Kurohashi - 2010

3 papers in library cite

[18]Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews

P. D. Turney - 2002

3 papers in library cite

[19]Contextual Valence Shifters

L. Polanyi, A. Zaenen - 2006

2 papers in library cite

[20]Linear Recursive Distributed Representations

T. Voegtlin, P. Dominey - 2005

2 papers in library cite

[21]Yahoo! For Amazon: Extracting Market Sentiment From Stock Message Boards

S. Das, Mark Chen - 2001

2 papers in library cite

[22]A Holistic Lexicon-Based Approach to Opinion Mining

X. Ding, Bing Liu, P. S. Yu - 2008

1 paper in library cites

[23]Coupling Niche Browsers and Affect Analysis for an Opinion Mining Application

G. Grefenstette, Y. Qu, J. G. Shanahan, D. A. Evans - 2004

1 paper in library cites

[24]Crystal: Analyzing Predictive Opinions on the Web

S. Kim, Eduard Hovy - 2007

1 paper in library cites

[25]Dynamic Auto-Encoders for Semantic Indexing

P. Mirowski, Marc'aurelio Ranzato, Yann Lecun - 2010

1 paper in library cites

[26]Exploring Sentiment Summarization

P. Beineke, T. Hastie, Christopher D. Manning, S. Vaithyanathan - 2004

1 paper in library cites

[27]Isotonic Conditional Random Fields and Local Sentiment Flow

Y. Mao, G. Lebanon - 2007

1 paper in library cites

[28]Learning to Shift the Polarity of Words for Sentiment Classification

D. Ikeda, H. Takamura, L. Ratinov, M. Okumura - 2008

1 paper in library cites

[29]Learning With Compositional Semantics as Structural Inference for Subsentential Sentiment Analysis

Yejin Choi, C. Cardie - 2008

1 paper in library cites

[30]Linguistic Inquiry and Word Count: Liwc2007 Operators Manual

J. W. Pennebaker, R. J. Booth, M. E. Francis - 2007

1 paper in library cites

[31]Mining the Peanut gallery: Opinion Extraction and Semantic Classification of Product Reviews

K. Dave, S. Lawrence, D. M. Pennock - 2003

1 paper in library cites

[32]On the Negativity of Negation

Christopher Potts - 2010

1 paper in library cites

[33]Pageranking Wordnet Synsets: An Application to Opinion Mining

A. Esuli, F. Sebastiani - 2007

1 paper in library cites

[34]Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis

T. Wilson, J. Wiebe, P. Hoffmann - 2005

1 paper in library cites

[35]The General Inquirer: A Computer Approach to Content Analysis

P. J. Stone - 1966

1 paper in library cites

[36]The Viability of Web-Derived Polarity Lexicons

L. Velikovich, S. B. Goldensohn, K. Hannan, R. Mcdonald - 2010

1 paper in library cites

[37]Towards Answering Opinion Questions: Separating Facts From Opinions and Identifying the Polarity of Opinion Sentences

H. Yu, V. Hatzivassiloglou - 2003

1 paper in library cites

Cited by

10

papers in your library

Cites

6

papers in your library

Read

on February 14, 2026

Very nice and very simple method. I like this waaay better than the other Stanford recursive autoencoder papers.

Tags

Paper Aliases

No aliases