Papperoni

2013

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Richard Socher, A. Perelygin, Jeffrey Wu, J. Chuang, C. Manning, A. Ng, Christopher Potts

Open PDF Google Scholar

citations

Cite Score

86

AI summary

This paper introduces a sentiment treebank and a Recursive Neural Tensor Network (RNTN) for sentiment compositionality analysis. The treebank contains fine-grained sentiment labels for phrases in movie review parse trees. The RNTN model achieves state-of-the-art results on sentiment classification and captures negation effects.

Main Contributions

Introduces a new Sentiment Treebank with fine-grained sentiment labels for phrases.
Proposes the Recursive Neural Tensor Network (RNTN) for sentiment compositionality.
Achieves state-of-the-art results on single sentence positive/negative classification (85.4%).
Reaches 80.7% accuracy in predicting fine-grained sentiment labels for all phrases.
Accurately captures the effects of negation and its scope at various tree levels.

Abstract

Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. Further progress towards understanding compositionality in tasks such as sentiment detection requires richer supervised training and evaluation resources and more powerful models of composition. To remedy this, we introduce a Sentiment Treebank. It includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality. To address them, we introduce the Recursive Neural Tensor Network. When trained on the new treebank, this model outperforms all previous methods on several metrics. It pushes the state of the art in single sentence positive/negative classification from 80% up to 85.4%. The accuracy of predicting fine-grained sentiment labels for all phrases reaches 80.7%, an improvement of 9.7% over bag of features baselines. Lastly, it is the only model that can accurately capture the effects of negation and its scope at various tree levels for both positive and negative phrases.

Citation Graph

Loading graph...

References [44]

Sort:

Filter:

[1]Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

John Duchi, Elad Hazan, Yoram Singer - 2011

19 papers in library cite

I actually skimmed through most of this. It's not a bad paper, but it's a math paper, not AI.

[2]A Neural Probabilistic Language Model

Yoshua Bengio, R. Ducharme, Pascal Vincent - 2001

62 papers in library cite

What started it all. Very simple and elegant.

[3]A Unified Architecture for Natural Language Processing: Deep Neural Networks With Multitask Learning

Ronan Collobert, Jason Weston - 2008

32 papers in library cite

Really did not add much to the game. I think this was more of a small perf. improvement over other existing things and set a few methodological standards. Maybe main contribution is Multitask Learning + Deep learning

[4]Parsing Natural Scenes and Natural Language With Recursive Neural Networks

Richard Socher, C. C. Lin, C. Manning, Andrew Y. Ng - 2011

10 papers in library cite

Good idea and nice results, but not my thing right now.

[5]Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions

Richard Socher, Jeffrey Pennington, Eric H. Huang, Andrew Y. Ng, Christopher D. Manning - 2011

10 papers in library cite

Very nice and very simple method. I like this waaay better than the other Stanford recursive autoencoder papers.

[6]Improving Word Representations via Global Context and Multiple Word Prototypes

Eric H. Huang, Richard Socher, C. Manning, Andrew Y. Ng - 2012

7 papers in library cite

Good paper. I would say not too relevant, but has some nice concepts like a "global context vector" (document embedding) that comes from averaging word embeddings, and a joint training objective.

[7]Recursive Distributed Representations

J. B. Pollack - 1990

7 papers in library cite

[8]Seeing Stars: Exploiting Class Relationships for Sentiment Categorization With Respect to Rating Scales

Bo Pang, L. Lee - 2005

13 papers in library cite

[9]Accurate Unlexicalized Parsing

Dan Klein, Christopher D. Manning - 2003

7 papers in library cite

[10]Semantic Compositionality Through Recursive Matrix-Vector Spaces

Richard Socher, B. Huval, Christopher D. Manning, Andrew Y. Ng - 2012

7 papers in library cite

[11]From Frequency to Meaning: Vector Space Models of Semantics

P. D. Turney, P. Pantel - 2010

6 papers in library cite

[12]Composition in Distributional Models of Semantics

J. Mitchell, Mirella Lapata - 2010

5 papers in library cite

[13]Multiple Aspect Ranking Using the Good Grief algorithm

B. Snyder, R. Barzilay - 2007

5 papers in library cite

[14]Learning Continuous Phrase Representations and Syntactic Parsing With Recursive Neural Networks

Richard Socher, Christopher D. Manning, Andrew Y. Ng - 2010

4 papers in library cite

[15]Learning Task-Dependent Distributed Representations by Backpropagation Through Structure

C. Goller, A. Kuchler - 1996

4 papers in library cite

[16]Opinion Mining and Sentiment Analysis

Bo Pang, L. Lee - 2008

4 papers in library cite

[17]A Structured Vector Space Model for Word Meaning in Context

K. Erk, S. Pado - 2008

3 papers in library cite

[18]Dependency Tree-Based Sentiment Classification Using CRFs With Hidden Variables

T. Nakagawa, K. Inui, S. Kurohashi - 2010

3 papers in library cite

[19]Estimating Linear Models for Compositional Distributional Semantics

F. Zanzotto, I. Korkontzelos, F. Fallucchi, S. Manandhar - 2010

3 papers in library cite

[20]Experimental Support for a Categorical Compositional Distributional Model of Meaning

Edward Grefenstette, M. Sadrzadeh - 2011

3 papers in library cite

[21]Factored 3-Way Restricted Boltzmann machines for Modeling Natural Images

Alex Krizhevsky, Geoffrey E. Hinton - 2010

3 papers in library cite

[22]From Machine Learning to Machine Reasoning

Leon Bottou - 2011

3 papers in library cite

[23]Combining Symbolic and Distributional Models of Meaning

S. Clark, S. Pulman - 2007

2 papers in library cite

[24]Compositional Matrix-Space Models for Sentiment Analysis

A. Yessenalina, C. Cardie - 2011

2 papers in library cite

[25]Contextual Valence Shifters

L. Polanyi, A. Zaenen - 2006

2 papers in library cite

[26]Distributional Memory: A General Framework for Corpus-Based Semantics

M. Baroni, A. Lenci - 2010

2 papers in library cite

[27]Learning to Map Sentences to Logical Form: Structured Classification With Probabilistic Categorial Grammars

L. S. Zettlemoyer, Michael Collins - 2005

2 papers in library cite

[28]Multi-Step Regression Learning for Compositional Distributional Semantics

Edward Grefenstette, G. Dinu, Y. Z. Zhang, M. Sadrzadeh, M. Baroni - 2013

2 papers in library cite

[29]A Latent Factor Model for Highly Multi-Relational Data

R. Jenatton, N. L. Roux, Antoine Bordes, G. Obozinski - 2012

1 paper in library cites

[30]A Natural History of Negation

L. R. Horn - 1989

1 paper in library cites

[31]A System for Real-Time Twitter Sentiment Analysis of 2012 u.s. presidential Election Cycle

Haiming Wang, D. Can, A. Kazemzadeh, F. Bar, S. Narayanan - 2012

1 paper in library cites

[32]Compositional Matrix-Space Models of Language

S. Rudolph, E. Giesbrecht - 2010

1 paper in library cites

[33]Denial and Contrast: A Relevance Theoretic Analysis of 'But'

D. Blakemore - 1989

1 paper in library cites

[34]Dependency-Based Construction of Semantic Space Models

S. Pado, Mirella Lapata - 2007

1 paper in library cites

[35]Holographic Reduced Representations

T. A. Plate - 1995

1 paper in library cites

[36]If's, And's, and But's About Conjunction

R. Lakoff - 1971

1 paper in library cites

[37]Information, Relevance, and Social Decisionmaking: Some Principles and Results of Decision-Theoretic Semantics

A. Merin - 1999

1 paper in library cites

[38]Large Vocabulary Speech Recognition Using Deep Tensor Neural Networks

D. Yu, L. Deng, F. Seide - 2012

1 paper in library cites

[39]Mapping Part-Whole Hierarchies Into Connectionist Networks

Geoffrey E. Hinton - 1990

1 paper in library cites

[40]Minimizers, Maximizers, and the Rhetoric of Scalar Reasoning

M. Israel - 2001

1 paper in library cites

[41]Modelling Relational Data Using Bayesian Clustered Tensor Factorization

Ilya Sutskever, Ruslan Salakhutdinov, Joshua B. Tenenbaum - 2009

1 paper in library cites

[42]Semantic Vector Products: Some Initial Investigations

D. Widdows - 2008

1 paper in library cites

[43]Sentiment Composition

K. Moilanen, S. Pulman - 2007

1 paper in library cites

[44]United We Stand: Improving Sentiment Analysis by Joining Machine Learning and Rule Based Methods

V. Rentoumi, S. Petrakis, M. Klenner, G. A. Vouros, V. Karkaletsis - 2010

1 paper in library cites

Cited by

24

papers in your library

Cites

7

papers in your library

Read

on October 18, 2025

I didn't really like the first paper and I don't really like this one. I think the dataset is more influential than the methodology. I think Stanford folks are too focused on old school NLP.

Tags

Paper Aliases

No aliases