Papperoni

2015

A Large Annotated Corpus for Learning Natural Language Inference

Samuel R. Bowman, G. Angeli, Christopher Potts, Christopher D. Manning

Open PDF Google Scholar

citations

Cite Score

75

AI summary

This paper introduces the Stanford Natural Language Inference (SNLI) corpus, a new collection of 570K sentence pairs labeled for entailment, contradiction, and semantic independence. It allows a neural network-based model to perform competitively on natural language inference benchmarks for the first time.

Main Contributions

Introduces the Stanford Natural Language Inference (SNLI) corpus, a new, freely available collection of 570K sentence pairs labeled for entailment, contradiction, and semantic independence.
The corpus was created with a novel grounded task based on image captioning.
Shows that a neural network-based model can perform competitively on natural language inference benchmarks for the first time using the new corpus.
Demonstrates that the representations learned by a neural network model on the corpus can be used to dramatically improve performance on a standard challenge dataset (SICK).
Achieved best reported performance for an unaugmented neural network model, surpassing available EOP models, and approaching both the overall state of the art and the level of interannotator agreement on SICK.

Abstract

Understanding entailment and contradiction is fundamental to understanding natural language, and inference about entailment and contradiction is a valuable testing ground for the development of semantic representations. However, machine learning research in this area has been dramatically limited by the lack of large-scale resources. To address this, we introduce the Stanford Natural Language Inference corpus, a new, freely available collection of labeled sentence pairs, written by humans doing a novel grounded task based on image captioning. At 570K pairs, it is two orders of magnitude larger than all other resources of its type. This increase in scale allows lexicalized classifiers to outperform some sophisticated existing entailment models, and it allows a neural network-based model to perform competitively on natural language inference benchmarks for the first time.

Citation Graph

Loading graph...

References [35]

Sort:

Filter:

[1]Long Short-Term Memory

Sepp Hochreiter, Jürgen Schmidhuber - 1997

94 papers in library cite

LSTMs FTW!

[2]Dropout: A Simple Way to Prevent Neural Networks From Overfitting

N. Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov - 2014

20 papers in library cite

Good paper, but it's mostly a review of the method described in the other paper with more results. It's longer as well, so I would suggest just reading the other one.

[3]GloVe: Global Vectors for Word Representation

Jeffrey Pennington, Richard Socher, Christopher D. Manning - 2014

31 papers in library cite

Not a bad paper, I just don't like the motivation and I think the methodology is poorly explained and hard to follow. I can't deny the good results though...

[4]WordNet: A Lexical Database for English

G. Miller - 1995

5 papers in library cite

Meh. It seems like it was publish so that people could cite the dataset somehow. Nothing interesting, but quick read and very used.

[5]Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Richard Socher, A. Perelygin, Jeffrey Wu, J. Chuang, C. Manning, A. Ng, Christopher Potts - 2013

24 papers in library cite

I didn't really like the first paper and I don't really like this one. I think the dataset is more influential than the methodology. I think Stanford folks are too focused on old school NLP.

[6]Adadelta: An Adaptive Learning Rate Method

Matthew D. Zeiler - 2012

13 papers in library cite

Cool contribution. Simple and straightforward, and not too mathy. Based on ADAGRAD.

[7]The PASCAL Recognising Textual Entailment Challenge

Ido Dagan, O. Glickman, Bernardo Magnini - 2005

19 papers in library cite

It's very nice how they had the foresight to create a challenge that became relevant like 10 years later.

[8]Memory Networks

Jason Weston, S. Chopra, Antoine Bordes - 2015

18 papers in library cite

The first half of the paper (when they discuss the concept in a very abstract way) is amazing. However, the actual methodology was very convoluted - I did not like it. I thought that Neural Turing Machines were inspired in this, but actually they are contemporary... So anyway, the concept is nice, execution is not.

[9]A Sick Cure for the Evaluation of Compositional Distributional Semantic Models

Marco Marelli, S. Menini, M. Baroni, L. Bentivogli, R. Bernardi, R. Z. Elli - 2014

7 papers in library cite

Just a basic dataset paper. Nothing much.

[10]Towards AI-complete Question Answering: A Set of Prerequisite Toy Tasks

Jason Weston, Antoine Bordes, S. Chopra, Tomas Mikolov - 2015

11 papers in library cite

It's a good idea and a nice read but the bad part is that most of the tasks are already easy.

[11]Direct Transfer of Learned Information Among Neural Networks

Lorien Y. Pratt, Jack Mostow, Candace A. Kamm - 1991

3 papers in library cite

Boring read and uninteresting results. I think they stumbled onto transfer learning but didn't know it yet, so they explored it in a bad way.

[12]On Our Best Behaviour

Hector J. Levesque - 2013

2 papers in library cite

I loved this paper. So relevant despite being 10+ years old! Everyone that works with AI should read this.

[13]Understanding Natural Language

T. Winograd - 1972

7 papers in library cite

Book, but early work on NLU

[14]Accurate Unlexicalized Parsing

Dan Klein, Christopher D. Manning - 2003

7 papers in library cite

[15]Baselines and Bigrams: Simple, Good Sentiment and Topic Classification

Shijie Wang, Manning, C. Christopher - 2012

7 papers in library cite

[16]Semeval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences Through Semantic Relatedness and Textual Entailment

Marco Marelli, L. Bentivogli, M. Baroni, R. Bernardi, S. Menini, R. Zamparelli - 2014

7 papers in library cite

[17]The Third PASCAL Recognizing Textual Entailment Challenge

D. Giampiccolo, Bernardo Magnini, Ido Dagan, B. Dolan - 2007

7 papers in library cite

[18]Entailment, Intensionality and Text Understanding

C. Condoravdi, D. Crouch, V. D. Paiva, R. Stolle, D. G. Bobrow - 2003

5 papers in library cite

[19]From Image Descriptions to Visual Denotations: New Similarity Metrics for Semantic Inference Over Event Descriptions

P. Young, A. L. M. Hodosh, J. Hockenmaier - 2014

5 papers in library cite

[20]Illinois-lh: A Denotational and Distributional Approach to Semantics

A. Lai, J. Hockenmaier - 2014

5 papers in library cite

[21]A Natural Logic Inference System

Y. Fyodorov, Y. Winter, N. Francez - 2000

4 papers in library cite

[22]An Extended Model of Natural Logic

B. Maccartney, Christopher D. Manning - 2009

4 papers in library cite

[23]Recognising Textual Entailment With Logical Inference

J. Bos, Katja Markert - 2005

4 papers in library cite

[24]Finding Contradictions in Text

M. C. D. Marneffe, A. N. Rafferty, Christopher D. Manning - 2008

3 papers in library cite

[25]A Brief History of Natural Logic

J. V. Benthem - 2008

2 papers in library cite

[26]Brown Corpus Manual

W. N. Francis, H. Kucera - 1979

2 papers in library cite

[27]Recursive Neural Networks Can Learn Logical Semantics

Samuel R. Bowman, Christopher Potts, Christopher D. Manning - 2015

2 papers in library cite

[28]Semantic Theory

J. J. Katz - 1972

2 papers in library cite

[29]Design and Realization of a Modular Architecture for Textual Entailment

S. Pado, T. G. Noh, A. Stern, R. Wang, R. Zanoli - 2014

1 paper in library cites

[30]Focused Entailment Graphs for Open IE Propositions

Omer Levy, Ido Dagan, J. Goldberger - 2014

1 paper in library cites

[31]Information Structure and Non-Canonical Syntax

G. Ward, B. Birner - 2004

1 paper in library cites

[32]PPDB 2.0: Better Paraphrase Ranking, Fine-Grained Entailment Relations, Word Embeddings, and Style Classification

Ellie Pavlick, J. Bos, M. Nissim, C. Beller, B. V. Durme, Chris Callison Burch - 2015

1 paper in library cites

[33]Recognizing Textual Entailment Using Sentence Similarity Based on Dependency Tree Skeletons

R. Wang, G. Neumann - 2007

1 paper in library cites

[34]The Excitement Open Platform for Textual Inferences

Bernardo Magnini, R. Zanoli, Ido Dagan, K. Eichler, G. Neumann, T. G. Noh, S. Pado, A. Stern, Omer Levy - 2014

1 paper in library cites

[35]Verb-Ocean: Mining the Web for Fine-Grained Semantic Verb Relations

T. Chklovski, P. Pantel - 2004

1 paper in library cites

Cited by

25

papers in your library

Cites

13

papers in your library

Read

on October 23, 2025

Dataset collection is ok. The model that they create seems very low effort.

Tags

Paper Aliases

No aliases