Papperoni

2015

A Neural Attention Model for Abstractive Sentence Summarization

Alexander M. Rush, S. Chopra, Jason Weston

Open PDF Google Scholar

citations

Cite Score

68

AI summary

This paper introduces an attention-based neural network model for abstractive sentence summarization, trained end-to-end, and achieving significant performance gains on the DUC-2004 shared task compared with several strong baselines. The model leverages a local attention mechanism to generate summaries conditioned on input sentences.

Main Contributions

Introduces a fully data-driven approach to abstractive sentence summarization.
Utilizes a local attention-based neural network model for generating summaries.
Demonstrates that the model can be trained end-to-end and scales to large training datasets.
Achieves significant performance gains on the DUC-2004 shared task compared with several strong baselines.
Explores the use of additional features to trade-off the abstractive/extractive tendency of the system

Abstract

Summarization based on text extraction is inherently limited, but generation-style abstractive methods have proven challenging to build. In this work, we propose a fully data-driven approach to abstractive sentence summarization. Our method utilizes a local attention-based model that generates each word of the summary conditioned on the input sentence. While the model is structurally simple, it can easily be trained end-to-end and scales to a large amount of training data. The model shows significant performance gains on the DUC-2004 shared task compared with several strong baselines.

Citation Graph

Loading graph...

References [28]

Sort:

Filter:

[1]Neural Machine Translation by Jointly Learning to Align and Translate

D. Bahdanau, Kyunghyun Cho, Yoshua Bengio - 2014

59 papers in library cite

Introduces the attention mechanism - amazing overall

[2]Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation

Kyunghyun Cho, B. V. Merrienboer, C. G. Gulcehre, D. Bahdanau, F. Bougares, Holger Schwenk, Yoshua Bengio - 2014

38 papers in library cite

Introduces RNN encoder-decoder. I love it :)

[3]Sequence to Sequence Learning With Neural Networks

Ilya Sutskever, Oriol Vinyals, Quoc V. Le - 2014

58 papers in library cite

Good paper, but I think it only got famous because they set a new good baseline for NNs in MT. Their main contribution was reversing the source sentence TBH.

[4]ROUGE: A Package for Automatic Evaluation of Summaries

Chin Yew Lin - 2004

9 papers in library cite

Wow, very poorly written and badly explained. This is the exact opposite of BLEU

[5]A Neural Probabilistic Language Model

Yoshua Bengio, R. Ducharme, Pascal Vincent - 2001

62 papers in library cite

What started it all. Very simple and elegant.

[6]Improving Neural Networks by Preventing Co-Adaptation of Feature Detectors

Geoffrey E. Hinton, N. Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov - 2012

25 papers in library cite

Dropout, super impactful. The idea that you are training many estimators at once is also very nice.

[7]Recurrent Continuous Translation Models

N. Kalchbrenner, Phil Blunsom - 2013

27 papers in library cite

Good paper, probably the first that used an encoder-decoder. But they used a conv. NN instead of a tradicional decoder, which I don't really like.

[8]Addressing the Rare Word Problem in Neural Machine Translation

T. Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, Wojciech Zaremba - 2014

14 papers in library cite

The method was very poorly explained. It was also worse than a paper released sooner, and more complicated. Overall not that good.

[9]Moses: Open Source Toolkit for Statistical Machine Translation

P. Koehn, H. Hoang, Alexandra Birch, Chris Callison Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin, E. Herbst - 2007

8 papers in library cite

[10]The stanford coreNLP Natural Language Processing Toolkit

Christopher D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, D. Mcclosky - 2014

6 papers in library cite

[11]English Gigaword

R. Parker, D. Graff, J. Kong, K. Chen, K. Maeda - 2011

5 papers in library cite

[12]Hedge Trimmer: A Parse-and-Trim Approach to Headline Generation

B. Dorr, D. Zajic, Richard Schwartz - 2003

3 papers in library cite

[13]Annotated Gigaword

C. Napoles, M. Gormley, B. V. Durme - 2012

2 papers in library cite

[14]BBN/Umd at Duc-2004: Topiary

D. Zajic, B. J. Dorr, Richard Schwartz - 2004

2 papers in library cite

[15]Duc in Context

P. Over, H. Dang, D. Harman - 2007

2 papers in library cite

[16]Headline Generation Based on Statistical Translation

M. Banko, V. O. Mittal, M. J. Witbrock - 2000

2 papers in library cite

[17]Introduction to Information Retrieval

Christopher D. Manning, P. Raghavan, H. Schtze - 2008

2 papers in library cite

[18]Minimum Error Rate Training in Statistical Machine Translation

F. J. Och - 2003

2 papers in library cite

[19]Overcoming the Lack of Parallel Data in Sentence Compression

K. Filippova, Y. Altun - 2013

2 papers in library cite

[20]Sentence Compression Beyond Word Deletion

T. Cohn, Mirella Lapata - 2008

2 papers in library cite

[21]Title Generation With Quasi-Synchronous Grammar

K. Woodsend, Y. Feng, Mirella Lapata - 2010

2 papers in library cite

[22]A Noisy-Channel Model for Document Compression

H. D. Iii, D. Marcu - 2002

1 paper in library cites

[23]Global Inference for Sentence Compression: An integer linear Programming Approach

J. Clarke, Mirella Lapata - 2008

1 paper in library cites

Missing year

[24]Sentence Compression

J. Clarke, Mirella Lapata

1 paper in library cites

[25]Sentence Simplification by Monolingual Machine Translation

S. Wubben, A. V. D. Bosch, E. Krahmer - 2012

1 paper in library cites

[26]Summarization Beyond Sentence Extraction: A Probabilistic Approach to Sentence Compression

K. Knight, D. Marcu - 2002

1 paper in library cites

[27]Using Hidden Markov Modeling to Decompose Human-Written Summaries

H. Jing - 2002

1 paper in library cites

[28]Z-Mert: A Fully Configurable Open Source Tool for Minimum Error Rate Training of Machine Translation Systems

O. Zaidan - 2009

1 paper in library cites

Cited by

13

papers in your library

Cites

8

papers in your library

Read

on November 26, 2025

TBH the paper is a bit boring and nothing new after reading a bunch of more modern techniques. I feel that they could have done a better job considering that seq2seq existed at the time. Either way, points for being the first to propose summarization with NNs.

Tags

Paper Aliases

No aliases