Papperoni

2017

Get to the Point: Summarization With Pointer-Generator Networks

A. See, P. J. Liu, Christopher D. Manning

Open PDF Google Scholar

citations

Cite Score

70

AI summary

The paper introduces a pointer-generator network with a coverage mechanism for abstractive summarization, using the CNN/Daily Mail dataset. The model facilitates copying words and reduces repetition, outperforming existing abstractive summarization methods by at least 2 ROUGE points.

Main Contributions

Introduces a hybrid pointer-generator network that copies words from the source text via pointing, improving accuracy and handling of OOV words.
Proposes a novel variant of the coverage vector to track and control coverage of the source document, effectively eliminating repetition.
Applies the model to the CNN/Daily Mail dataset, demonstrating its effectiveness on multi-sentence summaries.
Achieves state-of-the-art abstractive summarization results, outperforming existing systems by at least 2 ROUGE points.
Demonstrates that coverage is remarkably effective for eliminating repetition.

Abstract

Neural sequence-to-sequence models have provided a viable new approach for abstractive text summarization (meaning they are not restricted to simply selecting and rearranging passages from the original text). However, these models have two shortcomings: they are liable to reproduce factual details inaccurately, and they tend to repeat themselves. In this work we propose a novel architecture that augments the standard sequence-to-sequence attentional model in two orthogonal ways. First, we use a hybrid pointer-generator network that can copy words from the source text via pointing, which aids accurate reproduction of information, while retaining the ability to produce novel words through the generator. Second, we use coverage to keep track of what has been summarized, which discourages repetition. We apply our model to the CNN / Daily Mail summarization task, outperforming the current abstractive state-of-the-art by at least 2 ROUGE points.

Citation Graph

Loading graph...

References [31]

Sort:

Filter:

[1]Neural Machine Translation by Jointly Learning to Align and Translate

D. Bahdanau, Kyunghyun Cho, Yoshua Bengio - 2014

59 papers in library cite

Introduces the attention mechanism - amazing overall

[2]Sequence to Sequence Learning With Neural Networks

Ilya Sutskever, Oriol Vinyals, Quoc V. Le - 2014

58 papers in library cite

Good paper, but I think it only got famous because they set a new good baseline for NNs in MT. Their main contribution was reversing the source sentence TBH.

[3]ROUGE: A Package for Automatic Evaluation of Summaries

Chin Yew Lin - 2004

9 papers in library cite

Wow, very poorly written and badly explained. This is the exact opposite of BLEU

[4]Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

John Duchi, Elad Hazan, Yoram Singer - 2011

19 papers in library cite

I actually skimmed through most of this. It's not a bad paper, but it's a math paper, not AI.

[5]Show, Attend and Tell: Neural Image Caption Generation With Visual Attention

K. Xu, Jimmy Lei Ba, R. Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, R. Zemel, Yoshua Bengio - 2015

12 papers in library cite

It's a nice paper. I liked the soft attention way more than the hard one, and I am a bit mad that it wasn't the best lol And also it's the first paper I read about multimodality, but it seems that this was bustling at the time. Also results are kinda bad.

[6]Teaching Machines to Read and Comprehend

K. M. Hermann, T. Kocisky, Edward Grefenstette, L. Espeholt, W. Kay, M. Suleyman, Phil Blunsom - 2015

31 papers in library cite

Nice way of converting unsupervised data to train for Q&A - and nice visualizations as well :) But I think their main contribution is the dataset. Maybe with the dataset they "unlocked" summarization?

[7]Pointer Networks

Oriol Vinyals, M. Fortunato, Navdeep Jaitly - 2015

10 papers in library cite

Cool concept. Nice that it works and can find good solutions for TSP.

[8]A Neural Attention Model for Abstractive Sentence Summarization

Alexander M. Rush, S. Chopra, Jason Weston - 2015

13 papers in library cite

TBH the paper is a bit boring and nothing new after reading a bunch of more modern techniques. I feel that they could have done a better job considering that seq2seq existed at the time. Either way, points for being the first to propose summarization with NNs.

[9]Pointer Sentinel Mixture Models

S. Merity, Caiming Xiong, J. Bradbury, Richard Socher - 2017

12 papers in library cite

I really liked the methodology, but I had to read it a few times to understand it intuitively - I think they should have done a better job at explaining it.

[10]Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond

R. Nallapati, B. Zhou, C. N. D. Santos, C. G. Gulcehre, Bing Xiang - 2016

10 papers in library cite

Cool that they use pointer switching, and they introduce the CNN/daily mail dataset for summarization, which is a nice insight.

[11]Meteor Universal: Language Specific Translation Evaluation for Any Target Language

Alon Lavie - 2014

2 papers in library cite

So boring! Maybe the original Meteor paper is better though

[12]Sequence Level Training With Recurrent Neural Networks

Marc'aurelio Ranzato, S. Chopra, Michael Auli, Wojciech Zaremba - 2015

6 papers in library cite

Exposure bias

[13]Abstractive Sentence Summarization With Attentive Recurrent Neural Networks

S. Chopra, Michael Auli, A. Rush, S. Harvard - 2016

5 papers in library cite

Summarization with attention

[14]Modeling Coverage for Neural Machine Translation

Zhuowen Tu, Z. L. Lu, Yibo Liu, Xiaodong Liu, H. Li - 2016

4 papers in library cite

Beam search coverage study

[15]Pointing the Unknown Words

C. G. Gulcehre, S. Ahn, R. Nallapati, B. Zhou, Yoshua Bengio - 2016

7 papers in library cite

Bengio, pointer networks

[16]Temporal Attention Model for Neural Machine Translation

B. Sankaran, H. Mi, Y. A. Onaizan, A. Ittycheriah - 2016

3 papers in library cite

Intra-attention

[17]Statistical Machine Translation

P. Koehn - 2010

5 papers in library cite

[18]Incorporating Copying Mechanism in Sequence-to-Sequence Learning

J. Gu, Z. L. Lu, H. Li, V. O. K. Li - 2016

4 papers in library cite

[19]Distraction-Based Neural Networks for Modeling Documents

Qinlang Chen, X. Zhu, Z. Ling, S. Wei, H. Jiang - 2016

2 papers in library cite

[20]Efficient Summarization With Read-Again and Copy Mechanism

W. Zeng, W. Luo, Sanja Fidler, R. Urtasun - 2016

2 papers in library cite

[21]Summarunner: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents

R. Nallapati, F. Zhai, B. Zhou - 2017

2 papers in library cite

[22]A Trainable Document Summarizer

J. Kupiec, J. Pedersen, F. Chen - 1995

1 paper in library cites

[23]Automatic Text Summarization: Past, Present and Future

H. Saggion, T. Poibeau - 2013

1 paper in library cites

[24]Constructing Literature Abstracts by Computer: Techniques and Prospects

C. Paice - 1990

1 paper in library cites

[25]Coverage Embedding Models for Neural Machine Translation

H. Mi, B. Sankaran, Zhengtao Wang, A. Ittycheriah - 2016

1 paper in library cites

[26]Language as a Latent Variable: Discrete Generative Models for Sentence Compression

Y. Miao, Phil Blunsom - 2016

1 paper in library cites

[27]Looking for a Few Good Metrics: Automatic Summarization Evaluation-How Many Samples Are Enough?

C. Lin - 2004

1 paper in library cites

[28]Neural Headline Generation on Abstract Meaning Representation

S. Takase, J. Suzuki, N. Okazaki, T. Hirao, M. Nagata - 2016

1 paper in library cites

[29]RNN-based Encoder-Decoder Approach With Word Frequency Estimation

J. Suzuki, M. Nagata - 2016

1 paper in library cites

[30]Sentence Reduction for Automatic Text Summarization

H. Jing - 2000

1 paper in library cites

[31]Unsupervised Sentence Enhancement for Automatic Summarization

J. C. K. Cheung, G. Penn - 2014

1 paper in library cites

Cited by

8

papers in your library

Cites

16

papers in your library

Read

on November 6, 2025

It's a bit of the same thing of the other ones. I am not sure if this was the first or not, but I am getting a bit bored of this

Tags

Paper Aliases

No aliases