Papperoni

2017

Simple and Effective Multi-Paragraph Reading Comprehension

C. Clark, Matt Gardner

Open PDF Google Scholar

citations

Cite Score

23

AI summary

This paper introduces a simple and effective approach to multi-paragraph reading comprehension, using a TF-IDF heuristic for paragraph selection and a shared-normalization training objective to produce globally correct output, achieving a score of 71.3 F1 on the web portion of TriviaQA.

Main Contributions

Proposes an improved pipelined method that achieves state-of-the-art results.
Introduces a method for training models to produce accurate per-paragraph confidence scores.
Shows how combining this method with multiple paragraph selection further increases performance.
Proposes a TF-IDF heuristic to select which paragraphs to train and test on.
Uses a summed objective function that marginalizes the model's output over all locations the answer text occurs.

Abstract

We consider the problem of adapting neural paragraph-level question answering models to the case where entire documents are given as input. Our proposed solution trains models to produce well calibrated confidence scores for their results on individual paragraphs. We sample multiple paragraphs from the documents during training, and use a shared-normalization training objective that encourages the model to produce globally correct output. We combine this method with a state-of-the-art pipeline for training models on document QA data. Experiments demonstrate strong performance on several document QA datasets. Overall, we are able to achieve a score of 71.3 F1 on the web portion of TriviaQA, a large improvement from the 56.7 F1 of the previous best system.

Citation Graph

Loading graph...

References [29]

Sort:

Filter:

[1]GloVe: Global Vectors for Word Representation

Jeffrey Pennington, Richard Socher, Christopher D. Manning - 2014

31 papers in library cite

Not a bad paper, I just don't like the motivation and I think the methodology is poorly explained and hard to follow. I can't deny the good results though...

[2]Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation

Kyunghyun Cho, B. V. Merrienboer, C. G. Gulcehre, D. Bahdanau, F. Bougares, Holger Schwenk, Yoshua Bengio - 2014

38 papers in library cite

Introduces RNN encoder-decoder. I love it :)

[3]SQuAD: 100,000+ Questions for Machine Comprehension of Text

P. Rajpurkar, J. Zhang, K. Lopyrev, Percy Liang - 2016

37 papers in library cite

Nice paper that introduced an important dataset. Not much else though.

[4]Adadelta: An Adaptive Learning Rate Method

Matthew D. Zeiler - 2012

13 papers in library cite

Cool contribution. Simple and straightforward, and not too mathy. Based on ADAGRAD.

[5]Teaching Machines to Read and Comprehend

K. M. Hermann, T. Kocisky, Edward Grefenstette, L. Espeholt, W. Kay, M. Suleyman, Phil Blunsom - 2015

31 papers in library cite

Nice way of converting unsupervised data to train for Q&A - and nice visualizations as well :) But I think their main contribution is the dataset. Maybe with the dataset they "unlocked" summarization?

[6]TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

M. Joshi, E. Choi, D. Weld, Luke Zettlemoyer - 2017

18 papers in library cite

I like the way they collect the data, and I think this is a nice dataset. However, it seems like they didn't even try to make a good baseline.

[7]Bidirectional Attention Flow for Machine Comprehension

M. Seo, A. Kembhavi, Ali Farhadi, Hananneh Hajishirzi - 2017

13 papers in library cite

It's alright but the method seems absurdly complex. Maybe I am a bit biased because it's like the 20th paper that I read with attention + LSTMs...

[8]A Theoretically Grounded Application of Dropout in Recurrent Neural Networks

Yarin Gal - 2015

9 papers in library cite

The methodology is very simple and effective, but maybe it was too simple for a paper, so they put A TON of math on top of it... Unnecessary really.

[9]Adversarial Examples for Evaluating Reading Comprehension Systems

R. Jia, Percy Liang - 2017

11 papers in library cite

I liked it a lot! It's good to see people testing things rather than just trying to beat SotA!

[10]Long Short-Term Memory-Networks for Machine Reading

Mirella Lapata - 2016

8 papers in library cite

I read this more as an example of intra-attention, but this is not the main focus of the paper. I think visualization/explanation is a bit bad, and it doesn't seem too impactful. I kept thinking that this is starting to get too complicated, and indeed it was surpassed by transformers right after that.

[11]The Goldilocks Principle: Reading Children's Books With Explicit Memory Representations

F. Hill, Antoine Bordes, S. Chopra, Jason Weston - 2015

14 papers in library cite

Cool use of memory networks.

[12]Reading Wikipedia to Answer Open-Domain Questions

Deli Chen, Adam Fisch, Jason Weston, Antoine Bordes - 2017

10 papers in library cite

Open Domain QA with wikipedia

[13]MS MARCO: A Human Generated Machine Reading Comprehension Dataset

T. N. Nguyen, M. Rosenberg, X. Song, Jianfeng Gao, S. Tiwary, R. Majumder, L. Deng - 2016

8 papers in library cite

I am not sure if this is very relevant, but it has a few citations.

[14]Gated Self-Matching Networks for Reading Comprehension and Question Answering

Wenyi Wang, N. Yang, F. Wei, B. Chang, M. Zhou - 2017

5 papers in library cite

QA Model

[15]Machine Comprehension Using Match-LSTM and Answer Pointer

Shijie Wang, J. J. Jiang - 2017

6 papers in library cite

Answer pointer

[16]Text Understanding With the Attention Sum Reader Network

R. Kadlec, M. Schmid, O. Bajgar, Jan Kleindienst - 2016

7 papers in library cite

Not too cited but cited a lot internally

[17]Semantic Parsing on Freebase From Question-Answer Pairs

Jonathan Berant, A. Chou, R. Frostig, Percy Liang - 2013

8 papers in library cite

[18]The trec-8 Question Answering Track Evaluation

E. M. Voorhees, D. M. Tice - 1999

5 papers in library cite

[19]Question Answering Through Transfer Learning From Large Fine-Grained Supervision Data

S. Min, M. J. Seo, Hananneh Hajishirzi - 2017

4 papers in library cite

[20]Wikireading: A Novel Large-Scale Language Understanding Task Over Wikipedia

D. Hewlett, A. Lacoste, Llion Jones, Illia Polosukhin, A. Fandrianto, J. Han, M. Kelcey, D. Berthelot - 2016

4 papers in library cite

[21]Clueweb09 Data Set

J. Callan, M. Hoy, C. Yoo, L. Zhao - 2009

2 papers in library cite

[22]Memen: Multi-Layer Embedding With Memory Networks for Machine Comprehension

B. Pan, H. Li, Zhuoye Zhao, B. Cao, D. Cai, X. He - 2017

2 papers in library cite

[23]Quasar: Datasets for Question Answering by Search and Reading

B. Dhingra, K. Mazaitis, W. W. Cohen - 2017

2 papers in library cite

[24]R3: Reinforced Ranker-Reader for Open-Domain Question Answering

Shijie Wang, M. Yu, X. Guo, Zhengtao Wang, T. Klinger, Wenxuan Zhang, S. Chang, Gerald Tesauro, B. Zhou, J. J. Jiang - 2018

2 papers in library cite

[25]Dynamic Integration of Background Knowledge in Neural NLU Systems

Dirk Weissenborn, T. Koisk, C. Dyer - 2017

1 paper in library cites

[26]FastQA: A Simple and Efficient Neural Architecture for Question Answering

Dirk Weissenborn, G. Wiese, L. Seiffe - 2017

1 paper in library cites

[27]Mnemonic Reader: Machine Comprehension With Iterative Aligning and Multi-Hop Answer Pointing

M. Hu, Y. Peng, X. Qiu - 2017

1 paper in library cites

[28]S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension

C. Tan, F. Wei, N. Yang, W. Lv, M. Zhou - 2017

1 paper in library cites

[29]YodaQA: A Modular Question Answering System Pipeline

P. Baudis - 2015

1 paper in library cites

Cited by

7

papers in your library

Cites

16

papers in your library

Read

on October 28, 2025

Very nice paper! I think it's a stretch to call it "simple", but the paper is very well written and easy to follow.

Tags

Paper Aliases

No aliases