Papperoni

2019

Passage Re-Ranking With BERT

Rodrigo Nogueira, Kyunghyun Cho

Open PDF Google Scholar

citations

Cite Score

46

AI summary

This paper re-implements BERT for query-based passage re-ranking, achieving state-of-the-art results on the TREC-CAR dataset and MS MARCO passage retrieval task, outperforming previous methods by 27% (relative) in MRR@10.

Main Contributions

A simple re-implementation of BERT is described for query-based passage re-ranking.
Achieved state-of-the-art results on the TREC-CAR dataset.
Achieved state-of-the-art results on the MS MARCO passage retrieval task, outperforming previous state-of-the-art by 27% (relative) in MRR@10.
Demonstrated that BERT-based models surpass previous state-of-the-art with only a fraction of available training data.
The code to reproduce the results is made publicly available.

Abstract

Recently, neural models pretrained on a language modeling task, such as ELMO (Peters et al., 2017), OpenAI GPT (Radford et al., 2018), and BERT (Devlin et al., 2018), have achieved impressive results on various natural language processing tasks such as question-answering and natural language inference. In this paper, we describe a simple re-implementation of BERT for query-based passage re-ranking. Our system is the state of the art on the TREC-CAR dataset and the top entry in the leaderboard of the MS MARCO passage retrieval task, outperforming the previous state of the art by 27% (relative) in MRR@10. The code to reproduce our results is available at https://github.com/nyu-dl/dl4marco-bert

Citation Graph

Loading graph...

References [22]

Sort:

Filter:

[1]Adam: A Method for Stochastic Optimization

D. P. Kingma, Jimmy Lei Ba - 2014

49 papers in library cite

Amazing paper! Very well explained and huge impact. I am amazed that they made something so simple even when it requires a lot of background mathematical knowledge

[2]BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding

Jacob Devlin, M. W. Chang, K. Lee, Kristina Toutanova - 2018

39 papers in library cite

Simply amazing. It's very impressive how they make a leap vs. existing stuff (you can see from the references, pretty much no one is doing what they are doing, other than GPT)

[3]Improving Language Understanding by Generative Pre-Training

Alec Radford, K. Narasimhan, T. Salimans, Ilya Sutskever - 2018

23 papers in library cite

Very simple and very nice! Easy to understand and revolutionary maybe?

[4]SQuAD: 100,000+ Questions for Machine Comprehension of Text

P. Rajpurkar, J. Zhang, K. Lopyrev, Percy Liang - 2016

37 papers in library cite

Nice paper that introduced an important dataset. Not much else though.

[5]TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

M. Joshi, E. Choi, D. Weld, Luke Zettlemoyer - 2017

18 papers in library cite

I like the way they collect the data, and I think this is a nice dataset. However, it seems like they didn't even try to make a good baseline.

[6]Bidirectional Attention Flow for Machine Comprehension

M. Seo, A. Kembhavi, Ali Farhadi, Hananneh Hajishirzi - 2017

13 papers in library cite

It's alright but the method seems absurdly complex. Maybe I am a bit biased because it's like the 20th paper that I read with attention + LSTMs...

[7]Semi-Supervised Sequence Tagging With Bidirectional Language Models

M. E. Peters, W. Ammar, C. Bhagavatula, Russell Power - 2017

5 papers in library cite

Nothing too different from other LSTM + Attention papers

[8]Simple and Effective Multi-Paragraph Reading Comprehension

C. Clark, Matt Gardner - 2017

7 papers in library cite

Very nice paper! I think it's a stretch to call it "simple", but the paper is very well written and easy to follow.

[9]Reading Wikipedia to Answer Open-Domain Questions

Deli Chen, Adam Fisch, Jason Weston, Antoine Bordes - 2017

10 papers in library cite

Open Domain QA with wikipedia

[10]MS MARCO: A Human Generated Machine Reading Comprehension Dataset

T. N. Nguyen, M. Rosenberg, X. Song, Jianfeng Gao, S. Tiwary, R. Majumder, L. Deng - 2016

8 papers in library cite

I am not sure if this is very relevant, but it has a few citations.

[11]SearchQA: A New Q&A Dataset Augmented With Context From a Search Engine

M. Dunn, L. Sagun, M. Higgins, V. U. Guney, V. Cirik, Kyunghyun Cho - 2017

5 papers in library cite

[12]The Neural Hype and Comparisons Against Weak Baseline

Junyang Lin - 2019

1 paper in library cites

Reranking with BERT cites, seems like a info retrieval comparison vs. neural

[13]Trec complex Answer Retrieval Overview

L. Dietz, M. Verma, F. Radlinski, N. Craswell - 2017

1 paper in library cites

[14]Qanet: Combining Local Convolution With Global Self-Attention for Reading Compre- Hension

A. W. Yu, David Dohan, M. T. Luong, R. Zhao, K. Chen, M. Norouzi, Quoc V. Le - 2018

3 papers in library cite

[15]A Deep Relevance Matching Model for Ad-Hoc Retrieval

J. Guo, Yu Fan, Q. Ai, W. B. Croft - 2016

2 papers in library cite

[16]Convolutional Neural Networks for Soft-Matching N-Grams in Ad-Hoc Search

Z. Dai, Caiming Xiong, J. Callan, Ze Liu - 2018

2 papers in library cite

[17]End-to-End Neural Ad-Hoc Ranking With Kernel Pooling

Caiming Xiong, Z. Dai, J. Callan, Ze Liu, Russell Power - 2017

2 papers in library cite

[18]Quasar: Datasets for Question Answering by Search and Reading

B. Dhingra, K. Mazaitis, W. W. Cohen - 2017

2 papers in library cite

[19]Anserini: Reproducible Ranking Baselines Using Lucene

P. Yang, H. Fang, Junyang Lin - 2018

1 paper in library cites

[20]Co-Pacrr: A Context-Aware Neural Ir Model for Ad-Hoc Retrieval

K. Hui, A. Yates, K. Berberich, Gerard De Melo - 2018

1 paper in library cites

[21]Contextualized Pacrr for Complex Answer Retrieval

S. Macavaney, A. Yates, K. Hui - 2017

1 paper in library cites

[22]Learning to Match Using Local and Distributed Representations of Text for Web Search

B. Mitra, F. Diaz, N. Craswell - 2017

1 paper in library cites

Cited by

1

papers in your library

Cites

13

papers in your library

Read

on January 11, 2026

It's good but very very short. I think it's just good because it's SotA and straightforward, but lacks content.

Tags

Paper Aliases

No aliases