Papperoni

2014

Microsoft COCO: Common Objects in Context

T. Y. Lin, M. Maire, S. Belongie, James Hays, Pietro Perona, D. Ramanan, Piotr Dollar, C. L. Zitnick

Open PDF Google Scholar

citations

Cite Score

98

AI summary

This paper introduces the MS COCO dataset, a large-scale dataset for object recognition and scene understanding with 2.5 million labeled instances in 328k images. It uses per-instance segmentation and crowd worker involvement, providing baseline performance analysis using a Deformable Parts Model, outperforming models trained on prior datasets due to the diversity of non-iconic views.

Main Contributions

Introduces a new large-scale dataset (MS COCO) for object recognition and scene understanding.
The dataset contains 2.5 million labeled instances in 328k images.
Presents a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN.
Provides baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.
Emphasizes the importance of non-iconic views and contextual reasoning for object recognition.

Abstract

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.

Citation Graph

Loading graph...

References [51]

Sort:

Filter:

[1]ImageNet Classification With Deep Convolutional Neural Networks

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton - 2012

71 papers in library cite

I'm giving this a 5 just because of the impact, but this is VEEERY derivative of earlier work. Kudos for them for putting it all together, but really there's nothing revolutionary here.

[2]ImageNet: A Large-Scale Hierarchical Image Database

J. Deng, W. Dong, Richard Socher, L. J. Li, K. Li, Li Fei Fei - 2009

28 papers in library cite

Very nice idea and huge impact!

[3]Microsoft COCO: Common Objects in Context

T. Y. Lin, M. Maire, S. Belongie, James Hays, Pietro Perona, D. Ramanan, Piotr Dollar, C. L. Zitnick - 2014

14 papers in library cite

I liked this paper a lot. It's a bit long and I was already a bit tired, but it was nice overall.

[4]Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Ross Girshick, J. Donahue, Trevor Darrell, Jitendra Malik - 2014

18 papers in library cite

Good results, beat overfeat, used pretraining for improving performance. Only issue is that the paper is overly long...

[5]Learning Multiple Layers of Features From Tiny Images

Alex Krizhevsky - 2009

27 papers in library cite

It's alright. It mainly focuses on RBMs and their features and the actual part that describes the dataset is like 1 page. However, it's maybe the best intuitive description of an RBM I have seen. Other than that, it reads very much like an undergraduate thesis.

[6]The PASCAL Visual Object Classes (VOC) Challenge

Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn, Andrew Zisserman - 2010

7 papers in library cite

Good overview of the VOC challenge and good read, but too long and the part that they describe methods and results is very boring. Also, not related to NNs at the time!

[7]Learning Generative Visual Models From Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

Li Fei Fei, Rob Fergus, Pietro Perona - 2004

15 papers in library cite

I think most people cite this thinking this is where the Caltech 101 dataset comes from (it's not). Anyway, it's just an extension of the other dataset and it's very mathy, not NNs, and uninteresting.

[8]Labelme: A Database and Web-Based Tool for Image Annotation

Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, William T. Freeman - 2008

10 papers in library cite

It's a good paper overall but not worth the read. They just describe the platform (which for the time may have been a paradigm shift from in-house datasets). Maybe the basis for Amazon MT?

[9]Sun Database: Large-Scale Scene Recognition From Abbey to Zoo

Jianxiong Xiao, James Hays, K. Ehinger, Aude Oliva, Antonio Torralba - 2010

2 papers in library cite

I like the dataset and the approach. Solid methodology also. However, I don't really think "scene within a scene" is good for practical use. Also not too sure about impact.

[10]Caltech-256 Object Category Dataset

Greg Griffin, Alex Holub, Pietro Perona - 2007

9 papers in library cite

Boring read, but at least they were somewhat impactful.

[11]80 Million Tiny Images: A Large Dataset for Non-Parametric Object and Scene Recognition

Antonio Torralba, Rob Fergus, W. Freeman - 2008

8 papers in library cite

The initial part about data collection and dataset description was nice, but the part of classifying was a bit overkill

[12]OverFeat: Integrated Recognition, Localization and Detection Using Convolutional Networks

P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, Rob Fergus, Yann Lecun - 2014

16 papers in library cite

Very convoluted method, was SotA for only a bit of time, and the paper is very boring.

[13]The MNIST Database of Handwritten Digits

Yann Lecun - 1998

8 papers in library cite

Not a paper - it's actually a dataset

[14]WordNet: An Electronic Lexical Database

C. Fellbaum - 1998

12 papers in library cite

It's huge and I don't think it will add much (it is a book)

[15]Histograms of Oriented Gradients for Human Detection

N. Dalal, B. Triggs - 2005

12 papers in library cite

[16]Object Detection With Discriminatively Trained Part-Based Models

P. F. Felzenszwalb, Ross Girshick, D. Mcallester, D. Ramanan - 2010

8 papers in library cite

[17]Unbiased Look at dataset Bias

Antonio Torralba, A. Efros - 2011

5 papers in library cite

[18]Columbia Object Image Library: Coil

S. Nene, S. Nayar, H. Murase - 1996

4 papers in library cite

[19]Diagnosing Error in Object Detectors

D. Hoiem, Y. Chodpathumwan, Q. Dai - 2012

4 papers in library cite

[20]Im2text: Describing Images Using 1 Million Captioned Photographs

V. Ordonez, G. Kulkarni, T. L. Berg - 2011

3 papers in library cite

[21]A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms

D. Scharstein, R. Szeliski - 2002

2 papers in library cite

[22]Caltech-Ucsd Birds 200

Peter Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, Pietro Perona - 2010

2 papers in library cite

[23]Collecting Image Annotations Using Amazon's Mechanical Turk

C. Rashtchian, P. Young, M. Hodosh, J. Hockenmaier - 2010

2 papers in library cite

[24]Describing Objects by Their Attributes

Ali Farhadi, I. Endres, D. Hoiem, David Forsyth - 2009

2 papers in library cite

[25]Do We Need More Training Data or Better Models for Object Detection?

X. Zhu, C. Vondrick, D. Ramanan, C. Fowlkes - 2012

2 papers in library cite

[26]Evaluation of Gist Descriptors for Web-Scale Image Search

M. Douze, Hervé Jégou, H. Sandhawalia, L. Amsaleg, Cordelia Schmid - 2009

2 papers in library cite

[27]Face Detection: A Survey

E. Hjelmas, B. Low - 2001

2 papers in library cite

[28]Indoor Segmentation and Support Inference From RGBD Images

N. Silberman, D. Hoiem, P. Kohli, Rob Fergus - 2012

2 papers in library cite

[29]Scalable Multi-Label Annotation

J. Deng, O. Russakovsky, J. Krause, M. Bernstein, A. C. Berg, Li Fei Fei - 2014

2 papers in library cite

[30]A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms

S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, R. Szeliski - 2006

1 paper in library cites

[31]A Database and Evaluation Methodology for Optical Flow

S. Baker, D. Scharstein, J. Lewis, S. Roth, M. Black, R. Szeliski - 2011

1 paper in library cites

[32]Canonical Perspective and the Perception of Objects

S. Palmer, E. Rosch, P. Chase - 1981

1 paper in library cites

[33]Contour Detection and Hierarchical Image Segmentation

P. Arbelaez, M. Maire, C. Fowlkes, Jitendra Malik - 2011

1 paper in library cites

[34]Detecting Avocados to zucchinis: What Have We Done, and Where Are We Going?

O. Russakovsky, J. Deng, Zhongqiang Huang, A. Berg, Li Fei Fei - 2013

1 paper in library cites

[35]Discriminatively Trained Deformable Part Models, Release 5

Ross Girshick, P. F. Felzenszwalb, D. Mcallester - 2012

1 paper in library cites

[36]Finding Iconic Images

T. Berg, A. Berg - 2009

1 paper in library cites

[37]From Large Scale Image Categorization to Entry-Level Categories

V. Ordonez, J. Deng, Yejin Choi, A. Berg, T. Berg - 2013

1 paper in library cites

[38]Labeled Faces in the Wild

G. B. Huang, M. Ramesh, T. Berg, E. L. Miller - 2007

1 paper in library cites

[39]Layered Object Models for Image Segmentation

Yining Yang, S. Hallman, D. Ramanan, C. Fowlkes - 2012

1 paper in library cites

[40]Learning Spatial Context: Using Stuff to Find Things

G. Heitz, D. Koller - 2008

1 paper in library cites

[41]Learning to Detect Unseen Object Classes by Between-Class Attribute Transfer

C. Lampert, H. Nickisch, S. Harmeling - 2009

1 paper in library cites

[42]Learning to Localize Detected Objects

Q. Dai, D. Hoiem - 2012

1 paper in library cites

[43]Object Segmentation by Alignment of Poselet Activations to Image Contours

T. Brox, L. Bourdev, S. Maji, Jitendra Malik - 2011

1 paper in library cites

[44]OpenSurfaces: A Richly Annotated Catalog of Surface Appearance

S. Bell, P. Upchurch, N. Snavely, K. Bala - 2013

1 paper in library cites

[45]Pedestrian Detection: An Evaluation of the State of the Art

Piotr Dollar, C. Wojek, B. Schiele, Pietro Perona - 2012

1 paper in library cites

[46]Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations

L. Bourdev, Jitendra Malik - 2009

1 paper in library cites

[47]Semantic Object Classes in Video: A High-Definition Ground Truth Database

G. Brostow, J. Fauqueur, Roberto Cipolla - 2009

1 paper in library cites

[48]Spelling Sourcebook

R. Sitton - 1996

1 paper in library cites

[49]SUN Attribute Database: Discovering, Annotating, and Recognizing Scene Attributes

G. Patterson, James Hays - 2012

1 paper in library cites

[50]TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context

J. Shotton, John Winn, C. Rother, A. Criminisi - 2009

1 paper in library cites

[51]Using Segmentation to Verify Object Hypotheses

D. Ramanan - 2007

1 paper in library cites

Cited by

14

papers in your library

Cites

14

papers in your library

Read

on July 31, 2025

I liked this paper a lot. It's a bit long and I was already a bit tired, but it was nice overall.

Tags

Paper Aliases

No aliases