Papperoni

2008

Labelme: A Database and Web-Based Tool for Image Annotation

Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, William T. Freeman

Open PDF Google Scholar

citations

Cite Score

75

AI summary

This paper introduces LabelMe, a web-based image annotation tool and dataset featuring polygon annotations for object detection and recognition, enhanced with WordNet for semantic consistency, and capable of semi-automatic labeling to expand data with minimal user supervision.

Main Contributions

Introduction of LabelMe, a web-based annotation tool for creating large, high-quality image datasets.
Development of a dataset with polygonal ground truth labels for numerous object categories, useful for object detection and recognition.
Methodology to enhance object labels using WordNet for semantic consistency and querying at different hierarchy levels.
Heuristics for discovering object-part relationships and inferring depth ordering in cluttered scenes.
Proposal of a semi-automatic labeling framework to efficiently expand the dataset by training detectors on existing labels and applying them to web images.

Abstract

We seek to build a large collection of images with ground truth labels to be used for object detection and recognition research. Such data is useful for supervised learning and quantitative evaluation. To achieve this, we developed a web-based tool that allows easy image annotation and instant sharing of such annotations. Using this annotation tool, we have collected a large dataset that spans many object categories, often containing multiple instances over a wide variety of images. We quantify the contents of the dataset and compare against existing state of the art datasets used for object recognition and detection. Also, we show how to extend the dataset to automatically enhance object labels with WordNet, discover object parts, recover a depth ordering of objects in a scene, and increase the number of labels using minimal user supervision and images from the web.

Citation Graph

Loading graph...

References [45]

Sort:

Filter:

[1]Learning Generative Visual Models From Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

Li Fei Fei, Rob Fergus, Pietro Perona - 2004

15 papers in library cite

I think most people cite this thinking this is where the Caltech 101 dataset comes from (it's not). Anyway, it's just an extension of the other dataset and it's very mathy, not NNs, and uninteresting.

[2]Labelme: A Database and Web-Based Tool for Image Annotation

Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, William T. Freeman - 2008

10 papers in library cite

It's a good paper overall but not worth the read. They just describe the platform (which for the time may have been a paradigm shift from in-house datasets). Maybe the basis for Amazon MT?

[3]One-Shot Learning of Object Categories

Li Fei Fei, Rob Fergus, Pietro Perona - 2006

5 papers in library cite

It's a nice problem they are trying to solve, though I doubt the effectiveness. Maybe it was really relevant in a time where they didn't have data. Problem is it focuses too much on the methodology, and I am not too interested in it.

[4]Labeling Images With a Computer Game

Luis Von Ahn, Laura Dabbish - 2004

5 papers in library cite

It's alright. TBH I expected a bit more. It's a good idea but I feel that they were very optimist around people playing - and it didn't work

[5]Learning Methods for Generic Object Recognition With Invariance to Pose and Lighting

Yann Lecun, Fu Jie Huang, Leon Bottou - 2004

18 papers in library cite

Good paper, nice methodology for creating different images. However, I think that this was not too impactful... I don't see this being used a lot.

[6]Learning Object Categories From Google's Image Search

Rob Fergus, Li Fei Fei, Pietro Perona, Andrew Zisserman - 2005

3 papers in library cite

Wow, they focus totally on the LSA stuff, and not on the Google Image Search stuff. Anticlimatic.

[7]WordNet: An Electronic Lexical Database

C. Fellbaum - 1998

12 papers in library cite

It's huge and I don't think it will add much (it is a book)

[8]Rapid Object Detection Using a Boosted Cascade of Simple Features

P. Viola, M. J. Jones - 2001

10 papers in library cite

[9]The Nature of Statistical Learning Theory

V. Vapnik - 1995

9 papers in library cite

Book, 300+ pages

[10]Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope

Aude Oliva, Antonio Torralba - 2001

7 papers in library cite

[11]The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results

Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn, Andrew Zisserman - 2007

7 papers in library cite

[12]A Bayesian Approach to Unsupervised One-Shot Learning of Object Categories

Li Fei Fei, Rob Fergus, Pietro Perona - 2003

4 papers in library cite

[13]Combined Object Categorization and Segmentation With an Implicit Shape Model

B. Leibe, A. Leonardis, B. Schiele - 2004

4 papers in library cite

[14]Putting Objects in Perspective

D. Hoiem, A. A. Efros, M. Hebert - 2006

4 papers in library cite

[15]Sharing Features: Efficient Boosting Procedures for Multiclass Object Detection

Antonio Torralba, Kevin P. Murphy, William T. Freeman - 2004

4 papers in library cite

[16]Color Indexing

M. Swain, D. Ballard - 1991

3 papers in library cite

[17]Learning to Detect Objects in Images via a Sparse, Part-Based Representation

Sandhini Agarwal, A. Awan, Dan Roth - 2004

3 papers in library cite

[18]Recognition-by-Components: A Theory of Human Image Understanding

I. Biederman - 1987

3 papers in library cite

[19]The 2005 PASCAL Visual Object Classes Challenge

Mark Everingham, Andrew Zisserman, Christopher K. I. Williams, Luc Van Gool - 2006

3 papers in library cite

66 pages

[20]The Caltech 256

Greg Griffin, Alex Holub, Pietro Perona - 2006

3 papers in library cite

[21]Analyzing Appearance and Contour Based Methods for Object Categorization

B. Leibe, B. Schiele - 2003

2 papers in library cite

[22]Animals on the Web

T. L. Berg, D. A. Forsyth - 2006

2 papers in library cite

[23]Contextual Priming for Object Detection

Antonio Torralba - 2003

2 papers in library cite

[24]Discovering Objects and Their Location in Images

Josef Sivic, B. Russell, A. Efros, Andrew Zisserman, W. Freeman - 2005

2 papers in library cite

[25]Modeling Scenes With Local Descriptors and Latent Aspects

P. Quelhas, F. Monay, J. M. Odobez, D. Gatica, T. Tuytelaars, Luc Van Gool - 2005

2 papers in library cite

[26]Peekaboom: A Game for Locating Objects in Images

Luis Von Ahn, Rosanne Liu, M. Blum - 2006

2 papers in library cite

[27]Photo Tourism: Exploring Photo Collections in 3D

N. Snavely, S. M. Seitz, R. Szeliski - 2006

2 papers in library cite

[28]The Open Mind Initiative

D. G. Stork - 1999

2 papers in library cite

[29]A Bootstrapping Algorithm for Learning Linear Models of Object Classes

T. Vetter, M. J. Jones, T. Poggio - 1997

1 paper in library cites

[30]A Boundary-Fragment-Model for Object Detection

A. Opelt, A. Pinz, Andrew Zisserman - 2006

1 paper in library cites

[31]CBCL Streetscenes

S. Bileschi - 2006

1 paper in library cites

[32]Consistent Line Clusters for Building Recognition in Cbir

Yiwei Li, L. G. Shapiro - 2002

1 paper in library cites

[33]Describing Visual Scenes Using Transformed Dirichlet Processes

E. Sudderth, Antonio Torralba, William T. Freeman, W. Willsky - 2005

1 paper in library cites

[34]Eigenfaces for Recognition

M. Turk, A. Pentland - 1991

1 paper in library cites

[35]Feature Reduction and Hierarchy of Classifiers for Fast Object Detection in Video Images

B. Heisele, T. Serre, S. Mukherjee, T. Poggio - 2001

1 paper in library cites

Missing author listMissing year

[36]Flickr Photo Sharing Service

1 paper in library cites

[37]Generic Object Recognition With Boosting

A. Opelt, A. Pinz, M. Fussenegger, P. Auer - 2006

1 paper in library cites

[38]Interleaved Object Categorization and Segmentation

B. Leibe - 2005

1 paper in library cites

[39]Introduction to the Special Issue on Word Sense Disambiguation: The State of the Art

N. Ide, J. Vronis - 1998

1 paper in library cites

[40]Learning Hierarchical Models of Scenes, Objects, and Parts

E. Sudderth, Antonio Torralba, William T. Freeman, W. Willsky - 2005

1 paper in library cites

[41]Object Categorization by Learned Universal Visual dictionary

John Winn, A. Criminisi, T. Minka - 2005

1 paper in library cites

[42]Semi-Automatic Visual Learning (Seville): a Tutorial on Active Learning for Visual Object Recognition

Y. Abramson, Y. Freund - 2005

1 paper in library cites

[43]Soil-47, the Surrey Object Image Library

J. Burianek, A. Ahmadyfard, J. Kittler - 2000

1 paper in library cites

[44]Using Multiple Segmentations to Discover Objects and Their Extent in Image Collections

Bryan C. Russell, A. A. Efros, Josef Sivic, William T. Freeman, Andrew Zisserman - 2006

1 paper in library cites

[45]Word: Wiry Object Recognition Database

O. Carmichael, M. Hebert - 2004

1 paper in library cites

Cited by

10

papers in your library

Cites

7

papers in your library

Read

on February 1, 2026

It's a good paper overall but not worth the read. They just describe the platform (which for the time may have been a paradigm shift from in-house datasets). Maybe the basis for Amazon MT?

Tags

Paper Aliases

No aliases