Papperoni

2015

Fully Convolutional Networks for Semantic Segmentation

J. Long, E. Shelhamer, Trevor Darrell

Open PDF Google Scholar

citations

Cite Score

98

AI summary

This paper introduces fully convolutional networks (FCNs) for semantic segmentation, adapting classification networks like AlexNet, VGGNet, and GoogLeNet. It achieves state-of-the-art results on PASCAL VOC, NYUDv2, and SIFT Flow datasets, demonstrating efficient inference.

Main Contributions

Introduces fully convolutional networks (FCNs) for semantic segmentation, enabling end-to-end, pixel-to-pixel training.
Adapts existing classification networks (AlexNet, VGGNet, GoogLeNet) into FCNs via fine-tuning.
Defines a novel skip architecture that combines deep, coarse semantic information with shallow, fine appearance information.
Achieves state-of-the-art segmentation results on PASCAL VOC, NYUDv2, and SIFT Flow datasets.
Demonstrates efficient inference, with typical image segmentation taking less than one fifth of a second.

Abstract

Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic segmentation. Our key insight is to build "fully convolutional" networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet [19], the VGG net [31], and GoogLeNet [32]) into fully convolutional networks and transfer their learned representations by fine-tuning [4] to the segmentation task. We then define a novel architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional network achieves state-of-the-art segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012), NYUDv2, and SIFT Flow, while inference takes less than one fifth of a second for a typical image.

Citation Graph

Loading graph...

References [39]

Sort:

Filter:

[1]Very Deep Convolutional Networks for Large-Scale Image Recognition

K. Simonyan, Andrew Zisserman - 2014

20 papers in library cite

This is very good! The great thing here is small filters and depth analysis, but truly they do some other stuff as well: SotA, generalization for other tasks, and open source their models. Very nice.

[2]ImageNet Classification With Deep Convolutional Neural Networks

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton - 2012

71 papers in library cite

I'm giving this a 5 just because of the impact, but this is VEEERY derivative of earlier work. Kudos for them for putting it all together, but really there's nothing revolutionary here.

[3]Going Deeper With Convolutions

Christian Szegedy, Weizhou Liu, Y. Jia, P. Sermanet, S. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich - 2015

20 papers in library cite

Introduced the inception algorithm, which is nice. The paper is quite good, but I had to google some stuff to understand it fully. Nice contribution and SotA, but TBH I felt that it wasn't toooo good of a read.

[4]Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Ross Girshick, J. Donahue, Trevor Darrell, Jitendra Malik - 2014

18 papers in library cite

Good results, beat overfeat, used pretraining for improving performance. Only issue is that the paper is overly long...

[5]Visualizing and Understanding Convolutional Networks

Matthew D. Zeiler, Rob Fergus - 2014

15 papers in library cite

Very good explanation and visualization of CNNs, and also nice that they use their findings to improve the performance. The ablation study is also nice.

[6]Backpropagation Applied to Handwritten Zip-Code Recognition

Yann Lecun, B. Boser, John S. Denker, D. Henderson, R. E. Howard, W. Hubbard, L. D. Jackal - 1989

24 papers in library cite

The first convolution NN! Very simple concept and very simply explained. Very good results and overall a good read.

[7]Caffe: Convolutional Architecture for Fast Feature Embedding

Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, Ross Girshick, S. Guadarrama, Trevor Darrell - 2014

12 papers in library cite

Nothing new really, but worth the read. It's nice because it's the precursor to current AI frameworks + has a Python interface. Also good that model representation is separate from implementation

[8]Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

K. He, X. Zhang, S. Ren, Jian Sun - 2014

6 papers in library cite

Very simple, general and effective method. The paper ends at page ~4 TBH, the rest is just results and gets boring. Good contribution though.

[9]Efficient Backprop

Yann Lecun, Leon Bottou, G. B. Orr, Klaus Robert Muller - 1998

20 papers in library cite

The first half is very very good. The remainder is very boring.

[10]DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

J. Donahue, Y. Jia, Oriol Vinyals, J. Hoffman, N. Zhang, E. Tzeng, Trevor Darrell - 2014

15 papers in library cite

Very nice paper. First I've seen (and based on the text, first ever) about feature extraction for images. It's very nice to see embeddings doing SotA

[11]Regularization of Neural Networks Using Dropconnect

L. Wan, M. Zeiler, S. Zhang, Rob Fergus - 2013

8 papers in library cite

I feel that the method is very complex and does not improve much on top of regular dropout.

[12]Building High-Level Features Using Large Scale Unsupervised Learning

Quoc V. Le, M. A. Ranzato, R. Monga, M. Devin, K. Chen, G. S. Corrado, Jeffrey Dean, Andrew Y. Ng - 2012

10 papers in library cite

Very nice and very early work - seems very simple but very insightful to use an autoencoder to detect objects. Also, very similar to the neocognitron :)

[13]Restoring an Image Taken Through a Window Covered With Dirt or Rain

Rob Fergus - 2013

1 paper in library cites

Good paper, nice to see how image denoising is done. Nothing "wow" but good contribution.

[14]Multi-Digit Recognition Using a Space Displacement Neural Network

O. Matan, C. J. C. Burges, Yann Lecun, John S. Denker - 1992

3 papers in library cite

Meh. They just use a CNN to read more digits, but doesn't seem to have a ton of impact.

[15]OverFeat: Integrated Recognition, Localization and Detection Using Convolutional Networks

P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, Rob Fergus, Yann Lecun - 2014

16 papers in library cite

Very convoluted method, was SotA for only a bit of time, and the paper is very boring.

[16]Learning Hierarchical Features for Scene Labeling

Clement Farabet, C. Couprie, L. Najman, Yann Lecun - 2013

6 papers in library cite

Cited by deep learning paper (and 4 more)

[17]Semantic Segmentation With Second-Order Pooling

J. Carreira, R. Caseiro, J. Batista, C. Sminchisescu - 2012

4 papers in library cite

[18]Toward Automatic Phenotyping of Developing Embryos From Videos

F. Ning, D. Delhomme, Yann Lecun, F. Piano, Leon Bottou, P. Barbano - 2005

4 papers in library cite

[19]Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images

Dan C. Ciresan, A. Giusti, Jürgen Schmidhuber - 2012

3 papers in library cite

[20]Convolutional Feature Masking for Joint Object and Stuff Segmentation

Josef Dai, K. He, Jian Sun - 2015

2 papers in library cite

[21]Indoor Segmentation and Support Inference From RGBD Images

N. Silberman, D. Hoiem, P. Kohli, Rob Fergus - 2012

2 papers in library cite

[22]Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation

J. Tompson, A. Jain, Yann Lecun, C. Bregler - 2014

2 papers in library cite

[23]Recurrent Convolutional Neural Networks for Scene Labeling

P. Pinheiro, Ronan Collobert - 2014

2 papers in library cite

[24]Superparsing: Scalable Nonparametric Image Parsing With Superpixels

J. Tighe, Svetlana Lazebnik - 2010

2 papers in library cite

[25]Depth Map Prediction From a Single Image Using a Multi-Scale Deep Network

D. Eigen, C. Puhrsch, Rob Fergus - 2014

1 paper in library cites

[26]Descriptor Matching With Convolutional Neural Networks: A Comparison to SIFT

P. Fischer, Alexey Dosovitskiy, T. Brox - 2014

1 paper in library cites

[27]Do Convnets Learn Correspondence?

J. Long, N. Zhang, Trevor Darrell - 2014

1 paper in library cites

[28]Finding Things: Image Parsing With Regions and Per-Exemplar Detectors

J. Tighe, Svetlana Lazebnik - 2013

1 paper in library cites

[29]Learning Rich Features From RGB-D images for Object Detection and Segmentation

S. Gupta, Ross Girshick, P. Arbelaez, Jitendra Malik - 2014

1 paper in library cites

[30]N⁴-fields: Neural Network Nearest Neighbor Fields for Image Transforms

Yaroslav Ganin, Victor Lempitsky - 2014

1 paper in library cites

[31]Part-Based R-CNNS for Fine-Grained Category Detection

N. Zhang, J. Donahue, Ross Girshick, Trevor Darrell - 2014

1 paper in library cites

[32]Perceptual Organization and Recognition of Indoor Scenes From RGB-D images

S. Gupta, P. Arbelaez, Jitendra Malik - 2013

1 paper in library cites

[33]Postal Address Block Location Using a Convolutional Locator Network

R. Wolf, John C. Platt - 1994

1 paper in library cites

[34]Semantic Contours From Inverse Detectors

B. Hariharan, P. Arbelaez, L. Bourdev, S. Maji, Jitendra Malik - 2011

1 paper in library cites

[35]Sift Flow: Dense Correspondence Across Scenes and Its Applications

C. L. Liu, J. Yuen, Antonio Torralba - 2011

1 paper in library cites

[36]Simultaneous Detection and Segmentation

B. Hariharan, P. Arbelaez, Ross Girshick, Jitendra Malik - 2014

1 paper in library cites

[37]The Gaussian Scale-Space Paradigm and the Multiscale Local Jet

L. Florack, B. T. H. Romeny, M. Viergever, J. Koenderink - 1996

1 paper in library cites

Missing author listMissing year

[38]The PASCAL Visual Object Classes Challenge 2011 (VOC2011) Results

1 paper in library cites

[39]The Role of Context for Object Detection and Semantic Segmentation in the Wild

R. Mottaghi, X. Chen, Xiaodong Liu, N. G. Cho, S. W. Lee, Sanja Fidler, R. Urtasun, A. Yuille - 2014

1 paper in library cites

Cited by

7

papers in your library

Cites

16

papers in your library

Read

on October 21, 2025

I didn't really like the way the paper is written and the results seem a bit underwhelming. However, it's nice that they do it in a fully convolutional way and that they increase the speed.

Tags

Paper Aliases

No aliases