Papperoni

2010

Deconvolutional Networks

Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor, Rob Fergus

Open PDF Google Scholar

citations

Cite Score

59

AI summary

This paper introduces Deconvolutional Networks, an unsupervised learning framework for hierarchical image representations that generate robust features for image analysis and synthesis, achieving competitive performance on Caltech-101 for object recognition and demonstrating improved denoising.

Main Contributions

Proposes Deconvolutional Networks, a framework for unsupervised construction of hierarchical image representations.
Introduces a top-down, generative model that reconstructs input signals from learned feature maps using convolutions and learned filters.
Develops an efficient optimization technique for inferring feature maps, which involves solving a multi-component deconvolution problem with sparsity regularization.
Demonstrates that the learned features capture mid-level concepts like edge junctions, parallel lines, curves, and geometric elements.
Achieves competitive object recognition performance on Caltech-101 and improves image denoising, linking high-level recognition with low-level tasks.

Abstract

Building robust low and mid-level image representations, beyond edge primitives, is a long-standing goal in vision. Many existing feature detectors spatially pool edge information which destroys cues such as edge intersections, parallelism and symmetry. We present a learning framework where features that capture these mid-level cues spontaneously emerge from image data. Our approach is based on the convolutional decomposition of images under a sparsity constraint and is totally unsupervised. By building a hierarchy of such decompositions we can learn rich feature sets that are a robust image representation for both the analysis and synthesis of images.

Citation Graph

Loading graph...

References [31]

Sort:

Filter:

[1]Gradient-Based Learning Applied to Document Recognition

Yann Lecun, Leon Bottou, Yoshua Bengio, Patrick Haffner - 1998

62 papers in library cite

I absolutely hated this paper. Has ~50 pages but seems like 200 pages. Takes too long to explain some things that really is just repeating itself. Also doesn't seem to add too much on top of LeNet-5. Also, focuses a lot on GTNs, which really didn't stick.

[2]A Fast Learning Algorithm for Deep Belief Nets

Geoffrey E. Hinton, S. Osindero, Y. Teh - 2006

43 papers in library cite

The paper does not explain anything. It just throws the idea and a bunch of math, but doesn't really care to explain the concepts.

[3]Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

Svetlana Lazebnik, Cordelia Schmid, Jean Ponce - 2006

14 papers in library cite

It's a fun read, but in the end is just an application of the spatial pyramid matching kernel from the other paper.

[4]Extracting and Composing Robust Features With Denoising Autoencoders

Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre Antoine Manzagol - 2008

25 papers in library cite

I am *so* glad we found an alternative to DBNs. Also, introduced the idea of denoising which is nice.

[5]Greedy Layer-Wise Training of Deep Networks

Yoshua Bengio, P. Lamblin, D. Popovici, Hugo Larochelle - 2006

33 papers in library cite

Bengio is perfect. This is everything that Hinton's paper hoped to be. Very well explained, and also tying back to real use cases (not just "hey, the math works and it reduced the score")

[6]What Is the Best Multi-Stage Architecture for Object Recognition?

K. Jarrett, Koray Kavukcuoglu, Marc'aurelio Ranzato, Yann Lecun - 2009

20 papers in library cite

So boring and convoluted... They try to make things more abstract by saying "filter banks" and "2 stages", but in the end it's CNNs. It seems like they are trying to tie NNs to the rest of ML, but it doesn't work.

[7]Self-Taught Learning: Transfer Learning From Unlabeled Data

Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer, A. Ng - 2007

7 papers in library cite

It's nice to make the distinction of self-taught learning vs. semi-supervised learning. However, it doesn't talk about NNs. Overall, LeCun's CNN unsupervised learning was better.

[8]Efficient Learning of Sparse Representations With an Energy-Based Model

Marc'aurelio Ranzato, C. Poultney, S. Chopra, Yann Lecun - 2006

20 papers in library cite

It's ok. Not really good, but alright.

[9]Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations

Honglak Lee, R. Grosse, R. Ranganath, Andrew Y. Ng - 2009

12 papers in library cite

[10]Sparse Feature Learning for Deep Belief Networks

Marc'aurelio Ranzato, Y. Boureau, Yann Lecun - 2008

12 papers in library cite

[11]Sparse Coding With an Overcomplete Basis Set: A Strategy Employed by V1?

Bruno A. Olshausen, David J. Field - 1997

10 papers in library cite

[12]Hierarchical Models of Object Recognition in Cortex

M. Riesenhuber, T. Poggio - 1999

8 papers in library cite

[13]Linear Spatial Pyramid Matching Using Sparse Coding for Image Classification

Jihan Yang, K. Yu, Y. Gong, T. Huang - 2009

8 papers in library cite

[14]Object Recognition With Features Inspired by Visual Cortex

T. Serre, Lior Wolf, T. Poggio - 2005

7 papers in library cite

[15]Efficient Sparse Coding Algorithms

Honglak Lee, Alexis Battle, Rajat Raina, A. Ng - 2007

6 papers in library cite

[16]Svm-knn: Discriminative Nearest Neighbor Classification for Visual Category Recognition

Haowei Zhang, A. C. Berg, M. Maire, Jitendra Malik - 2006

6 papers in library cite

[17]A Computational Model for Visual Selection

Y. Amit, D. Geman - 1999

3 papers in library cite

[18]A Stochastic Grammar of Images

S. C. Zhu, D. Mumford - 2006

3 papers in library cite

[19]Learning a Hierarchical Deformable Template for Rapid Deformable Object Parsing

L. Zhu, Yanru Chen, A. L. Yuille - 2009

2 papers in library cite

[20]Parsing Images Into Regions, Curves, and Curve Groups

Z. W. Tu, S. C. Zhu - 2006

2 papers in library cite

[21]Primal sketch: Integrating Texture and Structure

C. E. Guo, S. C. Zhu, Y. N. Wu - 2007

2 papers in library cite

[22]Similarity-Based Cross-Layered Hierarchical Representation for Object Categorization

Sanja Fidler, M. Boben, A. Leonardis - 2008

2 papers in library cite

[23]Supervised Dictionary Learning

Julien Mairal, F. Bach, Jean Ponce, G. Sapiro, Andrew Zisserman - 2009

2 papers in library cite

[24]Towards Scalable Representations of Object Categories: Learning a Hierarchy of Parts

Sanja Fidler, A. Leonardis - 2007

2 papers in library cite

D. Marr - 1982

2 papers in library cite

[26]A New Alternating Minimization algorithm for Total Variation Image Reconstruction

Yuzhi Wang, Jihan Yang, W. Yin, Y. Z. Zhang - 2008

1 paper in library cites

[27]Analytic Hyper-Laplacian Priors for Fast Image Deconvolution

Dilip Krishnan, Rob Fergus - 2009

1 paper in library cites

[28]Atomic Decomposition by Basis Pursuit

S. Chen, D. Donoho, M. Saunders - 1999

1 paper in library cites

[29]Context and Hierarchy in a Probabilistic Image Model

Y. Jin, S. Geman - 2006

1 paper in library cites

[30]Nonlinear Image Recovery With Half-Quadratic Regularization

D. Geman, Y. C. - 1995

1 paper in library cites

[31]Online Dictionary Learning for Sparse Coding

Julien Mairal, F. Bach, Jean Ponce, G. Sapiro - 2009

1 paper in library cites

Cited by

3

papers in your library

Cites

10

papers in your library

Read

on February 16, 2026

It's a very nice idea but I feel that there is a much more intuitive way of explaining the idea.

Tags

Paper Aliases

No aliases