Papperoni

2011

High-Performance Neural Networks for Visual Object Classification

Dan C. Ciresan, Ueli Meier, Jonathan Masci, Luca M. Gambardella, Jürgen Schmidhuber

Open PDF Google Scholar

citations

Cite Score

19

AI summary

This paper presents a fast, fully parameterizable GPU implementation of Convolutional Neural Network variants, achieving state-of-the-art results on NORB (2.53%), CIFAR10 (19.51%), and MNIST (0.35%) benchmarks.

Main Contributions

Introduces a fast, fully parameterizable GPU implementation of Convolutional Neural Network variants.
Demonstrates that feature extractors can be learned in a supervised way rather than being carefully designed or pre-wired.
Achieves state-of-the-art results on object classification benchmarks: NORB (2.53%), CIFAR10 (19.51%), and MNIST (0.35%).
Shows that deep hierarchical architectures trained by simple back-propagation outperform shallower ones and learn surprisingly rapidly.

Abstract

We present a fast, fully parameterizable GPU implementation of Convolutional Neural Network variants. Our feature extractors are neither carefully designed nor pre-wired, but rather learned in a supervised way. Our deep hierarchical architectures achieve the best published results on benchmarks for object classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple back-propagation perform better than more shallow ones. Learning is surprisingly rapid. NORB is completely trained within five epochs. Test error rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs, respectively.

Citation Graph

Loading graph...

References [23]

Sort:

Filter:

[1]Gradient-Based Learning Applied to Document Recognition

Yann Lecun, Leon Bottou, Yoshua Bengio, Patrick Haffner - 1998

62 papers in library cite

I absolutely hated this paper. Has ~50 pages but seems like 200 pages. Takes too long to explain some things that really is just repeating itself. Also doesn't seem to add too much on top of LeNet-5. Also, focuses a lot on GTNs, which really didn't stick.

[2]Learning Multiple Layers of Features From Tiny Images

Alex Krizhevsky - 2009

27 papers in library cite

It's alright. It mainly focuses on RBMs and their features and the actual part that describes the dataset is like 1 page. However, it's maybe the best intuitive description of an RBM I have seen. Other than that, it reads very much like an undergraduate thesis.

[3]Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position

Kunihiko Fukushima - 1980

8 papers in library cite

Good read! First conv net? :)

[4]Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis

John C. Platt - 2003

12 papers in library cite

Very good paper. Simple and pragmatic.

[5]Learning Methods for Generic Object Recognition With Invariance to Pose and Lighting

Yann Lecun, Fu Jie Huang, Leon Bottou - 2004

18 papers in library cite

Good paper, nice methodology for creating different images. However, I think that this was not too impactful... I don't see this being used a lot.

[6]High Performance Convolutional Neural Networks for Document Processing

K. Chellapilla, S. Puri, Patrice Y. Simard - 2006

3 papers in library cite

It's ok. It's not bad, it's just a bit boring, and I don't think it's bringing anything new. People already implemented unrolled CNN and GPUs, they just brought it all together.

[7]Large-Scale Object Recognition With CUDA-accelerated Hierarchical Neural Networks

R. Uetz, Sven Behnke - 2009

2 papers in library cite

It's not a bad paper, but it seems like they tried to put too much stuff in it - the CNN part is very forgettable and the CUDA part is not so well explained as other papers.

[8]Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition

Dan C. Ciresan, Ueli Meier, Luca M. Gambardella, Jürgen Schmidhuber - 2010

10 papers in library cite

It's short, simple and straight to the point (as the network in the paper). It's refreshing to read something less "academic" and more "in your face"

[9]Sparse Coding With an Overcomplete Basis Set: A Strategy Employed by V1?

Bruno A. Olshausen, David J. Field - 1997

10 papers in library cite

[10]Hierarchical Models of Object Recognition in Cortex

M. Riesenhuber, T. Poggio - 1999

8 papers in library cite

[11]An Analysis of Single-Layer Networks in Unsupervised Feature Learning

A. Coates, A. Ng, Honglak Lee - 2011

7 papers in library cite

[12]Object Recognition With Features Inspired by Visual Cortex

T. Serre, Lior Wolf, T. Poggio - 2005

7 papers in library cite

[13]Receptive Fields of Single Neurons in the Cat's Striate Cortex

D. H. Hubel, T. N. Wiesel - 1959

6 papers in library cite

[14]A High-Throughput Screening Approach to Discovering Good Forms of Biologically Inspired Visual Representation

N. Pinto, D. Doukhan, J. J. Dicarlo, D. D. Cox - 2009

3 papers in library cite

[15]Hierarchical Neural Networks for Image Interpretation

Sven Behnke - 2003

3 papers in library cite

[16]3-D Object Recognition With Deep Belief Nets

V. Nair, Geoffrey Hinton - 2009

2 papers in library cite

[17]Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition

D. Scherer, A. Muller, Sven Behnke - 2010

2 papers in library cite

[18]Improved Local Coordinate Coding Using Local Tangents

K. Yu, Tong Zhang - 2010

2 papers in library cite

[19]Learning Factorial Codes by Predictability Minimization

J. H. Schmidhuber - 1992

2 papers in library cite

[20]Independent Component Analysis Applied to Feature Extraction From Colour and Stero Images

P. O. Hoyer, A. Hyvarinen - 2000

1 paper in library cites

[21]Neocognitron for Handwritten Digit Recognition

Kunihiko Fukushima - 2003

1 paper in library cites

[22]Object Class Recognition and Localization Using Sparse Features With Limited Receptive Fields

J. Mutch, D. G. Lowe - 2008

1 paper in library cites

[23]Semilinear Predictability Minimization Produces Well-Known Feature Detectors

Jürgen Schmidhuber, M. Eldracher, B. Foltin - 1996

1 paper in library cites

Cited by

5

papers in your library

Cites

8

papers in your library

Read

on February 16, 2026

OK, I can see why this is important (implementing CNNs in GPUs before it was cool), but nothing too surprising. BTW, I read the 6 page version but this one has color so I kept this one.

Tags

Paper Aliases

Flexible, High Performance Convolutional Neural Networks for Image Classification