Papperoni

2014

Very Deep Convolutional Networks for Large-Scale Image Recognition

K. Simonyan, Andrew Zisserman

Open PDF Google Scholar

citations

Cite Score

100

AI summary

This paper introduces very deep convolutional networks with small 3x3 filters, achieving state-of-the-art results on the ILSVRC classification and localization tasks and demonstrating excellent performance on other image recognition datasets; the two best-performing ConvNet models are released to the public.

Main Contributions

The paper introduces very deep convolutional networks (up to 19 layers).
The paper evaluates the impact of network depth on image classification accuracy.
The paper shows that very small (3x3) convolution filters can achieve state-of-the-art results.
The paper achieves state-of-the-art results on the ILSVRC classification and localization tasks.
The paper demonstrates that the learned representations generalize well to other image recognition datasets.

Abstract

In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3 × 3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16–19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Citation Graph

Loading graph...

References [34]

Sort:

Filter:

[1]ImageNet Classification With Deep Convolutional Neural Networks

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton - 2012

71 papers in library cite

I'm giving this a 5 just because of the impact, but this is VEEERY derivative of earlier work. Kudos for them for putting it all together, but really there's nothing revolutionary here.

[2]ImageNet: A Large-Scale Hierarchical Image Database

J. Deng, W. Dong, Richard Socher, L. J. Li, K. Li, Li Fei Fei - 2009

28 papers in library cite

Very nice idea and huge impact!

[3]Going Deeper With Convolutions

Christian Szegedy, Weizhou Liu, Y. Jia, P. Sermanet, S. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich - 2015

20 papers in library cite

Introduced the inception algorithm, which is nice. The paper is quite good, but I had to google some stuff to understand it fully. Nice contribution and SotA, but TBH I felt that it wasn't toooo good of a read.

[4]Fully Convolutional Networks for Semantic Segmentation

J. Long, E. Shelhamer, Trevor Darrell - 2015

7 papers in library cite

I didn't really like the way the paper is written and the results seem a bit underwhelming. However, it's nice that they do it in a fully convolutional way and that they increase the speed.

[5]Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Ross Girshick, J. Donahue, Trevor Darrell, Jitendra Malik - 2014

18 papers in library cite

Good results, beat overfeat, used pretraining for improving performance. Only issue is that the paper is overly long...

[6]Understanding the Difficulty of Training Deep Feedforward Neural Networks

Yoshua Bengio - 2010

20 papers in library cite

Nice but underwhelming results (they still underperform vs. pretraining). I also didn't really like the way it's written. It's not bad, it's just a bit clunky. Worth the read though.

[7]Visualizing and Understanding Convolutional Networks

Matthew D. Zeiler, Rob Fergus - 2014

15 papers in library cite

Very good explanation and visualization of CNNs, and also nice that they use their findings to improve the performance. The ablation study is also nice.

[8]Backpropagation Applied to Handwritten Zip-Code Recognition

Yann Lecun, B. Boser, John S. Denker, D. Henderson, R. E. Howard, W. Hubbard, L. D. Jackal - 1989

24 papers in library cite

The first convolution NN! Very simple concept and very simply explained. Very good results and overall a good read.

[9]Caffe: Convolutional Architecture for Fast Feature Embedding

Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, Ross Girshick, S. Guadarrama, Trevor Darrell - 2014

12 papers in library cite

Nothing new really, but worth the read. It's nice because it's the precursor to current AI frameworks + has a Python interface. Also good that model representation is separate from implementation

[10]Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

K. He, X. Zhang, S. Ren, Jian Sun - 2014

6 papers in library cite

Very simple, general and effective method. The paper ends at page ~4 TBH, the rest is just results and gets boring. Good contribution though.

[11]Network in Network

M. Lin, Qinlang Chen, Shuicheng Yan - 2013

11 papers in library cite

I think this was badly written and explained. The idea is nice but I didn't like the paper at all.

[12]The Pascal Visual Object Classes Challenge: A Retrospective

Mark Everingham, S. M. Ali Eslami, Luc Van Gool, C. Williams, John Winn, Andrew Zisserman - 2015

1 paper in library cites

Dying with grace. Nice to see what happened at the end of PASCAL

[13]Learning Generative Visual Models From Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

Li Fei Fei, Rob Fergus, Pietro Perona - 2004

15 papers in library cite

I think most people cite this thinking this is where the Caltech 101 dataset comes from (it's not). Anyway, it's just an extension of the other dataset and it's very mathy, not NNs, and uninteresting.

[14]DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

J. Donahue, Y. Jia, Oriol Vinyals, J. Hoffman, N. Zhang, E. Tzeng, Trevor Darrell - 2014

15 papers in library cite

Very nice paper. First I've seen (and based on the text, first ever) about feature extraction for images. It's very nice to see embeddings doing SotA

[15]Large Scale Distributed Deep Networks

Jeffrey Dean, G. S. Corrado, R. Monga, K. Chen, M. Devin, Quoc V. Le, Mark Z. Mao, Marc'aurelio Ranzato, A. Senior, P. Tucker, K. Yang, Andrew Y. Ng - 2012

16 papers in library cite

Good paper, nice algorithm. Nothing too crazy, but I understand the impact. I think the work to create the system was larger than the algorithm itself.

[16]Caltech-256 Object Category Dataset

Greg Griffin, Alex Holub, Pietro Perona - 2007

9 papers in library cite

Boring read, but at least they were somewhat impactful.

[17]Flexible, High Performance Convolutional Neural Networks for Image Classification

Dan C. Ciresan, Ueli Meier, Jonathan Masci, Luca M. Gambardella, Jürgen Schmidhuber - 2011

3 papers in library cite

Schmidhuber + small filters CNN

[18]Unifying Visual-Semantic Embeddings With Multimodal Neural Language Models

Richard S. Zemel - 2014

5 papers in library cite

I think what I like the most about this paper is that they can use regularities, but I like that they put the images and text in the same embedding space by increasing similarity of positive samples and reducing similarity of negative ones - neat!

[19]One Weird Trick for Parallelizing Convolutional Neural Networks

Alex Krizhevsky - 2014

3 papers in library cite

Very nice paper. Good new approach to training and explains the idea and rationale well. Not too many citations but used by SotA CNNs at the time.

[20]OverFeat: Integrated Recognition, Localization and Detection Using Convolutional Networks

P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, Rob Fergus, Yann Lecun - 2014

16 papers in library cite

Very convoluted method, was SotA for only a bit of time, and the paper is very boring.

[21]Imagenet Large Scale Visual Recognition Challenge

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Zhongqiang Huang, A. Karpathy, A. Khosla, M. Bernstein - 2014

18 papers in library cite

Imagenet dataset challenge paper

[22]Deep Visual-Semantic Alignments for Generating Image Descriptions

A. Karpathy, Li Fei Fei - 2014

6 papers in library cite

Early multimodality

[23]CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

A. Razavian, H. Azizpour, J. Sullivan, S. Carlsson - 2014

6 papers in library cite

CNN feature extraction

[24]Return of the Devil in the Details: Delving Deep Into Convolutional Nets

K. Chatfield, K. Simonyan, A. Vedaldi, Andrew Zisserman - 2014

5 papers in library cite

Using CNNs as feature extractors

[25]Multi-Digit Number Recognition From Street View Imagery Using Deep Convolutional Neural Networks

Ian J. Goodfellow, Y. Bulatov, J. Ibarz, S. Arnoud, V. Shet - 2014

4 papers in library cite

Goodfellow, street number recog

[26]Some Improvements on Deep Convolutional Neural Network Based Image Classification

A. G. Howard - 2013

4 papers in library cite

CNNs with images of different scales

[27]Learning and Transferring Mid-Level Image Representations Using Convolutional Neural Networks

Maxime Oquab, Leon Bottou, I. Laptev, Josef Sivic - 2014

4 papers in library cite

[28]Improving the Fisher Kernel for Large-Scale Image Classification

F. Perronnin, J. Sanchez, T. Mensink - 2010

2 papers in library cite

[29]Actions and Attributes From Wholes and Parts

G. Gkioxari, Ross Girshick, Jitendra Malik - 2014

1 paper in library cites

[30]CNN: Single-Label to Multi-Label

Yixuan Wei, W. Xia, J. Huang, B. Ni, J. Dong, Y. Zhao, Shuicheng Yan - 2014

1 paper in library cites

[31]Deep Convolutional Filter Banks for Texture Recognition and Segmentation

M. Cimpoi, S. Maji, A. Vedaldi - 2014

1 paper in library cites

[32]Material Recognition in the Wild With the Materials in Context Database

S. Bell, P. Upchurch, N. Snavely, K. Bala - 2014

1 paper in library cites

[33]Regularized Max Pooling for Image Categorization

M. Hoai - 2014

1 paper in library cites

[34]Two-Stream Convolutional Networks for Action Recognition in Videos

K. Simonyan, Andrew Zisserman - 2014

1 paper in library cites

Cited by

20

papers in your library

Cites

26

papers in your library

Read

on July 20, 2025

This is very good! The great thing here is small filters and depth analysis, but truly they do some other stuff as well: SotA, generalization for other tasks, and open source their models. Very nice.

Tags

Paper Aliases

No aliases