Papperoni

2006

Reducing the Dimensionality of Data With Neural Networks

Geoffrey Hinton, Ruslan Salakhutdinov

citations

Cite Score

AI summary

This paper introduces a deep autoencoder network initialized with a layer-by-layer pretraining procedure for dimensionality reduction, achieving superior performance compared to PCA on synthetic curves, MNIST digits, Olivetti faces, and Reuters newswire stories datasets.

Main Contributions

Introduces a layer-by-layer pretraining procedure for initializing deep autoencoder networks.
Demonstrates that the pretraining algorithm allows for efficient fine-tuning of deep networks.
Shows that deep autoencoders outperform PCA for dimensionality reduction.
Applies the method to image and document datasets.
Achieves a 1.2% error rate on the MNIST handwritten digit recognition task.

Abstract

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such ‘‘autoencoder’’ networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.

Citation Graph

Loading graph...

References [16]

Sort:

Filter:

[1]Parallel Distributed Processing: Volume 1: Foundations

D. E. Rumelhart, J. L. Mcclelland - 1986

3 papers in library cite