AI summary

This paper introduces Optimal Brain Damage (OBD), a novel technique that uses second-derivative information to selectively delete unimportant weights from a neural network, improving generalization and reducing complexity, demonstrated on handwritten digit recognition.

Abstract

We have used information-theoretic ideas to derive a class of practical and nearly optimal schemes for adapting the size of a neural network. By removing unimportant weights from a network, several improvements can be expected: better generalization, fewer training examples required, and improved speed of learning and/or classification. The basic idea is to use second-derivative information to make a tradeoff between network complexity and training set error. Experiments confirm the usefulness of the methods on a real-world application.