Papperoni

1990

Learning From Hints in Neural Networks

Yaser S. Abu Mostafa

citations

Cite Score

AI summary

This paper introduces a method for incorporating invariance hints into neural network learning using descent methods, by expressing hints as examples, and also demonstrates that learning in neural networks remains NP-complete even with biologically plausible hints.

Main Contributions

Introduces 'learning from hints' as an extension of 'learning from examples', allowing general information about a function f beyond just input-output pairs.
Proposes a method to incorporate any invariance hint into descent-based learning algorithms by treating the hint itself as a set of examples, which allows for generating arbitrary hint examples without knowing the function f.
Discusses the information value and complexity value of hints, noting that hints can reduce candidate functions and search steps.
Shows that general learning in neural networks remains NP-complete, even with biologically plausible hints about weight signs or magnitudes, due to polynomial reducibility to the original problem.

Abstract

Learning from examples is the process of taking input–output examples of an unknown function f and infering an implementation of f. Learning from hints allows for general information about f to be used instead of just input–output examples. We introduce a method for incorporating any invariance hint about f in any descent method for learning from examples. We also show that learning in a neural network remains NP-complete with a certain, biologically plausible, hint about the network. We discuss the information value and the complexity value of hints. 1990 Academic Press, Inc.

Citation Graph

Loading graph...

References [7]

Sort:

Filter:

[1]Learning Internal Representations by Error Propagation

D. E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams - 1986

46 papers in library cite

Google Scholar

I expected very little of this, but was so good in explaining concepts! Very good read. It gets a bit boring when it starts explaining things by the end of the chapter, but good nonetheless.

[2]On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities

V. N. Vapnik, A. Y. Chervonenkis - 1971

3 papers in library cite

Google Scholar

[3]A Theory of the Learnable

L. G. Valiant - 1984

2 papers in library cite

Google Scholar

[4]On Connectionist Models

J. H. Hong - 1987

1 paper in library cites

Google Scholar

[5]On the Complexity of Loading Shallow Neural Networks

J. S. Judd - 1988

1 paper in library cites

Google Scholar

[6]Random Problems

Yaser S. Abu Mostafa - 1988

1 paper in library cites

Google Scholar

[7]The Vapnik-Chervonenkis Dimension: Information Versus Complexity in Learning

Yaser S. Abu Mostafa - 1989

1 paper in library cites

Google Scholar

Cited by

papers in your library

Cites

papers in your library

Read

on January 23, 2026

Very nice read, but the results are very underwhelming. I hoped for some more concrete concepts/results.