Papperoni

1989

Multilayer Feedforward Networks Are Universal Approximators

Kur Hornik, Maxwell Stinchcombe, Halbert White

Open PDF Google Scholar

citations

Cite Score

95

AI summary

This paper rigorously establishes that standard multilayer feedforward networks with a single hidden layer and arbitrary squashing functions are universal approximators, capable of approximating any Borel measurable function to any desired accuracy, provided sufficient hidden units are available.

Main Contributions

Rigorous proof that standard multilayer feedforward networks with one hidden layer are universal approximators for Borel measurable functions.
Utilizes the Stone-Weierstrass Theorem and the cosine squasher to establish approximation capabilities.
Shows that these networks can approximate functions to any desired degree of accuracy, regardless of the specific squashing function used.
Demonstrates that both Sigma-Pi and Sigma networks are universal approximators.
Addresses the theoretical underpinnings of why feedforward networks have been successful in various applications.

Abstract

This paper rigorously establishes that standard multilayer feedforward networks with as few as one hidden layer using arbitrary squashing functions are capable of approximating any Borel measurable function from one finite dimensional space to another to any desired degree of accuracy, provided sufficiently many hidden units are available. In this sense, multilayer feedforward networks are a class of universal approximators.

Citation Graph

Loading graph...

References [24]

Sort:

Filter:

[1]Multilayer Feedforward Networks Are Universal Approximators

Kur Hornik, Maxwell Stinchcombe, Halbert White - 1989

3 papers in library cite

It's a bunch of proofs, so not nice to read but it's very impactful.

M. Minsky, S. Papert - 1969

12 papers in library cite

Book, 500 pages

[3]Modèles Connexionnistes De l'Apprentissage

Yann Lecun - 1987

9 papers in library cite

[4]Approximation by Superpositions of a Sigmoidal Function

G. Cybenko - 1988

2 papers in library cite

[5]Principles of Mathematical Analysis

W. Rudin, Others - 1964

2 papers in library cite

[6]Theory of the Backpropagation neural Network

R. H. Nielsen - 1989

2 papers in library cite

[7]Abstract Inference

U. Grenander - 1981

1 paper in library cites

[8]Capabilities of Three Layer Perceptrons

B. Irie, S. Miyake - 1988

1 paper in library cites

[9]Convergence Rates of Maximum Likelihood and Related Estimates in General Parameter Spaces

J. A. Severini, W. H. Wong - 1987

1 paper in library cites

[10]E-Entropy and E-Capacity of Sets in Functional Spaces

Andrei N. Kolmogorov, V. M. Tihomirov - 1961

1 paper in library cites

[11]How Neural Networks Work

A. Lapedes, R. Farber - 1988

1 paper in library cites

[12]Kolmogorov's Mapping Neural Network Existence theorem

R. H. Nielsen - 1987

1 paper in library cites

[13]Measure Theory

P. R. Halmos - 1974

1 paper in library cites

[14]Multilayer Feedforward Networks Can Learn Arbitrary Mappings: Connectionist Nonparametric Regression With Automatic and Semi-Automatic Determination of Network Complexity

Halbert White - 1988

1 paper in library cites

[15]Nonlinear Dynamics of Artificial Neural Systems

Timothy Maxwell, G. L. Giles, Y. C. Lee, H. H. Chen - 1986

1 paper in library cites

[16]On the Representation of Continuous Functions of Many Variables by Superposition of Continuous Functions of One Variable and Addition

Andrei N. Kolmogorov - 1957

1 paper in library cites

[17]Probability and Measure

P. Billingsley - 1979

1 paper in library cites

Missing year

[18]Some Results for Sieve Estimation With Dependent Observations

Halbert White, J. M. Wooldridge

1 paper in library cites

[19]The Case for Conceptual and Operational Separation of Network Architectures and Learning Mechanisms

Halbert White - 1988

1 paper in library cites

[20]The Logic of Activation Functions

Ronald J. Williams - 1986

1 paper in library cites

[21]The Thirteenth Problem of Hilbert

G. G. Lorentz - 1976

1 paper in library cites

[22]There Exists a Neural Network That Does Not Make Avoidable Mistables

A. R. Gallant, Halbert White - 1988

1 paper in library cites

J. Dugundji - 1966

1 paper in library cites

[24]Universal Approximation Using Feedforward Networks With Non-Sigmoid Hidden Layer Activation Functions

Maxwell Stinchcombe, Halbert White - 1989

1 paper in library cites

Cited by

3

papers in your library

Cites

2

papers in your library

Read

on January 18, 2026

It's a bunch of proofs, so not nice to read but it's very impactful.

Tags

Paper Aliases

No aliases