Papperoni

2004

Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication

Herbert Jaeger, Harald Haas

Open PDF Google Scholar

citations

Cite Score

72

AI summary

This paper introduces Echo State Networks (ESNs), a computationally efficient method for learning nonlinear systems, which achieved a 2400-fold accuracy improvement on chaotic time series prediction and a two orders of magnitude signal error rate improvement in wireless communication channel equalization.

Main Contributions

Introduced Echo State Networks (ESNs) as a computationally efficient method for learning nonlinear systems.
ESNs utilize large recurrent neural networks with learning restricted to output readout connections, simplifying training to linear regression.
Achieved a 2400-fold accuracy improvement over previous techniques on chaotic time series prediction using the Mackey-Glass system benchmark.
Demonstrated practical engineering application by improving signal error rate by two orders of magnitude in wireless communication channel equalization.
ESNs capitalize on massive short-term memory, enabling them to 'remember' a large number of previous inputs.

Abstract

We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.

Citation Graph

Loading graph...

References [31]

Sort:

Filter:

[1]Backpropagation Through Time: What It Does and How to Do It

P. Werbos - 1990

9 papers in library cite

Amazing tutorial! Very pragmatic. Explains very basic concepts and focus on implementation

[2]Parallel Models of Associative Memory

Geoffrey E. Hinton, J. A. Anderson - 1981

4 papers in library cite

This is actually a book, I only found chapter 6 online

[3]The echo State Approach to Analysing and Training Recurrent Neural Networks

Herbert Jaeger - 2001

2 papers in library cite

Echo state networks

[4]A New Approach to Time Series Prediction With Recurrent Neural Networks

Xingcheng Yao, Yibo Liu - 1997

1 paper in library cites

[5]Advanced Signal Processing: Algorithms, Architectures, and Implementations V

V. J. Mathews, Jaehoon Lee - 1994

1 paper in library cites

[6]Advances in Neural Information Processing Systems 15

Herbert Jaeger, S. Becker, Sebastian Thrun, K. Obermayer - 2003

1 paper in library cites

[7]Applying LSTM to Time Series Predictable Through Time-Window Approaches

F. Gers, D. Eck, J. F. Schmidhuber - 2000

1 paper in library cites

[8]Approximation Capabilities of Feedforward Neural Networks for Multivariate Input Functions

K. I. Funahashi, Y. Nakamura - 1993

1 paper in library cites

[9]Computational Neuroscience: A Comprehensive Approach

W. Maass, T. Natschlager, H. Markram, J. Feng - 2003

1 paper in library cites

[10]Control of a Robot Arm With a Recurrent Neural Network

T. M. Martinetz, S. G. Berkovich, K. J. Schulten - 1993

1 paper in library cites

[11]Dynamic Properties of Neurons in the monkey Motor Cortex

W. M. Kistler, C. I. D. Zeeuw - 2002

1 paper in library cites

[12]Dynamics of Feedforward Neural Networks With Random Weights

G. B. Stanley - 2001

1 paper in library cites

[13]Identification of Nonlinear Dynamical Systems Using Support Vector Machines

J. Mcnames, J. A. K. Suykens, J. Vandewalle - 1999

1 paper in library cites

[14]Learning to Control in an Open-Loop System

David Zipser, Ronald J. Williams - 1989

1 paper in library cites

Missing author listMissing year

[15]Materials and Methods

1 paper in library cites

[16]Motor Learning and the Control of Prosthetic Devices

S. M. Ivaldi - 2000

1 paper in library cites

[17]Nonlinear Modeling: Advanced Black-Box Techniques

L. A. Feldkamp, D. V. Prokhorov, C. F. Eagen, F. Yuan, J. A. K. Suykens, J. Vandewalle - 1998

1 paper in library cites

[18]Oscillation and Chaos in a Feedback Control System With a Time Delay

M. C. Mackey, L. Glass - 1977

1 paper in library cites

[19]Pattern Recognition Using a Neural Network Trained With a Modified Backpropagation algorithm

G. B. Stanley, F. F. Li, Y. Dan - 1999

1 paper in library cites

[20]Prediction of Chaotic Time Series by Neural Networks

L. Chudy, I. Farkas - 1998

1 paper in library cites

[21]Proc. 15th Europ. Conf. On Art. Int. (ECAI 02)

J. Hertzberg, Herbert Jaeger, F. Schonherr, F. V. Harmelen - 2002

1 paper in library cites

[22]Proc. IEEE World Congr. On Computational Intelligence (IJCNN '98)

H. Bersini, M. Birattari, G. Bontempi - 1997

1 paper in library cites

[23]Proc. WSOM '97

J. Vesanto - 1997

1 paper in library cites

[24]Recurrent Neural Networks for Learning and Generating Motor Patterns

K. Doya, S. Yoshizawa - 1989

1 paper in library cites

[25]Short Term Memory in echo State Networks

Herbert Jaeger - 2002

1 paper in library cites

[26]Task-Related Encoding of Firing Rate in Prefrontal Cortex of Monkeys

D. G. Beiser, J. C. Houk - 1998

1 paper in library cites

[27]Temporal Processing in Feed-Forward Networks With Memory Traces

W. Maass, T. Natschlager, H. Markram - 2002

1 paper in library cites

Missing author listMissing year

[28]The First Author Thanks T. Christaller for Unfaltering Support and W. Maass for Friendly Cooperation. International Patents Are Claimed by Fraunhofer AIS

1 paper in library cites

[29]The Handbook of Brain Theory and Neural Networks

K. Doya, M. A. Arbib - 1995

1 paper in library cites

[30]The Organization of Motor Cortex: A New View of the Somatotopic Map

Stanislas Dehaene, J. P. Changeux, J. P. Nadal - 1987

1 paper in library cites

[31]Tutorial on Training Recurrent Neural Networks

Herbert Jaeger - 2002

1 paper in library cites

Cited by

4

papers in your library

Cites

2

papers in your library

Read

on January 30, 2026

Bad read. Doesn't explain anything of how things work. It's boring and I didn't like the figures. Maybe the main paper about ESNs is better, but frankly I don't want to read it.

Tags

Paper Aliases

No aliases