Papperoni

2017

Overcoming Catastrophic Forgetting in Neural Networks

J. Kirkpatrick, Razvan Pascanu, N. C. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. G. Barwinska, Demis Hassabis, C. Clopath, D. Kumaran, Raia Hadsell

Open PDF Google Scholar

citations

Cite Score

85

AI summary

This paper introduces elastic weight consolidation (EWC), a novel algorithm which selectively slows down learning on important weights to overcome catastrophic forgetting. EWC achieves state-of-the-art results on MNIST classification and Atari 2600 games.

Main Contributions

Introduces elastic weight consolidation (EWC), a novel algorithm that mitigates catastrophic forgetting in neural networks by selectively slowing down learning on the weights.
EWC leverages a Bayesian perspective, tempering new learning with a prior based on the posterior distribution of parameters from previous tasks.
Demonstrates that EWC enables continual learning in deep neural networks without significant performance degradation on previously learned tasks.
Applies EWC to supervised learning tasks on permuted MNIST dataset, achieving state-of-the-art results.
Extends EWC to reinforcement learning scenarios, showcasing its effectiveness in the challenging Atari 2600 domain.

Abstract

The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Neural networks are not, in general, capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks which they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on the MNIST hand written digit dataset and by learning several Atari 2600 games sequentially.

Citation Graph

Loading graph...

References [42]

Sort:

Filter:

[1]ImageNet Classification With Deep Convolutional Neural Networks

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton - 2012

71 papers in library cite

I'm giving this a 5 just because of the impact, but this is VEEERY derivative of earlier work. Kudos for them for putting it all together, but really there's nothing revolutionary here.

[2]Deep Learning

I. Goodfellow, Yoshua Bengio, Y. A. Courville, A. Aaron - 2016

5 papers in library cite

It's an awesome review for people that are not familiar with deep learning, but it's very basic as well. Not too relevant given the context of other papers I read, but huge impact in terms of scientific communication.

[3]Human-Level Control Through Deep Reinforcement Learning

V. Mnih - 2015

9 papers in library cite

Maybe the first instance of RL with NNs! I am impressed that it seems quite straightforward to implement (maybe tricky to get working).

[4]Catastrophic Forgetting in Connectionist Networks

Robert M. French - 1999

2 papers in library cite

Such a good review/survey on catastrophic forgetting! Very nice

[5]The MNIST Database of Handwritten Digits

Yann Lecun - 1998

8 papers in library cite

Not a paper - it's actually a dataset

[6]Deep Reinforcement Learning With Double Q-Learning

H. V. Hasselt, A. Guez, D. Silver - 2016

2 papers in library cite

Sounds important to RL

[7]Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem

M. Mccloskey, N. J. Cohen - 1989

4 papers in library cite

50+ pages, but the first to notice catastropgic forgetting

[8]Weight Uncertainty in Neural Network

C. Blundell, J. Cornebise, Koray Kavukcuoglu, Daan Wierstra - 2015

3 papers in library cite

Introduced Bayesian NNs. I am not tooo excited about it but sounds relevant

[9]Progressive Neural Networks

A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, Raia Hadsell - 2016

1 paper in library cites

Play attari games

[10]An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks

Ian J. Goodfellow, M. Mirza, D. Xiao, Aaron Courville, Yoshua Bengio - 2014

2 papers in library cite

Meta examination of catastrophic fogetting - nice!

[11]Connectionist Models of Recognition Memory: Constraints Imposed by Learning and Forgetting Functions

R. M. Ratcliff - 1990

3 papers in library cite

24 pages double column. One of the first to notice catastrophic forgetting.

[12]Policy Distillation

A. A. Rusu, S. G. Colmenarejo, C. G. Gulcehre, G. Desjardins, J. Kirkpatrick, Razvan Pascanu, V. Mnih, Koray Kavukcuoglu, Raia Hadsell - 2015

1 paper in library cites

Sounds interesting - related to RL

[13]Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning

E. Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov - 2015

2 papers in library cite

Multitask learning

[14]Compete to Compute

R. K. Srivastava, Jonathan Masci, S. Kazerounian, Faustino Gomez, Jürgen Schmidhuber - 2013

3 papers in library cite

I like the name + schmidhuber

[15]The Arcade Learning Environment: An Evaluation Platform for General Agents

M. G. Bellemare, Y. Naddaf, J. Veness, M. Bowling - 2013

5 papers in library cite

[16]Universal Intelligence: A Definition of Machine Intelligence

Shane Legg, M. Hutter - 2007

4 papers in library cite

[17]Revisiting Natural Gradient for Deep Networks

Razvan Pascanu, Yoshua Bengio - 2013

3 papers in library cite

[18]Why There Are Complementary Learning Systems in the hippocampus and Neocortex: Insights From the Successes and Failures of Connectionist Models of Learning and Memory

J. L. Mcclelland, B. L. Mcnaughton, R. C. O'reilly - 1995

3 papers in library cite

[19]A Practical Bayesian Framework for Backpropagation Networks

D. J. Mackay - 1992

2 papers in library cite

[20]Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia

R. C. O'reilly, M. J. Frank - 2005

2 papers in library cite

[21]Theory of the Backpropagation neural Network

R. H. Nielsen - 1989

2 papers in library cite

[22]An Integrative Theory of Prefrontal Cortex Function

E. K. Miller, J. D. Cohen - 2001

1 paper in library cites

[23]Branch-Specific Dendritic Ca2+ Spikes Cause Persistent Synaptic Plasticity

J. Cichon, W. B. Gan - 2015

1 paper in library cites

[24]Cascade Models of Synaptically Stored Memories

S. Fusi, P. J. Drew, L. Abbott - 2005

1 paper in library cites

[25]Child: A First Step Towards Continual Learning

M. B. Ring - 1998

1 paper in library cites

[26]Cognitive Control Over Learning: Creating, Clustering, and Generalizing Task-Set Structure

A. G. Collins, M. J. Frank - 2013

1 paper in library cites

[27]Computational Principles of Synaptic Memory Consolidation

M. K. Benna, S. Fusi - 2016

1 paper in library cites

[28]Context Tree Switching

J. Veness, K. S. Ng, M. Hutter, M. Bowling - 2012

1 paper in library cites

[29]Context-Dependent Computation by Recurrent Dynamics in Prefrontal Cortex

V. Mante, D. Sussillo, K. V. Shenoy, W. T. Newsome - 2013

1 paper in library cites

[30]Ella: An Efficient Lifelong Learning algorithm

E. Eaton, P. L. Ruvolo - 2013

1 paper in library cites

[31]Labelling and Optical Erasure of Synaptic Memory Traces in the Motor Cortex

A. H. Takagi, S. Yagishita, M. Nakamura, F. Shirai, Y. I. Wu, A. L. Loshbaugh, B. Kuhlman, K. M. Hahn, H. Kasai - 2015

1 paper in library cites

[32]Laplace Propagation

E. Eskin, A. J. Smola, S. V. N. Vishwanathan - 2004

1 paper in library cites

[33]Multiple Model-Based Reinforcement Learning

K. Doya, K. Samejima, K. I. Katagiri, M. Kawato - 2002

1 paper in library cites

[34]Sleep Promotes Branch-Specific Formation of Dendritic Spines After Learning

G. Yang, C. S. W. Lai, J. Cichon, L. Ma, Wentao Li, W. B. Gan - 2014

1 paper in library cites

[35]Stably Maintained Dendritic Spines Are Associated With Lifelong Memories

G. Yang, F. Pan, W. B. Gan - 2009

1 paper in library cites

[36]Synapses With Short-Term Plasticity Are Optimal Estimators of Presynaptic Membrane Potentials

J. P. Pfister, Peter Dayan, M. Lengyel - 2010

1 paper in library cites

[37]Synaptic Sampling: A Connection Between PSP Variability and Uncertainty Explains Neurophysiological Observations

L. Aitchison, P. E. Latham - 2015

1 paper in library cites

[38]Tag-Trigger-Consolidation: A Model of Early and Late Long-Term-Potentiation and Depression

C. Clopath, L. Ziegler, E. Vasilaki, L. Busing, W. Gerstner - 2008

1 paper in library cites

[39]The Forget Me Not Process

M. Kieran, J. Veness, M. Bowling, J. Kirkpatrick, A. Koop, Demis Hassabis - 2016

1 paper in library cites

[40]The Fréchet Distance Between Multivariate Normal Distributions

D. Dowson, B. Landau - 1982

1 paper in library cites

[41]Uniqueness of the Weights for Minimal Feedforward Nets With a Given Input-Output Map

H. J. Sussmann - 1992

1 paper in library cites

[42]Using Noise to Compute Error Surfaces in Connectionist Networks: A Novel Means of Reducing Catastrophic Forgetting

Robert M. French, N. Chater - 2002

1 paper in library cites

Cited by

5

papers in your library

Cites

14

papers in your library

Read

on November 2, 2025

Very nice, intuitive, and impressive results.

Tags

Paper Aliases

No aliases