Papperoni

1986

Learning Distributed Representations of Concepts

Geoffrey E. Hinton

citations

Cite Score

AI summary

This paper introduces a neural network that learns distributed representations of concepts using the back-propagation algorithm on family relationship data. The network captures similarities between concepts, generalizes new information, and utilizes isomorphisms between family trees.

Main Contributions

Demonstrates that neural networks can learn distributed representations of concepts.
Applies the back-propagation algorithm to train a network on family relationship data.
Shows the network can capture similarities between concepts and generalize new information.
Illustrates how the network utilizes isomorphisms between different family trees to share structure.
Introduces a method for the network to develop its own role-specific representations.

Abstract

Concepts can be represented by distributed patterns of activity in networks of neuron-like units. One advantage of this kind of representation is that it leads to automatic generalization. When the weights in the network are changed to incorporate new knowledge about one concept, the changes affect the knowledge associated with other concepts that are represented by similar activity patterns. There have been numerous demonstrations of sensible generalization which have depended on the experimenter choosing appropriately similar patterns for different concepts. This paper shows how the network can be made to choose the pattems itself when shown a set of propositions that use the concepts. It chooses patterns which make explicit the underlying features that are only implicit in the propositions it is shown.

Citation Graph

Loading graph...

References [12]

Sort:

Filter:

[1]Learning Internal Representations by Error Propagation

D. E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams - 1986

46 papers in library cite

Google Scholar

I expected very little of this, but was so good in explaining concepts! Very good read. It gets a bit boring when it starts explaining things by the end of the chapter, but good nonetheless.

[2]Neural Networks and Physical Systems With Emergent Collective Computational Abilities

J. J. Hopfield - 1982

8 papers in library cite

Google Scholar

Very theory heavy and doesn't seem to add much... Famous because it introduced the "Hopfield network" - "At a time when research in neural networks was facing a lull, Hopfield's paper injected a fresh perspective by demonstrating how a network of simple, interconnected units could exhibit complex computational abilities as a collective, emergent property."

[3]Distributed Representations

Geoffrey E. Hinton, J. L. Mcclelland, D. E. Rumelhart - 1986

5 papers in library cite

Google Scholar

Very relevant but also very abstract. Not really an awesome read, but interesting nonetheless.

[4]Experiments on Learning by Back-Propagation

D. C. Plaut, S. J. Nowlan, Geoffrey E. Hinton - 1986

5 papers in library cite

Google Scholar

45 Pages. Seems alright, but not sure

[5]An Interactive Activation Model of Context Effects in Letter Perception

J. Mcclelland - 1981

3 papers in library cite

Google Scholar

[6]Associative Memory: A System-Theorectical Approach

T. Kohonen - 1977

3 papers in library cite

Google Scholar

[7]Holography, Associative Memory, and Inductive Generalization

D. Willshaw - 1981

2 papers in library cite

Google Scholar

[8]Implementing Semantic Networks in Parallel Hardware

Geoffrey Hinton - 1981

2 papers in library cite

Google Scholar

[9]Single Units and Sensation: A Neuron Doctrine for Perceptual Psychology

H. B. Barlow - 1972

2 papers in library cite

Google Scholar

[10]Vision

D. Marr - 1982

2 papers in library cite

Google Scholar

[11]Massively Parallel Parsing: A Strongly Interactive Model of Natural Language Interpretation

D. Waltz, J. Pollack - 1985

1 paper in library cites

Google Scholar

[12]Nonholographic Associative Memory

D. Willshaw, O. Buneman, H. L. Higgins - 1969

1 paper in library cites

Google Scholar

Cited by

papers in your library

Cites

papers in your library

Read

on March 20, 2025

 Probably seminal, but a bit boring overall