AI summary

This paper explores how neural networks learn language, using a semi-realistic artificial language, and shows that networks are able to learn when they begin with limited working memory and gradually mature to the adult state, achieving better results than networks with full memory capacity.

Abstract

It is a striking fact that in humans the greatest learning occurs precisely at that point in time - childhood – when the most dramatic maturational changes also occur. This report describes possible synergistic interactions between maturational change and the ability to learn a complex domain (language), as investigated in connectionist networks. The networks are trained to process complex sentences involving relative clauses, number agreement, and several types of verb argument structure. Training fails in the case of networks which are fully formed and 'adultlike' in their capacity. Training succeeds only when networks begin with limited working memory and gradually 'mature' to the adult state. This result suggests that rather than being a limitation, developmental restrictions on resources may constitute a necessary prerequisite for mastering certain complex domains. Specifically, successful learning may depend on starting small.