AI summary

This paper demonstrates that Random Weight Guessing (RG) often outperforms more complex methods on long-term dependency benchmark problems for recurrent neural networks, suggesting that some benchmark solutions are dense in weight space and questioning the difficulty of these benchmarks.

Abstract

Numerous recent papers focus on standard recurrent nets' problems with tasks involving long-term dependencies. We solve such tasks by random weight guessing (RG). Although RG cannot be viewed as a reasonable learning algorithm we find that it often outperforms previous, more complex methods on widely used benchmark problems. One reason for RG's success is that the solutions to many of these benchmarks are dense in weight space. An analysis of cases in which RG works well versus those in which it does not can serve to improve the quality of benchmarks for novel recurrent net algorithms.