AI summary

This paper examines challenge problems, beginning with Turing's famous test. It contrasts the successful "divide and conquer" strategy with the promising but largely untested "developmental" strategy. The paper concludes that good challenge problems encourage the latter strategy, fostering a developmental research strategy in AI.

Abstract

If it is true that good problems produce good science, then it will be worthwhile to identify good problems, and even more worthwhile to discover the attributes that make them good problems. This discovery process is necessarily empirical, so we examine several challenge problems, beginning with Turing's famous test, and more than a dozen attributes that challenge problems might have. We are led to a contrast between research strategies the successful "divide and conquer" strategy and the promising but largely untested "developmental" strategy and we conclude that good challenge problems encourage the latter strategy.