AI summary

This paper introduces BLEU, a method for automatic machine translation evaluation that is quick, inexpensive, and language-independent. BLEU correlates highly with human evaluation and has little marginal cost per run, serving as an automated understudy to skilled human judges.

Abstract

Human evaluations of machine translation are extensive but expensive. Human evaluations can take months to finish and involve human labor that can not be reused. We propose a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run. We present this method as an automated understudy to skilled human judges which substitutes for them when there is need for quick or frequent evaluations.