AI summary

This paper introduces Recurrent Continuous Translation Models (RCTM) using continuous representations for words, phrases, and sentences, conditioned by a Convolutional Sentence Model, achieving over 43% lower perplexity than state-of-the-art alignment-based translation models and demonstrating sensitivity to word order and syntax.

Abstract

We introduce a class of probabilistic con tinuous translation models called Recurrent Continuous Translation Models that are purely based on continuous representations for words, phrases and sentences and do not rely on alignments or phrasal translation units. The models have a generation and a conditioning aspect. The generation of the transla tion is modelled with a target Recurrent Language Model, whereas the conditioning on the source sentence is modelled with a Convolu tional Sentence Model. Through various ex periments, we show first that our models ob tain a perplexity with respect to gold transla tions that is > 43% lower than that of state of-the-art alignment-based translation models. Secondly, we show that they are remarkably sensitive to the word order, syntax, and mean ing of the source sentence despite lacking alignments. Finally we show that they match a state-of-the-art system when rescoring n-best lists of translations.