2000

Can Artificial Neural Networks Learn Language Models

Weixin Xu, Alex Rudnicky

citations

Cite Score

9

AI summary

This paper introduces a language model using artificial neural networks (NN) and compares it with standard statistical methods. Using data collected from the Communicator Telephone Air Travel Information System, the neural network achieves comparable performance to Katz and Jelinek-Mercer smoothing techniques.

Main Contributions

  • Investigated the use of neural networks for language modeling, an area with limited prior research.
  • Demonstrated that neural networks can achieve performance comparable to standard statistical methods for language modeling.
  • Showed that a neural network model can perform even better than standard statistical methods without explicit smoothing techniques.
  • Analyzed the behavior of a simplified linear output network to understand how early-stopping contributes to generalization ability.
  • Explored the computational cost associated with training neural networks for language modeling and proposed a method to reduce it.

Abstract

Currently, N-gram models are the most common and widely used models for statistical language modeling. In this paper, we investigated an alternative way to build language models, i.e., using artificial neural networks to learn the language model. Our experiment result shows that the neural network can learn a language model that has performance even better than standard statistical methods.

Citation Graph

Loading graph...

References [7]

Sort:
Filter:

R. Kneser, Hermann Ney - 1995

11 papers in library cite

S. F. Chen, J. Goodman - 1998

13 papers in library cite

A. P. Dempster, N. M. Laird, D. B. Rubin - 1977

6 papers in library cite

S. Lawrence, C. L. Giles, S. Fong - 1996

1 paper in library cites

Alex Rudnicky, E. Thayer, P. Constantinides, C. Tchou, R. Shern, K. Lenzo, Weixin Xu, A. Oh - 1999

1 paper in library cites

Frederick Jelinek - 1994

1 paper in library cites

Cited by

5

papers in your library

Cites

2

papers in your library

Read

on March 21, 2025

Your review

Tags

Paper Aliases

No aliases