Papperoni

1994

An Application to Recurrent Nets to Phone Probability Estimation

A. Robinson

Open PDF Google Scholar

citations

Cite Score

29

AI summary

This paper introduces recurrent networks for phone probability estimation in speech recognition, utilizing the DARPA TIMIT and Resource Management tasks. It demonstrates that recurrent nets are competitive with traditional methods for phone probability estimation and shows that this method can incorporate acoustic context.

Main Contributions

Application of recurrent networks for phone probability estimation
Integration of recurrent networks with Markov models
Demonstration of competitive results on DARPA TIMIT and Resource Management tasks
Exploration of different training algorithms
Discussion of acoustic feature selection for recurrent networks

Abstract

This paper presents an application of recurrent networks for phone probability estimation in large vocabulary speech recognition. The need for efficient exploitation of context information is discussed; a role for which the recurrent net appears suitable. An overview of early developments of recurrent nets for phone recognition is given along with the more recent improvements which include their integration with Markov models. Recognition results are presented for the DARPA TIMIT and Resource Management tasks, and it is concluded that recurrent nets are competitive with traditional means for performing phone probability estimation.

Citation Graph

Loading graph...

References [54]

Sort:

Filter:

[1]Learning Internal Representations by Error Propagation

D. E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams - 1986

46 papers in library cite

I expected very little of this, but was so good in explaining concepts! Very good read. It gets a bit boring when it starts explaining things by the end of the chapter, but good nonetheless.

[2]Finding Structure in Time

Jeffrey L. Elman - 1990

23 papers in library cite

Good paper overall that introduces the concept of an RNN. However, applications and results are still very primitive.

[3]Backpropagation Through Time: What It Does and How to Do It

P. Werbos - 1990

9 papers in library cite

Amazing tutorial! Very pragmatic. Explains very basic concepts and focus on implementation

[4]Phoneme Recognition Using Time-Delay Neural Networks

A. H. Waibel, T. Hanazawa, Geoffrey Hinton, K. Shikano, K. Lang - 1989

13 papers in library cite

I didn't really like this paper, although it's a good early example of using backprop. I think it can be thought as a 1D convolution.

[5]Increased Rates of Convergence Through Learning Rate Adaptation

Robert A. Jacobs - 1988

4 papers in library cite

Very early work on learning rate adaptation. I like that it describes the concepts very thoroughly (as they are all new).

[6]Connectionist Learning Procedures

Geoffrey E. Hinton - 1987

11 papers in library cite

It's a very good overview of everything that was happening in 1987! A bit too long though, but a good start nonetheless.

[7]Probabilistic Interpretation of Feedforward Classification Network Outputs, With Relationship to Statistical Pattern Recognition

John S. Bridle - 1990

12 papers in library cite

Explains using softmax as output. Nice read, but I am not sure if too relevant.

[8]An Application to Recurrent Nets to Phone Probability Estimation

A. Robinson - 1994

9 papers in library cite

Very early work. Seems like a good overview of how to apply NNs to speech recog, but TBH there's nothing impressive here.

[9]A Learning algorithm for Continually Running Fully Recurrent Neural Networks

R. Williams, David Zipser - 1989

8 papers in library cite

Quick read and very simple concept. I wish they explained things using a more visual approach instead of maths, but it's ok to follow nonetheless.

[10]Optimization of the Backpropagation algorithm for Training Multilayer Perceptrons

W. Schiffmann, M. Joost, R. Werner - 1992

1 paper in library cites

[I couldn't find the ref] 30 pages; Seems like early work on adaptive SGD

[11]Connectionist Speech Recognition: A Hybrid Approach

H. Bourlard, N. Morgan - 1993

8 papers in library cite

[12]Interpolated Estimation of Markov Source Parameters From Sparse Data

Frederick Jelinek, R. L. Mercer - 1980

8 papers in library cite

[13]Learning Complex, Extended Sequences Using the Principle of History Compression

Jürgen Schmidhuber - 1992

8 papers in library cite

[14]Speaker-Independent Phone Recognition Using Hidden Markov Models

K. F. Lee, H. W. Hon - 1989

5 papers in library cite

[15]A Maximum Likelihood Approach to Continuous Speech Recognition

L. R. Bahl, Frederick Jelinek, R. L. Mercer - 1983

4 papers in library cite

[16]A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition

L. R. Rabiner - 1989

4 papers in library cite

[17]A First Look at Phonetic Discrimination Using a Connectionist Architecture With Recurrent Links

G. Kuhn - 1987

3 papers in library cite

[18]Continuous Speech Recognition Using Multilayer Perceptrons With Hidden Markov Models

N. Morgan, H. Bourlard - 1990

3 papers in library cite

[19]Several Improvements to a Recurrent Error Propagation Network Phone Recognition System

Tony Robinson - 1991

3 papers in library cite

[20]A Probabilistic Approach to the Understanding and Training of Neural Network Classifiers

H. Gish - 1990

2 papers in library cite

[21]CDNN: A Context-Dependent Neural Network for Continuous Speech Recognition

H. Bourlard, N. Morgan, C. Wooters, S. Renals - 1992

2 papers in library cite

[22]Combining Hidden Markov Models and Neural Network Classifiers

L. T. Niles, H. F. Silverman - 1990

2 papers in library cite

[23]Connectionist Probability Estimators in HMM Speech Recognition

S. Renals, N. Morgan, H. Boulard, Michael Cohen, H. Franco - 1994

2 papers in library cite

[24]Connectionist Probability Estimators in HMM Speech Recognition

S. Renals, N. Morgan, H. Boulard, Michael Cohen, H. Franco - 1994

2 papers in library cite

[25]Gradient-Based Learning Algorithms for Recurrent Connectionist Networks

Ronald J. Williams, David Zipser - 1990

2 papers in library cite

[26]Links Between Markov Models and Multilayer Perceptrons

H. Bourlard, C. J. Wellekens - 1990

2 papers in library cite

[27]MMI Training for Continuous Phoneme Recognition on the TIMIT Database

S. Kapadia, V. Valtchev, S. Young - 1993

2 papers in library cite

[28]Neural Network Classifiers Estimate Bayesian a Posteriori Probabilities

M. D. Richard, R. P. Lippmann - 1991

2 papers in library cite

[29]Supervised Learning of Probability Distributions by Neural Networks

E. B. Baum, F. Wilczek - 1988

2 papers in library cite

[30]A Comparison of Preprocessors for the Cambridge Recurrent Error Propagation Network Speech Recognition System

Tony Robinson, J. Holdsworth, R. Patterson, F. Fallside - 1990

1 paper in library cites

[31]A Course in Phonetics

P. Ladefoged - 1982

1 paper in library cites

[32]A Dynamic Connectionist Model for Phoneme Recognition

A. J. Robinson, F. Fallside - 1989

1 paper in library cites

[33]A Recurrent Error Propagation Network Speech Recognition System

Tony Robinson, F. Fallside - 1991

1 paper in library cites

[34]ALPHA-NETS: A Recurrent `Neural' Network Architecture With a Hidden Markov Model interpretation

John S. Bridle - 1990

1 paper in library cites

[35]An Alphanet Approach to Optimising Input Transformations for Continuous Speech Recognition

John S. Bridle, L. Dodd - 1991

1 paper in library cites

[36]Automatic Speech Recognition: The Development of the SPHINX System

K. Lee - 1989

1 paper in library cites

[37]Competitive Training in Hidden Markov Models

S. J. Young - 1990

1 paper in library cites

[38]Connectionist Probability Estimation in the Decipher Speech Recognition System

S. Renals, N. Morgan, Michael Cohen, H. Franco - 1992

1 paper in library cites

[39]Context-Dependent Multiple Distribution Phonetic Modelling With MLPs

Michael Cohen, H. Franco, N. Morgan, D. Rumelhart, V. Abrash - 1993

1 paper in library cites

[40]Dynamic Recurrent Neural Networks

B. A. Pearlmutter - 1990

1 paper in library cites

[41]Fast Algorithms for Phone Classification and Recognition Using Segment-Based Models

V. V. Digalakis, M. Ostendorf, J. R. Rohlicek - 1992

1 paper in library cites

[42]Hidden Markov Models for Speech Recognition

X. D. Huang, Y. Ariki, M. A. Jack - 1990

1 paper in library cites

[43]Linear Discriminant Analysis for Improved Large Vocabulary Speech Recognition

R. H. Umbach, Hermann Ney - 1992

1 paper in library cites

[44]New Developments in Phone Recognition Using an Ergodic Hidden Markov Model

A. Ljolje - 1991

1 paper in library cites

[45]Phonological Structures for Speech Recognition

M. H. Cohen - 1989

1 paper in library cites

[46]Practical Network Design and Implementation

Tony Robinson - 1992

1 paper in library cites

[47]Soft Weight-Sharing

S. J. Nowlan, Geoffrey E. Hinton - 1991

1 paper in library cites

[48]Speaker-Independent Isolated Word Recognition Using Dynamic Features of Speech Spectrum

S. Furui - 1986

1 paper in library cites

[49]Speech Database Development: Design and Analysis of the Acoustic-Phonetic Corpus

L. F. Lamel, R. H. Kasel, S. Seneff - 1987

1 paper in library cites

[50]Static and Dynamic Error Propagation Networks With Application to Speech Coding

A. J. Robinson, F. Fallside - 1988

1 paper in library cites

[51]The DARPA 1000-Word Resource Management Database for Continuous Speech Recognition

P. Price, W. M. Fisher, J. Bernstein, D. S. Pallett - 1988

1 paper in library cites

[52]The HTK Tied-State Continuous Speech Recogniser

P. C. Woodland, S. J. Young - 1993

1 paper in library cites

[53]The State Space and “Ideal Input" Representations of Recurrent Networks

Tony Robinson - 1992

1 paper in library cites

[54]The Use of State Tying in Continous Speech Recognition

S. J. Young, P. C. Woodland - 1993

1 paper in library cites

Cited by

9

papers in your library

Cites

9

papers in your library

Read

on August 7, 2025

Very early work. Seems like a good overview of how to apply NNs to speech recog, but TBH there's nothing impressive here.

Tags

Paper Aliases

No aliases