Papperoni

2005

Framewise Phoneme Classification With Bidirectional LSTM Networks

Jürgen Schmidhuber

Open PDF Google Scholar

citations

Cite Score

34

AI summary

This paper introduces bidirectional LSTM networks for framewise phoneme classification on the TIMIT dataset, achieving high recognition scores. It demonstrates the architectural advantage of bidirectional training and LSTM over conventional RNNs, capturing time dependencies in speech.

Main Contributions

Introduces bidirectional LSTM networks for the first time.
Calculates the full error gradient for LSTM weights.
Achieves high framewise recognition score on the TIMIT database.
Demonstrates the architectural advantage of bidirectional training over unidrectional LSTM.
Demonstrates the architectural advantage of LSTM over conventional RNNs.

Abstract

In this paper, we apply bidirectional training to a Long Short Term Memory (LSTM) network for the first time. We also present a modified, full gradient version of the LSTM learning algorithm. On the TIMIT speech database, we measure the framewise phoneme classification ability of bidirectional and unidirectional variants of both LSTM and conventional Recurrent Neural Networks (RNNs). We find that the LSTM architecture outperforms conventional RNNs and that bidirectional networks outperform unidirectional ones.

Citation Graph

Loading graph...

References [23]

Sort:

Filter:

[1]Long Short-Term Memory

Sepp Hochreiter, Jürgen Schmidhuber - 1997

94 papers in library cite

LSTMs FTW!

[2]Bidirectional Recurrent Neural Networks

M. Schuster, Kuldip K. Paliwal - 1997

10 papers in library cite

Introduced the BRNN concept. Good paper.

[3]Gradient Flow in Recurrent Nets: The Difficulty of Learning Long-Term Dependencies

Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber - 2001

16 papers in library cite

Wow, this is so much better than the other paper - I should have read it sooner. It's concise and not too abstract, and also gives very good context on RNN problems and how to solve them.

[4]An Application to Recurrent Nets to Phone Probability Estimation

A. Robinson - 1994

9 papers in library cite

Very early work. Seems like a good overview of how to apply NNs to speech recog, but TBH there's nothing impressive here.

[5]Neural Networks for Pattern Recognition

C. M. Bishop - 1995

12 papers in library cite

Book, 38k citations

[6]The Utility Driven Dynamic Error Propagation Network

A. J. Robinson, F. Fallside - 1987

10 papers in library cite

[7]Learning Precise Timing With LSTM Recurrent Networks

F. Gers, N. Schraudolph, Jürgen Schmidhuber - 2002

9 papers in library cite

[8]Connectionist Speech Recognition: A Hybrid Approach

H. Bourlard, N. Morgan - 1993

8 papers in library cite

[9]The DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus (TIMIT)

D. Isto - 1990

5 papers in library cite

[10]Fast Curvature Matrix-Vector Products for Second-Order Gradient Descent

N. N. Schraudolph - 2002

4 papers in library cite

[11]Gradient-Based Learning Algorithms for Recurrent Connectionist Networks

Ronald J. Williams, David Zipser - 1990

2 papers in library cite

[12]Kalman Filters Improve LSTM Network Performance in Problems Un- Solvable by Traditional Recurrent Nets

J. A. P. Ortiz, F. Gers, D. Eck, Jürgen Schmidhuber - 2002

2 papers in library cite

[13]A Comparison Between Spiking and Differentiable Recurrent Neural Networks on Spoken Digit Recognition

Alex Graves, N. Beringer, Jürgen Schmidhuber - 2004

1 paper in library cites

[14]A New Approach to Continuous Speech Recognition Using LSTM Recurrent Neural Networks

D. Eck, Alex Graves, Jürgen Schmidhuber - 2003

1 paper in library cites

[15]Bidirectional Dynamics for Protein Secondary Structure Prediction

P. Baldi, S. Brunak, Paolo Frasconi, G. Pollastri, G. Soda - 2001

1 paper in library cites

[16]Biologically Plausible Speech Recognition With Istm Neural Nets

Alex Graves, D. Eck, N. Beringer, Jürgen Schmidhuber - 2004

1 paper in library cites

[17]Capturing Long-Term Dependencies for Protein Secondary Structure Prediction

Jixuan Chen, N. S. Chaudhari - 2004

1 paper in library cites

[18]Experiments on the Implementation of Recurrent Neural Networks for Speech Phone Recognition

R. Chen, L. Jamieson - 1996

1 paper in library cites

[19]Exploiting the Past and the Future in Protein Secondary Structure Prediction

P. Baldi, S. Brunak, Paolo Frasconi, G. Soda, G. Pollastri - 1999

1 paper in library cites

[20]Human Language Acquisition Methods in a Machine Learning Task

N. Beringer - 2004

1 paper in library cites

[21]On Supervised Learning From Sequential Data With Applica- Tions for Speech Recognition

M. Schuster - 1999

1 paper in library cites

[22]Phoneme Boundary Estimation Using Bidirectional Recurrent Neural Networks and Its Applications

T. Fukada, M. Schuster, Y. Sagisaka - 1999

1 paper in library cites

[23]Rprop - A Fast Adaptive learning Algorithm

M. Riedmiller, H. Braun - 1992

1 paper in library cites

Cited by

14

papers in your library

Cites

5

papers in your library

Read

on June 30, 2025

Very nice paper! Simple, no bullshit. Just "hey, we have LSTM and we have BRNN, let's try to join it"

Tags

Paper Aliases

Framewise Phoneme Classification With Bidirectional LSTM and Other Neural Network Architectures