First European Conference on Speech Communication and Technology

Paris, France
September 27-29, 1989

Fast Back-Propagation Learning Methods for Large Phonemic Neural Networks

P. Haffner (3), Alex Waibel (2), H. Sawai (1), Kiyohiro Shikano (1)

(1) ATR Interpreting Telephony Research Laboratories, Seika-cho, Soraku-gun, Kyoto, Japan
(2) Centre National d'Etudes des Telecommunications, Lannion, France
(3) Carnegie Mellon University, Computer Science Dept, Pittsburgh, PA, USA

Several improvements in the Back-Propagation procedure are proposed to increase training speed, and we discuss their limitations with respect to generalization performance. The error surface is modeled to avoid local minima and flat areas. The synaptic weights are updated as often as possible. Both the step size and the momentum are dynamically scaled to the largest possible values that do not result in overshooting. Training for the speaker-dependent recognition of the phonemes /b/, /d/ and /g/ has been reduced from 2 days to 1 minute on an Alliant parallel computer, delivering the same 98.6% recognition performance. With a 55000-connection TDNN, the same algorithm needs 1 hour and 5000 training tokens to recognize the 18 Japanese consonants with 96.7% correct.

Full Paper

Bibliographic reference.  Haffner, P. / Waibel, Alex / Sawai, H. / Shikano, Kiyohiro (1989): "Fast back-propagation learning methods for large phonemic neural networks", In EUROSPEECH-1989, 2553-2556.