Robust speech recognition using missing feature theory and target speech enhancement based on degenerate unmixing and estimation technique

2011 ◽  
Author(s):  
Minook Kim ◽  
Ji-Seon Kim ◽  
Hyung-Min Park
Keyword(s):  
Speech Recognition ◽  
Speech Enhancement ◽  
Robust Speech Recognition ◽  
Estimation Technique ◽  
Missing Feature Theory ◽  
Missing Feature ◽  
Feature Theory

Psychologically-Inspired Audio-Visual Speech Recognition Using Coarse Speech Recognition and Missing Feature Theory

2017 ◽  
Vol 29 (1) ◽  
pp. 105-113 ◽  
Author(s):  
Kazuhiro Nakadai ◽  
◽  
Tomoaki Koiwa ◽  
Keyword(s):  
Speech Recognition ◽  
Real World ◽  
Visual Speech ◽  
Noise Robustness ◽  
Visual Integration ◽  
The Real ◽  
Visual Speech Recognition ◽  
Missing Feature Theory ◽  
Missing Feature ◽  
Feature Theory

[abstFig src='/00290001/10.jpg' width='300' text='System architecture of AVSR based on missing feature theory and P-V grouping' ] Audio-visual speech recognition (AVSR) is a promising approach to improving the noise robustness of speech recognition in the real world. For AVSR, the auditory and visual units are the phoneme and viseme, respectively. However, these are often misclassified in the real world because of noisy input. To solve this problem, we propose two psychologically-inspired approaches. One is audio-visual integration based on missing feature theory (MFT) to cope with missing or unreliable audio and visual features for recognition. The other is phoneme and viseme grouping based on coarse-to-fine recognition. Preliminary experiments show that these two approaches are effective for audio-visual speech recognition. Integration based on MFT with an appropriate weight improves the recognition performance by −5 dB. This is the case even in a noisy environment, in which most speech recognition systems do not work properly. Phoneme and viseme grouping further improved the AVSR performance, particularly at a low signal-to-noise ratio.**This work is an extension of our publication “Tomoaki Koiwa et al.: Coarse speech recognition by audio-visual integration based on missing feature theory, IROS 2007, pp.1751-1756, 2007.”

Sign in / Sign up

Export Citation Format

Share Document