scholarly journals Audio-Visual Speech Recognition Using Lip Information Extracted from Side-Face Images

2007 ◽  
Vol 2007 ◽  
pp. 1-9 ◽  
Author(s):  
Koji Iwano ◽  
Tomoaki Yoshinaga ◽  
Satoshi Tamura ◽  
Sadaoki Furui
Keyword(s):  
Speech Recognition ◽  
Visual Speech ◽  
Face Images ◽  
Visual Speech Recognition ◽  
Side Face

scholarly journals Multi-Angle Lipreading with Angle Classification-Based Feature Extraction and Its Application to Audio-Visual Speech Recognition

2021 ◽  
Vol 13 (7) ◽  
pp. 182
Author(s):  
Shinnosuke Isobe ◽  
Satoshi Tamura ◽  
Satoru Hayamizu ◽  
Yuuto Gotoh ◽  
Masaki Nose
Keyword(s):  
Feature Extraction ◽  
Speech Recognition ◽  
Automatic Speech Recognition ◽  
Decision Fusion ◽  
Visual Speech ◽  
Face Images ◽  
Visual Speech Recognition ◽  
Visual Database ◽  
Visual Results ◽  
Better Than

Recently, automatic speech recognition (ASR) and visual speech recognition (VSR) have been widely researched owing to development in deep learning. Most VSR research works focus only on frontal face images. However, assuming real scenes, it is obvious that a VSR system should correctly recognize spoken contents from not only frontal but also diagonal or profile faces. In this paper, we propose a novel VSR method that is applicable to faces taken at any angle. Firstly, view classification is carried out to estimate face angles. Based on the results, feature extraction is then conducted using the best combination of pre-trained feature extraction models. Next, lipreading is carried out using the features. We also developed audio-visual speech recognition (AVSR) using the VSR in addition to conventional ASR. Audio results were obtained from ASR, followed by incorporating audio and visual results in a decision fusion manner. We evaluated our methods using OuluVS2, a multi-angle audio-visual database. We then confirmed that our approach achieved the best performance among conventional VSR schemes in a phrase classification task. In addition, we found that our AVSR results are better than ASR and VSR results.

Asynchrony modeling for audio-visual speech recognition

2002 ◽  
Author(s):  
Guillaume Gravier ◽  
Gerasimos Potamianos ◽  
Chalapathy Neti
Keyword(s):  
Speech Recognition ◽  
Visual Speech ◽  
Visual Speech Recognition

scholarly journals Comparison between different feature extraction techniques for audio-visual speech recognition

2007 ◽  
Vol 1 (1) ◽  
pp. 7-20 ◽  
Author(s):  
Alin G. Chiţu ◽  
Leon J. M. Rothkrantz ◽  
Pascal Wiggers ◽  
Jacek C. Wojdel
Keyword(s):  
Feature Extraction ◽  
Speech Recognition ◽  
Visual Speech ◽  
Extraction Techniques ◽  
Visual Speech Recognition

European and American Audio-Visual Speech Recognition, Using SVM in Portuguese Language

2008 ◽  
Author(s):  
Adriano de Andrade Bresolin ◽  
Diamantino Rui da Silva da Silva Freitas ◽  
Adriao Duarte Doria Neto ◽  
Pablo Javier Alsina
Keyword(s):  
Speech Recognition ◽  
Visual Speech ◽  
Visual Speech Recognition
Sign in / Sign up

Export Citation Format

Share Document