To improve the accuracy of visual speech recognition systems, selection of visual features is of fundamental importance. Prominent features, which are of maximum relevance for speech classification, need to be selected from a large set of extracted visual attributes. Existing methods apply feature reduction and selection techniques on image pixels constituting region-of-interest (ROI) to reduce data dimensionality. We propose application of feature selection methods on geometrical features to select the most dominant physical features. Two techniques, Minimum Redundancy Maximum Relevance (mRMR) and Correlation-based Feature Selection (CFS), have been applied on the extracted visual features. Experimental results show that recognition accuracy is not compromised when a few selected features from the complete visual feature set are used for classification, thereby reducing processing time and storage overheads considerably. Results are compared with performance of principal components obtained by application of Principal Component Analysis (PCA) on our dataset. Our set of selected features outperforms the PCA transformed data. Results show that the center and corner segments of the mouth are major contributors to visual speech recognition. Teeth pixels are shown to be a prominent visual cue. It is also seen that lip width contributes more towards visual speech recognition accuracy as compared to lip height.
The Performance and Classifications of Audio-Visual Speech Recognition by Using the Dynamic Visual Features Extractions
