Visual speech recognition for small scale dataset using VGG16 convolution neural network

2021 ◽  
Author(s):  
Shashidhar R ◽  
Sudarshan Patilkulkarni
Keyword(s):  
Neural Network ◽  
Speech Recognition ◽  
Convolution Neural Network ◽  
Visual Speech ◽  
Small Scale ◽  
Visual Speech Recognition

Convolution Neural Network Based Visual Speech Recognition System for Syllable Identification

2020 ◽  
Vol 13 ◽  
Author(s):  
Hunny Pahuja ◽  
Priya Ranjan ◽  
Amit Ujlayan ◽  
Ayush Goyal
Keyword(s):  
Neural Network ◽  
Speech Recognition ◽  
Real Time ◽  
Acoustic Property ◽  
Recognition System ◽  
Convolution Neural Network ◽  
Visual Speech ◽  
Large Set ◽  
Speech Recognition System ◽  
Visual Speech Recognition

Introduction: This paper introduces novel and reliable approach for speech impaired people to assist them to communicate effectively in real time. A deep learning technique named as convolution neural network is used as its classifier. With the help of this algorithm, words are recognized from an input which is a visual speech, disregards with its audible or acoustic property. Methods: This network extracts the features from mouth stances and different images respectively. With the help of a source, non-audible mouth stances are taken as an input and then segregated as subsets to get desired output. The Complete Datum is then arranged to recognize the word as an affricate. Results: Convolution neural network is one of the most effective algorithms that extracts features, performs classification and provides the desired output from the input images for speech recognition system. Conclusion: Recognizing the syllables at real time from visual mouth stances input is the main objective of the proposed method. When tested, datum accuracy and quantity of training sets is giving satisfactory output. A small set of datum is taken as first step of learning. In future, large set of datum can be considered for analyzing the data. Discussion: On the basis of type of Datum, network proposed in this paper is tested to obtain its precision level. A network is maintained to identify the syllables but it fails when syllables are of same set. Requirement of Higher end graphics pro-cessing units is there to bring down the time consumption and increases the efficiency of network.

scholarly journals End-to-end visual speech recognition for small-scale datasets

2020 ◽  
Vol 131 ◽  
pp. 421-427 ◽  
Author(s):  
Stavros Petridis ◽  
Yujiang Wang ◽  
Pingchuan Ma ◽  
Zuwei Li ◽  
Maja Pantic
Keyword(s):  
Speech Recognition ◽  
Visual Speech ◽  
Small Scale ◽  
Visual Speech Recognition ◽  
End To End

scholarly journals Integration of Audio video Speech Recognition using LSTM and Feed Forward Convolutional Neural Network

2021 ◽  
Author(s):  
Shashidhar R ◽  
Sudarshan Patil Kulkarni
Keyword(s):  
Neural Network ◽  
Speech Recognition ◽  
Good Accuracy ◽  
Visual Speech ◽  
Feed Forward Neural Network ◽  
Final Model ◽  
Feed Forward ◽  
Speech Detection ◽  
Visual Speech Recognition ◽  
Lip Reading

Abstract In the current scenario, audio visual speech recognition is one of the emerging fields of research, but there is still deficiency of appropriate visual features for recognition of visual speech. Human lip-readers are increasingly being presented as useful in the gathering of forensic evidence but, like all human, suffer from unreliability in analyzing the lip movement. Here we used a custom dataset and design the system in such a way that it predicts the output for the lip reading. The problem of speaker independent lip-reading is very demanding due to unpredictable variations between people. Also due to recent developments and advances in the fields of signal processing and computer vision. The task of automating the lip reading is becoming a field of great interest. Here we use MFCC techniques for audio processing and LSTM method for visual speech recognition and finally integrate the audio and video using feed forward neural network (FFNN) and also got good accuracy. That is why the AVSR technique attract a great attention as a reliable solution for the speech detection problem. The final model was capable of taking more appropriate decision while predicting the spoken word. We were able to get a good accuracy of about 92.38% for the final model.

scholarly journals An Critical Analysis of Speech Recognition of Tamil and Malay Language Through Artificial Neural Network

2021 ◽  
Author(s):  
Kingston Pal Thamburaj ◽  
Kartheges Ponniah ◽  
Ilangkumaran Sivanathan ◽  
Muniisvaran Kumar
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Speech Recognition ◽  
Data Entry ◽  
Recognition System ◽  
Visual Speech ◽  
Document Preparation ◽  
Multiple Sensors ◽  
Visual Speech Recognition ◽  
Artificial Neural

Human and Computer interaction has been a part of our day-to-day life. Speech is one of the essential and comfortable ways of interacting through devices as well as a human being. The device, particularly smartphones have multiple sensors in camera and microphone, etc. speech recognition is the process of converting the acoustic signal to a smartphone as a set of words. The efficient performance of the speech recognition system highly enhances the interaction between humans and machines by making the latter more receptive to user needs. The recognized words can be applied for many applications such as Commands & Control, Data entry, and Document preparation. This research paper highlights speech recognition through ANN (Artificial Neural Network). Also, a hybrid model is proposed for audio-visual speech recognition of the Tamil and Malay language through SOM (Self-organizing map0 and MLP (Multilayer Perceptron). The Effectiveness of the different models of NN (Neural Network) utilized in speech recognition will be examined.

Sign in / Sign up

Export Citation Format

Share Document