Computer Vision-Enabled Character Recognition of Hand Gestures for Patients with Hearing and Speaking Disability

Hand gesture recognition is one of the most sought technologies in the field of machine learning and computer vision. There has been an unprecedented demand for applications through which one can detect the hand signs for deaf people and people who use sign language to communicate, thereby detecting hand signs and correspondingly predicting the next word or recommending the word that may be most appropriate, followed by producing the word that the deaf people and people who use sign language to communicate want to say. This article presents an approach to develop such a system by that we can determine the most appropriate character from the sign that is being shown by the user or the person to the system. To enable pattern recognition, various machine learning techniques have been explored and we have used the CNN networks as a reliable solution in our context. The creation of such a system involves several convolution layers through which features have been captured layer by layer. The gathered features from the image are further used for training the model. The trained model efficiently predicts the most appropriate character in response to the sign exposed to the model. Thereafter, the predicted character is used to predict further words from it according to the recommendation system used in this case. The proposed system attains a prediction accuracy of 91.07%.