Bangla Sign Language (BdSL) Alphabets and Numerals Classification Using a Deep Learning Model

A real-time Bangla Sign Language interpreter can enable more than 200 k hearing and speech-impaired people to the mainstream workforce in Bangladesh. Bangla Sign Language (BdSL) recognition and detection is a challenging topic in computer vision and deep learning research because sign language recognition accuracy may vary on the skin tone, hand orientation, and background. This research has used deep machine learning models for accurate and reliable BdSL Alphabets and Numerals using two well-suited and robust datasets. The dataset prepared in this study comprises of the largest image database for BdSL Alphabets and Numerals in order to reduce inter-class similarity while dealing with diverse image data, which comprises various backgrounds and skin tones. The papers compared classification with and without background images to determine the best working model for BdSL Alphabets and Numerals interpretation. The CNN model trained with the images that had a background was found to be more effective than without background. The hand detection portion in the segmentation approach must be more accurate in the hand detection process to boost the overall accuracy in the sign recognition. It was found that ResNet18 performed best with 99.99% accuracy, precision, F1 score, sensitivity, and 100% specificity, which outperforms the works in the literature for BdSL Alphabets and Numerals recognition. This dataset is made publicly available for researchers to support and encourage further research on Bangla Sign Language Interpretation so that the hearing and speech-impaired individuals can benefit from this research.

Architecture Design and VLSI Implementation of 3D Hand Gesture Recognition System

With advancements in technology, more and more research is being focused on enhancing daily life quality and convenience. Along with the increase in the development of gesture control systems, many controllers, such as the keyboard, mouse, and other devices, have been replaced with remote control products, which are gradually becoming more intuitive for users. However, vision-based hand gesture recognition systems still have many problems to overcome. Most hand detection methods adopt a skin filter or motion filter for pre-processing. However, in a noisy environment, it is not easy to correctly extract interesting objects. In this paper, a VLSI design with dual-cameras has been proposed to construct a depth map with a stereo matching algorithm and recognize hand gestures. The proposed system adopts an adaptive depth filter to separate interesting foreground objects from the background. We also propose dynamic gesture recognition using depth and coordinate information. The system can perform static and dynamic gesture recognition. The ASIC design is implemented in TSMC 90 nm with about 47.3 K gate counts, and 27.8 mW of power consumption. The average accuracy of each gesture recognition is 83.98%.

In-situ identification and recognition of multi-hand gestures using optimized deep residual network

The real-time perception of hand gestures in a deprived environment is a demanding machine vision task. The hand recognition operations are more strenuous with different illumination conditions and varying backgrounds. Robust recognition and classification are the vital steps to support effective human-machine interaction (HMI), virtual reality, etc. In this paper, the real-time hand action recognition is performed by using an optimized Deep Residual Network model. It incorporates a RetinaNet model for hand detection and a Depthwise Separable Convolutional (DSC) layer for precise hand gesture recognition. The proposed model overcomes the class imbalance problems encountered by the conventional single-stage hand action recognition algorithms. The integrated DSC layer reduces the computational parameters and enhances the recognition speed. The model utilizes a ResNet-101 CNN architecture as a Feature extractor. The model is trained and evaluated on the MITI-HD dataset and compared with the benchmark datasets (NUSHP-II, Senz-3D). The network achieved a higher Precision and Recall value for an IoU value of 0.5. It is realized that the RetinaNet-DSC model using ResNet-101 backbone network obtained higher Precision (99.21 %for AP0.5, 96.80%for AP0.75) for MITI-HD Dataset. Higher performance metrics are obtained for a value of γ= 2 and α= 0.25. The SGD with a momentum optimizer outperformed the other optimizers (Adam, RMSprop) for the datasets considered in the studies. The prediction time of the optimized deep residual network is 82 ms.

A Deep Convolutional Neural Network Approach to Sign Alphabet Recognition

In this paper, we present an application that has been developed to be used as a tool for the purposes of learning sign language for beginners that utilizes hand detection as part of the process. It uses a skin-color modelling technique, such as explicit thresholding in the skin-color space, which is based on modeling skin-color spaces. This predetermined range of skin-colors is used to determine how pixels (hand) will be extracted from non-pixels (background). To classify the images, convolutional neural networks (CNN) were fed the images for the creation of the classifier. The training of the images was done using Keras. A uniform background and proper lighting conditions enabled the system to achieve a test accuracy of 93.67%, of which 90.04% was attributed to ASL alphabet recognition, 93.44% for number recognition and 97.52% recognition of static words, surpassing other studies of the type. An approach which is based on this technique is used for fast computation as well as real-time processing. Deaf-dumb people face a number of social challenges as the communication barrier prevents them from accessing basic and essential services of the life that they are entitled to as members of the hearing community. In spite of the fact that a number of factors have been incorporated into the innovations in the automatic recognition of sign language, an adequate solution has yet to be reached because of a number of challenges. As far as I know, the vast majority of existing works focus on developing vision based recognizers by deriving complex feature descriptors from captured images of the gestures and applying a classical pattern analysis technique. Although utilizing these methods can be effective when dealing with small sign vocabulary captures with a complex and uncontrolled background, they are very limited when dealing with large sign vocabulary. This paper proposes a method for analyzing and representing hand gestures, which acts as the core component of the vocabulary for signing languages, using a deep convolutional neural networks (CNN) architecture. On two publicly accessible datasets (the NUS hand posture dataset and the American fingerspelling A dataset), the method was demonstrated to be more accurate in recognizing hand postures.

Hand Detection for Safety to Avoid Accidents

Detecting the hand when it crosses the safety level and in return it also raises an alert in the form of alarm. So that the threat can be identified and proper measures are taken to overcome that. The methodology of the project goes as follows, taking input from camera , Image processing to detect hand, Projecting a line using computer vision, Raising alarm when hand crosses this projected safety line. The real time data is taken from the camera as an input to the Image processing algorithm. Then this input is processed to find the hand in image in it and checks whether the hand is crossing that safety line. If that hand is crossing the safety line we can simply raise alarm. The applications of the project are to the Employees who are working at industry are pushing the material into shredder machine. But somehow while pushing these material into shredder machine the employees are pushing their hands itself in the flow of work and the hands of employees were cut in that cause. So from a certain distance from shredder machine input we project a imaginary line using computer vision, So that if any hand crossing that imaginary line which is for safety we will raise an alarm. In addition, we can also extend the applications, by just replacing hand with the Bike, we can detect the bike, which is crossing the staggered stop line, and we can punish or fine them. As a part of object detection we are using Single short multibox detector.

Industry 4.0: Hand Recognition on Assembly Supervision Process

In the assembly industry, the process of assembling components is very important in order to produce a quality product. Assembly of components should be carried out sequentially based on the standards set by the company. For companies that still operate the assembly process manually by employee, sometimes errors occur in the assembly process, which can affect the quality of production. In order to be carried out the assembly process according to the procedure, a system is needed that can detect employee hands when carrying out the assembly process automatically. This study proposes an artificial intelligence-based real-time employee hand detection system. This system will be the basis for the development of an automatic industrial product assembly process to welcome the Industry 4.0. To verify system performance, several experiments were carried out, such as; detecting the right and left hands of employees and detecting hands when using accessories or not. From the experimental results it can be concluded that the system is able to detect the right and left hands of employees well with the resulting FPS average of 15.4.