scholarly journals Hand Pose Recognition Using Parallel Multi Stream CNN

Sensors ◽  
2021 ◽  
Vol 21 (24) ◽  
pp. 8469
Author(s):  
Iram Noreen ◽  
Muhammad Hamid ◽  
Uzma Akram ◽  
Saadia Malik ◽  
Muhammad Saleem
Keyword(s):  
Body Movement ◽  
Computer Applications ◽  
Support Vector ◽  
First Person ◽  
Hand Posture ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Depth Data ◽  
Proposed Model ◽  
Hand Pose

Recently, several computer applications provided operating mode through pointing fingers, waving hands, and with body movement instead of a mouse, keyboard, audio, or touch input such as sign language recognition, robot control, games, appliances control, and smart surveillance. With the increase of hand-pose-based applications, new challenges in this domain have also emerged. Support vector machines and neural networks have been extensively used in this domain using conventional RGB data, which are not very effective for adequate performance. Recently, depth data have become popular due to better understating of posture attributes. In this study, a multiple parallel stream 2D CNN (two-dimensional convolution neural network) model is proposed to recognize the hand postures. The proposed model comprises multiple steps and layers to detect hand poses from image maps obtained from depth data. The hyper parameters of the proposed model are tuned through experimental analysis. Three publicly available benchmark datasets: Kaggle, First Person, and Dexter, are used independently to train and test the proposed approach. The accuracy of the proposed method is 99.99%, 99.48%, and 98% using the Kaggle hand posture dataset, First Person hand posture dataset, and Dexter dataset, respectively. Further, the results obtained for F1 and AUC scores are also near-optimal. Comparative analysis with state-of-the-art shows that the proposed model outperforms the previous methods.

scholarly journals Wi-SL: Contactless Fine-Grained Gesture Recognition Uses Channel State Information

Sensors ◽  
2020 ◽  
Vol 20 (14) ◽  
pp. 4025
Author(s):  
Zhanjun Hao ◽  
Yu Duan ◽  
Xiaochao Dang ◽  
Yang Liu ◽  
Daiyang Zhang
Keyword(s):  
Sign Language ◽  
Gesture Recognition ◽  
Channel State Information ◽  
Computational Cost ◽  
Support Vector ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Channel State ◽  
State Information ◽  
Fine Grained

In recent years, with the development of wireless sensing technology and the widespread popularity of WiFi devices, human perception based on WiFi has become possible, and gesture recognition has become an active topic in the field of human-computer interaction. As a kind of gesture, sign language is widely used in life. The establishment of an effective sign language recognition system can help people with aphasia and hearing impairment to better interact with the computer and facilitate their daily life. For this reason, this paper proposes a contactless fine-grained gesture recognition method using Channel State Information (CSI), namely Wi-SL. This method uses a commercial WiFi device to establish the correlation mapping between the amplitude and phase difference information of the subcarrier level in the wireless signal and the sign language action, without requiring the user to wear any device. We combine an efficient denoising method to filter environmental interference with an effective selection of optimal subcarriers to reduce the computational cost of the system. We also use K-means combined with a Bagging algorithm to optimize the Support Vector Machine (SVM) classification (KSB) model to enhance the classification of sign language action data. We implemented the algorithms and evaluated them for three different scenarios. The experimental results show that the average accuracy of Wi-SL gesture recognition can reach 95.8%, which realizes device-free, non-invasive, high-precision sign language gesture recognition.

Improving arm segmentation in sign language recognition systems using image processing

2020 ◽  
pp. 1-14
Author(s):  
Qiuhong Tian ◽  
Jiaxin Bao ◽  
Huimin Yang ◽  
Yingrou Chen ◽  
Qiaoli Zhuang
Keyword(s):  
Image Processing ◽  
Sign Language ◽  
Recognition Rate ◽  
Support Vector ◽  
Language Recognition ◽  
Segmentation Method ◽  
Sign Language Recognition ◽  
Skin Segmentation ◽  
Reconstruction Methods ◽  
Area Operator

BACKGROUND: For a traditional vision-based static sign language recognition (SLR) system, arm segmentation is a major factor restricting the accuracy of SLR. OBJECTIVE: To achieve accurate arm segmentation for different bent arm shapes, we designed a segmentation method for a static SLR system based on image processing and combined it with morphological reconstruction. METHODS: First, skin segmentation was performed using YCbCr color space to extract the skin-like region from a complex background. Then, the area operator and the location of the mass center were used to remove skin-like regions and obtain the valid hand-arm region. Subsequently, the transverse distance was calculated to distinguish different bent arm shapes. The proposed segmentation method then extracted the hand region from different types of hand-arm images. Finally, the geometric features of the spatial domain were extracted and the sign language image was identified using a support vector machine (SVM) model. Experiments were conducted to determine the feasibility of the method and compare its performance with that of neural network and Euclidean distance matching methods. RESULTS: The results demonstrate that the proposed method can effectively segment skin-like regions from complex backgrounds as well as different bent arm shapes, thereby improving the recognition rate of the SLR system.

scholarly journals Hand Gesture Recognition Using PCA, KNN and SVM

2019 ◽  
pp. 547-550
Author(s):  
Julakanti Likhitha Reddy ◽  
Bhavya Mallela ◽  
Lakshmi Lavanya Bannaravuri ◽  
Kotha Mohan Krishna
Keyword(s):  
Gesture Recognition ◽  
Skin Color ◽  
Template Matching ◽  
Hand Gesture Recognition ◽  
Support Vector ◽  
Hand Gesture ◽  
Color Model ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Skin Color Model

To interact with world using expressions or body movements is comparatively effective than just speaking. Gesture recognition can be a better way to convey meaningful information. Communication through gestures has been widely used by humans to express their thoughts and feelings. Gestures can be performed with any body part like head, face, hands and arms but most predominantly hand is use to perform gestures, Hand Gesture Recognition have been widely accepted for numerous applications such as human computer interactions, robotics, sign language recognition, etc. This paper focuses on bare hand gesture recognition system by proposing a scheme using a database-driven hand gesture recognition based upon skin color model approach and thresholding approach along with an effective template matching with can be effectively used for human robotics applications and similar other applications .Initially, hand region is segmented by applying skin color model in YCbCr color space. Y represents the luminance and Cb and Cr represents chrominance. In the next stage Otsu thresholding is applied to separate foreground and background. Finally, template based matching technique is developed using Principal Component Analysis (PCA), k-nearest neighbour (KNN) and Support Vector Machine (SVM) for recognition. KNN is used for statistical estimation and pattern recognition. SVM can be used for classification or regression problems.

scholarly journals Sign Language Recognition using Principal Component Analysis and Support Vector Machine

2021 ◽  
Vol 4 (01) ◽  
pp. 49
Author(s):  
Astri Novianty ◽  
Fairuz Azmi
Keyword(s):  
Principal Component Analysis ◽  
Support Vector Machine ◽  
Sign Language ◽  
Principal Component ◽  
Histogram Equalization ◽  
Component Analysis ◽  
World Health ◽  
Support Vector ◽  
Language Recognition ◽  
Sign Language Recognition

The World Health Organization (WHO) estimates that over five percent of the world's population are hearing-impaired. One of the communication problems that often arise between deaf or speech impaired with normal people is the low level of knowledge and understanding of the deaf or speech impaired's normal sign language in their daily communication. To overcome this problem, we build a sign language recognition system, especially for the Indonesian language. The sign language system for Bahasa Indonesia, called Bisindo, is unique from the others. Our work utilizes two image processing algorithms for the pre-processing, namely the grayscale conversion and the histogram equalization. Subsequently, the principal component analysis (PCA) is employed for dimensional reduction and feature extraction. Finally, the support vector machine (SVM) is applied as the classifier. Results indicate that the use of the histogram equalization significantly enhances the accuracy of the recognition. Comprehensive experiments by applying different random seeds for testing data confirm that our method achieves 76.8% accuracy. Accordingly, a more robust method is still open to enhance the accuracy in sign language recognition.

scholarly journals Vision-based Pakistani Sign Language Recognition Using Bag-of-Words and Support Vector Machines

2022 ◽  
Author(s):  
Muhammad Shaheer Mirza ◽  
Sheikh Muhammad Munaf ◽  
Shahid Ali ◽  
Fahad Azim ◽  
Saad Jawaid Khan
Keyword(s):  
Sign Language ◽  
Recognition System ◽  
Support Vector ◽  
Bag Of Words ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Sign Languages ◽  
Vector Machines ◽  
Rgb Images ◽  
Native Signers

Abstract In order to perform their daily activities, a person is required to communicating with others. This can be a major obstacle for the deaf population of the world, who communicate using sign languages (SL). Pakistani Sign Language (PSL) is used by more than 250,000 deaf Pakistanis. Developing a SL recognition system would greatly facilitate these people. This study aimed to collect data of static and dynamic PSL alphabets and to develop a vision-based system for their recognition using Bag-of-Words (BoW) and Support Vector Machine (SVM) techniques. A total of 5,120 images for 36 static PSL alphabet signs and 353 videos with 45,224 frames for 3 dynamic PSL alphabet signs were collected from 10 native signers of PSL. The developed system used the collected data as input, resized the data to various scales and converted the RGB images into grayscale. The resized grayscale images were segmented using Thresholding technique and features were extracted using Speeded Up Robust Feature (SURF). The obtained SURF descriptors were clustered using K-means clustering. A BoW was obtained by computing the Euclidean distance between the SURF descriptors and the clustered data. The codebooks were divided into training and testing using 5-fold cross validation. The highest overall classification accuracy for static PSL signs was 97.80% at 750×750 image dimensions and 500 Bags. For dynamic PSL signs a 96.53% accuracy was obtained at 480×270 video resolution and 200 Bags.

scholarly journals American Sign Language Recognition Using Leap Motion Controller with Machine Learning Approach

Sensors ◽  
2018 ◽  
Vol 18 (10) ◽  
pp. 3554 ◽  
Author(s):  
Teak-Wei Chong ◽  
Boon-Giin Lee
Keyword(s):  
American Sign Language ◽  
Sign Language ◽  
Recognition System ◽  
Support Vector ◽  
Leap Motion ◽  
American Sign ◽  
Language Recognition ◽  
Motion Controller ◽  
Sign Language Recognition ◽  
Leap Motion Controller

Sign language is intentionally designed to allow deaf and dumb communities to convey messages and to connect with society. Unfortunately, learning and practicing sign language is not common among society; hence, this study developed a sign language recognition prototype using the Leap Motion Controller (LMC). Many existing studies have proposed methods for incomplete sign language recognition, whereas this study aimed for full American Sign Language (ASL) recognition, which consists of 26 letters and 10 digits. Most of the ASL letters are static (no movement), but certain ASL letters are dynamic (they require certain movements). Thus, this study also aimed to extract features from finger and hand motions to differentiate between the static and dynamic gestures. The experimental results revealed that the sign language recognition rates for the 26 letters using a support vector machine (SVM) and a deep neural network (DNN) are 80.30% and 93.81%, respectively. Meanwhile, the recognition rates for a combination of 26 letters and 10 digits are slightly lower, approximately 72.79% for the SVM and 88.79% for the DNN. As a result, the sign language recognition system has great potential for reducing the gap between deaf and dumb communities and others. The proposed prototype could also serve as an interpreter for the deaf and dumb in everyday life in service sectors, such as at the bank or post office.

Sign in / Sign up

Export Citation Format

Share Document