scholarly journals Gesture Based Real-time Indian Sign Language Interpreter

2021 ◽  
pp. 387-394
Author(s):  
Akshay Divkar ◽  
Rushikesh Bailkar ◽  
Dr. Chhaya S. Pawar
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Sign Language ◽  
Recurrent Neural Network ◽  
Video Sequences ◽  
Basic Step ◽  
Spatial Features ◽  
Normal People ◽  
Temporal Features ◽  
Impaired People

Hand gesture is one of the methods used in sign language for non-verbal communication. It is most commonly used by hearing & speech impaired people who have hearing or speech problems to communicate among themselves or with normal people. Developing sign language applications for hearing impaired people can be very important, as hearing & speech impaired people will be able to communicate easily with even those who don’t understand sign language. This project aims at taking the basic step in bridging the communication gap between normal people, deaf and dumb people using sign language. The main focus of this work is to create a vision based system to identify sign language gestures from the video sequences. The reason for choosing a system based on vision relates to the fact that it provides a simpler and more intuitive way of communication between a human and a computer. Video sequences contain both temporal as well as spatial features. In this project, two different models are used to train the temporal as well as spatial features. To train the model on the spatial features of the video sequences a deep Convolutional Neural Network. Convolutional Neural Network was trained on the frames obtained from the video sequences of train data. To train the model on the temporal features Recurrent Neural Network is used. The Trained Convolutional Neural Network model was used to make predictions for individual frames to obtain a sequence of predictions. Now this sequence of prediction outputs was given to the Recurrent Neural Network to train on the temporal features. Collectively both the trained models i.e. Convolutional Neural Network and Recurrent Neural Network will produce the text output of the respective gesture.

scholarly journals Processo para Reconhecimento e Tradução de Sinais em LIBRAS Utilizando Redes Neurais Artificiais

2020 ◽  
Author(s):  
João Pedro C. Sobrinho ◽  
Lucas Pacheco H. da Silva ◽  
Gabriella Dalpra ◽  
Samuel Basilio
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Data Collection ◽  
Sign Language ◽  
Recurrent Neural Network ◽  
Official Language ◽  
Current Version ◽  
Network Methods ◽  
Impaired People ◽  
Brazilian Sign Language

Recognized by law, the Brazilian Sign Language (LIBRAS), is thesecond Brazilian official language and, according to IBGE (BrazilianInstitute of Geography and Statistics), Brazil has a large communityof hearing-impaired people, with approximately nine million ofdeaf people. Besides that, most of the non-deaf community cannotcommunicate or understand this language. Considering that, theuse of LIBRAS’ interpreters becomes extremely necessary in orderto allow a greater inclusion of people with this type of disabilitywith the whole community. However, an alternative solution tothis problem would be to use artificial neural network methods forthe LIBRAS recognition and translation. In this work, a processof LIBRAS’ recognition and translation is presented, using videosas input and a convolutional-recurrent neural network, known asConvLSTM. This type of neural network receives the sequence offrames from the videos and analyzes, frame by frame, if the framebelongs to the video and if the video belongs to a specific class.This analysis is done in two steps: first, the image is analyzed inthe convolutional layer of the network and, after that, it is sent tothe network recurrent layer. In the current version of the implementednetwork, data collection has already been carried out, theconvolutional-recurrent neural network has been trained and it ispossible to recognize when a given LIBRAS’ video represents ornot a specific sentence in this language.

scholarly journals A combined convolutional and recurrent neural network for enhanced glaucoma detection

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Soheila Gheisari ◽  
Sahar Shariflou ◽  
Jack Phu ◽  
Paul J. Kennedy ◽  
Ashish Agar ◽  
...  
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Fundus Images ◽  
Spatial Features ◽  
Glaucoma Detection ◽  
Temporal Features ◽  
Long Short Term Memory ◽  
Sequential Images ◽  
F Measure

AbstractGlaucoma, a leading cause of blindness, is a multifaceted disease with several patho-physiological features manifesting in single fundus images (e.g., optic nerve cupping) as well as fundus videos (e.g., vascular pulsatility index). Current convolutional neural networks (CNNs) developed to detect glaucoma are all based on spatial features embedded in an image. We developed a combined CNN and recurrent neural network (RNN) that not only extracts the spatial features in a fundus image but also the temporal features embedded in a fundus video (i.e., sequential images). A total of 1810 fundus images and 295 fundus videos were used to train a CNN and a combined CNN and Long Short-Term Memory RNN. The combined CNN/RNN model reached an average F-measure of 96.2% in separating glaucoma from healthy eyes. In contrast, the base CNN model reached an average F-measure of only 79.2%. This proof-of-concept study demonstrates that extracting spatial and temporal features from fundus videos using a combined CNN and RNN, can markedly enhance the accuracy of glaucoma detection.

A Novel Automatic Modulation Classification Method Using Attention Mechanism and Hybrid Parallel Neural Network

2021 ◽  
Vol 11 (3) ◽  
pp. 1327
Author(s):  
Rui Zhang ◽  
Zhendong Yin ◽  
Zhilu Wu ◽  
Siyang Zhou
Keyword(s):  
Neural Network ◽  
Communication Systems ◽  
Parallel Structure ◽  
Modulation Classification ◽  
Automatic Modulation Classification ◽  
Spatial Features ◽  
Temporal Features ◽  
Different Types ◽  
In Series ◽  
Parallel Network

Automatic Modulation Classification (AMC) is of paramount importance in wireless communication systems. Existing methods usually adopt a single category of neural network or stack different categories of networks in series, and rarely extract different types of features simultaneously in a proper way. When it comes to the output layer, softmax function is applied for classification to expand the inter-class distance. In this paper, we propose a hybrid parallel network for the AMC problem. Our proposed method designs a hybrid parallel structure which utilizes Convolution Neural Network (CNN) and Gate Rate Unit (GRU) to extract spatial features and temporal features respectively. Instead of superposing these two categories of features directly, three different attention mechanisms are applied to assign weights for different types of features. Finally, a cosine similarity metric named Additive Margin softmax function, which can expand the inter-class distance and compress the intra-class distance simultaneously, is adopted for output. Simulation results demonstrate that the proposed method can achieve remarkable performance on an open access dataset.

Composite load modeling by spatial-temporal deep attention network based on wide-area monitoring systems

2021 ◽  
pp. 1-12
Author(s):  
Omid Izadi Ghafarokhi ◽  
Mazda Moattari ◽  
Ahmad Forouzantabar
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Wide Area ◽  
Load Modeling ◽  
Wide Area Monitoring ◽  
Training Performance ◽  
Spatial Features ◽  
Temporal Features ◽  
Frequency Domains ◽  
Area Monitoring

With the development of the wide-area monitoring system (WAMS), power system operators are capable of providing an accurate and fast estimation of time-varying load parameters. This study proposes a spatial-temporal deep network-based new attention concept to capture the dynamic and static patterns of electrical load consumption through modeling complicated and non-stationary interdependencies between time sequences. The designed deep attention-based network benefits from long short-term memory (LSTM) based component to learning temporal features in time and frequency-domains as encoder-decoder based recurrent neural network. Furthermore, to inherently learn spatial features, a convolutional neural network (CNN) based attention mechanism is developed. Besides, this paper develops a loss function based on a pseudo-Huber concept to enhance the robustness of the proposed network in noisy conditions as well as improve the training performance. The simulation results on IEEE 68-bus demonstrates the effectiveness and superiority of the proposed network through comparison with several previously presented and state-of-the-art methods.

scholarly journals A Review of Plant Phenotypic Image Recognition Technology Based on Deep Learning

Electronics ◽  
2021 ◽  
Vol 10 (1) ◽  
pp. 81
Author(s):  
Jianbin Xiong ◽  
Dezheng Yu ◽  
Shuangyin Liu ◽  
Lei Shu ◽  
Xiaochan Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Plant Species ◽  
Image Recognition ◽  
Recurrent Neural Network ◽  
Plant Diseases ◽  
Learning Methods ◽  
Smart Agriculture ◽  
Important Branch

Plant phenotypic image recognition (PPIR) is an important branch of smart agriculture. In recent years, deep learning has achieved significant breakthroughs in image recognition. Consequently, PPIR technology that is based on deep learning is becoming increasingly popular. First, this paper introduces the development and application of PPIR technology, followed by its classification and analysis. Second, it presents the theory of four types of deep learning methods and their applications in PPIR. These methods include the convolutional neural network, deep belief network, recurrent neural network, and stacked autoencoder, and they are applied to identify plant species, diagnose plant diseases, etc. Finally, the difficulties and challenges of deep learning in PPIR are discussed.

Sign in / Sign up

Export Citation Format

Share Document