scholarly journals Advances in Convolutional Neural Networks

2020 ◽  
Author(s):  
Wen Xu ◽  
Jing He ◽  
Yanfeng Shu ◽  
Hui Zheng
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Recurrent Neural Networks ◽  
Kernel Methods ◽  
Representation Learning ◽  
Multilayer Perceptrons ◽  
Learning Tasks ◽  
Comprehensive Survey ◽  
High Level ◽  
Advanced Applications

Deep Learning, also known as deep representation learning, has dramatically improved the performances on a variety of learning tasks and achieved tremendous successes in the past few years. Specifically, artificial neural networks are mainly studied, which mainly include Multilayer Perceptrons (MLPs), Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). Among these networks, CNNs got the most attention due to the kernel methods with the weight sharing mechanism, and achieved state-of-the-art in many domains, especially computer vision. In this research, we conduct a comprehensive survey related to the recent improvements in CNNs, and we demonstrate these advances from the low level to the high level, including the convolution operations, convolutional layers, architecture design, loss functions, and advanced applications.

scholarly journals Interpretation of Swedish Sign Language Using Convolutional Neural Networks and Transfer Learning

2021 ◽  
Vol 2 (3) ◽  
Author(s):  
Gustaf Halvardsson ◽  
Johanna Peterson ◽  
César Soto-Valero ◽  
Benoit Baudry
Keyword(s):  
Neural Networks ◽  
Sign Language ◽  
Transfer Learning ◽  
Convolutional Neural Networks ◽  
Web Application ◽  
Training Dataset ◽  
Motion Processing ◽  
Image Perception ◽  
Sign Languages ◽  
High Level

AbstractThe automatic interpretation of sign languages is a challenging task, as it requires the usage of high-level vision and high-level motion processing systems for providing accurate image perception. In this paper, we use Convolutional Neural Networks (CNNs) and transfer learning to make computers able to interpret signs of the Swedish Sign Language (SSL) hand alphabet. Our model consists of the implementation of a pre-trained InceptionV3 network, and the usage of the mini-batch gradient descent optimization algorithm. We rely on transfer learning during the pre-training of the model and its data. The final accuracy of the model, based on 8 study subjects and 9400 images, is 85%. Our results indicate that the usage of CNNs is a promising approach to interpret sign languages, and transfer learning can be used to achieve high testing accuracy despite using a small training dataset. Furthermore, we describe the implementation details of our model to interpret signs as a user-friendly web application.

Recurrent Neural Networks for Edge Intelligence

2021 ◽  
Vol 54 (4) ◽  
pp. 1-38
Author(s):  
Varsha S. Lalapura ◽  
J. Amudha ◽  
Hariramn Selvamuruga Satheesh
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Model Performance ◽  
Network Architectures ◽  
Training Procedure ◽  
Resource Constrained ◽  
Compression Technique ◽  
Creative Art ◽  
Comprehensive Survey ◽  
Intelligent Models

Recurrent Neural Networks are ubiquitous and pervasive in many artificial intelligence applications such as speech recognition, predictive healthcare, creative art, and so on. Although they provide accurate superior solutions, they pose a massive challenge “training havoc.” Current expansion of IoT demands intelligent models to be deployed at the edge. This is precisely to handle increasing model sizes and complex network architectures. Design efforts to meet these for greater performance have had inverse effects on portability on edge devices with real-time constraints of memory, latency, and energy. This article provides a detailed insight into various compression techniques widely disseminated in the deep learning regime. They have become key in mapping powerful RNNs onto resource-constrained devices. While compression of RNNs is the main focus of the survey, it also highlights challenges encountered while training. The training procedure directly influences model performance and compression alongside. Recent advancements to overcome the training challenges with their strengths and drawbacks are discussed. In short, the survey covers the three-step process, namely, architecture selection, efficient training process, and suitable compression technique applicable to a resource-constrained environment. It is thus one of the comprehensive survey guides a developer can adapt for a time-series problem context and an RNN solution for the edge.

scholarly journals Real-time classification of hand movements as a basis for intuitive control of grasp neuroprostheses

2020 ◽  
Vol 6 (2) ◽  
Author(s):  
Dmitry Amelin ◽  
Ivan Potapov ◽  
Josep Cardona Audí ◽  
Andreas Kogut ◽  
Rüdiger Rupp ◽  
...  
Keyword(s):  
Neural Networks ◽  
Standard Deviation ◽  
Real Time ◽  
Convolutional Neural Networks ◽  
Recurrent Neural Networks ◽  
Healthy Subjects ◽  
Hand Movements ◽  
Cord Injury ◽  
Field Programmable

AbstractThis paper reports on the evaluation of recurrent and convolutional neural networks as real-time grasp phase classifiers for future control of neuroprostheses for people with high spinal cord injury. A field-programmable gate array has been chosen as an implementation platform due to its form factor and ability to perform parallel computations, which are specific for the selected neural networks. Three different phases of two grasp patterns and the additional open hand pattern were predicted by means of surface Electromyography (EMG) signals (i.e. Seven classes in total). Across seven healthy subjects, CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Networks) had a mean accuracy of 85.23% with a standard deviation of 4.77% and 112 µs per prediction and 83.30% with a standard deviation of 4.36% and 40 µs per prediction, respectively.

scholarly journals Fully Automatic Cell Segmentation with Fourier Descriptors

2021 ◽  
Author(s):  
Dominik Hirling ◽  
Peter Horvath
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Fundamental Problem ◽  
Cell Segmentation ◽  
Fourier Descriptors ◽  
Control Points ◽  
Coefficient Vector ◽  
Fully Automatic ◽  
High Level ◽  
And Control

Cell segmentation is a fundamental problem in biology for which convolutional neural networks yield the best results nowadays. In this paper, we present HarmonicNet, a network, which is a modification of the popular StarDist and SplineDist architectures. While StarDist and SplineDist describe an object by the lengths of equiangular rays and control points respectively, our network utilizes Fourier descriptors, predicting a coefficient vector for every pixel on the image, which implicitly define the resulting segmentation. We evaluate our model on three different datasets, and show that Fourier descriptors can achieve a high level of accuracy with a small number of coefficients. HarmonicNet is also capable of accurately segmenting objects that are not star-shaped, a case where StarDist performs suboptimally according to our experiments.

Intrusion Detection Using Convolutional Neural Networks for Representation Learning

2017 ◽  
pp. 858-866 ◽  
Author(s):  
Zhipeng Li ◽  
Zheng Qin ◽  
Kai Huang ◽  
Xiao Yang ◽  
Shuxiong Ye
Keyword(s):  
Neural Networks ◽  
Intrusion Detection ◽  
Convolutional Neural Networks ◽  
Representation Learning

scholarly journals Representation learning of genomic sequence motifs with convolutional neural networks

2019 ◽  
Vol 15 (12) ◽  
pp. e1007560 ◽  
Author(s):  
Peter K. Koo ◽  
Sean R. Eddy
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Genomic Sequence ◽  
Representation Learning ◽  
Sequence Motifs

scholarly journals A Guide for Using Deep Learning for Complex Trait Genomic Prediction

Genes ◽  
2019 ◽  
Vol 10 (7) ◽  
pp. 553 ◽  
Author(s):  
Pérez-Enciso ◽  
Zingaretti
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Genomic Prediction ◽  
Genetic Risk ◽  
Complex Trait ◽  
Molecular Data ◽  
Multilayer Perceptrons ◽  
Complex Data ◽  
Theoretical Foundations

Deep learning (DL) has emerged as a powerful tool to make accurate predictions from complex data such as image, text, or video. However, its ability to predict phenotypic values from molecular data is less well studied. Here, we describe the theoretical foundations of DL and provide a generic code that can be easily modified to suit specific needs. DL comprises a wide variety of algorithms which depend on numerous hyperparameters. Careful optimization of hyperparameter values is critical to avoid overfitting. Among the DL architectures currently tested in genomic prediction, convolutional neural networks (CNNs) seem more promising than multilayer perceptrons (MLPs). A limitation of DL is in interpreting the results. This may not be relevant for genomic prediction in plant or animal breeding but can be critical when deciding the genetic risk to a disease. Although DL technologies are not ”plug-and-play”, they are easily implemented using Keras and TensorFlow public software. To illustrate the principles described here, we implemented a Keras-based code in GitHub.

Sign in / Sign up

Export Citation Format

Share Document