Optimizing Convolutional Neural Network Accelerator on Low-Cost FPGA

Low Cost ◽

Optimal Number ◽

Data Reuse ◽

Logic Element ◽

Input Buffer ◽

Classification Tasks ◽

Processing Engine ◽

Convolutional neural network (CNN) is one of the most promising algorithms that outweighs other traditional methods in terms of accuracy in classification tasks. However, several CNNs, such as VGG, demand a huge computation in convolutional layers. Many accelerators implemented on powerful FPGAs have been introduced to address the problems. In this paper, we present a VGG-based accelerator which is optimized for a low-cost FPGA. In order to optimize the FPGA resource of logic element and memory, we propose a dedicated input buffer that maximizes the data reuse. In addition, we design a low resource processing engine with the optimal number of Multiply Accumulate (MAC) units. In the experiments, we use VGG16 model for inference to evaluate the performance of our accelerator and achieve a throughput of 38.8[Formula: see text]GOPS at a clock speed of 150[Formula: see text]MHz on Intel Cyclone V SX SoC. The experimental results show that our design is better than previous works in terms of resource efficiency.

Detection of exomoons in simulated light curves with a regularized convolutional neural network

Astronomy and Astrophysics ◽

10.1051/0004-6361/201937059 ◽

2020 ◽

Vol 640 ◽

pp. A41

Author(s):

Rasha Alshehhi ◽

Kai Rodenbeck ◽

Laurent Gizon ◽

Katepalli R. Sreenivasan

Keyword(s):

Neural Network ◽

Network Optimization ◽

Extrasolar Planets ◽

Classical Method ◽

Light Curves ◽

Training Process ◽

Classification Tasks ◽

Host Stars ◽

Context. Many moons have been detected around planets in our Solar System, but none has been detected unambiguously around any of the confirmed extrasolar planets. Aims. We test the feasibility of a supervised convolutional neural network to classify photometric transit light curves of planet-host stars and identify exomoon transits, while avoiding false positives caused by stellar variability or instrumental noise. Methods. Convolutional neural networks are known to have contributed to improving the accuracy of classification tasks. The network optimization is typically performed without studying the effect of noise on the training process. Here we design and optimize a 1D convolutional neural network to classify photometric transit light curves. We regularize the network by the total variation loss in order to remove unwanted variations in the data features. Results. Using numerical experiments, we demonstrate the benefits of our network, which produces results comparable to or better than the standard network solutions. Most importantly, our network clearly outperforms a classical method used in exoplanet science to identify moon-like signals. Thus the proposed network is a promising approach for analyzing real transit light curves in the future.

End-to-end convolutional neural network enables COVID-19 detection from breath and cough audio: a pilot study

BMJ Innovations ◽

10.1136/bmjinnov-2021-000668 ◽

2021 ◽

Vol 7 (2) ◽

pp. 356-362

Author(s):

Harry Coppock ◽

Alex Gaskell ◽

Panagiotis Tzirakis ◽

Alice Baird ◽

Lyn Jones ◽

...

Keyword(s):

Neural Network ◽

Low Cost ◽

Area Under The Curve ◽

Alternative Form ◽

Economic Damage ◽

Audio Signals ◽

Operating Characteristics ◽

Data Set ◽

Empirical Performance

BackgroundSince the emergence of COVID-19 in December 2019, multidisciplinary research teams have wrestled with how best to control the pandemic in light of its considerable physical, psychological and economic damage. Mass testing has been advocated as a potential remedy; however, mass testing using physical tests is a costly and hard-to-scale solution.MethodsThis study demonstrates the feasibility of an alternative form of COVID-19 detection, harnessing digital technology through the use of audio biomarkers and deep learning. Specifically, we show that a deep neural network based model can be trained to detect symptomatic and asymptomatic COVID-19 cases using breath and cough audio recordings.ResultsOur model, a custom convolutional neural network, demonstrates strong empirical performance on a data set consisting of 355 crowdsourced participants, achieving an area under the curve of the receiver operating characteristics of 0.846 on the task of COVID-19 classification.ConclusionThis study offers a proof of concept for diagnosing COVID-19 using cough and breath audio signals and motivates a comprehensive follow-up research study on a wider data sample, given the evident advantages of a low-cost, highly scalable digital COVID-19 diagnostic tool.

Research on Real-Time Multiple Single Garbage Classification Based on Convolutional Neural Network

Mathematical Problems in Engineering ◽

10.1155/2020/5795976 ◽

2020 ◽

Vol 2020 ◽

pp. 1-6

Author(s):

Jian-ye Yuan ◽

Xin-yuan Nan ◽

Cheng-rong Li ◽

Le-le Sun

Keyword(s):

Neural Network ◽

Real Time ◽

Environmental Science ◽

Fine Tuning ◽

Classification Model ◽

Classification Models ◽

Data Set ◽

Classification Tasks ◽

Accuracy Rates

Considering that the garbage classification is urgent, a 23-layer convolutional neural network (CNN) model is designed in this paper, with the emphasis on the real-time garbage classification, to solve the low accuracy of garbage classification and recycling and difficulty in manual recycling. Firstly, the depthwise separable convolution was used to reduce the Params of the model. Then, the attention mechanism was used to improve the accuracy of the garbage classification model. Finally, the model fine-tuning method was used to further improve the performance of the garbage classification model. Besides, we compared the model with classic image classification models including AlexNet, VGG16, and ResNet18 and lightweight classification models including MobileNetV2 and SuffleNetV2 and found that the model GAF_dense has a higher accuracy rate, fewer Params, and FLOPs. To further check the performance of the model, we tested the CIFAR-10 data set and found the accuracy rates of the model (GAF_dense) are 0.018 and 0.03 higher than ResNet18 and SufflenetV2, respectively. In the ImageNet data set, the accuracy rates of the model (GAF_dense) are 0.225 and 0.146 higher than Resnet18 and SufflenetV2, respectively. Therefore, the garbage classification model proposed in this paper is suitable for garbage classification and other classification tasks to protect the ecological environment, which can be applied to classification tasks such as environmental science, children’s education, and environmental protection.

BengaliNet: A Low-Cost Novel Convolutional Neural Network for Bengali Handwritten Characters Recognition

Applied Sciences ◽

10.3390/app11156845 ◽

2021 ◽

Vol 11 (15) ◽

pp. 6845

Author(s):

Abu Sayeed ◽

Jungpil Shin ◽

Md. Al Mehedi Hasan ◽

Azmain Yakin Srizon ◽

Md. Mehedi Hasan

Keyword(s):

Neural Network ◽

Network Architecture ◽

Character Recognition ◽

High Performance ◽

Low Cost ◽

Traditional Learning ◽

Neural Network Architecture ◽

Handwritten Character Recognition ◽

Handwritten Character

As it is the seventh most-spoken language and fifth most-spoken native language in the world, the domain of Bengali handwritten character recognition has fascinated researchers for decades. Although other popular languages i.e., English, Chinese, Hindi, Spanish, etc. have received many contributions in the area of handwritten character recognition, Bengali has not received many noteworthy contributions in this domain because of the complex curvatures and similar writing fashions of Bengali characters. Previously, studies were conducted by using different approaches based on traditional learning, and deep learning. In this research, we proposed a low-cost novel convolutional neural network architecture for the recognition of Bengali characters with only 2.24 to 2.43 million parameters based on the number of output classes. We considered 8 different formations of CMATERdb datasets based on previous studies for the training phase. With experimental analysis, we showed that our proposed system outperformed previous works by a noteworthy margin for all 8 datasets. Moreover, we tested our trained models on other available Bengali characters datasets such as Ekush, BanglaLekha, and NumtaDB datasets. Our proposed architecture achieved 96–99% overall accuracies for these datasets as well. We believe our contributions will be beneficial for developing an automated high-performance recognition tool for Bengali handwritten characters.

Binary and Multiclass Text Classification by Means of Separable Convolutional Neural Network

Inventions ◽

10.3390/inventions6040070 ◽

2021 ◽

Vol 6 (4) ◽

pp. 70

Author(s):

Elena Solovyeva ◽

Ali Abdullah

Keyword(s):

Neural Network ◽

Neural Networks ◽

Recurrent Neural Networks ◽

Low Cost ◽

Computational Cost ◽

High Accuracy ◽

Activation Functions ◽

Fully Connected ◽

Fully Connected Networks

In this paper, the structure of a separable convolutional neural network that consists of an embedding layer, separable convolutional layers, convolutional layer and global average pooling is represented for binary and multiclass text classifications. The advantage of the proposed structure is the absence of multiple fully connected layers, which is used to increase the classification accuracy but raises the computational cost. The combination of low-cost separable convolutional layers and a convolutional layer is proposed to gain high accuracy and, simultaneously, to reduce the complexity of neural classifiers. Advantages are demonstrated at binary and multiclass classifications of written texts by means of the proposed networks under the sigmoid and Softmax activation functions in convolutional layer. At binary and multiclass classifications, the accuracy obtained by separable convolutional neural networks is higher in comparison with some investigated types of recurrent neural networks and fully connected networks.

Label Rectification Learning through Kernel Extreme Learning Machine

Wireless Communications and Mobile Computing ◽

10.1155/2021/6669081 ◽

2021 ◽

Vol 2021 ◽

pp. 1-6

Author(s):

Qiang Cai ◽

Fenghai Li ◽

Yifan Chen ◽

Haisheng Li ◽

Jian Cao ◽

...

Keyword(s):

Neural Network ◽

Image Classification ◽

Extreme Learning Machine ◽

Classification Performance ◽

Considerable Progress ◽

Strong Representation ◽

Kernel Extreme Learning Machine ◽

Classification Tasks ◽

Learning Machine

Along with the strong representation of the convolutional neural network (CNN), image classification tasks have achieved considerable progress. However, majority of works focus on designing complicated and redundant architectures for extracting informative features to improve classification performance. In this study, we concentrate on rectifying the incomplete outputs of CNN. To be concrete, we propose an innovative image classification method based on Label Rectification Learning (LRL) through kernel extreme learning machine (KELM). It mainly consists of two steps: (1) preclassification, extracting incomplete labels through a pretrained CNN, and (2) label rectification, rectifying the generated incomplete labels by the KELM to obtain the rectified labels. Experiments conducted on publicly available datasets demonstrate the effectiveness of our method. Notably, our method is extensible which can be easily integrated with off-the-shelf networks for improving performance.

A multi-objective opposition-based barnacles mating optimization for image super resolution using hyper-Spectral images

Journal of Engineering Design and Technology ◽

10.1108/jedt-01-2021-0030 ◽

2021 ◽

Vol ahead-of-print (ahead-of-print) ◽

Author(s):

Valli Bhasha A. ◽

Venkatramana Reddy B.D.

Keyword(s):

Neural Network ◽

Wavelet Transform ◽

Super Resolution ◽

Discrete Wavelet ◽

Content Type ◽

Resolution Model ◽

Deep Cnn ◽

Image Super Resolution ◽

Purpose The problems of Super resolution are broadly discussed in diverse fields. Rather than the progression toward the super resolution models for real-time images, operating hyperspectral images still remains a challenging problem. Design/methodology/approach This paper aims to develop the enhanced image super-resolution model using “optimized Non-negative Structured Sparse Representation (NSSR), Adaptive Discrete Wavelet Transform (ADWT), and Optimized Deep Convolutional Neural Network”. Once after converting the HR images into LR images, the NSSR images are generated by the optimized NSSR. Then the ADWT is used for generating the subbands of both NSSR and HRSB images. The residual image with this information is obtained by the optimized Deep CNN. All the improvements on the algorithms are done by the Opposition-based Barnacles Mating Optimization (O-BMO), with the objective of attaining the multi-objective function concerning the “Peak Signal-to-Noise Ratio (PSNR), and Structural similarity (SSIM) index”. Extensive analysis on benchmark hyperspectral image datasets shows that the proposed model achieves superior performance over typical other existing super-resolution models. Findings From the analysis, the overall analysis of the suggested and the conventional super resolution models relies that the PSNR of the improved O-BMO-(NSSR+DWT+CNN) was 38.8% better than bicubic, 11% better than NSSR, 16.7% better than DWT+CNN, 1.3% better than NSSR+DWT+CNN, and 0.5% better than NSSR+FF-SHO-(DWT+CNN). Hence, it has been confirmed that the developed O-BMO-(NSSR+DWT+CNN) is performing well in converting LR images to HR images. Originality/value This paper adopts a latest optimization algorithm called O-BMO with optimized Non-negative Structured Sparse Representation (NSSR), Adaptive Discrete Wavelet Transform (ADWT) and Optimized Deep Convolutional Neural Network for developing the enhanced image super-resolution model. This is the first work that uses O-BMO-based Deep CNN for image super-resolution model enhancement.

Object Detection in Ground-Penetrating Radar Images Using a Deep Convolutional Neural Network and Image Set Preparation by Migration

International Journal of Geophysics ◽

10.1155/2018/9365184 ◽

2018 ◽

Vol 2018 ◽

pp. 1-8 ◽

Cited By ~ 3

Author(s):

Kazuya Ishitsuka ◽

Shinichiro Iso ◽

Kyosuke Onishi ◽

Toshifumi Matsuoka

Keyword(s):

Neural Network ◽

Ground Penetrating Radar ◽

Deep Convolutional Neural Network ◽

Training Data ◽

Neural Network Approach ◽

Radar Images ◽

Image Set ◽

Ground Penetrating ◽

Ground-penetrating radar allows the acquisition of many images for investigation of the pavement interior and shallow geological structures. Accordingly, an efficient methodology of detecting objects, such as pipes, reinforcing steel bars, and internal voids, in ground-penetrating radar images is an emerging technology. In this paper, we propose using a deep convolutional neural network to detect characteristic hyperbolic signatures from embedded objects. As a first step, we developed a migration-based method to collect many training data and created 53510 categorized images. We then examined the accuracy of the deep convolutional neural network in detecting the signatures. The accuracy of the classification was 0.945 (94.5%)–0.979 (97.9%) when using several thousands of training images and was much better than the accuracy of the conventional neural network approach. Our results demonstrate the effectiveness of the deep convolutional neural network in detecting characteristic events in ground-penetrating radar images.

Pedestrian detection based on improved LeNet-5 convolutional neural network

Journal of Algorithms & Computational Technology ◽

10.1177/1748302619873601 ◽

2019 ◽

Vol 13 ◽

pp. 174830261987360 ◽

Cited By ~ 2

Author(s):

Chuan-Wei Zhang ◽

Meng-Yue Yang ◽

Hong-Jun Zeng ◽

Jian-Ping Wen

Keyword(s):

Neural Network ◽

Real Time ◽

Network Model ◽

Pedestrian Detection ◽

The Real ◽

In this article, according to the real-time and accuracy requirements of advanced vehicle-assisted driving in pedestrian detection, an improved LeNet-5 convolutional neural network is proposed. Firstly, the structure of LeNet-5 network model is analyzed, and the structure and parameters of the network are improved and optimized on the basis of this network to get a new LeNet network model, and then it is used to detect pedestrians. Finally, the miss rate of the improved LeNet convolutional neural network is found to be 25% by contrast and analysis. The experiment proves that this method is better than SA-Fast R-CNN and classical LeNet-5 CNN algorithm.

Deep Learning Convolutional Neural Network for the Retrieval of Land Surface Temperature from AMSR2 Data in China

Sensors ◽

10.3390/s19132987 ◽

2019 ◽

Vol 19 (13) ◽

pp. 2987 ◽

Cited By ~ 7

Author(s):

Jiancan Tan ◽

Nusseiba NourEldeen ◽

Kebiao Mao ◽

Jiancheng Shi ◽

Zhaoliang Li ◽

...

Keyword(s):

Neural Network ◽

Surface Temperature ◽

Land Surface Temperature ◽

Land Surface ◽

Measurement Data ◽

Retrieval Algorithm ◽

Observation Data ◽

Moderate Resolution Imaging Spectroradiometer ◽

A convolutional neural network (CNN) algorithm was developed to retrieve the land surface temperature (LST) from Advanced Microwave Scanning Radiometer 2 (AMSR2) data in China. Reference data were selected using the Moderate Resolution Imaging Spectroradiometer (MODIS) LST product to overcome the problem related to the need for synchronous ground observation data. The AMSR2 brightness temperature (TB) data and MODIS surface temperature data were randomly divided into training and test datasets, and a CNN was constructed to simulate passive microwave radiation transmission to invert the surface temperature. The twelve V/H channel combinations (7.3, 10.65, 18.7, 23.8, 36.5, 89 GHz) resulted in the most stable and accurate CNN retrieval model. Vertical polarizations performed better than horizontal polarizations; however, because CNNs rely heavily on large amounts of data, the combination of vertical and horizontal polarizations performed better than a single polarization. The retrievals in different regions indicated that the CNN accuracy was highest over large bare land areas. A comparison of the retrieval results with ground measurement data from meteorological stations yielded R2 = 0.987, RMSE = 2.69 K, and an average relative error of 2.57 K, which indicated that the accuracy of the CNN LST retrieval algorithm was high and the retrieval results can be applied to long-term LST sequence analysis in China.