The Impact of Soft Errors in Memory Units of Edge Devices Executing Convolutional Neural Networks

2022 ◽  
pp. 1-1
Author(s):  
Geancarlo Abich ◽  
Rafael Garibotti ◽  
Ricardo Reis ◽  
Luciano Ost
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Soft Errors ◽  
The Impact

scholarly journals Rectified Wing Loss for Efficient and Robust Facial Landmark Localisation with Convolutional Neural Networks

2019 ◽  
Vol 128 (8-9) ◽  
pp. 2126-2145 ◽  
Author(s):  
Zhen-Hua Feng ◽  
Josef Kittler ◽  
Muhammad Awais ◽  
Xiao-Jun Wu
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Loss Function ◽  
Data Augmentation ◽  
Small Sample Size ◽  
Small Sample ◽  
Imbalance Problem ◽  
Facial Landmark ◽  
The Impact ◽  
Coarse To Fine

AbstractEfficient and robust facial landmark localisation is crucial for the deployment of real-time face analysis systems. This paper presents a new loss function, namely Rectified Wing (RWing) loss, for regression-based facial landmark localisation with Convolutional Neural Networks (CNNs). We first systemically analyse different loss functions, including L2, L1 and smooth L1. The analysis suggests that the training of a network should pay more attention to small-medium errors. Motivated by this finding, we design a piece-wise loss that amplifies the impact of the samples with small-medium errors. Besides, we rectify the loss function for very small errors to mitigate the impact of inaccuracy of manual annotation. The use of our RWing loss boosts the performance significantly for regression-based CNNs in facial landmarking, especially for lightweight network architectures. To address the problem of under-representation of samples with large pose variations, we propose a simple but effective boosting strategy, referred to as pose-based data balancing. In particular, we deal with the data imbalance problem by duplicating the minority training samples and perturbing them by injecting random image rotation, bounding box translation and other data augmentation strategies. Last, the proposed approach is extended to create a coarse-to-fine framework for robust and efficient landmark localisation. Moreover, the proposed coarse-to-fine framework is able to deal with the small sample size problem effectively. The experimental results obtained on several well-known benchmarking datasets demonstrate the merits of our RWing loss and prove the superiority of the proposed method over the state-of-the-art approaches.

Deep Transfer Learning for Histopathological Diagnosis of Cervical Cancer Using Convolutional Neural Networks with Visualization Schemes

2020 ◽  
Vol 10 (2) ◽  
pp. 391-400 ◽  
Author(s):  
Ying Chen ◽  
Xiaomin Qin ◽  
Jingyu Xiong ◽  
Shugong Xu ◽  
Jun Shi ◽  
...  
Keyword(s):  
Neural Networks ◽  
Cervical Cancer ◽  
Learning Strategies ◽  
Transfer Learning ◽  
Convolutional Neural Networks ◽  
Characteristic Curve ◽  
Disease Classification ◽  
Natural Image ◽  
Histopathological Diagnosis ◽  
The Impact

This study aimed to propose a deep transfer learning framework for histopathological image analysis by using convolutional neural networks (CNNs) with visualization schemes, and to evaluate its usage for automated and interpretable diagnosis of cervical cancer. First, in order to examine the potential of the transfer learning for classifying cervix histopathological images, we pre-trained three state-of-the-art CNN architectures on large-size natural image datasets and then fine-tuned them on small-size histopathological datasets. Second, we investigated the impact of three learning strategies on classification accuracy. Third, we visualized both the multiple-layer convolutional kernels of CNNs and the regions of interest so as to increase the clinical interpretability of the networks. Our method was evaluated on a database of 4993 cervical histological images (2503 benign and 2490 malignant). The experimental results demonstrated that our method achieved 95.88% sensitivity, 98.93% specificity, 97.42% accuracy, 94.81% Youden's index and 99.71% area under the receiver operating characteristic curve. Our method can reduce the cognitive burden on pathologists for cervical disease classification and improve their diagnostic efficiency and accuracy. It may be potentially used in clinical routine for histopathological diagnosis of cervical cancer.

scholarly journals Convolutional Neural Networks for Decoding of Covert Attention Focus and Saliency Maps for EEG Feature Visualization

2019 ◽  
Author(s):  
Amr Farahat ◽  
Christoph Reichert ◽  
Catherine M. Sweeney-Reed ◽  
Hermann Hinrichs
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Cognitive Task ◽  
Event Related Potentials ◽  
Eeg Signal ◽  
Linear Discriminant ◽  
Saliency Maps ◽  
Significant Difference ◽  
Related Potentials ◽  
The Impact

ABSTRACTObjectiveConvolutional neural networks (CNNs) have proven successful as function approximators and have therefore been used for classification problems including electroencephalography (EEG) signal decoding for brain-computer interfaces (BCI). Artificial neural networks, however, are considered black boxes, because they usually have thousands of parameters, making interpretation of their internal processes challenging. Here we systematically evaluate the use of CNNs for EEG signal decoding and investigate a method for visualizing the CNN model decision process.ApproachWe developed a CNN model to decode the covert focus of attention from EEG event-related potentials during object selection. We compared the CNN and the commonly used linear discriminant analysis (LDA) classifier performance, applied to datasets with different dimensionality, and analyzed transfer learning capacity. Moreover, we validated the impact of single model components by systematically altering the model. Furthermore, we investigated the use of saliency maps as a tool for visualizing the spatial and temporal features driving the model output.Main resultsThe CNN model and the LDA classifier achieved comparable accuracy on the lower-dimensional dataset, but CNN exceeded LDA performance significantly on the higher-dimensional dataset (without hypothesis-driven preprocessing), achieving an average decoding accuracy of 90.7% (chance level = 8.3%). Parallel convolutions, tanh or ELU activation functions, and dropout regularization proved valuable for model performance, whereas the sequential convolutions, ReLU activation function, and batch normalization components, reduced accuracy or yielded no significant difference. Saliency maps revealed meaningful features, displaying the typical spatial distribution and latency of the P300 component expected during this task.SignificanceFollowing systematic evaluation, we provide recommendations for when and how to use CNN models in EEG decoding. Moreover, we propose a new approach for investigating the neural correlates of a cognitive task by training CNN models on raw high-dimensional EEG data and utilizing saliency maps for relevant feature extraction.

scholarly journals Aerial Images Processing for Car Detection using Convolutional Neural Networks: Comparison between Faster R-CNN and YoloV3

2019 ◽  
Author(s):  
Adel Ammar ◽  
Anis Koubaa ◽  
Mohanned Ahmed ◽  
Abdulrahman Saad
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Detection Algorithm ◽  
Aerial Images ◽  
Object Size ◽  
Car Detection ◽  
Images Processing ◽  
Different Characteristics ◽  
The Impact

In this paper, we address the problem of car detection from aerial images using Convolutional Neural Networks (CNN). This problem presents additional challenges as compared to car (or any object) detection from ground images because features of vehicles from aerial images are more difficult to discern. To investigate this issue, we assess the performance of two state-of-the-art CNN algorithms, namely Faster R-CNN, which is the most popular region-based algorithm, and YOLOv3, which is known to be the fastest detection algorithm. We analyze two datasets with different characteristics to check the impact of various factors, such as UAV’s altitude, camera resolution, and object size. The objective of this work is to conduct a robust comparison between these two cutting-edge algorithms. By using a variety of metrics, we show that none of the two algorithms outperforms the other in all cases.

scholarly journals Implementation of Lightweight Convolutional Neural Networks via Layer-Wise Differentiable Compression

Sensors ◽  
2021 ◽  
Vol 21 (10) ◽  
pp. 3464
Author(s):  
Huabin Diao ◽  
Yuexing Hao ◽  
Shaoyun Xu ◽  
Gongyan Li
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Compression Rate ◽  
Software Support ◽  
Model Size ◽  
Significant Compression ◽  
Selection Operator ◽  
The Impact ◽  
Constrained Devices

Convolutional neural networks (CNNs) have achieved significant breakthroughs in various domains, such as natural language processing (NLP), and computer vision. However, performance improvement is often accompanied by large model size and computation costs, which make it not suitable for resource-constrained devices. Consequently, there is an urgent need to compress CNNs, so as to reduce model size and computation costs. This paper proposes a layer-wise differentiable compression (LWDC) algorithm for compressing CNNs structurally. A differentiable selection operator OS is embedded in the model to compress and train the model simultaneously by gradient descent in one go. Instead of pruning parameters from redundant operators by contrast to most of the existing methods, our method replaces the original bulky operators with more lightweight ones directly, which only needs to specify the set of lightweight operators and the regularization factor in advance, rather than the compression rate for each layer. The compressed model produced by our method is generic and does not need any special hardware/software support. Experimental results on CIFAR-10, CIFAR-100 and ImageNet have demonstrated the effectiveness of our method. LWDC obtains more significant compression than state-of-the-art methods in most cases, while having lower performance degradation. The impact of lightweight operators and regularization factor on the compression rate and accuracy also is evaluated.

Sign in / Sign up

Export Citation Format

Share Document