FTT-NAS: Discovering Fault-tolerant Convolutional Neural Architecture

With the fast evolvement of embedded deep-learning computing systems, applications powered by deep learning are moving from the cloud to the edge. When deploying neural networks (NNs) onto the devices under complex environments, there are various types of possible faults: soft errors caused by cosmic radiation and radioactive impurities, voltage instability, aging, temperature variations, malicious attackers, and so on. Thus, the safety risk of deploying NNs is now drawing much attention. In this article, after the analysis of the possible faults in various types of NN accelerators, we formalize and implement various fault models from the algorithmic perspective. We propose Fault-Tolerant Neural Architecture Search (FT-NAS) to automatically discover convolutional neural network (CNN) architectures that are reliable to various faults in nowadays devices. Then, we incorporate fault-tolerant training (FTT) in the search process to achieve better results, which is referred to as FTT-NAS. Experiments on CIFAR-10 show that the discovered architectures outperform other manually designed baseline architectures significantly, with comparable or fewer floating-point operations (FLOPs) and parameters. Specifically, with the same fault settings, F-FTT-Net discovered under the feature fault model achieves an accuracy of 86.2% (VS. 68.1% achieved by MobileNet-V2), and W-FTT-Net discovered under the weight fault model achieves an accuracy of 69.6% (VS. 60.8% achieved by ResNet-18). By inspecting the discovered architectures, we find that the operation primitives, the weight quantization range, the capacity of the model, and the connection pattern have influences on the fault resilience capability of NN models.

Download Full-text

A Study on Fault Model and Fault-Tolerant Routing for Sensor Networks

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.182-183.1265 ◽

2012 ◽

Vol 182-183 ◽

pp. 1265-1269

Author(s):

Zu Ming Xu ◽

Xiong Fu

Keyword(s):

Sensor Networks ◽

Routing Protocols ◽

Energy Efficient ◽

Fault Tolerant ◽

Network Reliability ◽

Fault Model ◽

Cross Layer ◽

Cross Layer Design ◽

Fault Models ◽

Energy Aware

Wireless sensor networks require energy-efficient and robust routingprotocols. Most routing protocols for sensor networks try to extendnetwork lifetime by minimizing the energy consumption, but have not taken the network reliability into account. In this paper, we analyze the fault models and propose an ENergy-aware FAult-tolerantRouting scheme, termed as ENFAR. Firstly a link-based uniform fault model is presented, and we adopt a cross-layer design to measurethe transmission delay so as to detect the failed nodes.

Download Full-text

Efficient Resource-Aware Convolutional Neural Architecture Search for Edge Computing with Pareto-Bayesian Optimization

Sensors ◽

10.3390/s21020444 ◽

2021 ◽

Vol 21 (2) ◽

pp. 444

Author(s):

Zhao Yang ◽

Shengbing Zhang ◽

Ruxu Li ◽

Chuxi Li ◽

Miao Wang ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Network Structure ◽

High Accuracy ◽

Search Direction ◽

Search Process ◽

Search Efficiency ◽

Neural Architecture ◽

Efficient Resource ◽

Resource Aware

With the development of deep learning technologies and edge computing, the combination of them can make artificial intelligence ubiquitous. Due to the constrained computation resources of the edge device, the research in the field of on-device deep learning not only focuses on the model accuracy but also on the model efficiency, for example, inference latency. There are many attempts to optimize the existing deep learning models for the purpose of deploying them on the edge devices that meet specific application requirements while maintaining high accuracy. Such work not only requires professional knowledge but also needs a lot of experiments, which limits the customization of neural networks for varied devices and application scenarios. In order to reduce the human intervention in designing and optimizing the neural network structure, multi-objective neural architecture search methods that can automatically search for neural networks featured with high accuracy and can satisfy certain hardware performance requirements are proposed. However, the current methods commonly set accuracy and inference latency as the performance indicator during the search process, and sample numerous network structures to obtain the required neural network. Lacking regulation to the search direction with the search objectives will generate a large number of useless networks during the search process, which influences the search efficiency to a great extent. Therefore, in this paper, an efficient resource-aware search method is proposed. Firstly, the network inference consumption profiling model for any specific device is established, and it can help us directly obtain the resource consumption of each operation in the network structure and the inference latency of the entire sampled network. Next, on the basis of the Bayesian search, a resource-aware Pareto Bayesian search is proposed. Accuracy and inference latency are set as the constraints to regulate the search direction. With a clearer search direction, the overall search efficiency will be improved. Furthermore, cell-based structure and lightweight operation are applied to optimize the search space for further enhancing the search efficiency. The experimental results demonstrate that with our method, the inference latency of the searched network structure reduced 94.71% without scarifying the accuracy. At the same time, the search efficiency increased by 18.18%.

Download Full-text

Decentralized Validation for Non-malicious Arbitrary Fault Tolerance in Paxos

10.5753/wtf.2019.7713 ◽

2019 ◽

Author(s):

Rodrigo Barbieri ◽

Enrique dos Santos ◽

Gustavo Maciel Dias Vieira

Keyword(s):

Fault Tolerant ◽

Fault Injection ◽

High Reliability ◽

Fault Model ◽

Basic Type ◽

Fault Models ◽

Middle Ground ◽

Process Crash ◽

Usual Solution ◽

Hardware Failures

Fault-tolerant distributed systems offer high reliability because even if faults in their components occur, they do not exhibit erroneous behavior. Depending on the fault model adopted, hardware and software errors that do not result in a process crashing are usually not tolerated. To tolerate these rather common failures the usual solution is to adopt a stronger fault model, such as the arbitrary or Byzantine fault model. Algorithms created for this fault model, however, are considerably more complex and require more system resources than the ones developed for less strict fault models. One approach to reach a middle ground is the non-malicious arbitrary fault model. In this paper we describe how we incremented an implementation of active replication in the non-malicious fault model with a basic type of distributed validation, where a deviation from the expected algorithm behavior will make a process crash. We experimentally evaluate this implementation using a fault injection framework showing that it is feasible to extend the concept of non-malicious failures beyond hardware failures.

Download Full-text

2D and 3D Palmprint and Palm Vein Recognition Based on Neural Architecture Search

International Journal of Automation and Computing ◽

10.1007/s11633-021-1292-1 ◽

2021 ◽

Author(s):

Wei Jia ◽

Wei Xia ◽

Yang Zhao ◽

Hai Min ◽

Yan-Xiang Chen

Keyword(s):

Deep Learning ◽

Recognition Performance ◽

Research Direction ◽

Palmprint Recognition ◽

Neural Architecture ◽

Development Direction ◽

Vein Recognition ◽

Palm Vein ◽

2D And 3D ◽

Important Research Direction

AbstractPalmprint recognition and palm vein recognition are two emerging biometrics technologies. In the past two decades, many traditional methods have been proposed for palmprint recognition and palm vein recognition and have achieved impressive results. In recent years, in the field of artificial intelligence, deep learning has gradually become the mainstream recognition technology because of its excellent recognition performance. Some researchers have tried to use convolutional neural networks (CNNs) for palmprint recognition and palm vein recognition. However, the architectures of these CNNs have mostly been developed manually by human experts, which is a time-consuming and error-prone process. In order to overcome some shortcomings of manually designed CNN, neural architecture search (NAS) technology has become an important research direction of deep learning. The significance of NAS is to solve the deep learning model’s parameter adjustment problem, which is a cross-study combining optimization and machine learning. NAS technology represents the future development direction of deep learning. However, up to now, NAS technology has not been well studied for palmprint recognition and palm vein recognition. In this paper, in order to investigate the problem of NAS-based 2D and 3D palmprint recognition and palm vein recognition in-depth, we conduct a performance evaluation of twenty representative NAS methods on five 2D palmprint databases, two palm vein databases, and one 3D palmprint database. Experimental results show that some NAS methods can achieve promising recognition results. Remarkably, among different evaluated NAS methods, ProxylessNAS achieves the best recognition performance.

Download Full-text

A Systematic Deep Learning Based Overhead Tracking and Counting System Using RGB-D Remote Cameras

Applied Sciences ◽

10.3390/app11125503 ◽

2021 ◽

Vol 11 (12) ◽

pp. 5503

Author(s):

Munkhjargal Gochoo ◽

Syeda Amna Rizwan ◽

Yazeed Yasin Ghadi ◽

Ahmad Jalal ◽

Kibum Kim

Keyword(s):

Deep Learning ◽

Human Head ◽

Point Clouds ◽

Head Tracking ◽

Complex Environments ◽

People Tracking ◽

Practical Applications ◽

Remote Cameras ◽

Video Frames ◽

Benchmark Datasets

Automatic head tracking and counting using depth imagery has various practical applications in security, logistics, queue management, space utilization and visitor counting. However, no currently available system can clearly distinguish between a human head and other objects in order to track and count people accurately. For this reason, we propose a novel system that can track people by monitoring their heads and shoulders in complex environments and also count the number of people entering and exiting the scene. Our system is split into six phases; at first, preprocessing is done by converting videos of a scene into frames and removing the background from the video frames. Second, heads are detected using Hough Circular Gradient Transform, and shoulders are detected by HOG based symmetry methods. Third, three robust features, namely, fused joint HOG-LBP, Energy based Point clouds and Fused intra-inter trajectories are extracted. Fourth, the Apriori-Association is implemented to select the best features. Fifth, deep learning is used for accurate people tracking. Finally, heads are counted using Cross-line judgment. The system was tested on three benchmark datasets: the PCDS dataset, the MICC people counting dataset and the GOTPD dataset and counting accuracy of 98.40%, 98%, and 99% respectively was achieved. Our system obtained remarkable results.

Download Full-text