Dynamic Object Detection Algorithm Based on Lightweight Shared Feature Pyramid

Current object detection algorithms perform inference on all samples at a fixed computational cost in the inference stage, which wastes computing resources and is not flexible. To solve this problem, a dynamic object detection algorithm based on a lightweight shared feature pyramid is proposed, which performs adaptive inference according to computing resources and the difficulty of samples, greatly improving the efficiency of inference. Specifically, a lightweight shared feature pyramid network and lightweight detection head is proposed to reduce the amount of computation and parameters in the feature fusion part and detection head of the dynamic object detection model. On the PASCAL VOC dataset, under the two conditions of “anytime prediction” and “budgeted batch object detection”, the performance, computation amount and parameter amount are better than the dynamic object detection models constructed by networks such as ResNet, DenseNet and MSDNet.

Synergistically Exploiting CNN Pruning and HLS Versioning for Adaptive Inference on Multi-FPGAs at the Edge

FPGAs, because of their energy efficiency, reconfigurability, and easily tunable HLS designs, have been used to accelerate an increasing number of machine learning, especially CNN-based, applications. As a representative example, IoT Edge applications, which require low latency processing of resource-hungry CNNs, offload the inferences from resource-limited IoT end nodes to Edge servers featuring FPGAs. However, the ever-increasing number of end nodes pressures these FPGA-based servers with new performance and adaptability challenges. While some works have exploited CNN optimizations to alleviate inferences’ computation and memory burdens, others have exploited HLS to tune accelerators for statically defined optimization goals. However, these works have not tackled both CNN and HLS optimizations altogether; neither have they provided any adaptability at runtime, where the workload’s characteristics are unpredictable. In this context, we propose a hybrid two-step approach that, first, creates new optimization opportunities at design-time through the automatic training of CNN model variants (obtained via pruning) and the automatic generation of versions of convolutional accelerators (obtained during HLS synthesis); and, second, synergistically exploits these created CNN and HLS optimization opportunities to deliver a fully dynamic Multi-FPGA system that adapts its resources in a fully automatic or user-configurable manner. We implement this two-step approach as the AdaServ Framework and show, through a smart video surveillance Edge application as a case study, that it adapts to the always-changing Edge conditions: AdaServ processes at least 3.37× more inferences (using the automatic approach) and is at least 6.68× more energy-efficient (user-configurable approach) than original convolutional accelerators and CNN Models (VGG-16 and AlexNet). We also show that AdaServ achieves better results than solutions dynamically changing only the CNN model or HLS version, highlighting the importance of exploring both; and that it is always better than the best statically chosen CNN model and HLS version, showing the need for dynamic adaptability.

Mitigating State-Drift in Memristor Crossbar Arrays for Vector Matrix Multiplication

In this Chapter, we review the recent progress on resistance drift mitigation techniques for resistive switching memory devices (specifically memristors) and its impact on the accuracy in deep neural network applications. In the first section of the chapter, we investigate the importance of soft errors and their detrimental impact on memristor-based vector–matrix multiplication (VMM) platforms performance specially the memristance state-drift induced by long-term recurring inference operations with sub-threshold stress voltage. Also, we briefly review some currently developed state-drift mitigation methods. In the next section of the chapter, we will discuss an adaptive inference technique with low hardware overhead to mitigate the memristance drift in memristive VMM platform by using optimization techniques to adjust the inference voltage characteristic associated with different network layers. Also, we present simulation results and performance improvements achieved by applying the proposed inference technique by considering non-idealities for various deep network applications on memristor crossbar arrays. This chapter suggests that a simple low overhead inference technique can revive the functionality, enhance the performance of memristor-based VMM arrays and significantly increases their lifetime which can be a very important factor toward making this technology as a main stream player in future in-memory computing platforms.

A Fuzzy Radial Basis Adaptive Inference Network and Its Application to Time-Varying Signal Classification

A fuzzy radial basis adaptive inference network (FRBAIN) is proposed for multichannel time-varying signal fusion analysis and feature knowledge embedding. The model which combines the prior signal feature embedding mechanism of the radial basis kernel function with the rule-based logic inference ability of fuzzy system is composed of a multichannel time-varying signal input layer, a radial basis fuzzification layer, a rule layer, a regularization layer, and a T-S fuzzy classifier layer. The dynamic fuzzy clustering algorithm was used to divide the sample set pattern class into several subclasses with similar features. The fuzzy radial basis neurons (FRBNs) were defined and used as parameterized membership functions, and typical feature samples of each pattern subclass were used as kernel centers of the FRBN to realize the embedding of the diverse prior feature knowledge and the fuzzification of the input signals. According to the signal categories of FRBN kernel centers, nodes in the rule layer were selectively connected with nodes in the FRBN layer. A fuzzy multiplication operation was used to achieve synthesis of pattern class membership information and establishment of fuzzy inference rules. The excitation intensity of each rule was used as the input of T-S fuzzy classifier to classify the input signals. The FRBAIN can adaptively establish fuzzy set membership functions, fuzzy inference, and classification rules based on the learning of sample set, realize structural and data constraints of the model, and improve the modeling properties of imbalanced datasets. In this paper, the properties of FRBAIN were analyzed and a comprehensive learning algorithm was established. Experimental validation was performed with classification diagnoses from four complex cardiovascular diseases based on 12-lead ECG signals. Results demonstrated that, in the case of small-scale imbalanced datasets, the proposed method significantly improved both classification accuracy and generalizability comparing with other methods in the experiment.