scholarly journals An Effective Backward Filter Pruning Algorithm Using K1,n Bipartite Graph-Based Clustering and the Decreasing Pruning Rate Approach

2021 ◽  
Author(s):  
Kuo-Liang Chung ◽  
Yu-Lun Chang
Keyword(s):  
Neural Networks ◽  
Bipartite Graph ◽  
Binary Search ◽  
The Other ◽  
Floating Point ◽  
Clustering Method ◽  
Pruning Algorithm ◽  
Similar Accuracy ◽  
Graph Based Clustering ◽  
Pruning Methods

Setting a fixed pruning rate and/or specified threshold for pruning filters in convolutional layers has been widely used to reduce the number of parameters required in the convolutional neural networks (CNN) model. However, it fails to fully prune redundant filters for different layers whose redundant filters vary with layers. To overcome this disadvantage, we propose a new backward filter pruning algorithm using a sorted bipartite graph- and binary search-based (SBGBS-based) clustering and decreasing pruning rate (DPR) approach. We first represent each filter of the last layer by a bipartite graph 𝐾1 𝑛, with one root mean set and one 𝑛-weight set, where 𝑛 denotes the number of weights in the filter. Next, according to the accuracy loss tolerance, an SBGBS-based clustering method is used to partition all filters into clusters as maximal as possible. Then, for each cluster, we retain the filter corresponding to the bipartite graph with the median root mean among 𝑛 root means in the cluster, but we discard the other filters in the same cluster. Following the DPR approach, we repeat the above SBGBS-based filtering pruning approach to the backward layer until all layers are processed. Based on the CIFAR-10 and MNIST datasets, the proposed filter pruning algorithm has been deployed into VGG-16, AlexNet, LeNet, and ResNet. With similar accuracy, the thorough experimental results have demonstrated the substantial parameters and floating-point operations reduction merits of our filter pruning algorithm relative to the existing filter pruning methods.

scholarly journals An Effective Backward Filter Pruning Algorithm Using K1,n Bipartite Graph-Based Clustering and the Decreasing Pruning Rate Approach

2021 ◽  
Author(s):  
Kuo-Liang Chung ◽  
Yu-Lun Chang
Keyword(s):  
Neural Networks ◽  
Bipartite Graph ◽  
Binary Search ◽  
The Other ◽  
Floating Point ◽  
Clustering Method ◽  
Pruning Algorithm ◽  
Similar Accuracy ◽  
Graph Based Clustering ◽  
Pruning Methods

Setting a fixed pruning rate and/or specified threshold for pruning filters in convolutional layers has been widely used to reduce the number of parameters required in the convolutional neural networks (CNN) model. However, it fails to fully prune redundant filters for different layers whose redundant filters vary with layers. To overcome this disadvantage, we propose a new backward filter pruning algorithm using a sorted bipartite graph- and binary search-based (SBGBS-based) clustering and decreasing pruning rate (DPR) approach. We first represent each filter of the last layer by a bipartite graph 𝐾1 𝑛, with one root mean set and one 𝑛-weight set, where 𝑛 denotes the number of weights in the filter. Next, according to the accuracy loss tolerance, an SBGBS-based clustering method is used to partition all filters into clusters as maximal as possible. Then, for each cluster, we retain the filter corresponding to the bipartite graph with the median root mean among 𝑛 root means in the cluster, but we discard the other filters in the same cluster. Following the DPR approach, we repeat the above SBGBS-based filtering pruning approach to the backward layer until all layers are processed. Based on the CIFAR-10 and MNIST datasets, the proposed filter pruning algorithm has been deployed into VGG-16, AlexNet, LeNet, and ResNet. With similar accuracy, the thorough experimental results have demonstrated the substantial parameters and floating-point operations reduction merits of our filter pruning algorithm relative to the existing filter pruning methods.

A Neural Networks Pruning and Data Fusion Based Intrusion Detection Model

2014 ◽  
Vol 651-653 ◽  
pp. 1772-1775
Author(s):  
Wei Gong
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Intrusion Detection ◽  
Data Fusion ◽  
Input Data ◽  
The Other ◽  
Pruning Algorithm ◽  
Detection Model ◽  
Process Ability

The abilities of summarization, learning and self-fitting and inner-parallel computing make artificial neural networks suitable for intrusion detection. On the other hand, data fusion based IDS has been used to solve the problem of distorting rate and failing-to-report rate and improve its performance. However, multi-sensor input-data makes the IDS lose its efficiency. The research of neural network based data fusion IDS tries to combine the strong process ability of neural network with the advantages of data fusion IDS. A neural network is designed to realize the data fusion and intrusion analysis and Pruning algorithm of neural networks is used for filtering information from multi-sensors. In the process of intrusion analysis pruning algorithm of neural networks is used for filtering information from multi-sensors so as to increase its performance and save the bandwidth of networks.

Mining discriminative patches for script identification in natural scene images

2021 ◽  
Vol 40 (1) ◽  
pp. 551-563
Author(s):  
Liqiong Lu ◽  
Dong Wu ◽  
Ziwei Tang ◽  
Yaohua Yi ◽  
Faliang Huang
Keyword(s):  
Neural Networks ◽  
Experimental Results ◽  
The Other ◽  
Natural Scene ◽  
Fixed Size ◽  
Script Identification ◽  
Aspect Ratios ◽  
Novel Approach ◽  
Public Datasets ◽  
Natural Scene Images

This paper focuses on script identification in natural scene images. Traditional CNNs (Convolution Neural Networks) cannot solve this problem perfectly for two reasons: one is the arbitrary aspect ratios of scene images which bring much difficulty to traditional CNNs with a fixed size image as the input. And the other is that some scripts with minor differences are easily confused because they share a subset of characters with the same shapes. We propose a novel approach combing Score CNN, Attention CNN and patches. Attention CNN is utilized to determine whether a patch is a discriminative patch and calculate the contribution weight of the discriminative patch to script identification of the whole image. Score CNN uses a discriminative patch as input and predict the score of each script type. Firstly patches with the same size are extracted from the scene images. Secondly these patches are used as inputs to Score CNN and Attention CNN to train two patch-level classifiers. Finally, the results of multiple discriminative patches extracted from the same image via the above two classifiers are fused to obtain the script type of this image. Using patches with the same size as inputs to CNN can avoid the problems caused by arbitrary aspect ratios of scene images. The trained classifiers can mine discriminative patches to accurately identify some confusing scripts. The experimental results show the good performance of our approach on four public datasets.

scholarly journals Accelerating Spike-by-Spike Neural Networks on FPGA with Hybrid Custom Floating-Point and Logarithmic Dot-Product Approximation

IEEE Access ◽  
2021 ◽  
pp. 1-1
Author(s):  
Yarib Nevarez ◽  
David Rotermund ◽  
Klaus R. Pawelzik ◽  
Alberto Garcia-Ortiz
Keyword(s):  
Neural Networks ◽  
Floating Point ◽  
Dot Product

N. Belknap's Four-Valued Logic, Belknap Computer and New Proximity Functions for Comparing Discrete Objects

2021 ◽  
Vol 1 (2 (6)) ◽  
Author(s):  
Valerii Dmitrienko ◽  
Sergey Leonov ◽  
Mykola Mezentsev
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
The Other ◽  
Proximity Function ◽  
Input Information ◽  
Proximity Measures ◽  
The One ◽  
True Values

The idea of ​​Belknap's four-valued logic is that modern computers should function normally not only with the true values ​​of the input information, but also under the conditions of inconsistency and incompleteness of true failures. Belknap's logic introduces four true values: T (true - true), F (false - false), N (none - nobody, nothing, none), B (both - the two, not only the one but also the other).  For ease of work with these true values, the following designations are introduced: (1, 0, n, b). Belknap's logic can be used to obtain estimates of proximity measures for discrete objects, for which the functions Jaccard and Needhem, Russel and Rao, Sokal and Michener, Hamming, etc. are used. In this case, it becomes possible to assess the proximity, recognition and classification of objects in conditions of uncertainty when the true values ​​are taken from the set (1, 0, n, b). Based on the architecture of the Hamming neural network, neural networks have been developed that allow calculating the distances between objects described using true values ​​(1, 0, n, b). Keywords: four-valued Belknap logic, Belknap computer, proximity assessment, recognition and classification, proximity function, neural network.

scholarly journals Separable convolutional neural networks for facial expressions recognition

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Andry Chowanda
Keyword(s):  
Neural Networks ◽  
Social Interactions ◽  
Facial Expressions ◽  
Convolutional Neural Networks ◽  
The Other ◽  
Computational Power ◽  
Facial Cues ◽  
Emotional Recognition ◽  
Testing Accuracy ◽  
Facial Expressions Recognition

AbstractSocial interactions are important for us, humans, as social creatures. Emotions play an important part in social interactions. They usually express meanings along with the spoken utterances to the interlocutors. Automatic facial expressions recognition is one technique to automatically capture, recognise, and understand emotions from the interlocutor. Many techniques proposed to increase the accuracy of emotions recognition from facial cues. Architecture such as convolutional neural networks demonstrates promising results for emotions recognition. However, most of the current models of convolutional neural networks require an enormous computational power to train and process emotional recognition. This research aims to build compact networks with depthwise separable layers while also maintaining performance. Three datasets and three other similar architectures were used to be compared with the proposed architecture. The results show that the proposed architecture performed the best among the other architectures. It achieved up to 13% better accuracy and 6–71% smaller and more compact than the other architectures. The best testing accuracy achieved by the architecture was 99.4%.

scholarly journals INFERRING THE SCALE AND CONTENT OF A MAP USING DEEP LEARNING

2020 ◽  
Vol XLIII-B4-2020 ◽  
pp. 17-24
Author(s):  
G. Touya ◽  
F. Brisebard ◽  
F. Quinton ◽  
A. Courtial
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Visually Impaired ◽  
The Other ◽  
Visually Impaired People ◽  
Impaired People ◽  
Tactile Maps

Abstract. Visually impaired people cannot use classical maps but can learn to use tactile relief maps. These tactile maps are crucial at school to learn geography and history as well as the other students. They are produced manually by professional transcriptors in a very long and costly process. A platform able to generate tactile maps from maps scanned from geography textbooks could be extremely useful to these transcriptors, to fasten their production. As a first step towards such a platform, this paper proposes a method to infer the scale and the content of the map from its image. We used convolutional neural networks trained with a few hundred maps from French geography textbooks, and the results show promising results to infer labels about the content of the map (e.g. ”there are roads, cities and administrative boundaries”), and to infer the extent of the map (e.g. a map of France or of Europe).

Parallel Hardware for Artificial Neural Networks Using Fixed Floating Point Representation

2011 ◽  
pp. 295-308
Author(s):  
Nadia Nedjah ◽  
Rodrigo Martins da Silva ◽  
Luiza de Macedo Mourelle
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Activation Function ◽  
Floating Point ◽  
Weighted Sum ◽  
Design Environment ◽  
Silicon Area ◽  
Point Representation ◽  
Artificial Neural ◽  
Mathematical Computation

Artificial Neural Networks (ANNs) is a well known bio-inspired model that simulates human brain capabilities such as learning and generalization. ANNs consist of a number of interconnected processing units, wherein each unit performs a weighted sum followed by the evaluation of a given activation function. The involved computation has a tremendous impact on the implementation efficiency. Existing hardware implementations of ANNs attempt to speed up the computational process. However, these implementations require a huge silicon area that makes it almost impossible to fit within the resources available on a state-of-the-art FPGAs. In this chapter, a hardware architecture for ANNs that takes advantage of the dedicated adder blocks, commonly called MACs, to compute both the weighted sum and the activation function is devised. The proposed architecture requires a reduced silicon area considering the fact that the MACs come for free as these are FPGA’s built-in cores. Our system uses integer (fixed point) mathematics and operates with fractions to represent real numbers. Hence, floating point representation is not employed and any mathematical computation of the ANN hardware is based on combinational circuitry (performing only sums and multiplications). The hardware is fast because it is massively parallel. Besides, the proposed architecture can adjust itself on-the-fly to the user-defined configuration of the neural network, i.e., the number of layers and neurons per layer of the ANN can be settled with no extra hardware changes. This is a very nice characteristic in robot-like systems considering the possibility of the same hardware may be exploited in different tasks. The hardware also requires another system (a software) that controls the sequence of the hardware computation and provides inputs, weights and biases for the ANN in hardware. Thus, a co-design environment is necessary.

Sign in / Sign up

Export Citation Format

Share Document