scholarly journals Combining Learnable Low-dimensional Binary Filter Bases for Compressing Convolutional Neural Networks

2021 ◽  
Author(s):  
Weichao Lan ◽  
Yiu-ming Cheung ◽  
Liang Lan
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Real Life ◽  
Embedded Devices ◽  
Compression Performance ◽  
Benchmark Datasets ◽  
Significant Performance ◽  
Low Dimensional ◽  
Convolution Filters ◽  
Binary Filter

Existing convolutional neural networks (CNNs) have achieved significant performance on various real-life tasks, but a large number of parameters in convolutional layers requires huge storage and computation resources which makes it difficult to deploy CNNs on memory-constraint embedded devices. In this paper, we propose a novel compression method that generates the convolution filters in each layer by combining a set of learnable low-dimensional binary filter bases. The proposed method designs more compact convolution filters by stacking the linear combinations of these filter bases. Because of binary filters, the compact filters can be represented using less number of bits so that the network can be highly compressed. Furthermore, we explore the sparsity of coefficient through L1-ball projection when conducting linear combination to avoid overfitting. In addition, we analyze the compression performance of the proposed method in detail. Evaluations on four benchmark datasets under VGG-16 and ResNet-18 structures show that the proposed method can achieve a higher compression ratio with comparable accuracy compared with the existing state-of-the-art filter decomposition and network quantization methods.

scholarly journals Combining Learnable Low-dimensional Binary Filter Bases for Compressing Convolutional Neural Networks

2021 ◽  
Author(s):  
Weichao Lan ◽  
Yiu-ming Cheung ◽  
Liang Lan
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Real Life ◽  
Embedded Devices ◽  
Compression Performance ◽  
Benchmark Datasets ◽  
Significant Performance ◽  
Low Dimensional ◽  
Convolution Filters ◽  
Binary Filter

Existing convolutional neural networks (CNNs) have achieved significant performance on various real-life tasks, but a large number of parameters in convolutional layers requires huge storage and computation resources which makes it difficult to deploy CNNs on memory-constraint embedded devices. In this paper, we propose a novel compression method that generates the convolution filters in each layer by combining a set of learnable low-dimensional binary filter bases. The proposed method designs more compact convolution filters by stacking the linear combinations of these filter bases. Because of binary filters, the compact filters can be represented using less number of bits so that the network can be highly compressed. Furthermore, we explore the sparsity of coefficient through L1-ball projection when conducting linear combination to avoid overfitting. In addition, we analyze the compression performance of the proposed method in detail. Evaluations on four benchmark datasets under VGG-16 and ResNet-18 structures show that the proposed method can achieve a higher compression ratio with comparable accuracy compared with the existing state-of-the-art filter decomposition and network quantization methods.

scholarly journals Hybrid Mamdani Fuzzy Rules and Convolutional Neural Networks for Analysis and Identification of Animal Images

Computation ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 35
Author(s):  
Hind R. Mohammed ◽  
Zahir M. Hussain
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
High Speed ◽  
Moving Objects ◽  
Real Life ◽  
Fuzzy Rules ◽  
High Rate ◽  
Moving Images ◽  
Animal Images

Accurate, fast, and automatic detection and classification of animal images is challenging, but it is much needed for many real-life applications. This paper presents a hybrid model of Mamdani Type-2 fuzzy rules and convolutional neural networks (CNNs) applied to identify and distinguish various animals using different datasets consisting of about 27,307 images. The proposed system utilizes fuzzy rules to detect the image and then apply the CNN model for the object’s predicate category. The CNN model was trained and tested based on more than 21,846 pictures of animals. The experiments’ results of the proposed method offered high speed and efficiency, which could be a prominent aspect in designing image-processing systems based on Type 2 fuzzy rules characterization for identifying fixed and moving images. The proposed fuzzy method obtained an accuracy rate for identifying and recognizing moving objects of 98% and a mean square error of 0.1183464 less than other studies. It also achieved a very high rate of correctly predicting malicious objects equal to recall = 0.98121 and a precision rate of 1. The test’s accuracy was evaluated using the F1 Score, which obtained a high percentage of 0.99052.

scholarly journals Deep Learning for Detecting Building Defects Using Convolutional Neural Networks

Sensors ◽  
2019 ◽  
Vol 19 (16) ◽  
pp. 3556 ◽  
Author(s):  
Husein Perez ◽  
Joseph H. M. Tah ◽  
Amir Mosavi
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Effective Means ◽  
Scale Up ◽  
Real Life ◽  
Health And Safety ◽  
Condition Assessment ◽  
Automated Detection ◽  
Proposed Model ◽  
Current Asset

Clients are increasingly looking for fast and effective means to quickly and frequently survey and communicate the condition of their buildings so that essential repairs and maintenance work can be done in a proactive and timely manner before it becomes too dangerous and expensive. Traditional methods for this type of work commonly comprise of engaging building surveyors to undertake a condition assessment which involves a lengthy site inspection to produce a systematic recording of the physical condition of the building elements, including cost estimates of immediate and projected long-term costs of renewal, repair and maintenance of the building. Current asset condition assessment procedures are extensively time consuming, laborious, and expensive and pose health and safety threats to surveyors, particularly at height and roof levels which are difficult to access. This paper aims at evaluating the application of convolutional neural networks (CNN) towards an automated detection and localisation of key building defects, e.g., mould, deterioration, and stain, from images. The proposed model is based on pre-trained CNN classifier of VGG-16 (later compaired with ResNet-50, and Inception models), with class activation mapping (CAM) for object localisation. The challenges and limitations of the model in real-life applications have been identified. The proposed model has proven to be robust and able to accurately detect and localise building defects. The approach is being developed with the potential to scale-up and further advance to support automated detection of defects and deterioration of buildings in real-time using mobile devices and drones.

scholarly journals Robust Image Classification with Cognitive-Driven Color Priors

Electronics ◽  
2020 ◽  
Vol 9 (11) ◽  
pp. 1837
Author(s):  
Peng Gu ◽  
Chengfei Zhu ◽  
Xiaosong Lan ◽  
Jie Wang ◽  
Shuxiao Li
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Human Memory ◽  
Layer By Layer ◽  
Training Methods ◽  
Prior Model ◽  
Benchmark Datasets ◽  
Robust Image ◽  
Classification Probability

Existing image classification methods based on convolutional neural networks usually use a large number of samples to learn classification features hierarchically, causing the problems of over-fitting and error propagation layer by layer. Thus, they are vulnerable to adversarial samples generated by adding imperceptible disturbances to input samples. To address the above issue, we propose a cognitive-driven color prior model to memorize the color attributes of target samples inspired by the characteristics of human memory. At inference stage, color priors are indexed from the memory and fused with features of convolutional neural networks to achieve robust image classification. The proposed color prior model is cognitive-driven and has no training parameters, thus it has strong generalization and can effectively defend against adversarial samples. In addition, our method directly combines the features of the prior model with the classification probability of the convolutional neural network, without changing the network structure and its parameters of the existing algorithm. It can be combined with other adversarial attack defense methods, such as various preprocessing modules such as PixelDefense or adversarial training methods, to improve the robustness of image classification. Experiments on several benchmark datasets show that the proposed method improves the anti-interference ability of image classification algorithms.

scholarly journals Multiactivation Pooling Method in Convolutional Neural Networks for Image Recognition

2018 ◽  
Vol 2018 ◽  
pp. 1-15 ◽  
Author(s):  
Qi Zhao ◽  
Shuchang Lyu ◽  
Boxue Zhang ◽  
Wenquan Feng
Keyword(s):  
Neural Networks ◽  
Image Processing ◽  
Big Data ◽  
Convolutional Neural Networks ◽  
Image Recognition ◽  
Large Scale ◽  
Fog Computing ◽  
Feature Extractor ◽  
Benchmark Datasets ◽  
Classification Tasks

Convolutional neural networks (CNNs) are becoming more and more popular today. CNNs now have become a popular feature extractor applying to image processing, big data processing, fog computing, etc. CNNs usually consist of several basic units like convolutional unit, pooling unit, activation unit, and so on. In CNNs, conventional pooling methods refer to 2×2 max-pooling and average-pooling, which are applied after the convolutional or ReLU layers. In this paper, we propose a Multiactivation Pooling (MAP) Method to make the CNNs more accurate on classification tasks without increasing depth and trainable parameters. We add more convolutional layers before one pooling layer and expand the pooling region to 4×4, 8×8, 16×16, and even larger. When doing large-scale subsampling, we pick top-k activation, sum up them, and constrain them by a hyperparameter σ. We pick VGG, ALL-CNN, and DenseNets as our baseline models and evaluate our proposed MAP method on benchmark datasets: CIFAR-10, CIFAR-100, SVHN, and ImageNet. The classification results are competitive.

A CONVblock for Convolutional Neural Networks

2021 ◽  
pp. 100-113
Author(s):  
Hmidi Alaeddine ◽  
Malek Jihene
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Experimental Results ◽  
Classification Models ◽  
Deep Architecture ◽  
Improved Performance ◽  
Convolution Filters ◽  
Classification Database

The reduction in the size of convolution filters has been shown to be effective in image classification models. They make it possible to reduce the calculation and the number of parameters used in the operations of the convolution layer while increasing the efficiency of the representation. The authors present a deep architecture for classification with improved performance. The main objective of this architecture is to improve the main performances of the network thanks to a new design based on CONVblock. The proposal is evaluated on a classification database: CIFAR-10 and MNIST. The experimental results demonstrate the effectiveness of the proposed method. This architecture offers an error of 1.4% on CIFAR-10 and 0.055% on MNIST.

Hybrid pooling with wavelets for convolutional neural networks

2022 ◽  
pp. 1-10
Author(s):  
Daniel Trevino-Sanchez ◽  
Vicente Alarcon-Aquino
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Computational Cost ◽  
Relevant Information ◽  
Accuracy Improvement ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
Augmentation Techniques ◽  
High Computational Cost

The need to detect and classify objects correctly is a constant challenge, being able to recognize them at different scales and scenarios, sometimes cropped or badly lit is not an easy task. Convolutional neural networks (CNN) have become a widely applied technique since they are completely trainable and suitable to extract features. However, the growing number of convolutional neural networks applications constantly pushes their accuracy improvement. Initially, those improvements involved the use of large datasets, augmentation techniques, and complex algorithms. These methods may have a high computational cost. Nevertheless, feature extraction is known to be the heart of the problem. As a result, other approaches combine different technologies to extract better features to improve the accuracy without the need of more powerful hardware resources. In this paper, we propose a hybrid pooling method that incorporates multiresolution analysis within the CNN layers to reduce the feature map size without losing details. To prevent relevant information from losing during the downsampling process an existing pooling method is combined with wavelet transform technique, keeping those details "alive" and enriching other stages of the CNN. Achieving better quality characteristics improves CNN accuracy. To validate this study, ten pooling methods, including the proposed model, are tested using four benchmark datasets. The results are compared with four of the evaluated methods, which are also considered as the state-of-the-art.

scholarly journals Image Steganalysis via Diverse Filters and Squeeze-and-Excitation Convolutional Neural Network

Mathematics ◽  
2021 ◽  
Vol 9 (2) ◽  
pp. 189
Author(s):  
Feng Liu ◽  
Xuan Zhou ◽  
Xuehu Yan ◽  
Yuliang Lu ◽  
Shudong Wang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Image Steganalysis ◽  
Process Information ◽  
Effective Network ◽  
Convolution Filters

Steganalysis is a method to detect whether the objects contain secret messages. With the popularity of deep learning, using convolutional neural networks (CNNs), steganalytic schemes have become the chief method of combating steganography in recent years. However, the diversity of filters has not been fully utilized in the current research. This paper constructs a new effective network with diverse filter modules (DFMs) and squeeze-and-excitation modules (SEMs), which can better capture the embedding artifacts. As the essential parts, combining three different scale convolution filters, DFMs can process information diversely, and the SEMs can enhance the effective channels out from DFMs. The experiments presented that our CNN is effective against content-adaptive steganographic schemes with different payloads, such as S-UNIWARD and WOW algorithms. Moreover, some state-of-the-art methods are compared with our approach to demonstrate the outstanding performance.

Sign in / Sign up

Export Citation Format

Share Document