Multi-Feature Fusion with Convolutional Neural Network for Ship Classification in Optical Images

The appearance of ships is easily affected by external factors—illumination, weather conditions, and sea state—that make ship classification a challenging task. To facilitate realization of enhanced ship-classification performance, this study proposes a ship classification method based on multi-feature fusion with a convolutional neural network (CNN). First, an improved CNN characterized by shallow layers and few parameters is proposed to learn high-level features and capture structural information. Second, handcrafted features of the histogram of oriented gradients (HOG) and local binary patterns (LBP) are combined with high-level features extracted by the improved CNN in the last fully connected layer to obtain discriminative feature representation. The handcrafted features supplement the edge information and spatial texture information of the ship images. Then, the Softmax function is used to classify different types of ships in the output layer. Effectiveness of the proposed method is evaluated based on its application to two datasets—one self-built and the other publicly available, called visible and infrared spectrums (VAIS). As observed, the proposed method demonstrated attainment of average classification accuracies equal to 97.50% and 93.60%, respectively, when applied to these datasets. Additionally, results obtained in terms of the F1-score and confusion matrix demonstrate the proposed method to be superior to some state-of-the-art methods.

Download Full-text

Ship Classification Based on Multifeature Ensemble with Convolutional Neural Network

Remote Sensing ◽

10.3390/rs11040419 ◽

2019 ◽

Vol 11 (4) ◽

pp. 419 ◽

Cited By ~ 11

Author(s):

Qiaoqiao Shi ◽

Wei Li ◽

Ran Tao ◽

Xu Sun ◽

Lianru Gao

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Fractional Fourier Transform ◽

Gabor Filter ◽

Feature Learning ◽

Feature Representation ◽

Optical Remote Sensing ◽

Discriminative Feature ◽

Ship Classification ◽

High Level

As an important part of maritime traffic, ships play an important role in military and civilian applications. However, ships’ appearances are susceptible to some factors such as lighting, occlusion, and sea state, making ship classification more challenging. This is of great importance when exploring global and detailed information for ship classification in optical remote sensing images. In this paper, a novel method to obtain discriminative feature representation of a ship image is proposed. The proposed classification framework consists of a multifeature ensemble based on convolutional neural network (ME-CNN). Specifically, two-dimensional discrete fractional Fourier transform (2D-DFrFT) is employed to extract multi-order amplitude and phase information, which contains such important information as profiles, edges, and corners; completed local binary pattern (CLBP) is used to obtain local information about ship images; Gabor filter is used to gain the global information about ship images. Then, deep convolutional neural network (CNN) is applied to extract more abstract features based on the above information. CNN, extracting high-level features automatically, has performed well for object classification tasks. After high-feature learning, as the one of fusion strategies, decision-level fusion is investigated for the final classification result. The average accuracy of the proposed approach is 98.75% on the BCCT200-resize data, 92.50% on the original BCCT200 data, and 87.33% on the challenging VAIS data, which validates the effectiveness of the proposed method when compared to the existing state-of-art algorithms.

Download Full-text

A High-Density Crowd Counting Method Based on Convolutional Feature Fusion

Applied Sciences ◽

10.3390/app8122367 ◽

2018 ◽

Vol 8 (12) ◽

pp. 2367 ◽

Cited By ~ 5

Author(s):

Hongling Luo ◽

Jun Sang ◽

Weiqun Wu ◽

Hong Xiang ◽

Zhili Xiang ◽

...

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Large Scale ◽

Feature Fusion ◽

High Density ◽

Counting Problem ◽

Crowd Counting ◽

Low Level ◽

Density Map ◽

High Level

In recent years, the trampling events due to overcrowding have occurred frequently, which leads to the demand for crowd counting under a high-density environment. At present, there are few studies on monitoring crowds in a large-scale crowded environment, while there exists technology drawbacks and a lack of mature systems. Aiming to solve the crowd counting problem with high-density under complex environments, a feature fusion-based deep convolutional neural network method FF-CNN (Feature Fusion of Convolutional Neural Network) was proposed in this paper. The proposed FF-CNN mapped the crowd image to its crowd density map, and then obtained the head count by integration. The geometry adaptive kernels were adopted to generate high-quality density maps which were used as ground truths for network training. The deconvolution technique was used to achieve the fusion of high-level and low-level features to get richer features, and two loss functions, i.e., density map loss and absolute count loss, were used for joint optimization. In order to increase the sample diversity, the original images were cropped with a random cropping method for each iteration. The experimental results of FF-CNN on the ShanghaiTech public dataset showed that the fusion of low-level and high-level features can extract richer features to improve the precision of density map estimation, and further improve the accuracy of crowd counting.

Download Full-text

Dependency Exploitation: A Unified CNN-RNN Approach for Visual Emotion Recognition

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2017/503 ◽

2017 ◽

Cited By ~ 21

Author(s):

Xinge Zhu ◽

Liang Li ◽

Weigang Zhang ◽

Tianrong Rao ◽

Min Xu ◽

...

Keyword(s):

Neural Network ◽

Emotion Recognition ◽

Feature Fusion ◽

Feature Representation ◽

Low Level ◽

Learning Framework ◽

Independent Entity ◽

Internet Images ◽

High Level ◽

Different Levels

Visual emotion recognition aims to associate images with appropriate emotions. There are different visual stimuli that can affect human emotion from low-level to high-level, such as color, texture, part, object, etc. However, most existing methods treat different levels of features as independent entity without having effective method for feature fusion. In this paper, we propose a unified CNN-RNN model to predict the emotion based on the fused features from different levels by exploiting the dependency among them. Our proposed architecture leverages convolutional neural network (CNN) with multiple layers to extract different levels of features with in a multi-task learning framework, in which two related loss functions are introduced to learn the feature representation. Considering the dependencies within the low-level and high-level features, a new bidirectional recurrent neural network (RNN) is proposed to integrate the learned features from different layers in the CNN model. Extensive experiments on both Internet images and art photo datasets demonstrate that our method outperforms the state-of-the-art methods with at least 7% performance improvement.

Download Full-text

Ship Classification Based on Attention Mechanism and Multi-Scale Convolutional Neural Network for Visible and Infrared Images

Electronics ◽

10.3390/electronics9122022 ◽

2020 ◽

Vol 9 (12) ◽

pp. 2022

Author(s):

Yongmei Ren ◽

Jie Yang ◽

Zhiqiang Guo ◽

Qingnian Zhang ◽

Hui Cao

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Classification Performance ◽

Attention Mechanism ◽

Feature Representation ◽

Infrared Images ◽

Visible Image ◽

Multi Scale ◽

Ship Classification ◽

Fusion Features

Visible image quality is very susceptible to changes in illumination, and there are limitations in ship classification using images acquired by a single sensor. This study proposes a ship classification method based on an attention mechanism and multi-scale convolutional neural network (MSCNN) for visible and infrared images. First, the features of visible and infrared images are extracted by a two-stream symmetric multi-scale convolutional neural network module, and then concatenated to make full use of the complementary features present in multi-modal images. After that, the attention mechanism is applied to the concatenated fusion features to emphasize local details areas in the feature map, aiming to further improve feature representation capability of the model. Lastly, attention weights and the original concatenated fusion features are added element by element and fed into fully connected layers and Softmax output layer for final classification output. Effectiveness of the proposed method is verified on a visible and infrared spectra (VAIS) dataset, which shows 93.81% accuracy in classification results. Compared with other state-of-the-art methods, the proposed method could extract features more effectively and has better overall classification performance.

Download Full-text

A Novel Region-Extreme Convolutional Neural Network for Melanoma Malignancy Recognition

Mathematical Problems in Engineering ◽

10.1155/2021/6671498 ◽

2021 ◽

Vol 2021 ◽

pp. 1-18

Author(s):

Nudrat Nida ◽

Aun Irtaza ◽

Muhammad Haroon Yousaf

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Medical Image Analysis ◽

Feature Representation ◽

Machine Learning Techniques ◽

Feature Maps ◽

Model Learning ◽

Learning Machine ◽

Elm Classifier ◽

High Level

Melanoma malignancy recognition is a challenging task due to the existence of intraclass similarity, natural or clinical artefacts, skin contrast variation, and higher visual similarity among the normal or melanoma-affected skin. To overcome these problems, we propose a novel solution by leveraging “region-extreme convolutional neural network” for melanoma malignancy recognition as malignant or benign. Recent works on melanoma malignancy recognition employed the traditional machine learning techniques based on various handcrafted features or the recently introduced CNN network. However, the efficient training of these models is possible, if they localize the melanoma affected region and learn high-level feature representation from melanoma lesion to predict melanoma malignancy. In this paper, we incorporate this observation and propose a novel “region-extreme convolutional neural network” for melanoma malignancy recognition. Our proposed region-extreme convolutional neural network refines dermoscopy images to eliminate natural or clinical artefacts, localizes melanoma affected region, and defines precise boundary around the melanoma lesion. The defined melanoma lesion is used to generate deep feature maps for model learning using the extreme learning machine (ELM) classifier. The proposed model is evaluated on two challenge datasets (ISIC-2016 and ISIC-2017) and performs better than ISIC challenge winners. Our region-extreme convolutional neural network recognizes the melanoma malignancy 85% on ISIC-2016 and 93% on ISIC-2017 datasets. Our region-extreme convolutional neural network precisely segments the melanoma lesion with an average Jaccard index of 0.93 and Dice score of 0.94. The region-extreme convolutional neural network has several advantages: it eliminates the clinical and natural artefacts from dermoscopic images, precisely localizes and segments the melanoma lesion, and improves the melanoma malignancy recognition through feedforward model learning. The region-extreme convolutional neural network achieves significant performance improvement over existing methods that makes it adaptable for solving complex medical image analysis problems.

Download Full-text

Single Channel EEG signal for Automatic Detection of Absence Seizure using Convolutional Neural Network

Recent Advances in Computer Science and Communications ◽

10.2174/2666255813666191122114608 ◽

2019 ◽

Vol 13 ◽

Author(s):

Niha Kamal Basha ◽

Aisha Banu Wahab

Keyword(s):

Neural Network ◽

Support Vector Machine ◽

Convolutional Neural Network ◽

Monitoring System ◽

Single Channel ◽

Confusion Matrix ◽

Sudden Change ◽

Automatic Detection ◽

Support Vector ◽

Absence Seizure

: Absence seizure is a type of brain disorder in which subject get into sudden lapses in attention. Which means sudden change in brain stimulation. Most of this type of disorder is widely found in children’s (5-18 years). These Electroencephalogram (EEG) signals are captured with long term monitoring system and are analyzed individually. In this paper, a Convolutional Neural Network to extract single channel EEG seizure features like Power, log sum of wavelet transform, cross correlation, and mean phase variance of each frame in a windows are extracted after pre-processing and classify them into normal or absence seizure class, is proposed as an empowerment of monitoring system by automatic detection of absence seizure. The training data is collected from the normal and absence seizure subjects in the form of Electroencephalogram. The objective is to perform automatic detection of absence seizure using single channel electroencephalogram signal as input. Here the data is used to train the proposed Convolutional Neural Network to extract and classify absence seizure. The Convolutional Neural Network consist of three layers 1] convolutional layer – which extract the features in the form of vector 2] Pooling layer – the dimensionality of output from convolutional layer is reduced and 3] Fully connected layer–the activation function called soft-max is used to find the probability distribution of output class. This paper goes through the automatic detection of absence seizure in detail and provide the comparative analysis of classification between Support Vector Machine and Convolutional Neural Network. The proposed approach outperforms the performance of Support Vector Machine by 80% in automatic detection of absence seizure and validated using confusion matrix.

Download Full-text