Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification

Due to device limitations, small networks are necessary for some real-world scenarios, such as satellites and micro-robots. Therefore, the development of a network with both good performance and small size is an important area of research. Deep networks can learn well from large amounts of data, while manifold networks have outstanding feature representation at small sizes. In this paper, we propose an approach that exploits the advantages of deep networks and shallow Grassmannian manifold networks. Inspired by knowledge distillation, we use the information learned from convolutional neural networks to guide the training of the manifold networks. Our approach leads to a reduction in model size, which addresses the problem of deploying deep learning on resource-limited embedded devices. Finally, a series of experiments were conducted on four remote sensing scene classification datasets. The method in this paper improved the classification accuracy by 2.31% and 1.73% on the UC Merced Land Use and SIRIWHU datasets, respectively, and the experimental results demonstrate the effectiveness of our approach.

Download Full-text

A Dual-Model Architecture with Grouping-Attention-Fusion for Remote Sensing Scene Classification

Remote Sensing ◽

10.3390/rs13030433 ◽

2021 ◽

Vol 13 (3) ◽

pp. 433

Author(s):

Junge Shen ◽

Tong Zhang ◽

Yichen Wang ◽

Ruxin Wang ◽

Qi Wang ◽

...

Keyword(s):

Remote Sensing ◽

Feature Representation ◽

Dual Model ◽

Scene Classification ◽

Remote Sensing Images ◽

Single Model ◽

Fusion Strategy ◽

Multi Scale ◽

The Arts ◽

Scene Representation

Remote sensing images contain complex backgrounds and multi-scale objects, which pose a challenging task for scene classification. The performance is highly dependent on the capacity of the scene representation as well as the discriminability of the classifier. Although multiple models possess better properties than a single model on these aspects, the fusion strategy for these models is a key component to maximize the final accuracy. In this paper, we construct a novel dual-model architecture with a grouping-attention-fusion strategy to improve the performance of scene classification. Specifically, the model employs two different convolutional neural networks (CNNs) for feature extraction, where the grouping-attention-fusion strategy is used to fuse the features of the CNNs in a fine and multi-scale manner. In this way, the resultant feature representation of the scene is enhanced. Moreover, to address the issue of similar appearances between different scenes, we develop a loss function which encourages small intra-class diversities and large inter-class distances. Extensive experiments are conducted on four scene classification datasets include the UCM land-use dataset, the WHU-RS19 dataset, the AID dataset, and the OPTIMAL-31 dataset. The experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-arts.

Download Full-text

Learning Deep Networks under Noisy Labels for Remote Sensing Image Scene Classification

IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium ◽

10.1109/igarss.2019.8900497 ◽

2019 ◽

Cited By ~ 2

Author(s):

Yansheng Li ◽

Yongjun Zhang ◽

Zhihui Zhu

Keyword(s):

Remote Sensing ◽

Remote Sensing Image ◽

Scene Classification ◽

Deep Networks ◽

Noisy Labels

Download Full-text

A Fast Deep Perception Network for Remote Sensing Scene Classification

Remote Sensing ◽

10.3390/rs12040729 ◽

2020 ◽

Vol 12 (4) ◽

pp. 729 ◽

Cited By ~ 3

Author(s):

Ruchan Dong ◽

Dazhuan Xu ◽

Lichen Jiao ◽

Jin Zhao ◽

Jungang An

Keyword(s):

Remote Sensing ◽

Feature Fusion ◽

Remote Sensing Image ◽

Learning System ◽

Support Vector ◽

Scene Classification ◽

Deep Convolutional Neural Networks ◽

Directional Information ◽

Series Of Experiments ◽

New Feature

Current scene classification for high-resolution remote sensing images usually uses deep convolutional neural networks (DCNN) to extract extensive features and adopts support vector machine (SVM) as classifier. DCNN can well exploit deep features but ignore valuable shallow features like texture and directional information; and SVM can hardly train a large amount of samples in an efficient way. This paper proposes a fast deep perception network (FDPResnet) that integrates DCNN and Broad Learning System (BLS), a novel effective learning system, to extract both deep and shallow features and encapsulates a designed DPModel to fuse the two kinds of features. FDPResnet first extracts the shallow and the deep scene features of a remote sensing image through a pre-trained model on residual neural network-101 (Resnet101). Then, it inputs the two kinds of features into a designed deep perception module (DPModel) to obtain a new set of feature vectors that can describe both higher-level semantic and lower-level space information of the image. The DPModel is the key module responsible for dimension reduction and feature fusion. Finally, the obtained new feature vector is input into BLS for training and classification, and we can obtain a satisfactory classification result. A series of experiments are conducted on the challenging NWPU-RESISC45 remote sensing image dataset, and the results demonstrate that our approach outperforms some popular state-of-the-art deep learning methods, and present high-accurate scene classification within a shorter running time.

Download Full-text

Effective Multiscale Residual Network With High-Order Feature Representation for Optical Remote Sensing Scene Classification

IEEE Geoscience and Remote Sensing Letters ◽

10.1109/lgrs.2021.3075257 ◽

2021 ◽

pp. 1-5

Author(s):

Can Li ◽

Yin Zhuang ◽

Wenchao Liu ◽

Shan Dong ◽

Hailin Du ◽

...

Keyword(s):

Remote Sensing ◽

High Order ◽

Feature Representation ◽

Optical Remote Sensing ◽

Scene Classification ◽

Residual Network

Download Full-text

Knowledge distillation in deep learning and its applications

PeerJ Computer Science ◽

10.7717/peerj-cs.474 ◽

2021 ◽

Vol 7 ◽

pp. e474

Author(s):

Abdolmaged Alkhulaifi ◽

Fahad Alsahli ◽

Irfan Ahmad

Keyword(s):

Deep Learning ◽

Mobile Phones ◽

Learning Models ◽

Student Model ◽

Embedded Devices ◽

Research Directions ◽

Resource Limited ◽

Knowledge Distillation ◽

Teacher Model

Deep learning based models are relatively large, and it is hard to deploy such models on resource-limited devices such as mobile phones and embedded devices. One possible solution is knowledge distillation whereby a smaller model (student model) is trained by utilizing the information from a larger model (teacher model). In this paper, we present an outlook of knowledge distillation techniques applied to deep learning models. To compare the performances of different techniques, we propose a new metric called distillation metric which compares different knowledge distillation solutions based on models' sizes and accuracy scores. Based on the survey, some interesting conclusions are drawn and presented in this paper including the current challenges and possible research directions.

Download Full-text

Remote Sensing Image Scene Classification Based on Fusion Method

Journal of Sensors ◽

10.1155/2021/6659831 ◽

2021 ◽

Vol 2021 ◽

pp. 1-14

Author(s):

Liancheng Yin ◽

Peiyi Yang ◽

Keming Mao ◽

Qian Liu

Keyword(s):

Remote Sensing ◽

Model Performance ◽

Remote Sensing Image ◽

Research Area ◽

Feature Representation ◽

Back Side ◽

Scene Classification ◽

Single Side ◽

Fusion Methods ◽

Fusion Mode

Remote sensing image scene classification is a hot research area for its wide applications. More recently, fusion-based methods attract much attention since they are considered to be an useful way for scene feature representation. This paper explores the fusion-based method for remote sensing image scene classification from another viewpoint. First, it is categorized as front side fusion mode, middle side fusion mode, and back side fusion mode. For each fusion mode, the related methods are introduced and described. Then, classification performances of the single side fusion mode and hybrid side fusion mode (combinations of single side fusion) are evaluated. Comprehensive experiments on UC Merced, WHU-RS19, and NWPU-RESISC45 datasets give the comparison result among various fusion methods. The performance comparisons of various modes, and interactions among different fusion modes are also discussed. It is concluded that (1) fusion is an effective way to improve model performance, (2) back side fusion is the most powerful fusion mode, and (3) method with random crop+multiple backbone+average achieves the best performance.

Download Full-text

A Multi-Scale Approach for Remote Sensing Scene Classification Based on Feature Maps Selection and Region Representation

Remote Sensing ◽

10.3390/rs11212504 ◽

2019 ◽

Vol 11 (21) ◽

pp. 2504 ◽

Cited By ~ 3

Author(s):

Jun Zhang ◽

Min Zhang ◽

Lukui Shi ◽

Wenjie Yan ◽

Bin Pan

Keyword(s):

Remote Sensing ◽

High Resolution ◽

Remote Sensing Image ◽

Feature Representation ◽

Features Selection ◽

Scene Classification ◽

Feature Maps ◽

Global Features ◽

Convolutional Networks ◽

Multi Scale

Scene classification is one of the bases for automatic remote sensing image interpretation. Recently, deep convolutional neural networks have presented promising performance in high-resolution remote sensing scene classification research. In general, most researchers directly use raw deep features extracted from the convolutional networks to classify scenes. However, this strategy only considers single scale features, which cannot describe both the local and global features of images. In fact, the dissimilarity of scene targets in the same category may result in convolutional features being unable to classify them into the same category. Besides, the similarity of the global features in different categories may also lead to failure of fully connected layer features to distinguish them. To address these issues, we propose a scene classification method based on multi-scale deep feature representation (MDFR), which mainly includes two contributions: (1) region-based features selection and representation; and (2) multi-scale features fusion. Initially, the proposed method filters the multi-scale deep features extracted from pre-trained convolutional networks. Subsequently, these features are fused via two efficient fusion methods. Our method utilizes the complementarity between local features and global features by effectively exploiting the features of different scales and discarding the redundant information in features. Experimental results on three benchmark high-resolution remote sensing image datasets indicate that the proposed method is comparable to some state-of-the-art algorithms.

Download Full-text

An End-to-End Local-Global-Fusion Feature Extraction Network for Remote Sensing Image Scene Classification

Remote Sensing ◽

10.3390/rs11243006 ◽

2019 ◽

Vol 11 (24) ◽

pp. 3006 ◽

Cited By ~ 4

Author(s):

Yafei Lv ◽

Xiaohan Zhang ◽

Wei Xiong ◽

Yaqi Cui ◽

Mi Cai

Keyword(s):

Remote Sensing ◽

Feature Extraction ◽

Feature Fusion ◽

Remote Sensing Image ◽

Local Features ◽

Feature Representation ◽

Scene Classification ◽

Global Features ◽

End To End ◽

Fusion Feature

Remote sensing image scene classification (RSISC) is an active task in the remote sensing community and has attracted great attention due to its wide applications. Recently, the deep convolutional neural networks (CNNs)-based methods have witnessed a remarkable breakthrough in performance of remote sensing image scene classification. However, the problem that the feature representation is not discriminative enough still exists, which is mainly caused by the characteristic of inter-class similarity and intra-class diversity. In this paper, we propose an efficient end-to-end local-global-fusion feature extraction (LGFFE) network for a more discriminative feature representation. Specifically, global and local features are extracted from channel and spatial dimensions respectively, based on a high-level feature map from deep CNNs. For the local features, a novel recurrent neural network (RNN)-based attention module is first proposed to capture the spatial layout information and context information across different regions. Gated recurrent units (GRUs) is then exploited to generate the important weight of each region by taking a sequence of features from image patches as input. A reweighed regional feature representation can be obtained by focusing on the key region. Then, the final feature representation can be acquired by fusing the local and global features. The whole process of feature extraction and feature fusion can be trained in an end-to-end manner. Finally, extensive experiments have been conducted on four public and widely used datasets and experimental results show that our method LGFFE outperforms baseline methods and achieves state-of-the-art results.

Download Full-text

RS-SSKD: Self-Supervision Equipped with Knowledge Distillation for Few-Shot Remote Sensing Scene Classification

Sensors ◽

10.3390/s21051566 ◽

2021 ◽

Vol 21 (5) ◽

pp. 1566

Author(s):

Pei Zhang ◽

Ying Li ◽

Dong Wang ◽

Jiyue Wang

Keyword(s):

Remote Sensing ◽

State Of The Art ◽

Ground Truth ◽

Training Data ◽

Scene Classification ◽

Training Time ◽

Shot Classification ◽

Meta Learning ◽

Knowledge Distillation ◽

Few Data

While growing instruments generate more and more airborne or satellite images, the bottleneck in remote sensing (RS) scene classification has shifted from data limits toward a lack of ground truth samples. There are still many challenges when we are facing unknown environments, especially those with insufficient training data. Few-shot classification offers a different picture under the umbrella of meta-learning: digging rich knowledge from a few data are possible. In this work, we propose a method named RS-SSKD for few-shot RS scene classification from a perspective of generating powerful representation for the downstream meta-learner. Firstly, we propose a novel two-branch network that takes three pairs of original-transformed images as inputs and incorporates Class Activation Maps (CAMs) to drive the network mining, the most relevant category-specific region. This strategy ensures that the network generates discriminative embeddings. Secondly, we set a round of self-knowledge distillation to prevent overfitting and boost the performance. Our experiments show that the proposed method surpasses current state-of-the-art approaches on two challenging RS scene datasets: NWPU-RESISC45 and RSD46-WHU. Finally, we conduct various ablation experiments to investigate the effect of each component of the proposed method and analyze the training time of state-of-the-art methods and ours.

Download Full-text