Continual Learning Approach for Remote Sensing Scene Classification

Scene classification is an important aspect of image/video understanding and segmentation. However, remote-sensing scene classification is a challenging image recognition task, partly due to the limited training data, which causes deep-learning Convolutional Neural Networks (CNNs) to overfit. Another difficulty is that images often have very different scales and orientation (viewing angle). Yet another is that the resulting networks may be very large, again making them prone to overfitting and unsuitable for deployment on memory- and energy-limited devices. We propose an efficient deep-learning approach to tackle these problems. We use transfer learning to compensate for the lack of data, and data augmentation to tackle varying scale and orientation. To reduce network size, we use a novel unsupervised learning approach based on k-means clustering, applied to all parts of the network: most network reduction methods use computationally expensive supervised learning methods, and apply only to the convolutional or fully connected layers, but not both. In experiments, we set new standards in classification accuracy on four remote-sensing and two scene-recognition image datasets.

Download Full-text

JUNGLE-NET: USING EXPLAINABLE MACHINE LEARNING TO GAIN NEW INSIGHTS INTO THE APPEARANCE OF WILDERNESS IN SATELLITE IMAGERY

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-v-3-2021-317-2021 ◽

2021 ◽

Vol V-3-2021 ◽

pp. 317-324

Author(s):

T. Stomberg ◽

I. Weber ◽

M. Schmitt ◽

R. Roscher

Keyword(s):

Machine Learning ◽

Remote Sensing ◽

Land Cover ◽

Observational Data ◽

Satellite Imagery ◽

Learning Approach ◽

Scene Classification ◽

Machine Learning Approach ◽

New Knowledge

Abstract. Explainable machine learning has recently gained attention due to its contribution to understanding how a model works and why certain decisions are made. A so far less targeted goal, especially in remote sensing, is the derivation of new knowledge and scientific insights from observational data. In our paper, we propose an explainable machine learning approach to address the challenge that certain land cover classes such as wilderness are not well-defined in satellite imagery and can only be used with vague labels for mapping. Our approach consists of a combined U-Net and ResNet-18 that can perform scene classification while providing at the same time interpretable information with which we can derive new insights about classes. We show that our methodology allows us to deepen our understanding of what makes nature wild by automatically identifying simple concepts such as wasteland that semantically describes wilderness. It further quantifies a class’s sensitivity with respect to a concept and uses it as an indicator for how well a concept describes the class.

Download Full-text

LIL: Lightweight Incremental Learning Approach Through Feature Transfer for Remote Sensing Image Scene Classification

IEEE Transactions on Geoscience and Remote Sensing ◽

10.1109/tgrs.2021.3102629 ◽

2021 ◽

pp. 1-20

Author(s):

Xiaonan Lu ◽

Xian Sun ◽

Wenhui Diao ◽

Yingchao Feng ◽

Peijin Wang ◽

...

Keyword(s):

Remote Sensing ◽

Incremental Learning ◽

Remote Sensing Image ◽

Learning Approach ◽

Scene Classification

Download Full-text

Ensemble Learning Approaches Based on Covariance Pooling of CNN Features for High Resolution Remote Sensing Scene Classification

Remote Sensing ◽

10.3390/rs12203292 ◽

2020 ◽

Vol 12 (20) ◽

pp. 3292

Author(s):

Sara Akodad ◽

Lionel Bombrun ◽

Junshi Xia ◽

Yannick Berthoumieu ◽

Christian Germain

Keyword(s):

Machine Learning ◽

Remote Sensing ◽

Ensemble Learning ◽

Covariance Matrices ◽

Machine Learning Algorithms ◽

Learning Approach ◽

Learning Approaches ◽

Scene Classification ◽

First Order ◽

Fisher Vector Encoding

Remote sensing image scene classification, which consists of labeling remote sensing images with a set of categories based on their content, has received remarkable attention for many applications such as land use mapping. Standard approaches are based on the multi-layer representation of first-order convolutional neural network (CNN) features. However, second-order CNNs have recently been shown to outperform traditional first-order CNNs for many computer vision tasks. Hence, the aim of this paper is to show the use of second-order statistics of CNN features for remote sensing scene classification. This takes the form of covariance matrices computed locally or globally on the output of a CNN. However, these datapoints do not lie in an Euclidean space but a Riemannian manifold. To manipulate them, Euclidean tools are not adapted. Other metrics should be considered such as the log-Euclidean one. This consists of projecting the set of covariance matrices on a tangent space defined at a reference point. In this tangent plane, which is a vector space, conventional machine learning algorithms can be considered, such as the Fisher vector encoding or SVM classifier. Based on this log-Euclidean framework, we propose a novel transfer learning approach composed of two hybrid architectures based on covariance pooling of CNN features, the first is local and the second is global. They rely on the extraction of features from models pre-trained on the ImageNet dataset processed with some machine learning algorithms. The first hybrid architecture consists of an ensemble learning approach with the log-Euclidean Fisher vector encoding of region covariance matrices computed locally on the first layers of a CNN. The second one concerns an ensemble learning approach based on the covariance pooling of CNN features extracted globally from the deepest layers. These two ensemble learning approaches are then combined together based on the strategy of the most diverse ensembles. For validation and comparison purposes, the proposed approach is tested on various challenging remote sensing datasets. Experimental results exhibit a significant gain of approximately 2% in overall accuracy for the proposed approach compared to a similar state-of-the-art method based on covariance pooling of CNN features (on the UC Merced dataset).

Download Full-text

Memory Using Data Generator in Continual Learning for Remote Sensing Scene Classification

10.1109/igarss47720.2021.9553520 ◽

2021 ◽

Author(s):

Nassim Ammour

Keyword(s):

Remote Sensing ◽

Scene Classification ◽

Data Generator ◽

Using Data ◽

Continual Learning

Download Full-text

CLRS: Continual Learning Benchmark for Remote Sensing Image Scene Classification

Sensors ◽

10.3390/s20041226 ◽

2020 ◽

Vol 20 (4) ◽

pp. 1226

Author(s):

Haifeng Li ◽

Hao Jiang ◽

Xin Gu ◽

Jian Peng ◽

Wenbo Li ◽

...

Keyword(s):

Remote Sensing ◽

Large Scale ◽

Remote Sensing Data ◽

Remote Sensing Image ◽

Classification Model ◽

Scene Classification ◽

Learning Scenarios ◽

Batch Data ◽

Classification Database ◽

Continual Learning

Remote sensing image scene classification has a high application value in the agricultural, military, as well as other fields. A large amount of remote sensing data is obtained every day. After learning the new batch data, scene classification algorithms based on deep learning face the problem of catastrophic forgetting, that is, they cannot maintain the performance of the old batch data. Therefore, it has become more and more important to ensure that the scene classification model has the ability of continual learning, that is, to learn new batch data without forgetting the performance of the old batch data. However, the existing remote sensing image scene classification datasets all use static benchmarks and lack the standard to divide the datasets into a number of sequential learning training batches, which largely limits the development of continual learning in remote sensing image scene classification. First, this study gives the criteria for training batches that have been partitioned into three continual learning scenarios, and proposes a large-scale remote sensing image scene classification database called the Continual Learning Benchmark for Remote Sensing (CLRS). The goal of CLRS is to help develop state-of-the-art continual learning algorithms in the field of remote sensing image scene classification. In addition, in this paper, a new method of constructing a large-scale remote sensing image classification database based on the target detection pretrained model is proposed, which can effectively reduce manual annotations. Finally, several mainstream continual learning methods are tested and analyzed under three continual learning scenarios, and the results can be used as a baseline for future work.

Download Full-text