Crop Type Mapping from Optical and Radar Time Series Using Attention-Based Deep Learning

Crop maps are key inputs for crop inventory production and yield estimation and can inform the implementation of effective farm management practices. Producing these maps at detailed scales requires exhaustive field surveys that can be laborious, time-consuming, and expensive to replicate. With a growing archive of remote sensing data, there are enormous opportunities to exploit dense satellite image time series (SITS), temporal sequences of images over the same area. Generally, crop type mapping relies on single-sensor inputs and is solved with the help of traditional learning algorithms such as random forests or support vector machines. Nowadays, deep learning techniques have brought significant improvements by leveraging information in both spatial and temporal dimensions, which are relevant in crop studies. The concurrent availability of Sentinel-1 (synthetic aperture radar) and Sentinel-2 (optical) data offers a great opportunity to utilize them jointly; however, optimizing their synergy has been understudied with deep learning techniques. In this work, we analyze and compare three fusion strategies (input, layer, and decision levels) to identify the best strategy that optimizes optical-radar classification performance. They are applied to a recent architecture, notably, the pixel-set encoder–temporal attention encoder (PSE-TAE) developed specifically for object-based classification of SITS and based on self-attention mechanisms. Experiments are carried out in Brittany, in the northwest of France, with Sentinel-1 and Sentinel-2 time series. Input and layer-level fusion competitively achieved the best overall F-score surpassing decision-level fusion by 2%. On a per-class basis, decision-level fusion increased the accuracy of dominant classes, whereas layer-level fusion improves up to 13% for minority classes. Against single-sensor baseline, multi-sensor fusion strategies identified crop types more accurately: for example, input-level outperformed Sentinel-2 and Sentinel-1 by 3% and 9% in F-score, respectively. We have also conducted experiments that showed the importance of fusion for early time series classification and under high cloud cover condition.

Download Full-text

Evaluation of Five Deep Learning Models for Crop Type Mapping Using Sentinel-2 Time Series Images with Missing Information

Remote Sensing ◽

10.3390/rs13142790 ◽

2021 ◽

Vol 13 (14) ◽

pp. 2790

Author(s):

Hongwei Zhao ◽

Sibo Duan ◽

Jia Liu ◽

Liang Sun ◽

Louis Reymondin

Keyword(s):

Time Series ◽

Deep Learning ◽

Spatial Resolution ◽

Missing Information ◽

Type Mapping ◽

Crop Type ◽

Crop Types ◽

Optical Imagery ◽

Time Series Images ◽

Sentinel 2

Accurate crop type maps play an important role in food security due to their widespread applicability. Optical time series data (TSD) have proven to be significant for crop type mapping. However, filling in missing information due to clouds in optical imagery is always needed, which will increase the workload and the risk of error transmission, especially for imagery with high spatial resolution. The development of optical imagery with high temporal and spatial resolution and the emergence of deep learning algorithms provide solutions to this problem. Although the one-dimensional convolutional neural network (1D CNN), long short-term memory (LSTM), and gate recurrent unit (GRU) models have been used to classify crop types in previous studies, their ability to identify crop types using optical TSD with missing information needs to be further explored due to their different mechanisms for handling invalid values in TSD. In this research, we designed two groups of experiments to explore the performances and characteristics of the 1D CNN, LSTM, GRU, LSTM-CNN, and GRU-CNN models for crop type mapping using unfilled Sentinel-2 (Sentinel-2) TSD and to discover the differences between unfilled and filled Sentinel-2 TSD based on the same algorithm. A case study was conducted in Hengshui City, China, of which 70.3% is farmland. The results showed that the 1D CNN, LSTM-CNN, and GRU-CNN models achieved acceptable classification accuracies (above 85%) using unfilled TSD, even though the total missing rate of the sample values was 43.5%; these accuracies were higher and more stable than those obtained using filled TSD. Furthermore, the models recalled more samples on crop types with small parcels when using unfilled TSD. Although LSTM and GRU models did not attain accuracies as high as the other three models using unfilled TSD, their results were almost close to those with filled TSD. This research showed that crop types could be identified by deep learning features in Sentinel-2 dense time series images with missing information due to clouds or cloud shadows randomly, which avoided spending a lot of time on missing information reconstruction.

Download Full-text

Crop Rotation Modeling for Deep Learning-Based Parcel Classification from Satellite Time Series

Remote Sensing ◽

10.3390/rs13224599 ◽

2021 ◽

Vol 13 (22) ◽

pp. 4599

Author(s):

Félix Quinton ◽

Loic Landrieu

Keyword(s):

Time Series ◽

Deep Learning ◽

Crop Rotation ◽

Large Scale ◽

State Of The Art ◽

Crop Rotations ◽

Learning Approach ◽

Type Mapping ◽

Current State ◽

Crop Type

While annual crop rotations play a crucial role for agricultural optimization, they have been largely ignored for automated crop type mapping. In this paper, we take advantage of the increasing quantity of annotated satellite data to propose to model simultaneously the inter- and intra-annual agricultural dynamics of yearly parcel classification with a deep learning approach. Along with simple training adjustments, our model provides an improvement of over 6.3% mIoU over the current state-of-the-art of crop classification, and a reduction of over 21% of the error rate. Furthermore, we release the first large-scale multi-year agricultural dataset with over 300,000 annotated parcels.

Download Full-text

BREIZHCROPS: A TIME SERIES DATASET FOR CROP TYPE MAPPING

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b2-2020-1545-2020 ◽

2020 ◽

Vol XLIII-B2-2020 ◽

pp. 1545-1551

Author(s):

M. Rußwurm ◽

C. Pelletier ◽

M. Zollner ◽

S. Lefèvre ◽

M. Körner

Keyword(s):

Time Series ◽

Supervised Classification ◽

Deep Neural Networks ◽

Additional Data ◽

Type Mapping ◽

North East ◽

Time Series Dataset ◽

Crop Type ◽

Sentinel 2

Abstract. We present BreizhCrops, a novel benchmark dataset for the supervised classification of field crops from satellite time series. We aggregated label data and Sentinel-2 top-of-atmosphere as well as bottom-of-atmosphere time series in the region of Brittany (Breizh in local language), north-east France. We compare seven recently proposed deep neural networks along with a Random Forest baseline. The dataset, model (re-)implementations and pre-trained model weights are available at the associated GitHub repository (https://github.com/dl4sits/breizhcrops) that has been designed with applicability for practitioners in mind. We plan to maintain the repository with additional data and welcome contributions of novel methods to build a state-of-the-art benchmark on methods for crop type mapping.

Download Full-text

Decision-level fusion scheme for nasopharyngeal carcinoma identification using machine learning techniques

Neural Computing and Applications ◽

10.1007/s00521-018-3882-6 ◽

2018 ◽

Vol 32 (3) ◽

pp. 625-638 ◽

Cited By ~ 11

Author(s):

Mohd Khanapi Abd Ghani ◽

Mazin Abed Mohammed ◽

N. Arunkumar ◽

Salama A. Mostafa ◽

Dheyaa Ahmed Ibrahim ◽

...

Keyword(s):

Machine Learning ◽

Nasopharyngeal Carcinoma ◽

Machine Learning Techniques ◽

Decision Level ◽

Learning Techniques ◽

Decision Level Fusion ◽

Fusion Scheme ◽

Level Fusion

Download Full-text

RFCNN:Traffic Accident Severity Prediction based on Decision Level Fusion of Machine and Deep Learning Model

IEEE Access ◽

10.1109/access.2021.3112546 ◽

2021 ◽

pp. 1-1

Author(s):

Mubariz Manzoor ◽

Muhammad Umer ◽

Saima Sadiq ◽

Abid Ishaq ◽

Saleem Ullah ◽

...

Keyword(s):

Deep Learning ◽

Learning Model ◽

Decision Level ◽

Severity Prediction ◽

Accident Severity ◽

Decision Level Fusion ◽

Deep Learning Model ◽

Level Fusion

Download Full-text

Decision-Level Fusion Method Based on Deep Learning

Biometric Recognition - Lecture Notes in Computer Science ◽

10.1007/978-3-319-69923-3_72 ◽

2017 ◽

pp. 673-682

Author(s):

Kejun Wang ◽

Meichen Liu ◽

XueSen Hao ◽

Xianglei Xing

Keyword(s):

Deep Learning ◽

Fusion Method ◽

Decision Level ◽

Decision Level Fusion ◽

Level Fusion

Download Full-text

Monitoring of agricultural areas by using Sentinel 2 image time series and deep learning techniques

Image and Signal Processing for Remote Sensing XXVI ◽

10.1117/12.2574745 ◽

2020 ◽

Author(s):

Claudia Paris ◽

Giulio Weikmann ◽

Lorenzo Bruzzone

Keyword(s):

Time Series ◽

Deep Learning ◽

Learning Techniques ◽

Agricultural Areas ◽

Sentinel 2

Download Full-text

SEN2-AGRI – CROP TYPE MAPPING PILOT STUDY USING SENTINEL-2 SATELLITE IMAGERY IN INDIA

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-3-w6-175-2019 ◽

2019 ◽

Vol XLII-3/W6 ◽

pp. 175-180 ◽

Cited By ~ 1

Author(s):

D. Vijayasekaran

Keyword(s):

Land Use ◽

Time Series ◽

Satellite Imagery ◽

Classification Accuracy ◽

Earth Observation ◽

Field Size ◽

Observation Data ◽

Type Mapping ◽

Crop Type ◽

Sentinel 2

<p><strong>Abstract.</strong> Large-scale mapping and monitoring of agriculture land use are very important. It helps in forecast crop yields, assesses the factors influencing the crop stress and estimate the damage due to natural hazards. Also, more essentially, aids in calculating the irrigation water demand at the farm level and better water resource management. Recent developments in remote sensing satellite sensors spatial and temporal resolutions, global coverage and open access such as Sentinel-2, created new possibilities in mapping and monitoring land use/land cover features. The present study investigated the performance and applicability of Sen2-Agri system in the heterogeneous cropping system for operational crop type mapping at parcel resolution using time series Sentinel-2 multispectral satellite imagery. The parcel level crop type information was collected in the field by systematic sampling and used to train and validate the random forest (RF) classification in the system. The classification accuracy varied from 57% to 86% for different major crops. The overall classification accuracy was 70% with KAPPA index of 61%. The very small agriculture field size and persistent cloud cover are the major constraint to the improvement of classification accuracy. Combination of the time series imagery from multiple earth observation satellites for the monsoon cropping season and development of a robust system for in-situ data collection will further increase the mapping accuracy. Sen2-Agri system has the potential to handle a large amount of earth observation data and can be scaled up to the entire country, which will help in the efficient monitoring of crops.</p>

Download Full-text