Sampling Design for Accuracy Assessment of Large-Area, Land-Cover Maps

Substantial progress has been made in the field of large-area land cover mapping as the spatial resolution of remotely sensed data increases. However, a significant amount of human power is still required to label images for training and testing purposes, especially in high-resolution (e.g., 3-m) land cover mapping. In this research, we propose a solution that can produce 3-m resolution land cover maps on a national scale without human efforts being involved. First, using the public 10-m resolution land cover maps as an imperfect training dataset, we propose a deep learning based approach that can effectively transfer the existing knowledge. Then, we improve the efficiency of our method through a network pruning process for national-scale land cover mapping. Our proposed method can take the state-of-the-art 10-m resolution land cover maps (with an accuracy of 81.24% for China) as the training data, enable a transferred learning process that can produce 3-m resolution land cover maps, and further improve the overall accuracy (OA) to 86.34% for China. We present detailed results obtained over three mega cities in China, to demonstrate the effectiveness of our proposed approach for 3-m resolution large-area land cover mapping.

Download Full-text

Utilizing the Global Land Cover 2000 reference dataset for a comparative accuracy assessment of 1 km global land cover maps

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xl-7-w3-503-2015 ◽

2015 ◽

Vol XL-7/W3 ◽

pp. 503-510 ◽

Cited By ~ 4

Author(s):

M. Schultz ◽

N. E. Tsendbazazr ◽

M. Herold ◽

M. Jung ◽

P. Mayaux ◽

...

Keyword(s):

Land Cover ◽

Climate Models ◽

Accuracy Assessment ◽

Careful Consideration ◽

Global Climate Models ◽

Reference Dataset ◽

Heterogeneous Landscape ◽

Global Land Cover ◽

Global Land ◽

Land Cover Maps

Many investigators use global land cover (GLC) maps for different purposes, such as an input for global climate models. The current GLC maps used for such purposes are based on different remote sensing data, methodologies and legends. Consequently, comparison of GLC maps is difficult and information about their relative utility is limited. The objective of this study is to analyse and compare the thematic accuracies of GLC maps (i.e., IGBP-DISCover, UMD, MODIS, GLC2000 and SYNMAP) at 1 km resolutions by (a) re-analysing the GLC2000 reference dataset, (b) applying a generalized GLC legend and (c) comparing their thematic accuracies at different homogeneity levels. The accuracy assessment was based on the GLC2000 reference dataset with 1253 samples that were visually interpreted. The legends of the GLC maps and the reference datasets were harmonized into 11 general land cover classes. There results show that the map accuracy estimates vary up to 10-16% depending on the homogeneity of the reference point (HRP) for all the GLC maps. An increase of the HRP resulted in higher overall accuracies but reduced accuracy confidence for the GLC maps due to less number of accountable samples. The overall accuracy of the SYNMAP was the highest at any HRP level followed by the GLC2000. The overall accuracies of the maps also varied by up to 10% depending on the definition of agreement between the reference and map categories in heterogeneous landscape. A careful consideration of heterogeneous landscape is therefore recommended for future accuracy assessments of land cover maps.

Download Full-text

Trends in Remote Sensing Accuracy Assessment Approaches in the Context of Natural Resources

Remote Sensing ◽

10.3390/rs11192305 ◽

2019 ◽

Vol 11 (19) ◽

pp. 2305 ◽

Cited By ~ 11

Author(s):

Lucia Morales-Barquero ◽

Mitchell Lyons ◽

Stuart Phinn ◽

Chris Roelfsema

Keyword(s):

Remote Sensing ◽

Natural Resources ◽

Sampling Design ◽

Accuracy Assessment ◽

Assessment Practices ◽

Screening Process ◽

Error Matrix ◽

Remote Sensing Classification ◽

Considerable Work ◽

Land Cover Maps

The utility of land cover maps for natural resources management relies on knowing the uncertainty associated with each map. The continuous advances typical of remote sensing, including the increasing availability of higher spatial and temporal resolution satellite data and data analysis capabilities, have created both opportunities and challenges for improving the application of accuracy assessment. There are well established accuracy assessment methods, but their underlying assumptions have not changed much in the last couple decades. Consequently, revisiting how map error and accuracy have been performed and reported over the last two decades is timely, to highlight areas where there is scope for better utilization of emerging opportunities. We conducted a quantitative literature review on accuracy assessment practices for mapping via remote sensing classification methods, in both terrestrial and marine environments. We performed a structured search for land and benthic cover mapping, limiting our search to journals within the remote sensing field, and papers published between 1998–2017. After an initial screening process, we assembled a database of 282 papers, and extracted and standardized information on various components of their reported accuracy assessments. We discovered that only 56% of the papers explicitly included an error matrix, and a very limited number (14%) reported overall accuracy with confidence intervals. The use of kappa continues to be standard practice, being reported in 50.4% of the literature published on or after 2012. Reference datasets used for validation were collected using a probability sampling design in 54% of the papers. For approximately 11% of the studies, the sampling design used could not be determined. No association was found between classification complexity (i.e. number of classes) and measured accuracy, independent from the size of the study area. Overall, only 32% of papers included an accuracy assessment that could be considered reproducible; that is, they included a probability-based sampling scheme to collect the reference dataset, a complete error matrix, and provided sufficient characterization of the reference datasets and sampling unit. Our findings indicate that considerable work remains to identify and adopt more statistically rigorous accuracy assessment practices to achieve transparent and comparable land and benthic cover maps.

Download Full-text

Assessing the impacts of human uncertainty in the accuracy assessment of land-cover maps using linguistic scales and fuzzy intervals

International Journal of Remote Sensing ◽

10.1080/01431161.2015.1043034 ◽

2015 ◽

Vol 36 (10) ◽

pp. 2524-2547 ◽

Cited By ~ 1

Author(s):

Pedro Sarmento ◽

Cidália C. Fonte ◽

Joel Dinis ◽

Stephen V. Stehman ◽

Mário Caetano

Keyword(s):

Land Cover ◽

Accuracy Assessment ◽

Land Cover Maps

Download Full-text

Combining Accuracy Assessment of Land-Cover Maps With Environmental Monitoring Programs

Monitoring Ecological Condition in the Western United States ◽

10.1007/978-94-011-4343-1_11 ◽

2000 ◽

pp. 115-126

Author(s):

Stephen V. Stehman ◽

Raymond L. Czaplewski ◽

Sarah M. Nusser ◽

Limin Yang ◽

Zhiliang Zhu

Keyword(s):

Land Cover ◽

Environmental Monitoring ◽

Accuracy Assessment ◽

Monitoring Programs ◽

Land Cover Maps

Download Full-text

REVIEW OF HIGH-RESOLUTION GLOBAL LAND COVER

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b4-2021-175-2021 ◽

2021 ◽

Vol XLIII-B4-2021 ◽

pp. 175-182

Author(s):

G. Bratic ◽

A. Vavassori ◽

M. A. Brovelli

Keyword(s):

High Resolution ◽

Land Cover ◽

Spatial Resolution ◽

Accuracy Assessment ◽

General Information ◽

Tree Canopy ◽

Global Land Cover ◽

Tree Canopy Cover ◽

Global Land ◽

Land Cover Maps

Abstract. The land cover detection on our planet at high spatial resolution has a key role in many scientific and operational applications, such as climate modeling, natural resources management, biodiversity studies, urbanization analyses and spatial demography. Thanks to the progresses in Remote Sensing, accurate and high-resolution land cover maps have been developed over the last years, aiming at detecting the spatial resolution of different types of surfaces. In this paper we propose a review of the high-resolution global land cover products developed through Earth Observation technologies. A series of general information regarding imagery and data used to produce the map, the procedures employed for the map development and for the map accuracy assessment have been provided for every dataset. The land cover maps described in this paper concern the global distribution of settlements (Global Urban Footprint, Global Human Settlement Built-Up, World Settlement Footprint), water (Global Surface Water), forests (Forest/Non-forest, Tree canopy cover), and a two land cover maps describing world in 10 generic classes (GlobeLand30 and Finer Resolution Observation and Monitoring of Global Land Cover). The advantages and shortcomings of these maps and of the methods employed to produce them are summarized and compared in the conclusions.

Download Full-text

Quality assurance and assessment framework for large area land cover maps validation in the Copernicus high-resolution hot spot monitoring activity

European Journal of Remote Sensing ◽

10.1080/22797254.2021.1978001 ◽

2021 ◽

Vol 54 (1) ◽

pp. 537-556

Author(s):

Zoltan Szantoi ◽

Gabriel Jaffrain ◽

Heinz Gallaun ◽

Conrad Bielski ◽

Karl Ruf ◽

...

Keyword(s):

Quality Assurance ◽

High Resolution ◽

Land Cover ◽

Hot Spot ◽

Assessment Framework ◽

Large Area ◽

Land Cover Maps ◽

Monitoring Activity

Download Full-text

AREA ESTIMATION OF MULTI-TEMPORAL GLOBAL IMPERVIOUS LAND COVER BASED ON STRATIFIED RANDOM SAMPLING

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b4-2020-103-2020 ◽

2020 ◽

Vol XLIII-B4-2020 ◽

pp. 103-108

Author(s):

Y. Gong ◽

H. Xie ◽

X. Tong ◽

Y. Jin ◽

X. Xv ◽

...

Keyword(s):

Land Cover ◽

Random Sampling ◽

Accuracy Assessment ◽

Time Interval ◽

Stratified Random Sampling ◽

Regional Environment ◽

Area Estimation ◽

Research On Evaluation ◽

Multi Temporal ◽

Land Cover Maps

Abstract. Estimating area of impervious land cover is the most useful and one of the ecological assessment indexes of urban and regional environment. Global land cover maps are inevitably misclassified, which affects the quality and application of the data. Statistical approach for assessing the accuracy is critical to understand the global change information and area estimation is usually based on sample data with a probability-based estimator. However, research on evaluation of multi-temporal global impervious land cover maps has not been implemented. In this study, spatial characteristics of the data are considered to assess the thematic map accuracy with a two-stage stratified random sampling plan. The first stage of stratification is determined by the global urban ecoregion and the second one is determined by land cover classes. Additionally, sample size of both map stage and pixel stage are calculated using a probability sampling model. A response design is constructed for a per-pixel accuracy assessment and blind interpretation is implemented using sample pixels and its surrounding area. Our method is applied to the multi-temporal global impervious land cover maps between 2000 and 2010 with a time interval of 5 years and the estimated area in different epoch are listed in detail. The main contribution of our research is illustrating the details for calculating the proportion area of impervious land cover and corresponding confidence intervals based on the reference classification. The experimental results show that the increasing area of the impervious surface according to the sample unit shows good agreement with the urbanization and descriptive accuracy assessments by user’s, producer’s and overall accuracy are shown respectively.

Download Full-text

GLOBE Observer and the GO on a Trail Data Challenge: A Citizen Science Approach to Generating a Global Land Cover Land Use Reference Dataset

Frontiers in Climate ◽

10.3389/fclim.2021.620497 ◽

2021 ◽

Vol 3 ◽

Author(s):

Holli A. Kohl ◽

Peder V. Nelson ◽

John Pring ◽

Kristen L. Weaver ◽

Daniel M. Wiley ◽

...

Keyword(s):

Land Use ◽

Land Cover ◽

Reference Data ◽

Ground Cover ◽

The United States ◽

Large Area ◽

Natural Processes ◽

Land Cover Data ◽

Data Points ◽

Land Cover Maps

Land cover and land use are highly visible indicators of climate change and human disruption to natural processes. While land cover is frequently monitored over a large area using satellite data, ground-based reference data is valuable as a comparison point. The NASA-funded GLOBE Observer (GO) program provides volunteer-collected land cover photos tagged with location, date and time, and, in some cases, land cover type. When making a full land cover observation, volunteers take six photos of the site, one facing north, south, east, and west (N-S-E-W), respectively, one pointing straight up to capture canopy and sky, and one pointing down to document ground cover. Together, the photos document a 100-meter square of land. Volunteers may then optionally tag each N-S-E-W photo with the land cover types present. Volunteers collect the data through a smartphone app, also called GLOBE Observer, resulting in consistent data. While land cover data collected through GLOBE Observer is ongoing, this paper presents the results of a data challenge held between June 1 and October 15, 2019. Called “GO on a Trail,” the challenge resulted in more than 3,300 land cover data points from around the world with concentrated data collection in the United States and Australia. GLOBE Observer collections can serve as reference data, complementing satellite imagery for the improvement and verification of broad land cover maps. Continued collection using this protocol will build a database documenting climate-related land cover and land use change into the future.

Download Full-text