EXPLORING THE USE OF CLASSIFICATION UNCERTAINTY TO IMPROVE CLASSIFICATION ACCURACY

Supervised classification of remotely sensed images has been widely used to map land cover and land use. Since the performance of supervised methods depends on the quality of the training data, it is essential to develop methods to generate an enhanced training dataset. Active learning represents an alternative for such purpose as it proposes to create a dataset of optimized samples, normally collected based on classification uncertainty. However, it is heavily dependent on human interaction, since the user has to label selected samples over a number of iterations. In this paper, we explore the use of uncertainty to improve classification accuracy through a single iteration. We conducted experiments in a region of Portugal (Trás-os-Montes), using multi-temporal Sentinel-2 images. The proposed approach consisted in computing the classification uncertainty of a Random Forest to collect additional training data from areas of high uncertainty and perform a new classification. An accuracy assessment was performed to compare the overall accuracy of the initial and new classifications. The results exhibited an increase in accuracy, though considered not statistically significant. Obstacles related to labelling additional sampling units resulted in a lack of additional training data for various classes, which might have limited the accuracy improvement. Additionally, an uneven proportion of additional training sampling units per class and the collection of new sample data from a limited number of uncertainty regions might also have prevented a higher increase in accuracy. Nevertheless, visual inspection of the maps revealed that the new classification reduced the confusion between some classes.

NOVEL CLASSIFICATION UNCERTAINTY MEASUREMENT MODEL INTEGRATING SPATIAL INFORMATION FOR REMOTE SENSING IMAGE

Remote sensing image classification has important applications in many fields. However, the uncertainty of remote sensing image classification results will reduce its application value and reliability in these applications. Therefore, the uncertainty of remote sensing image classification results must be accurately and effectively measured. To address the shortcomings of the existing classification uncertainty measurement model in the utilization of image spatial information, this study proposes a novel uncertainty measurement model for remote sensing image classification, which considers the spatial correlation between pixels in images and the effects of local spatial heterogeneity during uncertainty measurement. Specifically, the proposed model first measures the classification uncertainty of an image at the pixel and local spatial levels on the basis of the posterior probability of image classification. Second, the local spatial heterogeneity of an image is quantified, and the proposed model uses the local spatial heterogeneity of the image as a weight to adaptively fuse the uncertainties of the pixel and local spatial levels. Accordingly, a joint uncertainty measurement index is generated for a more accurate and effective evaluation of the uncertainty of remote sensing image classification. Lastly, the classification verification experiments on three publicly available remote sensing images with different spatial resolutions confirm the validity of the proposed model. Moreover, experimental results show that the proposed model has relative superiority and better stability than the existing and commonly used uncertainty measurement models (e.g., information entropy and Eastman’s U) in improving image classification performance.

Multi-Spectral Image Classification Based on an Object-Based Active Learning Approach

In remote sensing, active learning (AL) is considered to be an effective solution to the problem of producing sufficient classification accuracy with a limited number of training samples. Though this field has been extensively studied, most papers exist in the pixel-based paradigm. In object-based image analysis (OBIA), AL has been comparatively less studied. This paper aims to propose a new AL method for selecting object-based samples. The proposed AL method solves the problem of how to identify the most informative segment-samples so that classification performance can be optimized. The advantage of this algorithm is that informativeness can be estimated by using various object-based features. The new approach has three key steps. First, a series of one-against-one binary random forest (RF) classifiers are initialized by using a small initial training set. This strategy allows for the estimation of the classification uncertainty in great detail. Second, each tested sample is processed by using the binary RFs, and a classification uncertainty value that can reflect informativeness is derived. Third, the samples with high uncertainty values are selected and then labeled by a supervisor. They are subsequently added into the training set, based on which the binary RFs are re-trained for the next iteration. The whole procedure is iterated until a stopping criterion is met. To validate the proposed method, three pairs of multi-spectral remote sensing images with different landscape patterns were used in this experiment. The results indicate that the proposed method can outperform other state-of-the-art AL methods. To be more specific, the highest overall accuracies for the three datasets were all obtained by using the proposed AL method, and the values were 88.32%, 85.77%, and 93.12% for “T1,” “T2,” and “T3,” respectively. Furthermore, since object-based features have a serious impact on the performance of AL, eight combinations of four feature types are investigated. The results show that the best feature combination is different for the three datasets due to the variation of the feature separability.

IDENTIFICATION OF LOW ACCURACY REGIONS IN LAND COVER MAPS USING UNCERTAINTY MEASURES AND CLASSIFICATION CONFIDENCE

<p><strong>Abstract.</strong> The aim of this article is to assess if the data provided by soft classifiers and uncertainty measures can be used to identify regions with different levels of accuracy in a classified image. To this aim a soft Bayesian classifier was used, which enables the assignment of classifications confidence levels to all pixels. Two uncertainty measures were also used, namely the Relative Maximum Deviation (RMD) uncertainty measure and the Normalized Entropy (NE). The approach was tested on a case study. A multispectral IKONOS image was classified and the classification uncertainty and confidence where computed and analysed. Regions with different levels of uncertainty and confidence were identified. Reference datasets were then used to assess the classification accuracy of the whole study area and also in the regions with different levels of uncertainty and confidence. A comparative analysis was made on the variation of accuracy and classification uncertainty and confidence along the map and per class. The results show that for the regions with more uncertainty or less confidence the spatially constrained confusion matrices always generate lower values of global accuracy than for global accuracy of the regions with less uncertainty or more confidence. The analysis of the user’s and producer’s accuracy also shows the same general tendency. Proposals are then made on methodologies to use the information provided by the uncertainty and confidence to identify less reliable regions and also to improve classification results using fully automated approaches.</p>

RefSeq database growth influences the accuracy of k-mer-based species identification

ABSTRACTAccurate species-level taxonomic classification and profiling of complex microbial communities remains a challenge due to homologous regions shared among closely related species and a sparse representation of non-human associated microbes in the database. Although the database undoubtedly has a strong influence on the sensitivity of taxonomic classifiers and profilers, to date, no study has carefully explored this topic on historical RefSeq releases and explored its impact on accuracy. In this study, we examined the influence of the database, over time, on k-mer based sequence classification and profiling. We present three major findings: (i) database growth over time resulted in more classified reads, but fewer species-level classifications and more species-level misclassifications; (ii) Bayesian re-estimation of abundance helped to recover species-level classifications when the exact target strain was present; and (iii) Bayesian reestimation struggled when the database lacked the target strain, resulting in a notable decrease in accuracy. In summary, our findings suggest that the growth of RefSeq over time has strongly influenced the accuracy of k-mer based classification and profiling methods, resulting in different classification results depending on the particular database used. These results suggest a need for new algorithms specially adapted for large genome collections and better measures of classification uncertainty.

Assessing Uncertainty in LULC Classification Accuracy by Using Bootstrap Resampling

Supervised land-use/land-cover (LULC) classifications are typically conducted using class assignment rules derived from a set of multiclass training samples. Consequently, classification accuracy varies with the training data set and is thus associated with uncertainty. In this study, we propose a bootstrap resampling and reclassification approach that can be applied for assessing not only the uncertainty in classification results of the bootstrap-training data sets, but also the classification uncertainty of individual pixels in the study area. Two measures of pixel-specific classification uncertainty, namely the maximum class probability and Shannon entropy, were derived from the class probability vector of individual pixels and used for the identification of unclassified pixels. Unclassified pixels that are identified using the traditional chi-square threshold technique represent outliers of individual LULC classes, but they are not necessarily associated with higher classification uncertainty. By contrast, unclassified pixels identified using the equal-likelihood technique are associated with higher classification uncertainty and they mostly occur on or near the borders of different land-cover.