A Hybrid Sparse Representation Model for Image Restoration

Group-based sparse representation (GSR) uses image nonlocal self-similarity (NSS) prior to grouping similar image patches, and then performs sparse representation. However, the traditional GSR model restores the image by training degraded images, which leads to the inevitable over-fitting of the data in the training model, resulting in poor image restoration results. In this paper, we propose a new hybrid sparse representation model (HSR) for image restoration. The proposed HSR model is improved in two aspects. On the one hand, the proposed HSR model exploits the NSS priors of both degraded images and external image datasets, making the model complementary in feature space and the plane. On the other hand, we introduce a joint sparse representation model to make better use of local sparsity and NSS characteristics of the images. This joint model integrates the patch-based sparse representation (PSR) model and GSR model, while retaining the advantages of the GSR model and the PSR model, so that the sparse representation model is unified. Extensive experimental results show that the proposed hybrid model outperforms several existing image recovery algorithms in both objective and subjective evaluations.

SI-STSAR-7: A Large SAR Images Dataset with Spatial and Temporal Information for Classification of Winter Sea Ice in Hudson Bay

Remote sensing satellites have been broadly applied to sea ice monitoring. The substantial increase in satellite imagery provides a large amount of data support for deep learning methods in the sea ice classification field. However, there is a lack of public remote sensing datasets to facilitate sea ice classification with spatial and temporal information and to benchmark the deep learning methods. In this paper, we provide a labeled large sea ice dataset derived from time-series sentinel-1 SAR images, dubbed SI-STSAR-7, and a validated dataset construction method for sea ice classification research. The SI-STSAR-7 dataset includes seven different sea ice types corresponding to different sea ice development stages in Hudson Bay during winter, and its samples are time sequences of SAR image patches in order to embody the differences of backscattering intensity and textures between different sea ice types, as well as the change of sea ice with time. We construct the dataset by first performing noise reduction and mitigation of incidence angle dependence on SAR images, and then producing data samples and labeling them based on our proposed sample-producing principles and the weekly regional ice charts provided by Canadian Ice Service. Three baseline classification methods are developed on SI-STSAR-7 to establish benchmarks, which are evaluated with accuracy and kappa coefficient. The sample-producing principles are verified through experiments. Based on the experimental results, sea ice classification can be implemented well on SI-STSAR-7.

Object Detection of Road Assets Using Transformer-Based YOLOX with Feature Pyramid Decoder on Thai Highway Panorama

Due to the various sizes of each object, such as kilometer stones, detection is still a challenge, and it directly impacts the accuracy of these object counts. Transformers have demonstrated impressive results in various natural language processing (NLP) and image processing tasks due to long-range modeling dependencies. This paper aims to propose an exceeding you only look once (YOLO) series with two contributions: (i) We propose to employ a pre-training objective to gain the original visual tokens based on the image patches on road asset images. By utilizing pre-training Vision Transformer (ViT) as a backbone, we immediately fine-tune the model weights on downstream tasks by joining task layers upon the pre-trained encoder. (ii) We apply Feature Pyramid Network (FPN) decoder designs to our deep learning network to learn the importance of different input features instead of simply summing up or concatenating, which may cause feature mismatch and performance degradation. Conclusively, our proposed method (Transformer-Based YOLOX with FPN) learns very general representations of objects. It significantly outperforms other state-of-the-art (SOTA) detectors, including YOLOv5S, YOLOv5M, and YOLOv5L. We boosted it to 61.5% AP on the Thailand highway corpus, surpassing the current best practice (YOLOv5L) by 2.56% AP for the test-dev data set.

EVALUATION OF EFFECTIVENESS OF PATCH BASED IMAGE CLASSIFICATION TECHNIQUE USING HIGH RESOLUTION WORLDVIEW-2 IMAGE

The aim of the current study was to evaluate the performance of patch-based classification technique in land use/land cover classification and to investigate the effect of patch size in thematic map accuracy. To reach desired goal, recently proposed ensemble learning classifiers (i.e., XGBoost and CatBoost) were utilized to classify produced image patches obtained from high-resolution WorldView-2 (WV-2) satellite image. . In order to analyse the effect of varying patch size on classification accuracy, three different window sizes (i.e., 3 × 3, 7 × 7 and 11 × 11) were applied to WV-2 imagery for extracting image patches. Constructed image patches were classified using XGBoost and CatBoost ensemble learning classifiers and thematic maps were constructed for varying patch sizes. Results showed that while XGBoost and CatBoost showed similar classification performances for varying patch size and the estimated highest overall accuracy were %68, %82 and %92 for 11x11, 7 × 7 and 11 × 11 patch sizes, respectively. These findings confirmed that defining class boundaries on the high-resolution image using smaller patches increases the accuracy of thematic maps. In addition, results of patch-based classification were compared the results of LULC maps produced by same classifiers using pixel-based classification method. Overall accuracy of pixel-by-pixel classification of WV-2 image reached to about %94. Furthermore, CatBoost showed superior classification performance in all time compared to XGBoost. All in all, pixel-based CatBoost was found to be more successful in LULC mapping of fine resolution image.

A multi-stage algorithm for image denoising based on PCA and adaptive TV-regularization

In this paper, we present an image denoising algorithm comprising three stages. In the first stage, Principal Component Analysis (PCA) is used to suppress the noise. PCA is applied to image blocks to characterize localized features and rare image patches. In the second stage, we use the Gaussian curvature to develop an adaptive total-variation-based (TV) denoising model to effectively remove visual artifacts and noise residual generated by the first stage. Finally, the denoised image is sharpened in order to enhance the contrast of the denoising result. Experimental results on natural images and computed tomography (CT) images demonstrated that the proposed algorithm yields denoising results better than competing algorithms in terms of both qualitative and quantitative aspects.

The lower bound of nonlocal gradient for non-convex and non-smooth image patches based regularization

In the inverse problem of image processing, we have witnessed that the non-convex and non-smooth regularizers can produce clearer image edges than convex ones such as total variation (TV). This fact can be explained by the uniform lower bound theory of the local gradient in non-convex and non-smooth regularization. In recent years, although it has been numerically shown that the nonlocal regularizers of various image patches based nonlocal methods can recover image textures well, we still desire a theoretical interpretation. To this end, we propose a non-convex non-smooth and block nonlocal (NNBN) regularization model based on image patches. By integrating the advantages of the non-convex and non-smooth potential function in the regularization term, the uniform lower bound theory of the image patches based nonlocal gradient is given. This approach partially explains why the proposed method can produce clearer image textures and edges. Compared to some classical regularization methods, such as total variation (TV), non-convex and non-smooth (NN) regularization, nonlocal total variation (NLTV) and block nonlocal total variation(BNLTV), our experimental results show that the proposed method improves restoration quality.

SiDroForest: A comprehensive forest inventory of Siberian boreal forest investigations including drone-based point clouds, individually labelled trees, synthetically generated tree crowns and Sentinel-2 labelled image patches

This data collection is an attempt to remedy the scarcity of tree level forest structure data in the circum-boreal region, whilst providing, as part of the data collection, adjusted and labelled tree level and vegetation plot level data for machine learning and upscaling practices. Publicly available comprehensive datasets on tree level forest structure are rare, due to the involvement of governmental agencies, public sectors, and private actors that all influence the availability of these datasets. We present datasets of vegetation composition and tree and plot level forest structure for two important vegetation transition zones in Siberia, Russia; the summergreen–evergreen transition zone in central Yakutia and the tundra–taiga transition zone in Chukotka (NE Siberia). The SiDroForest collection contains a variety of data mainly based on unmanned aerial vehicle (UAV) and field data collected from 64 vegetation plots during fieldwork jointly performed by the Alfred Wegener Institute for Polar and Marine Research (AWI) and the North-Eastern Federal University of Yakutsk (NEFU) during the Chukotka 2018 expedition to Siberia. The data collection consists of four separate datasets. The fieldwork locations are the anchors that bind the data types together based on the location of the vegetation plot. i) The first dataset (Kruse et al., 2021, https://doi.pangaea.de/10.1594/PANGAEA.933263) provides UAV-borne data products covering the 64 vegetation plots surveyed during fieldwork: including structure from motion (SfM) point clouds, point-cloud products such as Digital Elevation Model (DEM), Canopy Height Model (CHM), Digital Surface Model (DSM) and Digital Terrain Model (DTM) constructed from Red Green Blue (RGB) and Red Green Near Infrared (RGN) orthomosaics. Forest structure and vegetation composition data are crucial in the assessment of whether a forest is to act as a carbon sink under changing climate conditions. Fieldwork and UAV-products can provide such data in depth. ii) The second dataset contains spatial data in the form of points and polygon shape files of 872 labelled individual trees and shrubs that were recorded during fieldwork at the same vegetation plots with information on tree height, crown diameter, and species (van Geffen et al., 2021c, https://doi.pangaea.de/10.1594/PANGAEA.932821). These tree- and shrub-individual labelled point and polygon shape files were generated and are located on the UAV RGB orthoimages. The individual number links to the information collected during the expedition such as tree height, crown diameter and vitality provided in table format. This dataset can be used to link individual trees in the SfM point clouds, providing unique insights into the vegetation composition and also allows future monitoring of the individual trees and the contents of the recorded vegetation plots at large. iii) The third dataset contains a synthesis of 10 000 generated images and masks that have the tree crowns of two species of larch (Larix gmelinii and Larix cajanderi) automatically extracted from the RGB UAV images in the common objects in context (COCO) format (van Geffen et al., 2021a, https://doi.pangaea.de/10.1594/PANGAEA.932795). The synthetic dataset was created specifically to detect Siberian larch species. iv) If publicly available forest-structure datasets at tree level are rarely available for Siberia, even fewer ready-to-use tree and plot level data are available for machine learning approaches, for example optimised data formats containing annotated vegetation categories. The fourth set contains Sentinel-2 Level-2 bottom of atmosphere labelled image patches with seasonal information and annotated vegetation categories covering the vegetation plots (van Geffen et al., 2021b, https://doi.pangaea.de/10.1594/PANGAEA.933268). The dataset is created with the aim of providing a small ready-to use validation and training data set to be used in various vegetation-related machine-learning tasks. The SidroForest data collection serves a variety of user communities. First of all, the UAV-derived top of canopy structure information, orthomosaics and the detailed vegetation information in the labelled data set provide detailed information on forest type, structure and composition for scientific communities with ecological and biological applications. The detailed Land Cover and Vegetation structure information in the first two data sets are of use for the generation and validation of Land Cover remote sensing products in radar and optical remote sensing. In addition to providing information on forest structure and vegetation composition of the vegetation plots, parts of the SiDroForest dataset are prepared to be used as training and validation data for machine learning purposes. For example, the Synthetic tree crown dataset is generated from the raw UAV images and optimized to be used in neural networks. Furthermore, the fourth SiDroForest data set contains standardized Sentinel-2 labelled image patches that provide training data on vegetation class categories for machine learning classification with JSON labels provided. The SiDroForst data collective serves as a basis to add future data collected during expeditions performed by the Alfred Wegener Institute, creating a larger dataset in the upcoming years that can provide unique insights into remote hard to reach boreal regions of Siberia.

Investigating the Upper-Bound Performance of Sparse-Coding-Based Spectral Reconstruction from RGB Images

In Spectral Reconstruction (SR), we recover hyperspectral images from their RGB counterparts. Most of the recent approaches are based on Deep Neural Networks (DNN), where millions of parameters are trained mainly to extract and utilize the contextual features in large image patches as part of the SR process. On the other hand, the leading Sparse Coding method ‘A+’—which is among the strongest point-based baselines against the DNNs—seeks to divide the RGB space into neighborhoods, where locally a simple linear regression (comprised by roughly 102 parameters) suffices for SR. In this paper, we explore how the performance of Sparse Coding can be further advanced. We point out that in the original A+, the sparse dictionary used for neighborhood separations are optimized for the spectral data but used in the projected RGB space. In turn, we demonstrate that if the local linear mapping is trained for each spectral neighborhood instead of RGB neighborhood (and theoretically if we could recover each spectrum based on where it locates in the spectral space), the Sparse Coding algorithm can actually perform much better than the leading DNN method. In effect, our result defines one potential (and very appealing) upper-bound performance of point-based SR.