scholarly journals An Edge-Sense Bidirectional Pyramid Network for Stereo Matching of VHR Remote Sensing Images

2020 ◽  
Vol 12 (24) ◽  
pp. 4025
Author(s):  
Rongshu Tao ◽  
Yuming Xiang ◽  
Hongjian You
Keyword(s):  
Remote Sensing ◽  
Stereo Matching ◽  
Tall Buildings ◽  
Disparity Estimation ◽  
Complex Structures ◽  
Learning Networks ◽  
Remote Sensing Images ◽  
Essential Step ◽  
Disparity Range ◽  
The Cost

As an essential step in 3D reconstruction, stereo matching still faces unignorable problems due to the high resolution and complex structures of remote sensing images. Especially in occluded areas of tall buildings and textureless areas of waters and woods, precise disparity estimation has become a difficult but important task. In this paper, we develop a novel edge-sense bidirectional pyramid stereo matching network to solve the aforementioned problems. The cost volume is constructed from negative to positive disparities since the disparity range in remote sensing images varies greatly and traditional deep learning networks only work well for positive disparities. Then, the occlusion-aware maps based on the forward-backward consistency assumption are applied to reduce the influence of the occluded area. Moreover, we design an edge-sense smoothness loss to improve the performance of textureless areas while maintaining the main structure. The proposed network is compared with two baselines. The experimental results show that our proposed method outperforms two methods, DenseMapNet and PSMNet, in terms of averaged endpoint error (EPE) and the fraction of erroneous pixels (D1), and the improvements in occluded and textureless areas are significant.

scholarly journals A Disparity Refinement Algorithm for Satellite Remote Sensing Images Based on Mean-Shift Plane Segmentation

2021 ◽  
Vol 13 (10) ◽  
pp. 1903
Author(s):  
Zhihui Li ◽  
Jiaxin Liu ◽  
Yang Yang ◽  
Jing Zhang
Keyword(s):  
Remote Sensing ◽  
Satellite Remote Sensing ◽  
Stereo Matching ◽  
Mean Shift ◽  
Segmentation Method ◽  
Remote Sensing Images ◽  
Edge Matching ◽  
Refinement Method ◽  
Initial Segmentation ◽  
Refinement Algorithm

Objects in satellite remote sensing image sequences often have large deformations, and the stereo matching of this kind of image is so difficult that the matching rate generally drops. A disparity refinement method is needed to correct and fill the disparity. A method for disparity refinement based on the results of plane segmentation is proposed in this paper. The plane segmentation algorithm includes two steps: Initial segmentation based on mean-shift and alpha-expansion-based energy minimization. According to the results of plane segmentation and fitting, the disparity is refined by filling missed matching regions and removing outliers. The experimental results showed that the proposed plane segmentation method could not only accurately fit the plane in the presence of noise but also approximate the surface by plane combination. After the proposed plane segmentation method was applied to the disparity refinement of remote sensing images, many missed matches were filled, and the elevation errors were reduced. This proved that the proposed algorithm was effective. For difficult evaluations resulting from significant variations in remote sensing images of different satellites, the edge matching rate and the edge matching map are proposed as new stereo matching evaluation and analysis tools. Experiment results showed that they were easy to use, intuitive, and effective.

Semi-Global Stereo Matching of Remote Sensing Images Combined with Speeded up Robust Features

2020 ◽  
Vol 40 (16) ◽  
pp. 1628003
Author(s):  
王阳萍 Wang Yangping ◽  
秦安娜 Qin Anna ◽  
郝旗 Hao Qi ◽  
党建武 Dang Jianwu
Keyword(s):  
Remote Sensing ◽  
Stereo Matching ◽  
Remote Sensing Images ◽  
Speeded Up Robust Features

Stereo matching of remote sensing images using deep stereo matching

2021 ◽  
Author(s):  
Mang Chen ◽  
Johann Briffa ◽  
Gianluca Valentino ◽  
Reuben Farrugia
Keyword(s):  
Remote Sensing ◽  
Stereo Matching ◽  
Remote Sensing Images

scholarly journals Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching

2020 ◽  
Vol 34 (07) ◽  
pp. 12926-12934
Author(s):  
Youmin Zhang ◽  
Yimin Chen ◽  
Xiao Bai ◽  
Suihanjin Yu ◽  
Kun Yu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Loss Function ◽  
Stereo Matching ◽  
State Of The Art ◽  
Disparity Estimation ◽  
Model Matching ◽  
Regression Problem ◽  
Unimodal Distribution ◽  
Matching Process ◽  
The Cost

State-of-the-art deep learning based stereo matching approaches treat disparity estimation as a regression problem, where loss function is directly defined on true disparities and their estimated ones. However, disparity is just a byproduct of a matching process modeled by cost volume, while indirectly learning cost volume driven by disparity regression is prone to overfitting since the cost volume is under constrained. In this paper, we propose to directly add constraints to the cost volume by filtering cost volume with unimodal distribution peaked at true disparities. In addition, variances of the unimodal distributions for each pixel are estimated to explicitly model matching uncertainty under different contexts. The proposed architecture achieves state-of-the-art performance on Scene Flow and two KITTI stereo benchmarks. In particular, our method ranked the 1st place of KITTI 2012 evaluation and the 4th place of KITTI 2015 evaluation (recorded on 2019.8.20). The codes of AcfNet are available at: https://github.com/youmi-zym/AcfNet.

scholarly journals Semantic Segmentation for Buildings of Large Intra-Class Variation in Remote Sensing Images with O-GAN

2021 ◽  
Vol 13 (3) ◽  
pp. 475
Author(s):  
Shuting Sun ◽  
Lin Mu ◽  
Lizhe Wang ◽  
Peng Liu ◽  
Xiaolei Liu ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Economic Status ◽  
Back Propagation ◽  
Tall Buildings ◽  
Semantic Segmentation ◽  
Generative Adversarial Networks ◽  
Building Extraction ◽  
Remote Sensing Images ◽  
Feature Vectors ◽  
Classification Feature

Remote sensing building extraction is of great importance to many applications, such as urban planning and economic status assessment. Deep learning with deep network structures and back-propagation optimization can automatically learn features of targets in high-resolution remote sensing images. However, it is also obvious that the generalizability of deep networks is almost entirely dependent on the quality and quantity of the labels. Therefore, building extraction performances will be greatly affected if there is a large intra-class variation among samples of one class target. To solve the problem, a subdivision method for reducing intra-class differences is proposed to enhance semantic segmentation. We proposed that backgrounds and targets be separately generated by two orthogonal generative adversarial networks (O-GAN). The two O-GANs are connected by adding the new loss function to their discriminators. To better extract building features, drawing on the idea of fine-grained image classification, feature vectors for a target are obtained through an intermediate convolution layer of O-GAN with selective convolutional descriptor aggregation (SCDA). Subsequently, feature vectors are clustered into new, different subdivisions to train semantic segmentation networks. In the prediction stages, the subdivisions will be merged into one class. Experiments were conducted with remote sensing images of the Tibet area, where there are both tall buildings and herdsmen’s tents. The results indicate that, compared with direct semantic segmentation, the proposed subdivision method can make an improvement on accuracy of about 4%. Besides, statistics and visualizing building features validated the rationality of features and subdivisions.

scholarly journals Saliency-Guided Remote Sensing Image Super-Resolution

2021 ◽  
Vol 13 (24) ◽  
pp. 5144
Author(s):  
Baodi Liu ◽  
Lifei Zhao ◽  
Jiaoyue Li ◽  
Hengle Zhao ◽  
Weifeng Liu ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Saliency Detection ◽  
Super Resolution ◽  
Remote Sensing Image ◽  
Complex Structures ◽  
Remote Sensing Images ◽  
Saliency Maps ◽  
Image Super Resolution ◽  
The One ◽  
Visual Results

Deep learning has recently attracted extensive attention and developed significantly in remote sensing image super-resolution. Although remote sensing images are composed of various scenes, most existing methods consider each part equally. These methods ignore the salient objects (e.g., buildings, airplanes, and vehicles) that have more complex structures and require more attention in recovery processing. This paper proposes a saliency-guided remote sensing image super-resolution (SG-GAN) method to alleviate the above issue while maintaining the merits of GAN-based methods for the generation of perceptual-pleasant details. More specifically, we exploit the salient maps of images to guide the recovery in two aspects: On the one hand, the saliency detection network in SG-GAN learns more high-resolution saliency maps to provide additional structure priors. On the other hand, the well-designed saliency loss imposes a second-order restriction on the super-resolution process, which helps SG-GAN concentrate more on the salient objects of remote sensing images. Experimental results show that SG-GAN achieves competitive PSNR and SSIM compared with the advanced super-resolution methods. Visual results demonstrate our superiority in restoring structures while generating remote sensing super-resolution images.

scholarly journals BETTER GENERIC OBJECTS COUNTING WHEN ASKING QUESTIONS TO IMAGES: A MULTITASK APPROACH FOR REMOTE SENSING VISUAL QUESTION ANSWERING

2020 ◽  
Vol V-2-2020 ◽  
pp. 1021-1027
Author(s):  
S. Lobry ◽  
D. Marcos ◽  
B. Kellenberger ◽  
D. Tuia
Keyword(s):  
Remote Sensing ◽  
Natural Language ◽  
Question Answering ◽  
Classification Problem ◽  
Remote Sensing Images ◽  
Regression Problem ◽  
Visual Question Answering ◽  
Multiple Orders ◽  
Asking Questions ◽  
The Cost

Abstract. Visual Question Answering for Remote Sensing (RSVQA) aims at extracting information from remote sensing images through queries formulated in natural language. Since the answer to the query is also provided in natural language, the system is accessible to non-experts, and therefore dramatically increases the value of remote sensing images as a source of information, for example for journalism purposes or interactive land planning. Ideally, an RSVQA system should be able to provide an answer to questions that vary both in terms of topic (presence, localization, counting) and image content. However, aiming at such flexibility generates problems related to the variability of the possible answers. A striking example is counting, where the number of objects present in a remote sensing image can vary by multiple orders of magnitude, depending on both the scene and type of objects. This represents a challenge for traditional Visual Question Answering (VQA) methods, which either become intractable or result in an accuracy loss, as the number of possible answers has to be limited. To this end, we introduce a new model that jointly solves a classification problem (which is the most common approach in VQA) and a regression problem (to answer numerical questions more precisely). An evaluation of this method on the RSVQA dataset shows that this finer numerical output comes at the cost of a small loss of performance on non-numerical questions.

Sign in / Sign up

Export Citation Format

Share Document