Modality-Free Feature Detector and Descriptor for Multimodal Remote Sensing Image Registration

The nonlinear radiation distortions (NRD) among multimodal remote sensing images bring enormous challenges to image registration. The traditional feature-based registration methods commonly use the image intensity or gradient information to detect and describe the features that are sensitive to NRD. However, the nonlinear mapping of the corresponding features of the multimodal images often results in failure of the feature matching, as well as the image registration. In this paper, a modality-free multimodal remote sensing image registration method (SRIFT) is proposed for the registration of multimodal remote sensing images, which is invariant to scale, radiation, and rotation. In SRIFT, the nonlinear diffusion scale (NDS) space is first established to construct a multi-scale space. A local orientation and scale phase congruency (LOSPC) algorithm are then used so that the features of the images with NRD are mapped to establish a one-to-one correspondence, to obtain sufficiently stable key points. In the feature description stage, a rotation-invariant coordinate (RIC) system is adopted to build a descriptor, without requiring estimation of the main direction. The experiments undertaken in this study included one set of simulated data experiments and nine groups of experiments with different types of real multimodal remote sensing images with rotation and scale differences (including synthetic aperture radar (SAR)/optical, digital surface model (DSM)/optical, light detection and ranging (LiDAR) intensity/optical, near-infrared (NIR)/optical, short-wave infrared (SWIR)/optical, classification/optical, and map/optical image pairs), to test the proposed algorithm from both quantitative and qualitative aspects. The experimental results showed that the proposed method has strong robustness to NRD, being invariant to scale, radiation, and rotation, and the achieved registration precision was better than that of the state-of-the-art methods.

Download Full-text

Remote Sensing Image Registration Using Multiple Image Features

10.20944/preprints201705.0027.v2 ◽

2017 ◽

Author(s):

Kun Yang ◽

Anning Pan ◽

Yang Yang ◽

Su Zhang ◽

Sim Heng Ong ◽

...

Keyword(s):

Remote Sensing ◽

Image Registration ◽

Mixture Model ◽

Damage Assessment ◽

Remote Sensing Image ◽

Image Features ◽

Google Earth ◽

Geometric Distortion ◽

Remote Sensing Images ◽

Registration Method

Remote sensing image registration plays an important role in military and civilian fields, such as natural disaster damage assessment, military damage assessment and ground targets identification, etc. However, due to the ground relief variations and imaging viewpoint changes, non-rigid geometric distortion occurs between remote sensing images with different viewpoint, which further increases the difficulty of remote sensing image registration. To address the problem, we propose a multi-viewpoint remote sensing image registration method which contains the following contributions. (i) A multiple features based finite mixture model is constructed for dealing with different types of image features. (ii) Three features are combined and substituted into the mixture model to form a feature complementation, i.e., the Euclidean distance and shape context are used to measure the similarity of geometric structure, and the SIFT (scale-invariant feature transform) distance which is endowed with the intensity information is used to measure the scale space extrema. (iii) To prevent the ill-posed problem, a geometric constraint term is introduced into the L2E-based energy function for better behaving the non-rigid transformation. We evaluated the performances of the proposed method by three series of remote sensing images obtained from the unmanned aerial vehicle (UAV) and Google Earth, and compared with five state-of-the-art methods where our method shows the best alignments in most cases.

Download Full-text

Robust Multisource Remote Sensing Image Registration Method Based on Scene Shape Similarity

Photogrammetric Engineering & Remote Sensing ◽

10.14358/pers.85.10.725 ◽

2019 ◽

Vol 85 (10) ◽

pp. 725-736 ◽

Cited By ~ 3

Author(s):

Ming Hao ◽

Jian Jin ◽

Mengchao Zhou ◽

Yi Tian ◽

Wenzhong Shi

Keyword(s):

Remote Sensing ◽

Image Registration ◽

Similarity Measure ◽

Template Matching ◽

Remote Sensing Image ◽

Shape Similarity ◽

Remote Sensing Images ◽

Registration Method ◽

Sensing Applications ◽

Global Consistency

Image registration is an indispensable component of remote sensing applications, such as disaster monitoring, change detection, and classification. Grayscale differences and geometric distortions often occur among multisource images due to their different imaging mechanisms, thus making it difficult to acquire feature points and match corresponding points. This article proposes a scene shape similarity feature (SSSF) descriptor based on scene shape features and shape context algorithms. A new similarity measure called SSSFncc is then defined by computing the normalized correlation coefficient of the SSSF descriptors between multisource remote sensing images. Furthermore, the tie points between the reference and the sensed image are extracted via a template matching strategy. A global consistency check method is then used to remove the mismatched tie points. Finally, a piecewise linear transform model is selected to rectify the remote sensing image. The proposed SSSFncc aims to extract the scene shape similarity between multisource images. The accuracy of the proposed SSSFncc is evaluated using five pairs of experimental images from optical, synthetic aperture radar, and map data. Registration results demonstrate that the SSSFncc similarity measure is robust enough for complex nonlinear grayscale differences among multisource remote sensing images. The proposed method achieves more reliable registration outcomes compared with other popular methods.

Download Full-text

An Instance Segmentation Based Framework for Large-Sized High-Resolution Remote Sensing Images Registration

Remote Sensing ◽

10.3390/rs13091657 ◽

2021 ◽

Vol 13 (9) ◽

pp. 1657

Author(s):

Junyan Lu ◽

Hongguang Jia ◽

Tie Li ◽

Zhuqiang Li ◽

Jingyu Ma ◽

...

Keyword(s):

Remote Sensing ◽

High Resolution ◽

Image Registration ◽

Feature Matching ◽

Local Area ◽

Remote Sensing Image ◽

Remote Sensing Images ◽

Fine Grained ◽

Feature Based ◽

Instance Segmentation

Feature-based remote sensing image registration methods have achieved great accomplishments. However, they have faced some limitations of applicability, automation, accuracy, efficiency, and robustness for large high-resolution remote sensing image registration. To address the above issues, we propose a novel instance segmentation based registration framework specifically for large-sized high-resolution remote sensing images. First, we design an instance segmentation model based on a convolutional neural network (CNN), which can efficiently extract fine-grained instances as the deep features for local area matching. Then, a feature-based method combined with the instance segmentation results is adopted to acquire more accurate local feature matching. Finally, multi-constraints based on the instance segmentation results are introduced to work on the outlier removal. In the experiments of high-resolution remote sensing image registration, the proposal effectively copes with the circumstance of the sensed image with poor positioning accuracy. In addition, the method achieves superior accuracy and competitive robustness compared with state-of-the-art feature-based methods, while being rather efficient.

Download Full-text

An Image Registration Method Using Deep Residual Network Features for Multisource High-Resolution Remote Sensing Images

Remote Sensing ◽

10.3390/rs13173425 ◽

2021 ◽

Vol 13 (17) ◽

pp. 3425

Author(s):

Xin Zhao ◽

Hui Li ◽

Ping Wang ◽

Linhai Jing

Keyword(s):

Remote Sensing ◽

High Resolution ◽

Image Registration ◽

Feature Descriptor ◽

Remote Sensing Images ◽

Residual Network ◽

Scale Invariant ◽

Registration Method ◽

Sensing Applications ◽

Feature Based

Accurate registration for multisource high-resolution remote sensing images is an essential step for various remote sensing applications. Due to the complexity of the feature and texture information of high-resolution remote sensing images, especially for images covering earthquake disasters, feature-based image registration methods need a more helpful feature descriptor to improve the accuracy. However, traditional image registration methods that only use local features at low levels have difficulty representing the features of the matching points. To improve the accuracy of matching features for multisource high-resolution remote sensing images, an image registration method based on a deep residual network (ResNet) and scale-invariant feature transform (SIFT) was proposed. It used the fusion of SIFT features and ResNet features on the basis of the traditional algorithm to achieve image registration. The proposed method consists of two parts: model construction and training and image registration using a combination of SIFT and ResNet34 features. First, a registration sample set constructed from high-resolution satellite remote sensing images was used to fine-tune the network to obtain the ResNet model. Then, for the image to be registered, the Shi_Tomas algorithm and the combination of SIFT and ResNet features were used for feature extraction to complete the image registration. Considering the difference in image sizes and scenes, five pairs of images were used to conduct experiments to verify the effectiveness of the method in different practical applications. The experimental results showed that the proposed method can achieve higher accuracies and more tie points than traditional feature-based methods.

Download Full-text

A Robust False Matching Points Detection Method for Remote Sensing Image Registration

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xl-7-w3-699-2015 ◽

2015 ◽

Vol XL-7/W3 ◽

pp. 699-702

Author(s):

X. J. Shan ◽

P. Tang

Keyword(s):

Remote Sensing ◽

Image Registration ◽

Detection Method ◽

Graph Transformation ◽

Remote Sensing Image ◽

Transformation Model ◽

Superior Performance ◽

Geometric Distortion ◽

Remote Sensing Images ◽

Accuracy Result

Given the influences of illumination, imaging angle, and geometric distortion, among others, false matching points still occur in all image registration algorithms. Therefore, false matching points detection is an important step in remote sensing image registration. Random Sample Consensus (RANSAC) is typically used to detect false matching points. However, RANSAC method cannot detect all false matching points in some remote sensing images. Therefore, a robust false matching points detection method based on Knearest- neighbour (K-NN) graph (KGD) is proposed in this method to obtain robust and high accuracy result. The KGD method starts with the construction of the K-NN graph in one image. K-NN graph can be first generated for each matching points and its K nearest matching points. Local transformation model for each matching point is then obtained by using its K nearest matching points. The error of each matching point is computed by using its transformation model. Last, L matching points with largest error are identified false matching points and removed. This process is iterative until all errors are smaller than the given threshold. In addition, KGD method can be used in combination with other methods, such as RANSAC. Several remote sensing images with different resolutions and terrains are used in the experiment. We evaluate the performance of KGD method, RANSAC + KGD method, RANSAC, and Graph Transformation Matching (GTM). The experimental results demonstrate the superior performance of the KGD and RANSAC + KGD methods.

Download Full-text