Feature Matching Based on Triangle Guidance and Constraints

For images with distortions or repetitive patterns, the existing matching methods usually work well just on one of the two kinds of images. In this paper, we present novel triangle guidance and constraints (TGC)-based feature matching method, which can achieve good results on both kinds of images. We first extract stable matched feature points and combine these points into triangles as the initial matched triangles, and triangles combined by feature points are as the candidates to be matched. Then, triangle guidance based on the connection relationship via the shared feature point between the matched triangles and the candidates is defined to find the potential matching triangles. Triangle constraints, specially the location of a vertex relative to the inscribed circle center of the triangle, the scale represented by the ratio of corresponding side lengths of two matching triangles and the included angles between the sides of two triangles with connection relationship, are subsequently used to verify the potential matches and obtain the correct ones. Comparative experiments show that the proposed TGC can increase the number of the matched points with high accuracy under various image transformations, especially more effective on images with distortions or repetitive patterns due to the fact that the triangular structure are not only stable to image transformations but also provides more geometric constraints.

Download Full-text

Study on Reconstruction and Feature Tracking of Silicone Heart 3D Surface

Sensors ◽

10.3390/s21227570 ◽

2021 ◽

Vol 21 (22) ◽

pp. 7570

Author(s):

Ziyan Zhang ◽

Yan Liu ◽

Jiawei Tian ◽

Shan Liu ◽

Bo Yang ◽

...

Keyword(s):

Surface Reconstruction ◽

Feature Tracking ◽

Feature Matching ◽

Negative Impact ◽

Three Dimensional ◽

Geometric Constraints ◽

Feature Point ◽

Feature Points ◽

3D Surface Reconstruction ◽

3D Surface

At present, feature-based 3D reconstruction and tracking technology is widely applied in the medical field. In minimally invasive surgery, the surgeon can achieve three-dimensional reconstruction through the images obtained by the endoscope in the human body, restore the three-dimensional scene of the area to be operated on, and track the motion of the soft tissue surface. This enables doctors to have a clearer understanding of the location depth of the surgical area, greatly reducing the negative impact of 2D image defects and ensuring smooth operation. In this study, firstly, the 3D coordinates of each feature point are calculated by using the parameters of the parallel binocular endoscope and the spatial geometric constraints. At the same time, the discrete feature points are divided into multiple triangles using the Delaunay triangulation method. Then, the 3D coordinates of feature points and the division results of each triangle are combined to complete the 3D surface reconstruction. Combined with the feature matching method based on convolutional neural network, feature tracking is realized by calculating the three-dimensional coordinate changes of the same feature point in different frames. Finally, experiments are carried out on the endoscope image to complete the 3D surface reconstruction and feature tracking.

Download Full-text

3D Mesh Pre-processing Method Based on Feature Point Classification and Anisotropic Vertex Denoising Considering Scene Structure Characteristics

Remote Sensing ◽

10.3390/rs13112145 ◽

2021 ◽

Vol 13 (11) ◽

pp. 2145

Author(s):

Yawen Liu ◽

Bingxuan Guo ◽

Xiongwu Xiao ◽

Wei Qiu

Keyword(s):

Real Data ◽

Geometric Constraints ◽

Feature Point ◽

Feature Points ◽

Denoising Method ◽

3D Mesh ◽

Mesh Denoising ◽

Scene Structure ◽

Sharp Edges ◽

Update Strategy

3D mesh denoising plays an important role in 3D model pre-processing and repair. A fundamental challenge in the mesh denoising process is to accurately extract features from the noise and to preserve and restore the scene structure features of the model. In this paper, we propose a novel feature-preserving mesh denoising method, which was based on robust guidance normal estimation, accurate feature point extraction and an anisotropic vertex denoising strategy. The methodology of the proposed approach is as follows: (1) The dual weight function that takes into account the angle characteristics is used to estimate the guidance normals of the surface, which improved the reliability of the joint bilateral filtering algorithm and avoids losing the corner structures; (2) The filtered facet normal is used to classify the feature points based on the normal voting tensor (NVT) method, which raised the accuracy and integrity of feature classification for the noisy model; (3) The anisotropic vertex update strategy is used in triangular mesh denoising: updating the non-feature points with isotropic neighborhood normals, which effectively suppressed the sharp edges from being smoothed; updating the feature points based on local geometric constraints, which preserved and restored the features while avoided sharp pseudo features. The detailed quantitative and qualitative analyses conducted on synthetic and real data show that our method can remove the noise of various mesh models and retain or restore the edge and corner features of the model without generating pseudo features.

Download Full-text

Extraction and Research of Crop Feature Points Based on Computer Vision

Sensors ◽

10.3390/s19112553 ◽

2019 ◽

Vol 19 (11) ◽

pp. 2553 ◽

Cited By ~ 3

Author(s):

Jingwen Cui ◽

Jianping Zhang ◽

Guiling Sun ◽

Bowen Zheng

Keyword(s):

Computer Vision ◽

Point Cloud ◽

High Accuracy ◽

Depth Camera ◽

Positioning Error ◽

Feature Points ◽

Matching Method ◽

Depth Images ◽

Computer Vision Technology ◽

Rgb Images

Based on computer vision technology, this paper proposes a method for identifying and locating crops in order to successfully capture crops in the process of automatic crop picking. This method innovatively combines the YOLOv3 algorithm under the DarkNet framework with the point cloud image coordinate matching method, and can achieve the goal of this paper very well. Firstly, RGB (RGB is the color representing the three channels of red, green and blue) images and depth images are obtained by using the Kinect v2 depth camera. Secondly, the YOLOv3 algorithm is used to identify the various types of target crops in the RGB images, and the feature points of the target crops are determined. Finally, the 3D coordinates of the feature points are displayed on the point cloud images. Compared with other methods, this method of crop identification has high accuracy and small positioning error, which lays a good foundation for the subsequent harvesting of crops using mechanical arms. In summary, the method used in this paper can be considered effective.

Download Full-text

Robust Feature Matching with Spatial Smoothness Constraints

Remote Sensing ◽

10.3390/rs12193158 ◽

2020 ◽

Vol 12 (19) ◽

pp. 3158

Author(s):

Xu Huang ◽

Xue Wan ◽

Daifeng Peng

Keyword(s):

Energy Function ◽

Feature Matching ◽

Feature Point ◽

Limiting Factors ◽

Feature Points ◽

Point Sets ◽

Matching Accuracy ◽

Matching Problem ◽

Underlying Assumption ◽

Speeded Up Robust Features

Feature matching is to detect and match corresponding feature points in stereo pairs, which is one of the key techniques in accurate camera orientations. However, several factors limit the feature matching accuracy, e.g., image textures, viewing angles of stereo cameras, and resolutions of stereo pairs. To improve the feature matching accuracy against these limiting factors, this paper imposes spatial smoothness constraints over the whole feature point sets with the underlying assumption that feature points should have similar matching results with their surrounding high-confidence points and proposes a robust feature matching method with the spatial smoothness constraints (RMSS). The core algorithm constructs a graph structure from the feature point sets and then formulates the feature matching problem as the optimization of a global energy function with first-order, spatial smoothness constraints based on the graph. For computational purposes, the global optimization of the energy function is then broken into sub-optimizations of each feature point, and an approximate solution of the energy function is iteratively derived as the matching results of the whole feature point sets. Experiments on close-range datasets with some above limiting factors show that the proposed method was capable of greatly improving the matching robustness and matching accuracy of some feature descriptors (e.g., scale-invariant feature transform (SIFT) and Speeded Up Robust Features (SURF)). After the optimization of the proposed method, the inlier number of SIFT and SURF was increased by average 131.9% and 113.5%, the inlier percentages between the inlier number and the total matches number of SIFT and SURF were increased by average 259.0% and 307.2%, and the absolute matching accuracy of SIFT and SURF was improved by average 80.6% and 70.2%.

Download Full-text

A NEW PARADIGM FOR MATCHING UAV- AND AERIAL IMAGES

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsannals-iii-3-83-2016 ◽

2016 ◽

Vol III-3 ◽

pp. 83-90 ◽

Cited By ~ 3

Author(s):

T. Koch ◽

X. Zhuo ◽

P. Reinartz ◽

F. Fraundorfer

Keyword(s):

Real World ◽

Image Matching ◽

Feature Matching ◽

Ground Truth ◽

Geometric Constraints ◽

Feature Point ◽

Aerial Images ◽

Aerial Image ◽

Ratio Test ◽

New Paradigm

This paper investigates the performance of SIFT-based image matching regarding large differences in image scaling and rotation, as this is usually the case when trying to match images captured from UAVs and airplanes. This task represents an essential step for image registration and 3d-reconstruction applications. Various real world examples presented in this paper show that SIFT, as well as A-SIFT perform poorly or even fail in this matching scenario. Even if the scale difference in the images is known and eliminated beforehand, the matching performance suffers from too few feature point detections, ambiguous feature point orientations and rejection of many correct matches when applying the ratio-test afterwards. Therefore, a new feature matching method is provided that overcomes these problems and offers thousands of matches by a novel feature point detection strategy, applying a one-to-many matching scheme and substitute the ratio-test by adding geometric constraints to achieve geometric correct matches at repetitive image regions. This method is designed for matching almost nadir-directed images with low scene depth, as this is typical in UAV and aerial image matching scenarios. We tested the proposed method on different real world image pairs. While standard SIFT failed for most of the datasets, plenty of geometrical correct matches could be found using our approach. Comparing the estimated fundamental matrices and homographies with ground-truth solutions, mean errors of few pixels can be achieved.

Download Full-text

Flexible point cloud matching method based on three-dimensional image feature points

Advances in Mechanical Engineering ◽

10.1177/1687814018795032 ◽

2018 ◽

Vol 10 (9) ◽

pp. 168781401879503

Author(s):

Haihua Cui ◽

Wenhe Liao ◽

Xiaosheng Cheng ◽

Ning Dai ◽

Changye Guo

Keyword(s):

Point Cloud ◽

Three Dimensional ◽

Point Clouds ◽

Feature Point ◽

Image Feature ◽

Surface Measurement ◽

Surface Model ◽

Feature Points ◽

Matching Method ◽

Dimensional Image

Flexible and robust point cloud matching is important for three-dimensional surface measurement. This article proposes a new matching method based on three-dimensional image feature points. First, an intrinsic shape signature algorithm is used to detect the key shape feature points, using a weighted three-dimensional occupational histogram of the data points within the angular space, which is a view-independent representation of the three-dimensional shape. Then, the point feature histogram is used to represent the underlying surface model properties at a point whose computation is based on the combination of certain geometrical relations between the point’s nearest k-neighbors. The two-view point clouds are robustly matched using the proposed double neighborhood constraint of minimizing the sum of the Euclidean distances between the local neighbors of the point and feature point. The proposed optimization method is immune to noise, reduces the search range for matching points, and improves the correct feature point matching rate for a weak surface texture. The matching accuracy and stability of the proposed method are verified using experiments. This method can be used for a flat surface with weak features and in other applications. The method has a larger application range than the traditional methods.

Download Full-text

An Improved Method for Stable Feature Points Selection in Structure-from-Motion Considering Image Semantic and Structural Characteristics

Sensors ◽

10.3390/s21072416 ◽

2021 ◽

Vol 21 (7) ◽

pp. 2416

Author(s):

Fei Wang ◽

Zhendong Liu ◽

Hongchun Zhu ◽

Pengda Wu ◽

Chengming Li

Keyword(s):

Structure From Motion ◽

Feature Matching ◽

Structural Characteristics ◽

Feature Point ◽

Feature Points ◽

Improved Method ◽

Point Selection ◽

Aerial Triangulation ◽

Oblique Images ◽

Stable Feature

Feature matching plays a crucial role in the process of 3D reconstruction based on the structure from motion (SfM) technique. For a large collection of oblique images, feature matching is one of the most time-consuming steps, and the matching result directly affects the accuracy of subsequent tasks. Therefore, how to extract the reasonable feature points robustly and efficiently to improve the matching speed and quality has received extensive attention from scholars worldwide. Most studies perform quantitative feature point selection based on image Difference-of-Gaussian (DoG) pyramids in practice. However, the stability and spatial distribution of feature points are not considered enough, resulting in selected feature points that may not adequately reflect the scene structures and cannot guarantee the matching rate and the aerial triangulation accuracy. To address these issues, an improved method for stable feature point selection in SfM considering image semantic and structural characteristics is proposed. First, the visible-band difference vegetation index is used to identify the vegetation areas from oblique images, and the line feature in the image is extracted by the optimized line segment detector algorithm. Second, the feature point two-tuple classification model is established, in which the vegetation area recognition result is used as the semantic constraint, the line feature extraction result is used as the structural constraint, and the feature points are divided into three types. Finally, a progressive selection algorithm for feature points is proposed, in which feature points in the DoG pyramid are selected by classes and levels until the number of feature points is satisfied. Oblique images of a 40-km2 area in Dongying city, China, were used for validation. The experimental results show that compared to the state-of-the-art method, the method proposed in this paper not only effectively reduces the number of feature points but also better reflects the scene structure. At the same time, the average reprojection error of the aerial triangulation decrease by 20%, the feature point matching rate increase by 3%, the selected feature points are more stable and reasonable.

Download Full-text

A NEW PARADIGM FOR MATCHING UAV- AND AERIAL IMAGES

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iii-3-83-2016 ◽

2016 ◽

Vol III-3 ◽

pp. 83-90 ◽

Cited By ~ 1

Author(s):

T. Koch ◽

X. Zhuo ◽

P. Reinartz ◽

F. Fraundorfer

Keyword(s):

Real World ◽

Image Matching ◽

Feature Matching ◽

Ground Truth ◽

Geometric Constraints ◽

Feature Point ◽

Aerial Images ◽

Aerial Image ◽

Ratio Test ◽

New Paradigm

Download Full-text

A Robust Invariant Local Feature Matching Method for Changing Scenes

Wireless Communications and Mobile Computing ◽

10.1155/2021/8927822 ◽

2021 ◽

Vol 2021 ◽

pp. 1-13

Author(s):

Di Wang ◽

Hongying Zhang ◽

Yanhua Shao

Keyword(s):

Neural Network ◽

Feature Detection ◽

Feature Matching ◽

Local Features ◽

Local Feature ◽

Feature Point ◽

Visual Localization ◽

Matching Method ◽

Joint Training ◽

Image Pairs

The precise evaluation of camera position and orientation is a momentous procedure of most machine vision tasks, especially visual localization. Aiming at the shortcomings of local features of dealing with changing scenes and the problem of realizing a robust end-to-end network that worked from feature detection to matching, an invariant local feature matching method for changing scene image pairs is proposed, which is a network that integrates feature detection, descriptor constitution, and feature matching. In the feature point detection and descriptor construction stage, joint training is carried out based on a neural network. In the feature point extraction and descriptor construction stage, joint training is carried out based on a neural network. To obtain local features with solid robustness to viewpoint and illumination changes, the Vector of Locally Aggregated Descriptors based on Neural Network (NetVLAD) module is introduced to compute the degree of correlation of description vectors from one image to another counterpart. Then, to enhance the relationship between relevant local features of image pairs, the attentional graph neural network (AGNN) is introduced, and the Sinkhorn algorithm is used to match them; finally, the local feature matching results between image pairs are output. The experimental results show that, compared with the existed algorithms, the proposed method enhances the robustness of local features of varying sights, performs better in terms of homography estimation, matching precision, and recall, and when meeting the requirements of the visual localization system to the environment, the end-to-end network tasks can be realized.

Download Full-text