scholarly journals An Improved Method for Stable Feature Points Selection in Structure-from-Motion Considering Image Semantic and Structural Characteristics

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2416
Author(s):  
Fei Wang ◽  
Zhendong Liu ◽  
Hongchun Zhu ◽  
Pengda Wu ◽  
Chengming Li
Keyword(s):  
Structure From Motion ◽  
Feature Matching ◽  
Structural Characteristics ◽  
Feature Point ◽  
Feature Points ◽  
Improved Method ◽  
Point Selection ◽  
Aerial Triangulation ◽  
Oblique Images ◽  
Stable Feature

Feature matching plays a crucial role in the process of 3D reconstruction based on the structure from motion (SfM) technique. For a large collection of oblique images, feature matching is one of the most time-consuming steps, and the matching result directly affects the accuracy of subsequent tasks. Therefore, how to extract the reasonable feature points robustly and efficiently to improve the matching speed and quality has received extensive attention from scholars worldwide. Most studies perform quantitative feature point selection based on image Difference-of-Gaussian (DoG) pyramids in practice. However, the stability and spatial distribution of feature points are not considered enough, resulting in selected feature points that may not adequately reflect the scene structures and cannot guarantee the matching rate and the aerial triangulation accuracy. To address these issues, an improved method for stable feature point selection in SfM considering image semantic and structural characteristics is proposed. First, the visible-band difference vegetation index is used to identify the vegetation areas from oblique images, and the line feature in the image is extracted by the optimized line segment detector algorithm. Second, the feature point two-tuple classification model is established, in which the vegetation area recognition result is used as the semantic constraint, the line feature extraction result is used as the structural constraint, and the feature points are divided into three types. Finally, a progressive selection algorithm for feature points is proposed, in which feature points in the DoG pyramid are selected by classes and levels until the number of feature points is satisfied. Oblique images of a 40-km2 area in Dongying city, China, were used for validation. The experimental results show that compared to the state-of-the-art method, the method proposed in this paper not only effectively reduces the number of feature points but also better reflects the scene structure. At the same time, the average reprojection error of the aerial triangulation decrease by 20%, the feature point matching rate increase by 3%, the selected feature points are more stable and reasonable.

scholarly journals A Grid Feature-Point Selection Method for Large-Scale Street View Image Retrieval Based on Deep Local Features

2020 ◽  
Vol 12 (23) ◽  
pp. 3978
Author(s):  
Tianyou Chu ◽  
Yumin Chen ◽  
Liheng Huang ◽  
Zhiqiang Xu ◽  
Huangyuan Tan
Keyword(s):  
Image Retrieval ◽  
Large Scale ◽  
Local Features ◽  
Selection Method ◽  
Image Features ◽  
Feature Point ◽  
Feature Points ◽  
Point Selection ◽  
Retrieval Task ◽  
Street View

Street view image retrieval aims to estimate the image locations by querying the nearest neighbor images with the same scene from a large-scale reference dataset. Query images usually have no location information and are represented by features to search for similar results. The deep local features (DELF) method shows great performance in the landmark retrieval task, but the method extracts many features so that the feature file is too large to load into memory when training the features index. The memory size is limited, and removing the part of features simply causes a great retrieval precision loss. Therefore, this paper proposes a grid feature-point selection method (GFS) to reduce the number of feature points in each image and minimize the precision loss. Convolutional Neural Networks (CNNs) are constructed to extract dense features, and an attention module is embedded into the network to score features. GFS divides the image into a grid and selects features with local region high scores. Product quantization and an inverted index are used to index the image features to improve retrieval efficiency. The retrieval performance of the method is tested on a large-scale Hong Kong street view dataset, and the results show that the GFS reduces feature points by 32.27–77.09% compared with the raw feature. In addition, GFS has a 5.27–23.59% higher precision than other methods.

Feature Matching Based on Triangle Guidance and Constraints

2018 ◽  
Vol 32 (08) ◽  
pp. 1855014 ◽  
Author(s):  
Hongmin Liu ◽  
Hongya Zhang ◽  
Zhiheng Wang ◽  
Yiming Zheng
Keyword(s):  
Feature Matching ◽  
High Accuracy ◽  
Geometric Constraints ◽  
Feature Point ◽  
Feature Points ◽  
Matching Method ◽  
Shared Feature ◽  
Triangular Structure ◽  
Circle Center ◽  
Image Transformations

For images with distortions or repetitive patterns, the existing matching methods usually work well just on one of the two kinds of images. In this paper, we present novel triangle guidance and constraints (TGC)-based feature matching method, which can achieve good results on both kinds of images. We first extract stable matched feature points and combine these points into triangles as the initial matched triangles, and triangles combined by feature points are as the candidates to be matched. Then, triangle guidance based on the connection relationship via the shared feature point between the matched triangles and the candidates is defined to find the potential matching triangles. Triangle constraints, specially the location of a vertex relative to the inscribed circle center of the triangle, the scale represented by the ratio of corresponding side lengths of two matching triangles and the included angles between the sides of two triangles with connection relationship, are subsequently used to verify the potential matches and obtain the correct ones. Comparative experiments show that the proposed TGC can increase the number of the matched points with high accuracy under various image transformations, especially more effective on images with distortions or repetitive patterns due to the fact that the triangular structure are not only stable to image transformations but also provides more geometric constraints.

scholarly journals Robust Feature Matching with Spatial Smoothness Constraints

2020 ◽  
Vol 12 (19) ◽  
pp. 3158
Author(s):  
Xu Huang ◽  
Xue Wan ◽  
Daifeng Peng
Keyword(s):  
Energy Function ◽  
Feature Matching ◽  
Feature Point ◽  
Limiting Factors ◽  
Feature Points ◽  
Point Sets ◽  
Matching Accuracy ◽  
Matching Problem ◽  
Underlying Assumption ◽  
Speeded Up Robust Features

Feature matching is to detect and match corresponding feature points in stereo pairs, which is one of the key techniques in accurate camera orientations. However, several factors limit the feature matching accuracy, e.g., image textures, viewing angles of stereo cameras, and resolutions of stereo pairs. To improve the feature matching accuracy against these limiting factors, this paper imposes spatial smoothness constraints over the whole feature point sets with the underlying assumption that feature points should have similar matching results with their surrounding high-confidence points and proposes a robust feature matching method with the spatial smoothness constraints (RMSS). The core algorithm constructs a graph structure from the feature point sets and then formulates the feature matching problem as the optimization of a global energy function with first-order, spatial smoothness constraints based on the graph. For computational purposes, the global optimization of the energy function is then broken into sub-optimizations of each feature point, and an approximate solution of the energy function is iteratively derived as the matching results of the whole feature point sets. Experiments on close-range datasets with some above limiting factors show that the proposed method was capable of greatly improving the matching robustness and matching accuracy of some feature descriptors (e.g., scale-invariant feature transform (SIFT) and Speeded Up Robust Features (SURF)). After the optimization of the proposed method, the inlier number of SIFT and SURF was increased by average 131.9% and 113.5%, the inlier percentages between the inlier number and the total matches number of SIFT and SURF were increased by average 259.0% and 307.2%, and the absolute matching accuracy of SIFT and SURF was improved by average 80.6% and 70.2%.

scholarly journals Feature points selection with flocks of features constraint for visual simultaneous localization and mapping

2016 ◽  
Vol 14 (1) ◽  
pp. 172988141666678
Author(s):  
Hong Liu ◽  
Zhi Wang ◽  
Pengjin Chen
Keyword(s):  
Mobile Robots ◽  
Simultaneous Localization And Mapping ◽  
Selection Method ◽  
Feature Point ◽  
Feature Points ◽  
Point Selection ◽  
Localization Accuracy ◽  
Surrounding Environment ◽  
Localization And Mapping ◽  
Crucial Problem

Simultaneous localization and mapping is a crucial problem for mobile robots, which estimates the surrounding environment (the map) and, at the same time, computes the robot location in it. Most researchers working on simultaneous localization and mapping focus on localization accuracy. In visual simultaneous localization and mapping , localization is to calculate the robot’s position relative to the landmarks, which corresponds to the feature points in images. Therefore, feature points are of importance to localization accuracy and should be selected carefully. This article proposes a feature point selection method to improve the localization accuracy. First, theoretical and numerical analyses are conducted to demonstrate the importance of distribution of feature points. Then, an algorithm using flocks of features is proposed to select feature points. Experimental results show that the proposed flocks of features selector implemented in visual simultaneous localization and mapping enhances the accuracy of both localization and mapping, verifying the necessity of feature point selection.

scholarly journals TOWARDS EFFICIENCY OF OBLIQUE IMAGES ORIENTATION

2016 ◽  
Vol XL-3/W4 ◽  
pp. 91-96
Author(s):  
W. Ostrowski ◽  
K. Bakuła
Keyword(s):  
Structure From Motion ◽  
Laser Scanning ◽  
Bundle Adjustment ◽  
Commercial Software ◽  
Aerial Triangulation ◽  
Airborne Laser ◽  
Surface Models ◽  
Check Points ◽  
Oblique Images ◽  
The Impact

Many papers on both theoretical aspects of bundle adjustment of oblique images and new operators for detecting tie points on oblique images have been written. However, only a few achievements presented in the literature were practically implemented in commercial software. In consequence often aerial triangulation is performed either for nadir images obtained simultaneously with oblique photos or bundle adjustment for separate images captured in different directions. The aim of this study was to investigate how the orientation of oblique images can be carried out effectively in commercial software based on the structure from motion technology. The main objective of the research was to evaluate the impact of the orientation strategy on both duration of the process and accuracy of photogrammetric 3D products. Two, very popular software: Pix4D and Agisoft Photoscan were tested and two approaches for image blocks were considered. The first approach based only on oblique images collected in four directions and the second approach included nadir images. In this study, blocks for three test areas were analysed. Oblique images were collected with medium-format cameras in maltan cross configuration with registration of GNSS and INS data. As a reference both check points and digital surface models from airborne laser scanning were used.

scholarly journals Study on Reconstruction and Feature Tracking of Silicone Heart 3D Surface

Sensors ◽  
2021 ◽  
Vol 21 (22) ◽  
pp. 7570
Author(s):  
Ziyan Zhang ◽  
Yan Liu ◽  
Jiawei Tian ◽  
Shan Liu ◽  
Bo Yang ◽  
...  
Keyword(s):  
Surface Reconstruction ◽  
Feature Tracking ◽  
Feature Matching ◽  
Negative Impact ◽  
Three Dimensional ◽  
Geometric Constraints ◽  
Feature Point ◽  
Feature Points ◽  
3D Surface Reconstruction ◽  
3D Surface

At present, feature-based 3D reconstruction and tracking technology is widely applied in the medical field. In minimally invasive surgery, the surgeon can achieve three-dimensional reconstruction through the images obtained by the endoscope in the human body, restore the three-dimensional scene of the area to be operated on, and track the motion of the soft tissue surface. This enables doctors to have a clearer understanding of the location depth of the surgical area, greatly reducing the negative impact of 2D image defects and ensuring smooth operation. In this study, firstly, the 3D coordinates of each feature point are calculated by using the parameters of the parallel binocular endoscope and the spatial geometric constraints. At the same time, the discrete feature points are divided into multiple triangles using the Delaunay triangulation method. Then, the 3D coordinates of feature points and the division results of each triangle are combined to complete the 3D surface reconstruction. Combined with the feature matching method based on convolutional neural network, feature tracking is realized by calculating the three-dimensional coordinate changes of the same feature point in different frames. Finally, experiments are carried out on the endoscope image to complete the 3D surface reconstruction and feature tracking.

scholarly journals TOWARDS EFFICIENCY OF OBLIQUE IMAGES ORIENTATION

2016 ◽  
Vol XL-3/W4 ◽  
pp. 91-96 ◽  
Author(s):  
W. Ostrowski ◽  
K. Bakuła
Keyword(s):  
Structure From Motion ◽  
Laser Scanning ◽  
Bundle Adjustment ◽  
Commercial Software ◽  
Aerial Triangulation ◽  
Airborne Laser ◽  
Surface Models ◽  
Check Points ◽  
Oblique Images ◽  
The Impact

Many papers on both theoretical aspects of bundle adjustment of oblique images and new operators for detecting tie points on oblique images have been written. However, only a few achievements presented in the literature were practically implemented in commercial software. In consequence often aerial triangulation is performed either for nadir images obtained simultaneously with oblique photos or bundle adjustment for separate images captured in different directions. The aim of this study was to investigate how the orientation of oblique images can be carried out effectively in commercial software based on the structure from motion technology. The main objective of the research was to evaluate the impact of the orientation strategy on both duration of the process and accuracy of photogrammetric 3D products. Two, very popular software: Pix4D and Agisoft Photoscan were tested and two approaches for image blocks were considered. The first approach based only on oblique images collected in four directions and the second approach included nadir images. In this study, blocks for three test areas were analysed. Oblique images were collected with medium-format cameras in maltan cross configuration with registration of GNSS and INS data. As a reference both check points and digital surface models from airborne laser scanning were used.

Image Feature Matching Based on SIFT Algorithm

2014 ◽  
Vol 644-650 ◽  
pp. 4157-4161
Author(s):  
Xin Zhang ◽  
Ya Sheng Zhang ◽  
Hong Yao
Keyword(s):  
Image Matching ◽  
Feature Matching ◽  
Quadratic Function ◽  
Scale Space ◽  
Feature Point ◽  
Image Feature ◽  
Three Dimension ◽  
Feature Points ◽  
Gradient Distribution ◽  
Sift Feature

In the process of image matching, it is involved such as image rotation, scale zooming, brightness change and other problems. In order to improve the precision of image matching, image matching algorithm based on SIFT feature point is proposed. First, the method of generating scale space is introduced. Then, the scale and position of feature points are determined through three dimension quadratic function and feature vectors are determined through gradient distribution characteristic of neighborhood pixels of feature points. Finally, feature matching is completed based on the Euclidean distance. The experiment result indicates that using SIFT feature point can achieve image matching effectively.

scholarly journals Pose Measurement for Unmanned Aerial Vehicle Based on Rigid Skeleton

2021 ◽  
Vol 11 (4) ◽  
pp. 1373
Author(s):  
Jingyu Zhang ◽  
Zhen Liu ◽  
Guangjun Zhang
Keyword(s):  
Measurement Errors ◽  
Structural Characteristics ◽  
Geometric Model ◽  
Formation Flight ◽  
Depth Information ◽  
Feature Points ◽  
Feature Point Extraction ◽  
Aerial Vehicle ◽  
Uav Navigation ◽  
Rigid Skeleton

Pose measurement is a necessary technology for UAV navigation. Accurate pose measurement is the most important guarantee for a UAV stable flight. UAV pose measurement methods mostly use image matching with aircraft models or 2D points corresponding with 3D points. These methods will lead to pose measurement errors due to inaccurate contour and key feature point extraction. In order to solve these problems, a pose measurement method based on the structural characteristics of aircraft rigid skeleton is proposed in this paper. The depth information is introduced to guide and label the 2D feature points to eliminate the feature mismatch and segment the region. The space points obtained from the marked feature points fit the space linear equation of the rigid skeleton, and the UAV attitude is calculated by combining with the geometric model. This method does not need cooperative identification of the aircraft model, and can stably measure the position and attitude of short-range UAV in various environments. The effectiveness and reliability of the proposed method are verified by experiments on a visual simulation platform. The method proposed can prevent aircraft collision and ensure the safety of UAV navigation in autonomous refueling or formation flight.

scholarly journals FSD-BRIEF: A Distorted BRIEF Descriptor for Fisheye Image Based on Spherical Perspective Model

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1839
Author(s):  
Yutong Zhang ◽  
Jianmei Song ◽  
Yan Ding ◽  
Yating Yuan ◽  
Hua-Liang Wei
Keyword(s):  
Feature Matching ◽  
Field Of View ◽  
Image Feature ◽  
Feature Points ◽  
Matching Process ◽  
Feature Descriptors ◽  
Coordinate Mapping ◽  
Matching Performance ◽  
Perspective Model ◽  
Attitude Matrix

Fisheye images with a far larger Field of View (FOV) have severe radial distortion, with the result that the associated image feature matching process cannot achieve the best performance if the traditional feature descriptors are used. To address this challenge, this paper reports a novel distorted Binary Robust Independent Elementary Feature (BRIEF) descriptor for fisheye images based on a spherical perspective model. Firstly, the 3D gray centroid of feature points is designed, and the position and direction of the feature points on the spherical image are described by a constructed feature point attitude matrix. Then, based on the attitude matrix of feature points, the coordinate mapping relationship between the BRIEF descriptor template and the fisheye image is established to realize the computation associated with the distorted BRIEF descriptor. Four experiments are provided to test and verify the invariance and matching performance of the proposed descriptor for a fisheye image. The experimental results show that the proposed descriptor works well for distortion invariance and can significantly improve the matching performance in fisheye images.

Sign in / Sign up

Export Citation Format

Share Document