An In-Orbit Stereo Navigation Camera Self-Calibration Method for Planetary Rovers with Multiple Constraints

In order to complete the high-precision calibration of the planetary rover navigation camera using limited initial data in-orbit, we proposed a joint adjustment model with additional multiple constraints. Specifically, a base model was first established based on the bundle adjustment model, second-order radial and tangential distortion parameters. Then, combining the constraints of collinearity, coplanarity, known distance and relative pose invariance, a joint adjustment model was constructed to realize the in orbit self-calibration of the navigation camera. Given the problem of directionality in line extraction of the solar panel due to large differences in the gradient amplitude, an adaptive brightness-weighted line extraction method was proposed. Lastly, the Levenberg-Marquardt algorithm for nonlinear least squares was used to obtain the optimal results. To verify the proposed method, field experiments and in-orbit experiments were carried out. The results suggested that the proposed method was more accurate than the self-calibration bundle adjustment method, CAHVOR method (a camera model used in machine vision for three-dimensional measurements), and vanishing points method. The average error for the flag of China and the optical solar reflector was only 1 mm and 0.7 mm, respectively. In addition, the proposed method has been implemented in China’s deep space exploration missions.

Algorithm of 3D Virtual Reconstruction of Ancient Buildings in Qing Dynasty Based on Image Sequence

Aiming at the problem of low accuracy and poor integrity of traditional Qing Dynasty ancient architecture 3D virtual reconstruction algorithm, a 3D virtual reconstruction algorithm of Qing Dynasty ancient architecture based on image sequence is proposed. Acquire the sequence images of ancient buildings in the Qing Dynasty through the pinhole camera model, analyze the projective space and reconstruction space of the sequence images, redefine the similarity measurement coefficient according to the improved 2DPCA-SIFT feature matching algorithm, match the feature points of the ancient architecture images in the Qing Dynasty, and use random sampling to be consistent. The algorithm solves the basic matrix, removes the interference error in the image reconstruction process, and realizes the design of the three-dimensional reconstruction algorithm through image sequence fusion. The experimental results show that, compared with the existing methods, the completeness of the three-dimensional virtual reconstruction 3D model of ancient Qing Dynasty buildings constructed by the designed algorithm is 87.26% on average, and the completeness and accuracy of the 3D model construction of the subparts of the ancient Qing Dynasty buildings of this method are better. The height of the building fully shows that the designed building has good performance in the construction of the three-dimensional model of ancient buildings in the Qing Dynasty.

Application of 3D cameras to assess the quality of soil cultivation in agriculture

Currently, electronic devices are widely used in modern precision farming technology. Soil surface measurement is solved with single-purpose handheld instruments that are placed in the field before passing a tillage machine such as a plow or harrow and after tilling the soil. Laser, photogravimetric and synthetic aperture radar systems are used for non-contact measurement of soil roughness and uniformity. Measurements to calibrate the camera output were carried out at three height levels: 500 mm, 1000 mm and 1500 mm. The indicated measuring heights have been selected for possible applications in agricultural soil research, for example when determining the quality of the treated soil during plowing after passing the tillage machine. Measurements can be used to determine the condition of the soil before and after treatment. The system is easy to use and does not require special and additional expensive software. The use of 3D cameras is effective for determining the state of soil cultivation and a promising direction for adjusting the parameters of the tillage machine. Keywords: 3D CAMERA; MODEL; UNEVENNESS OF THE SOIL, TILLAGE MACHINE

Development and Experimental Validation of an Intelligent Camera Model for Automated Driving

The virtual testing and validation of advanced driver assistance system and automated driving (ADAS/AD) functions require efficient and realistic perception sensor models. In particular, the limitations and measurement errors of real perception sensors need to be simulated realistically in order to generate useful sensor data for the ADAS/AD function under test. In this paper, a novel sensor modeling approach for automotive perception sensors is introduced. The novel approach combines kernel density estimation with regression modeling and puts the main focus on the position measurement errors. The modeling approach is designed for any automotive perception sensor that provides position estimations at the object level. To demonstrate and evaluate the new approach, a common state-of-the-art automotive camera (Mobileye 630) was considered. Both sensor measurements (Mobileye position estimations) and ground-truth data (DGPS positions of all attending vehicles) were collected during a large measurement campaign on a Hungarian highway to support the development and experimental validation of the new approach. The quality of the model was tested and compared to reference measurements, leading to a pointwise position error of 9.60% in the lateral and 1.57% in the longitudinal direction. Additionally, the modeling of the natural scattering of the sensor model output was satisfying. In particular, the deviations of the position measurements were well modeled with this approach.

Reflectance estimation from snapshot multispectral images captured under unknown illumination

Multispectral images contain more spectral information of the scene objects compared to color images. The captured information of the scene reflectance is affected by several capture conditions, of which the scene illuminant is dominant. In this work, we implemented an imaging pipeline for a spectral filter array camera, where the focus is the estimation of the scene reflectances when the scene illuminant is unknown. We simulate three scenarios for reflectance estimation from multispectral images, and we evaluate the estimation accuracy on real captured data. We evaluate two camera model-based reflectance estimation methods that use a Wiener filter, and two other linear regression models for reflectance estimation that do not require an image formation model of the camera. Regarding the model-based approaches, we propose to use an estimate for the illuminant's spectral power distribution. The results show that our proposed approach stabilizes and marginally improves the estimation accuracy over the method that estimates the illuminant in the sensor space only. The results also provide a comparison of reflectance estimation using common approaches that are suited for different realistic scenarios.

A Multi-view Camera Model for Line-Scan Cameras with Telecentric Lenses

We propose a novel multi-view camera model for line-scan cameras with telecentric lenses. The camera model supports an arbitrary number of cameras and assumes a linear relative motion with constant velocity between the cameras and the object. We distinguish two motion configurations. In the first configuration, all cameras move with independent motion vectors. In the second configuration, the cameras are mounted rigidly with respect to each other and therefore share a common motion vector. The camera model can model arbitrary lens distortions by supporting arbitrary positions of the line sensor with respect to the optical axis. We propose an algorithm to calibrate a multi-view telecentric line-scan camera setup. To facilitate a 3D reconstruction, we prove that an image pair acquired with two telecentric line-scan cameras can always be rectified to the epipolar standard configuration, in contrast to line-scan cameras with entocentric lenses, for which this is possible only under very restricted conditions. The rectification allows an arbitrary stereo algorithm to be used to calculate disparity images. We propose an efficient algorithm to compute 3D coordinates from these disparities. Experiments on real images show the validity of the proposed multi-view telecentric line-scan camera model.

An Efficient Closed Form Solution to the Absolute Orientation Problem for Camera with Unknown Focal Length

In this paper we propose an efficient closed form solution to the absolute orientation problem for cameras with an unknown focal length, from two 2D–3D point correspondences and the camera position. The problem can be decomposed into two simple sub-problems and can be solved with angle constraints. A polynomial equation of one variable is solved to determine the focal length, and then a geometric approach is used to determine the absolute orientation. The geometric derivations are easy to understand and significantly improve performance. Rewriting the camera model with the known camera position leads to a simpler and more efficient closed form solution, and this gives a single solution, without the multi-solution phenomena of perspective-three-point (P3P) solvers. Experimental results demonstrated that our proposed method has a better performance in terms of numerical stability, noise sensitivity, and computational speed, with synthetic data and real images.