Image-based visual servoing with depth estimation

2021 ◽  
pp. 014233122110646
Author(s):  
Qingxuan Gongye ◽  
Peng Cheng ◽  
Jiuxiang Dong
Keyword(s):  
Real Time ◽  
State Vector ◽  
Visual Servoing ◽  
Jacobian Matrix ◽  
Fuzzy Controller ◽  
Image Plane ◽  
Depth Estimation ◽  
The State ◽  
Estimation Problem ◽  
Depth Information

For the depth estimation problem in the image-based visual servoing (IBVS) control, this paper proposes a new observer structure based on Kalman filter (KF) to recover the feature depth in real time. First, according to the number of states, two different mathematical models of the system are established. The first one is to extract the depth information from the Jacobian matrix as the state vector of the system. The other is to use the depth information and the coordinate point information of the two-dimensional image plane as the state vector of the system. The KF is used to estimate the unknown depth information of the system in real time. And an IBVS controller gain adjustment method for 6-degree-of-freedom (6-DOF) manipulator is obtained using fuzzy controller. This method can obtain the gain matrix by taking the depth and error information as the input of the fuzzy controller. Compared with the existing works, the proposed observer has less redundant motion while solving the Jacobian matrix depth estimation problem. At the same time, it will also be beneficial to reducing the time for the camera to reach the target. Conclusively, the experimental results of the 6-DOF robot with eye-in-hand configuration demonstrate the effectiveness and practicability of the proposed method.

scholarly journals Depth perception in single rgb camera system using lens aperture and object size: a geometrical approach for depth estimation

2021 ◽  
Vol 3 (6) ◽  
Author(s):  
P. J. A. Alphonse ◽  
K. V. Sriharsha
Keyword(s):  
Focal Length ◽  
Image Plane ◽  
Depth Estimation ◽  
Depth Information ◽  
Object Size ◽  
Geometrical Approach ◽  
Camera System ◽  
Object Distance ◽  
Lens Aperture ◽  
Human Movements

AbstractIn recent years, with increase in concern about public safety and security, human movements or action sequences are highly valued when dealing with suspicious and criminal activities. In order to estimate the position and orientation related to human movements, depth information is needed. This is obtained by fusing data obtained from multiple cameras at different viewpoints. In practice, whenever occlusion occurs in a surveillance environment, there may be a pixel-to-pixel correspondence between two images captured from two cameras and, as a result, depth information may not be accurate. Moreover use of more than one camera exclusively adds burden to the surveillance infrastructure. In this study, we present a mathematical model for acquiring object depth information using single camera by capturing the in focused portion of an object from a single image. When camera is in-focus, with the reference to camera lens center, for a fixed focal length for each aperture setting, the object distance is varied. For each aperture reading, for the corresponding distance, the object distance (or depth) is estimated by relating the three parameters namely lens aperture radius, object distance and object size in image plane. The results show that the distance computed from the relationship approximates actual with a standard error estimate of 2.39 to 2.54, when tested on Nikon and Cannon versions with an accuracy of 98.1% at 95% confidence level.

Vision-Based Fuzzy Controller for Quadrotor Tracking a Ground Target

2016 ◽  
Vol 101 ◽  
pp. 121-126 ◽  
Author(s):  
Xu Chao Chen ◽  
Zhi Qiang Cao ◽  
Yue Quan Yang ◽  
Chao Zhou
Keyword(s):  
Fuzzy Control ◽  
Target Tracking ◽  
Vertical Velocity ◽  
Visual Servoing ◽  
Fuzzy Controller ◽  
Image Plane ◽  
Tracking Problem ◽  
Accurate Model ◽  
Ground Target ◽  
Ground Target Tracking

A vision-based fuzzy controller for a quadrotor is proposed in this paper to realize ground target tracking. Due to the under-actuated property of quadrotors as well as the coupled dynamics in the image plane, it is challenging to design an image-based visual servoing controller for the quadrotor. Since the fuzzy control does not require an accurate model, a fuzzy-based approach is presented to solve the image-based tracking problem. Fuzzy controller takes image moments as inputs and its outputs are used to control the position of quadrotor in a form of tilt angles and vertical velocity adjustment. The proposed approach is verified by experiment.

scholarly journals DDTree: A Hybrid Deep Learning Model for Real-Time Waterway Depth Prediction and Smart Navigation

2020 ◽  
Vol 10 (8) ◽  
pp. 2770 ◽  
Author(s):  
Fan Yang ◽  
Yanan Qiao ◽  
Wei Wei ◽  
Xiao Wang ◽  
Difang Wan ◽  
...  
Keyword(s):  
Real Time ◽  
Situational Awareness ◽  
Depth Estimation ◽  
Automatic Identification ◽  
Depth Information ◽  
Large Variability ◽  
Depth Data ◽  
Depth Prediction ◽  
Maritime Navigation ◽  
Deep Learning Model

Timely and accurate depth estimation of a shallow waterway can improve shipping efficiency and reduce the danger of waterway transport accidents. However, waterway depth data measured during actual maritime navigation is limited, and the depth values can have large variability. Big data collected in real time by automatic identification systems (AIS) might provide a way to estimate accurate waterway depths, although these data include no direct channel depth information. We suggest a deep neural network (DNN) based model, called DDTree, for using the real-time AIS data and the data from Global Mapper to predict waterway depth for ships in an accurate and timely way. The model combines a decision tree and DNN, which is trained and tested on the AIS and Global Mapper data from the Nantong and Fangcheng ports on the southeastern and southwestern coast of China. The actual waterway depth data were used together with the AIS data as the input to DDTree. The latest data on waterway depths from the Chinese maritime agency were used to verify the results. The experiments show that the DDTree model has a prediction accuracy of 91.15%. Therefore, the DDTree model can provide an accurate prediction of waterway depth and compensate for the shortage of waterway depth monitoring means. The proposed hybrid DDTree model could improve marine situational awareness, navigation safety, and shipping efficiency, and contribute to smart navigation.

scholarly journals RGB-D Image Processing Algorithm for Target Recognition and Pose Estimation of Visual Servo System

Sensors ◽  
2020 ◽  
Vol 20 (2) ◽  
pp. 430 ◽  
Author(s):  
Shipeng Li ◽  
Di Li ◽  
Chunhua Zhang ◽  
Jiafu Wan ◽  
Mingyou Xie
Keyword(s):  
Image Processing ◽  
Real Time ◽  
Pose Estimation ◽  
Visual Servoing ◽  
Rapid Identification ◽  
Processing Algorithm ◽  
Depth Information ◽  
Target Image ◽  
Image Processing Algorithm ◽  
Depth Direction

This paper studies the control performance of visual servoing system under the planar camera and RGB-D cameras, the contribution of this paper is through rapid identification of target RGB-D images and precise measurement of depth direction to strengthen the performance indicators of visual servoing system such as real time and accuracy, etc. Firstly, color images acquired by the RGB-D camera are segmented based on optimized normalized cuts. Next, the gray scale is restored according to the histogram feature of the target image. Then, the obtained 2D graphics depth information and the enhanced gray image information are distort merged to complete the target pose estimation based on the Hausdorff distance, and the current image pose is matched with the target image pose. The end angle and the speed of the robot are calculated to complete a control cycle and the process is iterated until the servo task is completed. Finally, the performance index of this control system based on proposed algorithm is tested about accuracy, real-time under position-based visual servoing system. The results demonstrate and validate that the RGB-D image processing algorithm proposed in this paper has the performance in the above aspects of the visual servoing system.

scholarly journals Real-Time Identification of Dynamic Loads Using Inverse Solution and Kalman Filter

2020 ◽  
Vol 10 (19) ◽  
pp. 6767
Author(s):  
Jinhui Jiang ◽  
Shuyi Luo ◽  
M. Shadi Mohamed ◽  
Zhongzai Liang
Keyword(s):  
Kalman Filter ◽  
Real Time ◽  
Earthquake Engineering ◽  
State Vector ◽  
The State ◽  
Dynamic Loads ◽  
Unbiased Estimation ◽  
System Response ◽  
Structural Dynamic ◽  
Online Identification

Evaluating dynamic loads in real time is crucial for health monitoring, fault diagnosis and fatigue analysis in aerospace, automotive and earthquake engineering among other vibration related applications. Developing such algorithms can be vital for several safety and performance functionalities. Therefore, over the past few years the identification of dynamic loads has attracted a lot of attention; however, little literature on the online identification can be found. In this paper, we propose an online-identification method of structural dynamic loads so that the dynamic load is evaluated in real time and while the system response is still being measured. This is achieved by significantly improving the identification efficiency while retaining a high accuracy. The proposed method which is based on Kalman filter, is introduced in detail for a finite as well as an infinite number of degrees of freedom. Starting from an initial guess of the state vector we evaluate the error covariance, which then helps to identify the value of the excitation force using a weighted least square method and minimizing the covariance unbiased estimation. This is repeated at certain time intervals i.e., time steps where the state vector is updated in real time as acceleration measurements are updated. The feasibility of the method is validated using numerical simulations and an experimental verification where a detailed LabVIEW (National Instruments Ltd.) implementation is provided.

State estimation problems in PRF-shift magnetic resonance thermometry

2018 ◽  
Vol 28 (2) ◽  
pp. 315-335 ◽  
Author(s):  
César Pacheco ◽  
Helcio R.B. Orlande ◽  
Marcelo Colaco ◽  
George S. Dulikravich
Keyword(s):  
Kalman Filter ◽  
Steady State ◽  
Magnetic Resonance ◽  
State Estimation ◽  
Real Time ◽  
The State ◽  
Estimation Problem ◽  
Content Type ◽  
Magnetic Resonance Thermometry ◽  
Direct Inversion

Purpose The purpose of this paper is to apply the Steady State Kalman Filter for temperature measurements of tissues via magnetic resonance thermometry. Instead of using classical direct inversion, a methodology is proposed that couples the magnetic resonance thermometry with the bioheat transfer problem and the local temperatures can be identified through the solution of a state estimation problem. Design/methodology/approach Heat transfer in the tissues is given by Pennes’ bioheat transfer model, while the Proton Resonance Frequency (PRF)-Shift technique is used for the magnetic resonance thermometry. The problem of measuring the transient temperature field of tissues is recast as a state estimation problem and is solved through the Steady-State Kalman filter. Noisy synthetic measurements are used for testing the proposed methodology. Findings The proposed approach is more accurate for recovering the local transient temperatures from the noisy PRF-Shift measurements than the direct data inversion. The methodology used here can be applied in real time due to the reduced computational cost. Idealized test cases are examined that include the actual geometry of a forearm. Research limitations/implications The solution of the state estimation problem recovers the temperature variations in the region more accurately than the direct inversion. Besides that, the estimation of the temperature field in the region was possible with the solution of the state estimation problem via the Steady-State Kalman filter, but not with the direct inversion. Practical implications The recursive equations of the Steady-State Kalman filter can be calculated in computational times smaller than the supposed physical times, thus demonstrating that the present approach can be used for real-time applications, such as in control of the heating source in the hyperthermia treatment of cancer. Originality/value The original and novel contributions of the manuscript include: formulation of the PRF-Shift thermometry as a state estimation problem, which results in reduced uncertainties of the temperature variation as compared to the classical direct inversion; estimation of the actual temperature in the region with the solution of the state estimation problem, which is not possible with the direct inversion that is limited to the identification of the temperature variation; solution of the state estimation problem with the Steady-State Kalman filter, which allows for fast computations and real-time calculations.

scholarly journals Object Detection and Depth Estimation Approach Based on Deep Convolutional Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4755
Author(s):  
Huai-Mu Wang ◽  
Huei-Yung Lin ◽  
Chin-Chen Chang
Keyword(s):  
Neural Networks ◽  
Object Detection ◽  
Real Time ◽  
Convolutional Neural Networks ◽  
Prediction Accuracy ◽  
Depth Estimation ◽  
Disparity Estimation ◽  
Depth Information ◽  
Deep Convolutional Neural Networks ◽  
Epipolar Constraint

In this paper, we present a real-time object detection and depth estimation approach based on deep convolutional neural networks (CNNs). We improve object detection through the incorporation of transfer connection blocks (TCBs), in particular, to detect small objects in real time. For depth estimation, we introduce binocular vision to the monocular-based disparity estimation network, and the epipolar constraint is used to improve prediction accuracy. Finally, we integrate the two-dimensional (2D) location of the detected object with the depth information to achieve real-time detection and depth estimation. The results demonstrate that the proposed approach achieves better results compared to conventional methods.

Real-time Depth Estimation and View Interpolation using Quasar

2016 ◽  
Vol 2016 (19) ◽  
pp. 1-6 ◽  
Author(s):  
Bart Goossens ◽  
Simon Donné ◽  
Jan Aelterman ◽  
Jonas De Vylder ◽  
Dirk Van Haerenborgh ◽  
...  
Keyword(s):  
Real Time ◽  
Depth Estimation ◽  
View Interpolation

Operators and meaning of wave function

2016 ◽  
pp. 4039-4042
Author(s):  
Viliam Malcher
Keyword(s):  
Wave Function ◽  
Quantum Theory ◽  
State Vector ◽  
The State ◽  
Physical Significance ◽  
Central Problem ◽  
Quantum Mechanical ◽  
Mechanical Description ◽  
Quantum Mechanical Description ◽  
Interpretation Of Quantum Theory

The interpretation problems of quantum theory are considered. In the formalism of quantum theory the possible states of a system are described by a state vector. The state vector, which will be represented as |ψ> in Dirac notation, is the most general form of the quantum mechanical description. The central problem of the interpretation of quantum theory is to explain the physical significance of the |ψ>. In this paper we have shown that one of the best way to make of interpretation of wave function is to take the wave function as an operator.

Sign in / Sign up

Export Citation Format

Share Document