Coupling active depth estimation and visual servoing via a large projection operator

SUMMARYIn this paper we propose a dynamic image-based visual servoing (IBVS) control for a rotary wing unmanned aerial vehicle (UAV) which directly accounts for the vehicle's underactuated dynamic model. The motion control objective is to follow parallel lines and is motivated by power line inspection tasks where the UAV's relative position and orientation to the lines are controlled. The design is based on a virtual camera whose motion follows the onboard physical camera but which is constrained to point downwards independent of the vehicle's roll and pitch angles. A set of image features is proposed for the lines projected into the virtual camera frame. These features are chosen to simplify the interaction matrix which in turn leads to a simpler IBVS control design which is globally asymptotically stable. The proposed scheme is adaptive and therefore does not require depth estimation. Simulation results are presented to illustrate the performance of the proposed control and its robustness to calibration parameter error.

Download Full-text

Catadioptric panoramic stereovision for humanoid robots

Robotica ◽

10.1017/s0263574711001068 ◽

2011 ◽

Vol 30 (5) ◽

pp. 799-811 ◽

Cited By ~ 5

Author(s):

C. Salinas ◽

H. Montes ◽

G. Fernandez ◽

P. Gonzalez de Santos ◽

M. Armada

Keyword(s):

Visual Servoing ◽

Humanoid Robot ◽

Moving Objects ◽

Control Strategies ◽

Depth Estimation ◽

Humanoid Robots ◽

Robust Detection ◽

Motion System ◽

Wide Range ◽

3D Information

SUMMARYThis paper proposes a novel design of a reconfigurable humanoid robot head, based on biological likeness of human being so that the humanoid robot could agreeably interact with people in various everyday tasks. The proposed humanoid head has a modular and adaptive structural design and is equipped with three main components: frame, neck motion system and omnidirectional stereovision system modules. The omnidirectional stereovision system module being the last module, a motivating contribution with regard to other computer vision systems implemented in former humanoids, it opens new research possibilities for achieving human-like behaviour. A proposal for a real-time catadioptric stereovision system is presented, including stereo geometry for rectifying the system configuration and depth estimation. The methodology for an initial approach for visual servoing tasks is divided into two phases, first related to the robust detection of moving objects, their depth estimation and position calculation, and second the development of attention-based control strategies. Perception capabilities provided allow the extraction of 3D information from a wide range of visions from uncontrolled dynamic environments, and work results are illustrated through a number of experiments.

Download Full-text

Image-based visual servoing with depth estimation

Transactions of the Institute of Measurement and Control ◽

10.1177/01423312211064681 ◽

2021 ◽

pp. 014233122110646

Author(s):

Qingxuan Gongye ◽

Peng Cheng ◽

Jiuxiang Dong

Keyword(s):

Real Time ◽

State Vector ◽

Visual Servoing ◽

Jacobian Matrix ◽

Fuzzy Controller ◽

Image Plane ◽

Depth Estimation ◽

The State ◽

Estimation Problem ◽

Depth Information

For the depth estimation problem in the image-based visual servoing (IBVS) control, this paper proposes a new observer structure based on Kalman filter (KF) to recover the feature depth in real time. First, according to the number of states, two different mathematical models of the system are established. The first one is to extract the depth information from the Jacobian matrix as the state vector of the system. The other is to use the depth information and the coordinate point information of the two-dimensional image plane as the state vector of the system. The KF is used to estimate the unknown depth information of the system in real time. And an IBVS controller gain adjustment method for 6-degree-of-freedom (6-DOF) manipulator is obtained using fuzzy controller. This method can obtain the gain matrix by taking the depth and error information as the input of the fuzzy controller. Compared with the existing works, the proposed observer has less redundant motion while solving the Jacobian matrix depth estimation problem. At the same time, it will also be beneficial to reducing the time for the camera to reach the target. Conclusively, the experimental results of the 6-DOF robot with eye-in-hand configuration demonstrate the effectiveness and practicability of the proposed method.

Download Full-text

Depth-estimation for visual servoing based on fuzzy logic

Proceedings of IEEE 5th International Fuzzy Systems ◽

10.1109/fuzzy.1996.552322 ◽

2002 ◽

Cited By ~ 1

Author(s):

Zeungnam Bien ◽

Jongcheol Park

Keyword(s):

Fuzzy Logic ◽

Visual Servoing ◽

Depth Estimation

Download Full-text

Adaptive control of nonlinear visual servoing systems for 3D cartesian tracking

Sba Controle & Automação Sociedade Brasileira de Automatica ◽

10.1590/s0103-17592006000400001 ◽

2006 ◽

Vol 17 (4) ◽

pp. 381-390 ◽

Cited By ~ 3

Author(s):

Alessandro R. L. Zachi ◽

Hsu Liu ◽

Fernando Lizarralde ◽

Antonio C. Leite

Keyword(s):

Adaptive Control ◽

Visual Servoing ◽

Feedback Loop ◽

Control Method ◽

Optical Axis ◽

Robot Manipulator ◽

Depth Estimation ◽

Target Object ◽

Image Area ◽

Real Robot

This paper presents a control strategy for robot manipulators to perform 3D cartesian tracking using visual servoing. Considering a fixed camera, the 3D cartesian motion is decomposed in a 2D motion on a plane orthogonal to the optical axis and a 1D motion parallel to this axis. An image-based visual servoing approach is used to deal with the nonlinear control problem generated by the depth variation without requiring direct depth estimation. Due to the lack of camera calibration, an adaptive control method is used to ensure both depth and planar tracking in the image frame. The depth feedback loop is closed by measuring the image area of a target object attached to the robot end-effector. Simulation and experimental results obtained with a real robot manipulator illustrate the viability of the proposed scheme.

Download Full-text