Mix Frame Visual Servo Control Framework for Autonomous Assistive Robotic Arms

Assistive robotic arms (ARAs) that provide care to the elderly and people with disabilities, are a significant part of Human-Robot Interaction (HRI). Presently available ARAs provide non-intuitive interfaces such as joysticks for control and thus, lacks the autonomy to perform daily activities. This study proposes that, for inducing autonomous behavior in ARAs, visual sensors integration is vital, and visual servoing in the direct Cartesian control mode is the preferred method. Generally, ARAs are designed in a configuration where its end-effector’s position is defined in the fixed base frame while orientation is expressed in the end-effector frame. We denoted this configuration as ‘mixed frame robotic arms’. Consequently, conventional visual servo controllers which operate in a single frame of reference are incompatible with mixed frame ARAs. Therefore, we propose a mixed-frame visual servo control framework for ARAs. Moreover, we enlightened the task space kinematics of a mixed frame ARAs, which led us to the development of a novel “mixed frame Jacobian matrix”. The proposed framework was validated on a mixed frame JACO-2 7 DoF ARA using an adaptive proportional derivative controller for achieving image-based visual servoing (IBVS), which showed a significant increase of 31% in the convergence rate, outperforming conventional IBVS joint controllers, especially in the outstretched arm positions and near the base frame. Our Results determine the need for the mixed frame controller for deploying visual servo control on modern ARAs, that can inherently cater to the robotic arm’s joint limits, singularities, and self-collision problems.

Model Reference Tracking Control Solutions for a Visual Servo System Based on a Virtual State from Unknown Dynamics

This paper focuses on validating a model-free Value Iteration Reinforcement Learning (MFVI-RL) control solution on a visual servo tracking system in a comprehensive manner starting from theoretical convergence analysis to detailed hardware and software implementation. Learning is based on a virtual state representation reconstructed from input-output (I/O) system samples under nonlinear observability and unknown dynamics assumptions, while the goal is to ensure linear output reference model (ORM) tracking. Secondary, a competitive model-free Virtual State-Feedback Reference Tuning (VSFRT) is learned from the same I/O data using the same virtual state representation, demonstrating the framework’s learning capability. A model-based two degrees-of-freedom (2DOF) output feedback controller serving as a comparisons baseline is designed and tuned using an identified system model. With similar complexity and linear controller structure, MFVI-RL is shown to be superior, confirming that the model-based design issue of poor identified system model and control performance degradation can be solved in a direct data-driven style. Apart from establishing a formal connection between output feedback control, state feedback control and also between classical control and artificial intelligence methods, the results also point out several practical trade-offs, such as I/O data exploration quality and control performance leverage with data volume, control goal and controller complexity.

Design of Robot Vision Servo Control System Based on Image

Visual servo is a closed-loop control system of robot, which takes the image information obtained by visual sensor as feedback. Generally speaking, visual servo plays an important role in robot control, which is one of the main research directions in the field of robot control and plays a decisive role in the development of intelligent robots. In order to make the robot competent for more complex tasks and work more intelligently, autonomously and reliably, it is necessary not only to improve the control system of the robot, but also to obtain more and better information about the working environment of the robot. This paper introduces the principle and basic realization method of robot visual servo based on image, and expounds the problems and solutions in image feature extraction and visual servo controller design. In order to further expand the application field of robots and improve the operation performance of robots, robots must have higher intelligence level and stronger adaptability to the environment, so as to manufacture intelligent robots that can replace human labor.

A Robot Dynamic Target Grasping Method Based on Affine Group Improved Gaussian Resampling Particle Filter

Tracking and grasping a moving target is currently a challenging topic in the field of robotics. The current visual servo grasping method is still inadequate, as the real-time performance and robustness of target tracking both need to be improved. A target tracking method is proposed based on improved geometric particle filtering (IGPF). Following the geometric particle filtering (GPF) tracking framework, affine groups are proposed as state particles. Resampling is improved by incorporating an improved conventional Gaussian resampling algorithm. It addresses the problem of particle diversity loss and improves tracking performance. Additionally, the OTB2015 dataset and typical evaluation indicators in target tracking are adopted. Comparative experiments are performed using PF, GPF and the proposed IGPF algorithm. A dynamic target tracking and grasping method for the robot is proposed. It combines an improved Gaussian resampling particle filter algorithm based on affine groups and the positional visual servo control of the robot. Finally, the robot conducts simulation and experiments on capturing dynamic targets in the simulation environment and actual environment. It verifies the effectiveness of the method proposed in this paper.

Development of a Visual Servo System for Robotic Fruit Harvesting

One of the challenges in the future of food production, amidst increasing population and decreasing resources, is developing a sustainable food production system. It is anticipated that robotics will play a significant role in maintaining the food production system, specifically in labor-intensive operations. Therefore, the main goal of this project is to develop a robotic fruit harvesting system, initially focused on the harvesting of apples. The robotic harvesting system is composed of a six-degrees-of-freedom (DOF) robotic manipulator, a two-fingered gripper, a color camera, a depth sensor, and a personal computer. This paper details the development and performance of a visual servo system that can be used for fruit harvesting. Initial test evaluations were conducted in an indoor laboratory using plastic fruit and artificial trees. Subsequently, the system was tested outdoors in a commercial fruit orchard. Evaluation parameters included fruit detection performance, response time of the visual servo, and physical time to harvest a fruit. Results of the evaluation showed that the developed visual servo system has the potential to guide the robot for fruit harvesting.

Research on a Visual Servo Method of a Manipulator Based on Velocity Feedforward

In this paper, a method of predicting the motion state of a moving target in the base coordinate system by hand-eye vision and the position and attitude of the end is proposed. The predicted value is used as the velocity feedforward, and the position-based visual servo method is used to plan the velocity of the end of the manipulator. It overcomes the influence of end coordinate system motion on target prediction in a discrete system and introduces an integral control method to compensate for the prediction velocity, eliminating the end tracking error caused by target velocity prediction error. The effectiveness of this method is verified by simulation and experiment.