Deep Vision Servo Hand-Eye Coordination Planning Study for Sorting Robots

Within the context of large-scale symmetry, a study on deep vision servo hand-eye coordination planning for sorting robots was conducted according to the problems of low recognition-sorting accuracy and efficiency in existing sorting robots. In order to maintain the symmetry of the picking robot, a small telescopic sorting robot with RealSense depth vision servo embedded in the manipulator was developed. The workspace and posture of picking parcels were analyzed, and the coordinate transformation model of hand-eye coordination was established for the “Eye-in-hand” mode. The hand-eye coordinated sorting test shows that the average positioning accuracy of the end in the X, Y and Z directions is 3.49 mm, 2.76 mm and 3.32 mm respectively, and the average time is 19.19 s. Among them, the average time for the mechanical arm to pick up the package from the initial position is 12.02 s, the average time for intermediate identification and calculation is 3.79 s, and the average time for placing the package is 6.9 s. The time consumed by robot arm’s action accounts for 79.8% of the whole cycle. The robot structure and the hand-eye coordination strategy with RealSense depth vision servo embedded in the robot can meet picking operation requirements, and the design of a picking robot proposed in this paper can greatly improve the coordination symmetry of fruit target recognition, detection, and picking.

Predictive Control-Based Completeness Analysis and Global Calibration of Robot Vision Features

This paper provides an in-depth study and analysis of robot vision features for predictive control and a global calibration of their feature completeness. The acquisition and use of the complete macrofeature set are studied in the context of a robot task by defining the complete macrofeature set at the level of the overall purpose and constraints of the robot vision servo task. The visual feature set that can fully characterize the macropurpose and constraints of a vision servo task is defined as the complete macrofeature set. Due to the complexity of the task, a part of the features of the complete macrofeature set is obtained directly from the image, and another part of the features is obtained from the image by inference. The task is guaranteed to be completely based on a robust calibration-free visual serving strategy based on interference observer that is proposed to complete the visual serving task with high performance. To address the problems of singular values, local minima, and insufficient robustness in the traditional scale-free vision servo algorithm, a new scale-free vision servo method is proposed to construct a dual closed-loop vision servo structure based on interference observer, which ensures the closed-loop stability of the system through the Q-filter-based interference observer, while estimating and eliminating the interference consisting of hand-eye mapping model uncertainty and controlled robot input interference. The equivalent interference consisting of hand-eye mapping model uncertainty, controlled robot input interference, and detection noise is estimated and eliminated to obtain an inner-loop structure that presents a nominal model externally, and then an outer-loop controller is designed according to the nominal model to achieve the best performance of the system dynamic performance and robustness to optimally perform the vision servo task.

A Vision Servo System for Automated Harvest of Sweet Pepper in Korean Greenhouse Environment

Recently, farmers of sweet pepper suffer from the increase of its unit production costs due to the rise of labor costs. The rise of unit production costs of sweet pepper, on the other hand, decreases its productivity and causes the lack of its farming expertise, thus resulting in the quality degradation of products. In this regard, it is necessary to introduce an automated robot harvest system into the farming of sweet pepper. In this study, the authors developed an image-based closed-loop control system (a vision servo system) and an automated sweet pepper harvesting robot system and then carried out experiments to verify its efficiency. The working area of the manipulator that detects products through an imaging sensor in the farming environment of sweet pepper, decides whether to harvest it or not, and then informs the location of the product to the control center, which is set up at the distance scope of 350~600 mm from the center of the system and 1000 mm vertically. In order to confirm the performance of the sweet pepper recognition in this study, 269 sweet pepper images were used to extract fruits. Of 269 sweet pepper images, 82.16% were recognized successfully. The harvesting experiment of the system developed in this study was carried out with 100 sweet peppers. The result of experiment with 100 sweet peppers presents the fact that its approach rate to peduncle is about 86.7%, and via four sessions of repetitive harvest experiment it achieves a maximal 70% harvest rate, and its average time of harvest is 51.1 s.