Deep Vision Servo Hand-Eye Coordination Planning Study for Sorting Robots

Within the context of large-scale symmetry, a study on deep vision servo hand-eye coordination planning for sorting robots was conducted according to the problems of low recognition-sorting accuracy and efficiency in existing sorting robots. In order to maintain the symmetry of the picking robot, a small telescopic sorting robot with RealSense depth vision servo embedded in the manipulator was developed. The workspace and posture of picking parcels were analyzed, and the coordinate transformation model of hand-eye coordination was established for the “Eye-in-hand” mode. The hand-eye coordinated sorting test shows that the average positioning accuracy of the end in the X, Y and Z directions is 3.49 mm, 2.76 mm and 3.32 mm respectively, and the average time is 19.19 s. Among them, the average time for the mechanical arm to pick up the package from the initial position is 12.02 s, the average time for intermediate identification and calculation is 3.79 s, and the average time for placing the package is 6.9 s. The time consumed by robot arm’s action accounts for 79.8% of the whole cycle. The robot structure and the hand-eye coordination strategy with RealSense depth vision servo embedded in the robot can meet picking operation requirements, and the design of a picking robot proposed in this paper can greatly improve the coordination symmetry of fruit target recognition, detection, and picking.

Grasp Stability Prediction for a Dexterous Robotic Hand Combining Depth Vision and Haptic Bayesian Exploration

Grasp stability prediction of unknown objects is crucial to enable autonomous robotic manipulation in an unstructured environment. Even if prior information about the object is available, real-time local exploration might be necessary to mitigate object modelling inaccuracies. This paper presents an approach to predict safe grasps of unknown objects using depth vision and a dexterous robot hand equipped with tactile feedback. Our approach does not assume any prior knowledge about the objects. First, an object pose estimation is obtained from RGB-D sensing; then, the object is explored haptically to maximise a given grasp metric. We compare two probabilistic methods (i.e. standard and unscented Bayesian Optimisation) against random exploration (i.e. uniform grid search). Our experimental results demonstrate that these probabilistic methods can provide confident predictions after a limited number of exploratory observations, and that unscented Bayesian Optimisation can find safer grasps, taking into account the uncertainty in robot sensing and grasp execution.

Indoor Navigation Aid Systems for the Blind and Visually Impaired Based on Depth Sensors

This chapter focuses on the contributions made in the development of assistive technologies for the navigation of blind and visually impaired (BVI) individuals. A special interest is placed on vision-based systems that make use of image (RGB) and depth (D) information to assist their indoor navigation. Many commercial RGB-D cameras exist on the market, but for many years the Microsoft Kinect has been used as a tool for research in this field. Therefore, first-hand experience and advances on the use of Kinect for the development of an indoor navigation aid system for BVI individuals is presented. Limitations that can be encountered in building such a system are addressed at length. Finally, an overview of novel avenues of research in indoor navigation for BVI individuals such as integration of computer vision algorithms, deep learning for the classification of objects, and recent developments with stereo depth vision are discussed.

Choroba Stargardta funkcjonalnie – o sposobach radzenia sobie z niepełnosprawnością wzroku w stopniu znacznym

Visual disability affects about 2 million Poles. This group includes people who have been diagnosed with Stargardt’s disease. This disease occurs in the general population with a frequency of one in 10,000. It leads to legal blindness with visual acuity lower than 5%. Visual disturbances in this disease may occur in children, i.e. 7–12 years of age, as well as in adolescents and adults. People suffering from this disease experience a significant reduction in visual acuity, difficulty in recognizing colors, impaired depth vision, difficulties with accommodation, impaired central vision and often severe photophobia. The aim of the research was to collect information about the assistive technologies used by people with Stargardt’s disease and to identify their ability to move independently despite a significant reduction in visual acuity. The research results show that most of the 102 surveyed people use various types of assistive technology and experience significant visual impairment. At the same time, a significant part of this group declares that despite the inability to read the black print text, they can still move quite freely.