Real-time grasping strategies using event camera

Robotic vision plays a key role for perceiving the environment in grasping applications. However, the conventional framed-based robotic vision, suffering from motion blur and low sampling rate, may not meet the automation needs of evolving industrial requirements. This paper, for the first time, proposes an event-based robotic grasping framework for multiple known and unknown objects in a cluttered scene. With advantages of microsecond-level sampling rate and no motion blur of event camera, the model-based and model-free approaches are developed for known and unknown objects’ grasping respectively. The event-based multi-view approach is used to localize the objects in the scene in the model-based approach, and then point cloud processing is utilized to cluster and register the objects. The proposed model-free approach, on the other hand, utilizes the developed event-based object segmentation, visual servoing and grasp planning to localize, align to, and grasp the targeting object. Using a UR10 robot with an eye-in-hand neuromorphic camera and a Barrett hand gripper, the proposed approaches are experimentally validated with objects of different sizes. Furthermore, it demonstrates robustness and a significant advantage over grasping with a traditional frame-based camera in low-light conditions.

A FRAMEWORK FOR DESIGNING AN AUGMENTED REALITY APPLICATION FOCUSING ON OBJECT FUNCTION FOR CHILDREN WITH AUTISM

Augmented Reality (AR) is the evolution of the concept of Virtual Reality (VR). Its goal is to enhance a person's perception of the surrounding world. Augmented reality techniques are often applied to facilitate understanding and create attractive educational and health tools. As such, augmented reality is deemed suitable to be implemented as one of the potential intervention methods as the treatment of autism in a fun environment. Hence, this study is aimed to develop a conceptual framework to design augmented reality applications based on object function to help in the communication of children with autism. The study framework will be developed based on vision-based object recognition. Object recognition has been used in many applications, especially in bio-imaging, industrial inspection, and robotic vision. The findings of this study will benefit autistic children in visual communication and indirectly help them to effectively link objects with their functions. This framework will then help designers to develop augmented reality applications suited to be an intervention tool that fits the need of autistic children.