Autonomous Intubation Robot System based on Visual Servoing and Hybrid Control

During COVID-19 and other pandemics, endotracheal intubation is an effective and common method to save patients as the virus causes lung fibrosis and thus patients are unable to breathe spontaneously. Medical staff need to insert a tube close to the patient’s mouth, thereby leading to a high risk of cross-infection. To protect medical staff, we propose an autonomous intubation robot system (AIRS). With the developed visual servoing and hybrid control method, the entire system can simulate doctors for satisfying repeatability and safety of intubation operations. This system includes a self-driving/teleoperation platform, two co-robot arms, a new multi-functional laryngoscope, force sensors, and several cameras. In the visual servoing part, we realize recognition and location of the patient’s face, medical devices, and main physiological structures to provide real-time navigation. In the hybrid control part, we establish an oral model, propose an offline planning method and PID controllers by combining force, vision, and motion, and apply Virtual Fixture to insert safely. AIRS's validation is with a phantom model under a 2-min operation. Our proposed robot is original and promising in the area of emergent medical robots. We will further validate AIRS in clinical applications and extend the developed techniques in other general treatments.

Autonomous Intubation Robot System based on Visual Servoing and Hybrid Control

During COVID-19 and other pandemics, endotracheal intubation is an effective and common method to save patients as the virus causes lung fibrosis and thus patients are unable to breathe spontaneously. Medical staff need to insert a tube close to the patient’s mouth, thereby leading to a high risk of cross-infection. To protect medical staff, we propose an autonomous intubation robot system (AIRS). With the developed visual servoing and hybrid control method, the entire system can simulate doctors for satisfying repeatability and safety of intubation operations. This system includes a self-driving/teleoperation platform, two co-robot arms, a new multi-functional laryngoscope, force sensors, and several cameras. In the visual servoing part, we realize recognition and location of the patient’s face, medical devices, and main physiological structures to provide real-time navigation. In the hybrid control part, we establish an oral model, propose an offline planning method and PID controllers by combining force, vision, and motion, and apply Virtual Fixture to insert safely. AIRS's validation is with a phantom model under a 2-min operation. Our proposed robot is original and promising in the area of emergent medical robots. We will further validate AIRS in clinical applications and extend the developed techniques in other general treatments.

Monocular-Vision-Based Retinal Membrane Peeling with a Handheld Robot

Retinal membrane peeling requires delicate manipulation. The presence of the surgeon's physiological tremor, the high variability and often low quality of the ophthalmic image, and excessive forces make the tasks more challenging. Preventing unintended movement caused by tremor and unintentional forces can reduce membrane injury. With the use of an actively stabilized handheld robot, we employ a monocular camera-based surface reconstruction method to estimate the retinal plane and we propose the use of a virtual fixture with application of hard and soft stops and motion scaling to improve control of the tool tip during delaminating in a laboratory simulation of retinal membrane peeling. A hard stop just below the membrane surface helps to limit downward force exerted on the surface. Motion scaling also improves the user's control of contact force when delaminating. We demonstrate a reduction of maximum force and maximum surface-penetration distance from the estimated retinal plane using the proposed technique.