Precise and Effective Robotic Tool Change Strategy Using Visual Servoing With RGB-D Camera

In modern industrial manufacturing processes, robotic manipulators are routinely used in the assembly, packaging, and material handling operations. During production, changing end-of-arm tooling is frequently necessary for process flexibility and reuse of robotic resources. In conventional operation, a tool changer is sometimes employed to load and unload end-effectors, however, the robot must be manually taught to locate the tool changers by operators via a teach pendant. During tool change teaching, the operator takes considerable effort and time to align the master and tool side of the coupler by adjusting the motion speed of the robotic arm and observing the alignment from different viewpoints. In this paper, a custom robotic system, the NeXus, was programmed to locate and change tools automatically via an RGB-D camera. The NeXus was configured as a multi-robot system for multiple tasks including assembly, bonding, and 3D printing of sensor arrays, solar cells, and microrobot prototypes. Thus, different tools are employed by an industrial robotic arm to position grippers, printers, and other types of end-effectors in the workspace. To improve the precision and cycle-time of the robotic tool change, we mounted an eye-in-hand RGB-D camera and employed visual servoing to automate the tool change process. We then compared the teaching time of the tool location using this system and compared the cycle time with those of 6 human operators in the manual mode. We concluded that the tool location time in automated mode, on average, more than two times lower than the expert human operators.

First Full-Electric Shared-Actuation Control for Subsea Manifolds in Brazilian Ultra-Deep Waters: A Discussion of the Technological Development up to Field Commissioning

In projects to develop offshore production in Brazilian pre-salt fields, an innovative model of subsea manifolds is being used, based on shared-actuation control (SAC) for the remote operation of valves. The control solution, which comprises the first full-electric robotic tool designed to operate in ultra-deep waters, has achieved an important mark in 2020, with the commissioning and start-of-operation of the first fabricated unit. In this article, we present lessons learned and discuss relevant specifications and programs of the technological development that contributed for the results obtained so far. Considering aspects on conception, technology, and environment of application, the pre-salt SAC required the adoption of new solutions on several disciplines of subsea engineering. As a typical case of technological development, the design process comprised decisions on engineering requirements and the establishment of a comprehensive qualification program. Now, after the first robot completing critical stages at field, such as subsea deployment, functional testing, and integration with the subsea system, we obtain a set of performance results that serve us to evaluate e.g. how effective were the selected technical specifications and testing routines, used throughout the engineering program. This discussion also provides possible adjustments in the overall development plan, considering its application as new generations of SAC arise. The commissioning in 2020 of the first robot resulted in its full integration with the subsea manifold and the correspondent production system, contributing to water-alternating-gas injection in the pre-salt field Tupi Extremo Sul. A second subsea system featuring the same model of robotic tool, for manifold control, is in advanced schedule in 2021 for integration in Búzios II, another pre-salt field in Brazil. Confirming the advantages that we could expect with the adoption of SAC in subsea equipment, the pre-salt SAC allowed a series of optimizations on design of the robot-controlled manifold. The robot tool replaced all the hydraulic actuators that traditional control systems, based on electric-hydraulic multiplexing, would require to implement remote controlling of the manifold valves. This led to a significant reduction on sizes and weight of the manifold structure.

Autonomous pick-and-place using the dVRK

Purpose Robotic-assisted partial nephrectomy (RAPN) is a tissue-preserving approach to treating renal cancer, where ultrasound (US) imaging is used for intra-operative identification of tumour margins and localisation of blood vessels. With the da Vinci Surgical System (Sunnyvale, CA), the US probe is inserted through an auxiliary access port, grasped by the robotic tool and moved over the surface of the kidney. Images from US probe are displayed separately to the surgical site video within the surgical console leaving the surgeon to interpret and co-registers information which is challenging and complicates the procedural workflow. Methods We introduce a novel software architecture to support a hardware soft robotic rail designed to automate intra-operative US acquisition. As a preliminary step towards complete task automation, we automatically grasp the rail and position it on the tissue surface so that the surgeon is then able to manipulate manually the US probe along it. Results A preliminary clinical study, involving five surgeons, was carried out to evaluate the potential performance of the system. Results indicate that the proposed semi-autonomous approach reduced the time needed to complete a US scan compared to manual tele-operation. Conclusion Procedural automation can be an important workflow enhancement functionality in future robotic surgery systems. We have shown a preliminary study on semi-autonomous US imaging, and this could support more efficient data acquisition.

Robotic Fertilisation Using Localisation Systems Based on Point Clouds in Strip-Cropping Fields

The use of robotic systems in organic farming has taken on a leading role in recent years; the Sureveg CORE Organic Cofund ERA-Net project seeks to evaluate the benefits of strip-cropping to produce organic vegetables. This includes, among other objectives, the development of a robotic tool that facilitates the automation of the fertilisation process, allowing the individual treatment (at the plant level). In organic production, the slower nutrient release of the used fertilisers poses additional difficulties, as a tardy detection of deficiencies can no longer be corrected. To improve the detection, as well as counter the additional labour stemming from the strip-cropping configuration, an integrated robotic tool is proposed to detect individual crop deficiencies and react on a single-crop basis. For the development of this proof-of-concept, one of the main objectives of this work is implementing a robust localisation method within the vegetative environment based on point clouds, through the generation of general point cloud maps (G-PC) and local point cloud maps (L-PC) of a crop row. The plants’ geometric characteristics were extracted from the G-PC as a framework in which the robot’s positioning is defined. Through the processing of real-time lidar data, the L-PC is then defined and compared to the predefined reference system previously deduced. Both subsystems are integrated with ROS (Robot Operating System), alongside motion planning, and an inverse kinematics CCD (Cyclic Coordinate Descent) solver, among others. Tests were performed using a simulated environment of the crop row developed in Gazebo, followed by actual measurements in a strip-cropping field. During real-time data-acquisition, the localisation error is reduced from 13 mm to 11 mm within the first 120 cm of measurement. The encountered real-time geometric characteristics were found to coincide with those in the G-PC to an extend of 98.6%.