Field Robotics: Applications and Fundamentals

Field robotics is an area that is impelled by an application-driven approach by its nature. In this paper, I first review certain actual application areas of field robotics. Then, I discuss the current status of the application of field robotics in three common technologies: (1) mapping and path planning; (2) self-localization, recognition, and decision-making; and (3) dynamics and control. I then conclude by presenting future perspectives.

Special Issue on Field Robotics with Vision Systems

Field robotics has been undergoing rapid progress in recent years. It addresses a wide range of activities performed in outdoor environments, and its applications are being developed in areas where it was previously considered difficult to apply. This rapid progress is largely supported by AI-based improvements in computer vision systems with monocular cameras, stereo cameras, RGB-D cameras, LiDAR systems, and/or other sensors. Field robotics is impelled by an application-driven approach by its nature, and it contributes to the resolution of social problems and the creation of new innovations, including autonomous driving to reduce casualties, autonomous working machines/robots to resolve the problems of labor shortages or dangers, disaster-response robots to aid rescue parties, various kinds of aerial robots to do searches or make deliveries, underwater robots to perform search missions, etc. In this special issue on “Field Robotics with Vision Systems,” we highlight sixteen interesting papers, including one review paper, fourteen research papers, and one development report. They cover various application areas, ranging from underwater to space environments, and they propose interesting integration methods or element technologies to use in outdoor environments where vision systems and robot systems have great difficulty performing robustly. We thank all authors and reviewers, and we hope that this special issue contributes to future research and development in area of field robotics, which promises new innovations.

Development of AUV MONACA - Hover-Capable Platform for Detailed Observation Under Ice –

The melting of ice and changes in ocean currents in Antarctica must be investigated to understand global climate change. In this regard, the volume changes of sea ice and ice shelves, bathymetry, and ocean currents in the Antarctic Ocean must be measured in three dimensions. Therefore, the use of autonomous underwater vehicles (AUVs), which can directly observe under ice, is being considered. The authors developed an AUV named Mobility Oriented Nadir AntarctiC Adventurer (MONACA) to observe sea ice and the lower region of the ice shelf in the Antarctic Ocean. Herein, we describe MONACA and its basic autonomous navigation methods (altitude control, depth control, and waypoint tracking), as well as report the results of a sea experiment conducted in Shimoda Bay, Japan. During the 5-day sea trial, the MONACA successfully measured bathymetry by tracking 15 waypoints in sequence, switching the control criteria in the -axis direction between 3 m depth and 3 m altitude.

Autonomous Path Travel Control of Mobile Robot Using Internal and External Camera Images in GPS-Denied Environments

In this study, we developed a system for calculating the relative position and angle between a mobile robot and a marker using information such as the size of the marker of the internal camera of the mobile robot. Using this information, the mobile robot runs autonomously along the path given by the placement of the marker. In addition, we provide a control system that can follow a trajectory using information obtained by recognizing the mobile robot when reflected in an external camera using deep learning. The proposed method can easily achieve autonomous path travel control for mobile robots in environments where GPS cannot be received. The effectiveness of the proposed system is demonstrated under several actual experiments.

A Novel Method for Goal Recognition from 10 m Distance Using Deep Learning in CanSat

In this study, we propose a method for CanSat to recognize and guide a goal using deep learning image classification even 10 m away from the goal, and describe the results of demonstrative evaluation to confirm the effectiveness of the method. We applied deep learning image classification to goal recognition in CanSat for the first time at ARLISS 2019, and succeeded in guiding it almost all the way to the goal in all three races, winning the first place as overall winner. However, the conventional method has a drawback in that the goal recognition rate drops significantly when the CanSat is more than 6–7 m away from the goal, making it difficult to guide the CanSat to the goal when it moves away from the goal because of various factors. To enable goal recognition from a distance of 10 m from the goal, we investigated the number of horizontal regions of interest divisions and the method of vertical shifts during image recognition, and clarified the effective number of divisions and recognition rate using experiments. Although object detection is commonly used to detect the position of an object from an image by deep learning, we confirmed that the proposed method has a higher recognition rate at long distances and a shorter computation time than SSD MobileNet V1. In addition, we participated in the CanSat contest ACTS 2020 to evaluate the effectiveness of the proposed method and achieved the zero-distance goal in all three competitions, demonstrating its effectiveness by winning first place in the comeback category.

Development of an Automatic Tracking Camera System Integrating Image Processing and Machine Learning

This paper describes the development of a robust object tracking system that combines detection methods based on image processing and machine learning for automatic construction machine tracking cameras at unmanned construction sites. In recent years, unmanned construction technology has been developed to prevent secondary disasters from harming workers in hazardous areas. There are surveillance cameras on disaster sites that monitor the environment and movements of construction machines. By watching footage from the surveillance cameras, machine operators can control the construction machines from a safe remote site. However, to control surveillance cameras to follow the target machines, camera operators are also required to work next to machine operators. To improve efficiency, an automatic tracking camera system for construction machines is required. We propose a robust and scalable object tracking system and robust object detection algorithm, and present an accurate and robust tracking system for construction machines by integrating these two methods. Our proposed image-processing algorithm is able to continue tracking for a longer period than previous methods, and the proposed object detection method using machine learning detects machines robustly by focusing on their component parts of the target objects. Evaluations in real-world field scenarios demonstrate that our methods are more accurate and robust than existing off-the-shelf object tracking algorithms while maintaining practical real-time processing performance.

Attractive Force Estimation of a Magnetic Adsorption Unit for Inspection UAVs

This paper presents an attractive force estimation system and an automatic activation system for an electropermanent magnet (EPM) for an inspection UAV. Adsorption to infrastructures for inspection at a distance is extremely difficult to perform safely because the operator cannot detect the state of adsorption of the drone equipped with a magnetic adsorption device. Therefore, in this paper, we clarify the relationship between the magnetic flux density and attractive force of the EPM through experiments, and develop an estimation algorithm for the attractive force based on the results. An automatic activation system, using the induced voltage in the coil when the EPM approaches the magnetic substance, is developed and mounted on a quadrotor for a flight experiment along with the estimation system for the attractive force. The developed system is verified using flight and adsorption experiments on the quadrotor.

Image Mosaicking and Localization Using a Camera Mounted on a Hanging-Type Wall Climbing Robot

Owing to the revision of Japanese building law in 2008, the demand for wall inspections has been increasing. Currently, wall inspections are performed by workers using hammering devices; this involves dangerous work at high elevations. Therefore, we developed an inspection system using NOBORIN®, a hanging-type wall climbing robot. In this paper, we introduce the robot and its hammering inspection system, and propose a method for image mosaicking and localization using images captured from an equipped camera. The estimated values are used to correct the elevation motion(s) of the robot.

Numerical and Experimental Analysis of Portable Underwater Robots with a Movable Float Device

This report describes a numerical and experimental study of a posture control device based on a movable float for portable underwater robots. We numerically analyzed the static stability using a stability curve and allowable spatial range of a center-of-gravity shift caused by a payload shift or manipulator configuration. Further, we proposed a feedback controller based on direct pitch and roll signals to change and maintain robot posture. We tested the feedback control using a numerical simulator and conducted experiments in a water tank using two portable underwater robots to demonstrate the effectiveness of the movable float device and proposed controller. The results of the field experiments showed that the device and proposed controller can be employed for effective underwater operations of portable underwater robots.

Robotic Forklift for Stacking Multiple Pallets with RGB-D Cameras

We propose a robotic forklift system for stacking multiple mesh pallets. The stacking of mesh pallets is an essential task for the shipping and storage of loads. However, stacking, the placement of pallet feet on pallet edges, is a complex problem owing to the small sizes of the feet and edges, leading to a complexity in the detection and the need for high accuracy in adjusting the pallets. To detect the pallets accurately, we utilize multiple RGB-D (RGB Depth) cameras that produce dense depth data under the limitations of the sensor position. However, the depth data contain noise. Hence, we implement a region growing-based algorithm to extract the pallet feet and edges without removing them. In addition, we design the control law based on path following control for the forklift to adjust the position and orientation of two pallets. To evaluate the performance of the proposed system, we conducted an experiment assuming a real task. The experimental results demonstrated that the proposed system can achieve a stacking operation with a real forklift and mesh pallets.