Shape binary patterns: an efficient local descriptor and keypoint detector for point clouds

Many of the research problems in robot vision involve the detection of keypoints, areas with salient information in the input images and the generation of local descriptors, that encode relevant information for such keypoints. Computer vision solutions have recently relied on Deep Learning techniques, which make extensive use of the computational capabilities available. In autonomous robots, these capabilities are usually limited and, consequently, images cannot be processed adequately. For this reason, some robot vision tasks still benefit from a more classic approach based on keypoint detectors and local descriptors. In 2D images, the use of binary representations for visual tasks has shown that, with lower computational requirements, they can obtain a performance comparable to classic real-value techniques. However, these achievements have not been fully translated to 3D images, where research is mainly focused on real-value approaches. Thus, in this paper, we propose a keypoint detector and local descriptor based on 3D binary patterns. The experimentation demonstrates that our proposal is competitive against state-of-the-art techniques, while its processing can be performed more efficiently.

A Simulated Environment for Robot Vision Experiments

Training on simulation data has proven invaluable in applying machine learning in robotics. However, when looking at robot vision in particular, simulated images cannot be directly used no matter how realistic the image rendering is, as many physical parameters (temperature, humidity, wear-and-tear in time) vary and affect texture and lighting in ways that cannot be encoded in the simulation. In this article we propose a different approach for extracting value from simulated environments: although neither of the trained models can be used nor are any evaluation scores expected to be the same on simulated and physical data, the conclusions drawn from simulated experiments might be valid. If this is the case, then simulated environments can be used in early-stage experimentation with different network architectures and features. This will expedite the early development phase before moving to (harder to conduct) physical experiments in order to evaluate the most promising approaches. In order to test this idea we created two simulated environments for the Unity engine, acquired simulated visual datasets, and used them to reproduce experiments originally carried out in a physical environment. The comparison of the conclusions drawn in the physical and the simulated experiments is promising regarding the validity of our approach.

Comparative Study of Computer Vision Based Line Followers Using Raspberry Pi and Jetson Nano

The line follower robot is a mobile robot which can navigate and traverse to another place by following a trajectory which is generally in the form of black or white lines. This robot can also assist human in carrying out transportation and industrial automation. However, this robot also has several challenges with regard to the calibration issue, incompatibility on wavy surfaces, and also the light sensor placement due to the line width variation. Robot vision utilizes image processing and computer vision technology for recognizing objects and controlling the robot motion. This study discusses the implementation of vision based line follower robot using a camera as the only sensor used to capture objects. A comparison of robot performance employing different CPU controllers, namely Raspberry Pi and Jetson Nano, is made. The image processing uses an edge detection method which detect the border to discriminate two image areas and mark different parts. This method aims to enable the robot to control its motion based on the object captured by the webcam. The results show that the accuracies of the robot employing the Raspberry Pi and Jetson Nano are 96% and 98%, respectively.

Image Processing Test Bench for Robot Vision Using C#

Purpose: Nowadays, image processing is a well-known technological term. In some of the industries, it has practical needs. It is an essential tool for the process and robotic industry. Various popular frameworks and libraries are available to process the image. The OpenCV is one of the best and popular libraries for image processing. It was originally written in C++ by Intel. Now various wrappers are available to implement into the different programming languages. The OpenCvSharp is the wrapper of OpenCV. Those who are familiar with C# can use it. The new researcher who wants to integrate image processing into their project takes some time for setup, function writing, and integration. Here we created a test bench application for Image processing demonstration. It has been made with some usual function to process the image. It was created using visual studio 2022 and OpenCvSharp wrapper in C# language. The researcher can learn about various image processing algorithms without writing any code or giving little bits of effort. The complete project is available on GitHub. Anyone can download, experiment, and integrate into their project without any issue. Design/Methodology/Approach: We created a GUI (Graphical User Interface) based C# application. Using Nuget Package manager, installed two OpenCV wrapper packages. To invoke several functions, we add some buttons, and for changing the method’s parameter, we integrate some text boxes. We created some abstraction layers Between the OpenCvSharp wrapper and GUI. We made our custom module as portable as possible so that our researchers could easily incorporate it into their project. Findings/result: This unique image processing test bench is designed for new researchers trying to integrate image processing capability into their research work. It can take still images or moving images through the connected webcam, automatically sending the various commands and promptly observing the result. Originality/Value: This test bench has been arranged uniquely for the researcher. It might have some value to their research work. The unique feature like automatic trigger can help them send the series of commands without repeatedly typing or pressing the button to see the result. Paper Type: Experiment-based Research

Predictive Control-Based Completeness Analysis and Global Calibration of Robot Vision Features

This paper provides an in-depth study and analysis of robot vision features for predictive control and a global calibration of their feature completeness. The acquisition and use of the complete macrofeature set are studied in the context of a robot task by defining the complete macrofeature set at the level of the overall purpose and constraints of the robot vision servo task. The visual feature set that can fully characterize the macropurpose and constraints of a vision servo task is defined as the complete macrofeature set. Due to the complexity of the task, a part of the features of the complete macrofeature set is obtained directly from the image, and another part of the features is obtained from the image by inference. The task is guaranteed to be completely based on a robust calibration-free visual serving strategy based on interference observer that is proposed to complete the visual serving task with high performance. To address the problems of singular values, local minima, and insufficient robustness in the traditional scale-free vision servo algorithm, a new scale-free vision servo method is proposed to construct a dual closed-loop vision servo structure based on interference observer, which ensures the closed-loop stability of the system through the Q-filter-based interference observer, while estimating and eliminating the interference consisting of hand-eye mapping model uncertainty and controlled robot input interference. The equivalent interference consisting of hand-eye mapping model uncertainty, controlled robot input interference, and detection noise is estimated and eliminated to obtain an inner-loop structure that presents a nominal model externally, and then an outer-loop controller is designed according to the nominal model to achieve the best performance of the system dynamic performance and robustness to optimally perform the vision servo task.

Video Captioning Based on Both Egocentric and Exocentric Views of Robot Vision for Human-Robot Interaction

Robot vision provides the most important information to robots so that they can read the context and interact with human partners successfully. Moreover, to allow humans recognize the robot’s visual understanding during human-robot interaction (HRI), the best way is for the robot to provide an explanation of its understanding in natural language. In this paper, we propose a new approach by which to interpret robot vision from an egocentric standpoint and generate descriptions to explain egocentric videos particularly for HRI. Because robot vision equals to egocentric video on the robot’s side, it contains as much egocentric view information as exocentric view information. Thus, we propose a new dataset, referred to as the global, action, and interaction (GAI) dataset, which consists of egocentric video clips and GAI descriptions in natural language to represent both egocentric and exocentric information. The encoder-decoder based deep learning model is trained based on the GAI dataset and its performance on description generation assessments is evaluated. We also conduct experiments in actual environments to verify whether the GAI dataset and the trained deep learning model can improve a robot vision system

Multipath Lightweight Deep Network Using Randomly Selected Dilated Convolution

Robot vision is an essential research field that enables machines to perform various tasks by classifying/detecting/segmenting objects as humans do. The classification accuracy of machine learning algorithms already exceeds that of a well-trained human, and the results are rather saturated. Hence, in recent years, many studies have been conducted in the direction of reducing the weight of the model and applying it to mobile devices. For this purpose, we propose a multipath lightweight deep network using randomly selected dilated convolutions. The proposed network consists of two sets of multipath networks (minimum 2, maximum 8), where the output feature maps of one path are concatenated with the input feature maps of the other path so that the features are reusable and abundant. We also replace the 3×3 standard convolution of each path with a randomly selected dilated convolution, which has the effect of increasing the receptive field. The proposed network lowers the number of floating point operations (FLOPs) and parameters by more than 50% and the classification error by 0.8% as compared to the state-of-the-art. We show that the proposed network is efficient.

Method for Estimating Time of Wireless Transfer of Energy Resources Between Two Robots

The energy capacity of the batteries used as the main power source in mobile robotic devices determines the autonomous operation of the robot. To plan the execution of tasks by a group of robotic tools in terms of time consumption, it is important to take into account the time during which the battery of each individual robot is charged. When using wireless power transfer, this time depends on the efficiency of the power transfer system, on the power of the transferring part of the system, as well as on the level of charge required to recharge. In this paper, we propose a method for estimating the time of transfer of energy resources between two robots, taking into account these parameters. The proposed method takes into account the application of the algorithm for the final positioning of robots, the assessment of linear offsets between robots, includes the calculation of efficiency, as well as the determination of the battery charge time, taking into account the parameters obtained at the previous stages of the method. The final positioning algorithm for robots uses algorithms for processing data from a robot vision system to search for fiducial markers and determine their spatial characteristics to ensure the final positioning of mobile robotic platforms. These characteristics are also used to determine the linear offsets between robots, on which the efficiency of energy transfer depends. To determine it, the method uses a mathematical model of the energy characteristics of the wireless power transfer system and the obtained linear offsets. At the last stage of the method, the time for charging the battery of the mobile robot is calculated, taking into account the data from the previous stages. Application of the proposed method to simulate the positioning of robots in a certain set of points in the working space will reduce the time spent on charging the robot battery when using wireless power transfer. As a result of the simulation, it was determined that the transfer of energy resources between robots took place with an efficiency in the range from 58.11% to 68.22%, and out of 14 positioning points, 3 were identified with the shortest energy transfer time.