scholarly journals Visual Servoing of Unknown Objects for Family Service Robots

2021 ◽  
Vol 104 (1) ◽  
Author(s):  
Jing Xin ◽  
Caixia Dong ◽  
Youmin Zhang ◽  
Yumeng Yao ◽  
Ailing Gong
Keyword(s):  
Visual Servoing ◽  
Image Features ◽  
Image Feature ◽  
Service Robots ◽  
Natural Scenes ◽  
Rigid Object ◽  
Current Image ◽  
Family Service ◽  
Tree Classifier ◽  
Unknown Objects

AbstractAiming at satisfying the increasing demand of family service robots for housework, this paper proposes a robot visual servoing scheme based on the randomized trees to complete the visual servoing task of unknown objects in natural scenes. Here, “unknown” means that there is no prior information on object models, such as template or database of the object. Firstly, an object to be manipulated is randomly selected by user prior to the visual servoing task execution. Then, the raw image information about the object can be obtained and used to train a randomized tree classifier online. Secondly, the current image features can be computed using the well-trained classifier. Finally, the visual controller can be designed according to the error of image feature, which is defined as the difference between the desired image features and current image features. Five visual positioning of unknown objects experiments, including 2D rigid object and 3D non-rigid object, are conducted on a MOTOMAN-SV3X six degree-of-freedom (DOF) manipulator robot. Experimental results show that the proposed scheme can effectively position an unknown object in complex natural scenes, such as occlusion and illumination changes. Furthermore, the developed robot visual servoing scheme has an excellent positioning accuracy within 0.05 mm positioning error.

scholarly journals Visual servoing of robot manipulator with weak field-of-view constraints

2021 ◽  
Vol 18 (1) ◽  
pp. 172988142199032
Author(s):  
Jing Xin ◽  
Han Cheng ◽  
Baojing Ran
Keyword(s):  
Visual Servoing ◽  
Robot Manipulator ◽  
Image Features ◽  
Tracking Task ◽  
Field Of View ◽  
Image Feature ◽  
Visual Object ◽  
Natural Scenes ◽  
Current Frame ◽  
Task Failure

Aiming at the problem of servoing task failure caused by the manipulated object deviating from the camera field-of-view (FOV) during the robot manipulator visual servoing (VS) process, a new VS method based on an improved tracking learning detection (TLD) algorithm is proposed in this article, which allows the manipulated object to deviate from the camera FOV in several continuous frames and maintains the smoothness of the robot manipulator motion during VS. Firstly, to implement the robot manipulator visual object tracking task with strong robustness under the weak FOV constraints, an improved TLD algorithm is proposed. Then, the algorithm is used to extract the image features (object in the camera FOV) or predict image features (object out of the camera FOV) of the manipulated object in the current frame. And then, the position of the manipulated object in the current image is further estimated. Finally, the visual sliding mode control law is designed according to the image feature errors to control the motion of the robot manipulator so as to complete the visual tracking task of the robot manipulator to the manipulated object in complex natural scenes with high robustness. Several robot manipulator VS experiments were conducted on a six-degrees-of-freedom MOTOMANSV3 industrial manipulator under different natural scenes. The experimental results show that the proposed robot manipulator VS method can relax the FOV constraint requirements on real-time visibility of manipulated object and effectively solve the problem of servoing task failure caused by the object deviating from the camera FOV during the VS.

scholarly journals Enhanced Switch Image-Based Visual Servoing Dealing with FeaturesLoss

Electronics ◽  
2019 ◽  
Vol 8 (8) ◽  
pp. 903 ◽  
Author(s):  
Ahmad Ghasemi ◽  
Pengcheng Li ◽  
Wen-Fang Xie ◽  
Wei Tian
Keyword(s):  
Visual Servoing ◽  
Experimental Tests ◽  
Reconstruction Algorithm ◽  
Response Speed ◽  
Image Features ◽  
Image Feature ◽  
System Response ◽  
Switch Control ◽  
Six Degree Of Freedom ◽  
Feature Reconstruction

In this paper, an enhanced switch image-based visual servoing controller for a six-degree-of-freedom (DOF) robot with a monocular eye-in-hand camera configuration is presented. The switch control algorithm separates the rotating and translational camera motions and divides the image-based visual servoing (IBVS) control into three distinct stages with different gains. In the proposed method, an image feature reconstruction algorithm based on the Kalman filter is proposed to handle the situation where the image features go outside the camera’s field of view (FOV). The combination of the switch controller and the feature reconstruction algorithm improves the system response speed and tracking performance of IBVS, while ensuring the success of servoing in the case of the feature loss. Extensive simulation and experimental tests are carried out on a 6-DOF robot to verify the effectiveness of the proposed method.

High-resolution electron microscopy of the atomic structure of some grain boundaries in Au

1986 ◽  
Vol 44 ◽  
pp. 414-415
Author(s):  
W. Krakow ◽  
D. A. Smith
Keyword(s):  
Atomic Structure ◽  
High Resolution Electron Microscopy ◽  
Image Features ◽  
Image Feature ◽  
Resolution Electron ◽  
Tilt Boundaries ◽  
Processing Techniques ◽  
Atom Position ◽  
Structure Configuration

The successful determination of the atomic structure of [110] tilt boundaries in Au stems from the investigation of microscope performance at intermediate accelerating voltages (200 and 400kV) as well as a detailed understanding of how grain boundary image features depend on dynamical diffraction processes variation with specimen and beam orientations. This success is also facilitated by improving image quality by digital image processing techniques to the point where a structure image is obtained and each atom position is represented by a resolved image feature. Figure 1 shows an example of a low angle (∼10°) Σ = 129/[110] tilt boundary in a ∼250Å Au film, taken under tilted beam brightfield imaging conditions, to illustrate the steps necessary to obtain the atomic structure configuration from the image. The original image of Fig. 1a shows the regular arrangement of strain-field images associated with the cores of ½ [10] primary dislocations which are separated by ∼15Å.

Structural displacement and strain monitoring based on the edge detection operator

2016 ◽  
Vol 20 (2) ◽  
pp. 191-201 ◽  
Author(s):  
Wei Lu ◽  
Yan Cui ◽  
Jun Teng
Keyword(s):  
Edge Detection ◽  
Image Features ◽  
Image Feature ◽  
Monitoring Method ◽  
Strain Monitoring ◽  
Morphological Operator ◽  
Filtering Effect ◽  
Circular Target ◽  
Plane Displacement ◽  
The Cost

To decrease the cost of instrumentation for the strain and displacement monitoring method that uses sensors as well as considers the structural health monitoring challenges in sensor installation, it is necessary to develop a machine vision-based monitoring method. For this method, the most important step is the accurate extraction of the image feature. In this article, the edge detection operator based on multi-scale structure elements and the compound mathematical morphological operator is proposed to provide improved image feature extraction. The proposed method can not only achieve an improved filtering effect and anti-noise ability but can also detect the edge more accurately. Furthermore, the required image features (vertex of a square calibration board and centroid of a circular target) can be accurately extracted using the extracted image edge information. For validation, the monitoring tests for the structural local mean strain and in-plane displacement were designed accordingly. Through analysis of the error between the measured and calculated values of the structural strain and displacement, the feasibility and effectiveness of the proposed edge detection operator are verified.

scholarly journals A Lightweight Fusion Distillation Network for Image Deblurring and Deraining

Sensors ◽  
2021 ◽  
Vol 21 (16) ◽  
pp. 5312
Author(s):  
Yanni Zhang ◽  
Yiming Liu ◽  
Qiang Li ◽  
Jianzhong Wang ◽  
Miao Qi ◽  
...  
Keyword(s):  
Deep Learning ◽  
Image Deblurring ◽  
Image Features ◽  
Image Feature ◽  
Model Complexity ◽  
Small Scale ◽  
Feature Maps ◽  
Learning Framework ◽  
Channel Information ◽  
Scale Spaces

Recently, deep learning-based image deblurring and deraining have been well developed. However, most of these methods fail to distill the useful features. What is more, exploiting the detailed image features in a deep learning framework always requires a mass of parameters, which inevitably makes the network suffer from a high computational burden. We propose a lightweight fusion distillation network (LFDN) for image deblurring and deraining to solve the above problems. The proposed LFDN is designed as an encoder–decoder architecture. In the encoding stage, the image feature is reduced to various small-scale spaces for multi-scale information extraction and fusion without much information loss. Then, a feature distillation normalization block is designed at the beginning of the decoding stage, which enables the network to distill and screen valuable channel information of feature maps continuously. Besides, an information fusion strategy between distillation modules and feature channels is also carried out by the attention mechanism. By fusing different information in the proposed approach, our network can achieve state-of-the-art image deblurring and deraining results with a smaller number of parameters and outperform the existing methods in model complexity.

scholarly journals 3D Affine: An Embedding of Local Image Features for Viewpoint Invariance Using RGB-D Sensor Data

Sensors ◽  
2019 ◽  
Vol 19 (2) ◽  
pp. 291 ◽  
Author(s):  
Hamdi Sahloul ◽  
Shouhei Shirafuji ◽  
Jun Ota
Keyword(s):  
Low Cost ◽  
Image Features ◽  
Image Feature ◽  
Sensor Data ◽  
Invariant Representation ◽  
Depth Sensors ◽  
Viewpoint Invariance ◽  
Surface Discontinuities ◽  
Local Image ◽  
Local Image Features

Local image features are invariant to in-plane rotations and robust to minor viewpoint changes. However, the current detectors and descriptors for local image features fail to accommodate out-of-plane rotations larger than 25°–30°. Invariance to such viewpoint changes is essential for numerous applications, including wide baseline matching, 6D pose estimation, and object reconstruction. In this study, we present a general embedding that wraps a detector/descriptor pair in order to increase viewpoint invariance by exploiting input depth maps. The proposed embedding locates smooth surfaces within the input RGB-D images and projects them into a viewpoint invariant representation, enabling the detection and description of more viewpoint invariant features. Our embedding can be utilized with different combinations of descriptor/detector pairs, according to the desired application. Using synthetic and real-world objects, we evaluated the viewpoint invariance of various detectors and descriptors, for both standalone and embedded approaches. While standalone local image features fail to accommodate average viewpoint changes beyond 33.3°, our proposed embedding boosted the viewpoint invariance to different levels, depending on the scene geometry. Objects with distinct surface discontinuities were on average invariant up to 52.8°, and the overall average for all evaluated datasets was 45.4°. Similarly, out of a total of 140 combinations involving 20 local image features and various objects with distinct surface discontinuities, only a single standalone local image feature exceeded the goal of 60° viewpoint difference in just two combinations, as compared with 19 different local image features succeeding in 73 combinations when wrapped in the proposed embedding. Furthermore, the proposed approach operates robustly in the presence of input depth noise, even that of low-cost commodity depth sensors, and well beyond.

scholarly journals Experimental Evaluation of PSO Based Transfer Learning Method for Meteorological Visibility Estimation

Atmosphere ◽  
2021 ◽  
Vol 12 (7) ◽  
pp. 828
Author(s):  
Wai Lun Lo ◽  
Henry Shu Hung Chung ◽  
Hong Fu
Keyword(s):  
Transfer Learning ◽  
Experimental Evaluation ◽  
Estimation Method ◽  
Image Features ◽  
Parameters Estimation ◽  
Image Feature ◽  
Support Vector ◽  
Learning Method ◽  
Feature Vectors ◽  
Visibility Estimation

Estimation of Meteorological visibility from image characteristics is a challenging problem in the research of meteorological parameters estimation. Meteorological visibility can be used to indicate the weather transparency and this indicator is important for transport safety. This paper summarizes the outcomes of the experimental evaluation of a Particle Swarm Optimization (PSO) based transfer learning method for meteorological visibility estimation method. This paper proposes a modified approach of the transfer learning method for visibility estimation by using PSO feature selection. Image data are collected at fixed location with fixed viewing angle. The database images were gone through a pre-processing step of gray-averaging so as to provide information of static landmark objects for automatic extraction of effective regions from images. Effective regions are then extracted from image database and the image features are then extracted from the Neural Network. Subset of Image features are selected based on the Particle Swarming Optimization (PSO) methods to obtain the image feature vectors for each effective sub-region. The image feature vectors are then used to estimate the visibilities of the images by using the Multiple Support Vector Regression (SVR) models. Experimental results show that the proposed method can give an accuracy more than 90% for visibility estimation and the proposed method is effective and robust.

scholarly journals A Multi-Model Stereo Similarity Function Based on Monogenic Signal Analysis in Poisson Scale Space

2011 ◽  
Vol 2011 ◽  
pp. 1-14 ◽  
Author(s):  
Jinjun Li ◽  
Hong Zhao ◽  
Chengying Shi ◽  
Xiang Zhou
Keyword(s):  
Color Image ◽  
Image Features ◽  
Image Feature ◽  
Similarity Function ◽  
Local Orientation ◽  
Stereo Images ◽  
Interest Points ◽  
Monogenic Signal ◽  
Local Color ◽  
Instantaneous Phase

A stereo similarity function based on local multi-model monogenic image feature descriptors (LMFD) is proposed to match interest points and estimate disparity map for stereo images. Local multi-model monogenic image features include local orientation and instantaneous phase of the gray monogenic signal, local color phase of the color monogenic signal, and local mean colors in the multiscale color monogenic signal framework. The gray monogenic signal, which is the extension of analytic signal to gray level image using Dirac operator and Laplace equation, consists of local amplitude, local orientation, and instantaneous phase of 2D image signal. The color monogenic signal is the extension of monogenic signal to color image based on Clifford algebras. The local color phase can be estimated by computing geometric product between the color monogenic signal and a unit reference vector in RGB color space. Experiment results on the synthetic and natural stereo images show the performance of the proposed approach.

Concepts of Augmented Image Space and Transformed Feature Space for Efficient Visual Servoing of an “Eye-in-Hand Robot”

Robotica ◽  
1991 ◽  
Vol 9 (2) ◽  
pp. 203-212 ◽  
Author(s):  
Won Jang ◽  
Kyungjin Kim ◽  
Myungjin Chung ◽  
Zeungnam Bien
Keyword(s):  
Visual Feedback ◽  
Visual Servoing ◽  
Industrial Robot ◽  
Robot Vision ◽  
Feature Space ◽  
Image Features ◽  
Image Space ◽  
Formal Definition ◽  
Computational Burden ◽  
Definition Of

SUMMARYFor efficient visual servoing of an “eye-in-hand” robot, the concepts of Augmented Image Space and Transformed Feature Space are presented in the paper. A formal definition of image features as functionals is given along with a technique to use defined image features for visual servoing. Compared with other known methods, the proposed concepts reduce the computational burden for visual feedback, and enhance the flexibility in describing the vision-based task. Simulations and real experiments demonstrate that the proposed concepts are useful and versatile tools for the industrial robot vision tasks, and thus the visual servoing problem can be dealt with more systematically.

Sign in / Sign up

Export Citation Format

Share Document