scholarly journals Experiments on learning-based industrial bin-picking with iterative visual recognition

2018 ◽  
Vol 45 (4) ◽  
pp. 446-457 ◽  
Author(s):  
Kensuke Harada ◽  
Weiwei Wan ◽  
Tokuo Tsuji ◽  
Kohei Kikuchi ◽  
Kazuyuki Nagata ◽  
...  
Keyword(s):  
Random Forest ◽  
Point Cloud ◽  
Visual Recognition ◽  
Experimental Results ◽  
Training Data ◽  
Robotic Assembly ◽  
Depth Sensor ◽  
Content Type ◽  
New Finding ◽  
Bin Picking

Purpose This paper aims to automate the picking task needed in robotic assembly. Parts supplied to an assembly process are usually randomly staked in a box. If randomized bin-picking is introduced to a production process, we do not need any part-feeding machines or human workers to once arrange the objects to be picked by a robot. The authors introduce a learning-based method for randomized bin-picking. Design/methodology/approach The authors combine the learning-based approach on randomized bin-picking (Harada et al., 2014b) with iterative visual recognition (Harada et al., 2016a) and show additional experimental results. For learning, we use random forest explicitly considering the contact between a finger and a neighboring object. The iterative visual recognition method iteratively captures point cloud to obtain more complete point cloud of piled object by using 3D depth sensor attached at the wrist. Findings Compared with the authors’ previous research (Harada et al., 2014b) (Harada et al., 2016a), their new finding is as follows: by using random forest, the number of training data becomes extremely small. By adding penalty to occluded area, the learning-based method predicts the success after point cloud with less occluded area. We analyze the calculation time of the iterative visual recognition. We furthermore make clear the cases where a finger contacts neighboring objects. Originality/value The originality exists in the part where the authors combined the learning-based approach with the iterative visual recognition and supplied additional experimental results. After obtaining the complete point cloud of the piled object, prediction becomes effective.

scholarly journals Planning and optimization of robotic pick-and-place operations in highly constrained industrial environments

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Bence Tipary ◽  
András Kovács ◽  
Ferenc Gábor Erdős
Keyword(s):  
Process Planning ◽  
Robotic Assembly ◽  
Efficient Operation ◽  
Grasp Planning ◽  
Content Type ◽  
Online Planning ◽  
Assembly Cell ◽  
Pick And Place ◽  
Bin Picking ◽  
Localization Sequence

Purpose The purpose of this paper is to give a comprehensive solution method for the manipulation of parts with complex geometries arriving in bulk into a robotic assembly cell. As bin-picking applications are still not reliable in intricate workcells, first, the problem is transformed to a semi-structured pick-and-place application, then by collecting and organizing the required process planning steps, a methodology is formed to achieve reliable factory applications even in crowded assembly cell environments. Design/methodology/approach The process planning steps are separated into offline precomputation and online planning. The offline phase focuses on preparing the operation and reducing the online computational burdens. During the online phase, the parts laying in a semi-structured arrangement are first recognized and localized based on their stable equilibrium using two-dimensional vision. Then, the picking sequence and corresponding collision-free robot trajectories are planned and optimized. Findings The proposed method was evaluated in a geometrically complex experimental workcell, where it ensured precise, collision-free operation. Moreover, the applied planning processes could significantly reduce the execution time compared to heuristic approaches. Research limitations/implications The methodology can be further generalized by considering multiple part types and grasping modes. Additionally, the automation of grasp planning and the enhancement of part localization, sequence planning and path smoothing with more advanced solutions are further research directions. Originality/value The paper proposes a novel methodology that combines geometrical computations, image processing and combinatorial optimization, adapted to the requirements of flexible pick-and-place applications. The methodology covers each required planning step to reach reliable and more efficient operation.

scholarly journals Towards Autonomous Robotic Assembly: Using Combined Visual and Tactile Sensing for Adaptive Task Execution

2021 ◽  
Vol 101 (3) ◽  
Author(s):  
Korbinian Nottensteiner ◽  
Arne Sachtler ◽  
Alin Albu-Schäffer
Keyword(s):  
Autonomous Robots ◽  
Vision System ◽  
Dynamic Environments ◽  
Bayesian Framework ◽  
Experimental Results ◽  
Robotic Assembly ◽  
Tactile Sensing ◽  
Task Execution ◽  
Industrial Assembly ◽  
Assembly Tasks

AbstractRobotic assembly tasks are typically implemented in static settings in which parts are kept at fixed locations by making use of part holders. Very few works deal with the problem of moving parts in industrial assembly applications. However, having autonomous robots that are able to execute assembly tasks in dynamic environments could lead to more flexible facilities with reduced implementation efforts for individual products. In this paper, we present a general approach towards autonomous robotic assembly that combines visual and intrinsic tactile sensing to continuously track parts within a single Bayesian framework. Based on this, it is possible to implement object-centric assembly skills that are guided by the estimated poses of the parts, including cases where occlusions block the vision system. In particular, we investigate the application of this approach for peg-in-hole assembly. A tilt-and-align strategy is implemented using a Cartesian impedance controller, and combined with an adaptive path executor. Experimental results with multiple part combinations are provided and analyzed in detail.

scholarly journals Multi-Dimensional Underwater Point Cloud Detection Based on Deep Learning

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 884
Author(s):  
Chia-Ming Tsai ◽  
Yi-Horng Lai ◽  
Yung-Da Sun ◽  
Yu-Jen Chung ◽  
Jau-Woei Perng
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Training Data ◽  
Network Architectures ◽  
Point Cloud Data ◽  
Data Types ◽  
Raw Data ◽  
Cloud Data

Numerous sensors can obtain images or point cloud data on land, however, the rapid attenuation of electromagnetic signals and the lack of light in water have been observed to restrict sensing functions. This study expands the utilization of two- and three-dimensional detection technologies in underwater applications to detect abandoned tires. A three-dimensional acoustic sensor, the BV5000, is used in this study to collect underwater point cloud data. Some pre-processing steps are proposed to remove noise and the seabed from raw data. Point clouds are then processed to obtain two data types: a 2D image and a 3D point cloud. Deep learning methods with different dimensions are used to train the models. In the two-dimensional method, the point cloud is transferred into a bird’s eye view image. The Faster R-CNN and YOLOv3 network architectures are used to detect tires. Meanwhile, in the three-dimensional method, the point cloud associated with a tire is cut out from the raw data and is used as training data. The PointNet and PointConv network architectures are then used for tire classification. The results show that both approaches provide good accuracy.

“Please tidy up before leaving”: nudging Airbnb guests toward altruistic behavior

2019 ◽  
Vol 13 (4) ◽  
pp. 524-530 ◽  
Author(s):  
Paul Ranson ◽  
Daniel Guttentag
Keyword(s):  
Social Presence ◽  
Design Methodology ◽  
Field Experiments ◽  
Experimental Results ◽  
Altruistic Behavior ◽  
Content Type ◽  
The Social ◽  
Relationship Model ◽  
Social Market ◽  
Practical Implications

Purpose This study aimed to investigate whether increasing the social presence within an Airbnb lodging environment could nudge guests toward altruistic cleaning behaviors. Design/methodology/approach The study was based around a theoretical framework combining the social-market versus money-market relationship model, nudge theory and social presence theory. A series of three field experiments were conducted, in which social presence was manipulated to test its impact on guest cleaning behaviors prior to departure. Findings The experimental results confirmed the underlying hypothesis that an Airbnb listing’s enhanced social presence can subtly induce guests to help clean their rental units prior to departure. Originality/value This study is the first to examine behavioral nudging in an Airbnb context. It is also one of the first field experiments involving Airbnb. The study findings offer clear theoretical and practical implications.

scholarly journals BLAINDER—A Blender AI Add-On for Generation of Semantically Labeled Depth-Sensing Data

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2144
Author(s):  
Stefan Reitmann ◽  
Lorenzo Neumann ◽  
Bernhard Jung
Keyword(s):  
Point Clouds ◽  
Training Data ◽  
Sensor Data ◽  
Generation Process ◽  
Data Generation ◽  
Depth Sensor ◽  
Depth Sensing ◽  
Depth Sensors ◽  
3D Point Clouds ◽  
Wide Range

Common Machine-Learning (ML) approaches for scene classification require a large amount of training data. However, for classification of depth sensor data, in contrast to image data, relatively few databases are publicly available and manual generation of semantically labeled 3D point clouds is an even more time-consuming task. To simplify the training data generation process for a wide range of domains, we have developed the BLAINDER add-on package for the open-source 3D modeling software Blender, which enables a largely automated generation of semantically annotated point-cloud data in virtual 3D environments. In this paper, we focus on classical depth-sensing techniques Light Detection and Ranging (LiDAR) and Sound Navigation and Ranging (Sonar). Within the BLAINDER add-on, different depth sensors can be loaded from presets, customized sensors can be implemented and different environmental conditions (e.g., influence of rain, dust) can be simulated. The semantically labeled data can be exported to various 2D and 3D formats and are thus optimized for different ML applications and visualizations. In addition, semantically labeled images can be exported using the rendering functionalities of Blender.

scholarly journals Possibility of Autonomous Estimation of Shiba Goat’s Estrus and Non-Estrus Behavior by Machine Learning Methods

Animals ◽  
2020 ◽  
Vol 10 (5) ◽  
pp. 771
Author(s):  
Toshiya Arakawa
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Random Forest ◽  
Markov Models ◽  
Tracking System ◽  
Video Tracking ◽  
Training Data ◽  
Support Vector ◽  
Learning Methods ◽  
Machine Learning Methods

Mammalian behavior is typically monitored by observation. However, direct observation requires a substantial amount of effort and time, if the number of mammals to be observed is sufficiently large or if the observation is conducted for a prolonged period. In this study, machine learning methods as hidden Markov models (HMMs), random forests, support vector machines (SVMs), and neural networks, were applied to detect and estimate whether a goat is in estrus based on the goat’s behavior; thus, the adequacy of the method was verified. Goat’s tracking data was obtained using a video tracking system and used to estimate whether they, which are in “estrus” or “non-estrus”, were in either states: “approaching the male”, or “standing near the male”. Totally, the PC of random forest seems to be the highest. However, The percentage concordance (PC) value besides the goats whose data were used for training data sets is relatively low. It is suggested that random forest tend to over-fit to training data. Besides random forest, the PC of HMMs and SVMs is high. However, considering the calculation time and HMM’s advantage in that it is a time series model, HMM is better method. The PC of neural network is totally low, however, if the more goat’s data were acquired, neural network would be an adequate method for estimation.

Human skill integrated motion planning of assembly manipulation for 6R industrial robot

2019 ◽  
Vol 46 (1) ◽  
pp. 171-180
Author(s):  
Yi Liu ◽  
Ming Cong ◽  
Hang Dong ◽  
Dong Liu
Keyword(s):  
Deep Learning ◽  
Motion Planning ◽  
Industrial Robot ◽  
Robotic Assembly ◽  
3D Vision ◽  
Content Type ◽  
Human Skill ◽  
Depth Images ◽  
Positioning Method ◽  
Assembly Technology

Purpose The purpose of this paper is to propose a new method based on three-dimensional (3D) vision technologies and human skill integrated deep learning to solve assembly positioning task such as peg-in-hole. Design/methodology/approach Hybrid camera configuration was used to provide the global and local views. Eye-in-hand mode guided the peg to be in contact with the hole plate using 3D vision in global view. When the peg was in contact with the workpiece surface, eye-to-hand mode provided the local view to accomplish peg-hole positioning based on trained CNN. Findings The results of assembly positioning experiments proved that the proposed method successfully distinguished the target hole from the other same size holes according to the CNN. The robot planned the motion according to the depth images and human skill guide line. The final positioning precision was good enough for the robot to carry out force controlled assembly. Practical implications The developed framework can have an important impact on robotic assembly positioning process, which combine with the existing force-guidance assembly technology as to build a whole set of autonomous assembly technology. Originality/value This paper proposed a new approach to the robotic assembly positioning based on 3D visual technologies and human skill integrated deep learning. Dual cameras swapping mode was used to provide visual feedback for the entire assembly motion planning process. The proposed workpiece positioning method provided an effective disturbance rejection, autonomous motion planning and increased overall performance with depth images feedback. The proposed peg-hole positioning method with human skill integrated provided the capability of target perceptual aliasing avoiding and successive motion decision for the robotic assembly manipulation.

scholarly journals Random Forest with Adaptive Local Template for Pedestrian Detection

2015 ◽  
Vol 2015 ◽  
pp. 1-11 ◽  
Author(s):  
Tao Xiang ◽  
Tao Li ◽  
Mao Ye ◽  
Zijian Liu
Keyword(s):  
Computer Vision ◽  
Random Forest ◽  
Classification Accuracy ◽  
Template Matching ◽  
Detection Method ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Sliding Window ◽  
Experimental Results ◽  
Training Samples

Pedestrian detection with large intraclass variations is still a challenging task in computer vision. In this paper, we propose a novel pedestrian detection method based on Random Forest. Firstly, we generate a few local templates with different sizes and different locations in positive exemplars. Then, the Random Forest is built whose splitting functions are optimized by maximizing class purity of matching the local templates to the training samples, respectively. To improve the classification accuracy, we adopt a boosting-like algorithm to update the weights of the training samples in a layer-wise fashion. During detection, the trained Random Forest will vote the category when a sliding window is input. Our contributions are the splitting functions based on local template matching with adaptive size and location and iteratively weight updating method. We evaluate the proposed method on 2 well-known challenging datasets: TUD pedestrians and INRIA pedestrians. The experimental results demonstrate that our method achieves state-of-the-art or competitive performance.

Dual mass MEMS gyroscope temperature drift compensation based on TFPF-MEA-BP algorithm

Sensor Review ◽  
2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Huiliang Cao ◽  
Rang Cui ◽  
Wei Liu ◽  
Tiancheng Ma ◽  
Zekai Zhang ◽  
...  
Keyword(s):  
Training Data ◽  
Experimental Result ◽  
Intrinsic Mode Functions ◽  
Temperature Drift ◽  
Connection Weight ◽  
Bp Network ◽  
Content Type ◽  
Mems Gyroscope ◽  
Time Frequency ◽  
Drift Compensation

Purpose To reduce the influence of temperature on MEMS gyroscope, this paper aims to propose a temperature drift compensation method based on variational modal decomposition (VMD), time-frequency peak filter (TFPF), mind evolutionary algorithm (MEA) and BP neural network. Design/methodology/approach First, VMD decomposes gyro’s temperature drift sequence to obtain multiple intrinsic mode functions (IMF) with different center frequencies and then Sample entropy calculates, according to the complexity of the signals, they are divided into three categories, namely, noise signals, mixed signals and temperature drift signals. Then, TFPF denoises the mixed-signal, the noise signal is directly removed and the denoised sub-sequence is reconstructed, which is used as training data to train the MEA optimized BP to obtain a temperature drift compensation model. Finally, the gyro’s temperature characteristic sequence is processed by the trained model. Findings The experimental result proved the superiority of this method, the bias stability value of the compensation signal is 1.279 × 10–3°/h and the angular velocity random walk value is 2.132 × 10–5°/h/vHz, which is improved compared to the 3.361°/h and 1.673 × 10–2°/h/vHz of the original output signal of the gyro. Originality/value This study proposes a multi-dimensional processing method, which treats different noises separately, effectively protects the low-frequency characteristics and provides a high-precision training set for drift modeling. TFPF can be optimized by SEVMD parallel processing in reducing noise and retaining static characteristics, MEA algorithm can search for better threshold and connection weight of BP network and improve the model’s compensation effect.

Comparison of flexural capacity of different hollow slab beams with/without pavement layer reinforced by steel plates

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Long Liu ◽  
Lifeng Wang ◽  
Ziwang Xiao
Keyword(s):  
Finite Element ◽  
Bearing Capacity ◽  
Steel Plate ◽  
Plate Thickness ◽  
Experimental Results ◽  
Nonlinear Material ◽  
Finite Element Models ◽  
Rc Beams ◽  
Flexural Capacity ◽  
Content Type

PurposeReinforcement of reinforced concrete (RC) beams in-service have always been an important research field, anchoring steel plate in the bottom of the beams is a kind of common reinforcement methods. In actual engineering, the contribution of pavement layer to the bearing capacity of RC beams is often ignored, which underestimates the bearing capacity and stiffness of RC beams to a certain extent. The purpose of this paper is to study the effect of pavement layer on the RC beams before and after reinforcement.Design/methodology/approachFirst, static load experiments are carried out on three in-service RC hollow slab beams, meanwhile, nonlinear finite element models are built to study the bearing capacity of them. The nonlinear material and shear slip effect of studs are considered in the models. Second, the finite element models are verified, and the numerical simulation results are in good agreement with the experimental results. Last, the finite element models are adopted to carry out the research on the influence of different steel plate thicknesses on the flexural bearing capacity and ductility.FindingsThe experimental results showed that pavement layers increase the flexural capacity of hollow slab beams by 16.7%, and contribute to increasing stiffness. Ductility ratio of SPRCB3 and PRCB2 was 30% and 24% lower than that of RCB1, respectively. The results showed that when the steel plate thickness was 1 mm–6 mm, the bearing capacity of the hollow slab beam increased gradually from 2158.0 kN.m to 2656.6 kN.m. As the steel plate thickness continuously increased to 8 mm, the ultimate bearing capacity increased to 2681.0 kN.m. The increased thickness did not cause difference to the bearing capacity, because of concrete crushing at the upper edge.Originality/valueIn this paper, based on the experimental study, the bearing capacity of hollow beam strengthened by steel plate with different thickness is extrapolated by finite element simulation, and its influence on ductility is discussed. This method not only guarantees the accuracy of the bearing capacity evaluation, but also does not require a large number of samples, and has certain economy. The research results provide a basis for the reinforcement design of similar bridges.

Sign in / Sign up

Export Citation Format

Share Document