scholarly journals Failure Handling of Robotic Pick and Place Tasks With Multimodal Cues Under Partial Object Occlusion

2021 ◽  
Vol 15 ◽  
Author(s):  
Fan Zhu ◽  
Liangliang Wang ◽  
Yilin Wen ◽  
Lei Yang ◽  
Jia Pan ◽  
...  
Keyword(s):  
Visual Cues ◽  
Failure Detection ◽  
Robotic Manipulator ◽  
Open Loop ◽  
System Control ◽  
Grasp Planning ◽  
Loop Control ◽  
Failure Handling ◽  
Pick And Place ◽  
Place Task

The success of a robotic pick and place task depends on the success of the entire procedure: from the grasp planning phase, to the grasp establishment phase, then the lifting and moving phase, and finally the releasing and placing phase. Being able to detect and recover from grasping failures throughout the entire process is therefore a critical requirement for both the robotic manipulator and the gripper, especially when considering the almost inevitable object occlusion by the gripper itself during the robotic pick and place task. With the rapid rising of soft grippers, which rely heavily on their under-actuated body and compliant, open-loop control, less information is available from the gripper for effective overall system control. Tackling on the effectiveness of robotic grasping, this work proposes a hybrid policy by combining visual cues and proprioception of our gripper for the effective failure detection and recovery in grasping, especially using a proprioceptive self-developed soft robotic gripper that is capable of contact sensing. We solved failure handling of robotic pick and place tasks and proposed (1) more accurate pose estimation of a known object by considering the edge-based cost besides the image-based cost; (2) robust object tracking techniques that work even when the object is partially occluded in the system and achieve mean overlap precision up to 80%; (3) contact and contact loss detection between the object and the gripper by analyzing internal pressure signals of our gripper; (4) robust failure handling with the combination of visual cues under partial occlusion and proprioceptive cues from our soft gripper to effectively detect and recover from different accidental grasping failures. The proposed system was experimentally validated with the proprioceptive soft robotic gripper mounted on a collaborative robotic manipulator, and a consumer-grade RGB camera, showing that combining visual cues and proprioception from our soft actuator robotic gripper was effective in improving the detection and recovery from the major grasping failures in different stages for the compliant and robust grasping.

scholarly journals Pick-and-Place Task Implementation Using Visual Open-Loop Control

2019 ◽  
Vol 13 (3) ◽  
pp. 211-216
Author(s):  
Paweł Kołosowski ◽  
Adam Wolniakowski ◽  
Mariusz Bogdan
Keyword(s):  
Visual Processing ◽  
Template Matching ◽  
Visual Recognition ◽  
Open Loop ◽  
Work Piece ◽  
Loop Control ◽  
Pick And Place ◽  
Place Task ◽  
Sorting System ◽  
And Control

Abstract In the ever increasing number of robotic system applications in the industry, the robust and fast visual recognition and pose estimation of workpieces are of utmost importance. One of the ubiquitous tasks in industrial settings is the pick-and-place task where the object recognition is often important. In this paper, we present a new implementation of a work-piece sorting system using a template matching method for recognizing and estimating the position of planar workpieces with sparse visual features. The proposed framework is able to distinguish between the types of objects presented by the user and control a serial manipulator equipped with parallel finger gripper to grasp and sort them automatically. The system is furthermore enhanced with a feature that optimizes the visual processing time by automatically adjusting the template scales. We test the proposed system in a real-world setup equipped with a UR5 manipulator and provide experimental results documenting the performance of our approach.

Plant-Limited Co-Design of an Energy-Efficient Counterbalanced Robotic Manipulator

2012 ◽  
Author(s):  
James T. Allison
Keyword(s):  
System Design ◽  
Physical System ◽  
Control Design ◽  
Robotic Manipulator ◽  
Plant Design ◽  
Design Modification ◽  
Baseline Design ◽  
Design Changes ◽  
Pick And Place ◽  
Place Task

Modifying the design of an existing system to meet the needs of a new task is a common activity in mechatronic system development. Often engineers seek to meet requirements for the new task via control design changes alone, but in many cases new requirements are impossible to meet using control design only; physical system design modifications must be considered. Plant-Limited Co-Design (PLCD) is a design methodology for meeting new requirements at minimum cost through limited physical system (plant) design changes in concert with control system redesign. The most influential plant changes are identified to narrow the set of candidate plant changes. PLCD provides quantitative evidence to support strategic plant design modification decisions, including tradeoff analyses of redesign cost and requirement violation. In this article the design of a counterbalanced robotic manipulator is used to illustrate successful PLCD application. A baseline system design is obtained that exploits synergy between manipulator passive dynamics and control to minimize energy consumption for a specific pick-and-place task. The baseline design cannot meet requirements for a second pick-and-place task through control design changes alone. A limited set of plant design changes is identified using sensitivity analysis, and the PLCD result meets the new requirements at a cost significantly less than complete system redesign.

Plant-Limited Co-Design of an Energy-Efficient Counterbalanced Robotic Manipulator

2013 ◽  
Vol 135 (10) ◽  
Author(s):  
James T. Allison
Keyword(s):  
System Design ◽  
Physical System ◽  
Control Design ◽  
Robotic Manipulator ◽  
Plant Design ◽  
Design Modification ◽  
Baseline Design ◽  
Design Changes ◽  
Pick And Place ◽  
Place Task

Modifying the design of an existing system to meet the needs of a new task is a common activity in mechatronic system development. Often, engineers seek to meet requirements for the new task via control design changes alone, but in many cases new requirements are impossible to meet using control design only; physical system design modifications must be considered. Plant-limited co-design (PLCD) is a design methodology for meeting new requirements at minimum cost through limited physical system (plant) design changes in concert with control system redesign. The most influential plant changes are identified to narrow the set of candidate plant changes. PLCD provides quantitative evidence to support strategic plant design modification decisions, including tradeoff analyses of redesign cost and requirement violation. In this article the design of a counterbalanced robotic manipulator is used to illustrate successful PLCD application. A baseline system design is obtained that exploits synergy between manipulator passive dynamics and control to minimize energy consumption for a specific pick-and-place task. The baseline design cannot meet requirements for a second pick-and-place task through control design changes alone. A limited set of plant design changes is identified using sensitivity analysis, and the PLCD result meets the new requirements at a cost significantly less than complete system redesign.

Near-Minimum Time Visual Servoing of an Underactuated Robotic Manipulator

2013 ◽  
Vol 373-375 ◽  
pp. 217-220
Author(s):  
Yacine Benbelkacem ◽  
Rosmiwati Mohd-Mokhtar
Keyword(s):  
Rate Of Convergence ◽  
Visual Information ◽  
Visual Servoing ◽  
Open Loop ◽  
Minimum Time ◽  
Robotic Arm ◽  
Speed Of Convergence ◽  
Visually Guided ◽  
Pick And Place ◽  
Place Task

Rate of convergence to the desired pose to grasp an object using visual information may be important in some applications, such as a pick and place routine in assembly where the time between two stops of the conveyor is very short. The visually guided robot is required to move fast if vision is to bring the sought benefits to industrial setups. In this paper, the three most famous techniques to visual servoing, mainly the image-based, position-based and hybrid visual servoing are evaluated in terms of their speed of convergence to the grasping pose in a pick and place task of a momentarily motionless target. An alternative open-loop near-minimum time approach is also presented and tested on a 5DOF under-actuated robotic arm. The performance is compared and result shows significant reduction for its time of convergence, to the aforementioned techniques.

A Wearable Interface for Intuitive Control of Robotic Manipulators Without User Training

2014 ◽  
Author(s):  
Vinicius B. P. Fernandes ◽  
Jared A. Frank ◽  
Vikram Kapila
Keyword(s):  
Low Cost ◽  
Arm Movements ◽  
Robotic Manipulator ◽  
User Training ◽  
Game Controller ◽  
Pick And Place ◽  
Place Task ◽  
Kinematic Models ◽  
Wearable Interface ◽  
Alternative Interfaces

This paper describes the development of a wearable interface that exploits the user’s natural arm movements to intuitively control a robotic manipulator. The design is intended to alleviate the time and effort spent in operating the robotic manipulator, regardless of the age and technological experience of the user. The interface is made to be low-cost, comfortably worn, and easy to put on and remove. Kinematic models of human and robot arms are used to produce a natural mapping from the user’s arm movements to the commanded movements of the robotic manipulator. An experiment is conducted with 30 participants of varied ages and experience to assess the usability of the wearable interface. Each of the participants is assigned to perform a pick and place task using two of three different interfaces (the wearable interface, a game controller, and a mobile interface running on a tablet computer) for a total of 60 trials. The results of the study show that the wearable interface is easier to learn compared to the alternative interfaces and is chosen as the preferred interface by the participants. Performance data shows that the users complete the pick and place task faster with the wearable interface than with the alternative interfaces.

Open-loop control of compensation for optical propagation through a turbulent shear flow

1998 ◽  
Author(s):  
C. Truman ◽  
Lenore McMackin ◽  
Robert Pierson ◽  
Kenneth Bishop ◽  
Ellen Chen
Keyword(s):  
Shear Flow ◽  
Open Loop ◽  
Turbulent Shear ◽  
Turbulent Shear Flow ◽  
Open Loop Control ◽  
Loop Control ◽  
Optical Propagation

scholarly journals Sensuator: A Hybrid Sensor–Actuator Approach to Soft Robotic Proprioception Using Recurrent Neural Networks

Actuators ◽  
2021 ◽  
Vol 10 (2) ◽  
pp. 30
Author(s):  
Pornthep Preechayasomboon ◽  
Eric Rombokas
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Linear Models ◽  
Open Loop ◽  
Proof Of Concept ◽  
State Estimator ◽  
Loop Control ◽  
Practical Applications ◽  
Soft Actuator ◽  
The Cost

Soft robotic actuators are now being used in practical applications; however, they are often limited to open-loop control that relies on the inherent compliance of the actuator. Achieving human-like manipulation and grasping with soft robotic actuators requires at least some form of sensing, which often comes at the cost of complex fabrication and purposefully built sensor structures. In this paper, we utilize the actuating fluid itself as a sensing medium to achieve high-fidelity proprioception in a soft actuator. As our sensors are somewhat unstructured, their readings are difficult to interpret using linear models. We therefore present a proof of concept of a method for deriving the pose of the soft actuator using recurrent neural networks. We present the experimental setup and our learned state estimator to show that our method is viable for achieving proprioception and is also robust to common sensor failures.

Sign in / Sign up

Export Citation Format

Share Document