Planning for grasping cluttered objects based on obstruction degree

2021 ◽  
Vol 18 (6) ◽  
pp. 172988142110406
Author(s):  
Wenrui Zhao ◽  
Jingchuan Wang ◽  
Weidong Chen ◽  
Yi Huang
Keyword(s):  
Single Object ◽  
Multiple Objects ◽  
Object Interaction ◽  
Object Interactions

Grasping objects in clutter is more difficult than grasping a separated single object. An important issue is that unsafe grasps may occur, in case, one object sits or leans on another, which could cause the collapse of objects. In addition, reachability of each object surrounded by other obstacles also has to be considered. So the order of multiple objects for grasping and the grasp configuration of each object must be planned simultaneously. This article combines grasp order and grasp configuration planning to perform fast and safe multiobject grasping in cluttered scenes. First, a comprehensive grasp configuration database is built to provide enough feasible grasp configurations for the objects. Then, we propose an obstruction degree to estimate the likelihood of reachability of each grasp configuration as well as each object. This measurement also implicitly infers object interactions. Finally, grasp order and grasp configurations are planned together to deal with the constraints caused by reachability and object interaction. Simulations and experiments in a series of cluttered scenes demonstrate that our method can grasp objects efficiently and can greatly reduce unsafe grasps.

Tracking Multiple Objects Is Limited Only by Object Spacing, Not by Speed, Time, or Capacity

2010 ◽  
Vol 21 (7) ◽  
pp. 920-925 ◽  
Author(s):  
S.L. Franconeri ◽  
S.V. Jonathan ◽  
J.M. Scimeca
Keyword(s):  
Object Tracking ◽  
Moving Objects ◽  
Multiple Object Tracking ◽  
Single Object ◽  
Multiple Objects ◽  
Performance Constraints ◽  
Root Cause ◽  
Multiple Object ◽  
Tracking Time ◽  
Environment Performance

In dealing with a dynamic world, people have the ability to maintain selective attention on a subset of moving objects in the environment. Performance in such multiple-object tracking is limited by three primary factors—the number of objects that one can track, the speed at which one can track them, and how close together they can be. We argue that this last limit, of object spacing, is the root cause of all performance constraints in multiple-object tracking. In two experiments, we found that as long as the distribution of object spacing is held constant, tracking performance is unaffected by large changes in object speed and tracking time. These results suggest that barring object-spacing constraints, people could reliably track an unlimited number of objects as fast as they could track a single object.

scholarly journals Detecting human—object interaction with multi-level pairwise feature network

2020 ◽  
Author(s):  
Hanchao Liu ◽  
Tai-Jiang Mu ◽  
Xiaolei Huang
Keyword(s):  
Spatial Configuration ◽  
State Of The Art ◽  
Body Part ◽  
Human Action ◽  
Image Understanding ◽  
Semantic Level ◽  
Object Interaction ◽  
Human Object ◽  
Multi Level ◽  
Object Interactions

Abstract Human–object interaction (HOI) detection is crucial for human-centric image understanding which aims to infer ⟨human, action, object⟩ triplets within an image. Recent studies often exploit visual features and the spatial configuration of a human–object pair in order to learn the action linking the human and object in the pair. We argue that such a paradigm of pairwise feature extraction and action inference can be applied not only at the whole human and object instance level, but also at the part level at which a body part interacts with an object, and at the semantic level by considering the semantic label of an object along with human appearance and human–object spatial configuration, to infer the action. We thus propose a multi-level pairwise feature network (PFNet) for detecting human–object interactions. The network consists of three parallel streams to characterize HOI utilizing pairwise features at the above three levels; the three streams are finally fused to give the action prediction. Extensive experiments show that our proposed PFNet outperforms other state-of-the-art methods on the V-COCO dataset and achieves comparable results to the state-of-the-art on the HICO-DET dataset.

Solutions for the Binding Problem

1998 ◽  
Vol 53 (7-8) ◽  
pp. 691-715 ◽  
Author(s):  
Pieter R. Roelfsema
Keyword(s):  
Present Article ◽  
Cortical Neurons ◽  
Motor System ◽  
Binding Problem ◽  
Single Object ◽  
Multiple Objects ◽  
Perceptual Object ◽  
Neuronal Synchrony ◽  
Visual Cortical ◽  
Shed Light

Abstract Visual cortical neurons are broadly tuned to one or a few feature dimensions, like color and motion. This is advantageous because broadly tuned neurons can contribute to the repre­sentation of many visual scenes. However, if there are multiple objects in a visual scene, the cortex is at risk to combine features of different objects as if they belong to a single object. The term “binding problem” was introduced to refer to the difficulties that may occur in sorting out those responses that are evoked by a single perceptual object. The present article reviews proposals suggesting that the binding problem is solved by labelling an assembly of neurons that is responsive to a single perceptual obejct. Evidence is reviewed in favor of two possible assembly-labels: rate enhancement due to visual attention and neuronal synchrony. Assembly-labels should be spread through the cortical network to all neurons that have to participate in an assembly The present article tries to shed light on the mechanisms that subserve such a selective spread of assembly labels. Moreover, it is suggested that assembly labels may fulfill an equivalent role in the motor system, since binding problems can also occur during the generation of useful patterns of motor activity.

A System to Match Behaviors and Performance of Learners From User-Object Interactions

2019 ◽  
Vol 12 (2) ◽  
pp. 82-103
Author(s):  
José Guillermo Hernández-Calderón ◽  
Edgard Benítez-Guerrero ◽  
José Rafael Rojano-Cáceres ◽  
Carmen Mezura-Godoy
Keyword(s):  
Data Acquisition ◽  
Student Performance ◽  
Task Performance ◽  
Learning Process ◽  
Intelligent System ◽  
Intelligent Environments ◽  
Object Interaction ◽  
Teachers And Students ◽  
And Performance ◽  
Object Interactions

Intelligent environments in educational settings are aimed at supporting the learning process with an unobtrusive monitoring of the student while doing his/her activities. A desk is a common object in these settings, so if it is enhanced with sensing capabilities, it would enable gathering information of user-object interaction in a natural and unobtrusive way. An intelligent system is needed to analyze that information to reach conclusions faster than with traditional, manual observational process, and to provide timely valuable information. In this article, a design of a system to semi-automatically identify relationships between behaviors and the task performance of learners from user-object interaction logs is presented. The aim of detecting such relationships is to help teachers and students in the learning process, supporting their activities to identify special needs. Its components are designed to address four main functions: data acquisition, behavior identification, student performance identification, and computing relationships between student task performance and behaviors.

scholarly journals Deep learning wavefront sensing and aberration correction in atmospheric turbulence

PhotoniX ◽  
2021 ◽  
Vol 2 (1) ◽  
Author(s):  
Kaiqiang Wang ◽  
MengMeng Zhang ◽  
Ju Tang ◽  
Lingke Wang ◽  
Liusen Hu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Atmospheric Turbulence ◽  
Wavefront Sensor ◽  
Wavefront Aberration ◽  
Wavefront Sensing ◽  
Aberration Correction ◽  
Single Object ◽  
Multiple Objects ◽  
Correction Effect ◽  
And Performance

AbstractDeep learning neural networks are used for wavefront sensing and aberration correction in atmospheric turbulence without any wavefront sensor (i.e. reconstruction of the wavefront aberration phase from the distorted image of the object). We compared and found the characteristics of the direct and indirect reconstruction ways: (i) directly reconstructing the aberration phase; (ii) reconstructing the Zernike coefficients and then calculating the aberration phase. We verified the generalization ability and performance of the network for a single object and multiple objects. What’s more, we verified the correction effect for a turbulence pool and the feasibility for a real atmospheric turbulence environment.

Asymmetric Interface and Interactions for Bimanual Virtual Assembly With Haptics

2012 ◽  
Author(s):  
Vikram S. Vyawahare ◽  
Richard T. Stone
Keyword(s):  
Multimodal Interaction ◽  
Virtual Assembly ◽  
Haptic Device ◽  
The Other ◽  
Single Object ◽  
Interaction Method ◽  
Tracking Device ◽  
The Cost ◽  
Object Interactions ◽  
Assembly Tasks

This paper discusses development of a new bimanual interface configuration for virtual assembly consisting of a haptic device at one hand and a 6DOF tracking device at the other hand. The two devices form a multimodal interaction configuration facilitating unique interactions for virtual assembly. Tasks for virtual assembly can consist of both “one hand one object” and “bimanual single object” interactions. For one hand one object interactions this device configuration offers advantages in terms of increased manipulation workspace and provides a tradeoff between the cost effectiveness and mode of feedback. For bimanual single object manipulation an interaction method developed using this device configuration improves the realism and facilitates variation in precision of task of bimanual single object orientation. Furthermore another interaction method to expand the haptic device workspace using this configuration is introduced. The applicability of both these methods to the field of virtual assembly is discussed.

scholarly journals Topology Based Object Tracking

2019 ◽  
Vol 24 (3) ◽  
pp. 84
Author(s):  
Padraig Corcoran
Keyword(s):  
Homology Theory ◽  
Connected Components ◽  
Tracking Problem ◽  
Topological Properties ◽  
Single Object ◽  
Multiple Objects ◽  
Proposed Model ◽  
Radar Imagery ◽  
Tracking Objects ◽  
Over Time

A model for tracking objects whose topological properties change over time is proposed. Such changes include the splitting of an object into multiple objects or the merging of multiple objects into a single object. The proposed model employs a novel formulation of the tracking problem in terms of homology theory whereby 0-dimensional homology classes, which correspond to connected components, are tracked. A generalisation of this model for tracking spatially close objects lying in an ambient metric space is also proposed. This generalisation is particularly suitable for tracking spatial-temporal phenomena such as rain clouds. The utility of the proposed model is demonstrated with respect to tracking communities in a social network and tracking rain clouds in radar imagery.

scholarly journals Multiobject Tracking in Videos Based on LSTM and Deep Reinforcement Learning

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-12 ◽  
Author(s):  
Ming-xin Jiang ◽  
Chao Deng ◽  
Zhi-geng Pan ◽  
Lan-fang Wang ◽  
Xing Sun
Keyword(s):  
Reinforcement Learning ◽  
Object Tracking ◽  
Short Term Memory ◽  
Single Object ◽  
Multiple Objects ◽  
Short Term ◽  
Markov Decision ◽  
Formal Strategy ◽  
Long Short Term Memory ◽  
Multiobject Tracking

Multiple-object tracking is a challenging issue in the computer vision community. In this paper, we propose a multiobject tracking algorithm in videos based on long short-term memory (LSTM) and deep reinforcement learning. Firstly, the multiple objects are detected by the object detector YOLO V2. Secondly, the problem of single-object tracking is considered as a Markov decision process (MDP) since this setting provides a formal strategy to model an agent that makes sequence decisions. The single-object tracker is composed of a network that includes a CNN followed by an LSTM unit. Each tracker, regarded as an agent, is trained by utilizing deep reinforcement learning. Finally, we conduct a data association using LSTM for each frame between the results of the object detector and the results of single-object trackers. From the experimental results, we can see that our tracker achieves better performance than the other state-of-the-art methods. Multiple targets can be steadily tracked even when frequent occlusions, similar appearances, and scale changes happened.

Sign in / Sign up

Export Citation Format

Share Document