Planning for grasping cluttered objects based on obstruction degree

Grasping objects in clutter is more difficult than grasping a separated single object. An important issue is that unsafe grasps may occur, in case, one object sits or leans on another, which could cause the collapse of objects. In addition, reachability of each object surrounded by other obstacles also has to be considered. So the order of multiple objects for grasping and the grasp configuration of each object must be planned simultaneously. This article combines grasp order and grasp configuration planning to perform fast and safe multiobject grasping in cluttered scenes. First, a comprehensive grasp configuration database is built to provide enough feasible grasp configurations for the objects. Then, we propose an obstruction degree to estimate the likelihood of reachability of each grasp configuration as well as each object. This measurement also implicitly infers object interactions. Finally, grasp order and grasp configurations are planned together to deal with the constraints caused by reachability and object interaction. Simulations and experiments in a series of cluttered scenes demonstrate that our method can grasp objects efficiently and can greatly reduce unsafe grasps.

From Affordances to Abstract Words: The Flexibility of Sensorimotor Grounding

The sensorimotor system plays a critical role in several cognitive processes. Here, we review recent studies documenting this interplay at different levels. First, we concentrate on studies that have shown how the sensorimotor system is flexibly involved in interactions with objects. We report evidence demonstrating how social context and situations influence affordance activation, and then focus on tactile and kinesthetic components in body–object interactions. Then, we turn to word use, and review studies that have shown that not only concrete words, but also abstract words are grounded in the sensorimotor system. We report evidence that abstract concepts activate the mouth effector more than concrete concepts, and discuss this effect in light of studies on adults, children, and infants. Finally, we pinpoint possible sensorimotor mechanisms at play in the acquisition and use of abstract concepts. Overall, we show that the involvement of the sensorimotor system is flexibly modulated by context, and that its role can be integrated and flanked by that of other systems such as the linguistic system. We suggest that to unravel the role of the sensorimotor system in cognition, future research should fully explore the complexity of this intricate, and sometimes slippery, relation.

The science and engineering behind sensitized brain-controlled bionic hands

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain machine interfaces (BMI), which harness neural signals to reanimate the limbs via electrical activation of the muscles, or to control extra-corporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements using bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. While the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and non-invasive strategies for sensory restoration.

Measuring fingerpad deformation during active object manipulation

During active object manipulation, the finger-object interactions give rise to complex fingertip skin deformations. These deformations are in turn encoded by the local tactile afferents and provide rich and behaviorally relevant information to the central nervous system. Most of the work studying the mechanical response of the finger to dynamic loading has been performed under a passive setup, thereby precisely controlling the kinematics or the dynamics of the loading. However, to identify aspects of the deformations that are relevant to online control during object manipulation, it is desirable to measure the skin response in an active setup. To that end, we developed a device that allows us to monitor finger forces, skin deformations, and kinematics during fine manipulation. We describe the device in detail and test it to precisely describe how the fingertip skin in contact with the object deforms during a simple vertical oscillation task. We show that the level of grip force directly influences the fingerpad skin strains and that the strain rates are substantial during active manipulation (norm up to 100%/s). The developed setup will enable us to causally relate sensory information, i.e. skin deformation, to online control, i.e. grip force adjustment, in future studies.

The dynamics of touch sensing studied in a mass-spring-damper model of the skin

It has been shown that the skin can provide highly resolvable, dynamic tactile information to the central nervous system. However, currently available skin models do not provide a matching level of dynamic complexity. Motivated by recent observations that everyday interactions create a diversity of widespread travelling waves of multiple overlaid frequencies in the skin, we here model the skin as a 3D-distributed mass-spring-damper model. Shear forces across each spring were reported back as separate sources of information, on which we performed information content analysis using principal component analysis. We found that a wide range of settings of spring constants, dampening coefficients and baseline tension resulted in highly resolvable dynamic information even for simple skin-object interactions. Optimization showed that there were some settings that were more beneficial for a higher temporal resolution, i.e. where multiple independent interactions could be more easily resolved temporally. Whereas even a single sensor reporting a skin shear force with infinite precision by itself can achieve infinite resolution, biological sensors are noisy. We therefore also analyzed the resolution of force direction in the dynamic skin model, when their simulated signal-to-noise ratio was varied. We conclude that biological skin due to its inherent dynamics can afford a low spatial resolution of sensors (subsampling) while still maintaining a very high resolution for detecting skin-object interaction dynamics, and that biological evolution moreover due to this construct likely has been free to play around with a variety of mechanical skin parameters and sensor densities without significantly compromising this resolution.