On the Efficiency of Haptic Based Object Identification: Determining Where to Grasp to Get the Most Distinguishing Information

Haptic perception is one of the key modalities in obtaining physical information of objects and in object identification. Most existing literature focused on improving the accuracy of identification algorithms with less attention paid to the efficiency. This work aims to investigate the efficiency of haptic object identification to reduce the number of grasps required to correctly identify an object out of a given object set. Thus, in a case where multiple grasps are required to characterise an object, the proposed algorithm seeks to determine where the next grasp should be on the object to obtain the most amount of distinguishing information. As such, the paper proposes the construction of the object description that preserves the association of the spatial information and the haptic information on the object. A clustering technique is employed both to construct the description of the object in a data set and for the identification process. An information gain (IG) based method is then employed to determine which pose would yield the most distinguishing information among the remaining possible candidates in the object set to improve the efficiency of the identification process. This proposed algorithm is validated experimentally. A Reflex TakkTile robotic hand with integrated joint displacement and tactile sensors is used to perform both the data collection for the dataset and the object identification procedure. The proposed IG approach was found to require a significantly lower number of grasps to identify the objects compared to a baseline approach where the decision was made by random choice of grasps.

Human Somatosensory Processing and Artificial Somatosensation

In the past few years, we have gained a better understanding of the information processing mechanism in the human brain, which has led to advances in artificial intelligence and humanoid robots. However, among the various sensory systems, studying the somatosensory system presents the greatest challenge. Here, we provide a comprehensive review of the human somatosensory system and its corresponding applications in artificial systems. Due to the uniqueness of the human hand in integrating receptor and actuator functions, we focused on the role of the somatosensory system in object recognition and action guidance. First, the low-threshold mechanoreceptors in the human skin and somatotopic organization principles along the ascending pathway, which are fundamental to artificial skin, were summarized. Second, we discuss high-level brain areas, which interacted with each other in the haptic object recognition. Based on this close-loop route, we used prosthetic upper limbs as an example to highlight the importance of somatosensory information. Finally, we present prospective research directions for human haptic perception, which could guide the development of artificial somatosensory systems.

Aging and haptic shape discrimination: the effects of variations in size

Seventy-two older and younger adults haptically discriminated the solid shape of natural objects (bell peppers, Capsicum annuum). Plastic copies of the original-sized fruits were used as experimental stimuli, as well as copies that were reduced in size to 1/8th and 1/27th of the original object volumes. If haptic object shape is represented in a part-based manner, then haptic shape discrimination performance should be at least partly size invariant, since changes only in scale do not affect an object’s constituent parts. On any given trial, participants sequentially explored two bell pepper replicas and were required to judge whether they possessed the same shape or had different shapes. For some participants, the objects to be discriminated possessed the same size, while for others, the two objects had different sizes. It was found that variations in scale did significantly reduce the participants’ haptic sensitivities to shape. Nevertheless, the discrimination performance obtained for large variations in size was no lower than that obtained for smaller variations in size. The results also demonstrated that increases in age modestly affect haptic shape discrimination performance: the d′ values of the older participants were 15.5% lower than those of the younger participants.

Different activation signatures in the primary sensorimotor and higher-level regions for haptic three-dimensional curved surface exploration

Haptic object perception begins with continuous exploratory contacts, and the human brain needs to accumulate sensory information continuously over time. However, it is still unclear how the primary sensorimotor cortex (PSC) interacts with these higher-level regions during haptic exploration across time. This functional magnetic resonance imaging (fMRI) study investigates time-dependent haptic object processing by examining brain activity during haptic 3D curve and roughness estimation. For this experiment, we designed sixteen haptic stimuli (4 kinds of curve × 4 kinds of roughness) for the haptic curve and roughness estimation tasks. Twenty participants were asked to move their right index and middle fingers along with the surface twice and to estimate one of the two features--roughness or curvature--dependent on the task instruction. We found that the brain activity in several higher-level regions (e.g., bilateral posterior parietal cortex) linearly increased with curvature through the haptic exploration phase. Surprisingly, we found that the contralateral PSC was parametrically modulated by the number of curves only during the late exploration phase, but not during the early exploration phase. In contrast, we found no similar parametric modulation activity patterns for haptic roughness estimation in either the contralateral PSC or in the higher-level regions. Together, our findings suggest that haptic 3D object perception is processed across the cortical hierarchy, while the contralateral PSC interacts with other higher-level regions across time in a manner that is dependent upon object features.HighlightsWe observed the brain activity of haptic object perception using parametric stimuli.Haptic curve estimation showed parametric modulation across the cortical hierarchy.Curve parametric modulation in the sensorimotor cortex showed time dependency.Roughness parametric modulation showed very little dependency in any regions of the brain.These findings reflect the nature of time-dependent haptic object processing in the brain.

Neural representations of haptic object size in the human brain revealed by multivoxel fMRI patterns

Our understanding of the neural basis of haptics (perceiving the world through touch) remains incomplete. We used functional MRI to study human haptic judgments of object size, which require integrating multiple afferent signals. Multivoxel pattern analyses identified intraparietal and prefrontal regions that encode size haptically in a metric and hand-invariant fashion. Effector-independent haptic size estimates are useful on their own and in combination with other sensory estimates for a variety of perceptual and motor tasks.

Task-Irrelevant Sound Corrects Leftward Spatial Bias in Blindfolded Haptic Placement Task

We often rely on our sense of vision for understanding the spatial location of objects around us. If vision cannot be used, one must rely on other senses, such as hearing and touch, in order to build spatial representations. Previous work has found evidence of a leftward spatial bias in visual and tactile tasks. In this study, we sought evidence of this leftward bias in a non-visual haptic object location memory task and assessed the influence of a task-irrelevant sound. In Experiment 1, blindfolded right-handed sighted participants used their non-dominant hand to haptically locate an object on the table, then used their dominant hand to place the object back in its original location. During placement, participants either heard nothing (no-sound condition) or a task-irrelevant repeating tone to the left, right, or front of the room. The results showed that participants exhibited a leftward placement bias on no-sound trials. On sound trials, this leftward bias was corrected; placements were faster and more accurate (regardless of the direction of the sound). One explanation for the leftward bias could be that participants were overcompensating their reach with the right hand during placement. Experiment 2 tested this explanation by switching the hands used for exploration and placement, but found similar results as Experiment 1. A third Experiment found evidence supporting the explanation that sound corrects the leftward bias by heightening attention. Together, these findings show that sound, even if task-irrelevant and semantically unrelated, can correct one’s tendency to place objects too far to the left.