scholarly journals Learning to Integrate an Artificial Sensory Device: From Decision Fusion to Sensory Integration

2020 ◽  
Author(s):  
Mohammad-Ali Nikouei Mahani ◽  
Karin Maria Bausenhart ◽  
Rolf Ulrich ◽  
Majid Nili Ahmadabadi
Keyword(s):  
Formal Analysis ◽  
Sensory System ◽  
Sensory Information ◽  
Discrimination Performance ◽  
Decision Fusion ◽  
Sensory Substitution ◽  
Visual Direction ◽  
High Signal ◽  
Integration Policy ◽  
Tactile Information

AbstractThe present study examines how artificial tactile stimulation from a novel non-invasive sensory device is learned and integrated with information from another sensory system. Participants were trained to identify the direction of visual dot motion stimuli with a low, medium, and high signal-to-noise ratio. In bimodal trials, this visual direction information was paired with reliable symbolic tactile information. Over several blocks of training, discrimination performance in unimodal tactile test trials improved, indicating that participants were able to associate the visual and tactile information and thus learned the meaning of the symbolic tactile cues. Formal analysis of the results in bimodal trials showed that the information from both modalities was integrated according to two different integration policies. Initially, participants seemed to rely on a linear decision integration policy based on the metacognitive experience of confidence. In later learning phases, however, our results are consistent with a Bayesian integration policy, that is, optimal integration of sensory information. Thus, the present study demonstrates that humans are capable of learning and integrating an artificial sensory device delivering symbolic tactile information. This finding connects the field of multisensory integration research to the development of sensory substitution systems.

Linking sensory neurons to visually guided behavior: Relating MST activity to steering in a virtual environment

2013 ◽  
Vol 30 (5-6) ◽  
pp. 315-330 ◽  
Author(s):  
SETH W. EGGER ◽  
KENNETH H. BRITTEN
Keyword(s):  
Cortical Neurons ◽  
Traditional Approach ◽  
Sensory System ◽  
Sensory Information ◽  
Neuronal Population ◽  
Difficult Problem ◽  
Lower Envelope ◽  
Visually Guided ◽  
Medial Superior Temporal ◽  
Visual Cortical

AbstractMany complex behaviors rely on guidance from sensations. To perform these behaviors, the motor system must decode information relevant to the task from the sensory system. However, identifying the neurons responsible for encoding the appropriate sensory information remains a difficult problem for neurophysiologists. A key step toward identifying candidate systems is finding neurons or groups of neurons capable of representing the stimuli adequately to support behavior. A traditional approach involves quantitatively measuring the performance of single neurons and comparing this to the performance of the animal. One of the strongest pieces of evidence in support of a neuronal population being involved in a behavioral task comes from the signals being sufficient to support behavior. Numerous experiments using perceptual decision tasks show that visual cortical neurons in many areas have this property. However, most visually guided behaviors are not categorical but continuous and dynamic. In this article, we review the concept of sufficiency and the tools used to measure neural and behavioral performance. We show how concepts from information theory can be used to measure the ongoing performance of both neurons and animal behavior. Finally, we apply these tools to dorsal medial superior temporal (MSTd) neurons and demonstrate that these neurons can represent stimuli important to navigation to a distant goal. We find that MSTd neurons represent ongoing steering error in a virtual-reality steering task. Although most individual neurons were insufficient to support the behavior, some very nearly matched the animal’s estimation performance. These results are consistent with many results from perceptual experiments and in line with the predictions of Mountcastle’s “lower envelope principle.”

Sensory Substitution Devices as Advanced Sensory Tools

2018 ◽  
pp. 188-204
Author(s):  
Thomas D. Wright ◽  
Jamie Ward
Keyword(s):  
Nervous System ◽  
Sensory Information ◽  
Sensory Substitution ◽  
Considerable Effort ◽  
Conceptual Space ◽  
System P ◽  
Starting Point ◽  
Sensory Modalities ◽  
Magnetic Sense ◽  
Definition Of

There has been considerable effort devoted towards understanding sensory substitution devices in terms of their relationship to canonical sensory modalities. The approach taken in this essay is rather different, although complementary, in that we seek to define a broad conceptual space of ‘sensory tools’ in which sensory substitution devices can be situated. Such devices range from telescopes, to cochlear implants, to attempts to create a magnetic sense. One feature of these devices is that they operate at the level of ‘raw’ sensory information. As such, systems such as Braille which operate at a symbolic/conceptual level do not count as a sensory tool (or a sensory substitution device) and nor would a device such as CCTV which, although capturing raw sensory information, would not meet a conventional definition of a tool. With this approach, we hope to avoid the circularity inherent in previous attempts at defining sensory substitution and provide a better starting point to explore the effects of sensory tools, more generally, on the functioning of the nervous system.

Design of a 3-D Printed, Modular Lateral Line Sensory System for Hydrodynamic Force Estimation

2017 ◽  
Vol 51 (5) ◽  
pp. 103-115 ◽  
Author(s):  
Kevin Nelson ◽  
Kamran Mohseni
Keyword(s):  
Lateral Line ◽  
Hydrodynamic Force ◽  
Sensory System ◽  
Sensory Information ◽  
Strain Sensors ◽  
Model Complexity ◽  
Lateral Line System ◽  
Force Estimation ◽  
Biologically Inspired ◽  
Line System

AbstractThis paper presents a sensory system that is biologically inspired by the lateral line sensory system found in fish. This artificial lateral line system provides sensory information to be used in vehicle control algorithms, both to reduce model complexity and to measure hydrodynamic disturbances. The system presented in this paper is a modular implementation that can fit around a vehicle without requiring modifications to the hull. The design and manufacturing processes are presented in detail along with considerations for sensor placement and port spacing. An algorithm for calculating the hydrodynamic forces acting on the surface of a vehicle is derived and experimentally validated. An underwater motion capture system and strain sensors are used to calculate a reference hydrodynamic force that compares favorably with the hydrodynamic force calculated by the lateral line system.

scholarly journals The Common Rhythm of Action and Perception

2020 ◽  
Vol 32 (2) ◽  
pp. 187-200 ◽  
Author(s):  
Alessandro Benedetto ◽  
Maria Concetta Morrone ◽  
Alice Tomassini
Keyword(s):  
Phase Locking ◽  
Continuous Process ◽  
Sensory System ◽  
Sensory Information ◽  
Empirical Support ◽  
General Nature ◽  
Action Perception ◽  
Neurophysiological Data ◽  
The Common ◽  
Coupling Mechanisms

Research in the last decade has undermined the idea of perception as a continuous process, providing strong empirical support for its rhythmic modulation. More recently, it has been revealed that the ongoing motor processes influence the rhythmic sampling of sensory information. In this review, we will focus on a growing body of evidence suggesting that oscillation-based mechanisms may structure the dynamic interplay between the motor and sensory system and provide a unified temporal frame for their effective coordination. We will describe neurophysiological data, primarily collected in animals, showing phase-locking of neuronal oscillations to the onset of (eye) movements. These data are complemented by novel evidence in humans, which demonstrate the behavioral relevance of these oscillatory modulations and their domain-general nature. Finally, we will discuss the possible implications of these modulations for action–perception coupling mechanisms.

scholarly journals A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback

2019 ◽  
Vol 4 (27) ◽  
pp. eaau8892 ◽  
Author(s):  
Edoardo D’Anna ◽  
Giacomo Valle ◽  
Alberto Mazzoni ◽  
Ivo Strauss ◽  
Francesco Iberite ◽  
...  
Keyword(s):  
Visual Cues ◽  
Tactile Feedback ◽  
Sensory Substitution ◽  
Tactile Information ◽  
Position Information ◽  
Implanted Electrodes ◽  
Myoelectric Prostheses ◽  
Position Feedback ◽  
The Rich ◽  
Improved Function

Current myoelectric prostheses allow transradial amputees to regain voluntary motor control of their artificial limb by exploiting residual muscle function in the forearm. However, the overreliance on visual cues resulting from a lack of sensory feedback is a common complaint. Recently, several groups have provided tactile feedback in upper limb amputees using implanted electrodes, surface nerve stimulation, or sensory substitution. These approaches have led to improved function and prosthesis embodiment. Nevertheless, the provided information remains limited to a subset of the rich sensory cues available to healthy individuals. More specifically, proprioception, the sense of limb position and movement, is predominantly absent from current systems. Here, we show that sensory substitution based on intraneural stimulation can deliver position feedback in real time and in conjunction with somatotopic tactile feedback. This approach allowed two transradial amputees to regain high and close-to-natural remapped proprioceptive acuity, with a median joint angle reproduction precision of 9.1° and a median threshold to detection of passive movements of 9.5°, which was comparable with results obtained in healthy participants. The simultaneous delivery of position information and somatotopic tactile feedback allowed both amputees to discriminate the size and compliance of four objects with high levels of performance (75.5%). These results demonstrate that tactile information delivered via somatotopic neural stimulation and position information delivered via sensory substitution can be exploited simultaneously and efficiently by transradial amputees. This study paves a way to more sophisticated bidirectional bionic limbs conveying richer, multimodal sensations.

Development of an Innovative Dual-Coded Multimedia Application to Improve Reading Comprehension of Students With Imagery Deficit

2017 ◽  
Vol 57 (1) ◽  
pp. 170-200
Author(s):  
Ling Wang ◽  
Jiang Li
Keyword(s):  
Reading Comprehension ◽  
Teacher Perceptions ◽  
Reading Instruction ◽  
Coding Theory ◽  
Sensory System ◽  
Sensory Information ◽  
Written Language ◽  
Multimedia Application ◽  
Dual Coding ◽  
Good Reading

This study describes the development of an innovative multimedia application and examines teacher perceptions of its usefulness in assisting students with imagery deficit to visualize the reading comprehension. Students with good reading skills can easily create mental representations from oral or written language, and their sensory system quickly brings parts to whole through imagery. Students with deficiency may have trouble creating images due to their sensory information from imagery is slow and dull, and they often get stuck on parts and pieces. Those unable to image what they read usually cannot understand and remember what they read, which is consistent with the referential connection between the verbal and visual symbolic systems according to the Dual-Coding Theory. This application helps students to develop the capacity to build mental images sentence by sentence. The students engage a touch screen to draw a picture associated with the meaning of each sentence. With the sentences hidden or pictures shuffled, they select and describe each picture in sequence to retell the story. Pictures can be saved to analyze students’ learning outcomes and needs. Teacher perceptions indicate their willingness to integrate this application into reading instruction to help improve reading comprehension of students with imagery deficit.

scholarly journals Spatio-temporal information in an artificial olfactory mucosa

2008 ◽  
Vol 464 (2092) ◽  
pp. 1057-1077 ◽  
Author(s):  
Manuel A Sánchez-Montañés ◽  
Julian W Gardner ◽  
Timothy C Pearce
Keyword(s):  
Analytical Model ◽  
Temporal Dynamics ◽  
Stationary Phases ◽  
Formal Analysis ◽  
Discrimination Performance ◽  
Olfactory Mucosa ◽  
Temporal Information ◽  
Information Measure ◽  
Space And Time ◽  
Spatio Temporal

Deploying chemosensor arrays in close proximity to stationary phases imposes stimulus-dependent spatio-temporal dynamics on their response and leads to improvements in complex odour discrimination. These spatio-temporal dynamics need to be taken into account explicitly when considering the detection performance of this new odour sensing technology, termed an artificial olfactory mucosa. For this purpose, we develop here a new measure of spatio-temporal information that combined with an analytical model of the artificial mucosa, chemosensor and noise dynamics completely characterizes the discrimination capability of the system. This spatio-temporal information measure allows us to quantify the contribution of both space and time to discrimination performance and may be used as part of optimization studies or calculated directly from an artificial mucosa output. Our formal analysis shows that exploiting both space and time in the mucosa response always outperforms the use of space alone and is further demonstrated by comparing the spatial versus spatio-temporal information content of mucosa experimental data. Together, the combination of the spatio-temporal information measure and the analytical model can be applied to extract the general principles of the artificial mucosa design as well as to optimize the physical and operating parameters that determine discrimination performance.

The Beneficial Effects of Meprobamate on Delayed Response Performance in the Frontal Monkey

1965 ◽  
Vol 17 (2) ◽  
pp. 118-124 ◽  
Author(s):  
L. Weiskrantz ◽  
C. G. Gross ◽  
V. Baltzer
Keyword(s):  
Sensory Information ◽  
Discrimination Performance ◽  
Delayed Response ◽  
Beneficial Effects ◽  
Logical Flaw ◽  
Response Performance

Previous studies of antagonistic treatments on delayed response performance by frontal monkeys suffer from a logical flaw in that the treatments may also improve the performance of normal monkeys. In a previous study (Gross and Weiskrantz, 1961) we have shown that meprobamate is without effect on delayed response in normal monkeys, but it does severely depress their discrimination performance. In the current study meprobamate is shown to produce a significant improvement in delayed response in each of three frontal monkeys. Nembutal is also effective. Various interpretations of the results are discussed, the view being favoured that the frontal monkey suffers from an excessive and inappropriately ordered intake of sensory information.

scholarly journals Interplay between Primary Cortical Areas and Crossmodal Plasticity

2021 ◽  
Author(s):  
Christian Xerri ◽  
Yoh’i Zennou-Azogui
Keyword(s):  
Sensory System ◽  
Sensory Substitution ◽  
Multimodal Integration ◽  
Cortical Areas ◽  
Multimodal Interactions ◽  
Multisensory Interactions ◽  
Crossmodal Plasticity ◽  
Perceptual Representations ◽  
Sensory Cortices ◽  
Compensatory Plasticity

Perceptual representations are built through multisensory interactions underpinned by dense anatomical and functional neural networks that interconnect primary and associative cortical areas. There is compelling evidence that primary sensory cortical areas do not work in segregation, but play a role in early processes of multisensory integration. In this chapter, we firstly review previous and recent literature showing how multimodal interactions between primary cortices may contribute to refining perceptual representations. Secondly, we discuss findings providing evidence that, following peripheral damage to a sensory system, multimodal integration may promote sensory substitution in deprived cortical areas and favor compensatory plasticity in the spared sensory cortices.

Sign in / Sign up

Export Citation Format

Share Document