Neuroplastic Reorganization Induced by Sensory Augmentation for Self-Localization During Locomotion

Sensory skills can be augmented through training and technological support. This process is underpinned by neural plasticity in the brain. We previously demonstrated that auditory-based sensory augmentation can be used to assist self-localization during locomotion. However, the neural mechanisms underlying this phenomenon remain unclear. Here, by using functional magnetic resonance imaging, we aimed to identify the neuroplastic reorganization induced by sensory augmentation training for self-localization during locomotion. We compared activation in response to auditory cues for self-localization before, the day after, and 1 month after 8 days of sensory augmentation training in a simulated driving environment. Self-localization accuracy improved after sensory augmentation training, compared with the control (normal driving) condition; importantly, sensory augmentation training resulted in auditory responses not only in temporal auditory areas but also in higher-order somatosensory areas extending to the supramarginal gyrus and the parietal operculum. This sensory reorganization had disappeared by 1 month after the end of the training. These results suggest that the use of auditory cues for self-localization during locomotion relies on multimodality in higher-order somatosensory areas, despite substantial evidence that information for self-localization during driving is estimated from visual cues on the proximal part of the road. Our findings imply that the involvement of higher-order somatosensory, rather than visual, areas is crucial for acquiring augmented sensory skills for self-localization during locomotion.

Backward spatial perception can be augmented through a novel visual-to-auditory sensory substitution algorithm

Can humans extend and augment their natural perceptions during adulthood? Here, we address this fascinating question by investigating the extent to which it is possible to successfully augment visual spatial perception to include the backward spatial field (a region where humans are naturally blind) via other sensory modalities (i.e., audition). We thus developed a sensory-substitution algorithm, the “Topo-Speech” which conveys identity of objects through language, and their exact locations via vocal-sound manipulations, namely two key features of visual spatial perception. Using two different groups of blindfolded sighted participants, we tested the efficacy of this algorithm to successfully convey location of objects in the forward or backward spatial fields following ~ 10 min of training. Results showed that blindfolded sighted adults successfully used the Topo-Speech to locate objects on a 3 × 3 grid either positioned in front of them (forward condition), or behind their back (backward condition). Crucially, performances in the two conditions were entirely comparable. This suggests that novel spatial sensory information conveyed via our existing sensory systems can be successfully encoded to extend/augment human perceptions. The implications of these results are discussed in relation to spatial perception, sensory augmentation and sensory rehabilitation.

Can perception be extended to a “feel of North”? Tests of automaticity with the NaviEar

This study investigated the potential for the development of a new perceptual experience through sustained training with a sensory augmentation device. For this purpose, we developed (1) a new sensory augmentation device, the NaviEar, and (2) a battery of tests for automaticity in the use of the device. The NaviEar translates cardinal direction into sound. Because the signal of this device is comparatively simple and easy to interpret it should facilitate automatization of use. The test battery assesses different effects of automaticity (interference, rigidity of responses, dynamic integration) assuming that automaticity is a necessary criterion to show the emergence of perceptual feel, i.e. the sensory quality of a percept. We measured performance in simple training tasks, administered the tests for automaticity, and assessed subjective reports through a questionnaire. Results suggest that the NaviEar is easy and comfortable to use and has a potential for applications in real-world situations; but 5-day training with the NaviEar does not reach levels of automaticity that are indicative of perceptual feel. We propose that the test battery for automaticity may be used as a benchmark test for the possibility of perceptual feel in sensory augmentation and in sensory substitution.

A New Sensory Skill Shows Automaticity and Integration Features in Multisensory Interactions

ABSTRACTPeople can learn new sensory skills that augment their perception, such as human echolocation. However, it is not clear to what extent these can become an integral part of the perceptual repertoire. Can they show automatic use, integrated with the other senses, or do they remain cognitively-demanding, cumbersome, and separate? Here, participants learned to judge distance using an echo-like auditory cue. We show that use of this new skill met three key criteria for automaticity and sensory integration: (1) enhancing the speed of perceptual decisions; (2) processing through a non-verbal route and (3) integration with vision in an efficient, Bayes-like manner. We also show some limits following short training: integration was less-than-optimal, and there was no mandatory fusion of signals. These results demonstrate key ways in which new sensory skills can become automatic and integrated, and suggest that sensory augmentation systems may have benefits beyond current applications for sensory loss.

Evaluating the concepts of weak and subcognitive integration - an experimental study using vibrotactile sensory augmentation for the blind

Navigating in foreign surroundings necessitates peak concentration for blind travellers. Yet, most navigational aids heavily rely on attentional resources as well as on audition. Audition is a modality of supreme importance for the blind, allowing to react to cues of the immediate environment. Thus, it would be highly beneficial for a navigational aid for the blind to not or only partially rely on attentional resources and be easily interpreted and integrated into behaviour. Following the sensorimotor contingency (SMC) theory, which is embedded in the theoretical framework of embodiment, such endeavour has the potential to succeed by employing sensory augmentation devices. According to SMC theory, statistic regularities termed sensorimotor contingencies coupling action and perception are constitutive of conscious perception. Consequentially, since those regularities differ in between modality, also the qualitative experience of different modalities differ. Following this line of thought, new SMCs can be created through sensory augmentation devices and learned by exploring the SMC. The objective of this study is to further investigate if and to what extent such sensory augmentation device can be integrated into behaviour. Therefore, the weak integration hypothesis and the sub-cognitive processing hypothesis as established by Nagel et al. (2005) will be employed to evaluate the integration according to their criteria.Eleven congenitally and adventitiously blind adult subjects were provided with vibrotactile directional information of the magnetic north around the waist through a device termed naviBelt for seven weeks. At the beginning and at the middle of the study the integration of the signal of five participants was assessed using a battery of behavioural tests. These tests consisted of a straight-line-walking task, an angular rotation task and a triangle completion task. Furthermore, throughout the period of study all participants completed preliminary, weekly and final questionnaires, inspired by Kärcher et al. (2012). The questionnaires allowed to gain a more holistic picture of the subjective experience and the self-assessed benefits of the belt. In addition, two deaf-blind participants were provided with the belt for three to four weeks and answered questionnaires adjusted to their needs.The straight-line-walking task showed instant improvements in path stabilization when provided with the belt. In two participants characteristic behaviour of the sub-cognitive processing hypothesis is obtained. An overall improvement independent of whether the belt is worn or not is especially evident after the training period in the angular rotation task. This indicates an enhanced direction estimation accuracy, which is highly related to the understanding of the belt signal. Evidence for enhanced path integration and navigational skills through the belt can be found in the results of the triangle completion task. For two participants the performance improved even with an additional attentional load, hinting towards sub-cognitive processing.Overall, the data supports the weak integration hypothesis and points towards the sub-cognitive processing hypothesis and thus show that SMCs can be learned, which is in line with the theory of embodiment. Crucially, the study further exemplifies how such integration into behaviour can be of great benefit as assistive device for blind and deaf-blind.

Dazzling Technologies: Avant-Gardes and Sensory Augmentation in the First World War

This Chapter identifies a new form of technicity that emerged in the First World War, in which enhancement and distortion effects generated by sensory augmentation technologies could be manipulated for strategic purposes by a variety of cultural agents. It argues that ‘dazzle camouflage’, a technology developed by the British Admiralty in 1917 to delay and confuse attacking U-boats, exemplifies this mediation of everyday life both on and off the battle fronts. Focusing on the Florentine journal Lacerba and the Vorticist magazine Blast, the first part of the chapter shows how the Futurists F. T. Marinetti, Armando Mazza, Alberto Viviani, Ardengo Soffici and Carlo Carrà, and the Vorticists Wyndham Lewis, Jessica Dismorr and Helen Saunders, developed their ‘dazzle poetics’ to analyse, critique and exploit the sensory overload of the Machine Age. The second part of the chapter explores ‘dazzle technologies’ produced by avant-gardes and the military. It identifies convergences between Luigi Russolo’s intonarumori noise machines and Alvin Langdon Coburn’s and Ezra Pound’s ‘Vortoscope’ camera apparatuses, offering new perspectives on their prescient manipulation of sensory augmentation ensembles. The chapter concludes with an analysis of Edward Wadsworth’s role in the development of dazzle technicity and the production of dazzle camouflage.