Developmental Designs and Adult Functions of Cortical Maps in Multiple Modalities: Perception, Attention, Navigation, Numbers, Streaming, Speech, and Cognition

Children with Rett Syndrome (RS) are reported to use multiple modalities to communicate although their intentionality is often questioned (Bartolotta, Zipp, Simpkins, & Glazewski, 2011; Hetzroni & Rubin, 2006; Sigafoos et al., 2000; Sigafoos, Woodyatt, Tuckeer, Roberts-Pennell, & Pittendreigh, 2000). This paper will present results of a study analyzing the unconventional vocalizations of a child with RS. The primary research question addresses the ability of familiar and unfamiliar listeners to interpret unconventional vocalizations as “yes” or “no” responses. This paper will also address the acoustic analysis and perceptual judgments of these vocalizations. Pre-recorded isolated vocalizations of “yes” and “no” were presented to 5 listeners (mother, father, 1 unfamiliar, and 2 familiar clinicians) and the listeners were asked to rate the vocalizations as either “yes” or “no.” The ratings were compared to the original identification made by the child's mother during the face-to-face interaction from which the samples were drawn. Findings of this study suggest, in this case, the child's vocalizations were intentional and could be interpreted by familiar and unfamiliar listeners as either “yes” or “no” without contextual or visual cues. The results suggest that communication partners should be trained to attend to eye-gaze and vocalizations to ensure the child's intended choice is accurately understood.

Download Full-text

Stable neural representations of mental states across target people and stimulus modalities

10.31234/osf.io/7axzj ◽

2020 ◽

Author(s):

Miriam E. Weaverdyck ◽

Mark Allen Thornton ◽

Diana Tamir

Keyword(s):

Mental State ◽

Mental States ◽

Brain Regions ◽

The Self ◽

Similarity Analysis ◽

Multiple Modalities ◽

Neural Representations ◽

Conceptual Core ◽

Stimulus Modalities ◽

Huge Variety

Each individual experiences mental states in their own idiosyncratic way, yet perceivers are able to accurately understand a huge variety of states across unique individuals. How do they accomplish this feat? Do people think about their own anger in the same ways as another person’s? Is reading about someone’s anxiety the same as seeing it? Here, we test the hypothesis that a common conceptual core unites mental state representations across contexts. Across three studies, participants judged the mental states of multiple targets, including a generic other, the self, a socially close other, and a socially distant other. Participants viewed mental state stimuli in multiple modalities, including written scenarios and images. Using representational similarity analysis, we found that brain regions associated with social cognition expressed stable neural representations of mental states across both targets and modalities. This suggests that people use stable models of mental states across different people and contexts.

Download Full-text

Faculty Opinions recommendation of Sensory cortex limits cortical maps and drives top-down plasticity in thalamocortical circuits.

Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature ◽

10.3410/f.718027800.793482656 ◽

2013 ◽

Author(s):

Linda Richards ◽

Laura Fenlon

Keyword(s):

Sensory Cortex ◽

Top Down ◽

Cortical Maps

Download Full-text

Modeling cortical maps with Topographica

Neurocomputing ◽

10.1016/j.neucom.2004.01.177 ◽

2004 ◽

Vol 58-60 ◽

pp. 1129-1135 ◽

Cited By ~ 10

Author(s):

James A. Bednar ◽

Yoonsuck Choe ◽

Judah De Paula ◽

Risto Miikkulainen ◽

Jefferson Provost ◽

...

Keyword(s):

Cortical Maps

Download Full-text

Structural and Functional Imaging in Glioma Management

Neurosurgery ◽

10.1093/neuros/nyaa360 ◽

2020 ◽

Author(s):

Bledi C Brahimaj ◽

Ryan B Kochanski ◽

John J Pearce ◽

Melike Guryildirim ◽

Carter S Gerard ◽

...

Keyword(s):

Diffusion Tensor ◽

Imaging Modalities ◽

Hemispheric Dominance ◽

Tumor Control ◽

Histopathological Diagnosis ◽

Glioma Surgery ◽

Multiple Modalities ◽

Advantages And Disadvantages ◽

Multiple Imaging

Abstract The goal of glioma surgery is maximal safe resection in order to provide optimal tumor control and survival benefit to the patient. There are multiple imaging modalities beyond traditional contrast-enhanced magnetic resonance imaging (MRI) that have been incorporated into the preoperative workup of patients presenting with gliomas. The aim of these imaging modalities is to identify cortical and subcortical areas of eloquence, and their relationship to the lesion. In this article, multiple modalities are described with an emphasis on the underlying technology, clinical utilization, advantages, and disadvantages of each. functional MRI and its role in identifying hemispheric dominance and areas of language and motor are discussed. The nuances of magnetoencephalography and transcranial magnetic stimulation in localization of eloquent cortex are examined, as well as the role of diffusion tensor imaging in defining normal white matter tracts in glioma surgery. Lastly, we highlight the role of stimulated Raman spectroscopy in intraoperative histopathological diagnosis of tissue to guide tumor resection. Tumors may shift the normal arrangement of functional anatomy in the brain; thus, utilization of multiple modalities may be helpful in operative planning and patient counseling for successful surgery.

Download Full-text

Subversive compliance and embodiment in remedial interchanges

Text & Talk - An Interdisciplinary Journal of Language Discourse Communication Studies ◽

10.1515/text-2020-2078 ◽

2020 ◽

Vol 40 (5) ◽

pp. 669-693

Author(s):

Asta Cekaite

Keyword(s):

Primary School ◽

Adult Child ◽

First Grade ◽

Moral Emotion ◽

Multiple Modalities ◽

Temporal Duration ◽

One Year

AbstractThis study examines normativity of affect and the affective embeddedness of normativity, instantiated as verbal and embodied stances taken by the participants in adult-child remedial interchanges. The data are based on one year of video fieldwork in a first-grade class at a Swedish primary school. An ethnographically informed analysis of talk and multimodal action is adopted. The findings show that the children’s affective and normative transgressions provided discursive spaces for adult moral instructions and socialization. However, the children’s compliant responses were resistant and subversive. They were designed as embodied double-voiced acts that indexed incongruent affective and moral stances. The findings further revealed several ways of configuring embodied double-voiced responses. The children juxtaposed multiple modalities and exploited the expectations of what constitutes appropriate temporal duration, timing, and shape of nonverbal responses. They (i) combined up-scaled verbal and embodied hyperbolic rhetoric when the teachers’ talk required but minimal responses, and (ii) configured antithetical affect displays, e.g., crying and smiling, or overlaid bodily displays of moral emotion (sadness, seriousness, and smiling) with aligning but exaggerated gestures and movements. Subversive, embodied double-voiced responses simultaneously acquiesced with and deflected the responsibility and effectively derailed a successful closure of remedial interchange.

Download Full-text

Correspondence Learning for Deep Multi-Modal Recognition and Fraud Detection

Electronics ◽

10.3390/electronics10070800 ◽

2021 ◽

Vol 10 (7) ◽

pp. 800

Author(s):

Jongchan Park ◽

Min-Hyun Kim ◽

Dong-Geol Choi

Keyword(s):

Deep Learning ◽

Simple Technique ◽

Detection Method ◽

Fraud Detection ◽

Complementary Information ◽

Multiple Modalities ◽

Vending Machines ◽

Emotion Intensity ◽

The Relationship ◽

Extract Information

Deep learning-based methods have achieved good performance in various recognition benchmarks mostly by utilizing single modalities. As different modalities contain complementary information to each other, multi-modal based methods are proposed to implicitly utilize them. In this paper, we propose a simple technique, called correspondence learning (CL), which explicitly learns the relationship among multiple modalities. The multiple modalities in the data samples are randomly mixed among different samples. If the modalities are from the same sample (not mixed), then they have positive correspondence, and vice versa. CL is an auxiliary task for the model to predict the correspondence among modalities. The model is expected to extract information from each modality to check correspondence and achieve better representations in multi-modal recognition tasks. In this work, we first validate the proposed method in various multi-modal benchmarks including CMU Multimodal Opinion-Level Sentiment Intensity (CMU-MOSI) and CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) sentiment analysis datasets. In addition, we propose a fraud detection method using the learned correspondence among modalities. To validate this additional usage, we collect a multi-modal dataset for fraud detection using real-world samples for reverse vending machines.

Download Full-text