scholarly journals Images of the unseen: extrapolating visual representations for abstract and concrete words in a data-driven computational model

2020 ◽  
Author(s):  
Fritz Günther ◽  
Marco Alessandro Petilli ◽  
Alessandra Vergallito ◽  
Marco Marelli
Keyword(s):  
Computational Model ◽  
Perceptual Experience ◽  
Learning Experience ◽  
Mapping Function ◽  
Grounded Cognition ◽  
Data Driven ◽  
Abstract Concepts ◽  
Text Corpora ◽  
Abstract Words ◽  
Concrete Words

AbstractTheories of grounded cognition assume that conceptual representations are grounded in sensorimotor experience. However, abstract concepts such as jealousy or childhood have no directly associated referents with which such sensorimotor experience can be made; therefore, the grounding of abstract concepts has long been a topic of debate. Here, we propose (a) that systematic relations exist between semantic representations learned from language on the one hand and perceptual experience on the other hand, (b) that these relations can be learned in a bottom-up fashion, and (c) that it is possible to extrapolate from this learning experience to predict expected perceptual representations for words even where direct experience is missing. To test this, we implement a data-driven computational model that is trained to map language-based representations (obtained from text corpora, representing language experience) onto vision-based representations (obtained from an image database, representing perceptual experience), and apply its mapping function onto language-based representations for abstract and concrete words outside the training set. In three experiments, we present participants with these words, accompanied by two images: the image predicted by the model and a random control image. Results show that participants’ judgements were in line with model predictions even for the most abstract words. This preference was stronger for more concrete items and decreased for the more abstract ones. Taken together, our findings have substantial implications in support of the grounding of abstract words, suggesting that we can tap into our previous experience to create possible visual representation we don’t have.

scholarly journals Images of the unseen: Extrapolating visual representations for abstract and concrete words in a data-driven computational model

2020 ◽  
Author(s):  
Fritz Guenther ◽  
Marco Alessandro Petilli ◽  
Alessandra Vergallito ◽  
Marco Marelli
Keyword(s):  
Computational Model ◽  
Perceptual Experience ◽  
Learning Experience ◽  
Mapping Function ◽  
Grounded Cognition ◽  
Data Driven ◽  
Abstract Concepts ◽  
Text Corpora ◽  
Abstract Words ◽  
Concrete Words

Theories of grounded cognition assume that conceptual representations are grounded in sensorimotor experience. However, abstract concepts such as jealousy or childhood have no directly associated referents with which such sensorimotor experience can be made; therefore, the grounding of abstract concepts has long been a topic of debate. Here, we propose (a) that systematic relations exist between semantic representations learned from language on the one hand and perceptual experience on the other hand, (b) that these relations can be learned in a bottom-up fashion, and (c) that it is possible to extrapolate from this learning experience to predict expected perceptual representations for words even where direct experience is missing. To test this, we implement a data-driven computational model that is trained to map language-based representations (obtained from text corpora, representing language experience) onto vision-based representations (obtained from an image database, representing perceptual experience), and apply its mapping function onto language-based representations for abstract and concrete words outside the training set. In three experiments, we present participants with these words, accompanied by two images: the image predicted by the model and a random control image. Results show that participants' judgements were in line with model predictions even for the most abstract words. This preference was stronger for more concrete items and decreased for the more abstract ones. Taken together, our findings have substantial implications in support of the grounding of abstract words, suggesting that we can tap into our previous experience to create possible visual representation we don't have.

scholarly journals Functional Neuroanatomy of Contextual Acquisition of Concrete and Abstract Words

2009 ◽  
Vol 21 (11) ◽  
pp. 2154-2171 ◽  
Author(s):  
Anna Mestres-Missé ◽  
Thomas F. Münte ◽  
Antoni Rodriguez-Fornells
Keyword(s):  
Native Speakers ◽  
Brain Regions ◽  
Functional Neuroanatomy ◽  
Linguistic Context ◽  
Abstract Concepts ◽  
New Words ◽  
Simulation Paradigm ◽  
Abstract Words ◽  
The Brain ◽  
Concrete Words

The meaning of a novel word can be acquired by extracting it from linguistic context. Here we simulated word learning of new words associated to concrete and abstract concepts in a variant of the human simulation paradigm that provided linguistic context information in order to characterize the brain systems involved. Native speakers of Spanish read pairs of sentences in order to derive the meaning of a new word that appeared in the terminal position of the sentences. fMRI revealed that learning the meaning associated to concrete and abstract new words was qualitatively different and recruited similar brain regions as the processing of real concrete and abstract words. In particular, learning of new concrete words selectively boosted the activation of the ventral anterior fusiform gyrus, a region driven by imageability, which has previously been implicated in the processing of concrete words.

scholarly journals Constructing Semantic Models From Words, Images, and Emojis

2020 ◽  
Author(s):  
Armand Stefan Rotaru ◽  
Gabriella Vigliocco
Keyword(s):  
Visual Information ◽  
Semantic Processing ◽  
Empirical Work ◽  
Abstract Concepts ◽  
Visual Model ◽  
Semantic Data ◽  
Text Corpora ◽  
Abstract Words ◽  
Best Fit ◽  
Better Than

A number of recent models of semantics combine linguistic information, derived from text corpora, and visual information, derived from image collections, demonstrating that the resulting multimodal models are better than either of their unimodal counterparts, in accounting for behavioural data. Empirical work on semantic processing has shown that emotion also plays an important role especially for abstract concepts, however, models integrating emotion along with linguistic and visual information are lacking. Here, we first improve on visual and affective representations, derived from state-of-the-art existing models, by choosing models that best fit available human semantic data and extending the number of concepts they cover. Crucially then, we assess whether adding affective representations (obtained from a neural network model designed to predict emojis from co-occurring text) improves the model’s ability to fit semantic similarity/relatedness judgements from a purely linguistic and linguistic-visual model. We find that, given specific weights assigned to the models, adding both visual and affective representations improve performance, with visual representations providing an improvement especially for more concrete words, and affective representations improving especially the fit for more abstract words.

scholarly journals Learning abstract words and concepts: insights from developmental language disorder

2018 ◽  
Vol 373 (1752) ◽  
pp. 20170140 ◽  
Author(s):  
Marta Ponari ◽  
Courtenay Frazier Norbury ◽  
Armand Rotaru ◽  
Alessandro Lenci ◽  
Gabriella Vigliocco
Keyword(s):  
Language Disorder ◽  
Abstract Word ◽  
Primary Role ◽  
Typically Developing ◽  
Developmental Language Disorder ◽  
Abstract Concepts ◽  
Abstract Knowledge ◽  
Linguistic Input ◽  
Abstract Words ◽  
Concrete Words

Some explanations of abstract word learning suggest that these words are learnt primarily from the linguistic input, using statistical co-occurrences of words in language, whereas concrete words can also rely on non-linguistic, experiential information. According to this hypothesis, we expect that, if the learner is not able to fully exploit the information in the linguistic input, abstract words should be affected more than concrete ones. Embodied approaches instead argue that both abstract and concrete words can rely on experiential information and, therefore, there might not be any linguistic primacy. Here, we test the role of linguistic input in the development of abstract knowledge with children with developmental language disorder (DLD) and typically developing children aged 8–13. We show that DLD children, who by definition have impoverished language, do not show a disproportionate impairment for abstract words in lexical decision and definition tasks. These results indicate that linguistic information does not have a primary role in the learning of abstract concepts and words; rather, it would play a significant role in semantic development across all domains of knowledge. This article is part of the theme issue ‘Varieties of abstract concepts: development, use and representation in the brain’.

scholarly journals From Affordances to Abstract Words: The Flexibility of the Sensorimotor Grounding

2021 ◽  
Author(s):  
Claudia Mazzuca ◽  
Chiara Fini ◽  
Arthur Henri Michalland ◽  
Ilenia Falcinelli ◽  
Federico Da Rold ◽  
...  
Keyword(s):  
Word Processing ◽  
Primary Source ◽  
Inner Speech ◽  
Future Research ◽  
Sensorimotor System ◽  
Abstract Concepts ◽  
Abstract Words ◽  
Complex Words ◽  
Different Levels ◽  
Concrete Words

Recent research has shown that the sensorimotor system plays a significant role in a variety of cognitive processes. In this paper, we will review recent studies performed in our lab (Body Action Language Lab, BALLAB) or in labs with which we collaborate, showing the involvement of the sensorimotor system at different levels. With the purpose of expounding on this aspect, we focus on studies that highlight two main characteristics of the involvement of the sensorimotor systems. First, we concentrate on the flexibility of sensorimotor grounding during interaction with objects. We report evidence showing how social context and current situations influence affordance activation. We then focus on the tactile and kinesthetic involvement in body-object interaction. Second, we illustrate flexible sensorimotor grounding in word use. We review studies showing that not only concrete words, like “bottle,” but also abstract words, like “freedom,” “thinking,” and “perhaps,” are grounded in the sensorimotor system. We report evidence showing that abstract words activate sensory modalities and involve the mouth effector more than concrete words due to their privileged relationship with language, both outer and inner speech. We discuss the activation of the mouth sensorimotor system in light of studies on adults (e.g., studies employing articulatory suppression), children (e.g., studies on the effects of pacifier use on word acquisition and processing), and infants (e.g. studies on emergence of new words). Finally, we pinpoint possible mechanisms at play in the acquisition and use of abstract concepts. We argue that with abstract concepts, we rely more on other people to learn or negotiate the meaning of words; we have called this mechanism social metacognition.Social metacognition is bidirectionally linked to our sensorimotor system. On the one hand, linguistic explanations constitute a primary source of grounding that may be re-enacted when retrieving a concept, for example through inner speech. On the other hand, it leads us to feel closer and be more synchronous in movement with others, who can help us understand the meaning of very complex words. Overall, we show that the sensorimotor system provides a grounding basis not only for objects and concrete words but also for more abstract and concrete ones. We conclude by arguing that future research should address and deepen two different and interrelated aspects concerning the involvement of the sensorimotor system during object and word processing. First, the sensorimotor system is flexibly modulated by the context, as studies on affordances reveal. Second, the sensorimotor system can be involved at different levels, and its role can be integrated and flanked by that of other systems, like the linguistic one, as studies on abstract concepts clearly show. We urge future research aimed at unravelling the role of the sensorimotor system in cognition to fully explore the complexity of this intricate-and sometimes slippery-relation.

scholarly journals Visual and Affective Grounding in Language and Mind

2018 ◽  
Author(s):  
Simon De Deyne ◽  
Danielle Navarro ◽  
Guillem Collell ◽  
Amy Perfors
Keyword(s):  
Mental Representations ◽  
Word Meaning ◽  
Language Models ◽  
Abstract Concepts ◽  
Text Corpora ◽  
Semantic Models ◽  
Abstract Words ◽  
Distributional Semantic Models ◽  
Affective Information ◽  
New Evidence

One of the main limitations in natural language-based approaches to meaning is that they are not grounded. In this study, we evaluate how well different kinds of models account for people’s representations of both concrete and abstract concepts. The models are both unimodal (language-based only) models and multimodal distributional semantic models (which additionallyincorporate perceptual and/or affective information). The language-based models include both external (based on text corpora) and internal (derived from word associations) language. We present two new studies and a re-analysis of a series of previous studies demonstrating that the unimodal performance is substantially higher for internal models, especially when comparisons at the basiclevel are considered. For multimodal models, our findings suggest that additional visual and affective features lead to only slightly more accurate mental representations of word meaning than what is already encoded in internal language models; however, for abstract concepts, visual andaffective features improve the predictions of external text-based models. Our work presents new evidence that the grounding problem includes abstract words as well and is therefore more widespread than previously suggested. Implications for both embodied and distributional views arediscussed.

Distinct Brain Systems for Processing Concrete and Abstract Concepts

2005 ◽  
Vol 17 (6) ◽  
pp. 905-917 ◽  
Author(s):  
J. R. Binder ◽  
C. F. Westbury ◽  
K. A. McKiernan ◽  
E. T. Possing ◽  
D. A. Medler
Keyword(s):  
Conceptual Knowledge ◽  
Word Processing ◽  
Semantic Processing ◽  
Verbal Working Memory ◽  
Concrete Word ◽  
Angular Gyrus ◽  
Abstract Concepts ◽  
Central Interest ◽  
Abstract Words ◽  
Concrete Words

Behavioral and neurophysiological effects of word imageability and concreteness remain a topic of central interest in cognitive neuroscience and could provide essential clues for understanding how the brain processes conceptual knowledge. We examined these effects using event-related functional magnetic resonance imaging while participants identified concrete and abstract words. Relative to nonwords, concrete and abstract words both activated a left-lateralized network of multimodal association areas previously linked with verbal semantic processing. Areas in the left lateral temporal lobe were equally activated by both word types, whereas bilateral regions including the angular gyrus and the dorsal prefrontal cortex were more strongly engaged by concrete words. Relative to concrete words, abstract words activated left inferior frontal regions previously linked with phonological and verbal working memory processes. The results show overlapping but partly distinct neural systems for processing concrete and abstract concepts, with greater involvement of bilateral association areas during concrete word processing, and processing of abstract concepts almost exclusively by the left hemisphere.

scholarly journals Learning and Using Abstract Words: Evidence from Clinical Populations

2017 ◽  
Vol 2017 ◽  
pp. 1-8 ◽  
Author(s):  
Maria Luisa Lorusso ◽  
Michele Burigo ◽  
Alessandro Tavano ◽  
Anna Milani ◽  
Sara Martelli ◽  
...  
Keyword(s):  
Down Syndrome ◽  
Williams Syndrome ◽  
Cognitive Profile ◽  
Typically Developing ◽  
Abstract Concepts ◽  
Visual Spatial ◽  
Abstract Words ◽  
Concrete Concepts ◽  
Concrete Words

It has been shown that abstract concepts are more difficult to process and are acquired later than concrete concepts. We analysed the percentage of concrete words in the narrative lexicon of individuals with Williams Syndrome (WS) as compared to individuals with Down Syndrome (DS) and typically developing (TD) peers. The cognitive profile of WS is characterized by visual-spatial difficulties, while DS presents with predominant impairments in linguistic abilities. We predicted that if linguistic abilities are crucial to the development and use of an abstract vocabulary, DS participants should display a higher concreteness index than both Williams Syndrome and typically developing individuals. Results confirm this prediction, thus supporting the hypothesis of a crucial role of linguistic processes in abstract language acquisition. Correlation analyses suggest that a maturational link exists between the level of abstractness in narrative production and syntactic comprehension.

scholarly journals Emotion and concreteness effects when learning novel concepts in the native language

2021 ◽  
Vol 42 (2) ◽  
pp. 177-191
Author(s):  
Marc Guasch ◽  
Pilar Ferré
Keyword(s):  
Native Language ◽  
Recognition Task ◽  
Intentional Learning ◽  
The Novel ◽  
Abstract Concepts ◽  
Word Forms ◽  
Abstract Words ◽  
Concrete Concepts ◽  
Concrete Words ◽  
Better Than

Abstract The aim of the present study was to test the proposal of Kousta et al. (2011), according to which abstract words are more affectively loaded than concrete words. To this end, we focused on the acquisition of novel concepts by means of an intentional learning experiment in which participants had to learn a set of 40 novel concepts in Spanish (definitions) associated with novel word forms (pseudowords). Concreteness (concrete vs. abstract concepts) and emotionality (neutral vs. negative concepts) were orthogonally manipulated. Acquisition was assessed through a recognition task in which participants were asked to match the novel word forms with their definitions. Results showed that concrete concepts were acquired better than abstract concepts. Importantly, the concreteness advantage disappeared when the content of the concept was negative. Hence, emotional (negative) content facilitated the acquisition of abstract concepts, but not of concrete concepts, giving support to the proposal of Kousta et al. (2011).

Abstract Semantic Associative Network Training: A Replication and Update of an Abstract Word Retrieval Therapy Program

2020 ◽  
Vol 29 (3) ◽  
pp. 1574-1595
Author(s):  
Chaleece W. Sandberg ◽  
Teresa Gray
Keyword(s):  
Concrete Word ◽  
Abstract Word ◽  
Distinct Pattern ◽  
Word Retrieval ◽  
Effect Sizes ◽  
Associative Network ◽  
Direct Training ◽  
Network Training ◽  
Abstract Words ◽  
Concrete Words

Purpose We report on a study that replicates previous treatment studies using Abstract Semantic Associative Network Training (AbSANT), which was developed to help persons with aphasia improve their ability to retrieve abstract words, as well as thematically related concrete words. We hypothesized that previous results would be replicated; that is, when abstract words are trained using this protocol, improvement would be observed for both abstract and concrete words in the same context-category, but when concrete words are trained, no improvement for abstract words would be observed. We then frame the results of this study with the results of previous studies that used AbSANT to provide better evidence for the utility of this therapeutic technique. We also discuss proposed mechanisms of AbSANT. Method Four persons with aphasia completed one phase of concrete word training and one phase of abstract word training using the AbSANT protocol. Effect sizes were calculated for each word type for each phase. Effect sizes for this study are compared with the effect sizes from previous studies. Results As predicted, training abstract words resulted in both direct training and generalization effects, whereas training concrete words resulted in only direct training effects. The reported results are consistent across studies. Furthermore, when the data are compared across studies, there is a distinct pattern of the added benefit of training abstract words using AbSANT. Conclusion Treatment for word retrieval in aphasia is most often aimed at concrete words, despite the usefulness and pervasiveness of abstract words in everyday conversation. We show the utility of AbSANT as a means of improving not only abstract word retrieval but also concrete word retrieval and hope this evidence will help foster its application in clinical practice.

Sign in / Sign up

Export Citation Format

Share Document