Cross-Format Integration of Auditory Number Words and Visual-Arabic Digits: An ERP Study

Converting visual-Arabic digits to auditory number words and vice versa is seemingly effortless for adults. However, it is still unclear whether this process takes place automatically and whether accessing the underlying magnitude representation is necessary during this process. In two event-related potential (ERP) experiments, adults were presented with identical (e.g., “one” and 1) or non-identical (e.g., “one” and 9) number pairs, either unimodally (two visual-Arabic digits) or cross-format (an auditory number word and a visual-Arabic digit). In Experiment 1 (N=17), active task demands required numerical judgments, whereas this was not the case in Experiment 2 (N=19). We found pronounced early ERP markers of numerical identity unimodally in both experiments. In the cross-format conditions, however, we only observed late neural correlates of identity and only if the task required semantic number processing (Experiment 1). These findings suggest that unimodal pairs of digits are automatically integrated, whereas cross-format integration of numerical information occurs more slowly and involves semantic access.

Impaired emotion perception and categorization in semantic aphasia

According to a constructionist model of emotion, conceptual knowledge plays a foundational role in emotion perception; reduced availability of relevant conceptual knowledge should therefore impair emotion perception. Conceptual deficits can follow both degradation of semantic knowledge (e.g., semantic ‘storage’ deficits in semantic dementia) and deregulation of retrieval (e.g., semantic ‘access’ deficits in semantic aphasia). While emotion recognition deficits are known to accompany degraded conceptual knowledge, less is known about the impact of semantic access deficits. Here, we examined emotion perception and categorization tasks in patients with semantic aphasia, who have difficulty accessing semantic information in a flexible and controlled fashion following left hemisphere stroke. In Study 1, participants were asked to sort faces according to the emotion they portrayed – with numbers, written labels and picture examples each provided as category anchors across tasks. Semantic aphasia patients made more errors and showed a larger benefit from word anchors that reduced the need to internally constrain categorization than age-matched controls. They successfully sorted portrayals that differed in valance (positive vs. negative) but had difficulty categorizing different negative emotions. In Study 2, participants matched facial emotion portrayals to written labels following vocal emotion prosody cues, miscues, or no cues. Patients presented with overall poorer performance and benefited from cue trials relative to within-valence miscue trials. This same effect was seen in controls, who also showed deleterious effects of within-valence miscue relative to no cue trials. Overall we found that patients with deregulated semantic retrieval have deficits in emotional perception but that word anchors and cue conditions can facilitate emotion perception by increasing access to relevant emotion concepts and reducing reliance on semantic control. Semantic control may be of particular importance in emotion perception when it is necessary to interpret ambiguous inputs, or when there is interference between conceptually similar emotional states. These findings extend constructionist accounts of emotion to encompass difficulties in controlled semantic retrieval.

Context-Based Facilitation of Semantic Access Follows Both Logarithmic and Linear Functions of Stimulus Probability

Stimuli are easier to process when the preceding context (e.g., a sentence, in the case of a word) makes them predictable. However, it remains unclear whether context-based facilitation arises due to predictive preactivation of a limited set of relatively probable upcoming stimuli (with facilitation then linearly related to probability) or, instead, arises because the system maintains and updates a probability distribution across all items, as posited by accounts (e.g., surprisal theory) assuming a logarithmic function between predictability and processing effort. To adjudicate between these accounts, we measured the N400 component, an index of semantic access, evoked by sentence-final words of varying probability, including unpredictable words, which are never generated in human production norms. Word predictability was measured using both cloze probabilities and a state-of-the-art machine learning language model (GPT-2). We reanalyzed five datasets (n=138) to first demonstrate and then replicate that context-based facilitation on the N400 is graded and dissociates even among words with cloze probabilities at or near 0, as a function of very small differences in model-estimated predictability. Furthermore, we established that the relationship between word predictability and context-based facilitation on the N400 is neither purely linear nor purely logarithmic but instead combines both functions. We argue that such a composite function reveals properties of the mapping between words and semantic features and how feature- and word- related information is activated during on-line processing. Overall, the results provide powerful evidence for the role of internal models in shaping how the brain apprehends incoming stimulus information.

Recruitment of representational components of the semantic system distinguishes successful from unsuccessful access to complex factual knowledge

Our ability to effectively retrieve complex semantic knowledge meaningfully impacts our daily lives, yet the interactions between semantic control and semantic representational systems that underly successful access and transient failures in access remain only partially understood. In this fMRI study, we contrast activation during successful semantic access, unsuccessful semantic access due to transient access-failures (i.e., "tip-of-the-tongue", "feeling-of-knowing"), and trials where the semantic knowledge was not possessed. Twenty-four participants were presented 240 trivia-based questions relating to person, place, object or scholastic knowledge-domains. Whole brain analyses of the recall event indicate comparable recruitment of prefrontal semantic control systems during successful and unsuccessful semantic access and greater activation in representational systems in successful access. Region-of-interest analysis of domain-selective areas showed that successful access was generally associated with increased responses for both preferred and non-preferred stimuli, with the exception of place-selective regions (PPA, TOS and RSC). Both whole brain and Region-of-interest analysis showed the particular recruitment of place-selective regions during unsuccessful attempts at semantic access, for all stimulus domains. Collectively, these results suggest that prefrontal semantic control systems and classical spatial-knowledge-selective regions work together to locate relevant information and that access to complex knowledge results in a broadening of semantic representation to include regions selective for other knowledge domains.

Does morphological structure modulate access to embedded word meaning in child readers?

Beginning readers have been shown to be sensitive to the meaning of embedded neighbors (e.g., CROW in CROWN). Moreover, developing readers are sensitive to the morphological structure of words (TEACH-ER). However, the interaction between orthographic and morphological processes in meaning activation during reading is not well established. What determines semantic access to orthographically embedded words? What is the role of suffixes in this process? And how does this change throughout development? To address these questions, we asked 80 Italian elementary school children (third, fourth, and fifth grade) to make category decisions on words (e.g., is CARROT a type of food?). Critically, some target words for no-answers (e.g., is CORNER a type of food?) contained category-congruent embedded stems (i.e., CORN). To gauge the role of morphology in this process, half of the embedded stems were accompanied by a pseudosuffix (CORN-ER) and half by a non-morphological ending (PEA-CE). Results revealed that words were harder to reject as members of a category when the embedded stem was category-congruent. This effect held both with and without a pseudosuffix, but was larger for pseudosuffixed words in the error rates. These results suggest that orthographic stems are activated and activation is fed forward to the semantic level regardless of morphological structure, followed by a decision-making process that might strategically use suffix-like endings.

Affordance Derivation Facilitates the Semantic Access in Comprehending Chinese Puns

The study employed ERP technique to explore whether the affordance derivation can facilitate semantic access in comprehending Chinese puns. ERPs were measured while participants read the pun sentences containing dual meanings and made a judgement about the following probes and statements. The results showed that highly related probes in pun sentences elicited a smaller N400 and a larger LPC than moderately related probes in pun sentences. As for the comparison of sentence types, both highly and moderately related probes in pun sentences produce a smaller N400 and a larger LPC than those in control sentences. These results indicate that in the early stage of pun comprehension, semantic access to the literal meaning is easier through affordance derivation because of meaning dominance and frequency. In the late stage of integration, however, the intended meaning of puns can be facilitated and accessed through its privilege of affordance derivation activated by pun words in a pun context because of the priming context and its underlying intention. The study has discovered empirically that it is the affordance derivation, which connects the context and the dual meanings indicated by the pun words, that contributes to the different time courses and dynamic underlying neurocognitive mechanisms in comprehending puns in Chinese.