scholarly journals An explicit, generative, and automatic syntactic representation of multi-digit numbers

2021 ◽  
Author(s):  
Dror Dotan ◽  
Nadin Brutman
Keyword(s):  
Ad Hoc ◽  
Short Term Memory ◽  
Syntactic Structure ◽  
Short Term ◽  
Generative Process ◽  
Size Limit ◽  
Number Words ◽  
Chunk Size ◽  
Syntactic Representation ◽  
External Cues

Representing the base-10 structure of numbers is a challenging cognitively ability, unique to humans, but it is yet unknown how precisely this is done. Here, we examined whether and how literate adults represent a number’s full syntactic structure. In 5 experiments, the participants repeated sequences of 6-7 number words, and we systematically varied the order of words within the sequence. Repetition was more accurate when the sequence was grammatical (e.g., ninety-seven) than when it was not (seven-ninety). The performance monotonously improved for sequences with increasingly longer grammatical segments, up to a limit of ~4 words per segment, irrespectively of the number of digits, and worsened thereafter. We conclude that at least for numbers up to 6 digits long, participants represented the number’s full syntactic structure and used it to merge number words into chunks in short-term memory. Short chunks improved memorization, but oversized chunks disrupted memorization. The existence of a chunk size limit suggests that the chunks are not memorized templates, whose size limit is not expected to be so low. Rather, they are created ad-hoc by a generative process, such as the hierarchical syntactic representation hypothesized in Michael McCloskey’s number-processing model. Chunking occurred even when it disrupted performance, and even when external cues for chunking were controlled for or were removed; we conclude that the above generative process operates automatically rather than voluntarily.

scholarly journals Dissociating sensory and cognitive theories of harmony perception through computational modeling

2018 ◽  
Author(s):  
Peter Harrison ◽  
Marcus Thomas Pearce
Keyword(s):  
Statistical Learning ◽  
Short Term Memory ◽  
Syntactic Structure ◽  
Explanatory Power ◽  
Memory Models ◽  
Acoustic Properties ◽  
Probabilistic Prediction ◽  
Short Term ◽  
Cognitive Approach ◽  
Term Memory

Two approaches exist for explaining harmonic expectation. The sensory approach claims that harmonic expectation is a low-level process driven by sensory responses to acoustic properties of musical sounds. Conversely, the cognitive approach describes harmonic expectation as a high-level cognitive process driven by the recognition of syntactic structure learned through experience. Many previous studies have sought to distinguish these two hypotheses, largely yielding support for the cognitive hypothesis. However, subsequent re-analysis has shown that most of these results can parsimoniously be explained by a computational model from the sensory tradition, namely Leman’s (2000) model of auditory short- term memory (Bigand, Delbé, Poulin-Charronnat, Leman, & Tillmann, 2014). In this research we re-examine the explanatory power of auditory short-term memory models, and compare them to a new model in the Information Dynamics Of Music (IDyOM) tradition, which simulates a cognitive theory of harmony perception based on statistical learning and probabilistic prediction. We test the ability of these models to predict the surprisingness of chords within chord sequences (N = 300), as reported by a sample group of university undergraduates (N = 50). In contrast to previous studies, which typically use artificial stimuli composed in a classical idiom, we use naturalistic chord sequences sampled from a large dataset of popular music. Our results show that the auditory short-term memory models have remarkably low explanatory power in this context. In contrast, the new statistical learning model predicts surprisingness ratings relatively effectively. We conclude that auditory short-term memory is insufficient to explain harmonic expectation, and that cognitive processes of statistical learning and probabilistic prediction provide a viable alternative.

scholarly journals Complexity Reduction in the Negotiation of New Lexical Conventions

2018 ◽  
Author(s):  
William Schueller ◽  
Vittorio Loreto ◽  
Pierre-Yves Oudeyer
Keyword(s):  
Ad Hoc ◽  
Short Term Memory ◽  
Fine Tuning ◽  
Global State ◽  
High Complexity ◽  
Short Term ◽  
New Words ◽  
Efficient Control ◽  
New Strategy ◽  
Multi Agent

In the process of collectively inventing new words for new con-cepts in a population, conflicts can quickly become numerous,in the form of synonymy and homonymy. Remembering all ofthem could cost too much memory, and remembering too fewmay slow down the overall process. Is there an efficient be-havior that could help balance the two? The Naming Game isa multi-agent computational model for the emergence of lan-guage, focusing on the negotiation of new lexical conventions,where a common lexicon self-organizes but going through aphase of high complexity. Previous work has been done onthe control of complexity growth in this particular model, byallowing agents to actively choose what they talk about. How-ever, those strategies were relying on ad hoc heuristics highlydependent on fine-tuning of parameters. We define here a newprincipled measure and a new strategy, based on the beliefsof each agent on the global state of the population. The mea-sure does not rely on heavy computation, and is cognitivelyplausible. The new strategy yields an efficient control of com-plexity growth, along with a faster agreement process. Also,we show that short-term memory is enough to build relevantbeliefs about the global lexicon.

Improving Fine-Grained Opinion Mining Approach with a Deep Constituency Tree-Long Short Term Memory Network and Word Embedding

2020 ◽  
Vol 13 ◽  
Author(s):  
Dalila Bouras ◽  
Mohamed Amroune ◽  
Hakim Bendjenna ◽  
Issam Bendib
Keyword(s):  
Opinion Mining ◽  
Short Term Memory ◽  
State Of The Art ◽  
Syntactic Structure ◽  
Product Reviews ◽  
Short Term ◽  
Term Memory ◽  
Fine Grained ◽  
Wide Range ◽  
Long Short Term Memory

Objective: One key task of fine-grained opinion mining on product review is to extract product aspects and their corresponding opinion expressed by users. Previous work has demonstrated that precise modeling of opinion targets within the surrounding context can improve performances. However, how to effectively and efficiently learn hidden word semantics and better represent targets and the context still needs to be further studied. Recent years have seen a revival of the long short-term memory (LSTM), with its effectiveness being demonstrated on a wide range of problems. However, LSTM based approaches are still limited to linear data processing since it processes the information sequentially. As a result, they may perform poorly on user-generated texts, such as product reviews, tweets, etc., whose syntactic structure is not precise.To tackle this challenge, <P> Methods: In this research paper, we propose a constituency tree long short term memory neural network-based approach. We compare our model with state-of-the-art baselines on SemEval 2014 datasets. <P> Results: Experiment results show that our models obtain competitive performances compared to various supervised LSTM architectures. <P> Conclusion: Our work contributes to the improvement of state-of-the-art aspect-level opinion mining methods and offers a new approach to support human decision-making process based on opinion mining results.

scholarly journals Adaptive Semiparametric Language Models

2021 ◽  
Vol 9 ◽  
pp. 362-373
Author(s):  
Dani Yogatama ◽  
Cyprien de Masson d’Autume ◽  
Lingpeng Kong
Keyword(s):  
Ad Hoc ◽  
Nearest Neighbor ◽  
Short Term Memory ◽  
Language Model ◽  
Language Models ◽  
Local Context ◽  
Long Term Memory ◽  
Short Term ◽  
Term Memory

Abstract We present a language model that combines a large parametric neural network (i.e., a transformer) with a non-parametric episodic memory component in an integrated architecture. Our model uses extended short-term context by caching local hidden states—similar to transformer-XL—and global long-term memory by retrieving a set of nearest neighbor tokens at each timestep. We design a gating function to adaptively combine multiple information sources to make a prediction. This mechanism allows the model to use either local context, short-term memory, or long-term memory (or any combination of them) on an ad hoc basis depending on the context. Experiments on word-based and character-based language modeling datasets demonstrate the efficacy of our proposed method compared to strong baselines.

scholarly journals Keras Model for Text Classification in Amazon Review Dataset using LSTM

2021 ◽  
Vol 3 (2) ◽  
pp. 72-89
Author(s):  
Thivaharan S ◽  
Srivatsun G
Keyword(s):  
Classification Accuracy ◽  
Industry 4.0 ◽  
Short Term Memory ◽  
Feedback Mechanism ◽  
Memory Model ◽  
Short Term ◽  
Number Words ◽  
Proposed Model ◽  
Long Short Term Memory ◽  
Made In

With the use of Ecommerce, Industry 4.0 is being effectively used in online product-based commercial transactions. An effort has been made in this article to extract positive and negative sentiments from Amazon review datasets. This will give an upper hold to the purchaser to decide upon a particular product, without considering the manual rating given in the reviews. Even the number words in an inherent positive review exceeds by one, where the present classifiers misclassify them under negative category. This article addresses the aforementioned issue by using LSTM (Long-Short-Term-Memory) model, as LSTM model has a feedback mechanism based progression unlike the other classifiers, which are dependent on feed-forward mechanism. For achieving better classification accuracy, the dataset is initially processed and a total of 100239 short and 411313 long reviews have been obtained. With the appropriate Epoch iterations, it is observed that, this proposed model has gain the ability to classify with 89% accuracy, while maintaining a non-bias between the train and test datasets. The entire model is deployed in TensorFlow2.1.0 platform by using the Keras framework and python 3.6.0.

Conceptual short-term memory (CSTM) supports core claims of Christiansen and Chater

2016 ◽  
Vol 39 ◽  
Author(s):  
Mary C. Potter
Keyword(s):  
Working Memory ◽  
Rapid Serial Visual Presentation ◽  
Language Comprehension ◽  
Short Term Memory ◽  
Visual Presentation ◽  
Long Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Use Of Knowledge

AbstractRapid serial visual presentation (RSVP) of words or pictured scenes provides evidence for a large-capacity conceptual short-term memory (CSTM) that momentarily provides rich associated material from long-term memory, permitting rapid chunking (Potter 1993; 2009; 2012). In perception of scenes as well as language comprehension, we make use of knowledge that briefly exceeds the supposed limits of working memory.

Working, Declarative, and Procedural Memory in Children With Developmental Language Disorder

2020 ◽  
Vol 63 (12) ◽  
pp. 4162-4178
Author(s):  
Emily Jackson ◽  
Suze Leitão ◽  
Mary Claessen ◽  
Mark Boyes
Keyword(s):  
Working Memory ◽  
Short Term Memory ◽  
Language Disorder ◽  
Declarative Memory ◽  
Memory Systems ◽  
Procedural Memory ◽  
Typically Developing ◽  
Short Term ◽  
Term Memory ◽  
Visual Spatial

Purpose Previous research into the working, declarative, and procedural memory systems in children with developmental language disorder (DLD) has yielded inconsistent results. The purpose of this research was to profile these memory systems in children with DLD and their typically developing peers. Method One hundred four 5- to 8-year-old children participated in the study. Fifty had DLD, and 54 were typically developing. Aspects of the working memory system (verbal short-term memory, verbal working memory, and visual–spatial short-term memory) were assessed using a nonword repetition test and subtests from the Working Memory Test Battery for Children. Verbal and visual–spatial declarative memory were measured using the Children's Memory Scale, and an audiovisual serial reaction time task was used to evaluate procedural memory. Results The children with DLD demonstrated significant impairments in verbal short-term and working memory, visual–spatial short-term memory, verbal declarative memory, and procedural memory. However, verbal declarative memory and procedural memory were no longer impaired after controlling for working memory and nonverbal IQ. Declarative memory for visual–spatial information was unimpaired. Conclusions These findings indicate that children with DLD have deficits in the working memory system. While verbal declarative memory and procedural memory also appear to be impaired, these deficits could largely be accounted for by working memory skills. The results have implications for our understanding of the cognitive processes underlying language impairment in the DLD population; however, further investigation of the relationships between the memory systems is required using tasks that measure learning over long-term intervals. Supplemental Material https://doi.org/10.23641/asha.13250180

Comparison of Auditory, Language, Memory, and Attention Abilities in Children With and Without Listening Difficulties

2020 ◽  
Vol 29 (4) ◽  
pp. 710-727
Author(s):  
Beula M. Magimairaj ◽  
Naveen K. Nagaraj ◽  
Alexander V. Sergeev ◽  
Natalie J. Benafield
Keyword(s):  
Auditory Processing ◽  
Short Term Memory ◽  
Group Differences ◽  
Long Term Memory ◽  
Short Term ◽  
Significant Group ◽  
Term Memory ◽  
Memory And Attention ◽  
Speed And Accuracy

Objectives School-age children with and without parent-reported listening difficulties (LiD) were compared on auditory processing, language, memory, and attention abilities. The objective was to extend what is known so far in the literature about children with LiD by using multiple measures and selective novel measures across the above areas. Design Twenty-six children who were reported by their parents as having LiD and 26 age-matched typically developing children completed clinical tests of auditory processing and multiple measures of language, attention, and memory. All children had normal-range pure-tone hearing thresholds bilaterally. Group differences were examined. Results In addition to significantly poorer speech-perception-in-noise scores, children with LiD had reduced speed and accuracy of word retrieval from long-term memory, poorer short-term memory, sentence recall, and inferencing ability. Statistically significant group differences were of moderate effect size; however, standard test scores of children with LiD were not clinically poor. No statistically significant group differences were observed in attention, working memory capacity, vocabulary, and nonverbal IQ. Conclusions Mild signal-to-noise ratio loss, as reflected by the group mean of children with LiD, supported the children's functional listening problems. In addition, children's relative weakness in select areas of language performance, short-term memory, and long-term memory lexical retrieval speed and accuracy added to previous research on evidence-based areas that need to be evaluated in children with LiD who almost always have heterogenous profiles. Importantly, the functional difficulties faced by children with LiD in relation to their test results indicated, to some extent, that commonly used assessments may not be adequately capturing the children's listening challenges. Supplemental Material https://doi.org/10.23641/asha.12808607

The Link Between Verbal Short-Term Memory and Anomia Treatment Gains

2019 ◽  
Vol 28 (3) ◽  
pp. 1039-1052
Author(s):  
Reva M. Zimmerman ◽  
JoAnn P. Silkes ◽  
Diane L. Kendall ◽  
Irene Minkina
Keyword(s):  
Short Term Memory ◽  
Controlled Trial ◽  
Treatment Success ◽  
Linear Regression Models ◽  
Language Performance ◽  
Short Term ◽  
Term Memory ◽  
Treatment Gains ◽  
Post Hoc

Purpose A significant relationship between verbal short-term memory (STM) and language performance in people with aphasia has been found across studies. However, very few studies have examined the predictive value of verbal STM in treatment outcomes. This study aims to determine if verbal STM can be used as a predictor of treatment success. Method Retrospective data from 25 people with aphasia in a larger randomized controlled trial of phonomotor treatment were analyzed. Digit and word spans from immediately pretreatment were run in multiple linear regression models to determine whether they predict magnitude of change from pre- to posttreatment and follow-up naming accuracy. Pretreatment, immediately posttreatment, and 3 months posttreatment digit and word span scores were compared to determine if they changed following a novel treatment approach. Results Verbal STM, as measured by digit and word spans, did not predict magnitude of change in naming accuracy from pre- to posttreatment nor from pretreatment to 3 months posttreatment. Furthermore, digit and word spans did not change from pre- to posttreatment or from pretreatment to 3 months posttreatment in the overall analysis. A post hoc analysis revealed that only the less impaired group showed significant changes in word span scores from pretreatment to 3 months posttreatment. Discussion The results suggest that digit and word spans do not predict treatment gains. In a less severe subsample of participants, digit and word span scores can change following phonomotor treatment; however, the overall results suggest that span scores may not change significantly. The implications of these findings are discussed within the broader purview of theoretical and empirical associations between aphasic language and verbal STM processing.

Sign in / Sign up

Export Citation Format

Share Document