scholarly journals Bigger Isn’t Better: The Ethical and Scientific Vices of Extra-Large Datasets in Language Models

2021 ◽  
Author(s):  
Trystan S. Goetze ◽  
Darren Abramson
Keyword(s):  
Large Datasets ◽  
Language Models

scholarly journals Transformer-based deep neural network language models for Alzheimer’s disease risk assessment from targeted speech

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Alireza Roshanzamir ◽  
Hamid Aghajan ◽  
Mahdieh Soleymani Baghshah
Keyword(s):  
Alzheimer’S Disease ◽  
Risk Assessment ◽  
Alzheimer's Disease ◽  
Natural Language ◽  
Language Processing ◽  
Disease Risk ◽  
Large Datasets ◽  
Language Models ◽  
Classification Model ◽  
Picture Description

Abstract Background We developed transformer-based deep learning models based on natural language processing for early risk assessment of Alzheimer’s disease from the picture description test. Methods The lack of large datasets poses the most important limitation for using complex models that do not require feature engineering. Transformer-based pre-trained deep language models have recently made a large leap in NLP research and application. These models are pre-trained on available large datasets to understand natural language texts appropriately, and are shown to subsequently perform well on classification tasks with small training sets. The overall classification model is a simple classifier on top of the pre-trained deep language model. Results The models are evaluated on picture description test transcripts of the Pitt corpus, which contains data of 170 AD patients with 257 interviews and 99 healthy controls with 243 interviews. The large bidirectional encoder representations from transformers (BERTLarge) embedding with logistic regression classifier achieves classification accuracy of 88.08%, which improves the state-of-the-art by 2.48%. Conclusions Using pre-trained language models can improve AD prediction. This not only solves the problem of lack of sufficiently large datasets, but also reduces the need for expert-defined features.

scholarly journals Transformer-based deep neural network language models for Alzheimer’s disease detection from targeted speech

2020 ◽  
Author(s):  
Alireza Roshanzamir ◽  
Hamid Aghajan ◽  
Mahdieh Soleymani Baghshah
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Natural Language ◽  
Language Processing ◽  
Language Model ◽  
Large Datasets ◽  
Language Models ◽  
Classification Model ◽  
Picture Description ◽  
Network Language

Abstract Background: We developed transformer-based deep learning models based on natural language processing for early diagnosis of Alzheimer’s disease from the picture description test.Methods: The lack of large datasets poses the most important limitation for using complex models that do not require feature engineering. Transformer-based pre-trained deep language models have recently made a large leap in NLP research and application. These models are pre-trained on available large datasets to understand natural language texts appropriately, and are shown to subsequently perform well on classification tasks with small training sets. The overall classification model is a simple classifier on top of the pre-trained deep language model.Results: The models are evaluated on picture description test transcripts of the Pitt corpus, which contains data of 170 AD patients with 257 interviews and 99 healthy controls with 243 interviews. The large bidirectional encoder representations from transformers (BERTLarge) embedding with logistic regression classifier achieves classification accuracy of 88.08%, which improves thestate-of-the-art by 2.48%.Conclusions: Using pre-trained language models can improve AD prediction. This not only solves the problem of lack of sufficiently large datasets, but also reduces the need for expert-defined features.

scholarly journals Transformer-based deep neural network language models for Alzheimer's disease risk assessment from targeted speech

2021 ◽  
Author(s):  
Alireza Roshanzamir ◽  
Hamid Aghajan ◽  
Mahdieh Soleymani Baghshah
Keyword(s):  
Alzheimer’S Disease ◽  
Risk Assessment ◽  
Alzheimer's Disease ◽  
Natural Language ◽  
Language Processing ◽  
Disease Risk ◽  
Large Datasets ◽  
Language Models ◽  
Classification Model ◽  
Picture Description

Abstract Background: We developed transformer-based deep learning models based on natural language processing for early risk assessment of Alzheimer’s disease from the picture description test.Methods: The lack of large datasets poses the most important limitation for using complex models that do not require feature engineering. Transformer-based pre-trained deep language models have recently made a large leap in NLP research and application. These models are pre-trained on available large datasets to understand natural language texts appropriately, and are shown to subsequently perform well on classification tasks with small training sets. The overall classification model is a simple classifier on top of the pre-trained deep language model.Results: The models are evaluated on picture description test transcripts of the Pitt corpus, which contains data of 170 AD patients with 257 interviews and 99 healthy controls with 243 interviews. The large bidirectional encoder representations from transformers (BERTLarge) embedding with logistic regression classifier achieves classification accuracy of 88.08%, which improves the state-of-the-art by 2.48%.Conclusions: Using pre-trained language models can improve AD prediction. This not only solves the problem of lack of sufficiently large datasets, but also reduces the need for expert-defined features.

scholarly journals Transformer-Based Deep Neural Network Language Models for Alzheimer's Disease Detection from Targeted Speech

2020 ◽  
Author(s):  
Alireza Roshanzamir ◽  
Hamid Aghajan ◽  
Mahdieh Soleymani Baghshah
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Natural Language ◽  
Language Processing ◽  
Large Datasets ◽  
Language Models ◽  
Classification Model ◽  
Complex Models ◽  
Picture Description ◽  
Network Language

Abstract Background: We developed transformer-based deep learning models based on natural language processing for early diagnosis of Alzheimer’s disease from the picture description test.Methods: The lack of large datasets poses the most important limitation for using complex models that do not require feature engineering. Transformer-based pre-trained deep language models have recently made a large leap in NLP research and application. These models are pre-trained on available large datasets to understand natural language texts appropriately, and are shown to subsequently perform well on classification tasks with small training sets. The overall classification model is a simple classifier on top of the pre-trained deep language model.Results: The models are evaluated on picture description test transcripts of the Pitt corpus, which contains data of 170 AD patients with 257 interviews and 99 healthy controls with 243 interviews. The large bidirectional encoder representations from transformers (BERTLarge) embedding with logistic regression classifier achieves classification accuracy of 88.08%, which improves the state-of-the-art by 2.48%.Conclusions: Using pre-trained language models can improve AD prediction. This not only solves the problem of lack of sufficiently large datasets, but also reduces the need for expert-defined features.

THE BALANCE OF PROBABILITY: STATISTICS AND THE DIACHRONIC STUDY OF ANCIENT HEBREW

2017 ◽  
Vol 25 (2) ◽  
pp. 927-960
Author(s):  
Jarod Jacobs
Keyword(s):  
Dead Sea Scrolls ◽  
Dead Sea ◽  
Biblical Hebrew ◽  
Large Datasets ◽  
Dual Function ◽  
Statistical Tools ◽  
Masoretic Text ◽  
Linguistic Research ◽  
Recent Debate ◽  
Diachronic Study

In this article, I discuss three statistical tools that have proven pivotal in linguistic research, particularly those studies that seek to evaluate large datasets. These tools are the Gaussian Curve, significance tests, and hierarchical clustering. I present a brief description of these tools and their general uses. Then, I apply them to an analysis of the variations between the “biblical” DSS and our other witnesses, focusing upon variations involving particles. Finally, I engage the recent debate surrounding the diachronic study of Biblical Hebrew. This article serves a dual function. First, it presents statistical tools that are useful for many linguistic studies. Second, it develops an analysis of the he-locale, as it is used in the “biblical” Dead Sea Scrolls, Masoretic Text, and Samaritan Pentateuch. Through that analysis, this article highlights the value of inferential statistical tools as we attempt to better understand the Hebrew of our ancient witnesses.

mmpdb: An Open Source Matched Molecular Pair Platform for Large Multi-Property Datasets

2018 ◽  
Author(s):  
Andrew Dalke ◽  
Jerome Hert ◽  
Christian Kramer
Keyword(s):  
Open Source ◽  
Large Datasets ◽  
Molecular Pair ◽  
New Algorithms

We present mmpdb, an open source Matched Molecular Pair (MMP) platform to create, compile, store, retrieve, and use MMP rules. mmpdb is suitable for the large datasets typically found in pharmaceutical and agrochemical companies and provides new algorithms for fragment canonicalization and stereochemistry handling. The platform is written in Python and based on the RDKit toolkit. mmpdb is freely available.

Adolescent Language: Models, Assessment, and Links to Reading

2019 ◽  
Author(s):  
Amanda Goodwin ◽  
Yaacov Petscher ◽  
Jamie Tock
Keyword(s):  
Reading Comprehension ◽  
Bifactor Model ◽  
Language Models ◽  
Multiple Group ◽  
Global Factor ◽  
Eighth Grade Students ◽  
Key Aspects ◽  
Future Work ◽  
The Relationship ◽  
Best Fit

Various models have highlighted the complexity of language. Building on foundational ideas regarding three key aspects of language, our study contributes to the literature by 1) exploring broader conceptions of morphology, vocabulary, and syntax, 2) operationalizing this theoretical model into a gamified, standardized, computer-adaptive assessment of language for fifth to eighth grade students entitled Monster, PI, and 3) uncovering further evidence regarding the relationship between language and standardized reading comprehension via this assessment. Multiple-group item response theory (IRT) across grades show that morphology was best fit by a bifactor model of task specific factors along with a global factor related to each skill. Vocabulary was best fit by a bifactor model that identifies performance overall and on specific words. Syntax, though, was best fit by a unidimensional model. Next, Monster, PI produced reliable scores suggesting language can be assessed efficiently and precisely for students via this model. Lastly, performance on Monster, PI explained more than 50% of variance in standardized reading, suggesting operationalizing language via Monster, PI can provide meaningful understandings of the relationship between language and reading comprehension. Specifically, considering just a subset of a construct, like identification of units of meaning, explained significantly less variance in reading comprehension. This highlights the importance of considering these broader constructs. Implications indicate that future work should consider a model of language where component areas are considered broadly and contributions to reading comprehension are explored via general performance on components as well as skill level performance.

Fast Learning of Generalized Minimum Enclosing Ball for Large Datasets

2012 ◽  
Vol 38 (11) ◽  
pp. 1831
Author(s):  
Wen-Jun HU ◽  
Shi-Tong WANG ◽  
Juan WANG ◽  
Wen-Hao YING
Keyword(s):  
Large Datasets ◽  
Fast Learning ◽  
Minimum Enclosing Ball
Sign in / Sign up

Export Citation Format

Share Document