Word Recognition as a First Step Towards Natural Language Processing with Artificial Neural Networks

1990 ◽  
pp. 221-225 ◽  
Author(s):  
Renate Deffner ◽  
Klaus Eder ◽  
Hans Geiger
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Natural Language Processing ◽  
Word Recognition ◽  
Natural Language ◽  
Language Processing ◽  
Artificial Neural

scholarly journals Linguistic generalization and compositionality in modern artificial neural networks

2019 ◽  
Vol 375 (1791) ◽  
pp. 20190307 ◽  
Author(s):  
Marco Baroni
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Natural Language ◽  
Language Processing ◽  
Mechanistic Models ◽  
Theme Issue ◽  
High Productivity ◽  
Deep Networks ◽  
Artificial Neural ◽  
New Perspective

In the last decade, deep artificial neural networks have achieved astounding performance in many natural language-processing tasks. Given the high productivity of language, these models must possess effective generalization abilities. It is widely assumed that humans handle linguistic productivity by means of algebraic compositional rules: are deep networks similarly compositional? After reviewing the main innovations characterizing current deep language-processing networks, I discuss a set of studies suggesting that deep networks are capable of subtle grammar-dependent generalizations, but also that they do not rely on systematic compositional rules. I argue that the intriguing behaviour of these devices (still awaiting a full understanding) should be of interest to linguists and cognitive scientists, as it offers a new perspective on possible computational strategies to deal with linguistic productivity beyond rule-based compositionality, and it might lead to new insights into the less systematic generalization patterns that also appear in natural language. This article is part of the theme issue ‘Towards mechanistic models of meaning composition’.

scholarly journals A Survey on Bias in Deep NLP

2021 ◽  
Vol 11 (7) ◽  
pp. 3184
Author(s):  
Ismael Garrido-Muñoz  ◽  
Arturo Montejo-Ráez  ◽  
Fernando Martínez-Santiago  ◽  
L. Alfonso Ureña-López 
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Natural Language Processing ◽  
Probability Distribution ◽  
Natural Language ◽  
Network Design ◽  
Language Processing ◽  
Deep Neural Networks ◽  
Learning Processes ◽  
Relevant Issue

Deep neural networks are hegemonic approaches to many machine learning areas, including natural language processing (NLP). Thanks to the availability of large corpora collections and the capability of deep architectures to shape internal language mechanisms in self-supervised learning processes (also known as “pre-training”), versatile and performing models are released continuously for every new network design. These networks, somehow, learn a probability distribution of words and relations across the training collection used, inheriting the potential flaws, inconsistencies and biases contained in such a collection. As pre-trained models have been found to be very useful approaches to transfer learning, dealing with bias has become a relevant issue in this new scenario. We introduce bias in a formal way and explore how it has been treated in several networks, in terms of detection and correction. In addition, available resources are identified and a strategy to deal with bias in deep NLP is proposed.

Prediction of Emergency Department Hospital Admission Based on Natural Language Processing and Neural Networks

2017 ◽  
Vol 56 (05) ◽  
pp. 377-389 ◽  
Author(s):  
Xingyu Zhang ◽  
Joyce Kim ◽  
Rachel E. Patzer ◽  
Stephen R. Pitts ◽  
Aaron Patzer ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Emergency Department ◽  
Logistic Regression ◽  
Natural Language Processing ◽  
Natural Language ◽  
Hospital Admission ◽  
Language Processing ◽  
Predictive Accuracy ◽  
Free Text

SummaryObjective: To describe and compare logistic regression and neural network modeling strategies to predict hospital admission or transfer following initial presentation to Emergency Department (ED) triage with and without the addition of natural language processing elements.Methods: Using data from the National Hospital Ambulatory Medical Care Survey (NHAMCS), a cross-sectional probability sample of United States EDs from 2012 and 2013 survey years, we developed several predictive models with the outcome being admission to the hospital or transfer vs. discharge home. We included patient characteristics immediately available after the patient has presented to the ED and undergone a triage process. We used this information to construct logistic regression (LR) and multilayer neural network models (MLNN) which included natural language processing (NLP) and principal component analysis from the patient’s reason for visit. Ten-fold cross validation was used to test the predictive capacity of each model and receiver operating curves (AUC) were then calculated for each model.Results: Of the 47,200 ED visits from 642 hospitals, 6,335 (13.42%) resulted in hospital admission (or transfer). A total of 48 principal components were extracted by NLP from the reason for visit fields, which explained 75% of the overall variance for hospitalization. In the model including only structured variables, the AUC was 0.824 (95% CI 0.818-0.830) for logistic regression and 0.823 (95% CI 0.817-0.829) for MLNN. Models including only free-text information generated AUC of 0.742 (95% CI 0.7310.753) for logistic regression and 0.753 (95% CI 0.742-0.764) for MLNN. When both structured variables and free text variables were included, the AUC reached 0.846 (95% CI 0.839-0.853) for logistic regression and 0.844 (95% CI 0.836-0.852) for MLNN.Conclusions: The predictive accuracy of hospital admission or transfer for patients who presented to ED triage overall was good, and was improved with the inclusion of free text data from a patient’s reason for visit regardless of modeling approach. Natural language processing and neural networks that incorporate patient-reported outcome free text may increase predictive accuracy for hospital admission.

Sign in / Sign up

Export Citation Format

Share Document