scholarly journals Dynamic Compositionality in Recursive Neural Networks with Structure-Aware Tag Representations

2019 ◽  
Vol 33 ◽  
pp. 6594-6601 ◽  
Author(s):  
Taeuk Kim ◽  
Jihun Choi ◽  
Daniel Edmiston ◽  
Sanghwan Bae ◽  
Sang-goo Lee
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Neural Models ◽  
Sentence Level ◽  
Recursive Neural Networks ◽  
Syntactic Information ◽  
Parse Trees ◽  
By Products

Most existing recursive neural network (RvNN) architectures utilize only the structure of parse trees, ignoring syntactic tags which are provided as by-products of parsing. We present a novel RvNN architecture that can provide dynamic compositionality by considering comprehensive syntactic information derived from both the structure and linguistic tags. Specifically, we introduce a structure-aware tag representation constructed by a separate tag-level tree-LSTM. With this, we can control the composition function of the existing wordlevel tree-LSTM by augmenting the representation as a supplementary input to the gate functions of the tree-LSTM. In extensive experiments, we show that models built upon the proposed architecture obtain superior or competitive performance on several sentence-level tasks such as sentiment analysis and natural language inference when compared against previous tree-structured models and other sophisticated neural models.

scholarly journals Pat-in-the-Loop: Declarative Knowledge for Controlling Neural Networks

2020 ◽  
Vol 12 (12) ◽  
pp. 218
Author(s):  
Dario Onorati ◽  
Pierfrancesco Tommasino ◽  
Leonardo Ranaldi ◽  
Francesca Fallucchi ◽  
Fabio Massimo Zanzotto
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Cost Effective ◽  
Specific Class ◽  
Human Control ◽  
The Neural Network ◽  
Parse Trees

The dazzling success of neural networks over natural language processing systems is imposing an urgent need to control their behavior with simpler, more direct declarative rules. In this paper, we propose Pat-in-the-Loop as a model to control a specific class of syntax-oriented neural networks by adding declarative rules. In Pat-in-the-Loop, distributed tree encoders allow to exploit parse trees in neural networks, heat parse trees visualize activation of parse trees, and parse subtrees are used as declarative rules in the neural network. Hence, Pat-in-the-Loop is a model to include human control in specific natural language processing (NLP)-neural network (NN) systems that exploit syntactic information, which we will generically call Pat. A pilot study on question classification showed that declarative rules representing human knowledge, injected by Pat, can be effectively used in these neural networks to ensure correctness, relevance, and cost-effective.

Prediction of Emergency Department Hospital Admission Based on Natural Language Processing and Neural Networks

2017 ◽  
Vol 56 (05) ◽  
pp. 377-389 ◽  
Author(s):  
Xingyu Zhang ◽  
Joyce Kim ◽  
Rachel E. Patzer ◽  
Stephen R. Pitts ◽  
Aaron Patzer ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Emergency Department ◽  
Logistic Regression ◽  
Natural Language Processing ◽  
Natural Language ◽  
Hospital Admission ◽  
Language Processing ◽  
Predictive Accuracy ◽  
Free Text

SummaryObjective: To describe and compare logistic regression and neural network modeling strategies to predict hospital admission or transfer following initial presentation to Emergency Department (ED) triage with and without the addition of natural language processing elements.Methods: Using data from the National Hospital Ambulatory Medical Care Survey (NHAMCS), a cross-sectional probability sample of United States EDs from 2012 and 2013 survey years, we developed several predictive models with the outcome being admission to the hospital or transfer vs. discharge home. We included patient characteristics immediately available after the patient has presented to the ED and undergone a triage process. We used this information to construct logistic regression (LR) and multilayer neural network models (MLNN) which included natural language processing (NLP) and principal component analysis from the patient’s reason for visit. Ten-fold cross validation was used to test the predictive capacity of each model and receiver operating curves (AUC) were then calculated for each model.Results: Of the 47,200 ED visits from 642 hospitals, 6,335 (13.42%) resulted in hospital admission (or transfer). A total of 48 principal components were extracted by NLP from the reason for visit fields, which explained 75% of the overall variance for hospitalization. In the model including only structured variables, the AUC was 0.824 (95% CI 0.818-0.830) for logistic regression and 0.823 (95% CI 0.817-0.829) for MLNN. Models including only free-text information generated AUC of 0.742 (95% CI 0.7310.753) for logistic regression and 0.753 (95% CI 0.742-0.764) for MLNN. When both structured variables and free text variables were included, the AUC reached 0.846 (95% CI 0.839-0.853) for logistic regression and 0.844 (95% CI 0.836-0.852) for MLNN.Conclusions: The predictive accuracy of hospital admission or transfer for patients who presented to ED triage overall was good, and was improved with the inclusion of free text data from a patient’s reason for visit regardless of modeling approach. Natural language processing and neural networks that incorporate patient-reported outcome free text may increase predictive accuracy for hospital admission.

scholarly journals Aspect-level sentiment analysis merged with knowledge graph and graph convolutional neural network

2021 ◽  
Vol 2083 (4) ◽  
pp. 042044
Author(s):  
Zuhua Dai ◽  
Yuanyuan Liu ◽  
Shilong Di ◽  
Qi Fan
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Structural Information ◽  
Knowledge Graph ◽  
Convolutional Network ◽  
Text Data ◽  
Short Text ◽  
Fine Grained ◽  
Syntactic Information ◽  
Text Information

Abstract Aspect level sentiment analysis belongs to fine-grained sentiment analysis, w hich has caused extensive research in academic circles in recent years. For this task, th e recurrent neural network (RNN) model is usually used for feature extraction, but the model cannot effectively obtain the structural information of the text. Recent studies h ave begun to use the graph convolutional network (GCN) to model the syntactic depen dency tree of the text to solve this problem. For short text data, the text information is not enough to accurately determine the emotional polarity of the aspect words, and the knowledge graph is not effectively used as external knowledge that can enrich the sem antic information. In order to solve the above problems, this paper proposes a graph co nvolutional neural network (GCN) model that can process syntactic information, know ledge graphs and text semantic information. The model works on the “syntax-knowled ge” graph to extract syntactic information and common sense information at the same t ime. Compared with the latest model, the model in this paper can effectively improve t he accuracy of aspect-level sentiment classification on two datasets.

Sarcasm Detection Using Multi-Channel Attention Based BLSTM On News Headline

2020 ◽  
Author(s):  
Azika Syahputra Azwar ◽  
Suharjito
Keyword(s):  
Neural Network ◽  
Social Media ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Important Task ◽  
Hybrid Neural Network ◽  
Negative Opinion ◽  
Proposed Model ◽  
Short Memory ◽  
Literary Devices

Abstract Sarcasm is often used to express a negative opinion using positive or intensified positive words in social media. This intentional ambiguity makes sarcasm detection, an important task of sentiment analysis. Detecting a sarcastic tone in natural language hinders the performance of sentiment analysis tasks. The majority of the studies on automatic sarcasm detection emphasize on the use of lexical, syntactic, or pragmatic features that are often unequivocally expressed through figurative literary devices such as words, emoticons, and exclamation marks. In this paper, we introduce a multi-channel attention-based bidirectional long-short memory (MCAB-BLSTM) network to detect sarcastic headline on the news. Multi-channel attention-based bidirectional long-short memory (MCAB-BLSTM) proposed model was evaluated on the news headline dataset, and the results-compared to the CNN-LSTM and Hybrid Neural Network were excellent.

scholarly journals Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop

2019 ◽  
Vol 25 (4) ◽  
pp. 543-557 ◽  
Author(s):  
Afra Alishahi ◽  
Grzegorz Chrupała ◽  
Tal Linzen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Natural Language Processing ◽  
Language Processing ◽  
Performance Testing ◽  
Network Architectures ◽  
Empirical Methods ◽  
Neural Models ◽  
The Impact ◽  
Systematic Manipulation

AbstractThe Empirical Methods in Natural Language Processing (EMNLP) 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired by neural models of language. Approaches included: systematic manipulation of input to neural networks and investigating the impact on their performance, testing whether interpretable knowledge can be decoded from intermediate representations acquired by neural networks, proposing modifications to neural network architectures to make their knowledge state or generated output more explainable, and examining the performance of networks on simplified or formal languages. Here we review a number of representative studies in each category.

scholarly journals Development and Performance Evaluation of a Computer Program Based on Neural Network Mathematical Models for Forecasting By- Product Yield

2020 ◽  
Vol 174 ◽  
pp. 03023
Author(s):  
Yelena Vasileva ◽  
Aleksandr Nevedrov ◽  
Sergey Subbotin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Mathematical Models ◽  
Design Method ◽  
Coal Tar ◽  
Product Yield ◽  
Production Values ◽  
And Performance ◽  
Crude Benzene ◽  
By Products

Process performance of coking plants are based on data on the yield of by-products of coking coal and their quality, therefore, much attention is paid to the issues of their analysis. In view of the complexity and insufficient knowledge of the relationship between these parameters, mathematical modeling of this dependence using neural networks is of great interest. Based on a mathematical analysis of experimental data on the quality indicators of coal, coal concentrates and the by-product yield, neural network mathematical models have been developed to forecast the parameters under study. The neural network is based on the Ward’s network. Based on the results of the research, the application “Intelligent Information System for Forecasting By-product Yield” was created, which implements neural networks [1]. The relative forecasting error for the parameter “coke” is 0.64±0.23%, “coal tar” is 19.53±5.25%, “crude benzene” is 10.02±2.83%, and “coke gas” is 5.11±1.34%. A comparative analysis of the data obtained using the developed design method is carried out, with the simulation results using existing methods, as well as with the production values of by-products yield.

Efficiency testing of artificial neural networks in predicting the properties of carbon nanomaterials as potential systems for nervous tissue stimulation and regeneration

2017 ◽  
Vol 13 (1) ◽  
Author(s):  
Martyna Sasiada ◽  
Aneta Fraczek-Szczypta ◽  
Ryszard Tadeusiewicz
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Nervous Tissue ◽  
Carbon Nanomaterials ◽  
Real Life ◽  
Neural Model ◽  
Carbon Nanomaterial ◽  
Neural Models ◽  
Predictive Learning ◽  
The Many

AbstractA new method of predicting the properties of carbon nanomaterials from carbon nanotubes and graphene oxide, using electrophoretic deposition (EPD) on a metal surface, was investigated. The main goal is to obtain the basis for nervous tissue stimulation and regeneration. Because of the many variations of the EPD method, costly and time-consuming experiments are necessary for optimization of the produced systems. To limit such costs and workload, we propose a neural network-based model that can predict the properties of selected carbon nanomaterial systems before they are produced. The choice of neural networks as predictive learning models is based on many studies in the literature that report neural models as good interpretations of real-life processes. The use of a neural network model can reduce experimentation with unpromising methods of systems processing and preparation. Instead, it allows a focus on experiments with these systems, which are promising according to the prediction given by the neural model. The performed tests showed that the proposed method of predictive learning of carbon nanomaterial properties is easy and effective. The experiments showed that the prediction results were consistent with those obtained in the real system.

Synergistic Neural Models of a Robot Sensor for Part Orientation Detection

1996 ◽  
Vol 210 (1) ◽  
pp. 69-76 ◽  
Author(s):  
D T Pham ◽  
S Sagiroglu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Inertial Sensor ◽  
Neural Models ◽  
Combined System ◽  
Number Of Components ◽  
Inertial Parameters ◽  
Orientation Detection ◽  
Part Orientation

This paper describes the use of neural networks to compute the orientation of a part from the output signals of an inertial sensor which is a device for determining the location of parts by measuring their inertial parameters. The paper investigates an approach for increasing the accuracy of the computed orientation. This involves employing a group of neural networks and combining their outputs. The paper presents the results obtained for several neural network combinations. These show that the accuracy achieved in a combined system is higher than that of its individual components provided the number of components is not too large.

Sign in / Sign up

Export Citation Format

Share Document