scholarly journals Replicate, Walk, and Stop on Syntax: An Effective Neural Network Model for Aspect-Level Sentiment Classification

2020 ◽  
Vol 34 (05) ◽  
pp. 9685-9692
Author(s):  
Yaowei Zheng ◽  
Richong Zhang ◽  
Samuel Mensah ◽  
Yongyi Mao
Keyword(s):  
Neural Network ◽  
Random Walk ◽  
Network Model ◽  
Neural Network Model ◽  
Syntactic Structure ◽  
Contextual Information ◽  
Empirical Studies ◽  
Specific Aspect ◽  
Sentiment Classification ◽  
Benchmark Datasets

Aspect-level sentiment classification (ALSC) aims at predicting the sentiment polarity of a specific aspect term occurring in a sentence. This task requires learning a representation by aggregating the relevant contextual features concerning the aspect term. Existing methods cannot sufficiently leverage the syntactic structure of the sentence, and hence are difficult to distinguish different sentiments for multiple aspects in a sentence. We perceive the limitations of the previous methods and propose a hypothesis about finding crucial contextual information with the help of syntactic structure. For this purpose, we present a neural network model named RepWalk which performs a replicated random walk on a syntax graph, to effectively focus on the informative contextual words. Empirical studies show that our model outperforms recent models on most of the benchmark datasets for the ALSC task. The results suggest that our method for incorporating syntactic structure enriches the representation for the classification.

scholarly journals TopFuzz4SA: An Integrated Fuzzy Neural Network with Topic-aware Auto-encoding for Sentiment Analysis

2021 ◽  
Author(s):  
Tham Vo
Keyword(s):  
Neural Network ◽  
Fuzzy Neural Network ◽  
Semantic Information ◽  
Contextual Information ◽  
Representation Learning ◽  
Sentiment Classification ◽  
Textual Data ◽  
Fuzzy Neural ◽  
Benchmark Datasets ◽  
Textual Representation

Abstract Recent advanced deep learning architectures, such as neural seq2seq, transformer, etc. have demonstrated remarkable improvements in multi-typed sentiment classification tasks. Even though recent transformer-based and seq2seq-based models have successfully enabled to capture rich-contextual information of texts, they are still lacking of attention on incorporating the global semantic information, such as topic, in order to sufficiently leverage the performance of downstream SA task. Moreover, emotional expressions of users are normally in forms of natural human-written textual data which might consist a lot of noise and ambiguity which impose great challenges on the processes of textual representation learning as well as sentiment polarity prediction. To meet these challenges, we propose a novel integrated fuzzy-neural architecture with a topic-driven textual representation learning approach for handling SA task, called as: TopFuzz4SA. Specifically, in the proposed TopFuzz4SA model, we first apply a topic-driven neural encoder-decoder architecture with the incorporation of latent topic embedding and attention mechanism to sufficiently learn both rich contextual and global semantic information of the given textual data. Then, the achieved rich semantic representations of texts are fed into a fused deep fuzzy neural network to effectively reduce the feature ambiguity and noise, forming the final textual representations for sentiment classification task. Extensive experiments in benchmark datasets demonstrate the effectiveness of our proposed TopFuzz4SA model in comparing with contemporary state-of-the-art baselines.

scholarly journals Integrated neural network model with pre-RBF kernels

2021 ◽  
Vol 104 (3) ◽  
pp. 003685042110261
Author(s):  
Hui Wen ◽  
Tao Yan ◽  
Zhiqiang Liu ◽  
Deli Chen
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Performance ◽  
Back Propagation ◽  
Nonlinear Problems ◽  
Bp Network ◽  
Rbf Network ◽  
Bp Networks ◽  
Benchmark Datasets

To improve the network performance of radial basis function (RBF) and back-propagation (BP) networks on complex nonlinear problems, an integrated neural network model with pre-RBF kernels is proposed. The proposed method is based on the framework of a single optimized BP network and an RBF network. By integrating and connecting the RBF kernel mapping layer and BP neural network, the local features of a sample set can be effectively extracted to improve separability; subsequently, the connected BP network can be used to perform learning and classification in the kernel space. Experiments on an artificial dataset and three benchmark datasets show that the proposed model combines the advantages of RBF and BP networks, as well as improves the performances of the two networks. Finally, the effectiveness of the proposed method is verified.

scholarly journals An autoencoder-based neural network model for selectional preference: evidence from pseudo-disambiguation and cloze tasks

2017 ◽  
Vol 8 (2) ◽  
pp. 93-125
Author(s):  
Aki-Juhani Kyröläinen ◽  
Juhani Luotolahti ◽  
Filip Ginter
Keyword(s):  
Neural Network ◽  
Semantic Similarity ◽  
Network Model ◽  
Neural Network Model ◽  
Semantic Information ◽  
Contextual Information ◽  
Neural Model ◽  
The Other ◽  
Excellent Performance

Intuitively, some predicates have a better fit with certain arguments than others. Usage-based models of language emphasize the importance of semantic similarity in shaping the structuring of constructions (form and meaning). In this study, we focus on modeling the semantics of transitive constructions in Finnish and present an autoencoder-based neural network model trained on semantic vectors based on Word2vec. This model builds on the distributional hypothesis according to which semantic information is primarily shaped by contextual information. Specifically, we focus on the realization of the object. The performance of the model is evaluated in two tasks: a pseudo-disambiguation and a cloze task. Additionally, we contrast the performance of the autoencoder with a previously implemented neural model. In general, the results show that our model achieves an excellent performance on these tasks in comparison to the other models. The results are discussed in terms of usage-based construction grammar.Kokkuvõte. Aki-Juhani Kyröläinen, M. Juhani Luotolahti ja Filip Ginter: Autokoodril põhinev närvivõrkude mudel valikulisel eelistamisel. Intuitiivselt tundub, et mõned argumendid sobivad teatud predikaatidega paremini kokku kui teised. Kasutuspõhised keelemudelid rõhutavad konstruktsioonide struktuuri (nii vormi kui tähenduse) kujunemisel tähendusliku sarnasuse olulisust. Selles uurimuses modelleerime soome keele transitiivsete konstruktsioonide semantikat ja esitame närvivõrkude mudeli ehk autokoodri. Mudel põhineb distributiivse semantika hüpoteesil, mille järgi kujuneb semantiline info peamiselt konteksti põhjal. Täpsemalt keskendume uurimuses objektile. Mudelit hindame nii valeühestamise kui ka lünkülesande abil. Kõrvutame autokoodri tulemusi varem välja töötatud neurovõrgumudelitega ja tõestame, et meie mudel töötab võrreldes teiste mudelitega väga hästi. Tulemused esitame kasutuspõhise konstruktsioonigrammatika kontekstis.Võtmesõnad: neurovõrk; autokooder; tähendusvektor; kasutuspõhine mudel; soome keel

scholarly journals Implicit semantic prediction error can account for N400 effects on articles that do not differ in meaning: A neural network model

2019 ◽  
Author(s):  
Milena Rabovsky
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Prediction Error ◽  
Sentence Comprehension ◽  
Empirical Studies ◽  
Probabilistic Representation ◽  
Cue Validity ◽  
Phonological Word ◽  
Word Forms

AbstractN400 effects on indefinite articles (a/an) compatible or incompatible with expected nouns have been initially taken as strong evidence for probabilistic pre-activation of phonological word forms, and recently been intensely debated because they have been difficult to replicate. Here, we simulate these effects using a neural network model of sentence comprehension that we previously used to simulate a broad range of empirical N400 effects. The model produces the effects when the cue validity of the articles concerning upcoming noun meaning is high, but fails to produce the effects when the cue validity of the articles is low due to adjectives presented between articles and nouns during training, providing a possible explanation for the small size of the effects in empirical studies. The model accounts for article induced N400 effects without assuming pre-activation of word forms, and instead simulates these effects as the stimulus-induced change in a probabilistic representation of meaning corresponding to an implicit semantic prediction error.

scholarly journals Encoding Longer-Term Contextual Information with Predictive Coding and Ego-Motion

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-15
Author(s):  
Junpei Zhong ◽  
Angelo Cangelosi ◽  
Tetsuya Ogata ◽  
Xinzheng Zhang
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Contextual Information ◽  
Predictive Coding ◽  
Sensory Information ◽  
Hierarchical Architecture ◽  
Motor Action ◽  
Neural Activities ◽  
Different Levels

Studies suggest that, within the hierarchical architecture, the topological higher level possibly represents the scenarios of the current sensory events with slower changing activities. They attempt to predict the neural activities on the lower level by relaying the predicted information after the scenario of the sensorimotor event has been determined. On the other hand, the incoming sensory information corrects such prediction of the events on the higher level by the fast-changing novel or surprising signal. From this point, we propose a predictive hierarchical artificial neural network model that examines this hypothesis on neurorobotic platforms. It integrates the perception and action in the predictive coding framework. Moreover, in this neural network model, there are different temporal scales of predictions existing on different levels of the hierarchical predictive coding architecture, which defines the temporal memories in recording the events occurring. Also, both the fast- and the slow-changing neural activities are modulated by the motor action. Therefore, the slow-changing neurons can be regarded as the representation of the recent scenario which the sensorimotor system has encountered. The neurorobotic experiments based on the architecture were also conducted.

Sign in / Sign up

Export Citation Format

Share Document