scholarly journals Syntactically Meaningful and Transferable Recursive Neural Networks for Aspect and Opinion Extraction

2020 ◽  
Vol 45 (4) ◽  
pp. 705-736
Author(s):  
Wenya Wang ◽  
Sinno Jialin Pan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Syntactic Structure ◽  
Feature Learning ◽  
Single Domain ◽  
Fine Grained ◽  
Adversarial Network ◽  
Dependency Tree ◽  
Recursive Neural Networks ◽  
Syntactic Relations

In fine-grained opinion mining, extracting aspect terms (a.k.a. opinion targets) and opinion terms (a.k.a. opinion expressions) from user-generated texts is the most fundamental task in order to generate structured opinion summarization. Existing studies have shown that the syntactic relations between aspect and opinion words play an important role for aspect and opinion terms extraction. However, most of the works either relied on predefined rules or separated relation mining with feature learning. Moreover, these works only focused on single-domain extraction, which failed to adapt well to other domains of interest where only unlabeled data are available. In real-world scenarios, annotated resources are extremely scarce for many domains, motivating knowledge transfer strategies from labeled source domain(s) to any unlabeled target domain. We observe that syntactic relations among target words to be extracted are not only crucial for single-domain extraction, but also serve as invariant “pivot” information to bridge the gap between different domains. In this article, we explore the constructions of recursive neural networks based on the dependency tree of each sentence for associating syntactic structure with feature learning. Furthermore, we construct transferable recursive neural networks to automatically learn the domain-invariant fine-grained interactions among aspect words and opinion words. The transferability is built on an auxiliary task and a conditional domain adversarial network to reduce domain distribution difference in the hidden spaces effectively in word level through syntactic relations. Specifically, the auxiliary task builds structural correspondences across domains by predicting the dependency relation for each path of the dependency tree in the recursive neural network. The conditional domain adversarial network helps to learn domain-invariant hidden representation for each word conditioned on the syntactic structure. In the end, we integrate the recursive neural network with a sequence labeling classifier on top that models contextual influence in the final predictions. Extensive experiments and analysis are conducted to demonstrate the effectiveness of the proposed model and each component on three benchmark data sets.

Chapter 4. Graph Reasoning Networks and Applications

2021 ◽  
Author(s):  
Qingxing Cao ◽  
Wentao Wan ◽  
Xiaodan Liang ◽  
Liang Lin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Domain Knowledge ◽  
Feature Learning ◽  
External Information ◽  
Feature Representations ◽  
Domain Specific ◽  
The Neural Network ◽  
Structured Knowledge ◽  
Knowledge Graphs

Despite the significant success in various domains, the data-driven deep neural networks compromise the feature interpretability, lack the global reasoning capability, and can’t incorporate external information crucial for complicated real-world tasks. Since the structured knowledge can provide rich cues to record human observations and commonsense, it is thus desirable to bridge symbolic semantics with learned local feature representations. In this chapter, we review works that incorporate different domain knowledge into the intermediate feature representation.These methods firstly construct a domain-specific graph that represents related human knowledge. Then, they characterize node representations with neural network features and perform graph convolution to enhance these symbolic nodes via the graph neural network(GNN).Lastly, they map the enhanced node feature back into the neural network for further propagation or prediction. Through integrating knowledge graphs into neural networks, one can collaborate feature learning and graph reasoning with the same supervised loss function and achieve a more effective and interpretable way to introduce structure constraints.

scholarly journals Template-Based Math Word Problem Solvers with Recursive Neural Networks

2019 ◽  
Vol 33 ◽  
pp. 7144-7151 ◽  
Author(s):  
Lei Wang ◽  
Dongxiang Zhang ◽  
Jipeng Zhang ◽  
Xing Xu ◽  
Lianli Gao ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Word Problem ◽  
Large Space ◽  
Wide Margin ◽  
Recursive Neural Networks ◽  
Recursive Neural Network ◽  
Operator Nodes ◽  
Problem Solvers ◽  
New Framework

The design of automatic solvers to arithmetic math word problems has attracted considerable attention in recent years and a large number of datasets and methods have been published. Among them, Math23K is the largest data corpus that is very helpful to evaluate the generality and robustness of a proposed solution. The best performer in Math23K is a seq2seq model based on LSTM to generate the math expression. However, the model suffers from performance degradation in large space of target expressions. In this paper, we propose a template-based solution based on recursive neural network for math expression construction. More specifically, we first apply a seq2seq model to predict a tree-structure template, with inferred numbers as leaf nodes and unknown operators as inner nodes. Then, we design a recursive neural network to encode the quantity with Bi-LSTM and self attention, and infer the unknown operator nodes in a bottom-up manner. The experimental results clearly establish the superiority of our new framework as we improve the accuracy by a wide margin in two of the largest datasets, i.e., from 58.1% to 66.9% in Math23K and from 62.8% to 66.8% in MAWPS.

scholarly journals Predicting Customer Turnover Using Recursive Neural Networks

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Abdullah Jafari Chashmi ◽  
Vahid Rahmati ◽  
Behrouz Rezasoroush ◽  
Masumeh Motevalli Alamoti ◽  
Mohsen Askari ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Algorithm ◽  
Customer Relationship ◽  
Loyalty Programs ◽  
Recurrent Nerve ◽  
The Neural Network ◽  
Recursive Neural Networks ◽  
The Neural Networks ◽  
A Company

The most valuable asset for a company is its customers’ base. As a result, customer relationship management (CRM) is an important task that drives companies. By identifying and understanding the valuable customer segments, appropriate marketing strategies can be used to enhance customer satisfaction and maintain loyalty, as well as increase company retention. Predicting customer turnover is an important tool for companies to stay competitive in a fast-growing market. In this paper, we use the recurrent nerve sketch to predict rejection based on the time series of the lifetime of the customer. In anticipation, a key aspect of identifying key triggers is to turn off. To overcome the weakness of recurrent neural networks, the research model of the combination of LRFMP with the neural network has been used. In this paper, it was found that clustering by LRFMP can be used to perform a more comprehensive analysis of customers’ turnover. In this solution, LRFMP is used to execute customer segregation. The objective is to provide a new framework for LRFMP for macrodata and macrodata analysis in order to increase the problem of business problem solving and customer depreciation. The results of the research show that the neural networks are capable of predicting the LRFMP precursors of the customers in an effective way. This model can be used in advocacy systems for advertising and loyalty programs management. In the previous research, the LRFM and RFM algorithms along with the neural network and the machine learning algorithm, etc., have been used, and in the proposed solution, the use of the LRFMP algorithm increases the accuracy of the desired.

scholarly journals Multi-Document Abstractive Summarization using Recursive Neural Network

2020 ◽  
Vol 9 (7) ◽  
pp. 364-370
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Text Summarization ◽  
Similarity Matrix ◽  
Text Documents ◽  
Short Text ◽  
Current State ◽  
Recursive Neural Networks ◽  
Semantic Concepts ◽  
Abstractive Summarization

Text summarization is an area of research with a goal to provide short text from huge text documents. Extractive text summarization methods have been extensively studied by many researchers. There are various type of multi document ranging from different formats to domains and topic specific. With the application of neural networks for text generation, interest for research in abstractive text summarization has increased significantly. This approach has been attempted for English and Telugu languages in this article. Recurrent neural networks are a subtype of recursive neural networks which try to predict the next sequence based on the current state and considering the information from previous states. The use of neural networks allows generation of summaries for long text sentences as well. The work implements semantic based filtering using a similarity matrix while keeping all stop-words. The similarity is calculated using semantic concepts and Jiang Similarity and making use of a Recurrent Neural Network (RNN) with an attention mechanism to generate summary. ROUGE score is used for measuring the performance of the applied method on Telugu and English langauges .

scholarly journals Inverse design of porous materials using artificial neural networks

2020 ◽  
Vol 6 (1) ◽  
pp. eaax9324 ◽  
Author(s):  
Baekjun Kim ◽  
Sangwon Lee ◽  
Jihan Kim
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Porous Materials ◽  
Fine Tuning ◽  
Inverse Design ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Artificial Neural ◽  
Simple Molecules

Generating optimal nanomaterials using artificial neural networks can potentially lead to a notable revolution in future materials design. Although progress has been made in creating small and simple molecules, complex materials such as crystalline porous materials have yet to be generated using any of the neural networks. Here, we have implemented a generative adversarial network that uses a training set of 31,713 known zeolites to produce 121 crystalline porous materials. Our neural network takes in inputs in the form of energy and material dimensions, and we show that zeolites with a user-desired range of 4 kJ/mol methane heat of adsorption can be reliably produced using our neural network. The fine-tuning of user-desired capability can potentially accelerate materials development as it demonstrates a successful case of inverse design of porous materials.

Application of an Improved Recurrent Neural Network in Network Security Situation Prediction

2014 ◽  
Vol 951 ◽  
pp. 173-176
Author(s):  
Jun Ping Cai ◽  
Lei Qu ◽  
Gang Chen ◽  
Jun Yang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Security ◽  
Network Topology ◽  
High Efficiency ◽  
High Accuracy ◽  
Data Traffic ◽  
Network Load ◽  
Nonlinear Complexity ◽  
Recursive Neural Networks

The expansion of the network becomes size, network mode is diversification, network topology structure becomes more complex, the data traffic rises rapidly in the network, causes the network load increases, attack, fault and other unexpected network security events are more severe. Neural network to deal with nonlinear, complexity advantage of this paper, network security situation prediction based on improved recursive neural networks, experimental results show that the high efficiency of the method, results are compared with the actual values, low error, high accuracy.

scholarly journals Neural Relation Extraction within and across Sentence Boundaries

2019 ◽  
Vol 33 ◽  
pp. 6513-6520 ◽  
Author(s):  
Pankaj Gupta ◽  
Subburam Rajaram ◽  
Hinrich Schütze ◽  
Thomas Runkler
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Binary Relation ◽  
State Of The Art ◽  
Relation Extraction ◽  
Shared Task ◽  
Single Sentence ◽  
Winning Team ◽  
Recursive Neural Networks ◽  
Better Than

Past work in relation extraction mostly focuses on binary relation between entity pairs within single sentence. Recently, the NLP community has gained interest in relation extraction in entity pairs spanning multiple sentences. In this paper, we propose a novel architecture for this task: inter-sentential dependency-based neural networks (iDepNN). iDepNN models the shortest and augmented dependency paths via recurrent and recursive neural networks to extract relationships within (intra-) and across (inter-) sentence boundaries. Compared to SVM and neural network baselines, iDepNN is more robust to false positives in relationships spanning sentences. We evaluate our models on four datasets from newswire (MUC6) and medical (BioNLP shared task) domains that achieve state-of-the-art performance and show a better balance in precision and recall for inter-sentential relationships. We perform better than 11 teams participating in the BioNLP shared task 2016 and achieve a gain of 5.2% (0.587 vs 0.558) in F1 over the winning team. We also release the crosssentence annotations for MUC6.

Inverse Design in Porous Materials Using Artificial Neural Networks

2019 ◽  
Author(s):  
Baekjun Kim ◽  
Sangwon Lee ◽  
Jihan Kim
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Porous Materials ◽  
Fine Tuning ◽  
Inverse Design ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Artificial Neural ◽  
Simple Molecules

Generating optimal nanomaterials using artificial neural networks can potentially lead to a significant revolution in future materials design. Although progress has been made in creating small and simple molecules, complex materials such as crystalline porous materials have yet to be generated using any of the neural networks. In this work, we have for the first time implemented a generative adversarial network that uses a training set of 31,713 known zeolites to produce 14 crystalline porous materials. Our neural network takes in inputs in the form of energy and material dimensions and we show that zeolites with a user-desired range of 4 kJ/mol methane heat of adsorption can be reliably produced using our neural network. The fine-tuning of user-desired capability can potentially accelerate materials development as it demonstrates a successful case of inverse design in porous materials.

scholarly journals Go Wider: An Efficient Neural Network for Point Cloud Analysis via Group Convolutions

2020 ◽  
Vol 10 (7) ◽  
pp. 2391
Author(s):  
Can Chen ◽  
Luca Zanotti Fragonara ◽  
Antonios Tsourdos
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Point Cloud ◽  
Large Scale ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Autonomous Driving ◽  
Fine Grained ◽  
Point Cloud Analysis ◽  
Cloud Analysis

In order to achieve a better performance for point cloud analysis, many researchers apply deep neural networks using stacked Multi-Layer-Perceptron (MLP) convolutions over an irregular point cloud. However, applying these dense MLP convolutions over a large amount of points (e.g., autonomous driving application) leads to limitations due to the computation and memory capabilities. To achieve higher performances but decrease the computational complexity, we propose a deep-wide neural network, named ShufflePointNet, which can exploit fine-grained local features, but also reduce redundancies using group convolution and channel shuffle operation. Unlike conventional operations that directly apply MLPs on the high-dimensional features of a point cloud, our model goes “wider” by splitting features into groups with smaller depth in advance, having the respective MLP computations applied only to a single group, which can significantly reduce complexity and computation. At the same time, we allow communication between groups by shuffling the feature channel to capture fine-grained features. We further discuss the multi-branch method for wider neural networks being also beneficial to feature extraction for point clouds. We present extensive experiments for shape classification tasks on a ModelNet40 dataset and semantic segmentation task on large scale datasets ShapeNet part, S3DIS and KITTI. Finally, we carry out an ablation study and compare our model to other state-of-the-art algorithms to show its efficiency in terms of complexity and accuracy.

scholarly journals Dynamic Compositionality in Recursive Neural Networks with Structure-Aware Tag Representations

2019 ◽  
Vol 33 ◽  
pp. 6594-6601 ◽  
Author(s):  
Taeuk Kim ◽  
Jihun Choi ◽  
Daniel Edmiston ◽  
Sanghwan Bae ◽  
Sang-goo Lee
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Neural Models ◽  
Sentence Level ◽  
Recursive Neural Networks ◽  
Syntactic Information ◽  
Parse Trees ◽  
By Products

Most existing recursive neural network (RvNN) architectures utilize only the structure of parse trees, ignoring syntactic tags which are provided as by-products of parsing. We present a novel RvNN architecture that can provide dynamic compositionality by considering comprehensive syntactic information derived from both the structure and linguistic tags. Specifically, we introduce a structure-aware tag representation constructed by a separate tag-level tree-LSTM. With this, we can control the composition function of the existing wordlevel tree-LSTM by augmenting the representation as a supplementary input to the gate functions of the tree-LSTM. In extensive experiments, we show that models built upon the proposed architecture obtain superior or competitive performance on several sentence-level tasks such as sentiment analysis and natural language inference when compared against previous tree-structured models and other sophisticated neural models.

Sign in / Sign up

Export Citation Format

Share Document