scholarly journals Entity Type Enhanced Neural Model for Distantly Supervised Relation Extraction (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13751-13752
Author(s):  
Long Bai ◽  
Xiaolong Jin ◽  
Chuanzhi Zhuang ◽  
Xueqi Cheng
Keyword(s):  
Neural Network ◽  
Semantic Information ◽  
Relation Extraction ◽  
Neural Model ◽  
Benchmark Dataset ◽  
Experimental Comparison ◽  
Type Information

Distantly Supervised Relation Extraction (DSRE) has been widely studied, since it can automatically extract relations from very large corpora. However, existing DSRE methods only use little semantic information about entities, such as the information of entity type. Thus, in this paper, we propose a method for integrating entity type information into a neural network based DSRE model. It also adopts two attention mechanisms, namely, sentence attention and type attention. The former selects the representative sentences for a sentence bag, while the latter selects appropriate type information for entities. Experimental comparison with existing methods on a benchmark dataset demonstrates its merits.

scholarly journals Network Embedding via a Bi-Mode and Deep Neural Network Model

2017 ◽  
Author(s):  
Yang Fang ◽  
Xiang Zhao ◽  
Zhen Tan
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Semantic Information ◽  
Dimensional Space ◽  
Relation Extraction ◽  
Network Embedding ◽  
Structure Information ◽  
Second Mode ◽  
Real World Datasets ◽  
Low Dimensional

Network Embedding (NE) is an important method to learn the representations of network via a low-dimensional space. Conventional NE models focus on capturing the structure information and semantic information of vertices while neglecting such information for edges. In this work, we propose a novel NE model named BimoNet to capture both the structure and semantic information of edges. BimoNet is composed of two parts, i.e., the bi-mode embedding part and the deep neural network part. For bi-mode embedding part, the first mode named add-mode is used to express the entity-shared features of edges and the second mode named subtract-mode is employed to represent the entity-specific features of edges. These features actually reflect the semantic information. For deep neural network part, we firstly regard the edges in a network as nodes, and the vertices as links, which will not change the overall structure of the whole network. Then we take the nodes' adjacent matrix as the input of the deep neural network as it can obtain similar representations for nodes with similar structure. Afterwards, by jointly optimizing the objective function of these two parts, BimoNet could preserve both the semantic and structure information of edges. In experiments, we evaluate BimoNet on three real-world datasets and task of relation extraction, and BimoNet is demonstrated to outperform state-of-the-art baseline models consistently and significantly.

scholarly journals Entity Relation Extraction Based on Entity Indicators

Symmetry ◽  
2021 ◽  
Vol 13 (4) ◽  
pp. 539
Author(s):  
Yongbin Qin ◽  
Weizhe Yang ◽  
Kai Wang ◽  
Ruizhang Huang ◽  
Feng Tian ◽  
...  
Keyword(s):  
Neural Network ◽  
Semantic Information ◽  
Chinese Literature ◽  
Relation Extraction ◽  
Relevant Information ◽  
Internal Symmetry ◽  
Named Entity ◽  
Systemic Analysis ◽  
The Neural Network ◽  
The Impact

Relation extraction aims to extract semantic relationships between two specified named entities in a sentence. Because a sentence often contains several named entity pairs, a neural network is easily bewildered when learning a relation representation without position and semantic information about the considered entity pair. In this paper, instead of learning an abstract representation from raw inputs, task-related entity indicators are designed to enable a deep neural network to concentrate on the task-relevant information. By implanting entity indicators into a relation instance, the neural network is effective for encoding syntactic and semantic information about a relation instance. Organized, structured and unified entity indicators can make the similarity between sentences that possess the same or similar entity pair and the internal symmetry of one sentence more obviously. In the experiment, a systemic analysis was conducted to evaluate the impact of entity indicators on relation extraction. This method has achieved state-of-the-art performance, exceeding the compared methods by more than 3.7%, 5.0% and 11.2% in F1 score on the ACE Chinese corpus, ACE English corpus and Chinese literature text corpus, respectively.

scholarly journals An autoencoder-based neural network model for selectional preference: evidence from pseudo-disambiguation and cloze tasks

2017 ◽  
Vol 8 (2) ◽  
pp. 93-125
Author(s):  
Aki-Juhani Kyröläinen ◽  
Juhani Luotolahti ◽  
Filip Ginter
Keyword(s):  
Neural Network ◽  
Semantic Similarity ◽  
Network Model ◽  
Neural Network Model ◽  
Semantic Information ◽  
Contextual Information ◽  
Neural Model ◽  
The Other ◽  
Excellent Performance

Intuitively, some predicates have a better fit with certain arguments than others. Usage-based models of language emphasize the importance of semantic similarity in shaping the structuring of constructions (form and meaning). In this study, we focus on modeling the semantics of transitive constructions in Finnish and present an autoencoder-based neural network model trained on semantic vectors based on Word2vec. This model builds on the distributional hypothesis according to which semantic information is primarily shaped by contextual information. Specifically, we focus on the realization of the object. The performance of the model is evaluated in two tasks: a pseudo-disambiguation and a cloze task. Additionally, we contrast the performance of the autoencoder with a previously implemented neural model. In general, the results show that our model achieves an excellent performance on these tasks in comparison to the other models. The results are discussed in terms of usage-based construction grammar.Kokkuvõte. Aki-Juhani Kyröläinen, M. Juhani Luotolahti ja Filip Ginter: Autokoodril põhinev närvivõrkude mudel valikulisel eelistamisel. Intuitiivselt tundub, et mõned argumendid sobivad teatud predikaatidega paremini kokku kui teised. Kasutuspõhised keelemudelid rõhutavad konstruktsioonide struktuuri (nii vormi kui tähenduse) kujunemisel tähendusliku sarnasuse olulisust. Selles uurimuses modelleerime soome keele transitiivsete konstruktsioonide semantikat ja esitame närvivõrkude mudeli ehk autokoodri. Mudel põhineb distributiivse semantika hüpoteesil, mille järgi kujuneb semantiline info peamiselt konteksti põhjal. Täpsemalt keskendume uurimuses objektile. Mudelit hindame nii valeühestamise kui ka lünkülesande abil. Kõrvutame autokoodri tulemusi varem välja töötatud neurovõrgumudelitega ja tõestame, et meie mudel töötab võrreldes teiste mudelitega väga hästi. Tulemused esitame kasutuspõhise konstruktsioonigrammatika kontekstis.Võtmesõnad: neurovõrk; autokooder; tähendusvektor; kasutuspõhine mudel; soome keel

scholarly journals A Boundary Determined Neural Model for Relation Extraction

2021 ◽  
Vol 16 (3) ◽  
Author(s):  
RUI XUE TANG
Keyword(s):  
Neural Network ◽  
Error Propagation ◽  
State Of The Art ◽  
Relation Extraction ◽  
Neural Model ◽  
Relevant Information ◽  
Propagation Problem ◽  
Distributed Representation ◽  
High Quality ◽  
Information Encoding

Existing models extract entity relations only after two entity spans have been precisely extracted that influenced the performance of relation extraction. Compared with recognizing entity spans, because the boundary has a small granularity and a less ambiguity, it can be detected precisely and incorporated to learn better representation. Motivated by the strengths of boundary, we propose a boundary determined neural (BDN) model, which leverages boundaries as task-related cues to predict the relation labels. Our model can predict high-quality relation instance via the pairs of boundaries, which can relieve error propagation problem. Moreover, our model fuses with boundary-relevant information encoding to represent distributed representation to improve the ability of capturing semantic and dependency information, which can increase the discriminability of neural network. Experiments show that our model achieves state-of-the-art performances on ACE05 corpus.

Oil Saturation Log Prediction Using Neural Network in New Steamflood Area

2020 ◽  
Author(s):  
A. Syahputra
Keyword(s):  
Neural Network ◽  
Water Saturation ◽  
Prediction Method ◽  
Neural Model ◽  
Training Model ◽  
Acquisition Time ◽  
Oil Saturation ◽  
Full Field ◽  
Remaining Oil ◽  
Observation Wells

Surveillance is very important in managing a steamflood project. On the current surveillance plan, Temperature and steam ID logs are acquired on observation wells at least every year while CO log (oil saturation log or SO log) every 3 years. Based on those surveillance logs, a dynamic full field reservoir model is updated quarterly. Typically, a high depletion rate happens in a new steamflood area as a function of drainage activities and steamflood injection. Due to different acquisition time, there is a possibility of misalignment or information gaps between remaining oil maps (ie: net pay, average oil saturation or hydrocarbon pore thickness map) with steam chest map, for example a case of high remaining oil on high steam saturation interval. The methodology that is used to predict oil saturation log is neural network. In this neural network method, open hole observation wells logs (static reservoir log) such as vshale, porosity, water saturation effective, and pay non pay interval), dynamic reservoir logs as temperature, steam saturation, oil saturation, and acquisition time are used as input. A study case of a new steamflood area with 16 patterns of single reservoir target used 6 active observation wells and 15 complete logs sets (temperature, steam ID, and CO log), 19 incomplete logs sets (only temperature and steam ID) since 2014 to 2019. Those data were divided as follows ~80% of completed log set data for neural network training model and ~20% of completed log set data for testing the model. As the result of neural model testing, R2 is score 0.86 with RMS 5% oil saturation. In this testing step, oil saturation log prediction is compared to actual data. Only minor data that shows different oil saturation value and overall shape of oil saturation logs are match. This neural network model is then used for oil saturation log prediction in 19 incomplete log set. The oil saturation log prediction method can fill the gap of data to better describe the depletion process in a new steamflood area. This method also helps to align steam map and remaining oil to support reservoir management in a steamflood project.

scholarly journals Two-Pass Technique for Clone Detection and Type Classification Using Tree-Based Convolution Neural Network

2021 ◽  
Vol 11 (14) ◽  
pp. 6613
Author(s):  
Young-Bin Jo ◽  
Jihyun Lee ◽  
Cheol-Jung Yoo
Keyword(s):  
Neural Network ◽  
Average Rate ◽  
Convolution Neural Network ◽  
Clone Detection ◽  
Code Clones ◽  
Classification Technique ◽  
Development Costs ◽  
Code Quality ◽  
Type Information ◽  
Type Classification

Appropriate reliance on code clones significantly reduces development costs and hastens the development process. Reckless cloning, in contrast, reduces code quality and ultimately adds costs and time. To avoid this scenario, many researchers have proposed methods for clone detection and refactoring. The developed techniques, however, are only reliably capable of detecting clones that are either entirely identical or that only use modified identifiers, and do not provide clone-type information. This paper proposes a two-pass clone classification technique that uses a tree-based convolution neural network (TBCNN) to detect multiple clone types, including clones that are not wholly identical or to which only small changes have been made, and automatically classify them by type. Our method was validated with BigCloneBench, a well-known and wildly used dataset of cloned code. Our experimental results validate that our technique detected clones with an average rate of 96% recall and precision, and classified clones with an average rate of 78% recall and precision.

scholarly journals Towards Enhanced Performance of Neural-Network-Based Fault Detection Using an Sequential D-Optimum Experimental Design

2018 ◽  
Vol 8 (8) ◽  
pp. 1290 ◽  
Author(s):  
Beata Mrugalska
Keyword(s):  
Neural Network ◽  
Experimental Design ◽  
Fault Detection ◽  
Network Models ◽  
Neural Model ◽  
Neural Network Models ◽  
Optimum Experimental Design ◽  
The Neural Network ◽  
Adaptive Thresholds ◽  
Linear Neural Network

Increasing expectations of industrial system reliability require development of more effective and robust fault diagnosis methods. The paper presents a framework for quality improvement on the neural model applied for fault detection purposes. In particular, the proposed approach starts with an adaptation of the modified quasi-outer-bounding algorithm towards non-linear neural network models. Subsequently, its convergence is proven using quadratic boundedness paradigm. The obtained algorithm is then equipped with the sequential D-optimum experimental design mechanism allowing gradual reduction of the neural model uncertainty. Finally, an emerging robust fault detection framework on the basis of the neural network uncertainty description as the adaptive thresholds is proposed.

scholarly journals Extended Feature Pyramid Network with Adaptive Scale Training Strategy and Anchors for Object Detection in Aerial Images

2020 ◽  
Vol 12 (5) ◽  
pp. 784 ◽  
Author(s):  
Wei Guo ◽  
Weihong Li ◽  
Weiguo Gong ◽  
Jinkai Cui
Keyword(s):  
Neural Network ◽  
Object Detection ◽  
Semantic Information ◽  
Aerial Images ◽  
Training Strategy ◽  
The Public ◽  
Multi Scale ◽  
Shallow Layer ◽  
The Neural Network ◽  
Feature Pyramid

Multi-scale object detection is a basic challenge in computer vision. Although many advanced methods based on convolutional neural networks have succeeded in natural images, the progress in aerial images has been relatively slow mainly due to the considerably huge scale variations of objects and many densely distributed small objects. In this paper, considering that the semantic information of the small objects may be weakened or even disappear in the deeper layers of neural network, we propose a new detection framework called Extended Feature Pyramid Network (EFPN) for strengthening the information extraction ability of the neural network. In the EFPN, we first design the multi-branched dilated bottleneck (MBDB) module in the lateral connections to capture much more semantic information. Then, we further devise an attention pathway for better locating the objects. Finally, an augmented bottom-up pathway is conducted for making shallow layer information easier to spread and further improving performance. Moreover, we present an adaptive scale training strategy to enable the network to better recognize multi-scale objects. Meanwhile, we present a novel clustering method to achieve adaptive anchors and make the neural network better learn data features. Experiments on the public aerial datasets indicate that the presented method obtain state-of-the-art performance.

scholarly journals Semiotic Aggregation in Deep Learning

Entropy ◽  
2020 ◽  
Vol 22 (12) ◽  
pp. 1365
Author(s):  
Bogdan Muşat ◽  
Răzvan Andonie
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Decision Model ◽  
Deep Neural Networks ◽  
Neural Model ◽  
Network Layers ◽  
Saliency Maps ◽  
Spatial Entropy ◽  
Insight Into

Convolutional neural networks utilize a hierarchy of neural network layers. The statistical aspects of information concentration in successive layers can bring an insight into the feature abstraction process. We analyze the saliency maps of these layers from the perspective of semiotics, also known as the study of signs and sign-using behavior. In computational semiotics, this aggregation operation (known as superization) is accompanied by a decrease of spatial entropy: signs are aggregated into supersign. Using spatial entropy, we compute the information content of the saliency maps and study the superization processes which take place between successive layers of the network. In our experiments, we visualize the superization process and show how the obtained knowledge can be used to explain the neural decision model. In addition, we attempt to optimize the architecture of the neural model employing a semiotic greedy technique. To the extent of our knowledge, this is the first application of computational semiotics in the analysis and interpretation of deep neural networks.

Sign in / Sign up

Export Citation Format

Share Document