scholarly journals Combining External Medical Knowledge for improving Obstetric Intelligent Diagnosis

2020 ◽  
Author(s):  
Kunli Zhang ◽  
Linkun Cai ◽  
Yu Song ◽  
Tao Liu ◽  
Yueshu Zhao
Keyword(s):  
Domain Knowledge ◽  
Information Source ◽  
Language Model ◽  
Medical Knowledge ◽  
Strong Support ◽  
Development Trend ◽  
Attention Mechanism ◽  
Intelligent Diagnosis ◽  
Model Combining ◽  
Improved Model

Abstract Background: Data-driven medical health information processing has become a new development trend in obstetrics. Electronic medical records (EMRs) are the basis of evidence-based medicine and important information source for intelligent diagnosis. To obtain diagnostic results, doctors combine clinical experience and medical knowledge in their diagnosis process. External medical knowledge provides strong support for diagnosis. Therefore, how to make full use of EMRs and medical knowledge in intelligent diagnosis is worth studying.Methods: As an EMR usually contains multiple types of diagnostic results, the intelligent diagnosis can be treated as a multi-label classification task. We propose a novel neural network model called Knowledge-aware Hierarchical Diagnosis Model (KHDM) in which Chinese obstetric EMRs and external medical knowledge can be synchronously and effectively used for intelligent diagnostics. In the KHDM, EMRs and external knowledge documents are integrated by the attention mechanism contained in the hierarchical deep learning framework. In this way, we enrich the language model with curated knowledge documents, combining the advantages of both to make a knowledge-aware diagnosis.Results: We evaluate our model on a real-world Chinese obstetric EMR dataset show that KHDM achieves an accuracy of 0.8929, which exceeds that of the most advanced classification benchmark methods.Conclusion: In this paper, an improved model combining medical knowledge and an attention mechanism is proposed, which is based on the problem of the diversity of diagnostic results in Chinese EMRs. KHDM can effectively integrate domain knowledge to greatly improve the accuracy of diagnosis, and we also verify the model's interpretability advantage.

scholarly journals Combining External Medical Knowledge for Improving Obstetric Intelligent Diagnosis: Model Development and Validation

10.2196/25304 ◽  
2021 ◽  
Vol 9 (5) ◽  
pp. e25304
Author(s):  
Kunli Zhang ◽  
Linkun Cai ◽  
Yu Song ◽  
Tao Liu ◽  
Yueshu Zhao
Keyword(s):  
Domain Knowledge ◽  
Information Source ◽  
Language Model ◽  
Model Development ◽  
Medical Knowledge ◽  
Strong Support ◽  
Development Trend ◽  
Attention Mechanism ◽  
Intelligent Diagnosis ◽  
Diagnosis Model

Background Data-driven medical health information processing has become a new development trend in obstetrics. Electronic medical records (EMRs) are the basis of evidence-based medicine and an important information source for intelligent diagnosis. To obtain diagnostic results, doctors combine clinical experience and medical knowledge in their diagnosis process. External medical knowledge provides strong support for diagnosis. Therefore, it is worth studying how to make full use of EMRs and medical knowledge in intelligent diagnosis. Objective This study aims to improve the performance of intelligent diagnosis in EMRs by combining medical knowledge. Methods As an EMR usually contains multiple types of diagnostic results, the intelligent diagnosis can be treated as a multilabel classification task. We propose a novel neural network knowledge-aware hierarchical diagnosis model (KHDM) in which Chinese obstetric EMRs and external medical knowledge can be synchronously and effectively used for intelligent diagnostics. In KHDM, EMRs and external knowledge documents are integrated by the attention mechanism contained in the hierarchical deep learning framework. In this way, we enrich the language model with curated knowledge documents, combining the advantages of both to make a knowledge-aware diagnosis. Results We evaluate our model on a real-world Chinese obstetric EMR dataset and showed that KHDM achieves an accuracy of 0.8929, which exceeds that of the most advanced classification benchmark methods. We also verified the model’s interpretability advantage. Conclusions In this paper, an improved model combining medical knowledge and an attention mechanism is proposed, based on the problem of diversity of diagnostic results in Chinese EMRs. KHDM can effectively integrate domain knowledge to greatly improve the accuracy of diagnosis.

scholarly journals Combining External Medical Knowledge for Improving Obstetric Intelligent Diagnosis: Model Development and Validation (Preprint)

2020 ◽  
Author(s):  
Kunli Zhang ◽  
Linkun Cai ◽  
Yu Song ◽  
Tao Liu ◽  
Yueshu Zhao
Keyword(s):  
Domain Knowledge ◽  
Information Source ◽  
Language Model ◽  
Model Development ◽  
Medical Knowledge ◽  
Strong Support ◽  
Development Trend ◽  
Attention Mechanism ◽  
Intelligent Diagnosis ◽  
Diagnosis Model

BACKGROUND Data-driven medical health information processing has become a new development trend in obstetrics. Electronic medical records (EMRs) are the basis of evidence-based medicine and an important information source for intelligent diagnosis. To obtain diagnostic results, doctors combine clinical experience and medical knowledge in their diagnosis process. External medical knowledge provides strong support for diagnosis. Therefore, it is worth studying how to make full use of EMRs and medical knowledge in intelligent diagnosis. OBJECTIVE This study aims to improve the performance of intelligent diagnosis in EMRs by combining medical knowledge. METHODS As an EMR usually contains multiple types of diagnostic results, the intelligent diagnosis can be treated as a multilabel classification task. We propose a novel neural network knowledge-aware hierarchical diagnosis model (KHDM) in which Chinese obstetric EMRs and external medical knowledge can be synchronously and effectively used for intelligent diagnostics. In KHDM, EMRs and external knowledge documents are integrated by the attention mechanism contained in the hierarchical deep learning framework. In this way, we enrich the language model with curated knowledge documents, combining the advantages of both to make a knowledge-aware diagnosis. RESULTS We evaluate our model on a real-world Chinese obstetric EMR dataset and showed that KHDM achieves an accuracy of 0.8929, which exceeds that of the most advanced classification benchmark methods. We also verified the model’s interpretability advantage. CONCLUSIONS In this paper, an improved model combining medical knowledge and an attention mechanism is proposed, based on the problem of diversity of diagnostic results in Chinese EMRs. KHDM can effectively integrate domain knowledge to greatly improve the accuracy of diagnosis.

scholarly journals Offensive keyword extraction based on the attention mechanism of BERT and the eigenvector centrality using a graph representation

2021 ◽  
Author(s):  
Gretel Liz De la Peña Sarracén ◽  
Paolo Rosso
Keyword(s):  
Language Model ◽  
Hybrid Approach ◽  
Attention Mechanism ◽  
Graph Representation ◽  
Keyword Extraction ◽  
Eigenvector Centrality ◽  
Model Learning ◽  
Harmful Content ◽  
The Impact ◽  
Offensive Language

AbstractThe proliferation of harmful content on social media affects a large part of the user community. Therefore, several approaches have emerged to control this phenomenon automatically. However, this is still a quite challenging task. In this paper, we explore the offensive language as a particular case of harmful content and focus our study in the analysis of keywords in available datasets composed of offensive tweets. Thus, we aim to identify relevant words in those datasets and analyze how they can affect model learning. For keyword extraction, we propose an unsupervised hybrid approach which combines the multi-head self-attention of BERT and a reasoning on a word graph. The attention mechanism allows to capture relationships among words in a context, while a language model is learned. Then, the relationships are used to generate a graph from what we identify the most relevant words by using the eigenvector centrality. Experiments were performed by means of two mechanisms. On the one hand, we used an information retrieval system to evaluate the impact of the keywords in recovering offensive tweets from a dataset. On the other hand, we evaluated a keyword-based model for offensive language detection. Results highlight some points to consider when training models with available datasets.

scholarly journals Attentional Reinforcement Learning in the Brain

2020 ◽  
Vol 38 (1) ◽  
pp. 49-64 ◽  
Author(s):  
Hiroshi Yamakawa
Keyword(s):  
Deep Learning ◽  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Language Processing ◽  
Information Source ◽  
Attention Mechanism ◽  
Transmission Route ◽  
Thalamic Relay ◽  
Signal Changes ◽  
The Brain

AbstractRecently, attention mechanisms have significantly boosted the performance of natural language processing using deep learning. An attention mechanism can select the information to be used, such as by conducting a dictionary lookup; this information is then used, for example, to select the next utterance word in a sentence. In neuroscience, the basis of the function of sequentially selecting words is considered to be the cortico-basal ganglia-thalamocortical loop. Here, we first show that the attention mechanism used in deep learning corresponds to the mechanism in which the cerebral basal ganglia suppress thalamic relay cells in the brain. Next, we demonstrate that, in neuroscience, the output of the basal ganglia is associated with the action output in the actor of reinforcement learning. Based on these, we show that the aforementioned loop can be generalized as reinforcement learning that controls the transmission of the prediction signal so as to maximize the prediction reward. We call this attentional reinforcement learning (ARL). In ARL, the actor selects the information transmission route according to the attention, and the prediction signal changes according to the context detected by the information source of the route. Hence, ARL enables flexible action selection that depends on the situation, unlike traditional reinforcement learning, wherein the actor must directly select an action.

Knowledge-Based Induction of Clinical Prediction Rules

2011 ◽  
pp. 350-375
Author(s):  
Mila Kwiatkowska ◽  
M. Stella Atkins ◽  
Les Matthews ◽  
Najib T. Ayas ◽  
C. Frank Ryan
Keyword(s):  
Domain Knowledge ◽  
Medical Knowledge ◽  
Data Sets ◽  
Clinical Prediction ◽  
Clinical Prediction Rules ◽  
Prediction Rules ◽  
Integrative Framework ◽  
Knowledge Based ◽  
The Creation ◽  
Definition Of

This chapter describes how to integrate medical knowledge with purely inductive (data-driven) methods for the creation of clinical prediction rules. It addresses three issues: representation of medical knowledge, secondary analysis of medical data, and evaluation of automatically induced predictive models in the context of existing knowledge. To address the complexity of the domain knowledge, the authors have introduced a semio-fuzzy framework, which has its theoretical foundations in semiotics and fuzzy logic. This integrative framework has been applied to the creation of clinical prediction rules for the diagnosis of obstructive sleep apnea, a serious and under-diagnosed respiratory disorder. The authors use a semio-fuzzy approach (1) to construct a knowledge base for the definition of diagnostic criteria, predictors, and existing prediction rules; (2) to describe and analyze data sets used in the data mining process; and (3) to interpret the induced models in terms of confirmation, contradiction, and contribution to existing knowledge.

scholarly journals Combined Self-Attention Mechanism for Chinese Named Entity Recognition in Military

2019 ◽  
Vol 11 (8) ◽  
pp. 180
Author(s):  
Fei Liao ◽  
Liangli Ma ◽  
Jingjing Pei ◽  
Linshan Tan
Keyword(s):  
Domain Knowledge ◽  
Short Term Memory ◽  
Contextual Information ◽  
Named Entity Recognition ◽  
Attention Mechanism ◽  
The Self ◽  
Entity Recognition ◽  
Named Entity ◽  
Vector Representations ◽  
The Military

Military named entity recognition (MNER) is one of the key technologies in military information extraction. Traditional methods for the MNER task rely on cumbersome feature engineering and specialized domain knowledge. In order to solve this problem, we propose a method employing a bidirectional long short-term memory (BiLSTM) neural network with a self-attention mechanism to identify the military entities automatically. We obtain distributed vector representations of the military corpus by unsupervised learning and the BiLSTM model combined with the self-attention mechanism is adopted to capture contextual information fully carried by the character vector sequence. The experimental results show that the self-attention mechanism can improve effectively the performance of MNER task. The F-score of the military documents and network military texts identification was 90.15% and 89.34%, respectively, which was better than other models.

Improve Language Modeling for Code Completion Through Learning General Token Repetition of Source Code with Optimized Memory

2019 ◽  
Vol 29 (11n12) ◽  
pp. 1801-1818
Author(s):  
Yixiao Yang ◽  
Xiang Chen ◽  
Jiaguang Sun
Keyword(s):  
Prediction Accuracy ◽  
Language Model ◽  
Source Code ◽  
Attention Mechanism ◽  
Data Set ◽  
Training Time ◽  
Unseen Data ◽  
Code Completion ◽  
High Chance ◽  
Better Than

In last few years, applying language model to source code is the state-of-the-art method for solving the problem of code completion. However, compared with natural language, code has more obvious repetition characteristics. For example, a variable can be used many times in the following code. Variables in source code have a high chance to be repetitive. Cloned code and templates, also have the property of token repetition. Capturing the token repetition of source code is important. In different projects, variables or types are usually named differently. This means that a model trained in a finite data set will encounter a lot of unseen variables or types in another data set. How to model the semantics of the unseen data and how to predict the unseen data based on the patterns of token repetition are two challenges in code completion. Hence, in this paper, token repetition is modelled as a graph, we propose a novel REP model which is based on deep neural graph network to learn the code toke repetition. The REP model is to identify the edge connections of a graph to recognize the token repetition. For predicting the token repetition of token [Formula: see text], the information of all the previous tokens needs to be considered. We use memory neural network (MNN) to model the semantics of each distinct token to make the framework of REP model more targeted. The experiments indicate that the REP model performs better than LSTM model. Compared with Attention-Pointer network, we also discover that the attention mechanism does not work in all situations. The proposed REP model could achieve similar or slightly better prediction accuracy compared to Attention-Pointer network and consume less training time. We also find other attention mechanism which could further improve the prediction accuracy.

scholarly journals SG-Net: Syntax-Guided Machine Reading Comprehension

2020 ◽  
Vol 34 (05) ◽  
pp. 9636-9643
Author(s):  
Zhuosheng Zhang ◽  
Yuwei Wu ◽  
Junru Zhou ◽  
Sufeng Duan ◽  
Hai Zhao ◽  
...  
Keyword(s):  
Reading Comprehension ◽  
Performance Improvement ◽  
Language Model ◽  
Attention Mechanism ◽  
Linguistic Knowledge ◽  
Attention Network ◽  
Syntactic Constraints ◽  
Syntactic Dependency ◽  
Machine Reading

For machine reading comprehension, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy passages and getting ride of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanism for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the SAN from the original Transformer encoder through a dual contextual architecture for better linguistics inspired representation. To verify its effectiveness, the proposed SG-Net is applied to typical pre-trained language model BERT which is right based on a Transformer encoder. Extensive experiments on popular benchmarks including SQuAD 2.0 and RACE show that the proposed SG-Net design helps achieve substantial performance improvement over strong baselines.

scholarly journals DATA-GRU: Dual-Attention Time-Aware Gated Recurrent Unit for Irregular Multivariate Time Series

2020 ◽  
Vol 34 (01) ◽  
pp. 930-937
Author(s):  
Qingxiong Tan ◽  
Mang Ye ◽  
Baoyao Yang ◽  
Siqi Liu ◽  
Andy Jinhua Ma ◽  
...  
Keyword(s):  
Time Series ◽  
Missing Values ◽  
Multivariate Time Series ◽  
Medical Knowledge ◽  
Attention Mechanism ◽  
Physiological Variables ◽  
Real World Datasets ◽  
Clinical Records ◽  
Gated Recurrent Unit ◽  
Time Aware

Due to the discrepancy of diseases and symptoms, patients usually visit hospitals irregularly and different physiological variables are examined at each visit, producing large amounts of irregular multivariate time series (IMTS) data with missing values and varying intervals. Existing methods process IMTS into regular data so that standard machine learning models can be employed. However, time intervals are usually determined by the status of patients, while missing values are caused by changes in symptoms. Therefore, we propose a novel end-to-end Dual-Attention Time-Aware Gated Recurrent Unit (DATA-GRU) for IMTS to predict the mortality risk of patients. In particular, DATA-GRU is able to: 1) preserve the informative varying intervals by introducing a time-aware structure to directly adjust the influence of the previous status in coordination with the elapsed time, and 2) tackle missing values by proposing a novel dual-attention structure to jointly consider data-quality and medical-knowledge. A novel unreliability-aware attention mechanism is designed to handle the diversity in the reliability of different data, while a new symptom-aware attention mechanism is proposed to extract medical reasons from original clinical records. Extensive experimental results on two real-world datasets demonstrate that DATA-GRU can significantly outperform state-of-the-art methods and provide meaningful clinical interpretation.

scholarly journals Application of Two-Level Joint Information Fusion Model in Intelligent Vehicle

2016 ◽  
Vol 12 (03) ◽  
pp. 77
Author(s):  
Yan Ting Lan ◽  
Jiinying Huang ◽  
Xiaodong Chen
Keyword(s):  
Information Fusion ◽  
Feature Fusion ◽  
Target Identification ◽  
Information Source ◽  
Evidence Theory ◽  
Intelligent Vehicles ◽  
Fusion Model ◽  
Multiple Features ◽  
Model Combining ◽  
Real Vehicle

This paper proposes a two-level joint information fusion model combining BP neural network and D-S evidence theory. The model of great practical value reduces target identification error probability by multiple features of the target information, shows good scalability with its two steps of information fusion model, and conveniently increases/reduces feature fusion information source according to different situations and different objects. The method used for intelligent vehicles has good flexibility and robustness in tracking and avoiding obstacle. The simulation and real vehicle tests have verified effectiveness of the method.

Sign in / Sign up

Export Citation Format

Share Document