Knowledge Base Question Answering Based on Deep Learning Models

Text classification is a fundamental task in several areas of natural language processing (NLP), including words semantic classification, sentiment analysis, question answering, or dialog management. This paper investigates three basic architectures of deep learning models for the tasks of text classification: Deep Belief Neural (DBN), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), these three main types of deep learning architectures, are largely explored to handled various classification tasks. DBN have excellent learning capabilities to extracts highly distinguishable features and good for general purpose. CNN have supposed to be better at extracting the position of various related features while RNN is modeling in sequential of long-term dependencies. This paper work shows the systematic comparison of DBN, CNN, and RNN on text classification tasks. Finally, we show the results of deep models by research experiment. The aim of this paper to provides basic guidance about the deep learning models that which models are best for the task of text classification.

Download Full-text

Open Domain Question Answering with Character-Level Deep Learning Models

2017 10th International Symposium on Computational Intelligence and Design (ISCID) ◽

10.1109/iscid.2017.58 ◽

2017 ◽

Cited By ~ 8

Author(s):

Kai Lei ◽

Yang Deng ◽

Bing Zhang ◽

Ying Shen

Keyword(s):

Deep Learning ◽

Question Answering ◽

Open Domain ◽

Learning Models

Download Full-text

Deep learning models in detection of dietary supplement adverse event signals from Twitter

JAMIA Open ◽

10.1093/jamiaopen/ooab081 ◽

2021 ◽

Vol 4 (4) ◽

Author(s):

Yefeng Wang ◽

Yunpeng Zhao ◽

Dalton Schutte ◽

Jiang Bian ◽

Rui Zhang

Keyword(s):

Adverse Event ◽

Deep Learning ◽

Knowledge Base ◽

Dietary Supplement ◽

Relation Extraction ◽

Learning Models ◽

Concept Extraction ◽

Annotation Guideline ◽

End To End ◽

Ae Signals

Abstract Objective The objective of this study is to develop a deep learning pipeline to detect signals on dietary supplement-related adverse events (DS AEs) from Twitter. Materials and Methods We obtained 247 807 tweets ranging from 2012 to 2018 that mentioned both DS and AE. We designed a tailor-made annotation guideline for DS AEs and annotated biomedical entities and relations on 2000 tweets. For the concept extraction task, we fine-tuned and compared the performance of BioClinical-BERT, PubMedBERT, ELECTRA, RoBERTa, and DeBERTa models with a CRF classifier. For the relation extraction task, we fine-tuned and compared BERT models to BioClinical-BERT, PubMedBERT, RoBERTa, and DeBERTa models. We chose the best-performing models in each task to assemble an end-to-end deep learning pipeline to detect DS AE signals and compared the results to the known DS AEs from a DS knowledge base (ie, iDISK). Results DeBERTa-CRF model outperformed other models in the concept extraction task, scoring a lenient microaveraged F1 score of 0.866. RoBERTa model outperformed other models in the relation extraction task, scoring a lenient microaveraged F1 score of 0.788. The end-to-end pipeline built on these 2 models was able to extract DS indication and DS AEs with a lenient microaveraged F1 score of 0.666. Conclusion We have developed a deep learning pipeline that can detect DS AE signals from Twitter. We have found DS AEs that were not recorded in an existing knowledge base (iDISK) and our proposed pipeline can as sist DS AE pharmacovigilance.

Download Full-text

Technical, Hard and Explainable Question Answering (THE-QA)

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/916 ◽

2019 ◽

Author(s):

Shailaja Sampat

Keyword(s):

Artificial Intelligence ◽

Deep Learning ◽

Knowledge Representation ◽

Question Answering ◽

State Of The Art ◽

Future Research ◽

Semantic Parsing ◽

Learning Models ◽

Knowledge Types ◽

Language And Vision

The ability of an agent to rationally answer questions about a given task is the key measure of its intelligence. While we have obtained phenomenal performance over various language and vision tasks separately, 'Technical, Hard and Explainable Question Answering' (THE-QA) is a new challenging corpus which addresses them jointly. THE-QA is a question answering task involving diagram understanding and reading comprehension. We plan to establish benchmarks over this new corpus using deep learning models guided by knowledge representation methods. The proposed approach will envisage detailed semantic parsing of technical figures and text, which is robust against diverse formats. It will be aided by knowledge acquisition and reasoning module that categorizes different knowledge types, identify sources to acquire that knowledge and perform reasoning to answer the questions correctly. THE-QA data will present a strong challenge to the community for future research and will bridge the gap between state-of-the-art Artificial Intelligence (AI) and 'Human-level' AI.

Download Full-text

Deep learning-based question answering system for intelligent humanoid robot

Journal Of Big Data ◽

10.1186/s40537-020-00341-6 ◽

2020 ◽

Vol 7 (1) ◽

Author(s):

Widodo Budiharto ◽

Vincent Andreas ◽

Alexander Agung Santoso Gunawan

Keyword(s):

Neural Network ◽

Deep Learning ◽

Knowledge Base ◽

Humanoid Robot ◽

Question Answering ◽

The Self ◽

Learning Approach ◽

Learning Capability ◽

Question Answering Systems ◽

Self Learning

Abstract Background The development of Intelligent Humanoid Robot focuses on question answering systems that can interact with people is very limited. In this research, we would like to propose an Intelligent Humanoid Robot with the self-learning capability for accepting and giving responses from people based on Deep Learning and Big Data knowledge base. This kind of robot can be used widely in hotels, universities, and public services. The Humanoid Robot should consider the style of questions and conclude the answer through conversation between robot and user. In our scenario, the robot will detect the user’s face and accept commands from the user to do an action. Findings The question from the user will be processed using deep learning, and the result will be compared to the knowledge base on the system. We proposed our Deep Learning approach, based on Recurrent Neural Network (RNN) encoder, Convolution Neural Network (CNN) encoder, with Bidirectional Attention Flow (BiDAF). Conclusions Our evaluation indicates that using RNN based encoder with BiDAF gives a higher score, than CNN encoder with the BiDAF. Based on our experiment, our model get 82.43% F1 score and the RNN based encoder will give a higher EM/F1 score than using the CNN encoder.

Download Full-text

Using deep learning models for learning semantic text similarity of Arabic questions

International Journal of Electrical and Computer Engineering (IJECE) ◽

10.11591/ijece.v11i4.pp3519-3528 ◽

2021 ◽

Vol 11 (4) ◽

pp. 3519

Author(s):

Mahmoud Hammad ◽

Mohammed Al-Smadi ◽

Qanita Bani Baker ◽

Sa’ad A. Al-Zboon

Keyword(s):

Deep Learning ◽

Question Answering ◽

Supervised Machine Learning ◽

Learning Models ◽

Text Similarity ◽

Baseline Model ◽

Life Problems ◽

Machine Learning Model ◽

Recurrent Architecture ◽

The Right

Question-answering platforms serve millions of users seeking knowledge and solutions for their daily life problems. However, many knowledge seekers are facing the challenge to find the right answer among similar answered questions and writer’s responding to asked questions feel like they need to repeat answers many times for similar questions. This research aims at tackling the problem of learning the semantic text similarity among different asked questions by using deep learning. Three models are implemented to address the aforementioned problem: i) a supervised-machine learning model using XGBoost trained with pre-defined features, ii) an adapted Siamese-based deep learning recurrent architecture trained with pre-defined features, and iii) a Pre-trained deep bidirectional transformer based on BERT model. Proposed models were evaluated using a reference Arabic dataset from the mawdoo3.com company. Evaluation results show that the BERT-based model outperforms the other two models with an F1=92.99%, whereas the Siamese-based model comes in the second place with F1=89.048%, and finally, the XGBoost as a baseline model achieved the lowest result of F1=86.086%.

Download Full-text

Deep Learning-Based Question Answering System for Intelligent Humanoid Robot

10.21203/rs.3.rs-26510/v2 ◽

2020 ◽

Author(s):

Widodo Budiharto ◽

Vincent Andreas ◽

Alexander Agung Santoso Gunawan

Keyword(s):

Neural Network ◽

Deep Learning ◽

Knowledge Base ◽

Humanoid Robot ◽

Question Answering ◽

The Self ◽

Learning Approach ◽

Learning Capability ◽

Question Answering Systems ◽

Self Learning

Abstract Background- The development of Intelligent Humanoid Robot focuses on question answering systems that can interact with people is very limited. In this research, we would like to propose an Intelligent Humanoid Robot with the self-learning capability for accepting and giving responses from people based on Deep Learning and Big Data knowledge base. This kind of robot can be used widely in hotels, universities, and public services. The Humanoid Robot should consider the style of questions and conclude the answer through conversation between robot and user. In our scenario, the robot will detect the user’s face and accept commands from the user to do an action. Findings- The question from the user will be processed using deep learning, and the result will be compared to the knowledge base on the system. We proposed our Deep Learning approach, based on Recurrent Neural Network (RNN) encoder, Convolution Neural Network (CNN) encoder, with Bidirectional Attention Flow (BiDAF). Conclusions- Our evaluation indicates that using RNN based encoder with BiDAF gives a higher score, than CNN encoder with the BiDAF. Based on our experiment, our model get 82.43% F1 score and the RNN based encoder will give a higher EM / F1 score than using the CNN encoder.

Download Full-text

A Model of Convolutional Neural Network Combined with External Knowledge to Measure the Question Similarity for Community Question Answering Systems

International Journal of Machine Learning and Computing ◽

10.18178/ijmlc.2021.11.3.1035 ◽

2021 ◽

Vol 11 (3) ◽

pp. 194-201

Author(s):

Van-Tu Nguyen ◽

◽

Anh-Cuong Le ◽

Ha-Nam Nguyen

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Question Answering ◽

Learning Models ◽

Learning Methods ◽

Data Set ◽

External Knowledge ◽

Community Question Answering ◽

Proposed Model

Automatically determining similar questions and ranking the obtained questions according to their similarities to each input question is a very important task to any community Question Answering system (cQA). Various methods have applied for this task including conventional machine learning methods with feature extraction and some recent studies using deep learning methods. This paper addresses the problem of how to combine advantages of different methods into one unified model. Moreover, deep learning models are usually only effective for large data, while training data sets in cQA problems are often small, so the idea of integrating external knowledge into deep learning models for this cQA problem becomes more important. To this objective, we propose a neural network-based model which combines a Convolutional Neural Network (CNN) with features from other methods so that the deep learning model is enhanced with addtional knowledge sources. In our proposed model, the CNN component will learn the representation of two given questions, then combined additional features through a Multilayer Perceptron (MLP) to measure similarity between the two questions. We tested our proposed model on the SemEval 2016 task-3 data set and obtain better results in comparison with previous studies on the same task.

Download Full-text

Deep learning approaches for extracting adverse events and indications of dietary supplements from clinical text

Journal of the American Medical Informatics Association ◽

10.1093/jamia/ocaa218 ◽

2020 ◽

Author(s):

Yadan Fan ◽

Sicheng Zhou ◽

Yifan Li ◽

Rui Zhang

Keyword(s):

Deep Learning ◽

Adverse Events ◽

Knowledge Base ◽

Dietary Supplements ◽

Short Term Memory ◽

Conditional Random Field ◽

Entity Recognition ◽

Learning Models ◽

Clinical Text ◽

Clinical Notes

Abstract Objective We sought to demonstrate the feasibility of utilizing deep learning models to extract safety signals related to the use of dietary supplements (DSs) in clinical text. Materials and Methods Two tasks were performed in this study. For the named entity recognition (NER) task, Bi-LSTM-CRF (bidirectional long short-term memory conditional random field) and BERT (bidirectional encoder representations from transformers) models were trained and compared with CRF model as a baseline to recognize the named entities of DSs and events from clinical notes. In the relation extraction (RE) task, 2 deep learning models, including attention-based Bi-LSTM and convolutional neural network as well as a random forest model were trained to extract the relations between DSs and events, which were categorized into 3 classes: positive (ie, indication), negative (ie, adverse events), and not related. The best performed NER and RE models were further applied on clinical notes mentioning 88 DSs for discovering DSs adverse events and indications, which were compared with a DS knowledge base. Results For the NER task, deep learning models achieved a better performance than CRF, with F1 scores above 0.860. The attention-based Bi-LSTM model performed the best in the RE task, with an F1 score of 0.893. When comparing DS event pairs generated by the deep learning models with the knowledge base for DSs and event, we found both known and unknown pairs. Conclusions Deep learning models can detect adverse events and indication of DSs in clinical notes, which hold great potential for monitoring the safety of DS use.

Download Full-text