Question Answering

2012 ◽  
Author(s):  
Sanda Harabagiu ◽  
Dan Moldovan
Keyword(s):  
Language Processing ◽  
Question Answering ◽  
Document Retrieval ◽  
Semantic Features ◽  
Knowledge Based ◽  
Answer Extraction ◽  
Small Set ◽  
Processing Module ◽  
On Line ◽  
Processing Techniques

Textual Question Answering (QA) identifies the answer to a question in large collections of on-line documents. By providing a small set of exact answers to questions, QA takes a step closer to information retrieval rather than document retrieval. A QA system comprises three modules: a question-processing module, a document-processing module, and an answer extraction and formulation module. Questions may be asked about any topic, in contrast with Information Extraction (IE), which identifies textual information relevant only to a predefined set of events and entities. The natural language processing (NLP) techniques used in open-domain QA systems may range from simple lexical and semantic disambiguation of question stems to complex processing that combines syntactic and semantic features of the questions with pragmatic information derived from the context of candidate answers. This article reviews current research in integrating knowledge-based NLP methods with shallow processing techniques for QA.

A Persian Medical Question Answering System

2020 ◽  
Vol 29 (06) ◽  
pp. 2050019
Author(s):  
Hadi Veisi ◽  
Hamed Fakour Shandi
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Document Retrieval ◽  
Main Concept ◽  
Question Answering System ◽  
Detection Algorithms ◽  
Persian Language ◽  
Part Of Speech ◽  
Answer Extraction

A question answering system is a type of information retrieval that takes a question from a user in natural language as the input and returns the best answer to it as the output. In this paper, a medical question answering system in the Persian language is designed and implemented. During this research, a dataset of diseases and drugs is collected and structured. The proposed system includes three main modules: question processing, document retrieval, and answer extraction. For the question processing module, a sequential architecture is designed which retrieves the main concept of a question by using different components. In these components, rule-based methods, natural language processing, and dictionary-based techniques are used. In the document retrieval module, the documents are indexed and searched using the Lucene library. The retrieved documents are ranked using similarity detection algorithms and the highest-ranked document is selected to be used by the answer extraction module. This module is responsible for extracting the most relevant section of the text in the retrieved document. During this research, different customized language processing tools such as part of speech tagger and lemmatizer are also developed for Persian. Evaluation results show that this system performs well for answering different questions about diseases and drugs. The accuracy of the system for 500 sample questions is 83.6%.

Machine learning for query formulation in question answering

2011 ◽  
Vol 17 (4) ◽  
pp. 425-454 ◽  
Author(s):  
CHRISTOF MONZ
Keyword(s):  
Machine Learning ◽  
Information Retrieval ◽  
Language Processing ◽  
Question Answering ◽  
Document Retrieval ◽  
Machine Learning Techniques ◽  
Strong Impact ◽  
Query Formulation ◽  
Answer Extraction ◽  
Question Answering Systems

AbstractResearch on question answering dates back to the 1960s but has more recently been revisited as part of TREC's evaluation campaigns, where question answering is addressed as a subarea of information retrieval that focuses on specific answers to a user's information need. Whereas document retrieval systems aim to return the documents that are most relevant to a user's query, question answering systems aim to return actual answers to a users question. Despite this difference, question answering systems rely on information retrieval components to identify documents that contain an answer to a user's question. The computationally more expensive answer extraction methods are then applied only to this subset of documents that are likely to contain an answer. As information retrieval methods are used to filter the documents in the collection, the performance of this component is critical as documents that are not retrieved are not analyzed by the answer extraction component. The formulation of queries that are used for retrieving those documents has a strong impact on the effectiveness of the retrieval component. In this paper, we focus on predicting the importance of terms from the original question. We use model tree machine learning techniques in order to assign weights to query terms according to their usefulness for identifying documents that contain an answer. Term weights are learned by inspecting a large number of query formulation variations and their respective accuracy in identifying documents containing an answer. Several linguistic features are used for building the models, including part-of-speech tags, degree of connectivity in the dependency parse tree of the question, and ontological information. All of these features are extracted automatically by using several natural language processing tools. Incorporating the learned weights into a state-of-the-art retrieval system results in statistically significant improvements in identifying answer-bearing documents.

LIS4: Lesk Inspired Sense Specific Semantic Similarity using WordNet

2021 ◽  
pp. 2150006
Author(s):  
Saravanakumar Kandasamy ◽  
Aswani Kumar Cherukuri
Keyword(s):  
Information Retrieval ◽  
Natural Language Processing ◽  
Natural Language ◽  
Semantic Similarity ◽  
Language Processing ◽  
Gold Standard ◽  
Question Answering ◽  
Knowledge Based ◽  
Benchmark Datasets ◽  
Processing Information

Semantic similarity quantification between concepts is one of the inevitable parts in domains like Natural Language Processing, Information Retrieval, Question Answering, etc. to understand the text and their relationships better. Last few decades, many measures have been proposed by incorporating various corpus-based and knowledge-based resources. WordNet and Wikipedia are two of the Knowledge-based resources. The contribution of WordNet in the above said domain is enormous due to its richness in defining a word and all of its relationship with others. In this paper, we proposed an approach to quantify the similarity between concepts that exploits the synsets and the gloss definitions of different concepts using WordNet. Our method considers the gloss definitions, contextual words that are helping in defining a word, synsets of contextual word and the confidence of occurrence of a word in other word’s definition for calculating the similarity. The evaluation based on different gold standard benchmark datasets shows the efficiency of our system in comparison with other existing taxonomical and definitional measures.

Artificial Intelligence in Software Engineering

2012 ◽  
pp. 1215-1236 ◽  
Author(s):  
Farid Meziane ◽  
Sunil Vadera
Keyword(s):  
Neural Networks ◽  
Fuzzy Logic ◽  
Software Engineering ◽  
Software Development ◽  
Language Processing ◽  
Knowledge Based ◽  
Development Life Cycle ◽  
Planning Methods ◽  
Software Development Life Cycle ◽  
Processing Techniques

Artificial intelligences techniques such as knowledge based systems, neural networks, fuzzy logic and data mining have been advocated by many researchers and developers as the way to improve many of the software development activities. As with many other disciplines, software development quality improves with the experience, knowledge of the developers, past projects and expertise. Software also evolves as it operates in changing and volatile environments. Hence, there is significant potential for using AI for improving all phases of the software development life cycle. This chapter provides a survey on the use of AI for software engineering that covers the main software development phases and AI methods such as natural language processing techniques, neural networks, genetic algorithms, fuzzy logic, ant colony optimization, and planning methods.

Artificial Intelligence in Software Engineering

2010 ◽  
pp. 278-299 ◽  
Author(s):  
Farid Meziane ◽  
Sunil Vadera
Keyword(s):  
Neural Networks ◽  
Fuzzy Logic ◽  
Software Engineering ◽  
Software Development ◽  
Language Processing ◽  
Knowledge Based ◽  
Development Life Cycle ◽  
Planning Methods ◽  
Software Development Life Cycle ◽  
Processing Techniques

Artificial intelligences techniques such as knowledge based systems, neural networks, fuzzy logic and data mining have been advocated by many researchers and developers as the way to improve many of the software development activities. As with many other disciplines, software development quality improves with the experience, knowledge of the developers, past projects and expertise. Software also evolves as it operates in changing and volatile environments. Hence, there is significant potential for using AI for improving all phases of the software development life cycle. This chapter provides a survey on the use of AI for software engineering that covers the main software development phases and AI methods such as natural language processing techniques, neural networks, genetic algorithms, fuzzy logic, ant colony optimization, and planning methods.

Artificial Intelligence and Investing

2011 ◽  
pp. 237-240 ◽  
Author(s):  
Roy Rada
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Risk Management ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Asset Valuation ◽  
Stock Valuation ◽  
Knowledge Based ◽  
Processing Techniques

The techniques of artificial intelligence include knowledgebased, machine learning, and natural language processing techniques. The discipline of investing requires data identification, asset valuation, and risk management. Artificial intelligence techniques apply to many aspects of financial investing, and published work has shown an emphasis on the application of knowledge-based techniques for credit risk assessment and machine learning techniques for stock valuation. However, in the future, knowledge-based, machine learning, and natural language processing techniques will be integrated into systems that simultaneously address data identification, asset valuation, and risk management.

CONTENT CLASSIFICATION AND RECOMMENDATION TECHNIQUES FOR VIEWING ELECTRONIC PROGRAMMING GUIDE ON A PORTABLE DEVICE

2007 ◽  
Vol 21 (02) ◽  
pp. 375-395 ◽  
Author(s):  
JINGBO ZHU ◽  
MATTHEW Y. MA ◽  
JINHONG K. GUO ◽  
ZHENXING WANG
Keyword(s):  
Language Processing ◽  
Training Data ◽  
Home Network ◽  
Digital Television ◽  
Amount Of Information ◽  
Retrieval Rate ◽  
Content Classification ◽  
Small Set ◽  
The Right ◽  
Processing Techniques

With the merge of digital television (DTV) and the exponential growth of broadcasting network, an overwhelmingly amount of information has been made available to a consumer's home. Therefore, how to provide consumers with the right amount of information becomes a challenging problem. In this paper, we propose an electronic programming guide (EPG) recommender based on natural language processing techniques, more specifically, text classification. This recommender has been implemented as a service on a home network that facilitates the personalized browsing and recommendation of TV programs on a portable remote device. Evaluations of our Maximum Entropy text classifier were performed on multiple categories of TV programs, and a near 80% retrieval rate is achieved using a small set of training data.

scholarly journals Unsupervised Event Coreference Resolution

2014 ◽  
Vol 40 (2) ◽  
pp. 311-347 ◽  
Author(s):  
Cosmin Bejan ◽  
Sanda Harabagiu
Keyword(s):  
Hybrid Model ◽  
Language Processing ◽  
Dirichlet Process ◽  
Question Answering ◽  
Latent Class ◽  
Latent Class Model ◽  
Critical Role ◽  
Coreference Resolution ◽  
Semantic Features ◽  
Topic Detection And Tracking

The task of event coreference resolution plays a critical role in many natural language processing applications such as information extraction, question answering, and topic detection and tracking. In this article, we describe a new class of unsupervised, nonparametric Bayesian models with the purpose of probabilistically inferring coreference clusters of event mentions from a collection of unlabeled documents. In order to infer these clusters, we automatically extract various lexical, syntactic, and semantic features for each event mention from the document collection. Extracting a rich set of features for each event mention allows us to cast event coreference resolution as the task of grouping together the mentions that share the same features (they have the same participating entities, share the same location, happen at the same time, etc.). Some of the most important challenges posed by the resolution of event coreference in an unsupervised way stem from (a) the choice of representing event mentions through a rich set of features and (b) the ability of modeling events described both within the same document and across multiple documents. Our first unsupervised model that addresses these challenges is a generalization of the hierarchical Dirichlet process. This new extension presents the hierarchical Dirichlet process's ability to capture the uncertainty regarding the number of clustering components and, additionally, takes into account any finite number of features associated with each event mention. Furthermore, to overcome some of the limitations of this extension, we devised a new hybrid model, which combines an infinite latent class model with a discrete time series model. The main advantage of this hybrid model stands in its capability to automatically infer the number of features associated with each event mention from data and, at the same time, to perform an automatic selection of the most informative features for the task of event coreference. The evaluation performed for solving both within- and cross-document event coreference shows significant improvements of these models when compared against two baselines for this task.

scholarly journals Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge

2022 ◽  
Vol 40 (1) ◽  
pp. 1-33
Author(s):  
Yang Deng ◽  
Yuexiang Xie ◽  
Yaliang Li ◽  
Min Yang ◽  
Wai Lam ◽  
...  
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Question Answering ◽  
Representation Learning ◽  
Attention Mechanism ◽  
Convolutional Network ◽  
Structure Information ◽  
Dialog Systems ◽  
External Knowledge ◽  
Knowledge Based

Answer selection, which is involved in many natural language processing applications, such as dialog systems and question answering (QA), is an important yet challenging task in practice, since conventional methods typically suffer from the issues of ignoring diverse real-world background knowledge. In this article, we extensively investigate approaches to enhancing the answer selection model with external knowledge from knowledge graph (KG). First, we present a context-knowledge interaction learning framework, Knowledge-aware Neural Network, which learns the QA sentence representations by considering a tight interaction with the external knowledge from KG and the textual information. Then, we develop two kinds of knowledge-aware attention mechanism to summarize both the context-based and knowledge-based interactions between questions and answers. To handle the diversity and complexity of KG information, we further propose a Contextualized Knowledge-aware Attentive Neural Network, which improves the knowledge representation learning with structure information via a customized Graph Convolutional Network and comprehensively learns context-based and knowledge-based sentence representation via the multi-view knowledge-aware attention mechanism. We evaluate our method on four widely used benchmark QA datasets, including WikiQA, TREC QA, InsuranceQA, and Yahoo QA. Results verify the benefits of incorporating external knowledge from KG and show the robust superiority and extensive applicability of our method.

scholarly journals A Knowledge-Based Sense Disambiguation Method to Semantically Enhanced NL Question for Restricted Domain

Information ◽  
2021 ◽  
Vol 12 (11) ◽  
pp. 452
Author(s):  
Ammar Arbaaeen ◽  
Asadullah Shah
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Word Sense Disambiguation ◽  
Knowledge Bases ◽  
Word Sense ◽  
Intended Meaning ◽  
Lexical Semantic ◽  
Knowledge Based ◽  
Sense Disambiguation

Within the space of question answering (QA) systems, the most critical module to improve overall performance is question analysis processing. Extracting the lexical semantic of a Natural Language (NL) question presents challenges at syntactic and semantic levels for most QA systems. This is due to the difference between the words posed by a user and the terms presently stored in the knowledge bases. Many studies have achieved encouraging results in lexical semantic resolution on the topic of word sense disambiguation (WSD), and several other works consider these challenges in the context of QA applications. Additionally, few scholars have examined the role of WSD in returning potential answers corresponding to particular questions. However, natural language processing (NLP) is still facing several challenges to determine the precise meaning of various ambiguities. Therefore, the motivation of this work is to propose a novel knowledge-based sense disambiguation (KSD) method for resolving the problem of lexical ambiguity associated with questions posed in QA systems. The major contribution is the proposed innovative method, which incorporates multiple knowledge sources. This includes the question’s metadata (date/GPS), context knowledge, and domain ontology into a shallow NLP. The proposed KSD method is developed into a unique tool for a mobile QA application that aims to determine the intended meaning of questions expressed by pilgrims. The experimental results reveal that our method obtained comparable and better accuracy performance than the baselines in the context of the pilgrimage domain.

Sign in / Sign up

Export Citation Format

Share Document