ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data

Contextualized representations trained over large raw text data have given remarkable improvements for NLP tasks including question answering and reading comprehension. There have been works showing that syntactic, semantic and word sense knowledge are contained in such representations, which explains why they benefit such tasks. However, relatively little work has been done investigating commonsense knowledge contained in contextualized representations, which is crucial for human question answering and reading comprehension. We study the commonsense ability of GPT, BERT, XLNet, and RoBERTa by testing them on seven challenging benchmarks, finding that language modeling and its variants are effective objectives for promoting models' commonsense ability while bi-directional context and larger training set are bonuses. We additionally find that current models do poorly on tasks require more necessary inference steps. Finally, we test the robustness of models by making dual test cases, which are correlated so that the correct prediction of one sample should lead to correct prediction of the other. Interestingly, the models show confusion on these test cases, which suggests that they learn commonsense at the surface rather than the deep level. We release a test set, named CATs publicly, for future research.

Download Full-text

Question answering system for deterministic fault diagnosis of intelligent railway signal equipment

Smart and Resilient Transport ◽

10.1108/srt-10-2020-0016 ◽

2021 ◽

Vol ahead-of-print (ahead-of-print) ◽

Author(s):

Haosen Liu ◽

Youwei Wang ◽

Xiabing Zhou ◽

Zhengzheng Lou ◽

Yangdong Ye

Keyword(s):

Question Answering ◽

Failure Diagnosis ◽

Semantic Matching ◽

Equipment Failure ◽

Text Data ◽

Data Set ◽

Content Type ◽

Question Answering System ◽

Railway Signal ◽

Matching Mechanism

Purpose The railway signal equipment failure diagnosis is a vital element to keep the railway system operating safely. One of the most difficulties in signal equipment failure diagnosis is the uncertainty of causality between the consequence and cause for the accident. The traditional method to solve this problem is based on Bayesian Network, which needs a rigid and independent assumption basis and prior probability knowledge but ignoring the semantic relationship in causality analysis. This paper aims to perform the uncertainty of causality in signal equipment failure diagnosis through a new way that emphasis on mining semantic relationships. Design/methodology/approach This study proposes a deterministic failure diagnosis (DFD) model based on the question answering system to implement railway signal equipment failure diagnosis. It includes the failure diagnosis module and deterministic diagnosis module. In the failure diagnosis module, this paper exploits the question answering system to recognise the cause of failure consequences. The question answering is composed of multi-layer neural networks, which extracts the position and part of speech features of text data from lower layers and acquires contextual features and interactive features of text data by Bi-LSTM and Match-LSTM, respectively, from high layers, subsequently generates the candidate failure cause set by proposed the enhanced boundary unit. In the second module, this study ranks the candidate failure cause set in the semantic matching mechanism (SMM), choosing the top 1st semantic matching degree as the deterministic failure causative factor. Findings Experiments on real data set railway maintenance signal equipment show that the proposed DFD model can implement the deterministic diagnosis of railway signal equipment failure. Comparing massive existing methods, the model achieves the state of art in the natural understanding semantic of railway signal equipment diagnosis domain. Originality/value It is the first time to use a question answering system executing signal equipment failure diagnoses, which makes failure diagnosis more intelligent than before. The EMU enables the DFD model to understand the natural semantic in long sequence contexture. Then, the SMM makes the DFD model acquire the certainty failure cause in the failure diagnosis of railway signal equipment.

Download Full-text

Where Do Their Answers Come from? A Study of the Inferences which Children Make when Answering Questions about Narrative Stories

Journal of Reading Behavior ◽

10.1080/10862968109547400 ◽

1981 ◽

Vol 13 (2) ◽

pp. 111-129 ◽

Cited By ~ 4

Author(s):

Tom Nicholson ◽

Robert Imlach

Keyword(s):

Repeated Measures ◽

Text Comprehension ◽

Question Answering ◽

Text Structure ◽

Narrative Texts ◽

Text Data ◽

Comprehension Process ◽

The Mind ◽

Inside Out ◽

Answering Questions

Young readers often seem to overlook explicitly stated causal statements In narrative texts and instead give their own versions of why a text event occurred. Some researchers would agree with Smith (1979) that children do this because they read for meaning rather than word-by-word. This is an “inside-out” (or, “schema-based”) view of text comprehension. Other researchers, however, agree with Thorndike (1917) that “errors” occur because “the mind is assailed by every word in the paragraph.” This is an “outside-in” (or, “text-based”) view of the comprehension process. The purpose of this study was to find out the relative influence of text data and prior knowledge on the kinds of inferences which children make when answering questions about stories. In Experiment 1, text structure was altered by embedding either predictable or unpredictable reasons for events in the text, and also by varying the position and distance of these reasons from the text event being asked about. Some of these stories were familiar; others less so. Text accessibility was also varied. In all, the design was a 24 × 3 factorial, using repeated measures. In Experiment 2, a causal “preference” factor was added, to take account of the fact that children seemed predisposed toward certain kinds of inferences, whether they are predictable or not. The results provide support for the notion that text data and background knowledge compete for priority in question-answering. They suggest that children may benefit from instruction which helps them to arbitrate between plausible, yet competing explanations for important text events.

Download Full-text

Vis Quelle: Visual Question based Elementary Learning Companion a system to Facilitate Learning Word Object Associations

International Journal of Innovative Technology and Exploring Engineering - Special Issue ◽

10.35940/ijitee.a9599.1111121 ◽

2021 ◽

Vol 11 (1) ◽

pp. 41-49

Author(s):

Sandhya Vidyashankar ◽

◽

Rakshit Vahi ◽

Yash Karkhanis ◽

Gowri Srinivasa ◽

...

Keyword(s):

Machine Learning ◽

Second Language ◽

Object Recognition ◽

Question Answering ◽

Learning Models ◽

Text Data ◽

Visual Question Answering ◽

Machine Learning Models ◽

Native Tongue

We present an automated, visual question answering based companion – VisQuelle - to facilitate elementary learning of word-object associations. In particular, we attempt to harness the power of machine learning models for object recognition and the understanding of combined processing of images and text data from visual-question answering to provide variety and nuance in the images associated with letters or words presented to the elementary learner. We incorporate elements such as gamification to motivate the learner by recording scores, errors, etc., to track the learner’s progress. Translation is also provided to reinforce word-object associations in the user’s native tongue, if the learner is using VisQuelle to learn a second language.

Download Full-text

ACTSMLT: Automatic Classification of Text Summarization using Machine Learning Technique

International Journal of Engineering and Advanced Technology - Regular Issue ◽

10.35940/ijeat.b2993.129219 ◽

2019 ◽

Vol 9 (2) ◽

pp. 5445-5457

Keyword(s):

Machine Learning ◽

Question Answering ◽

Multinomial Logistic Regression ◽

Machine Learning Algorithms ◽

Support Vector ◽

Retrieval Task ◽

Text Data ◽

Common Information ◽

Community Question Answering ◽

Internet Users

In today’s world, due to the steep rise in internet users, Community Question Answering (CQA) has attracted many research communities. In order to provide the correct and perfect answer to the user asked question from a given large collection of text data, understanding the question properly to suggest a precise answer is a challenging task. Therefore, Question Answering (QA) system is a challenging task than a common information retrieval task done by many search engines. In this paper, an automatic prediction of the quality of CQA answers is proposed. This is accomplished by using five well known machine learning algorithms. Usually, questions asked by the user are based on a topic or theme. We try to exploit this feature in our work by identifying the category of the question posted and further map with the corresponding question. Similarly, for the answers posted by the multiple user’s are processed as answer for category mapping. Here, the results show that for Question Classification (QA), Linear Support Vector Classification (LSVC) is found to be the best classifier and Multinomial Logistic Regression (MLR) is the most suitable for Answer Classification (AC). The MS Macro dataset is used as the underlying dataset for retrieving and testing the question and answer classifiers. The Yahoo Answers are used as a golden reference during the testing throughout our experiments. Experiments results show that the proposed technique is efficient and outperforms Metzler and Kanungo’s (MK++) [1] while providing the best answer summary satisfying the user’s queries.

Download Full-text

A Numerically Coded File of Operative Procedures Derived from a Free Text Data Collection System : A Measure of the Accuracy

Methods of Information in Medicine ◽

10.1055/s-0038-1635717 ◽

1976 ◽

Vol 15 (01) ◽

pp. 21-28 ◽

Cited By ~ 3

Author(s):

Carmen A. Scudiero ◽

Ruth L. Wong

Keyword(s):

Data Collection ◽

Pap Smear ◽

Operative Procedures ◽

Free Text ◽

Collection System ◽

Process Data ◽

Text Data ◽

Data Collection System ◽

History Of ◽

Correlation System

A free text data collection system has been developed at the University of Illinois utilizing single word, syntax free dictionary lookup to process data for retrieval. The source document for the system is the Surgical Pathology Request and Report form. To date 12,653 documents have been entered into the system.The free text data was used to create an IRS (Information Retrieval System) database. A program to interrogate this database has been developed to numerically coded operative procedures. A total of 16,519 procedures records were generated. One and nine tenths percent of the procedures could not be fitted into any procedures category; 6.1% could not be specifically coded, while 92% were coded into specific categories. A system of PL/1 programs has been developed to facilitate manual editing of these records, which can be performed in a reasonable length of time (1 week). This manual check reveals that these 92% were coded with precision = 0.931 and recall = 0.924. Correction of the readily correctable errors could improve these figures to precision = 0.977 and recall = 0.987. Syntax errors were relatively unimportant in the overall coding process, but did introduce significant error in some categories, such as when right-left-bilateral distinction was attempted.The coded file that has been constructed will be used as an input file to a gynecological disease/PAP smear correlation system. The outputs of this system will include retrospective information on the natural history of selected diseases and a patient log providing information to the clinician on patient follow-up.Thus a free text data collection system can be utilized to produce numerically coded files of reasonable accuracy. Further, these files can be used as a source of useful information both for the clinician and for the medical researcher.

Download Full-text

Diagnostics of professional competence of IT students based on digital footprint data

Informatics and Education ◽

10.32517/0234-0453-2020-35-4-4-11 ◽

2020 ◽

pp. 4-11

Author(s):

I. G. Zakharova ◽

Yu. V. Boganyuk ◽

M. S. Vorobyova ◽

E. A. Pavlova

Keyword(s):

Information Technology ◽

Educational Program ◽

Professional Competence ◽

Objective Data ◽

Text Data ◽

Data Set ◽

Job Requirements ◽

Digital Footprint ◽

Graduate Employment ◽

The University

The article goal is to demonstrate the possibilities of the approach to diagnosing the level of IT graduates’ professional competence, based on the analysis of the student’s digital footprint and the content of the corresponding educational program. We describe methods for extracting student professional level indicators from digital footprint text data — courses’ descriptions and graduation qualification works. We show methods of comparing these indicators with the formalized requirements of employers, reflected in the texts of vacancies in the field of information technology. The proposed approach was applied at the Institute of Mathematics and Computer Science of the University of Tyumen. We performed diagnostics using a data set that included texts of courses’ descriptions for IT areas of undergraduate studies, 542 graduation qualification works in these areas, 879 descriptions of job requirements and information on graduate employment. The presented approach allows us to evaluate the relevance of the educational program as a whole and the level of professional competence of each student based on objective data. The results were used to update the content of some major courses and to include new elective courses in the curriculum.

Download Full-text

Reports of the Workshops Held at the 2019 AAAI Conference on Artificial Intelligence

AI Magazine ◽

10.1609/aimag.v40i3.4981 ◽

2019 ◽

Vol 40 (3) ◽

pp. 67-78

Author(s):

Guy Barash ◽

Mauricio Castillo-Effen ◽

Niyati Chhaya ◽

Peter Clark ◽

Huáscar Espinoza ◽

...

Keyword(s):

Artificial Intelligence ◽

Language Processing ◽

Cyber Security ◽

Question Answering ◽

Intent Recognition ◽

Affective Content ◽

Learning Plan ◽

Dialog System ◽

Affective Content Analysis ◽

Games And Simulations

The workshop program of the Association for the Advancement of Artificial Intelligence’s 33rd Conference on Artificial Intelligence (AAAI-19) was held in Honolulu, Hawaii, on Sunday and Monday, January 27–28, 2019. There were fifteen workshops in the program: Affective Content Analysis: Modeling Affect-in-Action, Agile Robotics for Industrial Automation Competition, Artificial Intelligence for Cyber Security, Artificial Intelligence Safety, Dialog System Technology Challenge, Engineering Dependable and Secure Machine Learning Systems, Games and Simulations for Artificial Intelligence, Health Intelligence, Knowledge Extraction from Games, Network Interpretability for Deep Learning, Plan, Activity, and Intent Recognition, Reasoning and Learning for Human-Machine Dialogues, Reasoning for Complex Question Answering, Recommender Systems Meet Natural Language Processing, Reinforcement Learning in Games, and Reproducible AI. This report contains brief summaries of the all the workshops that were held.

Download Full-text