scholarly journals A Lexical Knowledge Representation Model for Natural Language Understanding

2012 ◽  
pp. 193-210
Author(s):  
Ping Chen ◽  
Wei Ding ◽  
Chengmin Ding
Keyword(s):  
Knowledge Representation ◽  
Natural Language ◽  
Language Processing ◽  
Large Scale ◽  
Natural Language Understanding ◽  
Commonsense Reasoning ◽  
Language Understanding ◽  
Representation Model ◽  
Intelligent Information ◽  
Representation Techniques

Knowledge representation is essential for semantics modeling and intelligent information processing. For decades researchers have proposed many knowledge representation techniques. However, it is a daunting problem how to capture deep semantic information effectively and support the construction of a large-scale knowledge base efficiently. This paper describes a new knowledge representation model, SenseNet, which provides semantic support for commonsense reasoning and natural language processing. SenseNet is formalized with a Hidden Markov Model. An inference algorithm is proposed to simulate human-like natural language understanding procedure. A new measurement, confidence, is introduced to facilitate the natural language understanding. The authors present a detailed case study of applying SenseNet to retrieving compensation information from company proxy filings.

scholarly journals A Lexical Knowledge Representation Model for Natural Language Understanding

2009 ◽  
Vol 1 (4) ◽  
pp. 17-35 ◽  
Author(s):  
Ping Chen ◽  
Wei Ding ◽  
Chengmin Ding
Keyword(s):  
Knowledge Representation ◽  
Natural Language ◽  
Language Processing ◽  
Large Scale ◽  
Natural Language Understanding ◽  
Commonsense Reasoning ◽  
Lexical Knowledge ◽  
Language Understanding ◽  
Representation Model ◽  
Intelligent Information

Knowledge representation is essential for semantics modeling and intelligent information processing. For decades researchers have proposed many knowledge representation techniques. However, it is a daunting problem how to capture deep semantic information effectively and support the construction of a large-scale knowledge base efficiently. This article describes a new knowledge representation model, SenseNet, which provides semantic support for commonsense reasoning and natural language processing. SenseNet is formalized with a Hidden Markov Model. An inference algorithm is proposed to simulate human-like natural language understanding procedure. A new measurement, confidence, is introduced to facilitate the natural language understanding. The authors present a detailed case study of applying SenseNet to retrieving compensation information from company proxy filings.

Guru

2012 ◽  
pp. 156-171 ◽  
Author(s):  
Andrew M. Olney ◽  
Natalie K. Person ◽  
Arthur C. Graesser
Keyword(s):  
Natural Language Processing ◽  
Knowledge Representation ◽  
Natural Language ◽  
Language Processing ◽  
Natural Language Understanding ◽  
Natural Language Generation ◽  
Language Understanding ◽  
Language Generation ◽  
Processing Techniques

The authors discuss Guru, a conversational expert ITS. Guru is designed to mimic expert human tutors using advanced applied natural language processing techniques including natural language understanding, knowledge representation, and natural language generation.

scholarly journals A Sequential and Intensive Weighted Language Modeling Scheme for Multi-Task Learning-Based Natural Language Understanding

2021 ◽  
Vol 11 (7) ◽  
pp. 3095
Author(s):  
Suhyune Son ◽  
Seonjeong Hwang ◽  
Sohyeun Bae ◽  
Soo Jun Park ◽  
Jang-Hwan Choi
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Empirical Investigation ◽  
Natural Language Understanding ◽  
Language Modeling ◽  
Language Understanding ◽  
Task Learning ◽  
Language Representation ◽  
Internal Transfer ◽  
Two Stages

Multi-task learning (MTL) approaches are actively used for various natural language processing (NLP) tasks. The Multi-Task Deep Neural Network (MT-DNN) has contributed significantly to improving the performance of natural language understanding (NLU) tasks. However, one drawback is that confusion about the language representation of various tasks arises during the training of the MT-DNN model. Inspired by the internal-transfer weighting of MTL in medical imaging, we introduce a Sequential and Intensive Weighted Language Modeling (SIWLM) scheme. The SIWLM consists of two stages: (1) Sequential weighted learning (SWL), which trains a model to learn entire tasks sequentially and concentrically, and (2) Intensive weighted learning (IWL), which enables the model to focus on the central task. We apply this scheme to the MT-DNN model and call this model the MTDNN-SIWLM. Our model achieves higher performance than the existing reference algorithms on six out of the eight GLUE benchmark tasks. Moreover, our model outperforms MT-DNN by 0.77 on average on the overall task. Finally, we conducted a thorough empirical investigation to determine the optimal weight for each GLUE task.

scholarly journals PIQA: Reasoning about Physical Commonsense in Natural Language

2020 ◽  
Vol 34 (05) ◽  
pp. 7432-7439
Author(s):  
Yonatan Bisk ◽  
Rowan Zellers ◽  
Ronan Le bras ◽  
Jianfeng Gao ◽  
Yejin Choi
Keyword(s):  
Natural Language ◽  
Question Answering ◽  
Natural Language Understanding ◽  
Physical World ◽  
Future Research ◽  
Physical Interaction ◽  
Commonsense Reasoning ◽  
Cotton Swab ◽  
Language Understanding ◽  
Abstract Domains

To apply eyeshadow without a brush, should I use a cotton swab or a toothpick? Questions requiring this kind of physical commonsense pose a challenge to today's natural language understanding systems. While recent pretrained models (such as BERT) have made progress on question answering over more abstract domains – such as news articles and encyclopedia entries, where text is plentiful – in more physical domains, text is inherently limited due to reporting bias. Can AI systems learn to reliably answer physical commonsense questions without experiencing the physical world?In this paper, we introduce the task of physical commonsense reasoning and a corresponding benchmark dataset Physical Interaction: Question Answering or PIQA. Though humans find the dataset easy (95% accuracy), large pretrained models struggle (∼75%). We provide analysis about the dimensions of knowledge that existing models lack, which offers significant opportunities for future research.

Sign in / Sign up

Export Citation Format

Share Document