Refining Automatically Extracted Knowledge Bases Using Crowdsourcing

Machine-constructed knowledge bases often contain noisy and inaccurate facts. There exists significant work in developing automated algorithms for knowledge base refinement. Automated approaches improve the quality of knowledge bases but are far from perfect. In this paper, we leverage crowdsourcing to improve the quality of automatically extracted knowledge bases. As human labelling is costly, an important research challenge is how we can use limited human resources to maximize the quality improvement for a knowledge base. To address this problem, we first introduce a concept of semantic constraints that can be used to detect potential errors and do inference among candidate facts. Then, based on semantic constraints, we propose rank-based and graph-based algorithms for crowdsourced knowledge refining, which judiciously select the most beneficial candidate facts to conduct crowdsourcing and prune unnecessary questions. Our experiments show that our method improves the quality of knowledge bases significantly and outperforms state-of-the-art automatic methods under a reasonable crowdsourcing cost.

Knowledge Base Refinement Using Limited Amount of Efforts from Experts

Knowledge bases are essential for supporting decision making during intelligent information processing. Automatic construction of knowledge bases becomes infeasible without labeled data, a complete table of data records including answers to queries. Preparing such information requires huge efforts from experts. The authors propose a new knowledge base refinement framework based on pattern mining and active learning using an existing available knowledge base constructed from a different domain (source domain) solving the same task as well as some data collected from the target domain. The knowledge base investigated in this paper is represented by a model known as Markov Logic Networks. The authors' proposed method first analyzes the unlabeled target domain data and actively asks the expert to provide labels (or answers) a very small amount of automatically selected queries. The idea is to identify the target domain queries whose underlying relations are not sufficiently described by the existing source domain knowledge base. Potential relational patterns are discovered and new logic relations are constructed for the target domain by exploiting the limited amount of labeled target domain data and the unlabeled target domain data. The authors have conducted extensive experiments by applying our approach to two different text mining applications, namely, pronoun resolution and segmentation of citation records, demonstrating consistent improvements.

S-SALT: A problem-solver, knowledge acquisition tool and associated knowledge base refinement mechanisms

SALT is the highly influential system implemented by Marcus (1988), which is able to acquire knowledge and solve problems using the propose-and-revise strategy. The primary task to which this has been applied is the design of elevators, or lifts (Marcus et al., 1988). Recently we have re-implemented and extended the system at Aberdeen, S-SALT, the new system, has the following enhancements: