Chapter 5. Answering Natural-Language Questions with Neuro-Symbolic Knowledge Bases

Symbolic reasoning systems based on first-order logics are computationally powerful, and feedforward neural networks are computationally efficient, so unless P=NP, neural networks cannot, in general, emulate symbolic logics. Hence bridging the gap between neural and symbolic methods requires achieving a delicate balance: one needs to incorporate just enough of symbolic reasoning to be useful for a task, but not so much as to cause computational intractability. In this chapter we first present results that make this claim precise, and then use these formal results to inform the choice of a neuro-symbolic knowledge-based reasoning system, based on a set-based dataflow query language. We then present experimental results with a number of variants of this neuro-symbolic reasoner, and also show that this neuro-symbolic reasoner can be closely integrated into modern neural language models.

Chapter 11. Neuro-Symbolic Semantic Reasoning

Humans have astounding reasoning capabilities. They can learn from very few examples while providing explanations for their decision-making process. In contrast, deep learning techniques–even though robust to noise and very effective in generalizing across several fields including machine vision, natural language understanding, speech recognition, etc. –require large amounts of data and are mostly unable to provide explanations for their decisions. Attaining human-level robust reasoning requires combining sound symbolic reasoning with robust connectionist learning. However, connectionist learning uses low-level representations–such as embeddings–rather than symbolic representations. This challenge constitutes what is referred to as the Neuro-Symbolic gap. A field of study to bridge this gap between the two paradigms has been called neuro-symbolic integration or neuro-symbolic computing. This chapter aims to present approaches that contribute towards bridging the Neuro-Symbolic gap specifically in the Semantic Web field, RDF Schema (RDFS) and EL+ reasoning and to discuss the benefits and shortcomings of neuro-symbolic reasoning.

Neurules and connectionist expert systems: Unexplored neuro-symbolic reasoning aspects

Neuro-symbolic approaches combine neural and symbolic methods. This paper explores aspects regarding the reasoning mechanisms of two neuro-symbolic approaches, that is, neurules and connectionist expert systems. Both provide reasoning and explanation facilities. Neurules are a type of neuro-symbolic rules tightly integrating the neural and symbolic components, giving pre-eminence to the symbolic component. Connectionist expert systems give pre-eminence to the connectionist component. This paper explores reasoning aspects about neurules and connectionist expert systems that have not been previously addressed. As far as neurules are concerned, an aspect playing a role in conflict resolution (i.e., order of neurules) is explored. Experimental results show an improvement in reasoning efficiency. As far as connectionist expert systems are concerned, variations of the reasoning mechanism are explored. Experimental results are presented for them as well showing that one of the variations generally performs better than the others.

Undergraduate Students’ Combinatorial Proof of Binomial Identities

Combinatorial proof serves both as an important topic in combinatorics and as a type of proof with certain properties and constraints. We report on a teaching experiment in which undergraduate students (who were novice provers) engaged in combinatorial reasoning as they proved binomial identities. We highlight ways of understanding that were important for their success with establishing combinatorial arguments; in particular, the students demonstrated referential symbolic reasoning within an enumerative representation system, and as the students engaged in successful combinatorial proof, they had to coordinate reasoning within algebraic and enumerative representation systems. We illuminate features of the students’ work that potentially contributed to their successes and highlight potential issues that students may face when working with binomial identities.

Coarsening optimization for differentiable programming

This paper presents a novel optimization for differentiable programming named coarsening optimization. It offers a systematic way to synergize symbolic differentiation and algorithmic differentiation (AD). Through it, the granularity of the computations differentiated by each step in AD can become much larger than a single operation, and hence lead to much reduced runtime computations and data allocations in AD. To circumvent the difficulties that control flow creates to symbolic differentiation in coarsening, this work introduces phi-calculus, a novel method to allow symbolic reasoning and differentiation of computations that involve branches and loops. It further avoids "expression swell" in symbolic differentiation and balance reuse and coarsening through the design of reuse-centric segment of interest identification. Experiments on a collection of real-world applications show that coarsening optimization is effective in speeding up AD, producing several times to two orders of magnitude speedups.

Compositional Neural Logic Programming

This paper introduces Compositional Neural Logic Programming (CNLP), a framework that integrates neural networks and logic programming for symbolic and sub-symbolic reasoning. We adopt the idea of compositional neural networks to represent first-order logic predicates and rules. A voting backward-forward chaining algorithm is proposed for inference with both symbolic and sub-symbolic variables in an argument-retrieval style. The framework is highly flexible in that it can be constructed incrementally with new knowledge, and it also supports batch reasoning in certain cases. In the experiments, we demonstrate the advantages of CNLP in discriminative tasks and generative tasks.

The Nature of Information, Semantics, and Effectiveness for Artificial Intelligence and Cognition

This manuscript puts forward claims to help address foundational gaps in understanding Cognition and Artificial General Intelligence (AGI), including the nature of Emergence, Semantics, and Information. This includes criteria for assessing true understanding in AI models. How symbolic reasoning conceptualizes phenomena is described. Without a subsymbolic perceptual level to generate concepts, there is no symbol grounding. Grounding requires dynamics outside of its own symbolization. Grounding forms the set of symbols used at the conceptual level. It is claimed that this role explains Semantics. This approach naturally leads to established research on Conceptual Spaces and has implications for Semantic Vector Spaces learned via Neural Embedding methods. It also has implications for Information Theories. A claim is made that Semantic Processes form Shannon-like microstates and macrostates, while Effective Processes constrain Semantic Processes. Unlike existing Semantic Information Theories, Semantic Processes are pre-informational. The claims provide perspective on the Mind. It is natural to conflate percepts with the modified version necessarily created when conceptualizing through explication. The ‘Hard Problem of Consciousness’ is related to this Percept/Concept distinction. Concepts are always subject to Eliminative Materialism. The nonconceptual properties of Percepts cannot be eliminated. Intrinsic are Extrinsic Emergence are distinguished. It is common to assume extrinsic emergent properties are intrinsic to the systems evoking them. This presents a challenge for proving intrinsic emergence in AI. However, criteria are proposed for claiming a theoretical system intrinsically processes information and grounds symbols. By leveraging the functional properties of Grounding, the criteria can be considered for actual systems.