Step-Wise Hierarchical Alignment Network for Image-Text Matching

Image-text matching plays a central role in bridging the semantic gap between vision and language. The key point to achieve precise visual-semantic alignment lies in capturing the fine-grained cross-modal correspondence between image and text. Most previous methods rely on single-step reasoning to discover the visual-semantic interactions, which lacks the ability of exploiting the multi-level information to locate the hierarchical fine-grained relevance. Different from them, in this work, we propose a step-wise hierarchical alignment network (SHAN) that decomposes image-text matching into multi-step cross-modal reasoning process. Specifically, we first achieve local-to-local alignment at fragment level, following by performing global-to-local and global-to-global alignment at context level sequentially. This progressive alignment strategy supplies our model with more complementary and sufficient semantic clues to understand the hierarchical correlations between image and text. The experimental results on two benchmark datasets demonstrate the superiority of our proposed method.

Nāgārjunian-Yogācārian Modal Logic versus Aristotelian Modal Logic

There are two different modal logics: the logic T assuming contingency and the logic K = assuming logical determinism. In the paper, I show that the Aristotelian treatise On Interpretation (Περί ερμηνείας, De Interpretatione) has introduced some modal-logical relationships which correspond to T. In this logic, it is supposed that there are contingent events. The Nāgārjunian treatise Īśvara-kartṛtva-nirākṛtiḥ-viṣṇoḥ-ekakartṛtva-nirākaraṇa has introduced some modal-logical relationships which correspond to K =. In this logic, it is supposed that there is a logical determinism: each event happens necessarily (siddha) or it does not happen necessarily (asiddha). The Nāgārjunian approach was inherited by the Yogācārins who developed, first, the doctrine of causality of all real entities (arthakriyātva) and, second, the doctrine of momentariness of all real entities (kṣaṇikavāda). Both doctrines were a philosophical ground of the Yogācārins for the logical determinism. Hence, Aristotle implicitly used the logic T in his modal reasoning. The Madhyamaka and Yogācāra schools implicitly used the logic K = in their modal reasoning.

A Formal Representation of Conjugate Verbs for Function Modeling

Modern design problems often require multi-modal, reconfigurable solutions. Function modeling is a common tool used to explore solutions in early mechanical design. Currently, function modeling formalisms minimally support the modeling of multi-modal systems in a formal manner. There is a need in function modeling to capture multi-modal system and analyze the effects of control signals and status signals on their operating modes. This paper presents the concept of functional conjugacy, where two function verbs or functional subgraphs are topological opposites of each other. The paper presents a formal representation of these conjugate verbs that formally captures the transition from one mode of operation to its topological opposite based on the existence of, or the value of, signal flows. Additionally, this paper extends functional conjugacy to functional features, which supports conjugacy-based reasoning at a higher level of abstraction. Through the example of a system-level function model of a geothermal heat pump operating in its heating and cooling modes, this paper demonstrates the ability to support modal reasoning on function models using functional conjugacy and illustrates the modeling efficacy of the extended representation.

Graphhopper: Multi-hop Scene Graph Reasoning for Visual Question Answering

Visual Question Answering (VQA) is concerned with answering free-form questions about an image. Since it requires a deep semantic and linguistic understanding of the question and the ability to associate it with various objects that are present in the image, it is an ambitious task and requires multi-modal reasoning from both computer vision and natural language processing. We propose Graphhopper, a novel method that approaches the task by integrating knowledge graph reasoning, computer vision, and natural language processing techniques. Concretely, our method is based on performing context-driven, sequential reasoning based on the scene entities and their semantic and spatial relationships. As a first step, we derive a scene graph that describes the objects in the image, as well as their attributes and their mutual relationships. Subsequently, a reinforcement learning agent is trained to autonomously navigate in a multi-hop manner over the extracted scene graph to generate reasoning paths, which are the basis for deriving answers. We conduct an experimental study on the challenging dataset GQA, based on both manually curated and automatically generated scene graphs. Our results show that we keep up with human performance on manually curated scene graphs. Moreover, we find that Graphhopper outperforms another state-of-the-art scene graph reasoning model on both manually curated and automatically generated scene graphs by a significant margin.

A Humean modal epistemology

I present an exemplary Humean modal epistemology. My version takes inspiration from but incurs no commitment to both Hume’s historical position and Lewis’s Humeanism. Modal epistemology should meet two challenges: the Integration challenge of integrating metaphysics and epistemology and the Reliability challenge of giving an account of how our epistemic capacities can be reliable in detecting modal truth. According to Lewis, modal reasoning starts from certain Humean principles: there is only the vast mosaic of spatiotemporally distributed local matters of fact. The facts can be arbitarily recombined. These principles cannot be taken for granted. I suggest a bottom-up approach instead: Humean principles of recombining the mosaic of facts can be retrieved from the evolutionarily instilled and empirically informed use of imagination in exploring everyday circumstantial possibilities. This use of imagination conforms to a primitive conception of matter as freely recombinable. The modal beliefs that can be obtained from generalizing the more elementary exercise of imagination have to be corrected. Recombination is limited by sortal criteria of identity. Moreover, the overall picture of a recombinable spatiotemporal mosaic must be weighed against the results of science.

Argumentation and counterfactual reasoning in Parmenides and Melissus

Parmenides and Melissus employ different deductive styles for their different kinds of argumentation. The former’s poem flows in an interesting sequence of passages: contents foreword, methodological premises, krisis, conclusions and corollaries. The latter, however, organizes an extensive process of deduction to show the characteristics of what is. In both cases, the strength of their argument rests on their deductive form, on the syntactical level of their texts: the formal structure of their reasonings help to identify the features and logical intersections of their thoughts. On the one hand, Parmenides uses modal reasoning, enforcing the employment of the principle of the excluded middle. On the other hand, Melissus radicalizes the use of modal reasoning and employs counterfactual statements in order to develop his doctrine of what is. Despite their differences, both deserve a place in the Stone Age of logic and theory of argumentation due to their common ambition to demonstrate what is.