The Study and Application of Object-oriented Hyper-graph Spatio-temporal Reasoning Model

Video Question Answering (Video QA) is a powerful testbed to develop new AI capabilities. This task necessitates learning to reason about objects, relations, and events across visual and linguistic domains in space-time. High-level reasoning demands lifting from associative visual pattern recognition to symbol like manipulation over objects, their behavior and interactions. Toward reaching this goal we propose an object-oriented reasoning approach in that video is abstracted as a dynamic stream of interacting objects. At each stage of the video event flow, these objects interact with each other, and their interactions are reasoned about with respect to the query and under the overall context of a video. This mechanism is materialized into a family of general-purpose neural units and their multi-level architecture called Hierarchical Object-oriented Spatio-Temporal Reasoning (HOSTR) networks. This neural model maintains the objects' consistent lifelines in the form of a hierarchically nested spatio-temporal graph. Within this graph, the dynamic interactive object-oriented representations are built up along the video sequence, hierarchically abstracted in a bottom-up manner, and converge toward the key information for the correct answer. The method is evaluated on multiple major Video QA datasets and establishes new state-of-the-arts in these tasks. Analysis into the model's behavior indicates that object-oriented reasoning is a reliable, interpretable and efficient approach to Video QA.

Download Full-text

Theories and Methods of Spatio-Temporal Reasoning in Geographic Space

10.1007/3-540-55966-3 ◽

1992 ◽

Cited By ~ 28

Keyword(s):

Temporal Reasoning ◽

Geographic Space ◽

Spatio Temporal

Download Full-text

DOOD'95 Post-Conference Workshop on Temporal Reasoning in Deductive and Object-Oriented Databases

The Knowledge Engineering Review ◽

10.1017/s0269888900007827 ◽

1996 ◽

Vol 11 (2) ◽

pp. 191-192 ◽

Cited By ~ 1

Author(s):

Stefan Conrad

Keyword(s):

Knowledge Discovery ◽

Object Oriented ◽

Temporal Reasoning ◽

International Conference ◽

First Time ◽

Object Oriented Databases

For the first time, post-conference workshops were organised for the International Conference on Deductive and Object-Oriented Databases (DOOD). There were two workshops focusing on knowledge discovery and temporal reasoning. This report is dedicated to one dealing with temporal reasoning.

Download Full-text

Study of the Spatio-Temporal Data Model in Sea Battlefield

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.246-247.744 ◽

2012 ◽

Vol 246-247 ◽

pp. 744-748

Author(s):

Yue Lin Sun ◽

Lei Bao ◽

Yi Hang Peng

Keyword(s):

Data Model ◽

Object Oriented ◽

Space Time ◽

Comprehensive Analysis ◽

Temporal Data ◽

Data Types ◽

Spatio Temporal ◽

The Right ◽

Effective Analysis

An effective analysis of the battlefield situation and spatio-temporal data model in a sea battlefield has great significance for the commander to perceive the battlefield situation and to make the right decisions. Based on the existing spatio-temporal data model, the present paper gives a comprehensive analysis of the characteristics of sea battlefield data, and chooses the object-oriented spatio-temporal data model to modify it; at the same time this paper introduces sea battlefield space-time algebra system to define various data types formally, which lays the foundation for the establishment of the sea battlefield spatio-temporal data model.

Download Full-text