VEKG: Video Event Knowledge Graph to Represent Video Streams for Complex Event Pattern Matching

Author(s):  
Piyush Yadav ◽  
Edward Curry
2020 ◽  
Vol 14 (03) ◽  
pp. 423-455
Author(s):  
Piyush Yadav ◽  
Dhaval Salwala ◽  
Dibya Prakash Das ◽  
Edward Curry

Complex Event Processing (CEP) is an event processing paradigm to perform real-time analytics over streaming data and match high-level event patterns. Presently, CEP is limited to process structured data stream. Video streams are complicated due to their unstructured data model and limit CEP systems to perform matching over them. This work introduces a graph-based structure for continuous evolving video streams, which enables the CEP system to query complex video event patterns. We propose the Video Event Knowledge Graph (VEKG), a graph-driven representation of video data. VEKG models video objects as nodes and their relationship interaction as edges over time and space. It creates a semantic knowledge representation of video data derived from the detection of high-level semantic concepts from the video using an ensemble of deep learning models. A CEP-based state optimization — VEKG-Time Aggregated Graph (VEKG-TAG) — is proposed over VEKG representation for faster event detection. VEKG-TAG is a spatiotemporal graph aggregation method that provides a summarized view of the VEKG graph over a given time length. We defined a set of nine event pattern rules for two domains (Activity Recognition and Traffic Management), which act as a query and applied over VEKG graphs to discover complex event patterns. To show the efficacy of our approach, we performed extensive experiments over 801 video clips across 10 datasets. The proposed VEKG approach was compared with other state-of-the-art methods and was able to detect complex event patterns over videos with [Formula: see text]-Score ranging from 0.44 to 0.90. In the given experiments, the optimized VEKG-TAG was able to reduce 99% and 93% of VEKG nodes and edges, respectively, with 5.19[Formula: see text] faster search time, achieving sub-second median latency of 4–20[Formula: see text]ms.


Author(s):  
Tingting Tang ◽  
Wei Liu ◽  
Weimin Li ◽  
Jinliang Wu ◽  
Haiyang Ren

Author(s):  
Wang Yunlong ◽  
Wang Tingchun ◽  
Mu Bo ◽  
Guo Xiaoyan ◽  
Zhang Guozhi ◽  
...  

Author(s):  
Charlotte Rudnik ◽  
Thibault Ehrhart ◽  
Olivier Ferret ◽  
Denis Teyssou ◽  
Raphael Troncy ◽  
...  

2021 ◽  
Vol 2078 (1) ◽  
pp. 012024
Author(s):  
Zhen Jia ◽  
Yang Chu ◽  
Zhi Liu

Abstract This paper proposes a new tactical decision aids method based on event knowledge graph (EventKG). In the warfare domain, EventKG can be constructed through event types design, event network construction and transition probability computation between events. Initially, four event classes are introduced in accordance with the OODA loop, and eighteen subclasses are further decomposed. With the aids of a common event template, all the events taking place in the battle field can be described. Event networks are built by adopting the hierarchical task network (HTN) and described through Bayesian network, to exhibit various relations between battle events. Transition probability, namely the occurrence probability of next possible event, is computed by using the prior probability and conditional probability of event occurring. On the basis of structured EventKG, entity knowledge graph (EKG) and entity relation knowledge graph (ERKG), tactical decision aid instructions can be generated by combining with the battlefield situation information.


Sign in / Sign up

Export Citation Format

Share Document