Video Object Detection Using Event-Aware Convolutional Lstm and Object Relation Networks

Common video-based object detectors exploit temporal contextual information to improve the performance of object detection. However, detecting objects under challenging conditions has not been thoroughly studied yet. In this paper, we focus on improving the detection performance for challenging events such as aspect ratio change, occlusion, or large motion. To this end, we propose a video object detection network using event-aware ConvLSTM and object relation networks. Our proposed event-aware ConvLSTM is able to highlight the area where those challenging events take place. Compared with traditional ConvLSTM, with the proposed method it is easier to exploit temporal contextual information to support video-based object detectors under challenging events. To further improve the detection performance, an object relation module using supporting frame selection is applied to enhance the pooled features for target ROI. It effectively selects the features of the same object from one of the reference frames rather than all of them. Experimental results on ImageNet VID dataset show that the proposed method achieves mAP of 81.0% without any post processing and can handle challenging events efficiently in video object detection.

Aspect-Oriented System Coupling Metric and its Validation

Background: Aspect-oriented programming promises to enhance the extensibility and reusability of code through the removal of tangled and crosscutting code. Determining the degree of coupling for Aspect- Oriented Systems (AOSs) would assist in the quantification of various software attributes and hence improve quality. Objective: The research aims to present a novel Aspect-oriented System Coupling Metric (COAO), that calculates the coupling for the complete aspect-oriented system as a whole, based on the properties of elements and the relationships among them. Methods: The process of defining a metric essentially requires a clear, unambiguous definition of primary and relevant concepts related to Aspect-Oriented Programming. As such, first and foremost, novel definitions of basic concepts such as system, element, relation, module, and attribute are specified concerning Aspect- Oriented Programming. Subsequently, a metric for Aspect-Oriented System Coupling is proposed. Subsequently, the proposed metric is validated theoretically against Braiand properties for coupling of software systems. Finally, an illustration for calculation of the proposed metric is demonstrated using an exemplary aspect-oriented system. Results: The findings reveal that the proposed Aspect-Oriented Coupling metric conforms to the five Property- Based software engineering measurements given by Braiand et al. for coupling. This indicates that the proposed metric for the Aspect-oriented System Coupling metric COAO is a valid metric for measuring coupling in Aspect-oriented Software Systems. Conclusion: Results of validation along with the supportive illustration show that single metric to assess coupling for the complete Aspect-oriented Software System is theoretically sound and also easies the calculation of coupling of a software system.

Video Question Answering for Surveillance

There are many task in surveillance monitoring such as object detection, person identification, activity and action recognition etc. Integrating variety of surveillance task through a multimodal interactive system will benefit real-life deployment, and will also support human operators. We first introduce a dataset which is first of its kind and named as Surveillance Video Question Answering (SVideoQA) dataset. The multi-camera surveillance monitoring aspect is considered through the multimodal context of Video Question Answering (VideoQA) in the SVideoQA dataset. This paper proposes a deep learning model where VideoQA task on the SVideoQA dataset is attempted to solved in a manner where memory-driven relationship among appearance and motion aspect of the video features are captured. At each level of the relational reasoning respective attentive parts of the context of the motion and appearance features are identified forwarded through frame level and clip level relational reasoning module. Also, respective memories are updated which are again forwarded to the memory-relation module to finally predict the answer word. The proposed memory-driven multilevel relational reasoning is made compatible with the surveillance monitoring task through the incorporation of multi-camera relation module, which is able to capture and reason over the relationships among the video feeds across multiple cameras. Experimental outcome exhibits that the proposed memory-driven multilevel relational reasoning perform significantly better on the open-ended VideoQA task compared to other state-of-the art systems. The proposed method achieves an accuracy of 57\% and 57.6\% respectively for the single-camera and multi-camera task of the SVideoQA dataset.

Video Question Answering for Surveillance

There are many task in surveillance monitoring such as object detection, person identification, activity and action recognition etc. Integrating variety of surveillance task through a multimodal interactive system will benefit real-life deployment, and will also support human operators. We first introduce a dataset which is first of its kind and named as Surveillance Video Question Answering (SVideoQA) dataset. The multi-camera surveillance monitoring aspect is considered through the multimodal context of Video Question Answering (VideoQA) in the SVideoQA dataset. This paper proposes a deep learning model where VideoQA task on the SVideoQA dataset is attempted to solved in a manner where memory-driven relationship among appearance and motion aspect of the video features are captured. At each level of the relational reasoning respective attentive parts of the context of the motion and appearance features are identified forwarded through frame level and clip level relational reasoning module. Also, respective memories are updated which are again forwarded to the memory-relation module to finally predict the answer word. The proposed memory-driven multilevel relational reasoning is made compatible with the surveillance monitoring task through the incorporation of multi-camera relation module, which is able to capture and reason over the relationships among the video feeds across multiple cameras. Experimental outcome exhibits that the proposed memory-driven multilevel relational reasoning perform significantly better on the open-ended VideoQA task compared to other state-of-the art systems. The proposed method achieves an accuracy of 57\% and 57.6\% respectively for the single-camera and multi-camera task of the SVideoQA dataset.

Relation-Aware Pedestrian Attribute Recognition with Graph Convolutional Networks

In this paper, we propose a new end-to-end network, named Joint Learning of Attribute and Contextual relations (JLAC), to solve the task of pedestrian attribute recognition. It includes two novel modules: Attribute Relation Module (ARM) and Contextual Relation Module (CRM). For ARM, we construct an attribute graph with attribute-specific features which are learned by the constrained losses, and further use Graph Convolutional Network (GCN) to explore the correlations among multiple attributes. For CRM, we first propose a graph projection scheme to project the 2-D feature map into a set of nodes from different image regions, and then employ GCN to explore the contextual relations among those regions. Since the relation information in the above two modules is correlated and complementary, we incorporate them into a unified framework to learn both together. Experiments on three benchmarks, including PA-100K, RAP, PETA attribute datasets, demonstrate the effectiveness of the proposed JLAC.

AVS-Net: Automatic Visual Surveillance Using Relation Network

Visual surveillance through closed circuit television (CCTV) can help to prevent crime. In this paper, we propose an automatic visual surveillance network (AVS-Net), which simultaneously performs image processing and object detection to determine the dangers of situations captured by CCTV. In addition, we add a relation module to infer the relationships of the objects in the images. Experimental results show that the relation module greatly improves classification accuracy, even if there is not enough information.

Explicit generators for the relation module in the example of Gruenberg–Linnell

Gruenberg and Linnell showed that the standard relation module of a free product of n groups of the form Cr × $\mathbb{Z}$ could be generated by just n + 1 generators, raising the possibility of a relation gap. We explicitly give such a set of generators.