VIDEO-SHOT TRANSITION DETECTION: A SPATIO-TEMPORAL SALIENCY APPROACH

2010 ◽  
Vol 24 (08) ◽  
pp. 1295-1309 ◽  
Author(s):  
XIAN WU ◽  
JIANHUANG LAI ◽  
PONG C. YUEN
Keyword(s):  
Error Probability ◽  
Spatial Information ◽  
Saliency Map ◽  
Detection Error ◽  
Novel Approach ◽  
Video Shot ◽  
Recent Method ◽  
Temporal And Spatial ◽  
Spatio Temporal ◽  
Shot Transitions

This paper proposes a novel approach for video-shot transition detection using spatio-temporal saliency. Both temporal and spatial information are combined to generate a saliency map, and features are available based on the change of saliency. Considering the context of shot changes, a statistical detector is constructed to determine all types of shot transitions by the minimization of the detection-error probability simultaneously under the same framework. The evaluation performed on videos of various content types demonstrates that the proposed approach outperforms a more recent method and two publicly available systems, namely VideoAnnex and VCM.

A FEATURE-BASED SCHEME FOR DETECTING AND CLASSIFYING VIDEO-SHOT TRANSITIONS BASED ON SPATIO-TEMPORAL ANALYSIS AND FUZZY CLASSIFICATION

2009 ◽  
Vol 23 (06) ◽  
pp. 1179-1200 ◽  
Author(s):  
GUO-SHIANG LIN ◽  
MIN-KUAN CHANG ◽  
SHIEN-TANG CHIU
Keyword(s):  
Feature Extraction ◽  
Temporal Analysis ◽  
Fuzzy Classification ◽  
Model Parameters ◽  
Fuzzy Classifier ◽  
Video Shot ◽  
Feature Based ◽  
Spatio Temporal ◽  
Shot Transitions ◽  
The Impact

In this paper, we propose a feature-based scheme for detecting different genres of video shot transitions based on spatio-temporal analysis and model parameter estimation. In feature extraction, the histogram difference and its modified versions are calculated from the effectiveness of detecting cuts and reducing the impact of fleeting lights. We propose a hybrid algorithm composed of adaptive thresholding, parameter calculation, and transition duration refinement to measure model parameters. Some properties of the associated model parameters of each transition are computed as features. A feature measuring the time gap between two consecutive shots is also adopted. After feature extraction, a fuzzy classifier integrates these features to distinguish nontransitions, cuts, and dissolve-type features from one to another. Many test videos having different types of shots are used for performance evaluation. The experimental results demonstrate that the proposed scheme not only detects cuts, dissolves, and fades well, but also accurately locates the duration of each dissolve-type transition. In addition, the proposed scheme outperforms some existing methods in terms of cut and dissolve detection.

scholarly journals Spatio-temporal patterns of leptospirosis in Thailand: is flooding a risk factor?

2015 ◽  
Vol 143 (10) ◽  
pp. 2106-2115 ◽  
Author(s):  
S. SUWANPAKDEE ◽  
J. KAEWKUNGWAL ◽  
L. J. WHITE ◽  
N. ASENSIO ◽  
P. RATANAKORN ◽  
...  
Keyword(s):  
Census Data ◽  
Spatial Information ◽  
Time Lags ◽  
Flood Events ◽  
Economic Activities ◽  
High Incidence ◽  
Temporal And Spatial ◽  
Spatio Temporal ◽  
The Relationship ◽  
Animal Reservoirs

SUMMARYWe studied the temporal and spatial patterns of leptospirosis, its association with flooding and animal census data in Thailand. Flood data from 2010 to 2012 were extracted from spatial information taken from satellite images. The incidence rate ratio (IRR) was used to determine the relationship between spatio-temporal flooding patterns and the number of human leptospirosis cases. In addition, the area of flood coverage, duration of waterlogging, time lags between flood events, and a number of potential animal reservoirs were considered in a sub-analysis. There was no significant temporal trend of leptospirosis over the study period. Statistical analysis showed an inconsistent relationship between IRR and flooding across years and regions. Spatially, leptospirosis occurred repeatedly and predominantly in northeastern Thailand. Our findings suggest that flooding is less influential in leptospirosis transmission than previously assumed. High incidence of the disease in the northeastern region is explained by the fact that agriculture and animal farming are important economic activities in this area. The periodic rise and fall of reported leptospirosis cases over time might be explained by seasonal exposure from rice farming activities performed during the rainy season when flood events often occur. We conclude that leptospirosis remains an occupational disease in Thailand.

scholarly journals Explaining Neural Networks Using Attentive Knowledge Distillation

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1280
Author(s):  
Hyeonseok Lee ◽  
Sungchan Kim
Keyword(s):  
Neural Networks ◽  
Model Prediction ◽  
Saliency Map ◽  
Model Parameters ◽  
Learning Capability ◽  
Fine Grained ◽  
Network Layers ◽  
Saliency Maps ◽  
Novel Approach ◽  
Knowledge Distillation

Explaining the prediction of deep neural networks makes the networks more understandable and trusted, leading to their use in various mission critical tasks. Recent progress in the learning capability of networks has primarily been due to the enormous number of model parameters, so that it is usually hard to interpret their operations, as opposed to classical white-box models. For this purpose, generating saliency maps is a popular approach to identify the important input features used for the model prediction. Existing explanation methods typically only use the output of the last convolution layer of the model to generate a saliency map, lacking the information included in intermediate layers. Thus, the corresponding explanations are coarse and result in limited accuracy. Although the accuracy can be improved by iteratively developing a saliency map, this is too time-consuming and is thus impractical. To address these problems, we proposed a novel approach to explain the model prediction by developing an attentive surrogate network using the knowledge distillation. The surrogate network aims to generate a fine-grained saliency map corresponding to the model prediction using meaningful regional information presented over all network layers. Experiments demonstrated that the saliency maps are the result of spatially attentive features learned from the distillation. Thus, they are useful for fine-grained classification tasks. Moreover, the proposed method runs at the rate of 24.3 frames per second, which is much faster than the existing methods by orders of magnitude.

scholarly journals Extracting Stops from Spatio-Temporal Trajectories within Dynamic Contextual Features

2021 ◽  
Vol 13 (2) ◽  
pp. 690
Author(s):  
Tao Wu ◽  
Huiqing Shen ◽  
Jianxin Qin ◽  
Longgang Xiang
Keyword(s):  
Moving Objects ◽  
Major Effect ◽  
Location Based Services ◽  
Trajectory Data ◽  
The Road ◽  
Contextual Features ◽  
Novel Approach ◽  
Gps Trajectories ◽  
Time Distance ◽  
Spatio Temporal

Identifying stops from GPS trajectories is one of the main concerns in the study of moving objects and has a major effect on a wide variety of location-based services and applications. Although the spatial and non-spatial characteristics of trajectories have been widely investigated for the identification of stops, few studies have concentrated on the impacts of the contextual features, which are also connected to the road network and nearby Points of Interest (POIs). In order to obtain more precise stop information from moving objects, this paper proposes and implements a novel approach that represents a spatio-temproal dynamics relationship between stopping behaviors and geospatial elements to detect stops. The relationship between the candidate stops based on the standard time–distance threshold approach and the surrounding environmental elements are integrated in a complex way (the mobility context cube) to extract stop features and precisely derive stops using the classifier classification. The methodology presented is designed to reduce the error rate of detection of stops in the work of trajectory data mining. It turns out that 26 features can contribute to recognizing stop behaviors from trajectory data. Additionally, experiments on a real-world trajectory dataset further demonstrate the effectiveness of the proposed approach in improving the accuracy of identifying stops from trajectories.

scholarly journals The Role of Spatio-Temporal Information to Govern the COVID-19 Pandemic: A European Perspective

2021 ◽  
Vol 10 (3) ◽  
pp. 166
Author(s):  
Hartmut Müller ◽  
Marije Louwsma
Keyword(s):  
European Union ◽  
Spatial Distribution ◽  
Spatial Information ◽  
National Level ◽  
Temporal Information ◽  
The European Union ◽  
Member States ◽  
Heavy Burden ◽  
Spatio Temporal

The Covid-19 pandemic put a heavy burden on member states in the European Union. To govern the pandemic, having access to reliable geo-information is key for monitoring the spatial distribution of the outbreak over time. This study aims to analyze the role of spatio-temporal information in governing the pandemic in the European Union and its member states. The European Nomenclature of Territorial Units for Statistics (NUTS) system and selected national dashboards from member states were assessed to analyze which spatio-temporal information was used, how the information was visualized and whether this changed over the course of the pandemic. Initially, member states focused on their own jurisdiction by creating national dashboards to monitor the pandemic. Information between member states was not aligned. Producing reliable data and timeliness reporting was problematic, just like selecting indictors to monitor the spatial distribution and intensity of the outbreak. Over the course of the pandemic, with more knowledge about the virus and its characteristics, interventions of member states to govern the outbreak were better aligned at the European level. However, further integration and alignment of public health data, statistical data and spatio-temporal data could provide even better information for governments and actors involved in managing the outbreak, both at national and supra-national level. The Infrastructure for Spatial Information in Europe (INSPIRE) initiative and the NUTS system provide a framework to guide future integration and extension of existing systems.

scholarly journals Spatio-Temporal Alignment and Trajectory Matching for Netted Radar Without Prior Spatial Information and Time Delay

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 126965-126976
Author(s):  
Xiaoyu Cong ◽  
Yubing Han ◽  
Weixing Sheng ◽  
Shanhong Guo ◽  
Renli Zhang
Keyword(s):  
Time Delay ◽  
Spatial Information ◽  
Temporal Alignment ◽  
Trajectory Matching ◽  
Spatio Temporal

scholarly journals IDENTIFYING OVERLAPPED OBJECTS FOR VIDEO INDEXING AND MODELING IN MULTIMEDIA DATABASE SYSTEMS

2001 ◽  
Vol 10 (04) ◽  
pp. 715-734 ◽  
Author(s):  
SHU-CHING CHEN ◽  
MEI-LING SHYU ◽  
CHENGCUI ZHANG ◽  
R. L. KASHYAP
Keyword(s):  
Spatial Information ◽  
Video Segmentation ◽  
Database Systems ◽  
Video Indexing ◽  
Video Data ◽  
Multimedia Database ◽  
Current Frame ◽  
Data Indexing ◽  
Spatio Temporal ◽  
Input Strings

The identification of the overlapped objects is a great challenge in object tracking and video data indexing. For this purpose, a backtrack-chain-updation split algorithm is proposed to assist an unsupervised video segmentation method called the "simultaneous partition and class parameter estimation" (SPCPE) algorithm to identify the overlapped objects in the video sequence. The backtrack-chain-updation split algorithm can identify the split segment (object) and use the information in the current frame to update the previous frames in a backtrack-chain manner. The split algorithm provides more accurate temporal and spatial information of the semantic objects so that the semantic objects can be indexed and modeled by multimedia input strings and the multimedia augmented transition network (MATN) model. The MATN model is based on the ATN model that has been used in artificial intelligence (AI) areas for natural language understanding systems, and its inputs are modeled by the multimedia input strings. In this paper, we will show that the SPCPE algorithm together with the backtrack-chain-updation split algorithm can significantly enhance the efficiency of spatio-temporal video indexing by improving the accuracy of multimedia database queries related to semantic objects.

VIDEO CONTENT REPRESENTATION FOR SHOT RETRIEVAL AND SCENE EXTRACTION

2001 ◽  
Vol 01 (03) ◽  
pp. 507-526 ◽  
Author(s):  
TONG LIN ◽  
HONG-JIANG ZHANG ◽  
QING-YUN SHI
Keyword(s):  
Similarity Measure ◽  
Video Content ◽  
Object Based ◽  
Scene Structure ◽  
Color Histograms ◽  
Extraction Algorithm ◽  
Content Representation ◽  
Video Shot ◽  
Shot Boundary ◽  
Spatio Temporal

In this paper, we present a novel scheme on video content representation by exploring the spatio-temporal information. A pseudo-object-based shot representation containing more semantics is proposed to measure shot similarity and force competition approach is proposed to group shots into scene based on content coherences between shots. Two content descriptors, color objects: Dominant Color Histograms (DCH) and Spatial Structure Histograms (SSH), are introduced. To represent temporal content variations, a shot can be segmented into several subshots that are of coherent content, and shot similarity measure is formulated as subshot similarity measure that serves to shot retrieval. With this shot representation, scene structure can be extracted by analyzing the splitting and merging force competitions at each shot boundary. Experimental results on real-world sports video prove that our proposed approach for video shot retrievals achieve the best performance on the average recall (AR) and average normalized modified retrieval rank (ANMRR), and Experiment on MPEG-7 test videos achieves promising results by the proposed scene extraction algorithm.

Sign in / Sign up

Export Citation Format

Share Document