scholarly journals Temporal Video Segmentation Using Optical Flow Estimation

2021 ◽  
pp. 4181-4194
Author(s):  
Eman Hato
Keyword(s):  
Optical Flow ◽  
Video Segmentation ◽  
Computational Cost ◽  
Superior Performance ◽  
Detection Accuracy ◽  
Motion Feature ◽  
Detection Process ◽  
Shot Detection ◽  
Temporal Video Segmentation ◽  
Shot Boundary

Shot boundary detection is the process of segmenting a video into basic units known as shots by discovering transition frames between shots. Researches have been conducted to accurately detect the shot boundaries. However, the acceleration of the shot detection process with higher accuracy needs improvement. A new method was introduced in this paper to find out the boundaries of abrupt shots in the video with high accuracy and lower computational cost. The proposed method consists of two stages. First, projection features were used to distinguish non boundary transitions and candidate transitions that may contain abrupt boundary. Only candidate transitions were conserved for next stage. Thus, the speed of shot detection was improved by reducing the detection scope. In the second stage, the candidate segments were refined using motion feature derived from the optical flow to remove non boundary frames. The results manifest that the proposed method achieved excellent detection accuracy (0.98 according to F-Score) and effectively speeded up detection process. In addition, the comparative analysis results confirmed the superior performance of the proposed method versus other methods.

scholarly journals An Efficient and Robust Temporal Video Segmentation

2019 ◽  
Vol 9 (1) ◽  
pp. 4332-4337
Keyword(s):  
Video Segmentation ◽  
Video Retrieval ◽  
Boundary Detection ◽  
Detection Methods ◽  
Shot Boundary Detection ◽  
Feature Descriptor ◽  
Local Contrast ◽  
Temporal Video Segmentation ◽  
Shot Boundary ◽  
Content Based Video Retrieval

Temporal video segmentation is the primary step of content based video retrieval. The whole processes of video management are coming under the focus of content based video retrieval, which includes, video indexing, video retrieval, and video summarization etc. In this paper, we proposed a computationally efficient and discriminating shot boundary detection method, which uses a local feature descriptor named local Contrast and Ordering (LCO) for feature extraction. The results of the experiments, which are conducted on the video dataset TRECVid, analyzed and compared with some existing shot boundary detection methods. The proposed method has given a promising result, even in the cases of illumination changes, rotated images etc.

scholarly journals Video Segmentation and Structuring for Indexing Applications

2011 ◽  
Vol 2 (4) ◽  
pp. 38-58 ◽  
Author(s):  
Ruxandra Tapu ◽  
Titus Zaharia
Keyword(s):  
Extraction Method ◽  
Video Segmentation ◽  
Computationally Efficient ◽  
Interest Points ◽  
Grouping Strategy ◽  
Color Histograms ◽  
Different Types ◽  
Temporal Video Segmentation ◽  
Shot Boundary ◽  
Complete Framework

This paper introduces a complete framework for temporal video segmentation. First, a computationally efficient shot extraction method is introduced, which adopts the normalized graph partition approach, enriched with a non-linear, multiresolution filtering of the similarity vectors involved. The shot boundary detection technique proposed yields high precision (90%) and recall (95%) rates, for all types of transitions, both abrupt and gradual. Next, for each detected shot, the authors construct a static storyboard by introducing a leap keyframe extraction method. The video abstraction algorithm is 23% faster than existing techniques for similar performances. Finally, the authors propose a shot grouping strategy that iteratively clusters visually similar shots under a set of temporal constraints. Two different types of visual features are exploited: HSV color histograms and interest points. In both cases, the precision and recall rates present average performances of 86%.

scholarly journals Video Segmentation and Structuring for Indexing Applications

2013 ◽  
pp. 205-225
Author(s):  
Ruxandra Tapu ◽  
Titus Zaharia
Keyword(s):  
Extraction Method ◽  
Video Segmentation ◽  
Shot Boundary Detection ◽  
Computationally Efficient ◽  
Interest Points ◽  
Grouping Strategy ◽  
Color Histograms ◽  
Temporal Video Segmentation ◽  
Shot Boundary ◽  
Complete Framework

This paper introduces a complete framework for temporal video segmentation. First, a computationally efficient shot extraction method is introduced, which adopts the normalized graph partition approach, enriched with a non-linear, multiresolution filtering of the similarity vectors involved. The shot boundary detection technique proposed yields high precision (90%) and recall (95%) rates, for all types of transitions, both abrupt and gradual. Next, for each detected shot, the authors construct a static storyboard by introducing a leap keyframe extraction method. The video abstraction algorithm is 23% faster than existing techniques for similar performances. Finally, the authors propose a shot grouping strategy that iteratively clusters visually similar shots under a set of temporal constraints. Two different types of visual features are exploited: HSV color histograms and interest points. In both cases, the precision and recall rates present average performances of 86%.

scholarly journals An efficient pruning scheme of deep neural networks for Internet of Things applications

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Chen Qi ◽  
Shibo Shen ◽  
Rongpeng Li ◽  
Zhifeng Zhao ◽  
Qing Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Internet Of Things ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Superior Performance ◽  
Compact Structure ◽  
Resource Limited ◽  
Benchmark Datasets ◽  
Iot Devices

AbstractNowadays, deep neural networks (DNNs) have been rapidly deployed to realize a number of functionalities like sensing, imaging, classification, recognition, etc. However, the computational-intensive requirement of DNNs makes it difficult to be applicable for resource-limited Internet of Things (IoT) devices. In this paper, we propose a novel pruning-based paradigm that aims to reduce the computational cost of DNNs, by uncovering a more compact structure and learning the effective weights therein, on the basis of not compromising the expressive capability of DNNs. In particular, our algorithm can achieve efficient end-to-end training that transfers a redundant neural network to a compact one with a specifically targeted compression rate directly. We comprehensively evaluate our approach on various representative benchmark datasets and compared with typical advanced convolutional neural network (CNN) architectures. The experimental results verify the superior performance and robust effectiveness of our scheme. For example, when pruning VGG on CIFAR-10, our proposed scheme is able to significantly reduce its FLOPs (floating-point operations) and number of parameters with a proportion of 76.2% and 94.1%, respectively, while still maintaining a satisfactory accuracy. To sum up, our scheme could facilitate the integration of DNNs into the common machine-learning-based IoT framework and establish distributed training of neural networks in both cloud and edge.

scholarly journals A Review of Algorithms and Hardware Implementations for Spiking Neural Networks

2021 ◽  
Vol 11 (2) ◽  
pp. 23
Author(s):  
Duy-Anh Nguyen ◽  
Xuan-Tu Tran ◽  
Francesca Iacopi
Keyword(s):  
Neural Networks ◽  
Computational Cost ◽  
Superior Performance ◽  
Training Algorithms ◽  
Current State ◽  
Hardware Implementations ◽  
Neuromorphic Hardware ◽  
High Level ◽  
Event Based ◽  
Hardware Platforms

Deep Learning (DL) has contributed to the success of many applications in recent years. The applications range from simple ones such as recognizing tiny images or simple speech patterns to ones with a high level of complexity such as playing the game of Go. However, this superior performance comes at a high computational cost, which made porting DL applications to conventional hardware platforms a challenging task. Many approaches have been investigated, and Spiking Neural Network (SNN) is one of the promising candidates. SNN is the third generation of Artificial Neural Networks (ANNs), where each neuron in the network uses discrete spikes to communicate in an event-based manner. SNNs have the potential advantage of achieving better energy efficiency than their ANN counterparts. While generally there will be a loss of accuracy on SNN models, new algorithms have helped to close the accuracy gap. For hardware implementations, SNNs have attracted much attention in the neuromorphic hardware research community. In this work, we review the basic background of SNNs, the current state and challenges of the training algorithms for SNNs and the current implementations of SNNs on various hardware platforms.

scholarly journals Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

2020 ◽  
Vol 34 (07) ◽  
pp. 10713-10720
Author(s):  
Mingyu Ding ◽  
Zhe Wang ◽  
Bolei Zhou ◽  
Jianping Shi ◽  
Zhiwu Lu ◽  
...  
Keyword(s):  
Optical Flow ◽  
Video Segmentation ◽  
Video Clip ◽  
Semantic Segmentation ◽  
Temporal Consistency ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Optical Flows ◽  
Benchmark Datasets ◽  
Spatio Temporal

A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from the rest of the frames. To exploit the spatio-temporal information in videos, many previous works use pre-computed optical flows, which encode the temporal consistency to improve the video segmentation. However, the video segmentation and optical flow estimation are still considered as two separate tasks. In this paper, we propose a novel framework for joint video semantic segmentation and optical flow estimation. Semantic segmentation brings semantic information to handle occlusion for more robust optical flow estimation, while the non-occluded optical flow provides accurate pixel-level temporal correspondences to guarantee the temporal consistency of the segmentation. Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference. Extensive experiments show that the proposed model makes the video semantic segmentation and optical flow estimation benefit from each other and outperforms existing methods under the same settings in both tasks.

Temporal video segmentation: A survey

2001 ◽  
Vol 16 (5) ◽  
pp. 477-500 ◽  
Author(s):  
Irena Koprinska ◽  
Sergio Carrato
Keyword(s):  
Video Segmentation ◽  
Temporal Video Segmentation

scholarly journals VIDEO COPY DETECTION UTILIZING THE LOG-POLAR TRANSFORMATION

2016 ◽  
pp. 8-13
Author(s):  
Daniel Reynolds ◽  
Richard A. Messner
Keyword(s):  
Data Reduction ◽  
Reduction Process ◽  
Video Data ◽  
Video Frame ◽  
Detection Accuracy ◽  
Video Copy Detection ◽  
Copy Detection ◽  
Frame Size ◽  
Detection Process ◽  
Processing Step

Video copy detection is the process of comparing and analyzing videos to extract a measure of their similarity in order to determine if they are copies, modified versions, or completely different videos. With video frame sizes increasing rapidly, it is important to allow for a data reduction process to take place in order to achieve fast video comparisons. Further, detecting video streaming and storage of legal and illegal video data necessitates the fast and efficient implementation of video copy detection algorithms. In this paper some commonly used algorithms for video copy detection are implemented with the Log-Polar transformation being used as a pre-processing step to reduce the frame size prior to signature calculation. Two global based algorithms were chosen to validate the use of Log-Polar as an acceptable data reduction stage. The results of this research demonstrate that the addition of this pre-processing step significantly reduces the computation time of the overall video copy detection process while not significantly affecting the detection accuracy of the algorithm used for the detection process.

Sign in / Sign up

Export Citation Format

Share Document