scholarly journals A Unified Framework for Shot Type Classification Based on Subject Centric Lens

2020 ◽  
pp. 17-34
Author(s):  
Anyi Rao ◽  
Jiaze Wang ◽  
Linning Xu ◽  
Xuekun Jiang ◽  
Qingqiu Huang ◽  
...  
Keyword(s):  
Unified Framework ◽  
Shot Type ◽  
Type Classification

scholarly journals Comparative Study of Movie Shot Classification Based on Semantic Segmentation

2020 ◽  
Vol 10 (10) ◽  
pp. 3390
Author(s):  
Hui-Yong Bak ◽  
Seung-Bo Park
Keyword(s):  
Neural Network ◽  
Performance Improvement ◽  
Semantic Segmentation ◽  
Additional Information ◽  
Shot Classification ◽  
Average Accuracy ◽  
The Face ◽  
Shot Type ◽  
Type Classification

The shot-type decision is a very important pre-task in movie analysis due to the vast information, such as the emotion, psychology of the characters, and space information, from the shot type chosen. In order to analyze a variety of movies, a technique that automatically classifies shot types is required. Previous shot type classification studies have classified shot types by the proportion of the face on-screen or using a convolutional neural network (CNN). Studies that have classified shot types by the proportion of the face on-screen have not classified the shot if a person is not on the screen. A CNN classifies shot types even in the absence of a person on the screen, but there are certain shots that cannot be classified because instead of semantically analyzing the image, the method classifies them only by the characteristics and patterns of the image. Therefore, additional information is needed to access the image semantically, which can be done through semantic segmentation. Consequently, in the present study, the performance of shot type classification was improved by preprocessing the semantic segmentation of the frame extracted from the movie. Semantic segmentation approaches the images semantically and distinguishes the boundary relationships among objects. The representative technologies of semantic segmentation include Mask R-CNN and Yolact. A study was conducted to compare and evaluate performance using these as pretreatments for shot type classification. As a result, the average accuracy of shot type classification using a frame preprocessed with semantic segmentation increased by 1.9%, from 93% to 94.9%, when compared with shot type classification using the frame without such preprocessing. In particular, when using ResNet-50 and Yolact, the classification of shot type showed a 3% performance improvement (to 96% accuracy from 93%).

A unified framework for automated person re-indentification

2020 ◽  
Vol 71 (7) ◽  
pp. 868-880
Author(s):  
Nguyen Hong-Quan ◽  
Nguyen Thuy-Binh ◽  
Tran Duc-Long ◽  
Le Thi-Lan
Keyword(s):  
Deep Learning ◽  
Video Analysis ◽  
Camera Network ◽  
Unified Framework ◽  
Person Detection ◽  
Practical Applications ◽  
Detection And Tracking ◽  
Analysis System ◽  
Bounding Boxes

Along with the strong development of camera networks, a video analysis system has been become more and more popular and has been applied in various practical applications. In this paper, we focus on person re-identification (person ReID) task that is a crucial step of video analysis systems. The purpose of person ReID is to associate multiple images of a given person when moving in a non-overlapping camera network. Many efforts have been made to person ReID. However, most of studies on person ReID only deal with well-alignment bounding boxes which are detected manually and considered as the perfect inputs for person ReID. In fact, when building a fully automated person ReID system the quality of the two previous steps that are person detection and tracking may have a strong effect on the person ReID performance. The contribution of this paper are two-folds. First, a unified framework for person ReID based on deep learning models is proposed. In this framework, the coupling of a deep neural network for person detection and a deep-learning-based tracking method is used. Besides, features extracted from an improved ResNet architecture are proposed for person representation to achieve a higher ReID accuracy. Second, our self-built dataset is introduced and employed for evaluation of all three steps in the fully automated person ReID framework.

A Unified Model for Photophysical and Electro-Optical Properties of Green Fluorescent Proteins

2019 ◽  
Author(s):  
Chi-Yun Lin ◽  
Matthew Romei ◽  
Luke Oltrogge ◽  
Irimpan Mathews ◽  
Steven Boxer
Keyword(s):  
Driving Force ◽  
Fluorescent Protein ◽  
Charge Transfer Band ◽  
Photophysical Properties ◽  
Polymethine Dyes ◽  
Emission Rates ◽  
Unified Framework ◽  
Underlying Factor ◽  
Green Fluorescent ◽  
Protein Environment

Green fluorescent protein (GFPs) have become indispensable imaging and optogenetic tools. Their absorption and emission properties can be optimized for specific applications. Currently, no unified framework exists to comprehensively describe these photophysical properties, namely the absorption maxima, emission maxima, Stokes shifts, vibronic progressions, extinction coefficients, Stark tuning rates, and spontaneous emission rates, especially one that includes the effects of the protein environment. In this work, we study the correlations among these properties from systematically tuned GFP environmental mutants and chromophore variants. Correlation plots reveal monotonic trends, suggesting all these properties are governed by one underlying factor dependent on the chromophore's environment. By treating the anionic GFP chromophore as a mixed-valence compound existing as a superposition of two resonance forms, we argue that this underlying factor is defined as the difference in energy between the two forms, or the driving force, which is tuned by the environment. We then introduce a Marcus-Hush model with the bond length alternation vibrational mode, treating the GFP absorption band as an intervalence charge transfer band. This model explains all the observed strong correlations among photophysical properties; related subtopics are extensively discussed in Supporting Information. Finally, we demonstrate the model's predictive power by utilizing the additivity of the driving force. The model described here elucidates the role of the protein environment in modulating photophysical properties of the chromophore, providing insights and limitations for designing new GFPs with desired phenotypes. We argue this model should also be generally applicable to both biological and non-biological polymethine dyes.<br>

Sign in / Sign up

Export Citation Format

Share Document