Unified framework for representation, analysis of multimedia content for correlation and prediction

Author(s):  
S. Nissi Paul ◽  
Y. Jayanta Singh
Author(s):  
B. Aparna ◽  
S. Madhavi ◽  
G. Mounika ◽  
P. Avinash ◽  
S. Chakravarthi

We propose a new design for large-scale multimedia content protection systems. Our design leverages cloud infrastructures to provide cost efficiency, rapid deployment, scalability, and elasticity to accommodate varying workloads. The proposed system can be used to protect different multimedia content types, including videos, images, audio clips, songs, and music clips. The system can be deployed on private and/or public clouds. Our system has two novel components: (i) method to create signatures of videos, and (ii) distributed matching engine for multimedia objects. The signature method creates robust and representative signatures of videos that capture the depth signals in these videos and it is computationally efficient to compute and compare as well as it requires small storage. The distributed matching engine achieves high scalability and it is designed to support different multimedia objects. We implemented the proposed system and deployed it on two clouds: Amazon cloud and our private cloud. Our experiments with more than 11,000 videos and 1 million images show the high accuracy and scalability of the proposed system. In addition, we compared our system to the protection system used by YouTube and our results show that the YouTube protection system fails to detect most copies of videos, while our system detects more than 98% of them.


2020 ◽  
Vol 71 (7) ◽  
pp. 868-880
Author(s):  
Nguyen Hong-Quan ◽  
Nguyen Thuy-Binh ◽  
Tran Duc-Long ◽  
Le Thi-Lan

Along with the strong development of camera networks, a video analysis system has been become more and more popular and has been applied in various practical applications. In this paper, we focus on person re-identification (person ReID) task that is a crucial step of video analysis systems. The purpose of person ReID is to associate multiple images of a given person when moving in a non-overlapping camera network. Many efforts have been made to person ReID. However, most of studies on person ReID only deal with well-alignment bounding boxes which are detected manually and considered as the perfect inputs for person ReID. In fact, when building a fully automated person ReID system the quality of the two previous steps that are person detection and tracking may have a strong effect on the person ReID performance. The contribution of this paper are two-folds. First, a unified framework for person ReID based on deep learning models is proposed. In this framework, the coupling of a deep neural network for person detection and a deep-learning-based tracking method is used. Besides, features extracted from an improved ResNet architecture are proposed for person representation to achieve a higher ReID accuracy. Second, our self-built dataset is introduced and employed for evaluation of all three steps in the fully automated person ReID framework.


2019 ◽  
Author(s):  
Chi-Yun Lin ◽  
Matthew Romei ◽  
Luke Oltrogge ◽  
Irimpan Mathews ◽  
Steven Boxer

Green fluorescent protein (GFPs) have become indispensable imaging and optogenetic tools. Their absorption and emission properties can be optimized for specific applications. Currently, no unified framework exists to comprehensively describe these photophysical properties, namely the absorption maxima, emission maxima, Stokes shifts, vibronic progressions, extinction coefficients, Stark tuning rates, and spontaneous emission rates, especially one that includes the effects of the protein environment. In this work, we study the correlations among these properties from systematically tuned GFP environmental mutants and chromophore variants. Correlation plots reveal monotonic trends, suggesting all these properties are governed by one underlying factor dependent on the chromophore's environment. By treating the anionic GFP chromophore as a mixed-valence compound existing as a superposition of two resonance forms, we argue that this underlying factor is defined as the difference in energy between the two forms, or the driving force, which is tuned by the environment. We then introduce a Marcus-Hush model with the bond length alternation vibrational mode, treating the GFP absorption band as an intervalence charge transfer band. This model explains all the observed strong correlations among photophysical properties; related subtopics are extensively discussed in Supporting Information. Finally, we demonstrate the model's predictive power by utilizing the additivity of the driving force. The model described here elucidates the role of the protein environment in modulating photophysical properties of the chromophore, providing insights and limitations for designing new GFPs with desired phenotypes. We argue this model should also be generally applicable to both biological and non-biological polymethine dyes.<br>


Sign in / Sign up

Export Citation Format

Share Document