scholarly journals Cross-Modal Learning Based on Semantic Correlation and Multi-Task Learning for Text-Video Retrieval

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2125
Author(s):  
Xiaoyu Wu ◽  
Tiantian Wang ◽  
Shengjin Wang
Keyword(s):  
Video Retrieval ◽  
Feature Space ◽  
Retrieval Algorithm ◽  
Learning Networks ◽  
Semantic Association ◽  
Semantic Consistency ◽  
Semantic Correlation ◽  
Task Learning ◽  
Feature Subspace ◽  
Semantic Similarity Measurement

Text-video retrieval tasks face a great challenge in the semantic gap between cross modal information. Some existing methods transform the text or video into the same subspace to measure their similarity. However, this kind of method does not consider adding a semantic consistency constraint when associating the two modalities of semantic encoding, and the associated result is poor. In this paper, we propose a multi-modal retrieval algorithm based on semantic association and multi-task learning. Firstly, the multi-level features of video or text are extracted based on multiple deep learning networks, so that the information of the two modalities can be fully encoded. Then, in the public feature space where the two modalities information are mapped together, we propose a semantic similarity measurement and semantic consistency classification based on text-video features for a multi-task learning framework. With the semantic consistency classification task, the learning of semantic association task is restrained. So multi-task learning guides the better feature mapping of two modalities and optimizes the construction of unified feature subspace. Finally, the experimental results of our proposed algorithm on the Microsoft Video Description dataset (MSVD) and MSR-Video to Text (MSR-VTT) are better than the existing research, which prove that our algorithm can improve the performance of cross-modal retrieval.

Learning Target Class Feature Subspace (LTC-FS) Using Eigenspace Analysis and N-ary Search-Based Autonomous Hyperparameter Tuning for OCSVM

2021 ◽  
Author(s):  
Sanjay Kumar Sonbhadra ◽  
Sonali Agarwal ◽  
P. Nagabhushan
Keyword(s):  
Principal Component ◽  
Feature Space ◽  
Support Vector ◽  
Feature Subset ◽  
Target Class ◽  
Significant Information ◽  
Feature Extraction Method ◽  
Specificity And Sensitivity ◽  
Feature Subspace ◽  
Novel Target

Existing dimensionality reduction (DR) techniques such as principal component analysis (PCA) and its variants are not suitable for target class mining due to the negligence of unique statistical properties of class-of-interest (CoI) samples. Conventionally, these approaches utilize higher or lower eigenvalued principal components (PCs) for data transformation; but the higher eigenvalued PCs may split the target class, whereas lower eigenvalued PCs do not contribute significant information and wrong selection of PCs leads to performance degradation. Considering these facts, the present research offers a novel target class-guided feature extraction method. In this approach, initially, the eigendecomposition is performed on variance–covariance matrix of only the target class samples, where the higher- and lower-valued eigenvectors are rejected via statistical analysis, and the selected eigenvectors are utilized to extract the most promising feature subspace. The extracted feature-subset gives a more tighter description of the CoI with enhanced associativity among target class samples and ensures the strong separation from nontarget class samples. One-class support vector machine (OCSVM) is evaluated to validate the performance of learned features. To obtain optimized values of hyperparameters of OCSVM a novel [Formula: see text]-ary search-based autonomous method is also proposed. Exhaustive experiments with a wide variety of datasets are performed in feature-space (original and reduced) and eigenspace (obtained from original and reduced features) to validate the performance of the proposed approach in terms of accuracy, precision, specificity and sensitivity.

Graph-based multi-space semantic correlation propagation for video retrieval

2010 ◽  
Vol 27 (1) ◽  
pp. 21-34 ◽  
Author(s):  
Bailan Feng ◽  
Juan Cao ◽  
Xiuguo Bao ◽  
Lei Bao ◽  
Yongdong Zhang ◽  
...  
Keyword(s):  
Video Retrieval ◽  
Semantic Correlation ◽  
Space Semantic

scholarly journals Generative models for capturing and exploiting the influence of process conditions on process curves

2021 ◽  
Author(s):  
Tarek Iraki ◽  
Norbert Link
Keyword(s):  
Latent Variable ◽  
Welding Process ◽  
Feature Space ◽  
Generative Models ◽  
Process Conditions ◽  
Process Data ◽  
Task Learning ◽  
Latent Space ◽  
Variable Space ◽  
Novel Method

AbstractVariations of dedicated process conditions (such as workpiece and tool properties) yield different process state evolutions, which are reflected by different time series of the observable quantities (process curves). A novel method is presented, which firstly allows to extract the statistical influence of these conditions on the process curves and its representation via generative models, and secondly represents their influence on the ensemble of curves by transformations of the representation space. A latent variable space is derived from sampled process data, which represents the curves with only few features. Generative models are formed based on conditional propability functions estimated in this space. Furthermore, the influence of conditions on the ensemble of process curves is represented by estimated transformations of the feature space, which map the process curve densities with different conditions on each other. The latent space is formed via Multi-Task-Learning of an auto-encoder and condition-detectors. The latter classifies the latent space representations of the process curves into the considered conditions. The Bayes framework and the Multi-task Learning models are used to obtain the process curve probabilty densities from the latent space densities. The methods are shown to reveal and represent the influence of combinations of workpiece and tool properties on resistance spot welding process curves.

scholarly journals A Multi-task Learning Model for Daily Activity Forecast in Smart Home

Sensors ◽  
2020 ◽  
Vol 20 (7) ◽  
pp. 1933 ◽  
Author(s):  
Hong Yang ◽  
Shanshan Gong ◽  
Yaqing Liu ◽  
Zhengkui Lin ◽  
Yi Qu
Keyword(s):  
Daily Activity ◽  
Short Term Memory ◽  
Feature Space ◽  
Forecast Model ◽  
Learning Model ◽  
Occurrence Time ◽  
Daily Lives ◽  
Task Learning ◽  
Proposed Model ◽  
In Series

Daily activity forecasts play an important role in the daily lives of residents in smart homes. Category forecasts and occurrence time forecasts of daily activity are two key tasks. Category forecasts of daily activity are correlated with occurrence time forecasts, however, existing research has only focused on one of the two tasks. Moreover, the performance of daily activity forecasts is low when the two tasks are performed in series. In this paper, a forecast model based on multi-task learning is proposed to forecast category and occurrence time of daily activity mutually and iteratively. Firstly, raw sensor events are pre-processed to form a feature space of daily activity. Secondly, a parallel multi-task learning model which combines a convolutional neural network (CNN) with bidirectional long short-term memory (Bi-LSTM) units are developed as the forecast model. Finally, five distinct datasets are used to evaluate the proposed model. The experimental results show that compared with the state-of-the-art single-task learning models, this model improves accuracy by at least 2.22%, and the metrics of NMAE, NRMSE and R2 are improved by at least 1.542%, 7.79% and 1.69%, respectively.

Git Recognition with Incomplete GEI Based on Random Forests

2014 ◽  
Vol 519-520 ◽  
pp. 661-666
Author(s):  
Qing Zhu ◽  
Jie Zhang
Keyword(s):  
Random Forests ◽  
Gait Recognition ◽  
Feature Space ◽  
High Dimensional ◽  
Random Forest Algorithm ◽  
Recognition Method ◽  
Locality Preserving ◽  
Feature Spaces ◽  
Feature Subspace ◽  
Low Dimensional

Abstract. This paper proposes an incomplete GEI gait recognition method based on Random Forests. There are numerous methods exist for git recognition,but they all lead to high dimensional feature spaces. To address the problem of high dimensional feature space, we propose the use of the Random Forest algorithm to rank features' importance . In order to efficiently search throughout subspaces, we apply a backward feature elimination search strategy.This demonstrate static areas of a GEI also contain useful information.Then, we project the selected feature to a low-dimensional feature subspace via the newly proposed two-dimensional locality preserving projections (2DLPP) method.Asa sequence,we further improve the discriminative power of the extracted features. Experimental results on the CASIA gait database demonstrate the effectiveness of the proposed method.

Sign in / Sign up

Export Citation Format

Share Document