scholarly journals An Efficient Approach to Informative Feature Extraction from Multimodal Data

2019 ◽  
Vol 33 ◽  
pp. 5281-5288 ◽  
Author(s):  
Lichen Wang ◽  
Jiaxiang Wu ◽  
Shao-Lun Huang ◽  
Lizhong Zheng ◽  
Xiangxiang Xu ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Empirical Evaluation ◽  
Multimodal Data ◽  
Informative Feature ◽  
Feature Representations ◽  
Multiple Data ◽  
Maximal Correlation ◽  
Feature Geometry ◽  
Primary Focus ◽  
Measure Of Dependence

One primary focus in multimodal feature extraction is to find the representations of individual modalities that are maximally correlated. As a well-known measure of dependence, the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation be-´ comes an appealing objective because of its operational meaning and desirable properties. However, the strict whitening constraints formalized in the HGR maximal correlation limit its application. To address this problem, this paper proposes Soft-HGR, a novel framework to extract informative features from multiple data modalities. Specifically, our framework prevents the “hard” whitening constraints, while simultaneously preserving the same feature geometry as in the HGR maximal correlation. The objective of Soft-HGR is straightforward, only involving two inner products, which guarantees the efficiency and stability in optimization. We further generalize the framework to handle more than two modalities and missing modalities. When labels are partially available, we enhance the discriminative power of the feature representations by making a semi-supervised adaptation. Empirical evaluation implies that our approach learns more informative feature mappings and is more efficient to optimize.

Investigation of automated feature extraction using multiple data sources

2003 ◽  
Author(s):  
Neal R. Harvey ◽  
Simon J. Perkins ◽  
Paul A. Pope ◽  
James P. Theiler ◽  
Nancy A. David ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Data Sources ◽  
Multiple Data Sources ◽  
Multiple Data ◽  
Automated Feature Extraction

scholarly journals Offline Signature Identification and Verification Based on Capsule Representations

2020 ◽  
Vol 20 (5) ◽  
pp. 60-67
Author(s):  
Dilara Gumusbas ◽  
Tulay Yildirim
Keyword(s):  
Feature Extraction ◽  
Daily Life ◽  
Extraction Methods ◽  
Signature Verification ◽  
Verification Task ◽  
Feature Representations ◽  
Automatic Feature Extraction ◽  
Offline Signature Verification ◽  
Orientation Information ◽  
Trained Network

AbstractOffline signature is one of the frequently used biometric traits in daily life and yet skilled forgeries are posing a great challenge for offline signature verification. To differentiate forgeries, a variety of research has been conducted on hand-crafted feature extraction methods until now. However, these methods have recently been set aside for automatic feature extraction methods such as Convolutional Neural Networks (CNN). Although these CNN-based algorithms often achieve satisfying results, they require either many samples in training or pre-trained network weights. Recently, Capsule Network has been proposed to model with fewer data by using the advantage of convolutional layers for automatic feature extraction. Moreover, feature representations are obtained as vectors instead of scalar activation values in CNN to keep orientation information. Since signature samples per user are limited and feature orientations in signature samples are highly informative, this paper first aims to evaluate the capability of Capsule Network for signature identification tasks on three benchmark databases. Capsule Network achieves 97 96, 94 89, 95 and 91% accuracy on CEDAR, GPDS-100 and MCYT databases for 64×64 and 32×32 resolutions, which are lower than usual, respectively. The second aim of the paper is to generalize the capability of Capsule Network concerning the verification task. Capsule Network achieves average 91, 86, and 89% accuracy on CEDAR, GPDS-100 and MCYT databases for 64×64 resolutions, respectively. Through this evaluation, the capability of Capsule Network is shown for offline verification and identification tasks.

scholarly journals Data-Dependent Conditional Priors for Unsupervised Learning of Multimodal Data

Entropy ◽  
2020 ◽  
Vol 22 (8) ◽  
pp. 888
Author(s):  
Frantzeska Lavda ◽  
Magda Gregorová ◽  
Alexandros Kalousis
Keyword(s):  
Latent Variables ◽  
Joint Distribution ◽  
Empirical Evaluation ◽  
Original Data ◽  
Sampling Procedure ◽  
Mixture Distributions ◽  
Multimodal Data ◽  
Generative Process ◽  
The Individual ◽  
Learning Data

One of the major shortcomings of variational autoencoders is the inability to produce generations from the individual modalities of data originating from mixture distributions. This is primarily due to the use of a simple isotropic Gaussian as the prior for the latent code in the ancestral sampling procedure for data generations. In this paper, we propose a novel formulation of variational autoencoders, conditional prior VAE (CP-VAE), with a two-level generative process for the observed data where continuous z and a discrete c variables are introduced in addition to the observed variables x. By learning data-dependent conditional priors, the new variational objective naturally encourages a better match between the posterior and prior conditionals, and the learning of the latent categories encoding the major source of variation of the original data in an unsupervised manner. Through sampling continuous latent code from the data-dependent conditional priors, we are able to generate new samples from the individual mixture components corresponding, to the multimodal structure over the original data. Moreover, we unify and analyse our objective under different independence assumptions for the joint distribution of the continuous and discrete latent variables. We provide an empirical evaluation on one synthetic dataset and three image datasets, FashionMNIST, MNIST, and Omniglot, illustrating the generative performance of our new model comparing to multiple baselines.

Kernel-based Informative Feature Extraction via Gradient Learning

2012 ◽  
Vol 7 (11) ◽  
Author(s):  
Songhua Liu ◽  
Jiansheng Liu ◽  
Caiying Ding ◽  
Chaoquan Zhang
Keyword(s):  
Feature Extraction ◽  
Informative Feature ◽  
Gradient Learning

scholarly journals On Task-specific Autonomy in Robotic Interventions: A Multimodal Learning-based Approach for Multi-level Skill Assessment during Cyborg Catheterization

2021 ◽  
Author(s):  
Olatunji Omisore ◽  
Wenke Duan ◽  
Wenjing Du ◽  
Shipeng Han ◽  
Toluwanimi Akinyemi ◽  
...  
Keyword(s):  
Skill Assessment ◽  
Multimodal Learning ◽  
Robotic Control ◽  
Multimodal Data ◽  
Multiple Data ◽  
Study Task ◽  
Catheter System ◽  
Multi Level ◽  
Effective Assessment

Lack of learning-based methods for characterizing the multimodal data generated during cyborg catheterization hinders the drive towards autonomous robotic control. Also, multiplexing salient features from multiple data-sources can enhance effective assessment and classification of domain skills for apt intelligent surgeon-robot (cyborg) catheterization during intravascular interventions. In this study, task-specific autonomous intervention is envisioned upon an isomorphic master-slave robotic catheter system that exhibit hand defter techniques used in Cath Labs. To drive cyborg catheterization, stacking-based deep neural network is developed for three-level skill assessment.<br>

scholarly journals Feature extraction from multiple data sources using genetic programming

2002 ◽  
Author(s):  
John J. Szymanski ◽  
Steven P. Brumby ◽  
Paul A. Pope ◽  
Damian R. Eads ◽  
Diana M. Esch-Mosher ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Genetic Programming ◽  
Data Sources ◽  
Multiple Data Sources ◽  
Multiple Data

scholarly journals A Harmonic Linear Dynamical System for Prominent ECG Feature Extraction

2014 ◽  
Vol 2014 ◽  
pp. 1-8 ◽  
Author(s):  
Ngoc Anh Nguyen Thi ◽  
Hyung-Jeong Yang ◽  
SunHee Kim ◽  
Luu Ngoc Do
Keyword(s):  
Dynamical System ◽  
Time Series ◽  
Feature Extraction ◽  
Biomedical Applications ◽  
Empirical Evaluation ◽  
Computation Time ◽  
Main Stream ◽  
Linear Dynamical System ◽  
Improved Performance ◽  
Real World Datasets

Unsupervised mining of electrocardiography (ECG) time series is a crucial task in biomedical applications. To have efficiency of the clustering results, the prominent features extracted from preprocessing analysis on multiple ECG time series need to be investigated. In this paper, a Harmonic Linear Dynamical System is applied to discover vital prominent features via mining the evolving hidden dynamics and correlations in ECG time series. The discovery of the comprehensible and interpretable features of the proposed feature extraction methodology effectively represents the accuracy and the reliability of clustering results. Particularly, the empirical evaluation results of the proposed method demonstrate the improved performance of clustering compared to the previous main stream feature extraction approaches for ECG time series clustering tasks. Furthermore, the experimental results on real-world datasets show scalability with linear computation time to the duration of the time series.

scholarly journals Deep learning with multimodal representation for pancancer prognosis prediction

2019 ◽  
Vol 35 (14) ◽  
pp. i446-i454 ◽  
Author(s):  
Anika Cheerla ◽  
Olivier Gevaert
Keyword(s):  
Cancer Patients ◽  
Expression Data ◽  
Patient Handling ◽  
Multimodal Data ◽  
Patients With Cancer ◽  
Multiple Data ◽  
Single Feature ◽  
Clinical Methods ◽  
Cancer Types ◽  
Single Cancer

Abstract Motivation Estimating the future course of patients with cancer lesions is invaluable to physicians; however, current clinical methods fail to effectively use the vast amount of multimodal data that is available for cancer patients. To tackle this problem, we constructed a multimodal neural network-based model to predict the survival of patients for 20 different cancer types using clinical data, mRNA expression data, microRNA expression data and histopathology whole slide images (WSIs). We developed an unsupervised encoder to compress these four data modalities into a single feature vector for each patient, handling missing data through a resilient, multimodal dropout method. Encoding methods were tailored to each data type—using deep highway networks to extract features from clinical and genomic data, and convolutional neural networks to extract features from WSIs. Results We used pancancer data to train these feature encodings and predict single cancer and pancancer overall survival, achieving a C-index of 0.78 overall. This work shows that it is possible to build a pancancer model for prognosis that also predicts prognosis in single cancer sites. Furthermore, our model handles multiple data modalities, efficiently analyzes WSIs and represents patient multimodal data flexibly into an unsupervised, informative representation. We thus present a powerful automated tool to accurately determine prognosis, a key step towards personalized treatment for cancer patients. Availability and implementation https://github.com/gevaertlab/MultimodalPrognosis

Sign in / Sign up

Export Citation Format

Share Document