Feature Space of Deep Learning and its Importance

Decoding human brain activities, especially reconstructing human visual stimuli via functional magnetic resonance imaging (fMRI), has gained increasing attention in recent years. However, the high dimensionality and small quantity of fMRI data impose restrictions on satisfactory reconstruction, especially for the reconstruction method with deep learning requiring huge amounts of labelled samples. When compared with the deep learning method, humans can recognize a new image because our human visual system is naturally capable of extracting features from any object and comparing them. Inspired by this visual mechanism, we introduced the mechanism of comparison into deep learning method to realize better visual reconstruction by making full use of each sample and the relationship of the sample pair by learning to compare. In this way, we proposed a Siamese reconstruction network (SRN) method. By using the SRN, we improved upon the satisfying results on two fMRI recording datasets, providing 72.5% accuracy on the digit dataset and 44.6% accuracy on the character dataset. Essentially, this manner can increase the training data about from n samples to 2n sample pairs, which takes full advantage of the limited quantity of training samples. The SRN learns to converge sample pairs of the same class or disperse sample pairs of different class in feature space.

Download Full-text

Boosting Automated Sleep Staging Performance in Big Datasets using Population Sub-grouping

SLEEP ◽

10.1093/sleep/zsab027 ◽

2021 ◽

Author(s):

Samaneh Nasiri ◽

Gari D Clifford

Keyword(s):

Deep Learning ◽

Feature Space ◽

Learning Approaches ◽

Sleep Staging ◽

Learning Framework ◽

Sensitivity Score ◽

Instrument Noise ◽

Novel Method ◽

Clustering Approach ◽

Statistical Relationships

Abstract Current approaches to automated sleep staging from the electroencephalogram (EEG) rely on constructing a large labeled training and test corpora by aggregating data from different individuals. However, many of the subjects in the training set may exhibit changes in the EEG that are very different from the subjects in the test set. Training an algorithm on such data without accounting for this diversity can cause underperformance. Moreover, test data may have unexpected sensor misplacement or different instrument noise and spectral responses. This work proposes a novel method to learn relevant individuals based on their similarities effectively. The proposed method embeds all training patients into a shared and robust feature space. Individuals that share strong statistical relationships and are similar based on their EEG signals are clustered in this feature space before being passed to a deep learning framework for classification. Using 994 patient EEGs from the 2018 Physionet Challenge (≈ 6,561 hours of recording), we demonstrate that the clustering approach significantly boosts performance compared to state-of-the-art deep learning approaches. The proposed method improves, on average, a precision score from 0.72 to 0.81, a sensitivity score from 0.74 to 0.82, and a Cohen’s Kappa coefficient from 0.64 to 0.75 under 10-fold cross-validation.

Download Full-text

Classification of Hydroacoustic Signals Based on Harmonic Wavelets and a Deep Learning Artificial Intelligence System

Applied Sciences ◽

10.3390/app10093097 ◽

2020 ◽

Vol 10 (9) ◽

pp. 3097

Author(s):

Dmitry Kaplun ◽

Alexander Voznesensky ◽

Sergei Romanov ◽

Valery Andreev ◽

Denis Butusov

Keyword(s):

Fourier Transform ◽

Deep Learning ◽

Feature Space ◽

Roc Curves ◽

Harmonic Wavelets ◽

Intelligence System ◽

Stationary Signals ◽

Harmonic Wavelet ◽

Short Time

This paper considers two approaches to hydroacoustic signal classification, taking the sounds made by whales as an example: a method based on harmonic wavelets and a technique involving deep learning neural networks. The study deals with the classification of hydroacoustic signals using coefficients of the harmonic wavelet transform (fast computation), short-time Fourier transform (spectrogram) and Fourier transform using a kNN-algorithm. Classification quality metrics (precision, recall and accuracy) are given for different signal-to-noise ratios. ROC curves were also obtained. The use of the deep neural network for classification of whales’ sounds is considered. The effectiveness of using harmonic wavelets for the classification of complex non-stationary signals is proved. A technique to reduce the feature space dimension using a ‘modulo N reduction’ method is proposed. A classification of 26 individual whales from the Whale FM Project dataset is presented. It is shown that the deep-learning-based approach provides the best result for the Whale FM Project dataset both for whale types and individuals.

Download Full-text

Towards Repayment Prediction in Peer-to-Peer Social Lending Using Deep Learning

Mathematics ◽

10.3390/math7111041 ◽

2019 ◽

Vol 7 (11) ◽

pp. 1041 ◽

Cited By ~ 1

Author(s):

Kim ◽

Cho

Keyword(s):

Deep Learning ◽

Design Feature ◽

Feature Space ◽

Peer To Peer ◽

Complex Data ◽

Complex Information ◽

Social Lending ◽

P2p Lending ◽

Complex Features ◽

Deep Learning Model

Peer-to-Peer (P2P) lending transactions take place by the lenders choosing a borrower and lending money. It is important to predict whether a borrower can repay because the lenders must bear the credit risk when the borrower defaults, but it is difficult to design feature extractors with very complex information about borrowers and loan products. In this paper, we present an architecture of deep convolutional neural network (CNN) for predicting the repayment in P2P social lending to extract features automatically and improve the performance. CNN is a deep learning model for classifying complex data, which extracts discriminative features automatically by convolution operation on lending data. We classify the borrower’s loan status by capturing the robust features and learning the patterns. Experimental results with 5-fold cross-validation show that our method automatically extracts complex features and is effective in repayment prediction on Lending Club data. In comparison with other machine learning methods, the standard CNN has achieved the highest performance with 75.86%. Exploiting various CNN models such as Inception, ResNet, and Inception-ResNet results in the state-of-the-art performance of 77.78%. We also demonstrate that the features extracted by our model are better performed by projecting the samples into the feature space.

Download Full-text

MalDeep: A Deep Learning Classification Framework against Malware Variants Based on Texture Visualization

Security and Communication Networks ◽

10.1155/2019/4895984 ◽

2019 ◽

Vol 2019 ◽

pp. 1-11

Author(s):

Yuntao Zhao ◽

Chunyu Xu ◽

Bo Bo ◽

Yongxin Feng

Keyword(s):

Deep Learning ◽

Classification Accuracy ◽

Feature Space ◽

Image Texture ◽

Accuracy Rate ◽

Texture Representation ◽

Malware Classification ◽

Classification Framework ◽

Average Accuracy ◽

New Feature

The increasing sophistication of malware variants such as encryption, polymorphism, and obfuscation calls for the new detection and classification technology. In this paper, MalDeep, a novel malware classification framework of deep learning based on texture visualization, is proposed against malicious variants. Through code mapping, texture partitioning, and texture extracting, we can study malware classification in a new feature space of image texture representation without decryption and disassembly. Furthermore, we built a malware classifier on convolutional neural network with two convolutional layers, two downsampling layers, and many full connection layers. We adopt the dataset, from Microsoft Malware Classification Challenge including 9 categories of malware families and 10868 variant samples, to train the model. The experiment results show that the established MalDeep has a higher accuracy rate for malware classification. In particular, for some backdoor families, the classification accuracy of the model reaches over 99%. Moreover, compared with other main antivirus software, MalDeep also outperforms others in the average accuracy for the variants from different families.

Download Full-text

A Survey on Feature Selection Based Spam Review Detection using Deep Learning Techniques

International Journal of Advanced Information and Communication Technology ◽

10.46532/ijaict-2020023 ◽

2020 ◽

pp. 102-108

Author(s):

Sophia S ◽

Rajamohana SP

Keyword(s):

Feature Selection ◽

Deep Learning ◽

Nearest Neighbor ◽

Feature Space ◽

Online Reviews ◽

High Dimensionality ◽

Product Reviews ◽

K Nearest Neighbor ◽

Good Decision ◽

Learning Techniques

In recent times, online shoppers are technically knowledgeable and open to product reviews. They usually read the buyer reviews and ratings before purchasing any product from ecommerce website. For the better understanding of products or services, reviews provided by the customers gives the vital source of information. In order to buy the right products for the individuals and to make the business decisions for the Organization online reviews are very important. These reviews or opinions in turn, allow us to find out the strength and weakness of the products. Spam reviews are written in order to falsely promote or demote a few target products or services. Also, detecting the spam reviews has also become more critical issue for the customer to make good decision during the purchase of the product. A major problem in identifying the fake review detection is high dimensionality of the feature space. Therefore, feature selection is an essential step in the fake review detection to reduce dimensionality of the feature space and to improve the classification accuracy. Hence it is important to detect the spam reviews but the major issues in spam review detection are the high dimensionality of feature space which contains redundant, noisy and irrelevant features. To resolve this, Deep Learning Techniques for selecting features is necessary. To classify the features, classifiers such as Naive Bayes, K Nearest Neighbor are used. An analysis of the various techniques employed to identify false and genuine reviews has been surveyed.

Download Full-text

Deep Optical Flow Estimation Via Multi-Scale Correspondence Structure Learning

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2017/488 ◽

2017 ◽

Cited By ~ 2

Author(s):

Shanshan Zhao ◽

Xi Li ◽

Omar El Farouk Bourahla

Keyword(s):

Deep Learning ◽

Optical Flow ◽

Structure Learning ◽

Feature Space ◽

Image Correlation ◽

Flow Estimation ◽

Optical Flow Estimation ◽

Multi Scale ◽

Benchmark Datasets ◽

End To End

As an important and challenging problem in computer vision, learning based optical flow estimation aims to discover the intrinsic correspondence structure between two adjacent video frames through statistical learning. Therefore, a key issue to solve in this area is how to effectively model the multi-scale correspondence structure properties in an adaptive end-to-end learning fashion. Motivated by this observation, we propose an end-to-end multi-scale correspondence structure learning (MSCSL) approach for optical flow estimation. In principle, the proposed MSCSL approach is capable of effectively capturing the multi-scale inter-image-correlation correspondence structures within a multi-level feature space from deep learning. Moreover, the proposed MSCSL approach builds a spatial Conv-GRU neural network model to adaptively model the intrinsic dependency relationships among these multi-scale correspondence structures. Finally, the above procedures for correspondence structure learning and multi-scale dependency modeling are implemented in a unified end-to-end deep learning framework. Experimental results on several benchmark datasets demonstrate the effectiveness of the proposed approach.

Download Full-text

Assessing PD-L1 Expression Level via Preoperative MRI in HCC Based on Integrating Deep Learning and Radiomics Features

Diagnostics ◽

10.3390/diagnostics11101875 ◽

2021 ◽

Vol 11 (10) ◽

pp. 1875

Author(s):

Yuchi Tian ◽

Temitope Emmanuel Komolafe ◽

Jian Zheng ◽

Guofeng Zhou ◽

Tao Chen ◽

...

Keyword(s):

Hepatocellular Carcinoma ◽

Deep Learning ◽

Cross Validation ◽

Predictive Ability ◽

Feature Space ◽

Integrated Model ◽

Expression Level ◽

Deep Learning Features ◽

Expression Status ◽

Fold Cross Validation

To assess if quantitative integrated deep learning and radiomics features can predict the PD-L1 expression level in preoperative MRI of hepatocellular carcinoma (HCC) patients. The data in this study consist of 103 hepatocellular carcinoma patients who received immunotherapy in a single center. These patients were divided into a high PD-L1 expression group (30 patients) and a low PD-L1 expression group (73 patients). Both radiomics and deep learning features were extracted from their MRI sequence of T2-WI, which were merged into an integrative feature space for machine learning for the prediction of PD-L1 expression. The five-fold cross-validation was adopted to validate the performance of the model, while the AUC was used to assess the predictive ability of the model. Based on the five-fold cross-validation, the integrated model achieved the best prediction performance, with an AUC score of 0.897 ± 0.084, followed by the deep learning-based model with an AUC of 0.852 ± 0.043 then the radiomics-based model with AUC of 0.794 ± 0.035. The feature set integrating radiomics and deep learning features is more effective in predicting PD-L1 expression level than only one feature type. The integrated model can achieve fast and accurate prediction of PD-L1 expression status in preoperative MRI of HCC patients.

Download Full-text

KNN Loss and Deep KNN

Fundamenta Informaticae ◽

10.3233/fi-2021-2068 ◽

2021 ◽

Vol 182 (2) ◽

pp. 95-110

Author(s):

Linh Le ◽

Ying Xie ◽

Vijay V. Raghavan

Keyword(s):

Deep Learning ◽

Supervised Learning ◽

Nearest Neighbor ◽

Learning Strategy ◽

Feature Space ◽

Kernel Functions ◽

Training Data ◽

K Nearest Neighbor ◽

Map Data

The k Nearest Neighbor (KNN) algorithm has been widely applied in various supervised learning tasks due to its simplicity and effectiveness. However, the quality of KNN decision making is directly affected by the quality of the neighborhoods in the modeling space. Efforts have been made to map data to a better feature space either implicitly with kernel functions, or explicitly through learning linear or nonlinear transformations. However, all these methods use pre-determined distance or similarity functions, which may limit their learning capacity. In this paper, we present two loss functions, namely KNN Loss and Fuzzy KNN Loss, to quantify the quality of neighborhoods formed by KNN with respect to supervised learning, such that minimizing the loss function on the training data leads to maximizing KNN decision accuracy on the training data. We further present a deep learning strategy that is able to learn, by minimizing KNN loss, pairwise similarities of data that implicitly maps data to a feature space where the quality of KNN neighborhoods is optimized. Experimental results show that this deep learning strategy (denoted as Deep KNN) outperforms state-of-the-art supervised learning methods on multiple benchmark data sets.

Download Full-text

Simple and Efficient Computational Intelligence Strategies for Effective Collaborative Decisions

Future Internet ◽

10.3390/fi11010024 ◽

2019 ◽

Vol 11 (1) ◽

pp. 24

Author(s):

Emelia Opoku Aboagye ◽

Rajesh Kumar

Keyword(s):

Deep Learning ◽

Feature Space ◽

Cold Start ◽

Personalized Recommendation ◽

Tensor Factorization ◽

Scalable Systems ◽

Collaborative Recommendation ◽

Real World Datasets ◽

Low Dimensional ◽

Cold Start Problem

We approach scalability and cold start problems of collaborative recommendation in this paper. An intelligent hybrid filtering framework that maximizes feature engineering and solves cold start problem for personalized recommendation based on deep learning is proposed in this paper. Present e-commerce sites mainly recommend pertinent items or products to a lot of users through personalized recommendation. Such personalization depends on large extent on scalable systems which strategically responds promptly to the request of the numerous users accessing the site (new users). Tensor Factorization (TF) provides scalable and accurate approach for collaborative filtering in such environments. In this paper, we propose a hybrid-based system to address scalability problems in such environments. We propose to use a multi-task approach which represent multiview data from users, according to their purchasing and rating history. We use a Deep Learning approach to map item and user inter-relationship to a low dimensional feature space where item-user resemblance and their preferred items is maximized. The evaluation results from real world datasets show that, our novel deep learning multitask tensor factorization (NeuralFil) analysis is computationally less expensive, scalable and addresses the cold-start problem through explicit multi-task approach for optimal recommendation decision making.

Download Full-text