Hypergraph Convolution on Nodes-Hyperedges Network for Semi-Supervised Node Classification

2022 ◽  
Vol 16 (4) ◽  
pp. 1-19
Author(s):  
Hanrui Wu ◽  
Michael K. Ng
Keyword(s):  
Deep Learning ◽  
Classification Problem ◽  
Cross Entropy ◽  
Learning Approaches ◽  
Entropy Loss ◽  
Order Relations ◽  
Data Representations ◽  
Node Classification ◽  
The Cross ◽  
Full Consideration

Hypergraphs have shown great power in representing high-order relations among entities, and lots of hypergraph-based deep learning methods have been proposed to learn informative data representations for the node classification problem. However, most of these deep learning approaches do not take full consideration of either the hyperedge information or the original relationships among nodes and hyperedges. In this article, we present a simple yet effective semi-supervised node classification method named Hypergraph Convolution on Nodes-Hyperedges network, which performs filtering on both nodes and hyperedges as well as recovers the original hypergraph with the least information loss. Instead of only reducing the cross-entropy loss over the labeled samples as most previous approaches do, we additionally consider the hypergraph reconstruction loss as prior information to improve prediction accuracy. As a result, by taking both the cross-entropy loss on the labeled samples and the hypergraph reconstruction loss into consideration, we are able to achieve discriminative latent data representations for training a classifier. We perform extensive experiments on the semi-supervised node classification problem and compare the proposed method with state-of-the-art algorithms. The promising results demonstrate the effectiveness of the proposed method.

scholarly journals Ranking Loss: Maximizing the Success Rate in Deep Learning Side-Channel Analysis

2020 ◽  
pp. 25-55
Author(s):  
Gabriel Zaid ◽  
Lilian Bossuet ◽  
François Dassance ◽  
Amaury Habrard ◽  
Alexandre Venelli
Keyword(s):  
Deep Learning ◽  
Success Rate ◽  
Loss Function ◽  
Estimation Error ◽  
Approximation Error ◽  
Cross Entropy ◽  
Side Channel ◽  
Entropy Loss ◽  
The Cross ◽  
Ranking Loss

The side-channel community recently investigated a new approach, based on deep learning, to significantly improve profiled attacks against embedded systems. Compared to template attacks, deep learning techniques can deal with protected implementations, such as masking or desynchronization, without substantial preprocessing. However, important issues are still open. One challenging problem is to adapt the methods classically used in the machine learning field (e.g. loss function, performance metrics) to the specific side-channel context in order to obtain optimal results. We propose a new loss function derived from the learning to rank approach that helps preventing approximation and estimation errors, induced by the classical cross-entropy loss. We theoretically demonstrate that this new function, called Ranking Loss (RkL), maximizes the success rate by minimizing the ranking error of the secret key in comparison with all other hypotheses. The resulting model converges towards the optimal distinguisher when considering the mutual information between the secret and the leakage. Consequently, the approximation error is prevented. Furthermore, the estimation error, induced by the cross-entropy, is reduced by up to 23%. When the ranking loss is used, the convergence towards the best solution is up to 23% faster than a model using the cross-entropy loss function. We validate our theoretical propositions on public datasets.

scholarly journals Deep Learning for Laryngopharyngeal Reflux Diagnosis

2021 ◽  
Vol 11 (11) ◽  
pp. 4753
Author(s):  
Gen Ye ◽  
Chen Du ◽  
Tong Lin ◽  
Yan Yan ◽  
Jack Jiang
Keyword(s):  
Deep Learning ◽  
Speech Processing ◽  
Data Augmentation ◽  
Laryngopharyngeal Reflux ◽  
Ph Monitoring ◽  
Binary Classification ◽  
Classification Problem ◽  
Learning Approaches ◽  
Learning Techniques ◽  
Auc Value

(1) Background: Deep learning has become ubiquitous due to its impressive performance in various domains, such as varied as computer vision, natural language and speech processing, and game-playing. In this work, we investigated the performance of recent deep learning approaches on the laryngopharyngeal reflux (LPR) diagnosis task. (2) Methods: Our dataset is composed of 114 subjects with 37 pH-positive cases and 77 control cases. In contrast to prior work based on either reflux finding score (RFS) or pH monitoring, we directly take laryngoscope images as inputs to neural networks, as laryngoscopy is the most common and simple diagnostic method. The diagnosis task is formulated as a binary classification problem. We first tested a powerful backbone network that incorporates residual modules, attention mechanism and data augmentation. Furthermore, recent methods in transfer learning and few-shot learning were investigated. (3) Results: On our dataset, the performance is the best test classification accuracy is 73.4%, while the best AUC value is 76.2%. (4) Conclusions: This study demonstrates that deep learning techniques can be applied to classify LPR images automatically. Although the number of pH-positive images used for training is limited, deep network can still be capable of learning discriminant features with the advantage of technique.

scholarly journals A Review on Deep Learning Approaches for 3D Data Representations in Retrieval and Classifications

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 57566-57593 ◽  
Author(s):  
Abubakar Sulaiman Gezawa ◽  
Yan Zhang ◽  
Qicong Wang ◽  
Lei Yunqi
Keyword(s):  
Deep Learning ◽  
Learning Approaches ◽  
3D Data ◽  
Data Representations

scholarly journals Personal Interest Attention Graph Neural Networks for Session-Based Recommendation

Entropy ◽  
2021 ◽  
Vol 23 (11) ◽  
pp. 1500
Author(s):  
Xiangde Zhang ◽  
Yuan Zhou ◽  
Jianping Wang ◽  
Xiaojun Lu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Objective Function ◽  
Cross Entropy ◽  
Personal Interest ◽  
Entropy Loss ◽  
Convolutional Networks ◽  
The Cross ◽  
Graph Neural Networks

Session-based recommendations aim to predict a user’s next click based on the user’s current and historical sessions, which can be applied to shopping websites and APPs. Existing session-based recommendation methods cannot accurately capture the complex transitions between items. In addition, some approaches compress sessions into a fixed representation vector without taking into account the user’s interest preferences at the current moment, thus limiting the accuracy of recommendations. Considering the diversity of items and users’ interests, a personalized interest attention graph neural network (PIA-GNN) is proposed for session-based recommendation. This approach utilizes personalized graph convolutional networks (PGNN) to capture complex transitions between items, invoking an interest-aware mechanism to activate users’ interest in different items adaptively. In addition, a self-attention layer is used to capture long-term dependencies between items when capturing users’ long-term preferences. In this paper, the cross-entropy loss is used as the objective function to train our model. We conduct rich experiments on two real datasets, and the results show that PIA-GNN outperforms existing personalized session-aware recommendation methods.

scholarly journals Deep Learning for COVID-19 Diagnosis from CT Images

2021 ◽  
Vol 11 (17) ◽  
pp. 8227 ◽  
Author(s):  
Andrea Loddo ◽  
Fabio Pili ◽  
Cecilia Di Ruberto
Keyword(s):  
Deep Learning ◽  
Real World ◽  
Heterogeneous Data ◽  
Learning Approaches ◽  
Reference Dataset ◽  
Heterogeneous Data Sources ◽  
Patient Status ◽  
Learning Techniques ◽  
The Cross ◽  
Public Datasets

COVID-19, an infectious coronavirus disease, caused a pandemic with countless deaths. From the outset, clinical institutes have explored computed tomography as an effective and complementary screening tool alongside the reverse transcriptase-polymerase chain reaction. Deep learning techniques have shown promising results in similar medical tasks and, hence, may provide solutions to COVID-19 based on medical images of patients. We aim to contribute to the research in this field by: (i) Comparing different architectures on a public and extended reference dataset to find the most suitable; (ii) Proposing a patient-oriented investigation of the best performing networks; and (iii) Evaluating their robustness in a real-world scenario, represented by cross-dataset experiments. We exploited ten well-known convolutional neural networks on two public datasets. The results show that, on the reference dataset, the most suitable architecture is VGG19, which (i) Achieved 98.87% accuracy in the network comparison; (ii) Obtained 95.91% accuracy on the patient status classification, even though it misclassifies some patients that other networks classify correctly; and (iii) The cross-dataset experiments exhibit the limitations of deep learning approaches in a real-world scenario with 70.15% accuracy, which need further investigation to improve the robustness. Thus, VGG19 architecture showed promising performance in the classification of COVID-19 cases. Nonetheless, this architecture enables extensive improvements based on its modification, or even with preprocessing step in addition to it. Finally, the cross-dataset experiments exposed the critical weakness of classifying images from heterogeneous data sources, compatible with a real-world scenario.

scholarly journals Using deep learning for trajectory classification in imbalanced dataset

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Nicksson Ckayo Arrais de Freitas ◽  
Ticiana L. Coelho Da Silva ◽  
José Antônio Fernandes De Macêdo ◽  
Leopoldo Melo Júnioer
Keyword(s):  
Deep Learning ◽  
Classification Problem ◽  
Learning Model ◽  
Learning Approaches ◽  
Imbalanced Dataset ◽  
Imbalanced Datasets ◽  
Learning Techniques ◽  
Trajectory Classification ◽  
Low Performance ◽  
Deep Learning Model

Deep learning has gained much popularity in the past years due to GPU advancements, cloud computing improvements, and its supremacy, considering the accuracy results when trained on massive datasets. As with machine learning, deep learning models may experience low performance when handled with imbalanced datasets. In this paper, we focus on the trajectory classification problem, and we examine deep learning techniques for coping with imbalanced class data. We extend a deep learning model, called DeepeST (Deep Learning for Sub-Trajectory classification), to predict the class or label for sub-trajectories from imbalanced datasets. DeepeST is the first deep learning model for trajectory classification that provides approaches for coping with imbalanced dataset problems from the authors' knowledge. In this paper, we perform the experiments with three real datasets from LBSN (Location-Based Social Network) trajectories to identify who is the user of a sub-trajectory (similar to the Trajectory-User Linking problem). We show that DeepeST outperforms other deep learning approaches from state-of-the-art concerning the accuracy, precision, recall, and F1-score.

scholarly journals HetEnc: A Deep Learning Predictive Model for Multi-type Biological Dataset

2019 ◽  
Author(s):  
Leihong Wu ◽  
Xiangwen Liu ◽  
Joshua Xu
Keyword(s):  
Neural Network ◽  
Gene Expression ◽  
Deep Learning ◽  
Classification Problem ◽  
Biological Data ◽  
Feature Representation ◽  
Biological Research ◽  
Integrated Analysis ◽  
Learning Approaches ◽  
Feed Forward Neural Network

Abstract Background: Researchers today are generating unprecedented amounts of biological data. One trend in current biological research is integrated analysis with multi-platform data. Effective integration of multi-platform data into the solution of a single or multi-task classification problem; however, is critical and challenging. In this study, we proposed HetEnc, a novel deep learning-based approach, for information domain separation. Results: HetEnc includes both an unsupervised feature representation module and a supervised neural network module to handle multi-platform gene expression datasets. It first constructs three different encoding networks to represent the original gene expression data using high-level abstracted features. A six-layer fully-connected feed-forward neural network is then trained using these abstracted features for each targeted endpoint. We applied HetEnc to the SEQC neuroblastoma dataset to demonstrate that it outperforms other machine learning approaches. Although we used multi-platform data in feature abstraction and model training, HetEnc does not need multi-platform data for prediction, enabling a broader application of the trained model by reducing the cost of gene expression profiling for new samples to a single platform. Thus, HetEnc provides a new solution to integrated gene expression analysis, accelerating modern biological research.

scholarly journals The HSIC Bottleneck: Deep Learning without Back-Propagation

2020 ◽  
Vol 34 (04) ◽  
pp. 5085-5092 ◽  
Author(s):  
Wan-Duo Kurt Ma ◽  
J. P. Lewis ◽  
W. Bastiaan Kleijn
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Back Propagation ◽  
Single Layer ◽  
Cross Entropy ◽  
Entropy Loss ◽  
Deep Networks ◽  
Independence Criterion

We introduce the HSIC (Hilbert-Schmidt independence criterion) bottleneck for training deep neural networks. The HSIC bottleneck is an alternative to the conventional cross-entropy loss and backpropagation that has a number of distinct advantages. It mitigates exploding and vanishing gradients, resulting in the ability to learn very deep networks without skip connections. There is no requirement for symmetric feedback or update locking. We find that the HSIC bottleneck provides performance on MNIST/FashionMNIST/CIFAR10 classification comparable to backpropagation with a cross-entropy target, even when the system is not encouraged to make the output resemble the classification labels. Appending a single layer trained with SGD (without backpropagation) to reformat the information further improves performance.

scholarly journals HetEnc: A Deep Learning Predictive Model for Multi-type Biological Dataset

2019 ◽  
Author(s):  
Leihong Wu ◽  
Xiangwen Liu ◽  
Joshua Xu
Keyword(s):  
Neural Network ◽  
Gene Expression ◽  
Deep Learning ◽  
Classification Problem ◽  
Biological Data ◽  
Feature Representation ◽  
Biological Research ◽  
Integrated Analysis ◽  
Learning Approaches ◽  
Feed Forward Neural Network

Abstract Background: Researchers today are generating unprecedented amounts of biological data. One trend in current biological research is integrated analysis with multi-platform data. Effective integration of multi-platform data into the solution of a single or multi-task classification problem; however, is critical and challenging. In this study, we proposed HetEnc, a novel deep learning-based approach, for information domain separation. Results: HetEnc includes both an unsupervised feature representation module and a supervised neural network module to handle multi-platform gene expression datasets. It first constructs three different encoding networks to represent the original gene expression data using high-level abstracted features. A six-layer fully-connected feed-forward neural network is then trained using these abstracted features for each targeted endpoint. We applied HetEnc to the SEQC neuroblastoma dataset to demonstrate that it outperforms other machine learning approaches. Although we used multi-platform data in feature abstraction and model training, HetEnc does not need multi-platform data for prediction, enabling a broader application of the trained model by reducing the cost of gene expression profiling for new samples to a single platform. Thus, HetEnc provides a new solution to integrated gene expression analysis, accelerating modern biological research.

scholarly journals Regression Loss in Transformer-based Supervised Neural Machine Translation

2021 ◽  
Vol 16 (4) ◽  
Author(s):  
Dongxing Li ◽  
Zuying Luo
Keyword(s):  
Machine Translation ◽  
Loss Function ◽  
Likelihood Estimation ◽  
Classification Problem ◽  
Cross Entropy ◽  
Slight Reduction ◽  
Neural Machine Translation ◽  
Entropy Loss ◽  
Semantic Difference ◽  
Reference Distance

Transformer-based model has achieved human-level performance in supervised neural machine translation (SNMT), much better than the models based on recurrent neural networks (RNNs) or convolutional neural network (CNN). The original Transformer-based model is trained through maximum likelihood estimation (MLE), which regards the machine translation task as a multilabel classification problem and takes the sum of the cross entropy loss of all the target tokens as the loss function. However, this model assumes that token generation is partially independent, without realizing that tokens are the components of a sequence. To solve the problem, this paper proposes a semantic regression loss for Transformer training, treating the generated sequence as a global. Upon finding that the semantic difference is proportional to candidate-reference distance, the authors considered the machine translation problem as a multi-task problem, and took the linear combination of cross entropy loss and semantic regression loss as the overall loss function. The semantic regression loss was proved to significantly enhance SNMT performance, with a slight reduction in convergence speed.

Sign in / Sign up

Export Citation Format

Share Document