scholarly journals PresB-Net: parametric binarized neural network with learnable activations and shuffled grouped convolution

2022 ◽  
Vol 8 ◽  
pp. e842
Author(s):  
Jungwoo Shin ◽  
HyunJin Kim
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
State Of The Art ◽  
Classification Performance ◽  
Performance Degradation ◽  
Critical Problem ◽  
Hardware Costs ◽  
Performance Enhancing

In this study, we present a novel performance-enhancing binarized neural network model called PresB-Net: Parametric Binarized Neural Network. A binarized neural network (BNN) model can achieve fast output computation with low hardware costs by using binarized weights and features. However, performance degradation is the most critical problem in BNN models. Our PresB-Net combines several state-of-the-art BNN structures including the learnable activation with additional trainable parameters and shuffled grouped convolution. Notably, we propose a new normalization approach, which reduces the imbalance between the shuffled groups occurring in shuffled grouped convolutions. Besides, the proposed normalization approach helps gradient convergence so that the unstableness of the learning can be amortized when applying the learnable activation. Our novel BNN model enhances the classification performance compared with other existing BNN models. Notably, the proposed PresB-Net-18 achieves 73.84% Top-1 inference accuracy for the CIFAR-100 dataset, outperforming other existing counterparts.

scholarly journals An Efficient Deep Learning Approach to Pneumonia Classification in Healthcare

2019 ◽  
Vol 2019 ◽  
pp. 1-7 ◽  
Author(s):  
Okeke Stephen ◽  
Mangal Sain ◽  
Uchenna Joseph Maduh ◽  
Do-Un Jeong
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Data Augmentation ◽  
Classification Performance ◽  
Learning Approaches ◽  
X Ray ◽  
Chest X Ray

This study proposes a convolutional neural network model trained from scratch to classify and detect the presence of pneumonia from a collection of chest X-ray image samples. Unlike other methods that rely solely on transfer learning approaches or traditional handcrafted techniques to achieve a remarkable classification performance, we constructed a convolutional neural network model from scratch to extract features from a given chest X-ray image and classify it to determine if a person is infected with pneumonia. This model could help mitigate the reliability and interpretability challenges often faced when dealing with medical imagery. Unlike other deep learning classification tasks with sufficient image repository, it is difficult to obtain a large amount of pneumonia dataset for this classification task; therefore, we deployed several data augmentation algorithms to improve the validation and classification accuracy of the CNN model and achieved remarkable validation accuracy.

scholarly journals DeepSeqPanII: an interpretable recurrent neural network model with attention mechanism for peptide-HLA class II binding prediction

2020 ◽  
Author(s):  
ZHONGHAO LIU ◽  
Jing Jin ◽  
Yuxin Cui ◽  
Zheng Xiong ◽  
Alireza Nasiri ◽  
...  
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Structure ◽  
State Of The Art ◽  
Class Ii ◽  
Hla Class Ii ◽  
Binding Prediction ◽  
Neural Network Structure ◽  
Art Performance

Abstract Background: Human leukocyte antigen (HLA) complex molecules play an essential role in immune interactions by presenting peptides on the cell surface to T cells. With significant progress in deep learning, a series of neural network based models have been proposed and demonstrated with their good performances for peptide-HLA class I binding prediction. However, there still lack effective binding prediction models for HLA class II protein binding with peptides due to its inherent challenges. In this work, we present a novel sequence-based pan-specific neural network structure, DeepSeaPanII, for peptide-HLA class II binding prediction. Compared with existing pan-specific models, our model is an end-to-end neural network model without the need for pre- or post-processing on input samples. Results: The leave-one-allele-out cross validation and benchmark evaluation results show that our proposed network model achieved state-of-the-art performance in HLA-II peptide binding. Besides state-of-the-art performance in binding affinity prediction, DeepSeqPanII can also extract biological insight on the binding mechanism over the peptide and HLA sequences by its attention mechanism based binding core prediction capability. Conclusions: In this work, we present a novel neural network structure for peptide-HLA class II binding prediction. It has state-of-the-art performance and could display insightful information it learned benefiting from attention module we carefully designed. Without requiring additional data, this structure could be applied to other related sequence problems. The source code and trained models are freely available at https://github.com/pcpLiu/DeepSeqPanII.

scholarly journals Spatiotemporal Graph Convolutional Recurrent Neural Network Model for Citywide Air Pollution Forecasting

2021 ◽  
Author(s):  
Van-Duc Le
Keyword(s):  
Neural Network ◽  
Air Pollution ◽  
Network Model ◽  
Recurrent Neural Network ◽  
Neural Network Model ◽  
State Of The Art ◽  
Influential Factors ◽  
Temporal Features ◽  
Air Pollution Forecasting

This paper applies a Spatiotemporal Graph Convolutional Recurrent Neural Network which is a tight combination of a Graph Neural Network (GNN) to a Recurrent Neural Network (RNN) architecture for air pollution forecasting in long-term for the entire city. Our model can effectively learn the spatial and temporal features of the air pollution data and its influential factors (e.g. weather, traffic, external areas) at the time. Our method achieves better performance than a state-of-the-art ConvLSTM model in air pollution forecasting and a hybrid GNN-based model that separates GNN and RNN in discrete layers.

Performing Multi-Target Regression via a Parameter Sharing-Based Deep Network

2019 ◽  
Vol 29 (09) ◽  
pp. 1950014 ◽  
Author(s):  
Oscar Reyes ◽  
Sebastián Ventura
Keyword(s):  
Neural Network ◽  
Experimental Study ◽  
Network Model ◽  
Neural Network Model ◽  
State Of The Art ◽  
Experimental Results ◽  
Input Output ◽  
Deep Architecture ◽  
Proposed Model ◽  
Input Variables

Multi-target regression (MTR) comprises the prediction of multiple continuous target variables from a common set of input variables. There are two major challenges when addressing the MTR problem: the exploration of the inter-target dependencies and the modeling of complex input–output relationships. This paper proposes a neural network model that is able to simultaneously address these two challenges in a flexible way. A deep architecture well suited for learning multiple continuous outputs is designed, providing some flexibility to model the inter-target relationships by sharing network parameters as well as the possibility to exploit target-specific patterns by learning a set of nonshared parameters for each target. The effectiveness of the proposal is analyzed through an extensive experimental study on 18 datasets, demonstrating the benefits of using a shared representation that exploits the commonalities between target variables. According to the experimental results, the proposed model is competitive with respect to the state-of-the-art in MTR.

scholarly journals A novel framework for Automatic Chinese Question Generation based on multi-feature neural network model

2018 ◽  
Vol 15 (3) ◽  
pp. 487-499 ◽  
Author(s):  
Hai-Tao Zheng ◽  
Jinxin Han ◽  
Jinyuan Chen ◽  
Arun Sangaiah
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Language Processing ◽  
Neural Network Model ◽  
State Of The Art ◽  
Ranking Methods ◽  
Question Generation ◽  
Human Evaluation ◽  
Automatic Question Generation

Automatic question generation from text or paragraph is a great challenging task which attracts broad attention in natural language processing. Because of the verbose texts and fragile ranking methods, the quality of top generated questions is poor. In this paper, we present a novel framework Automatic Chinese Question Generation (ACQG) to generate questions from text or paragraph. In ACQG, we use an adopted TextRank to extract key sentences and a template-based method to construct questions from key sentences. Then a multi-feature neural network model is built for ranking to obtain the top questions. The automatic evaluation result reveals that the proposed framework outperforms the state-of-the-art systems in terms of perplexity. In human evaluation, questions generated by ACQG rate a higher score.

scholarly journals DeepSeqPanII: an interpretable recurrent neural network model with attention mechanism for peptide-HLA class II binding prediction

2019 ◽  
Author(s):  
Zhonghao Liu ◽  
Jing Jin ◽  
Yuxin Cui ◽  
Zheng Xiong ◽  
Alireza Nasiri ◽  
...  
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Prediction Models ◽  
State Of The Art ◽  
Human Leukocyte ◽  
Class Ii ◽  
Hla Class Ii ◽  
Attention Mechanism ◽  
Binding Prediction

AbstractHuman leukocyte antigen (HLA) complex molecules play an essential role in immune interactions by presenting peptides on the cell surface to T cells. With significant progress in deep learning, a series of neural network based models have been proposed and demonstrated with their good performances for peptide-HLA class I binding prediction. However, there still lack effective binding prediction models for HLA class II protein binding with peptides due to its inherent challenges. In this work, we present a novel sequence-based pan-specific neural network structure, DeepSeaPanII, for peptide-HLA class II binding prediction. Compared with existing pan-specific models, our model is an end-to-end neural network model without the need for pre- or post-processing on input samples. Besides state-of-the-art peformance in binding affinity prediction, DeepSeqPanII can also extract biological insight on the binding mechanism over the peptide and HLA sequences by its attention mechanism based binding core prediction capability. The leave-one-allele-out cross validation and benchmark evaluation results show that our proposed network model achieved state-of-the-art performance in HLA-II peptide binding. The source code and trained models are freely available at https://github.com/pcpLiu/DeepSeqPanII.

scholarly journals MultiCapsNet: A General Framework for Data Integration and Interpretable Classification

2021 ◽  
Vol 12 ◽  
Author(s):  
Lifei Wang ◽  
Xuexia Miao ◽  
Rui Nie ◽  
Zhang Zhang ◽  
Jiang Zhang ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Random Forest ◽  
Data Integration ◽  
Network Model ◽  
Neural Network Model ◽  
Dna Interaction ◽  
Classification Performance ◽  
Cell Type ◽  
The Neural Network

The latest progresses of experimental biology have generated a large number of data with different formats and lengths. Deep learning is an ideal tool to deal with complex datasets, but its inherent “black box” nature needs more interpretability. At the same time, traditional interpretable machine learning methods, such as linear regression or random forest, could only deal with numerical features instead of modular features often encountered in the biological field. Here, we present MultiCapsNet (https://github.com/wanglf19/MultiCapsNet), a new deep learning model built on CapsNet and scCapsNet, which possesses the merits such as easy data integration and high model interpretability. To demonstrate the ability of this model as an interpretable classifier to deal with modular inputs, we test MultiCapsNet on three datasets with different data type and application scenarios. Firstly, on the labeled variant call dataset, MultiCapsNet shows a similar classification performance with neural network model, and provides importance scores for data sources directly without an extra importance determination step required by the neural network model. The importance scores generated by these two models are highly correlated. Secondly, on single cell RNA sequence (scRNA-seq) dataset, MultiCapsNet integrates information about protein-protein interaction (PPI), and protein-DNA interaction (PDI). The classification accuracy of MultiCapsNet is comparable to the neural network and random forest model. Meanwhile, MultiCapsNet reveals how each transcription factor (TF) or PPI cluster node contributes to classification of cell type. Thirdly, we made a comparison between MultiCapsNet and SCENIC. The results show several cell type relevant TFs identified by both methods, further proving the validity and interpretability of the MultiCapsNet.

Sign in / Sign up

Export Citation Format

Share Document