Self-Supervised Learning of a Deep Neural Network for Ultrafast Ultrasound Imaging as an Inverse Problem

AbstractProtein contact prediction is a crucially important step for protein structure prediction. To predict a contact, approaches of two types are used: evolutionary coupling analysis (ECA) and supervised learning. ECA uses a large multiple sequence alignment (MSA) of homologue sequences and extract correlation information between residues. Supervised learning uses ECA analysis results as input features and can produce higher accuracy. As described herein, we present a new approach to contact prediction which can both extract correlation information and predict contacts in a supervised manner directly from MSA using a deep neural network (DNN). Using DNN, we can obtain higher accuracy than with earlier ECA methods. Simultaneously, we can weight each sequence in MSA to eliminate noise sequences automatically in a supervised way. It is expected that the combination of our method and other meta-learning methods can provide much higher accuracy of contact prediction.

Download Full-text

Compression and Decompression which Use Deep Neural Network

10.36227/techrxiv.13514158.v2 ◽

2022 ◽

Author(s):

Jinxin Wei

Keyword(s):

Neural Network ◽

Supervised Learning ◽

Compression Ratio ◽

Deep Neural Network ◽

Inverse Function

an auto-encoder which can be split into two parts is designed. The two parts can work well separately. The top half is an abstract network which is trained by supervised learning and can be used to classify and regress. The bottom half is a concrete network which is accomplished by inverse function and trained by self-supervised learning. It can generate the input of abstract network from concept or label. It is tested by tensorflow and mnist dataset. The abstract network is like LeNet-5. The concrete network is the inverse of the abstract network.Lossy compression can achieved by the test. The large compression ratio which is 19.6 is achieved. The decompression performance is ok through regression which treats classification as regression.

Download Full-text

Compression and Decompression which Use Deep Neural Network

10.36227/techrxiv.13514158 ◽

2021 ◽

Author(s):

Jinxin Wei

Keyword(s):

Neural Network ◽

Supervised Learning ◽

Compression Ratio ◽

Deep Neural Network ◽

Inverse Function

an auto-encoder which can be split into two parts is designed. The two parts can work well separately. The top half is an abstract network which is trained by supervised learning and can be used to classify and regress. The bottom half is a concrete network which is accomplished by inverse function and trained by self-supervised learning. It can generate the input of abstract network from concept or label. It is tested by tensorflow and mnist dataset. The abstract network is like LeNet-5. The concrete network is the inverse of the abstract network.Lossy compression can achieved by the test. The large compression ratio which is 19.6 is achieved. The decompression performance is ok through regression which treats classification as regression.

Download Full-text

A Deep Learning Algorithm for the Max-Cut Problem Based on Pointer Network Structure with Supervised Learning and Reinforcement Learning Strategies

Mathematics ◽

10.3390/math8020298 ◽

2020 ◽

Vol 8 (2) ◽

pp. 298 ◽

Cited By ~ 2

Author(s):

Shenshen Gu ◽

Yue Yang

Keyword(s):

Neural Network ◽

Deep Learning ◽

Reinforcement Learning ◽

Combinatorial Optimization ◽

Supervised Learning ◽

Learning Strategies ◽

Large Scale ◽

Deep Neural Network ◽

Max Cut Problem ◽

Cut Problems

The Max-cut problem is a well-known combinatorial optimization problem, which has many real-world applications. However, the problem has been proven to be non-deterministic polynomial-hard (NP-hard), which means that exact solution algorithms are not suitable for large-scale situations, as it is too time-consuming to obtain a solution. Therefore, designing heuristic algorithms is a promising but challenging direction to effectively solve large-scale Max-cut problems. For this reason, we propose a unique method which combines a pointer network and two deep learning strategies (supervised learning and reinforcement learning) in this paper, in order to address this challenge. A pointer network is a sequence-to-sequence deep neural network, which can extract data features in a purely data-driven way to discover the hidden laws behind data. Combining the characteristics of the Max-cut problem, we designed the input and output mechanisms of the pointer network model, and we used supervised learning and reinforcement learning to train the model to evaluate the model performance. Through experiments, we illustrated that our model can be well applied to solve large-scale Max-cut problems. Our experimental results also revealed that the new method will further encourage broader exploration of deep neural network for large-scale combinatorial optimization problems.

Download Full-text

Assessment of Landslide Susceptibility Combining Deep Learning with Semi-Supervised Learning in Jiaohe County, Jilin Province, China

Applied Sciences ◽

10.3390/app10165640 ◽

2020 ◽

Vol 10 (16) ◽

pp. 5640

Author(s):

Jingyu Yao ◽

Shengwu Qin ◽

Shuangshuang Qiao ◽

Wenchao Che ◽

Yang Chen ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Supervised Learning ◽

Landslide Susceptibility ◽

Deep Neural Network ◽

Spatial Information ◽

Information Gain ◽

Jilin Province ◽

Landslide Susceptibility Mapping ◽

Support Vector

Accurate and timely landslide susceptibility mapping (LSM) is essential to effectively reduce the risk of landslide. In recent years, deep learning has been successfully applied to landslide susceptibility assessment due to the strong ability of fitting. However, in actual applications, the number of labeled samples is usually not sufficient for the training component. In this paper, a deep neural network model based on semi-supervised learning (SSL-DNN) for landslide susceptibility is proposed, which makes full use of a large number of spatial information (unlabeled data) with limited labeled data in the region to train the mode. Taking Jiaohe County in Jilin Province, China as an example, the landslide inventory from 2000 to 2017 was collected and 12 metrological, geographical, and human explanatory factors were compiled. Meanwhile, supervised models such as deep neural network (DNN), support vector machine (SVM), and logistic regression (LR) were implemented for comparison. Then, the landslide susceptibility was plotted and a series of evaluation tools such as class accuracy, predictive rate curves (AUC), and information gain ratio (IGR) were calculated to compare the prediction of models and factors. Experimental results indicate that the proposed SSL-DNN model (AUC = 0.898) outperformed all the comparison models. Therefore, semi-supervised deep learning could be considered as a potential approach for LSM.

Download Full-text

A Deep Neural Network Based on ELM for Semi-supervised Learning of Image Classification

Neural Processing Letters ◽

10.1007/s11063-017-9709-0 ◽

2017 ◽

Vol 48 (1) ◽

pp. 375-388 ◽

Cited By ~ 5

Author(s):

Peiju Chang ◽

Jiangshe Zhang ◽

Junying Hu ◽

Zengjie Song

Keyword(s):

Neural Network ◽

Image Classification ◽

Supervised Learning ◽

Deep Neural Network

Download Full-text

Model Prediksi Interaksi Senyawa dan Protein untuk Drug Repositioning menggunakan Deep Semi-Supervised Learning

Jurnal Teknologi Informasi dan Ilmu Komputer ◽

10.25126/jtiik.2020742236 ◽

2020 ◽

Vol 7 (4) ◽

pp. 727

Author(s):

Larasati Larasati ◽

Wisnu Ananta Kusuma ◽

Annisa Annisa

Keyword(s):

Neural Network ◽

Ion Channel ◽

Unsupervised Learning ◽

Supervised Learning ◽

Nuclear Receptor ◽

Deep Neural Network ◽

Drug Repositioning ◽

Imbalanced Data ◽

Unlabeled Data ◽

F Measure

Drug repositioning adalah penggunaan senyawa obat yang sudah lolos uji sebelumnya untuk mengatasi penyakit baru selain penyakit awal obat tersebut ditujukan. Drug repositioning dapat dilakukan dengan memprediksi interaksi senyawa obat dengan protein penyakit yang bereaksi positif. Salah satu tantangan dalam prediksi interaksi senyawa dan protein adalah masalah ketidakseimbangan data. Deep semi-supervised learning dapat menjadi alternatif untuk menangani model prediksi dengan data yang tidak seimbang. Proses pre-training berbasis unsupervised learning pada deep semi-supervised learning dapat merepresentasikan input dari unlabeled data (data mayoritas) dengan baik dan mengoptimasi inisialisasi bobot pada classifier. Penelitian ini mengimplementasikan Deep Belief Network (DBN) sebagai pre-training dan Deep Neural Network (DNN) sebagai classifier. Data yang digunakan pada penelitian ini adalah dataset ion channel, GPCR, dan nuclear receptor yang bersumber dari pangkalan data KEGG BRITE, BRENDA, SuperTarget, dan DrugBank. Hasil penelitian ini menunjukkan pada dataset tersebut, pre-training berupa ekstraksi fitur memberikan efek optimasi dilihat dari peningkatan performa model DNN pada akurasi (3-4.5%), AUC (4.5%), precision (5.9-6%), dan F-measure (3.8%). AbstractDrug repositioning is the reuse of an existing drug to treat a new disease other than its original medical indication. Drug repositioning can be done by predicting the interaction of drug compounds with disease proteins that react positively. One of the challenges in predicting the interaction of compounds and proteins is imbalanced data. Deep semi-supervised learning can be an alternative to handle prediction models with imbalanced data. The unsupervised learning based pre-training process in deep semi-supervised learning can represent input from unlabeled data (majority data) properly and optimize initialization of weights on the classifier. This study implements the Deep Belief Network (DBN) as a pre-training with Deep Neural Network (DNN) as a classifier. The data used in this study are ion channel, GPCR, and nuclear receptor dataset sourced from KEGG BRITE, BRENDA, SuperTarget, and DrugBank databases. The results of this study indicate that pre-training as feature extraction had an optimization effect. This can be seen from DNN performance improvement in accuracy (3-4.5%), AUC (4.5%), precision (5.9-6%), and F-measure (3.8%).

Download Full-text