A Graph Regularized Deep Neural Network for Unsupervised Image Representation Learning

With the rapid expanding of big data in all domains, data-driven and deep learning-based fault diagnosis methods in chemical industry have become a major research topic in recent years. In addition to a deep neural network, deep forest also provides a new idea for deep representation learning and overcomes the shortcomings of a deep neural network such as strong parameter dependence and large training cost. However, the ability of each base classifier is not taken into account in the standard cascade forest, which may lead to its indistinct discrimination. In this paper, a multigrained scanning-based weighted cascade forest (WCForest) is proposed and has been applied to fault diagnosis in chemical processes. In view of the high-dimensional nonlinear data in the process of chemical industry, WCForest first designs a set of relatively suitable windows for the multigrained scan strategy to learn its data representation. Next, considering the fitting quality of each forest classifier, a weighting strategy is proposed to calculate the weight of each forest in the cascade structure without additional calculation cost, so as to improve the overall performance of the model. In order to prove the effectiveness of WCForest, its application has been carried out in the benchmark Tennessee Eastman (TE) process. Experiments demonstrate that WCForest achieves better results than other related approaches across various evaluation metrics.

Download Full-text

A learning-based method for drug-target interaction prediction based on feature representation learning and deep neural network

BMC Bioinformatics ◽

10.1186/s12859-020-03677-1 ◽

2020 ◽

Vol 21 (S13) ◽

Author(s):

Jiajie Peng ◽

Jingyi Li ◽

Xuequn Shang

Keyword(s):

Neural Network ◽

Drug Discovery ◽

Drug Target ◽

Deep Neural Network ◽

Computational Prediction ◽

Representation Learning ◽

Feature Representation ◽

New Drugs ◽

Target Interaction ◽

Interaction Prediction

Abstract Background Drug-target interaction prediction is of great significance for narrowing down the scope of candidate medications, and thus is a vital step in drug discovery. Because of the particularity of biochemical experiments, the development of new drugs is not only costly, but also time-consuming. Therefore, the computational prediction of drug target interactions has become an essential way in the process of drug discovery, aiming to greatly reducing the experimental cost and time. Results We propose a learning-based method based on feature representation learning and deep neural network named DTI-CNN to predict the drug-target interactions. We first extract the relevant features of drugs and proteins from heterogeneous networks by using the Jaccard similarity coefficient and restart random walk model. Then, we adopt a denoising autoencoder model to reduce the dimension and identify the essential features. Third, based on the features obtained from last step, we constructed a convolutional neural network model to predict the interaction between drugs and proteins. The evaluation results show that the average AUROC score and AUPR score of DTI-CNN were 0.9416 and 0.9499, which obtains better performance than the other three existing state-of-the-art methods. Conclusions All the experimental results show that the performance of DTI-CNN is better than that of the three existing methods and the proposed method is appropriately designed.

Download Full-text

Domain-invariant representation learning using an unsupervised domain adversarial adaptation deep neural network

Neurocomputing ◽

10.1016/j.neucom.2019.04.033 ◽

2019 ◽

Vol 355 ◽

pp. 209-220 ◽

Cited By ~ 3

Author(s):

Xibin Jia ◽

Ya Jin ◽

Xing Su ◽

Yongli Hu

Keyword(s):

Neural Network ◽

Deep Neural Network ◽

Representation Learning ◽

Invariant Representation

Download Full-text

A Bearing Fault Diagnosis Method Using Multi-Branch Deep Neural Network

Machines ◽

10.3390/machines9120345 ◽

2021 ◽

Vol 9 (12) ◽

pp. 345

Author(s):

Van-Cuong Nguyen ◽

Duy-Tang Hoang ◽

Xuan-Toa Tran ◽

Mien Van ◽

Hee-Jun Kang

Keyword(s):

Neural Network ◽

Feature Extraction ◽

Fault Diagnosis ◽

Deep Neural Network ◽

Image Representation ◽

Vibration Signal ◽

Bearing Fault ◽

Bearing Fault Diagnosis ◽

Diagnosis Method ◽

Multiple Domain

Feature extraction from a signal is the most important step in signal-based fault diagnosis. Deep learning or deep neural network (DNN) is an effective method to extract features from signals. In this paper, a novel vibration signal-based bearing fault diagnosis method using DNN is proposed. First, the measured vibration signals are transformed into a new data form called multiple-domain image-representation. By this transformation, the task of signal-based fault diagnosis is transferred into the task of image classification. After that, a DNN with a multi-branch structure is proposed to handle the multiple-domain image representation data. The multi-branch structure of the proposed DNN helps to extract features in multiple domains simultaneously, and to lead to better feature extraction. Better feature extraction leads to a better performance of fault diagnosis. The effectiveness of the proposed method was verified via the experiments conducted with actual bearing fault signals and its comparisons with well-established published methods.

Download Full-text

Autoencoder Regularized Network For Driving Style Representation Learning

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2017/222 ◽

2017 ◽

Cited By ~ 12

Author(s):

Weishan Dong ◽

Ting Yuan ◽

Kai Yang ◽

Changsheng Li ◽

Shilei Zhang

Keyword(s):

Neural Network ◽

Deep Neural Network ◽

Estimation Error ◽

Feature Learning ◽

Representation Learning ◽

Identification Problem ◽

Identification Accuracy ◽

Unsupervised Feature Learning ◽

Driving Style ◽

Driver Identification

In this paper, we study learning generalized driving style representations from automobile GPS trip data. We propose a novel Autoencoder Regularized deep neural Network (ARNet) and a trip encoding framework trip2vec to learn drivers' driving styles directly from GPS records, by combining supervised and unsupervised feature learning in a unified architecture. Experiments on a challenging driver number estimation problem and the driver identification problem show that ARNet can learn a good generalized driving style representation: It significantly outperforms existing methods and alternative architectures by reaching the least estimation error on average (0.68, less than one driver) and the highest identification accuracy (by at least 3% improvement) compared with traditional supervised learning methods.

Download Full-text

Chapter 14. Combining Probabilistic Logic and Deep Learning for Self-Supervised Learning

10.3233/faia210361 ◽

2021 ◽

Author(s):

Hoifung Poon ◽

Hai Wang ◽

Hunter Lang

Keyword(s):

Neural Network ◽

Deep Learning ◽

Supervised Learning ◽

Latent Variables ◽

Domain Knowledge ◽

Deep Neural Network ◽

Representation Learning ◽

Probabilistic Logic ◽

Initial Seed ◽

Training Examples

Deep learning has proven effective for various application tasks, but its applicability is limited by the reliance on annotated examples. Self-supervised learning has emerged as a promising direction to alleviate the supervision bottleneck, but existing work focuses on leveraging co-occurrences in unlabeled data for task-agnostic representation learning, as exemplified by masked language model pretraining. In this chapter, we explore task-specific self-supervision, which leverages domain knowledge to automatically annotate noisy training examples for end applications, either by introducing labeling functions for annotating individual instances, or by imposing constraints over interdependent label decisions. We first present deep probabilistic logic (DPL), which offers a unifying framework for task-specific self-supervision by composing probabilistic logic with deep learning. DPL represents unknown labels as latent variables and incorporates diverse self-supervision using probabilistic logic to train a deep neural network end-to-end using variational EM. Next, we present self-supervised self-supervision (S4), which adds to DPL the capability to learn new self-supervision automatically. Starting from an initial seed self-supervision, S4 iteratively uses the deep neural network to propose new self supervision. These are either added directly (a form of structured self-training) or verified by a human expert (as in feature-based active learning). Experiments on real-world applications such as biomedical machine reading and various text classification tasks show that task-specific self-supervision can effectively leverage domain expertise and often match the accuracy of supervised methods with a tiny fraction of human effort.

Download Full-text

Label-Attended Hashing for Multi-Label Image Retrieval

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/133 ◽

2020 ◽

Author(s):

Yanzhao Xie ◽

Yu Liu ◽

Yangtao Wang ◽

Lianli Gao ◽

Peng Wang ◽

...

Keyword(s):

Neural Network ◽

Feature Extraction ◽

Image Retrieval ◽

Hash Function ◽

State Of The Art ◽

Image Representation ◽

Back Propagation ◽

Cauchy Distribution ◽

Representation Learning ◽

Hash Codes

For the multi-label image retrieval, the existing hashing algorithms neglect the dependency between objects and thus fail to capture the attention information in the feature extraction, which affects the precision of hash codes. To address this problem, we explore the inter-dependency between objects through their co-occurrence correlation from the label set and adopt Multi-modal Factorized Bilinear (MFB) pooling component so that the image representation learning can capture this attention information. We propose a Label-Attended Hashing (LAH) algorithm which enables an end-to-end hash model with inter-dependency feature extraction. LAH first combines Convolutional Neural Network (CNN) and Graph Convolution Network (GCN) to separately generate the image representation and label co-occurrence embeddings, then adopts MFB to fuse these two modal vectors, finally learns the hash function with a Cauchy distribution based loss function via back propagation. Extensive experiments on public multi-label datasets demonstrate that (1) LAH can achieve the state-of-the-art retrieval results and (2) the usage of co-occurrence relationship and MFB not only promotes the precision of hash codes but also accelerates the hash learning. GitHub address: https://github.com/IDSM-AI/LAH.

Download Full-text