Deep learning based network similarity for model selection

Capturing data in the form of networks is becoming an increasingly popular approach for modeling, analyzing and visualising complex phenomena, to understand the important properties of the underlying complex processes. Access to many large-scale network datasets is restricted due to the privacy and security concerns. Also for several applications (such as functional connectivity networks), generating large scale real data is expensive. For these reasons, there is a growing need for advanced mathematical and statistical models (also called generative models) that can account for the structure of these large-scale networks, without having to materialize them in the real world. The objective is to provide a comprehensible description of the network properties and to be able to infer previously unobserved properties. Various models have been developed by researchers, which generate synthetic networks that adhere to the structural properties of real networks. However, the selection of the appropriate generative model for a given real-world network remains an important challenge. In this paper, we investigate this problem and provide a novel technique (named as TripletFit) for model selection (or network classification) and estimation of structural similarities of the complex networks. The goal of network model selection is to select a generative model that is able to generate a structurally similar synthetic network for a given real-world (target) network. We consider six outstanding generative models as the candidate models. The existing model selection methods mostly suffer from sensitivity to network perturbations, dependency on the size of the networks, and low accuracy. To overcome these limitations, we considered a broad array of network features, with the aim of representing different structural aspects of the network and employed deep learning techniques such as deep triplet network architecture and simple feed-forward network for model selection and estimation of structural similarities of the complex networks. Our proposed method, outperforms existing methods with respect to accuracy, noise-tolerance, and size independence on a number of gold standard data set used in previous studies.

Download Full-text

Link Prediction through Deep Generative Model

10.1101/247577 ◽

2018 ◽

Author(s):

Xu-Wen Wang ◽

Yize Chen ◽

Yang-Yu Liu

Keyword(s):

Complex Networks ◽

Real World ◽

Link Prediction ◽

Prediction Method ◽

Generative Models ◽

Generative Model ◽

Superior Performance ◽

Prediction Problem ◽

Structural Patterns ◽

Domain Specific

AbstractInferring missing links or predicting future ones based on the currently observed network is known as link prediction, which has tremendous real-world applications in biomedicine1–3, e-commerce4, social media5 and criminal intelligence6. Numerous methods have been proposed to solve the link prediction problem7–9. Yet, many of these existing methods are designed for undirected networks only. Moreover, most methods are based on domain-specific heuristics10, and hence their performances differ greatly for networks from different domains. Here we developed a new link prediction method based on deep generative models11 in machine learning. This method does not rely on any domain-specific heuristic and works for general undirected or directed complex networks. Our key idea is to represent the adjacency matrix of a network as an image and then learn hierarchical feature representations of the image by training a deep generative model. Those features correspond to structural patterns in the network at different scales, from small subgraphs to mesoscopic communities12. Conceptually, taking into account structural patterns at different scales all together should outperform any domain-specific heuristics that typically focus on structural patterns at a particular scale. Indeed, when applied to various real-world networks from different domains13–17, our method shows overall superior performance against existing methods. Moreover, it can be easily parallelized by splitting a large network into several small subnetworks and then perform link prediction for each subnetwork in parallel. Our results imply that deep learning techniques can be effectively applied to complex networks and solve the classical link prediction problem with robust and superior performance.SummaryWe propose a new link prediction method based on deep generative models.

Download Full-text

Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽

10.3390/s21082852 ◽

2021 ◽

Vol 21 (8) ◽

pp. 2852

Author(s):

Parvathaneni Naga Srinivasu ◽

Jalluri Gnana SivaSai ◽

Muhammad Fazal Ijaz ◽

Akash Kumar Bhoi ◽

Wonjoon Kim ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Skin Disease ◽

Network Architecture ◽

Large Scale ◽

Short Term Memory ◽

Convolutional Networks ◽

Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

Download Full-text

Curriculum Learning for Natural Answer Generation

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/587 ◽

2018 ◽

Cited By ~ 11

Author(s):

Cao Liu ◽

Shizhu He ◽

Kang Liu ◽

Jun Zhao

Keyword(s):

Full Advantage ◽

Real World ◽

Large Scale ◽

Question Answering ◽

Generative Models ◽

The Internet ◽

Basic Model ◽

The Arts ◽

Learning Data

By reason of being able to obtain natural language responses, natural answers are more favored in real-world Question Answering (QA) systems. Generative models learn to automatically generate natural answers from large-scale question answer pairs (QA-pairs). However, they are suffering from the uncontrollable and uneven quality of QA-pairs crawled from the Internet. To address this problem, we propose a curriculum learning based framework for natural answer generation (CL-NAG), which is able to take full advantage of the valuable learning data from a noisy and uneven-quality corpus. Specifically, we employ two practical measures to automatically measure the quality (complexity) of QA-pairs. Based on the measurements, CL-NAG firstly utilizes simple and low-quality QA-pairs to learn a basic model, and then gradually learns to produce better answers with richer contents and more complete syntaxes based on more complex and higher-quality QA-pairs. In this way, all valuable information in the noisy and uneven-quality corpus could be fully exploited. Experiments demonstrate that CL-NAG outperforms the state-of-the-arts, which increases 6.8% and 8.7% in the accuracy for simple and complex questions, respectively.

Download Full-text

Characterizing the Feasibility and Performance of Real-World Tumor Progression End Points and Their Association With Overall Survival in a Large Advanced Non–Small-Cell Lung Cancer Data Set

JCO Clinical Cancer Informatics ◽

10.1200/cci.19.00013 ◽

2019 ◽

pp. 1-13 ◽

Cited By ~ 14

Author(s):

Sandra D. Griffith ◽

Rebecca A. Miksad ◽

Geoff Calkins ◽

Paul You ◽

Nicole G. Lipitz ◽

...

Keyword(s):

Lung Cancer ◽

Small Cell Lung Cancer ◽

Cell Lung Cancer ◽

Real World ◽

Large Scale ◽

Progression Free Survival ◽

Small Cell ◽

Small Cell Lung ◽

Data Set ◽

End Points

PURPOSE Large, generalizable real-world data can enhance traditional clinical trial results. The current study evaluates reliability, clinical relevance, and large-scale feasibility for a previously documented method with which to characterize cancer progression outcomes in advanced non–small-cell lung cancer from electronic health record (EHR) data. METHODS Patients who were diagnosed with advanced non–small-cell lung cancer between January 1, 2011, and February 28, 2018, with two or more EHR-documented visits and one or more systemic therapy line initiated were identified in Flatiron Health’s longitudinal EHR-derived database. After institutional review board approval, we retrospectively characterized real-world progression (rwP) dates, with a random duplicate sample to ascertain interabstractor agreement. We calculated real-world progression-free survival, real-world time to progression, real-world time to next treatment, and overall survival (OS) using the Kaplan-Meier method (index date was the date of first-line therapy initiation), and correlations between OS and other end points were assessed at the patient level (Spearman’s ρ). RESULTS Of 30,276 eligible patients,16,606 (55%) had one or more rwP event. Of these patients, 11,366 (68%) had subsequent death, treatment discontinuation, or new treatment initiation. Correlation of real-world progression-free survival with OS was moderate to high (Spearman’s ρ, 0.76; 95% CI, 0.75 to 0.77; evaluable patients, n = 20,020), and for real-world time to progression correlation with OS was lower (Spearman’s ρ, 0.69; 95% CI, 0.68 to 0.70; evaluable patients, n = 11,902). Interabstractor agreement on rwP occurrence was 0.94 (duplicate sample, n = 1,065) and on rwP date 0.85 (95% CI, 0.81 to 0.89; evaluable patients n = 358 [patients with two independent event captures within 30 days]). Median rwP abstraction time from individual EHRs was 18.0 minutes (interquartile range, 9.7 to 34.4 minutes). CONCLUSION We demonstrated that rwP-based end points correlate with OS, and that rwP curation from a large, contemporary EHR data set can be reliable, clinically relevant, and feasible on a large scale.

Download Full-text

A novel out of step relaying algorithm based on wavelet transform and a deep learning machine model

Protection and Control of Modern Power Systems ◽

10.1186/s41601-021-00221-y ◽

2021 ◽

Vol 6 (1) ◽

Author(s):

Jigneshkumar Pramodbhai Desai ◽

Vijay Hiralal Makwana

Keyword(s):

Deep Learning ◽

Wavelet Transform ◽

Power System ◽

Large Scale ◽

Half Cycle ◽

Synchronous Generators ◽

Power Swing ◽

Renewable Power ◽

Three Phase ◽

Feed Forward Network

AbstractOut-of-step protection of one or a group of synchronous generators is unreliable in a power system which has significant renewable power penetration. In this work, an innovative out-of-step protection algorithm using wavelet transform and deep learning is presented to protect synchronous generators and transmission lines. The specific patterns are generated from both stable and unstable power swing, and three-phase fault using the wavelet transform technique. Data containing 27,008 continuous samples of 48 different features is used to train a two-layer feed-forward network. The proposed algorithm gives an automatic, setting free and highly accurate classification for the three-phase fault, stable power swing, and unstable power swing through pattern recognition within a half cycle. The proposed algorithm uses the Kundur 2-area system and a 29-bus electric network for testing under different swing center locations and levels of renewable power penetration. Hardware-in-the-loop (HIL) tests show the hardware compatibility of the developed out-of-step algorithm. The proposed algorithm is also compared with recently reported algorithms. The comparison and test results on different large-scale systems show that the proposed algorithm is simple, fast, accurate, and HIL tested, and not affected by changes in power system parameters.

Download Full-text

Deep Learning-based Image Analysis for High-content Screening

10.26686/wgtn.16892482 ◽

2021 ◽

Author(s):

◽

Dylon Zeng

Keyword(s):

Image Analysis ◽

Deep Learning ◽

Network Architecture ◽

Estimating Equation ◽

Time Frame ◽

High Content Screening ◽

Similarity Analysis ◽

Analysis Pipeline ◽

Data Set ◽

A Cell

High-content screening is an empirical strategy in drug discovery toidentify substances capable of altering cellular phenotype — the set ofobservable characteristics of a cell — in a desired way. Throughout thepast two decades, high-content screening has gathered significant attentionfrom academia and the pharmaceutical industry. However, imageanalysis remains a considerable hindrance to the widespread applicationof high-content screening. Standard image analysis relies on feature engineeringand suffers from inherent drawbacks such as the dependence onannotated inputs. There is an urging need for reliable and more efficientmethods to cope with increasingly large amounts of data produced. This thesis centres around the design and implementation of a deeplearning-based image analysis pipeline for high-content screening. Theend goal is to identify and cluster hit compounds that significantly alterthe phenotype of a cell. The proposed pipeline replaces feature engineeringwith a k-nearest neighbour-based similarity analysis. In addition, featureextraction using convolutional autoencoders is applied to reduce thenegative effects of noise on hit selection. As a result, the feature engineeringprocess is circumvented. A novel similarity measure is developed tofacilitate similarity analysis. Moreover, we combine deep learning withstatistical modelling to achieve optimal results. Preliminary explorationssuggest that the choice of hyperparameters have a direct impact on neuralnetwork performance. Generalised estimating equation models are usedto predict the most suitable neural network architecture for the input data. Using the proposed pipeline, we analyse an extensive set of images acquiredfrom a series of cell-based assays examining the effect of 282 FDAapproved drugs. The analysis of this data set produces a shortlist of drugsthat can significantly alter a cell’s phenotype, then further identifies fiveclusters of the shortlisted drugs. The clustering results present groups ofexisting drugs that have the potential to be repurposed for new therapeuticuses. Furthermore, our findings align with published studies. Comparedwith other neural networks, the image analysis pipeline proposedin this thesis provides reliable and better results in a shorter time frame.

Download Full-text

Scale-Covariant and Scale-Invariant Gaussian Derivative Networks

Journal of Mathematical Imaging and Vision ◽

10.1007/s10851-021-01057-9 ◽

2021 ◽

Author(s):

Tony Lindeberg

Keyword(s):

Deep Learning ◽

Network Architecture ◽

Large Scale ◽

Hybrid Approach ◽

Multiple Scale ◽

Scale Space ◽

Training Data ◽

Scale Invariant ◽

Gaussian Derivative ◽

Space Operations

AbstractThis paper presents a hybrid approach between scale-space theory and deep learning, where a deep learning architecture is constructed by coupling parameterized scale-space operations in cascade. By sharing the learnt parameters between multiple scale channels, and by using the transformation properties of the scale-space primitives under scaling transformations, the resulting network becomes provably scale covariant. By in addition performing max pooling over the multiple scale channels, or other permutation-invariant pooling over scales, a resulting network architecture for image classification also becomes provably scale invariant. We investigate the performance of such networks on the MNIST Large Scale dataset, which contains rescaled images from the original MNIST dataset over a factor of 4 concerning training data and over a factor of 16 concerning testing data. It is demonstrated that the resulting approach allows for scale generalization, enabling good performance for classifying patterns at scales not spanned by the training data.

Download Full-text

Learning effective physical laws for generating cosmological hydrodynamics with Lagrangian deep learning

Proceedings of the National Academy of Sciences ◽

10.1073/pnas.2020324118 ◽

2021 ◽

Vol 118 (16) ◽

pp. e2020324118

Author(s):

Biwei Dai ◽

Uroš Seljak

Keyword(s):

Dark Matter ◽

Deep Learning ◽

Generative Models ◽

Generative Model ◽

Effective Theory ◽

High Dimensions ◽

Wide Range ◽

Hydrodynamical Simulations ◽

Physical Laws ◽

Very High

The goal of generative models is to learn the intricate relations between the data to create new simulated data, but current approaches fail in very high dimensions. When the true data-generating process is based on physical processes, these impose symmetries and constraints, and the generative model can be created by learning an effective description of the underlying physics, which enables scaling of the generative model to very high dimensions. In this work, we propose Lagrangian deep learning (LDL) for this purpose, applying it to learn outputs of cosmological hydrodynamical simulations. The model uses layers of Lagrangian displacements of particles describing the observables to learn the effective physical laws. The displacements are modeled as the gradient of an effective potential, which explicitly satisfies the translational and rotational invariance. The total number of learned parameters is only of order 10, and they can be viewed as effective theory parameters. We combine N-body solver fast particle mesh (FastPM) with LDL and apply it to a wide range of cosmological outputs, from the dark matter to the stellar maps, gas density, and temperature. The computational cost of LDL is nearly four orders of magnitude lower than that of the full hydrodynamical simulations, yet it outperforms them at the same resolution. We achieve this with only of order 10 layers from the initial conditions to the final output, in contrast to typical cosmological simulations with thousands of time steps. This opens up the possibility of analyzing cosmological observations entirely within this framework, without the need for large dark-matter simulations.

Download Full-text

Memristor Based Binary Convolutional Neural Network Architecture With Configurable Neurons

Frontiers in Neuroscience ◽

10.3389/fnins.2021.639526 ◽

2021 ◽

Vol 15 ◽

Author(s):

Lixing Huang ◽

Jietao Diao ◽

Hongshan Nie ◽

Wei Wang ◽

Zhiwei Li ◽

...

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

High Precision ◽

Network Architecture ◽

Large Scale ◽

Network Performance ◽

Recognition Accuracy ◽

Activation Function ◽

Data Set ◽

Neuron Activation

The memristor-based convolutional neural network (CNN) gives full play to the advantages of memristive devices, such as low power consumption, high integration density, and strong network recognition capability. Consequently, it is very suitable for building a wearable embedded application system and has broad application prospects in image classification, speech recognition, and other fields. However, limited by the manufacturing process of memristive devices, high-precision weight devices are currently difficult to be applied in large-scale. In the same time, high-precision neuron activation function also further increases the complexity of network hardware implementation. In response to this, this paper proposes a configurable full-binary convolutional neural network (CFB-CNN) architecture, whose inputs, weights, and neurons are all binary values. The neurons are proportionally configured to two modes for different non-ideal situations. The architecture performance is verified based on the MNIST data set, and the influence of device yield and resistance fluctuations under different neuron configurations on network performance is also analyzed. The results show that the recognition accuracy of the 2-layer network is about 98.2%. When the yield rate is about 64% and the hidden neuron mode is configured as −1 and +1, namely ±1 MD, the CFB-CNN architecture achieves about 91.28% recognition accuracy. Whereas the resistance variation is about 26% and the hidden neuron mode configuration is 0 and 1, namely 01 MD, the CFB-CNN architecture gains about 93.43% recognition accuracy. Furthermore, memristors have been demonstrated as one of the most promising devices in neuromorphic computing for its synaptic plasticity. Therefore, the CFB-CNN architecture based on memristor is SNN-compatible, which is verified using the number of pulses to encode pixel values in this paper.

Download Full-text