scholarly journals Graph-Driven Generative Models for Heterogeneous Multi-Task Learning

2020 ◽  
Vol 34 (01) ◽  
pp. 979-988
Author(s):  
Wenlin Wang ◽  
Hongteng Xu ◽  
Zhe Gan ◽  
Bai Li ◽  
Guoyin Wang ◽  
...  
Keyword(s):  
Generative Models ◽  
Healthcare Applications ◽  
Convolutional Network ◽  
Learning Tasks ◽  
Proposed Model ◽  
Heterogeneous Learning ◽  
Clinical Topic ◽  
Generative Processes ◽  
Admission Type ◽  
Uniform Manner

We propose a novel graph-driven generative model, that unifies multiple heterogeneous learning tasks into the same framework. The proposed model is based on the fact that heterogeneous learning tasks, which correspond to different generative processes, often rely on data with a shared graph structure. Accordingly, our model combines a graph convolutional network (GCN) with multiple variational autoencoders, thus embedding the nodes of the graph (i.e., samples for the tasks) in a uniform manner, while specializing their organization and usage to different tasks. With a focus on healthcare applications (tasks), including clinical topic modeling, procedure recommendation and admission-type prediction, we demonstrate that our method successfully leverages information across different tasks, boosting performance in all tasks and outperforming existing state-of-the-art approaches.

Equivariant Adversarial Network for Image-to-image Translation

2021 ◽  
Vol 17 (2s) ◽  
pp. 1-14
Author(s):  
Masoumeh Zareapoor ◽  
Jie Yang
Keyword(s):  
State Of The Art ◽  
Generative Models ◽  
Generative Model ◽  
Target Domain ◽  
Adversarial Network ◽  
Proposed Model ◽  
Image Translation ◽  
Great Performance ◽  
Representative Model ◽  
The Ideal

Image-to-Image translation aims to learn an image from a source domain to a target domain. However, there are three main challenges, such as lack of paired datasets, multimodality, and diversity, that are associated with these problems and need to be dealt with. Convolutional neural networks (CNNs), despite of having great performance in many computer vision tasks, they fail to detect the hierarchy of spatial relationships between different parts of an object and thus do not form the ideal representative model we look for. This article presents a new variation of generative models that aims to remedy this problem. We use a trainable transformer, which explicitly allows the spatial manipulation of data within training. This differentiable module can be augmented into the convolutional layers in the generative model, and it allows to freely alter the generated distributions for image-to-image translation. To reap the benefits of proposed module into generative model, our architecture incorporates a new loss function to facilitate an effective end-to-end generative learning for image-to-image translation. The proposed model is evaluated through comprehensive experiments on image synthesizing and image-to-image translation, along with comparisons with several state-of-the-art algorithms.

scholarly journals Predictive Uncertainty Estimation for Tractable Deep Probabilistic Models

2020 ◽  
Author(s):  
Julissa Villanueva Llerena
Keyword(s):  
Probabilistic Models ◽  
Linear Time ◽  
Generative Models ◽  
Uncertainty Estimation ◽  
Image Completion ◽  
Predictive Uncertainty ◽  
Learning Tasks ◽  
Challenging Tasks ◽  
Statistical Support ◽  
Marginal Inference

Tractable Deep Probabilistic Models (TPMs) are generative models based on arithmetic circuits that allow for exact marginal inference in linear time. These models have obtained promising results in several machine learning tasks. Like many other models, TPMs can produce over-confident incorrect inferences, especially on regions with small statistical support. In this work, we will develop efficient estimators of the predictive uncertainty that are robust to data scarcity and outliers. We investigate two approaches. The first approach measures the variability of the output to perturbations of the model weights. The second approach captures the variability of the prediction to changes in the model architecture. We will evaluate the approaches on challenging tasks such as image completion and multilabel classification.

A Novel Neural Model with Lateral Interaction for Learning Tasks

2020 ◽  
pp. 1-24
Author(s):  
Dequan Jin ◽  
Ziyan Qin ◽  
Murong Yang ◽  
Penghe Chen
Keyword(s):  
Numerical Experiments ◽  
State Of The Art ◽  
Neural Model ◽  
Parameter Tuning ◽  
Small Sample ◽  
Lateral Interaction ◽  
Learning Tasks ◽  
Proposed Model ◽  
High Level ◽  
Elementary Field

We propose a novel neural model with lateral interaction for learning tasks. The model consists of two functional fields: an elementary field to extract features and a high-level field to store and recognize patterns. Each field is composed of some neurons with lateral interaction, and the neurons in different fields are connected by the rules of synaptic plasticity. The model is established on the current research of cognition and neuroscience, making it more transparent and biologically explainable. Our proposed model is applied to data classification and clustering. The corresponding algorithms share similar processes without requiring any parameter tuning and optimization processes. Numerical experiments validate that the proposed model is feasible in different learning tasks and superior to some state-of-the-art methods, especially in small sample learning, one-shot learning, and clustering.

Artificial Bee Colony-Based Approach for Privacy Preservation of Medical Data

2021 ◽  
pp. 1203-1221
Author(s):  
Shivlal Mewada ◽  
Sita Sharan Gautam ◽  
Pradeep Sharma
Keyword(s):  
Privacy Preservation ◽  
Artificial Bee Colony ◽  
Model Simulation ◽  
Privacy Preserving ◽  
Global Network ◽  
Sensitive Information ◽  
Healthcare Applications ◽  
Efficient Manner ◽  
Bee Colony ◽  
Proposed Model

A large amount of data is generated through healthcare applications and medical equipment. This data is transferred from one piece of equipment to another and sometimes also communicated over a global network. Hence, security and privacy preserving are major concerns in the healthcare sector. It is seen that traditional anonymization algorithms are viable for sanitization process, but not for restoration task. In this work, artificial bee colony-based privacy preserving model is developed to address the aforementioned issues. In the proposed model, ABC-based algorithm is adopted to generate the optimal key for sanitization of sensitive information. The effectiveness of the proposed model is tested through restoration analysis. Furthermore, several popular attacks are also considered for evaluating the performance of the proposed privacy preserving model. Simulation results of the proposed model are compared with some popular existing privacy preserving models. It is observed that the proposed model is capable of preserving the sensitive information in an efficient manner.

scholarly journals Variational Autoencoder-Based Multiple Image Captioning Using a Caption Attention Map

2019 ◽  
Vol 9 (13) ◽  
pp. 2699 ◽  
Author(s):  
Boeun Kim ◽  
Saim Shin ◽  
Hyedong Jung
Keyword(s):  
Generative Models ◽  
Research Topic ◽  
Video Data ◽  
Image Feature ◽  
Image Captioning ◽  
Multiple Image ◽  
Visually Impaired People ◽  
Proposed Model ◽  
Variational Autoencoder ◽  
Latent Distribution

Image captioning is a promising research topic that is applicable to services that search for desired content in a large amount of video data and a situation explanation service for visually impaired people. Previous research on image captioning has been focused on generating one caption per image. However, to increase usability in applications, it is necessary to generate several different captions that contain various representations for an image. We propose a method to generate multiple captions using a variational autoencoder, which is one of the generative models. Because an image feature plays an important role when generating captions, a method to extract a Caption Attention Map (CAM) of the image is proposed, and CAMs are projected to a latent distribution. In addition, methods for the evaluation of multiple image captioning tasks are proposed that have not yet been actively researched. The proposed model outperforms in the aspect of diversity compared with the base model when the accuracy is comparable. Moreover, it is verified that the model using CAM generates detailed captions describing various content in the image.

scholarly journals Revisiting Graph Based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach

2020 ◽  
Vol 34 (01) ◽  
pp. 27-34 ◽  
Author(s):  
Lei Chen ◽  
Le Wu ◽  
Richang Hong ◽  
Kun Zhang ◽  
Meng Wang
Keyword(s):  
Collaborative Filtering ◽  
Representation Learning ◽  
Superior Performance ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Proposed Model ◽  
Non Linear ◽  
Efficiency And Effectiveness ◽  
Residual Graph ◽  
Interaction Modeling

Graph Convolutional Networks~(GCNs) are state-of-the-art graph based representation learning models by iteratively stacking multiple layers of convolution aggregation operations and non-linear activation operations. Recently, in Collaborative Filtering~(CF) based Recommender Systems~(RS), by treating the user-item interaction behavior as a bipartite graph, some researchers model higher-layer collaborative signals with GCNs. These GCN based recommender models show superior performance compared to traditional works. However, these models suffer from training difficulty with non-linear activations for large user-item graphs. Besides, most GCN based models could not model deeper layers due to the over smoothing effect with the graph convolution operation. In this paper, we revisit GCN based CF models from two aspects. First, we empirically show that removing non-linearities would enhance recommendation performance, which is consistent with the theories in simple graph convolutional networks. Second, we propose a residual network structure that is specifically designed for CF with user-item interaction modeling, which alleviates the over smoothing problem in graph convolution aggregation operation with sparse user-item interaction data. The proposed model is a linear model and it is easy to train, scale to large datasets, and yield better efficiency and effectiveness on two real datasets. We publish the source code at https://github.com/newlei/LR-GCCF.

scholarly journals Variational Approach for Learning Community Structures

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-13 ◽  
Author(s):  
Jun Jin Choong ◽  
Xin Liu ◽  
Tsuyoshi Murata
Keyword(s):  
Deep Learning ◽  
Community Structure ◽  
Learning Community ◽  
Community Detection ◽  
Representation Learning ◽  
Semisupervised Learning ◽  
Detection Methods ◽  
Community Structures ◽  
Learning Tasks ◽  
Proposed Model

Discovering and modeling community structure exist to be a fundamentally challenging task. In domains such as biology, chemistry, and physics, researchers often rely on community detection algorithms to uncover community structures from complex systems yet no unified definition of community structure exists. Furthermore, existing models tend to be oversimplified leading to a neglect of richer information such as nodal features. Coupled with the surge of user generated information on social networks, a demand for newer techniques beyond traditional approaches is inevitable. Deep learning techniques such as network representation learning have shown tremendous promise. More specifically, supervised and semisupervised learning tasks such as link prediction and node classification have achieved remarkable results. However, unsupervised learning tasks such as community detection remain widely unexplored. In this paper, a novel deep generative model for community detection is proposed. Extensive experiments show that the proposed model, empowered with Bayesian deep learning, can provide insights in terms of uncertainty and exploit nonlinearities which result in better performance in comparison to state-of-the-art community detection methods. Additionally, unlike traditional methods, the proposed model is community structure definition agnostic. Leveraging on low-dimensional embeddings of both network topology and feature similarity, it automatically learns the best model configuration for describing similarities in a community.

scholarly journals A simple model for learning in volatile environments

2019 ◽  
Author(s):  
Payam Piray ◽  
Nathaniel D. Daw
Keyword(s):  
Kalman Filter ◽  
State Space Model ◽  
Choice Data ◽  
Exact Inference ◽  
Second Order Statistics ◽  
Decision Neuroscience ◽  
Learning Tasks ◽  
Neuroscience Research ◽  
Proposed Model ◽  
Psychological Models

AbstractSound principles of statistical inference dictate that uncertainty shapes learning. In this work, we revisit the question of learning in volatile environments, in which both the first and second-order statistics of observations dynamically evolve over time. We propose a new model, the volatile Kalman filter (VKF), which is based on a tractable state-space model of uncertainty and extends the Kalman filter algorithm to volatile environments. The proposed model is algorithmically simple and encompasses the Kalman filter as a special case. Specifically, in addition to the error-correcting rule of Kalman filter for learning observations, the VKF learns volatility according to a second error-correcting rule. These dual updates echo and contextualize classical psychological models of learning, in particular hybrid accounts of Pearce-Hall and Rescorla-Wagner. At the computational level, compared with existing models, the VKF gives up some flexibility in the generative model to enable a more faithful approximation to exact inference. When fit to empirical data, the VKF is better behaved than alternatives and better captures human choice data in two independent datasets of probabilistic learning tasks. The proposed model provides a coherent account of learning in stable or volatile environments and has implications for decision neuroscience research.

scholarly journals Visual Attention Through Uncertainty Minimization in Recurrent Generative Models

2020 ◽  
Author(s):  
Kai Standvoss ◽  
Silvan C. Quax ◽  
Marcel A.J. van Gerven
Keyword(s):  
Eye Movements ◽  
Visual Attention ◽  
Intelligent Agents ◽  
Saccadic Eye Movements ◽  
Generative Models ◽  
Task Demands ◽  
Relevant Stimulus ◽  
Predictive Uncertainty ◽  
Proposed Model ◽  
Uncertainty Minimization

AbstractAllocating visual attention through saccadic eye movements is a key ability of intelligent agents. Attention is both influenced through bottom-up stimulus properties as well as top-down task demands. The interaction of these two attention mechanisms is not yet fully understood. A parsimonious reconciliation posits that both processes serve the minimization of predictive uncertainty. We propose a recurrent generative neural network model that predicts a visual scene based on foveated glimpses. The model shifts its attention in order to minimize the uncertainty in its predictions. We show that the proposed model produces naturalistic eye movements focusing on informative stimulus regions. Introducing additional tasks modulates the saccade patterns towards task-relevant stimulus regions. The model’s saccade characteristics correspond well with previous experimental data in humans, providing evidence that uncertainty minimization could be a fundamental mechanisms for the allocation of visual attention.

scholarly journals Analyzing the instructions vulnerability of dense convolutional network on GPUS

2021 ◽  
Vol 11 (5) ◽  
pp. 4481
Author(s):  
Khalid Adam ◽  
Izzeldin I. Mohd ◽  
Younis Ibrahim
Keyword(s):  
Graphics Processing Units ◽  
Deep Neural Networks ◽  
Fault Injection ◽  
Soft Errors ◽  
Small Error ◽  
Mitigation Strategy ◽  
Healthcare Applications ◽  
Convolutional Network ◽  
Safety Critical ◽  
Graphics Processing

Recently, deep neural networks (DNNs) have been increasingly deployed in various healthcare applications, which are considered safety-critical applications. Thus, the reliability of these DNN models should be remarkably high, because even a small error in healthcare applications can lead to injury or death. Due to the high computations of the DNN models, DNNs are often executed on the graphics processing units (GPUs). However, the GPUs have been reportedly impacted by soft errors, which are extremely serious issues in the healthcare applications. In this paper, we show how the fault injection can provide a deeper understanding of DenseNet201 model instructions vulnerability on the GPU. Then, we analyze vulnerable instructions of the DenseNet201 on the GPU. Our results show that the most significant vulnerable instructions against soft errors PR, STORE, FADD, FFMA, SETP and LD can be reduced from 4.42% to 0.14% of injected faults, after we applied our mitigation strategy.

Sign in / Sign up

Export Citation Format

Share Document