scholarly journals Rate-optimal denoising with deep neural networks

2020 ◽  
Author(s):  
Reinhard Heckel ◽  
Wen Huang ◽  
Paul Hand ◽  
Vladislav Voroninski
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Large Data ◽  
Large Data Sets ◽  
Gradient Algorithm ◽  
Data Sets ◽  
Underlying Principle ◽  
Noise Energy ◽  
Noisy Observation ◽  
Low Dimensional

Abstract Deep neural networks provide state-of-the-art performance for image denoising, where the goal is to recover a near noise-free image from a noisy observation. The underlying principle is that neural networks trained on large data sets have empirically been shown to be able to generate natural images well from a low-dimensional latent representation of the image. Given such a generator network, a noisy image can be denoised by (i) finding the closest image in the range of the generator or by (ii) passing it through an encoder-generator architecture (known as an autoencoder). However, there is little theory to justify this success, let alone to predict the denoising performance as a function of the network parameters. In this paper, we consider the problem of denoising an image from additive Gaussian noise using the two generator-based approaches. In both cases, we assume the image is well described by a deep neural network with ReLU activations functions, mapping a $k$-dimensional code to an $n$-dimensional image. In the case of the autoencoder, we show that the feedforward network reduces noise energy by a factor of $O(k/n)$. In the case of optimizing over the range of a generative model, we state and analyze a simple gradient algorithm that minimizes a non-convex loss function and provably reduces noise energy by a factor of $O(k/n)$. We also demonstrate in numerical experiments that this denoising performance is, indeed, achieved by generative priors learned from data.

scholarly journals Artificial Intelligence Explained for Nonexperts

2020 ◽  
Vol 24 (01) ◽  
pp. 003-011 ◽  
Author(s):  
Narges Razavian ◽  
Florian Knoll ◽  
Krzysztof J. Geras
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Clinical Practice ◽  
Medical Imaging ◽  
Deep Neural Networks ◽  
Large Data ◽  
Large Data Sets ◽  
Natural Images ◽  
Data Sets ◽  
Current State

AbstractArtificial intelligence (AI) has made stunning progress in the last decade, made possible largely due to the advances in training deep neural networks with large data sets. Many of these solutions, initially developed for natural images, speech, or text, are now becoming successful in medical imaging. In this article we briefly summarize in an accessible way the current state of the field of AI. Furthermore, we highlight the most promising approaches and describe the current challenges that will need to be solved to enable broad deployment of AI in clinical practice.

scholarly journals Online Deep Learning: Learning Deep Neural Networks on the Fly

2018 ◽  
Author(s):  
Doyen Sahoo ◽  
Quang Pham ◽  
Jing Lu ◽  
Steven C. H. Hoi
Keyword(s):  
Neural Networks ◽  
Online Learning ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Large Data ◽  
Learning Task ◽  
Large Data Sets ◽  
Training Data ◽  
Data Sets ◽  
Online Setting

Deep Neural Networks (DNNs) are typically trained by backpropagation in a batch setting, requiring the entire training data to be made available prior to the learning task. This is not scalable for many real-world scenarios where new data arrives sequentially in a stream. We aim to address an open challenge of ``Online Deep Learning" (ODL) for learning DNNs on the fly in an online setting. Unlike traditional online learning that often optimizes some convex objective function with respect to a shallow model (e.g., a linear/kernel-based hypothesis), ODL is more challenging as the optimization objective is non-convex, and regular DNN with standard backpropagation does not work well in practice for online settings. We present a new ODL framework that attempts to tackle the challenges by learning DNN models which dynamically adapt depth from a sequence of training data in an online learning setting. Specifically, we propose a novel Hedge Backpropagation (HBP) method for online updating the parameters of DNN effectively, and validate the efficacy on large data sets (both stationary and concept drifting scenarios).

scholarly journals Generation of geometric interpolations of building types with deep variational autoencoders

2020 ◽  
Vol 6 ◽  
Author(s):  
Jaime de Miguel Rodríguez ◽  
Maria Eugenia Villafañe ◽  
Luka Piškorec ◽  
Fernando Sancho Caparrini
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Large Data ◽  
Learning Model ◽  
Large Data Sets ◽  
Data Sets ◽  
Connectivity Map ◽  
Data Set ◽  
3D Objects ◽  
Machine Learning Model

Abstract This work presents a methodology for the generation of novel 3D objects resembling wireframes of building types. These result from the reconstruction of interpolated locations within the learnt distribution of variational autoencoders (VAEs), a deep generative machine learning model based on neural networks. The data set used features a scheme for geometry representation based on a ‘connectivity map’ that is especially suited to express the wireframe objects that compose it. Additionally, the input samples are generated through ‘parametric augmentation’, a strategy proposed in this study that creates coherent variations among data by enabling a set of parameters to alter representative features on a given building type. In the experiments that are described in this paper, more than 150 k input samples belonging to two building types have been processed during the training of a VAE model. The main contribution of this paper has been to explore parametric augmentation for the generation of large data sets of 3D geometries, showcasing its problems and limitations in the context of neural networks and VAEs. Results show that the generation of interpolated hybrid geometries is a challenging task. Despite the difficulty of the endeavour, promising advances are presented.

Modular Grammatical Evolution for the Generation of Artificial Neural Networks

2021 ◽  
pp. 1-36
Author(s):  
Khabat Soltanian ◽  
Ali Ebnenasir ◽  
Mohsen Afsharchi
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Solution Space ◽  
Single Layer ◽  
Large Data ◽  
Grammatical Evolution ◽  
Large Data Sets ◽  
Data Sets ◽  
Novel Method ◽  
Weak Locality

Abstract This paper presents a novel method, called Modular Grammatical Evolution (MGE), towards validating the hypothesis that restricting the solution space of NeuroEvolution to modular and simple neural networks enables the efficient generation of smaller and more structured neural networks while providing acceptable (and in some cases superior) accuracy on large data sets. MGE also enhances the state-of-the-art Grammatical Evolution (GE) methods in two directions. First, MGE's representation is modular in that each individual has a set of genes, and each gene is mapped to a neuron by grammatical rules. Second, the proposed representation mitigates two important drawbacks of GE, namely the low scalability and weak locality of representation, towards generating modular and multi-layer networks with a high number of neurons. We define and evaluate five different forms of structures with and without modularity using MGE and find single-layer modules with no coupling more productive. Our experiments demonstrate that modularity helps in finding better neural networks faster. We have validated the proposed method using ten well-known classification benchmarks with different sizes, feature counts, and output class counts. Our experimental results indicate that MGE provides superior accuracy with respect to existing NeuroEvolution methods and returns classifiers that are significantly simpler than other machine learning generated classifiers. Finally, we empirically demonstrate that MGE outperforms other GE methods in terms of locality and scalability properties.

Methodological support of the expert system in the problem of interaction of business ecosystems

2021 ◽  
Vol 4 ◽  
pp. 47-53
Author(s):  
K. V. Simonov ◽  
◽  
V. V. Kuimov ◽  
M. V. Kobalinsky ◽  
S. V. Kirillova ◽  
...  
Keyword(s):  
Neural Networks ◽  
Expert System ◽  
Business Models ◽  
Quantitative Description ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Effective Solution ◽  
Business Ecosystems ◽  
The Government

The paper discusses modern approaches and digital transformations in business models and interactions. In this regard for a quantitative description of interactions in ecosystems a variant of methodological support based on neural networks is proposed for fast nonlinear multiparametric regression of large data sets within the projected expert system. The possibility of effective solution of the problem of filling gaps in the observational data arrays and processing of not precisely specified information is shown. This approach is proposed for solving predictive problems in the problem of interaction of objects of interest in business ecosystems. The article was prepared within the framework of the Grant of the RFBR and the Government of the Krasnoyarsk Territory No. 20-410-242916 / 20 r_mk Krasnoyarsk.

Feed Forward Backpropagation Neural Networks and their Use in Predicting the Acute Toxicity of Chemicals to the Fathead Minnow

1997 ◽  
Vol 32 (3) ◽  
pp. 637-658 ◽  
Author(s):  
Klaus L.E. Kaiser ◽  
Stefan P. Niculescu ◽  
Gerrit Schüürmann
Keyword(s):  
Neural Networks ◽  
Fathead Minnow ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Explicit Equation ◽  
Data Set ◽  
Feed Forward ◽  
Backpropagation Neural Networks ◽  
Water Partition Coefficient

Abstract Various aspects connected to the use of feed forward backpropagation neural networks to build multivariate QSARs based on large data sets containing considerable amounts of important information are investigated. Based on such a model and a 419 compound data set, the explicit equation of one of the resulting multivariate QSARs for the computation of toxicity to the fathead minnow is presented as function of measured Microtox, logarithms of molecular weight and octanol/water partition coefficient, and 48 other functional group and discrete descriptors.

scholarly journals A survey of deep meta-learning

2021 ◽  
Author(s):  
Mike Huisman ◽  
Jan N. van Rijn ◽  
Aske Plaat
Keyword(s):  
Deep Neural Networks ◽  
Theoretical Foundation ◽  
Large Data ◽  
Large Data Sets ◽  
Performance Evaluations ◽  
Data Sets ◽  
Computational Costs ◽  
New Concepts ◽  
Meta Learning ◽  
Computational Resources

AbstractDeep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into (i) metric-, (ii) model-, and (iii) optimization-based techniques. In addition, we identify the main open challenges, such as performance evaluations on heterogeneous benchmarks, and reduction of the computational costs of meta-learning.

Sign in / Sign up

Export Citation Format

Share Document