scholarly journals High-dimensional distribution generation through deep neural networks

2021 ◽  
Vol 2 (5) ◽  
Author(s):  
Dmytro Perekrestenko ◽  
Léandre Eberhard ◽  
Helmut Bölcskei
Keyword(s):  
Probability Distributions ◽  
Approximation Error ◽  
Wasserstein Distance ◽  
Independent Random Variables ◽  
Target Distribution ◽  
Neural Information ◽  
Neural Information Processing ◽  
Dimensional Distribution ◽  
Red Hook ◽  
Distribution Generation

AbstractWe show that every d-dimensional probability distribution of bounded support can be generated through deep ReLU networks out of a 1-dimensional uniform input distribution. What is more, this is possible without incurring a cost—in terms of approximation error measured in Wasserstein-distance—relative to generating the d-dimensional target distribution from d independent random variables. This is enabled by a vast generalization of the space-filling approach discovered in Bailey and Telgarsky (in: Bengio (eds) Advances in neural information processing systems vol 31, pp 6489–6499. Curran Associates, Inc., Red Hook, 2018). The construction we propose elicits the importance of network depth in driving the Wasserstein distance between the target distribution and its neural network approximation to zero. Finally, we find that, for histogram target distributions, the number of bits needed to encode the corresponding generative network equals the fundamental limit for encoding probability distributions as dictated by quantization theory.

Neural Information Processing

1967 ◽  
Vol 12 (11) ◽  
pp. 558-559
Author(s):  
STEPHAN L. CHOROVER
Keyword(s):  
Information Processing ◽  
Neural Information ◽  
Neural Information Processing

scholarly journals Cortical response to naturalistic stimuli is largely predictable with deep neural networks

2021 ◽  
Vol 7 (22) ◽  
pp. eabe7547
Author(s):  
Meenakshi Khosla ◽  
Gia H. Ngo ◽  
Keith Jamison ◽  
Amy Kuceyeski ◽  
Mert R. Sabuncu
Keyword(s):  
Brain Function ◽  
Deep Neural Networks ◽  
Neural Responses ◽  
Dynamic Interactions ◽  
Hierarchical Processing ◽  
Human Connectome Project ◽  
Neural Information ◽  
Neural Information Processing ◽  
Visual Interactions ◽  
High Level

Naturalistic stimuli, such as movies, activate a substantial portion of the human brain, invoking a response shared across individuals. Encoding models that predict neural responses to arbitrary stimuli can be very useful for studying brain function. However, existing models focus on limited aspects of naturalistic stimuli, ignoring the dynamic interactions of modalities in this inherently context-rich paradigm. Using movie-watching data from the Human Connectome Project, we build group-level models of neural activity that incorporate several inductive biases about neural information processing, including hierarchical processing, temporal assimilation, and auditory-visual interactions. We demonstrate how incorporating these biases leads to remarkable prediction performance across large areas of the cortex, beyond the sensory-specific cortices into multisensory sites and frontal cortex. Furthermore, we illustrate that encoding models learn high-level concepts that generalize to task-bound paradigms. Together, our findings underscore the potential of encoding models as powerful tools for studying brain function in ecologically valid conditions.

scholarly journals Consciousness, decision making, and volition: freedom beyond chance and necessity

2021 ◽  
Author(s):  
Hans Liljenström
Keyword(s):  
Decision Making ◽  
Free Will ◽  
Brain Activity ◽  
Brain Structures ◽  
Complex Process ◽  
Neural Information ◽  
Neural Information Processing ◽  
Stochastic Population Model ◽  
The Brain

AbstractWhat is the role of consciousness in volition and decision-making? Are our actions fully determined by brain activity preceding our decisions to act, or can consciousness instead affect the brain activity leading to action? This has been much debated in philosophy, but also in science since the famous experiments by Libet in the 1980s, where the current most common interpretation is that conscious free will is an illusion. It seems that the brain knows, up to several seconds in advance what “you” decide to do. These studies have, however, been criticized, and alternative interpretations of the experiments can be given, some of which are discussed in this paper. In an attempt to elucidate the processes involved in decision-making (DM), as an essential part of volition, we have developed a computational model of relevant brain structures and their neurodynamics. While DM is a complex process, we have particularly focused on the amygdala and orbitofrontal cortex (OFC) for its emotional, and the lateral prefrontal cortex (LPFC) for its cognitive aspects. In this paper, we present a stochastic population model representing the neural information processing of DM. Simulation results seem to confirm the notion that if decisions have to be made fast, emotional processes and aspects dominate, while rational processes are more time consuming and may result in a delayed decision. Finally, some limitations of current science and computational modeling will be discussed, hinting at a future development of science, where consciousness and free will may add to chance and necessity as explanation for what happens in the world.

scholarly journals Autism as a disorder of neural information processing: Directions for research and targets for therapy

2019 ◽  
Author(s):  
Matthew Belmonte ◽  
John Rubenstein
Keyword(s):  
Brain Development ◽  
Autism Spectrum ◽  
Behavioural Syndrome ◽  
Neural Systems ◽  
Developmental Courses ◽  
Genetic Syndromes ◽  
Causal Factors ◽  
Neural Information ◽  
Neural Information Processing ◽  
Broad Variation

The broad variation in phenotypes and severities within autism spectrum disorders suggests the involvement of multiple predisposing factors, interacting in complex ways with normal developmental courses and gradients. Identification of these factors, and the common developmental path into which they feed, is hampered by the large degrees of convergence from causal factors to altered brain development, and divergence from abnormal brain development into altered cognition and behaviour. Genetic, neurochemical, neuroimaging and behavioural findings on autism, as well as studies of normal development and of genetic syndromes that share symptoms with autism, offer hypotheses as to the nature of causal factors and their possible effects on the structure and dynamics of neural systems. Such alterations in neural properties may in turn perturb activity-dependent development, giving rise to a complex behavioural syndrome many steps removed from the root causes. Animal models based on genetic, neurochemical, neurophysiological, and behavioural manipulations offer the possibility of exploring these developmental processes in detail, as do human studies addressing endophenotypes beyond the diagnosis itself.

Sign in / Sign up

Export Citation Format

Share Document