Advances in Neural Computation, Machine Learning, and Cognitive Research III

2020 ◽  
Keyword(s):  
Machine Learning ◽  
Neural Computation ◽  
Cognitive Research

scholarly journals Memristors for the Curious Outsiders

Technologies ◽  
2018 ◽  
Vol 6 (4) ◽  
pp. 118 ◽  
Author(s):  
Francesco Caravelli ◽  
Juan Carbajal
Keyword(s):  
Machine Learning ◽  
Physical Mechanism ◽  
Neural Computation ◽  
Dynamic Resistance ◽  
Passive Component ◽  
Internal Parameter ◽  
New Approaches ◽  
Memristive Behavior ◽  
Memristive Circuits

We present both an overview and a perspective of recent experimental advances and proposed new approaches to performing computation using memristors. A memristor is a 2-terminal passive component with a dynamic resistance depending on an internal parameter. We provide an brief historical introduction, as well as an overview over the physical mechanism that lead to memristive behavior. This review is meant to guide nonpractitioners in the field of memristive circuits and their connection to machine learning and neural computation.

Might a Single Neuron Solve Interesting Machine Learning Problems through Successive Computations on Its Dendritic Tree?

2021 ◽  
pp. 1-18
Author(s):  
Ilenna Simone Jones ◽  
Konrad Paul Kording
Keyword(s):  
Machine Learning ◽  
Binary Tree ◽  
Single Neuron ◽  
Model Performance ◽  
Dendritic Tree ◽  
Neural Computation ◽  
Learning Problems ◽  
Synaptic Inputs ◽  
Learning Tasks ◽  
Dendritic Branches

Abstract Physiological experiments have highlighted how the dendrites of biological neurons can nonlinearly process distributed synaptic inputs. However, it is unclear how aspects of a dendritic tree, such as its branched morphology or its repetition of presynaptic inputs, determine neural computation beyond this apparent nonlinearity. Here we use a simple model where the dendrite is implemented as a sequence of thresholded linear units. We manipulate the architecture of this model to investigate the impacts of binary branching constraints and repetition of synaptic inputs on neural computation. We find that models with such manipulations can perform well on machine learning tasks, such as Fashion MNIST or Extended MNIST. We find that model performance on these tasks is limited by binary tree branching and dendritic asymmetry and is improved by the repetition of synaptic inputs to different dendritic branches. These computational experiments further neuroscience theory on how different dendritic properties might determine neural computation of clearly defined tasks.

scholarly journals Memristors for the curious outsiders

2018 ◽  
Author(s):  
Juan Pablo Carbajal ◽  
Francesco Caravelli
Keyword(s):  
Machine Learning ◽  
Physical Mechanism ◽  
Neural Computation ◽  
Dynamic Resistance ◽  
Passive Component ◽  
Internal Parameter ◽  
New Approaches ◽  
Memristive Behavior ◽  
Memristive Circuits

We present both an overview and a perspective of recent experimental advances and proposed new approaches to performing computation using memristors. A memristor is a 2-terminal passive component with a dynamic resistance depending on an internal parameter. We provide an brief historical introduction, as well as an overview over the physical mechanism that lead to memristive behavior. This review is meant to guide nonpractitioners in the field of memristive circuits and their connection to machine learning and neural computation.

scholarly journals Multiplicative Updates for Nonnegative Quadratic Programming

2007 ◽  
Vol 19 (8) ◽  
pp. 2004-2031 ◽  
Author(s):  
Fei Sha ◽  
Yuanqing Lin ◽  
Lawrence K. Saul ◽  
Daniel D. Lee
Keyword(s):  
Machine Learning ◽  
Quadratic Programming ◽  
Statistical Learning ◽  
Global Minimum ◽  
Neural Computation ◽  
Multiplicative Updates ◽  
Convex Problems ◽  
Nonnegative Orthant ◽  
Nonnegativity Constraints ◽  
Free Parameters

Many problems in neural computation and statistical learning involve optimizations with nonnegativity constraints. In this article, we study convex problems in quadratic programming where the optimization is confined to an axis-aligned region in the nonnegative orthant. For these problems, we derive multiplicative updates that improve the value of the objective function at each iteration and converge monotonically to the global minimum. The updates have a simple closed form and do not involve any heuristics or free parameters that must be tuned to ensure convergence. Despite their simplicity, they differ strikingly in form from other multiplicative updates used in machine learning. We provide complete proofs of convergence for these updates and describe their application to problems in signal processing and pattern recognition.

scholarly journals Interrogating theoretical models of neural computation with deep inference

2019 ◽  
Author(s):  
Sean R. Bittner ◽  
Agostina Palmigiano ◽  
Alex T. Piet ◽  
Chunyu A. Duan ◽  
Carlos D. Brody ◽  
...  
Keyword(s):  
Machine Learning ◽  
Ad Hoc ◽  
Neural Mechanism ◽  
Theoretical Models ◽  
Generative Models ◽  
Neural Computation ◽  
Machine Learning Techniques ◽  
Model Parameters ◽  
The Relationship ◽  
Computational Properties

1AbstractA cornerstone of theoretical neuroscience is the circuit model: a system of equations that captures a hypothesized neural mechanism. Such models are valuable when they give rise to an experimentally observed phenomenon – whether behavioral or in terms of neural activity – and thus can offer insights into neural computation. The operation of these circuits, like all models, critically depends on the choices of model parameters. Historically, the gold standard has been to analytically derive the relationship between model parameters and computational properties. However, this enterprise quickly becomes infeasible as biologically realistic constraints are included into the model increasing its complexity, often resulting in ad hoc approaches to understanding the relationship between model and computation. We bring recent machine learning techniques – the use of deep generative models for probabilistic inference – to bear on this problem, learning distributions of parameters that produce the specified properties of computation. Importantly, the techniques we introduce offer a principled means to understand the implications of model parameter choices on computational properties of interest. We motivate this methodology with a worked example analyzing sensitivity in the stomatogastric ganglion. We then use it to go beyond linear theory of neuron-type input-responsivity in a model of primary visual cortex, gain a mechanistic understanding of rapid task switching in superior colliculus models, and attribute error to connectivity properties in recurrent neural networks solving a simple mathematical task. More generally, this work suggests a departure from realism vs tractability considerations, towards the use of modern machine learning for sophisticated interrogation of biologically relevant models.

Sign in / Sign up

Export Citation Format

Share Document