INSTRUCTION TOOLS FOR SIGNAL PROCESSING AND MACHINE LEARNING FOR ION-CHANNEL SENSORS

Ion Channel sensors have several applications including DNA sequencing, biothreat detection, and medical applications. Ion-channel sensors mimic the selective transport mechanism of cell membranes and can detect a wide range of analytes at the molecule level. Analytes are sensed through changes in signal patterns. Papers in the literature have described different methods for ion channel signal analysis. In this paper, we describe a series of new graphical tools for ion channel signal analysis which can be used for research and education. The paper focuses on the utility of this tools in biosensor classes. Teaching signal processing and machine learning for ion channel sensors is challenging because of the multidisciplinary content and student backgrounds which include physics, chemistry, biology and engineering. The paper describes graphical ion channel analysis tools developed for an on-line simulation environment called J-DSP. The tools are integrated and assessed in a graduate bio-sensor course through computer laboratory exercises.

Ethanol-guided behavior in Drosophila larvae

Chemosensory signals allow vertebrates and invertebrates not only to orient in its environment toward energy-rich food sources to maintain nutrition but also to avoid unpleasant or even poisonous substrates. Ethanol is a substance found in the natural environment of Drosophila melanogaster. Accordingly, D. melanogaster has evolved specific sensory systems, physiological adaptations, and associated behaviors at its larval and adult stage to perceive and process ethanol. To systematically analyze how D. melanogaster larvae respond to naturally occurring ethanol, we examined ethanol-induced behavior in great detail by reevaluating existing approaches and comparing them with new experiments. Using behavioral assays, we confirm that larvae are attracted to different concentrations of ethanol in their environment. This behavior is controlled by olfactory and other environmental cues. It is independent of previous exposure to ethanol in their food. Moreover, moderate, naturally occurring ethanol concentration of 4% results in increased larval fitness. On the contrary, higher concentrations of 10% and 20% ethanol, which rarely or never appear in nature, increase larval mortality. Finally, ethanol also serves as a positive teaching signal in learning and memory and updates valence associated with simultaneously processed odor information. Since information on how larvae perceive and process ethanol at the genetic and neuronal level is limited, the establishment of standardized assays described here is an important step towards their discovery.

A Neurorobotic Embodiment for Exploring the Dynamical Interactions of a Spiking Cerebellar Model and a Robot Arm During Vision-Based Manipulation Tasks

While the original goal for developing robots is replacing humans in dangerous and tedious tasks, the final target shall be completely mimicking the human cognitive and motor behavior. Hence, building detailed computational models for the human brain is one of the reasonable ways to attain this. The cerebellum is one of the key players in our neural system to guarantee dexterous manipulation and coordinated movements as concluded from lesions in that region. Studies suggest that it acts as a forward model providing anticipatory corrections for the sensory signals based on observed discrepancies from the reference values. While most studies consider providing the teaching signal as error in joint-space, few studies consider the error in task-space and even fewer consider the spiking nature of the cerebellum on the cellular-level. In this study, a detailed cellular-level forward cerebellar model is developed, including modeling of Golgi and Basket cells which are usually neglected in previous studies. To preserve the biological features of the cerebellum in the developed model, a hyperparameter optimization method tunes the network accordingly. The efficiency and biological plausibility of the proposed cerebellar-based controller is then demonstrated under different robotic manipulation tasks reproducing motor behavior observed in human reaching experiments.

Ethanol-guided behavior in Drosophila larvae

Chemosensory signals allow vertebrates and invertebrates not only to orient in its environment toward energy-rich food sources to maintain nutrition but also to avoid unpleasant or even poisonous substrates. Ethanol is a substance found in the natural environment of Drosophila melanogaster. Accordingly, D. melanogaster has evolved specific sensory systems, physiological adaptations, and associated behaviors at its larval and adult stage to perceive and process ethanol.To systematically analyze how D. melanogaster larvae respond to naturally occurring ethanol, we examined ethanol-induced behavior in great detail by parametrically reevaluating existing approaches and comparing them with new experiments. Using behavioral assays, we confirm that larvae are attracted to different concentrations of ethanol in their environment. This behavior is controlled both by olfactory and contact cues. It is independent of previous exposure to ethanol in their food. Moreover, moderate, naturally occurring ethanol concentration of 4% results in increased larval fitness. On the contrary, higher concentrations of 10% and 20% ethanol, which rarely or never appear in nature, increase larval mortality. Finally, ethanol also serves as a positive teaching signal in learning and memory and updates valence associated with simultaneously processed odor information.Since information on how larvae perceive and process ethanol at the genetic and neuronal level is limited, the establishment of standardized assays described here is an important step towards their discovery.

What is dopamine doing in model-based reinforcement learning?

Experiments have implicated dopamine in model-based reinforcement learning (RL). These findings are unexpected as dopamine is thought to encode a reward prediction error (RPE), which is the key teaching signal in model-free RL. Here we examine two possible accounts for dopamine’s involvement in model-based RL: the first that dopamine neurons carry a prediction error used to update a type of predictive state representation called a successor representation, the second that two well established aspects of dopaminergic activity, RPEs and surprise signals, can together explain dopamine’s involvement in model-based RL.

A gradual backward shift of dopamine responses during associative learning

It has been proposed that the activity of dopamine neurons approximates temporal difference (TD) prediction error, a teaching signal developed in reinforcement learning, a field of machine learning. However, whether this similarity holds true during learning remains elusive. In particular, some TD learning models predict that the error signal gradually shifts backward in time from reward delivery to a reward-predictive cue, but previous experiments failed to observe such a gradual shift in dopamine activity. Here we demonstrate conditions in which such a shift can be detected experimentally. These shared dynamics of TD error and dopamine activity narrow the gap between machine learning theory and biological brains, tightening a long-sought link.

Learning temporal integration from internal feedback

There has been much focus on the mechanisms of temporal integration, but little on how circuits learn to integrate. In the adult oculomotor system, where a neural integrator maintains fixations, changes in integration dynamics can be driven by visual error signals. However, we show through dark-rearing experiments that visual inputs are not necessary for initial integrator development. We therefore propose a vision-independent learning mechanism whereby a recurrent network learns to integrate via a ‘teaching’ signal formed by low-pass filtered feedback of its population activity. The key is the segregation of local recurrent inputs onto a dendritic compartment and teaching inputs onto a somatic compartment of an integrator neuron. Model instantiation for oculomotor control shows how a self-corrective teaching signal through the cerebellum can generate an integrator with both the dynamical and tuning properties necessary to drive eye muscles and maintain gaze angle. This bootstrap learning paradigm may be relevant for development and plasticity of temporal integration more generally.Highlights- A neuronal architecture that learns to integrate saccadic commands for eye position.- Learning is based on the recurrent dendritic prediction of somatic teaching signals.- Experiment and model show that no visual feedback is required for initial integrator learning.- Cerebellum is an internal teacher for motor nuclei and integrator population.

Dopamine in the basal amygdala signals salient somatosensory events during fear learning

SummaryThe amygdala is a brain area critical for the formation of threat memories. However, the nature of the teaching signal(s) that drive plasticity in the amygdala are still under debate. Here, we use optogenetic methods to investigate whether dopamine release in the amygdala contributes to fear learning. Antero- and retrograde labeling showed that a sparse, and relatively evenly distributed population of ventral tegmental area (VTA) neurons projects to the basal amygdala (BA). In-vivo optrode recordings in behaving mice showed that many VTA neurons, amongst them putative dopamine neurons, are excited by footshocks. Correspondingly, in-vivo fiber photometry of dopamine in the BA revealed robust dopamine concentration transients upon footshock presentation. Finally, silencing VTA dopamine neurons, or their axon terminals in the BA during the footshock, reduced the extent of threat memory retrieval one day later. Thus, VTA dopamine neurons projecting to the BA code for the saliency of the footshock event, and the resulting dopamine release in the BA facilitates threat memory formation.