scholarly journals Mid-lateral cerebellar complex spikes encode multiple independent reward-related signals during reinforcement learning

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Naveen Sendhilnathan ◽  
Anna Ipata ◽  
Michael E. Goldberg
Keyword(s):  
Reinforcement Learning ◽  
Stimulus Onset ◽  
Cell Type ◽  
Independent Manner ◽  
Distinct Cell Type ◽  
Information Encoding ◽  
Cognitive Behaviors ◽  
Wide Range ◽  
Neural Information ◽  
Complex Spikes

AbstractAlthough the cerebellum has been implicated in simple reward-based learning recently, the role of complex spikes (CS) and simple spikes (SS), their interaction and their relationship to complex reinforcement learning and decision making is still unclear. Here we show that in a context where a non-human primate learned to make novel visuomotor associations, classifying CS responses based on their SS properties revealed distinct cell-type specific encoding of the probability of failure after the stimulus onset and the non-human primate’s decision. In a different context, CS from the same cerebellar area also responded in a cell-type and learning independent manner to the stimulus that signaled the beginning of the trial. Both types of CS signals were independent of changes in any motor kinematics and were unlikely to instruct the concurrent SS activity through an error based mechanism, suggesting the presence of context dependent, flexible, multiple independent channels of neural encoding by CS and SS. This diversity in neural information encoding in the mid-lateral cerebellum, depending on the context and learning state, is well suited to promote exploration and acquisition of wide range of cognitive behaviors that entail flexible stimulus-action-reward relationships but not necessarily motor learning.

scholarly journals Complex spikes encode reward expectation signals during visuomotor association learning

2019 ◽  
Author(s):  
Naveen Sendhilnathan ◽  
Anna Ipata ◽  
Michael E. Goldberg
Keyword(s):  
Stimulus Onset ◽  
Higher Order ◽  
Independent Manner ◽  
General Role ◽  
Order Processing ◽  
Association Learning ◽  
Reward Delivery ◽  
Spike Signals ◽  
Complex Spikes

AbstractClimbing fiber input to Purkinje cells has been thought to instruct learning related changes in simple spikes and cause behavioral changes through an error-based learning mechanism. Although, this framework explains simple motor learning, it cannot be extended to learning higher-order skills. Recently the cerebellum has been implicated in a variety of cognitive tasks and reward-based learning. Here we show that when a monkey learns a new visuomotor association, complex spikes predict the time of the beginning of the trial in a learning independent manner as well as encode a learning contingent reward expectation signal after the stimulus onset and reward delivery. These complex spike signals are unrelated to and were unlikely to instruct the reward based signal found in the simple spikes. Our results provide a more general role of complex spikes in learning and higher-order processing while gathering evidence for their participation in reward based learning.

scholarly journals Congenital hypothyroidism impairs spine growth of dentate granule cells by downregulation of CaMKIV

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Qingying Tang ◽  
Shuxia Chen ◽  
Hui Wu ◽  
Honghua Song ◽  
Yongjun Wang ◽  
...  
Keyword(s):  
Congenital Hypothyroidism ◽  
Granule Cells ◽  
Spine Density ◽  
Drug Induced ◽  
Distinct Cell Type ◽  
Cognitive Behaviors ◽  
Dentate Granule Cells ◽  
Rat Pups ◽  
Cognitive Deficiency ◽  
Underlying Mechanisms

AbstractCongenital hypothyroidism (CH), a common neonatal endocrine disorder, can result in cognitive deficits if delay in diagnose and treatment. Dentate gyrus (DG) is the severely affected subregion of the hippocampus by the CH, where the dentate granule cells (DGCs) reside in. However, how CH impairs the cognitive function via affecting DGCs and the underlying mechanisms are not fully elucidated. In the present study, the CH model of rat pups was successfully established, and the aberrant dendrite growth of the DGCs and the impaired cognitive behaviors were observed in the offspring. Transcriptome analysis of hippocampal tissues following rat CH successfully identified that calcium/calmodulin-dependent protein kinase IV (CaMKIV) was the prominent regulator involved in mediating deficient growth of DGC dendrites. CaMKIV was shown to be dynamically regulated in the DG subregion of the rats following drug-induced CH. Interference of CaMKIV expression in the primary DGCs significantly reduced the spine density of dendrites, while addition of T3 to the primary DGCs isolated from CH pups could facilitate the spine growth of dendrites. Insights into relevant mechanisms revealed that CH-mediated CaMKIV deficiency resulted in the significant decrease of phosphorylated CREB in DGCs, in association with the abnormality of dendrites. Our results have provided a distinct cell type in hippocampus that is affected by CH, which would be beneficial for the treatment of CH-induced cognitive deficiency.

scholarly journals A Reinforcement Learning Framework for Spiking Networks with Dynamic Synapses

2011 ◽  
Vol 2011 ◽  
pp. 1-12 ◽  
Author(s):  
Karim El-Laithy ◽  
Martin Bogdan
Keyword(s):  
Reinforcement Learning ◽  
Spike Timing ◽  
Neural Representation ◽  
Model Parameters ◽  
Learning Framework ◽  
Reference Target ◽  
Wide Range ◽  
Spiking Network ◽  
Dynamic Synapses ◽  
Exclusive Or

An integration of both the Hebbian-based and reinforcement learning (RL) rules is presented for dynamic synapses. The proposed framework permits the Hebbian rule to update the hidden synaptic model parameters regulating the synaptic response rather than the synaptic weights. This is performed using both the value and the sign of the temporal difference in the reward signal after each trial. Applying this framework, a spiking network with spike-timing-dependent synapses is tested to learn the exclusive-OR computation on a temporally coded basis. Reward values are calculated with the distance between the output spike train of the network and a reference target one. Results show that the network is able to capture the required dynamics and that the proposed framework can reveal indeed an integrated version of Hebbian and RL. The proposed framework is tractable and less computationally expensive. The framework is applicable to a wide class of synaptic models and is not restricted to the used neural representation. This generality, along with the reported results, supports adopting the introduced approach to benefit from the biologically plausible synaptic models in a wide range of intuitive signal processing.

scholarly journals Goal-driven active learning

2021 ◽  
Vol 35 (2) ◽  
Author(s):  
Nicolas Bougie ◽  
Ryutaro Ichise
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Learning Process ◽  
Real World ◽  
Imitation Learning ◽  
Learning Approaches ◽  
Wide Range ◽  
Fixed Set ◽  
Complex Decision Making ◽  
Complex Decision

AbstractDeep reinforcement learning methods have achieved significant successes in complex decision-making problems. In fact, they traditionally rely on well-designed extrinsic rewards, which limits their applicability to many real-world tasks where rewards are naturally sparse. While cloning behaviors provided by an expert is a promising approach to the exploration problem, learning from a fixed set of demonstrations may be impracticable due to lack of state coverage or distribution mismatch—when the learner’s goal deviates from the demonstrated behaviors. Besides, we are interested in learning how to reach a wide range of goals from the same set of demonstrations. In this work we propose a novel goal-conditioned method that leverages very small sets of goal-driven demonstrations to massively accelerate the learning process. Crucially, we introduce the concept of active goal-driven demonstrations to query the demonstrator only in hard-to-learn and uncertain regions of the state space. We further present a strategy for prioritizing sampling of goals where the disagreement between the expert and the policy is maximized. We evaluate our method on a variety of benchmark environments from the Mujoco domain. Experimental results show that our method outperforms prior imitation learning approaches in most of the tasks in terms of exploration efficiency and average scores.

Predicting Human Mobility with Reinforcement-Learning-Based Long-Term Periodicity Modeling

2021 ◽  
Vol 12 (6) ◽  
pp. 1-23
Author(s):  
Shuo Tao ◽  
Jingang Jiang ◽  
Defu Lian ◽  
Kai Zheng ◽  
Enhong Chen
Keyword(s):  
Reinforcement Learning ◽  
Human Mobility ◽  
Recurrent Network ◽  
Mobility Prediction ◽  
Learning Framework ◽  
Temporal Features ◽  
Wide Range ◽  
Spatio Temporal ◽  
Historical Trajectory

Mobility prediction plays an important role in a wide range of location-based applications and services. However, there are three problems in the existing literature: (1) explicit high-order interactions of spatio-temporal features are not systemically modeled; (2) most existing algorithms place attention mechanisms on top of recurrent network, so they can not allow for full parallelism and are inferior to self-attention for capturing long-range dependence; (3) most literature does not make good use of long-term historical information and do not effectively model the long-term periodicity of users. To this end, we propose MoveNet and RLMoveNet. MoveNet is a self-attention-based sequential model, predicting each user’s next destination based on her most recent visits and historical trajectory. MoveNet first introduces a cross-based learning framework for modeling feature interactions. With self-attention on both the most recent visits and historical trajectory, MoveNet can use an attention mechanism to capture the user’s long-term regularity in a more efficient way. Based on MoveNet, to model long-term periodicity more effectively, we add the reinforcement learning layer and named RLMoveNet. RLMoveNet regards the human mobility prediction as a reinforcement learning problem, using the reinforcement learning layer as the regularization part to drive the model to pay attention to the behavior with periodic actions, which can help us make the algorithm more effective. We evaluate both of them with three real-world mobility datasets. MoveNet outperforms the state-of-the-art mobility predictor by around 10% in terms of accuracy, and simultaneously achieves faster convergence and over 4x training speedup. Moreover, RLMoveNet achieves higher prediction accuracy than MoveNet, which proves that modeling periodicity explicitly from the perspective of reinforcement learning is more effective.

scholarly journals Non-negative Independent Factor Analysis for single cell RNA-seq

2020 ◽  
Author(s):  
Weiguang Mao ◽  
Maziyar Baran Pouyan ◽  
Dennis Kostka ◽  
Maria Chikina
Keyword(s):  
Factor Analysis ◽  
Single Cell ◽  
Matrix Factorization ◽  
Component Analysis ◽  
Computational Techniques ◽  
Cell Type ◽  
Analysis Model ◽  
Factor Analysis Model ◽  
Type Identity ◽  
Wide Range

AbstractMotivationSingle cell RNA sequencing (scRNA-seq) enables transcriptional profiling at the level of individual cells. With the emergence of high-throughput platforms datasets comprising tens of thousands or more cells have become routine, and the technology is having an impact across a wide range of biomedical subject areas. However, scRNA-seq data are high-dimensional and affected by noise, so that scalable and robust computational techniques are needed for meaningful analysis, visualization and interpretation. Specifically, a range of matrix factorization techniques have been employed to aid scRNA-seq data analysis. In this context we note that sources contributing to biological variability between cells can be discrete (or multi-modal, for instance cell-types), or continuous (e.g. pathway activity). However, no current matrix factorization approach is set up to jointly infer such mixed sources of variability.ResultsTo address this shortcoming, we present a new probabilistic single-cell factor analysis model, Non-negative Independent Factor Analysis (NIFA), that combines features of complementary approaches like Independent Component Analysis (ICA), Principal Component Analysis (PCA), and Non-negative Matrix Factorization (NMF). NIFA simultaneously models uni- and multi-modal latent factors and can so isolate discrete cell-type identity and continuous pathway-level variations into separate components. Similar to NMF, NIFA constrains factor loadings to be non-negative in order to increase biological interpretability. We apply our approach to a range of data sets where cell-type identity is known, and we show that NIFA-derived factors outperform results from ICA, PCA and NMF in terms of cell-type identification and biological interpretability. Studying an immunotherapy dataset in detail, we show that NIFA identifies biomedically meaningful sources of variation, derive an improved expression signature for regulatory T-cells, and identify a novel myeloid cell subtype associated with treatment response. Overall, NIFA is a general approach advancing scRNA-seq analysis capabilities and it allows researchers to better take advantage of their data. NIFA is available at https://github.com/wgmao/[email protected]

scholarly journals A Possible Role of Allatostatin C in Inhibiting Ecdysone Biosynthesis Revealed in the Mud Crab Scylla paramamosain

2021 ◽  
Vol 8 ◽  
Author(s):  
An Liu ◽  
Wenyuan Shi ◽  
Dongdong Lin ◽  
Haihui Ye
Keyword(s):  
Scylla Paramamosain ◽  
Dependent Manner ◽  
Mud Crab ◽  
Methyl Farnesoate ◽  
Independent Manner ◽  
Wide Range ◽  
Negative Effect

C-type allatostatins (C-type ASTs) are a family of structurally related neuropeptides found in a wide range of insects and crustaceans. To date, the C-type allatostatin receptor in crustaceans has not been deorphaned, and little is known about its physiological functions. In this study, we aimed to functionally define a C-type ASTs receptor in the mud crab, Scylla paramamosian. We showed that C-type ASTs receptor can be activated by ScypaAST-C peptide in a dose-independent manner and by ScypaAST-CCC peptide in a dose-dependent manner with an IC50 value of 6.683 nM. Subsequently, in vivo and in vitro experiments were performed to investigate the potential roles of ScypaAST-C and ScypaAST-CCC peptides in the regulation of ecdysone (20E) and methyl farnesoate (MF) biosynthesis. The results indicated that ScypaAST-C inhibited biosynthesis of 20E in the Y-organ, whereas ScypaAST-CCC had no effect on the production of 20E. In addition, qRT-PCR showed that both ScypaAST-C and ScypaAST-CCC significantly decreased the level of expression of the MF biosynthetic enzyme gene in the mandibular organ, suggesting that the two neuropeptides have a negative effect on the MF biosynthesis in mandibular organs. In conclusion, this study provided new insight into the physiological roles of AST-C in inhibiting ecdysone biosynthesis. Furthermore, it was revealed that AST-C family peptides might inhibit MF biosynthesis in crustaceans.

scholarly journals Hydroxy Pentacyclic Triterpene Acid, Kaempferol, Inhibits the Human 5-Hydroxytryptamine Type 3A Receptor Activity

2022 ◽  
Vol 23 (1) ◽  
pp. 544
Author(s):  
Shinhui Lee ◽  
Hee-Soo Seol ◽  
Sanung Eom ◽  
Jaeeun Lee ◽  
Chaelin Kim ◽  
...  
Keyword(s):  
Nervous System ◽  
Binding Sites ◽  
Molecular Level ◽  
Neuroprotective Effect ◽  
Site Directed Mutagenesis ◽  
Pentacyclic Triterpene ◽  
Independent Manner ◽  
Triterpene Acid ◽  
Cudrania Tricuspidata ◽  
Wide Range

Monoamine serotonin is a major neurotransmitter that acts on a wide range of central nervous system and peripheral nervous system functions and is known to have a role in various processes. Recently, it has been found that 5-HT is involved in cognitive and memory functions through interaction with cholinergic pathways. The natural flavonoid kaempferol (KAE) extracted from Cudrania tricuspidata is a secondary metabolite of the plant. Recently studies have confirmed that KAE possesses a neuroprotective effect because of its strong antioxidant activity. It has been confirmed that KAE is involved in the serotonergic pathway through an in vivo test. However, these results need to be confirmed at the molecular level, because the exact mechanism that is involved in such effects of KAE has not yet been elucidated. Therefore, the objective of this study is to confirm the interaction of KAE with 5-HT3A through electrophysiological studies at the molecular level using KAE extracted from Cudrania tricuspidata. This study confirmed the interaction between 5-HT3A and KAE at the molecular level. KAE inhibited 5-HT3A receptors in a concentration-dependent and voltage-independent manner. Site-directed mutagenesis and molecular-docking studies confirmed that the binding sites D177 and F199 are the major binding sites of human 5-HT3A receptors of KAE.

scholarly journals Panoramic stitching of heterogeneous single-cell transcriptomic data

2018 ◽  
Author(s):  
Brian Hie ◽  
Bryan Bryson ◽  
Bonnie Berger
Keyword(s):  
Single Cell ◽  
Cell Types ◽  
Data Sets ◽  
Cell Type ◽  
Data Set ◽  
Wide Range ◽  
Data Set Integration ◽  
Biological Patterns ◽  
Insight Into ◽  
Comprehensive Reference

AbstractResearchers are generating single-cell RNA sequencing (scRNA-seq) profiles of diverse biological systems1–4 and every cell type in the human body.5 Leveraging this data to gain unprecedented insight into biology and disease will require assembling heterogeneous cell populations across multiple experiments, laboratories, and technologies. Although methods for scRNA-seq data integration exist6,7, they often naively merge data sets together even when the data sets have no cell types in common, leading to results that do not correspond to real biological patterns. Here we present Scanorama, inspired by algorithms for panorama stitching, that overcomes the limitations of existing methods to enable accurate, heterogeneous scRNA-seq data set integration. Our strategy identifies and merges the shared cell types among all pairs of data sets and is orders of magnitude faster than existing techniques. We use Scanorama to combine 105,476 cells from 26 diverse scRNA-seq experiments across 9 different technologies into a single comprehensive reference, demonstrating how Scanorama can be used to obtain a more complete picture of cellular function across a wide range of scRNA-seq experiments.

scholarly journals Optogenetic delivery of trophic signals in a genetic model of Parkinson’s disease

2020 ◽  
Author(s):  
Álvaro Inglés-Prieto ◽  
Nikolas Furthmann ◽  
Samuel Crossman ◽  
Nina Hoyer ◽  
Meike Petersen ◽  
...  
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Growth Factor ◽  
Tissue Repair ◽  
Genetic Model ◽  
Human Cells ◽  
Cell Type ◽  
Degenerative Disorders ◽  
Wide Range ◽  
Cell Type Specific

AbstractOptogenetics has been harnessed to shed new mechanistic light on current therapies and to develop future treatment strategies. This has been to date achieved by the correction of electrical signals in neuronal cells and neural circuits that are affected by disease. In contrast, the optogenetic delivery of trophic biochemical signals, which support cell survival and thereby may modify progression of degenerative disorders, has never been demonstrated in an animal disease model. Here, we reengineered the human and Drosophila melanogaster REarranged during Transfection (hRET and dRET) receptors to be activated by light, creating one-component optogenetic tools termed Opto-hRET and Opto-dRET. Upon blue light stimulation, these receptors robustly induced the MAPK/ERK proliferative signaling pathway in cultured cells. In PINK1B9 flies that exhibit loss of PTEN-induced putative kinase 1 (PINK1), a kinase associated with familial Parkinson’s disease (PD), light activation of Opto-dRET suppressed mitochondrial defects, tissue degeneration and behavioral deficits. In human cells with PINK1 loss-of-function, mitochondrial fragmentation was rescued using Opto-dRET via the PI3K/NF-кB pathway. Our results demonstrate that a light-activated receptor can ameliorate disease hallmarks in a genetic model of PD. The optogenetic delivery of trophic signals is cell type-specific and reversible and thus has the potential to overcome limitations of current strategies towards a spatio-temporal regulation of tissue repair.Significance StatementThe death of physiologically important cell populations underlies of a wide range of degenerative disorders, including Parkinson’s disease (PD). Two major strategies to counter cell degeneration, soluble growth factor injection and growth factor gene therapy, can lead to the undesired activation of bystander cells and non-natural permanent signaling responses. Here, we employed optogenetics to deliver cell type-specific pro-survival signals in a genetic model of PD. In Drosophila and human cells exhibiting loss of the PINK1 kinase, akin to autosomal recessive PD, we efficiently suppressed disease phenotypes using a light-activated tyrosine kinase receptor. This work demonstrates a spatio-temporally precise strategy to interfere with degeneration and may open new avenues towards tissue repair in disease models.

Sign in / Sign up

Export Citation Format

Share Document