scholarly journals Contrastive Learning and Neural Oscillations

1991 ◽  
Vol 3 (4) ◽  
pp. 526-545 ◽  
Author(s):  
Pierre Baldi ◽  
Fernando Pineda
Keyword(s):  
Gradient Descent ◽  
Learning Algorithms ◽  
Learning Rule ◽  
Unified Framework ◽  
Contrast Function ◽  
New Learning ◽  
Different Types ◽  
Free Network ◽  
Two Phases ◽  
The Brain

The concept of Contrastive Learning (CL) is developed as a family of possible learning algorithms for neural networks. CL is an extension of Deterministic Boltzmann Machines to more general dynamical systems. During learning, the network oscillates between two phases. One phase has a teacher signal and one phase has no teacher signal. The weights are updated using a learning rule that corresponds to gradient descent on a contrast function that measures the discrepancy between the free network and the network with a teacher signal. The CL approach provides a general unified framework for developing new learning algorithms. It also shows that many different types of clamping and teacher signals are possible. Several examples are given and an analysis of the landscape of the contrast function is proposed with some relevant predictions for the CL curves. An approach that may be suitable for collective analog implementations is described. Simulation results and possible extensions are briefly discussed together with a new conjecture regarding the function of certain oscillations in the brain. In the appendix, we also examine two extensions of contrastive learning to time-dependent trajectories.

scholarly journals A solution to the learning dilemma for recurrent networks of spiking neurons

2019 ◽  
Author(s):  
Guillaume Bellec ◽  
Franz Scherr ◽  
Anand Subramoney ◽  
Elias Hajek ◽  
Darjan Salaj ◽  
...  
Keyword(s):  
Recurrent Neural Networks ◽  
Energy Efficient ◽  
Gradient Descent ◽  
Spiking Neurons ◽  
Recurrent Networks ◽  
Network Learning ◽  
Backpropagation Through Time ◽  
New Learning ◽  
On Chip ◽  
The Brain

AbstractRecurrently connected networks of spiking neurons underlie the astounding information processing capabilities of the brain. But in spite of extensive research, it has remained open how they can learn through synaptic plasticity to carry out complex network computations. We argue that two pieces of this puzzle were provided by experimental data from neuroscience. A new mathematical insight tells us how these pieces need to be combined to enable biologically plausible online network learning through gradient descent, in particular deep reinforcement learning. This new learning method – called e-prop – approaches the performance of BPTT (backpropagation through time), the best known method for training recurrent neural networks in machine learning. In addition, it suggests a method for powerful on-chip learning in novel energy-efficient spike-based hardware for AI.

scholarly journals Local online learning in recurrent networks with random feedback

2018 ◽  
Author(s):  
James M. Murray
Keyword(s):  
Online Learning ◽  
Computational Neuroscience ◽  
Learning Rule ◽  
Recurrent Networks ◽  
Biological Features ◽  
Mathematical Arguments ◽  
Learning Rules ◽  
New Learning ◽  
Gradient Based ◽  
The Brain

AbstractA longstanding challenge for computational neuroscience has been the development of biologically plausible learning rules for recurrent neural networks (RNNs) enabling the production and processing of time-dependent signals such as those that might drive movement or facilitate working memory. Classic gradient-based algorithms for training RNNs have been available for decades, but they are inconsistent with known biological features of the brain, such as causality and locality. In this work we derive an approximation to gradient-based learning that comports with these biologically motivated constraints. Specifically, the online learning rule for the synaptic weights involves only local information about the pre- and postsynaptic activities, in addition to a random feedback projection of the RNN output error. In addition to providing mathematical arguments for the effectiveness of the new learning rule, we show through simulations that it can be used to train an RNN to successfully perform a variety of tasks. Finally, to overcome the difficulty of training an RNN over a very large number of timesteps, we propose an augmented circuit architecture that allows the RNN to concatenate short-duration patterns into sequences of longer duration.

scholarly journals Local online learning in recurrent networks with random feedback

eLife ◽  
2019 ◽  
Vol 8 ◽  
Author(s):  
James M Murray
Keyword(s):  
Synaptic Weight ◽  
Learning Rule ◽  
Local Information ◽  
Recurrent Networks ◽  
Biological Features ◽  
Mathematical Arguments ◽  
Large Numbers ◽  
New Learning ◽  
Gradient Based ◽  
The Brain

Recurrent neural networks (RNNs) enable the production and processing of time-dependent signals such as those involved in movement or working memory. Classic gradient-based algorithms for training RNNs have been available for decades, but are inconsistent with biological features of the brain, such as causality and locality. We derive an approximation to gradient-based learning that comports with these constraints by requiring synaptic weight updates to depend only on local information about pre- and postsynaptic activities, in addition to a random feedback projection of the RNN output error. In addition to providing mathematical arguments for the effectiveness of the new learning rule, we show through simulations that it can be used to train an RNN to perform a variety of tasks. Finally, to overcome the difficulty of training over very large numbers of timesteps, we propose an augmented circuit architecture that allows the RNN to concatenate short-duration patterns into longer sequences.

“Where is Dementia?” A Systematic Literature Review Exploring Neuroanatomical Aspects of Dementia

2017 ◽  
Vol 2 (15) ◽  
pp. 9-23 ◽  
Author(s):  
Chorong Oh ◽  
Leonard LaPointe
Keyword(s):  
English Language ◽  
Signs And Symptoms ◽  
Behavioral Symptoms ◽  
Journal Articles ◽  
People With Dementia ◽  
Different Types ◽  
Precise Diagnosis ◽  
Physical Changes ◽  
The Brain ◽  
Language Literature

Dementia is a condition caused by and associated with separate physical changes in the brain. The signs and symptoms of dementia are very similar across the diverse types, and it is difficult to diagnose the category by behavioral symptoms alone. Diagnostic criteria have relied on a constellation of signs and symptoms, but it is critical to understand the neuroanatomical differences among the dementias for a more precise diagnosis and subsequent management. With this regard, this review aims to explore the neuroanatomical aspects of dementia to better understand the nature of distinctive subtypes, signs, and symptoms. This is a review of English language literature published from 1996 to the present day of peer-reviewed academic and medical journal articles that report on older people with dementia. This review examines typical neuroanatomical aspects of dementia and reinforces the importance of a thorough understanding of the neuroanatomical characteristics of the different types of dementia and the differential diagnosis of them.

The Acute Effect of Alcohol on Various Memory Processes

2010 ◽  
Vol 24 (4) ◽  
pp. 249-252 ◽  
Author(s):  
Márk Molnár ◽  
Roland Boha ◽  
Balázs Czigler ◽  
Zsófia Anna Gaál
Keyword(s):  
Low Dose ◽  
Short Term Memory ◽  
Acute Effect ◽  
Long Term Memory ◽  
Term Memory ◽  
Memory Processes ◽  
Different Types ◽  
The Brain ◽  
Legal Consequences

This review surveys relevant and recent data of the pertinent literature regarding the acute effect of alcohol on various kinds of memory processes with special emphasis on working memory. The characteristics of different types of long-term memory (LTM) and short-term memory (STM) processes are summarized with an attempt to relate these to various structures in the brain. LTM is typically impaired by chronic alcohol intake but according to some data a single dose of ethanol may have long lasting effects if administered at a critically important age. The most commonly seen deleterious acute effect of alcohol to STM appears following large doses of ethanol in conditions of “binge drinking” causing the “blackout” phenomenon. However, with the application of various techniques and well-structured behavioral paradigms it is possible to detect, albeit occasionally, subtle changes of cognitive processes even as a result of a low dose of alcohol. These data may be important for the consideration of legal consequences of low-dose ethanol intake in conditions such as driving, etc.

Types of refractive errors and methods for their diagnosis

2020 ◽  
pp. 30-36
Author(s):  
Olga Lemzyakova
Keyword(s):  
Optical System ◽  
Contact Lenses ◽  
Vision Impairment ◽  
Refractive Errors ◽  
Vision Correction ◽  
Light Rays ◽  
Different Types ◽  
Almost All ◽  
Degrees Of Severity ◽  
The Brain

Refraction of the eye means its ability to bend (refract) light in its own optical system. In a normal state, which is called emmetropia, light rays passing through the optical system of the eye focus on the retina, from where the impulse is transmitted to the visual cortex of the brain and is analyzed there. A person sees equally well both in the distance and near in this situation. However, very often, refractive errors develop as a result of various types of influences. Myopia, or short-sightedness, occurs when the light rays are focused in front of the retina as a result of passing through the optical system of the eye. In this case, a person will clearly distinguish close objects and have difficulties in seeing distant objects. On the opposite side is development of farsightedness (hypermetropia), in which the focusing of light rays occurs behind the retina — such a person sees distant objects clearly, but outlines of closer objects are out of focus. Near vision impairment in old age is a natural process called presbyopia, it develops due to the lens thickening. Both myopia and hypermetropia can have different degrees of severity. The variant, when different refractive errors are observed in different eyes, is called anisometropia. In the same case, if different types of refraction are observed in the same eye, it is astigmatism, and most often it is a congenital pathology. Almost all of the above mentioned refractive errors require correction with spectacles or use of contact lenses. Recently, people are increasingly resorting to the methods of surgical vision correction.

Ligand and Structure-based Modeling of Passive Diffusion through the Blood-Brain Barrier

2018 ◽  
Vol 25 (9) ◽  
pp. 1073-1089 ◽  
Author(s):  
Santiago Vilar ◽  
Eduardo Sobarzo-Sanchez ◽  
Lourdes Santana ◽  
Eugenio Uriarte
Keyword(s):  
Blood Brain Barrier ◽  
In Silico ◽  
Passive Diffusion ◽  
Brain Barrier ◽  
Barrier Permeability ◽  
Blood Brain Barrier Permeability ◽  
Blood Brain ◽  
Brain Barrier Permeability ◽  
Different Types ◽  
The Brain

Background: Blood-brain barrier transport is an important process to be considered in drug candidates. The blood-brain barrier protects the brain from toxicological agents and, therefore, also establishes a restrictive mechanism for the delivery of drugs into the brain. Although there are different and complex mechanisms implicated in drug transport, in this review we focused on the prediction of passive diffusion through the blood-brain barrier. Methods: We elaborated on ligand-based and structure-based models that have been described to predict the blood-brain barrier permeability. Results: Multiple 2D and 3D QSPR/QSAR models and integrative approaches have been published to establish quantitative and qualitative relationships with the blood-brain barrier permeability. We explained different types of descriptors that correlate with passive diffusion along with data analysis methods. Moreover, we discussed the applicability of other types of molecular structure-based simulations, such as molecular dynamics, and their implications in the prediction of passive diffusion. Challenges and limitations of experimental measurements of permeability and in silico predictive methods were also described. Conclusion: Improvements in the prediction of blood-brain barrier permeability from different types of in silico models are crucial to optimize the process of Central Nervous System drug discovery and development.

scholarly journals Multiple objects tracking in the UAV system based on hierarchical deep high-resolution network

2021 ◽  
Author(s):  
Wei Huang ◽  
Xiaoshu Zhou ◽  
Mingchao Dong ◽  
Huaiyu Xu
Keyword(s):  
High Resolution ◽  
Object Tracking ◽  
High Performance ◽  
State Of The Art ◽  
Class Imbalance ◽  
Unified Framework ◽  
Multiple Objects ◽  
Tracking Process ◽  
Objects Tracking ◽  
Different Types

AbstractRobust and high-performance visual multi-object tracking is a big challenge in computer vision, especially in a drone scenario. In this paper, an online Multi-Object Tracking (MOT) approach in the UAV system is proposed to handle small target detections and class imbalance challenges, which integrates the merits of deep high-resolution representation network and data association method in a unified framework. Specifically, while applying tracking-by-detection architecture to our tracking framework, a Hierarchical Deep High-resolution network (HDHNet) is proposed, which encourages the model to handle different types and scales of targets, and extract more effective and comprehensive features during online learning. After that, the extracted features are fed into different prediction networks for interesting targets recognition. Besides, an adjustable fusion loss function is proposed by combining focal loss and GIoU loss to solve the problems of class imbalance and hard samples. During the tracking process, these detection results are applied to an improved DeepSORT MOT algorithm in each frame, which is available to make full use of the target appearance features to match one by one on a practical basis. The experimental results on the VisDrone2019 MOT benchmark show that the proposed UAV MOT system achieves the highest accuracy and the best robustness compared with state-of-the-art methods.

Characterization of hybrid fillers based on carbon black of different types obtained by impregnation

2015 ◽  
Vol 231 (7) ◽  
pp. 584-599 ◽  
Author(s):  
Ahmed A Al-Ghamdi ◽  
Omar A Al-Hartomy ◽  
Falleh R Al-Solamy ◽  
Nikolay Dishovsky ◽  
Petrunka Malinova ◽  
...  
Keyword(s):  
Carbon Black ◽  
Inductively Coupled Plasma ◽  
Total Pore Volume ◽  
Silica Content ◽  
Dual Phase ◽  
Hybrid Filler ◽  
Average Pore ◽  
Hybrid Fillers ◽  
Different Types ◽  
Two Phases

The paper presents the investigations on obtaining dual phase fillers with preset silica content running a successful impregnation of two completely different types of conventional carbon black with silicasol. The hybrid fillers studied were characterized by atomic absorption spectroscopy and inductively coupled plasma–optical emission spectroscopy. The total pore volume, the average pore diameter, the specific surface area, the oil absorption number, and iodine adsorption of the fillers were also investigated. The distribution of both phases within the hybrid filler obtained and their interpenetration were investigated with scanning transmission electron microscopy-energy dispersive X-ray spectroscopy. The hybrid products obtained were investigated as reinforcing fillers of natural rubber-based composites. The results obtained show that the suggested impregnation with silicasol of conventional carbon black is a perspective method for preparation of carbon-silica dual phase fillers. The method provides an easy control over the quantitative ratio between the two phases. The fillers thus prepared do not change significantly the curing and mechanical characteristics of the vulcanizates, but improve their thermal aging resistance. The isolation of the carbon black aggregates by the silica phase, and the interpenetration of the two phases is a prerequisite to obtain elastomer composites of good mechanical and microwave properties suitable for producing of microwave shielding devices.

Sign in / Sign up

Export Citation Format

Share Document