Optimizing One-Shot Learning with Binary Synapses

2008 ◽  
Vol 20 (8) ◽  
pp. 1928-1950 ◽  
Author(s):  
Sandro Romani ◽  
Daniel J. Amit ◽  
Yali Amit
Keyword(s):  
Memory Retrieval ◽  
Hebbian Learning ◽  
Signal To Noise Ratio ◽  
Excitatory Synapses ◽  
Steady State Distribution ◽  
Capacity Limits ◽  
Fast Learning ◽  
Excess Number ◽  
High Level ◽  
Slow Learning

A network of excitatory synapses trained with a conservative version of Hebbian learning is used as a model for recognizing the familiarity of thousands of once-seen stimuli from those never seen before. Such networks were initially proposed for modeling memory retrieval (selective delay activity). We show that the same framework allows the incorporation of both familiarity recognition and memory retrieval, and estimate the network's capacity. In the case of binary neurons, we extend the analysis of Amit and Fusi (1994) to obtain capacity limits based on computations of signal-to-noise ratio of the field difference between selective and non-selective neurons of learned signals. We show that with fast learning (potentiation probability approximately 1), the most recently learned patterns can be retrieved in working memory (selective delay activity). A much higher number of once-seen learned patterns elicit a realistic familiarity signal in the presence of an external field. With potentiation probability much less than 1 (slow learning), memory retrieval disappears, whereas familiarity recognition capacity is maintained at a similarly high level. This analysis is corroborated in simulations. For analog neurons, where such analysis is more difficult, we simplify the capacity analysis by studying the excess number of potentiated synapses above the steady-state distribution. In this framework, we derive the optimal constraint between potentiation and depression probabilities that maximizes the capacity.

scholarly journals Fabric Vest Socket with Embroidered Electrodes for Control of Myoelectric Prosthesis

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1196 ◽  
Author(s):  
Seulah Lee ◽  
Babar Jamil ◽  
Sunhong Kim ◽  
Youngjin Choi
Keyword(s):  
Signal To Noise Ratio ◽  
Cost Effective ◽  
Electrode Impedance ◽  
Rigid Support ◽  
Daily Lives ◽  
Myoelectric Prosthesis ◽  
Myoelectric Prostheses ◽  
Average Classification Accuracy ◽  
High Level ◽  
Semg Signals

Myoelectric prostheses assist users to live their daily lives. However, the majority of users are primarily confined to forearm amputees because the surface electromyography (sEMG) that understands the motion intents should be acquired from a residual limb for control of the myoelectric prosthesis. This study proposes a novel fabric vest socket that includes embroidered electrodes suitable for a high-level upper amputee, especially for shoulder disarticulation. The fabric vest socket consists of rigid support and a fabric vest with embroidered electrodes. Several experiments were conducted to verify the practicality of the developed vest socket with embroidered electrodes. The sEMG signals were measured using commercial Ag/AgCl electrodes for a comparison to verify the performance of the embroidered electrodes in terms of signal amplitudes, the skin-electrode impedance, and signal-to-noise ratio (SNR). These results showed that the embroidered electrodes were as effective as the commercial electrodes. Then, posture classification was carried out by able-bodied subjects for the usability of the developed vest socket. The average classification accuracy for each subject reached 97.92%, and for all the subjects it was 93.2%. In other words, the fabric vest socket with the embroidered electrodes could measure sEMG signals with high accuracy. Therefore, it is expected that it can be readily worn by high-level amputees to control their myoelectric prostheses, as well as it is cost effective for fabrication as compared with the traditional socket.

scholarly journals Motor Sequences - Separating The Sequence From The Motor. A longitudinal rsfMRI Study

2021 ◽  
Author(s):  
ATP Jäger ◽  
JM Huntenburg ◽  
SA Tremblay ◽  
U Schneider ◽  
S Grahl ◽  
...  
Keyword(s):  
Functional Connectivity ◽  
Motor Learning ◽  
Resting State ◽  
Primary Motor Cortex ◽  
Control Group ◽  
Motor Execution ◽  
Motor Sequence ◽  
Resting State Functional Connectivity ◽  
Fast Learning ◽  
Slow Learning

AbstractIn motor learning, sequence-specificity, i.e. the learning of specific sequential associations, has predominantly been studied using task-based fMRI paradigms. However, offline changes in resting state functional connectivity after sequence-specific motor learning are less well understood. Previous research has established that plastic changes following motor learning can be divided into stages including fast learning, slow learning and retention. A description of how resting state functional connectivity after sequence-specific motor sequence learning (MSL) develops across these stages is missing. This study aimed to identify plastic alterations in whole-brain functional connectivity after learning a complex motor sequence by contrasting an active group who learned a complex sequence with a control group who performed a control task matched for motor execution. Resting state fMRI and behavioural performance were collected in both groups over the course of 5 consecutive training days and at follow-up after 12 days to encompass fast learning, slow learning, overall learning and retention. Between-group interaction analyses showed sequence-specific increases in functional connectivity during fast learning in the sensorimotor territory of the internal segment of right globus pallidus (GPi), and sequence-specific decreases in right supplementary motor area (SMA) in overall learning. We found that connectivity changes in key regions of the motor network including the superior parietal cortex (SPC) and primary motor cortex (M1) were not a result of sequence-specific learning but were instead linked to motor execution. Our study confirms the sequence-specific role of SMA and GPi that has previously been identified in online task-based learning studies in humans and primates, and extends it to resting state network changes after sequence-specific MSL. Finally, our results shed light on a timing-specific plasticity mechanism between GPi and SMA following MSL.

scholarly journals On cross-correlations, averages and noise in electron microscopy

2019 ◽  
Vol 75 (1) ◽  
pp. 12-18 ◽  
Author(s):  
Michael Radermacher ◽  
Teresa Ruiz
Keyword(s):  
High Resolution ◽  
Cross Correlation ◽  
Signal To Noise Ratio ◽  
3D Structure ◽  
Signal To Noise ◽  
High Resolution Structure ◽  
Multiple Copies ◽  
High Level ◽  
2D Images ◽  
Resolution Structure

Biological samples are radiation-sensitive and require imaging under low-dose conditions to minimize damage. As a result, images contain a high level of noise and exhibit signal-to-noise ratios that are typically significantly smaller than 1. Averaging techniques, either implicit or explicit, are used to overcome the limitations imposed by the high level of noise. Averaging of 2D images showing the same molecule in the same orientation results in highly significant projections. A high-resolution structure can be obtained by combining the information from many single-particle images to determine a 3D structure. Similarly, averaging of multiple copies of macromolecular assembly subvolumes extracted from tomographic reconstructions can lead to a virtually noise-free high-resolution structure. Cross-correlation methods are often used in the alignment and classification steps of averaging processes for both 2D images and 3D volumes. However, the high noise level can bias alignment and certain classification results. While other approaches may be implicitly affected, sensitivity to noise is most apparent in multireference alignments, 3D reference-based projection alignments and projection-based volume alignments. Here, the influence of the image signal-to-noise ratio on the value of the cross-correlation coefficient is analyzed and a method for compensating for this effect is provided.

Unconstrained global simulations of ocean tides up to degree 3 for satellite gravimetry

2021 ◽  
Author(s):  
Roman Sulzbach ◽  
Henryk Dobslaw ◽  
Maik Thomas
Keyword(s):  
Signal To Noise Ratio ◽  
Relative Accuracy ◽  
Ocean Tide ◽  
Groundwater Depletion ◽  
Altimetry Data ◽  
Satellite Gravity ◽  
Tidal Model ◽  
Rotation Scheme ◽  
Gravimetric Data ◽  
High Level

<p>Tidal de-aliasing of satellite gravimetric data is a critical task in order to correctly extract gravimetric signatures of climate signals like glacier melting or groundwater depletion and poses a high demand on the accuracy of the employed tidal solutions (Flechtner et al., 2016). Modern tidal atlases that are constrained by altimetry data possess a high level of accuracy, especially for partial tides exhibiting large open ocean signals (e.g. M2, K1). Since the achievable precision directly depends on the available density and quality of altimetry data, the accuracy relative to the tidal amplitude drops for minor tidal excitations (worse signal-to-noise ratio) as well as in polar latitudes (sparse satellite-data). In contrast, this drop in relative accuracy can be reduced by employing an unconstrained tidal model acting independently of altimetric data.<br>We will present recent results from the purely-hydrodynamic, barotropic tidal model TiME (Weis et al., 2008) that benefit from a set of recently implemented upgrades. Among others, these include a revised scheme for dynamic feedbacks of self-attraction and loading; energy-dissipation by parametrized internal wavedrag; partial tide excitations by the tide-generating potential up to degree 3; and a pole-rotation scheme allowing for simulations dedicated to polar areas. Benefiting from the recent updates, the obtained solutions for major tides are on the same level of accuracy as comparable modern unconstrained tidal models. Furthermore, we show that the relative accuracy level only drops moderately for tidal excitations with small excitation strength (e.g. for minor tides), thus narrowing down the accuracy gap to data-constrained tidal atlases. Exemplarily for this, we introduce solutions for minor tidal excitations of degrees 2 and 3 that represent valuable constraints for the expected ocean tide dynamics. While they are currently not considered for GRACE-FO de-aliasing we demonstrate that third-degree tides can lead to relevant aliasing of satellite gravity fields and correspond closely to recently published empirical solutions (Ray, 2020).</p>

Signal coding in cockroach photoreceptors is tuned to dim environments

2012 ◽  
Vol 108 (10) ◽  
pp. 2641-2652 ◽  
Author(s):  
K. Heimonen ◽  
E.-V. Immonen ◽  
R. V. Frolov ◽  
I. Salmela ◽  
M. Juusola ◽  
...  
Keyword(s):  
Information Transfer ◽  
Periplaneta Americana ◽  
Single Photon ◽  
Signal To Noise Ratio ◽  
Temporal Integration ◽  
High Sensitivity ◽  
Single Photons ◽  
Low Frequencies ◽  
Whole Cell Patch Clamp ◽  
High Level

In dim light, scarcity of photons typically leads to poor vision. Nonetheless, many animals show visually guided behavior with dim environments. We investigated the signaling properties of photoreceptors of the dark active cockroach ( Periplaneta americana) using intracellular and whole-cell patch-clamp recordings to determine whether they show selective functional adaptations to dark. Expectedly, dark-adapted photoreceptors generated large and slow responses to single photons. However, when light adapted, responses of both phototransduction and the nontransductive membrane to white noise (WN)-modulated stimuli remained slow with corner frequencies ∼20 Hz. This promotes temporal integration of light inputs and maintains high sensitivity of vision. Adaptive changes in dynamics were limited to dim conditions. Characteristically, both step and frequency responses stayed effectively unchanged for intensities >1,000 photons/s/photoreceptor. A signal-to-noise ratio (SNR) of the light responses was transiently higher at frequencies <5 Hz for ∼5 s after light onset but deteriorated to a lower value upon longer stimulation. Naturalistic light stimuli, as opposed to WN, evoked markedly larger responses with higher SNRs at low frequencies. This allowed realistic estimates of information transfer rates, which saturated at ∼100 bits/s at low-light intensities. We found, therefore, selective adaptations beneficial for vision in dim environments in cockroach photoreceptors: large amplitude of single-photon responses, constant high level of temporal integration of light inputs, saturation of response properties at low intensities, and only transiently efficient encoding of light contrasts. The results also suggest that the sources of the large functional variability among different photoreceptors reside mostly in phototransduction processes and not in the properties of the nontransductive membrane.

scholarly journals Image Shadow Removal Using End-to-End Deep Convolutional Neural Networks

2019 ◽  
Vol 9 (5) ◽  
pp. 1009 ◽  
Author(s):  
Hui Fan ◽  
Meng Han ◽  
Jinjiang Li
Keyword(s):  
Signal To Noise Ratio ◽  
Network Models ◽  
Structural Similarity ◽  
Shadow Removal ◽  
Deep Convolutional Neural Networks ◽  
Edge Information ◽  
Qualitative And Quantitative ◽  
Automatic Methods ◽  
End To End ◽  
High Level

Image degradation caused by shadows is likely to cause technological issues in image segmentation and target recognition. In view of the existing shadow removal methods, there are problems such as small and trivial shadow processing, the scarcity of end-to-end automatic methods, the neglecting of light, and high-level semantic information such as materials. An end-to-end deep convolutional neural network is proposed to further improve the image shadow removal effect. The network mainly consists of two network models, an encoder–decoder network and a small refinement network. The former predicts the alpha shadow scale factor, and the latter refines to obtain sharper edge information. In addition, a new image database (remove shadow database, RSDB) is constructed; and qualitative and quantitative evaluations are made on databases such as UIUC, UCF and newly-created databases (RSDB) with various real images. Using the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) for quantitative analysis, the algorithm has a big improvement on the PSNR and the SSIM as opposed to other methods. In terms of qualitative comparisons, the network shadow has a clearer and shadow-free image that is consistent with the original image color and texture, and the detail processing effect is much better. The experimental results show that the proposed algorithm is superior to other algorithms, and it is more robust in subjective vision and objective quantization.

A Feature Recognition Approach with Fuzzy ART Neural Network

2011 ◽  
Vol 279 ◽  
pp. 406-411
Author(s):  
Cong Lu ◽  
Jun Zha
Keyword(s):  
Neural Network ◽  
Feature Recognition ◽  
Boundary Representation ◽  
Solid Model ◽  
Fuzzy Art ◽  
Fast Learning ◽  
Key Technologies ◽  
Learning Modes ◽  
Slow Learning

This paper proposes a feature recognition approach from a boundary representation solid model with Fuzzy ART neural network. To recognize the feature efficiently, some key technologies in Fuzzy ART neural network are used. The influence of the vigilance parameter on feature recognition is studied, and two learning modes, fast learning and slow learning are adopted and compared in feature recognition. Finally, a case study is given to verify the proposed approach.

Reward-Modulated Hebbian Learning of Decision Making

2010 ◽  
Vol 22 (6) ◽  
pp. 1399-1444 ◽  
Author(s):  
Michael Pfeiffer ◽  
Bernhard Nessler ◽  
Rodney J. Douglas ◽  
Wolfgang Maass
Keyword(s):  
Decision Making ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Final Decision ◽  
Bayesian Decision ◽  
Decision Stage ◽  
Fast Learning ◽  
Hebb Rule ◽  
Adaptive Processes ◽  
Perceptron Learning

We introduce a framework for decision making in which the learning of decision making is reduced to its simplest and biologically most plausible form: Hebbian learning on a linear neuron. We cast our Bayesian-Hebb learning rule as reinforcement learning in which certain decisions are rewarded and prove that each synaptic weight will on average converge exponentially fast to the log-odd of receiving a reward when its pre- and postsynaptic neurons are active. In our simple architecture, a particular action is selected from the set of candidate actions by a winner-take-all operation. The global reward assigned to this action then modulates the update of each synapse. Apart from this global reward signal, our reward-modulated Bayesian Hebb rule is a pure Hebb update that depends only on the coactivation of the pre- and postsynaptic neurons, not on the weighted sum of all presynaptic inputs to the postsynaptic neuron as in the perceptron learning rule or the Rescorla-Wagner rule. This simple approach to action-selection learning requires that information about sensory inputs be presented to the Bayesian decision stage in a suitably preprocessed form resulting from other adaptive processes (acting on a larger timescale) that detect salient dependencies among input features. Hence our proposed framework for fast learning of decisions also provides interesting new hypotheses regarding neural nodes and computational goals of cortical areas that provide input to the final decision stage.

scholarly journals Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy

2021 ◽  
Vol 15 ◽  
Author(s):  
Shirin Dora ◽  
Sander M. Bohte ◽  
Cyriel M. A. Pennartz
Keyword(s):  
Network Architecture ◽  
Hebbian Learning ◽  
Predictive Coding ◽  
Learning Rule ◽  
Neuronal Populations ◽  
Hebbian Learning Rule ◽  
Cortical Hierarchy ◽  
Latent Representations ◽  
High Level ◽  
Visual Cortical

Predictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule and mimics the feedforward and feedback connectivity of the cortex. After training on image datasets, the models formed latent representations in higher areas that allowed reconstruction of the original images. We analyzed low- and high-level properties such as orientation selectivity, object selectivity and sparseness of neuronal populations in the model. As reported experimentally, image selectivity increased systematically across ascending areas in the model hierarchy. Depending on the strength of regularization factors, sparseness also increased from lower to higher areas. The results suggest a rationale as to why experimental results on sparseness across the cortical hierarchy have been inconsistent. Finally, representations for different object classes became more distinguishable from lower to higher areas. Thus, deep neural networks trained using a gated Hebbian formulation of predictive coding can reproduce several properties associated with neuronal responses along the visual cortical hierarchy.

scholarly journals Inference of Brain States under Anesthesia with Meta Learning Based Deep Learning Models

2021 ◽  
Author(s):  
Qihang Wang ◽  
Feng Liu ◽  
Guihong Wan ◽  
Ying Chen
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Short Term Memory ◽  
Signal To Noise Ratio ◽  
General Anesthetics ◽  
Learning Models ◽  
Multi Stage ◽  
Meta Learning ◽  
Brain States ◽  
High Level

AbstractMonitoring the depth of unconsciousness during anesthesia is useful in both clinical settings and neuroscience investigations to understand brain mechanisms. Electroencephalogram (EEG) has been used as an objective means of characterizing brain altered arousal and/or cognition states induced by anesthetics in real-time. Different general anesthetics affect cerebral electrical activities in different ways. However, the performance of conventional machine learning models on EEG data is unsatisfactory due to the low Signal to Noise Ratio (SNR) in the EEG signals, especially in the office-based anesthesia EEG setting. Deep learning models have been used widely in the field of Brain Computer Interface (BCI) to perform classification and pattern recognition tasks due to their capability of good generalization and handling noises. Compared to other BCI applications, where deep learning has demonstrated encouraging results, the deep learning approach for classifying different brain consciousness states under anesthesia has been much less investigated. In this paper, we propose a new framework based on meta-learning using deep neural networks, named Anes-MetaNet, to classify brain states under anesthetics. The Anes-MetaNet is composed of Convolutional Neural Networks (CNN) to extract power spectrum features, and a time consequence model based on Long Short-Term Memory (LSTM) Networks to capture the temporal dependencies, and a meta-learning framework to handle large cross-subject variability. We used a multi-stage training paradigm to improve the performance, which is justified by visualizing the high-level feature mapping. Experiments on the office-based anesthesia EEG dataset demonstrate the effectiveness of our proposed Anes-MetaNet by comparison of existing methods.

Sign in / Sign up

Export Citation Format

Share Document