Markovian Bias of Neural-based Architectures With Feedback Connections

Author(s):  
Peter Tiňo ◽  
Barbara Hammer ◽  
Mikael Bodén
Keyword(s):  
2020 ◽  
Vol 124 (2) ◽  
pp. 404-417 ◽  
Author(s):  
Peter W. Campbell ◽  
Gubbi Govindaiah ◽  
Sean P. Masterson ◽  
Martha E. Bickford ◽  
William Guido

The thalamic reticular nucleus (TRN) modulates thalamocortical transmission through inhibition. In mouse, TRN terminals in the dorsal lateral geniculate nucleus (dLGN) form synapses with relay neurons but not interneurons. Stimulation of TRN terminals in dLGN leads to a frequency-dependent form of inhibition, with higher rates of stimulation leading to a greater suppression of spike firing. Thus, TRN inhibition appears more dynamic than previously recognized, having a graded rather than an all-or-none impact on thalamocortical transmission.


1999 ◽  
Vol 82 (5) ◽  
pp. 2667-2675 ◽  
Author(s):  
Susana Martinez-Conde ◽  
Javier Cudeiro ◽  
Kenneth L. Grieve ◽  
Rosa Rodriguez ◽  
Casto Rivadulla ◽  
...  

In the absence of a direct geniculate input, area 17 cells in the cat are nevertheless able to respond to visual stimuli because of feedback connections from area 18. Anatomic studies have shown that, in the cat visual cortex, layer 5 of area 18 projects to layer 5 of area 17, and layers 2/3 of area 18 project to layers 2/3 of area 17. What is the specific role of these connections? Previous studies have examined the effect of area 18 layer 5 blockade on cells in area 17 layer 5. Here we examine whether the feedback connections from layers 2/3 of area 18 influence the orientation tuning and velocity tuning of cells in layers 2/3 of area 17. Experiments were carried out in anesthetized and paralyzed cats. We blocked reversibly a small region (300 μm radius) in layers 2/3 of area 18 by iontophoretic application of GABA and recorded simultaneously from cells in layers 2/3 of area 17 while stimulating with oriented sweeping bars. Area 17 cells showed either enhanced or suppressed visual responses to sweeping bars of various orientations and velocities during area 18 blockade. For most area 17 cells, orientation bandwidths remained unaltered, and we never observed visual responses during blockade that were absent completely in the preblockade condition. This suggests that area 18 layers 2/3 modulate visual responses in area 17 layers 2/3 without fundamentally altering their specificity.


2019 ◽  
Author(s):  
Hedyeh Rezaei ◽  
Ad Aertsen ◽  
Arvind Kumar ◽  
Alireza Valizadeh

AbstractTransient oscillations in the network activity upon sensory stimulation have been reported in different sensory areas. These evoked oscillations are the generic response of networks of excitatory and inhibitory neurons (EI-networks) to a transient external input. Recently, it has been shown that this resonance property of EI-networks can be exploited for communication in modular neuronal networks by enabling the transmission of sequences of synchronous spike volleys (‘pulse packets’), despite the sparse and weak connectivity between the modules. The condition for successful transmission is that the pulse packet (PP) intervals match the period of the modules’ resonance frequency. Hence, the mechanism was termed communication through resonance (CTR). This mechanism has three sever constraints, though. First, it needs periodic trains of PPs, whereas single PPs fail to propagate. Second, the inter-PP interval needs to match the network resonance. Third, transmission is very slow, because in each module, the network resonance needs to build-up over multiple oscillation cycles. Here, we show that, by adding appropriate feedback connections to the network, the CTR mechanism can be improved and the aforementioned constraints relaxed. Specifically, we show that adding feedback connections between two upstream modules, called the resonance pair, in an otherwise feedforward modular network can support successful propagation of a single PP throughout the entire network. The key condition for successful transmission is that the sum of the forward and backward delays in the resonance pair matches the resonance frequency of the network modules. The transmission is much faster, by more than a factor of two, than in the original CTR mechanism. Moreover, it distinctly lowers the threshold for successful communication by synchronous spiking in modular networks of weakly coupled networks. Thus, our results suggest a new functional role of bidirectional connectivity for the communication in cortical area networks.Author summaryThe cortex is organized as a modular system, with the modules (cortical areas) communicating via weak long-range connections. It has been suggested that the intrinsic resonance properties of population activities in these areas might contribute to enabling successful communication. A module’s intrinsic resonance appears in the damped oscillatory response to an incoming spike volley, enabling successful communication during the peaks of the oscillation. Such communication can be exploited in feedforward networks, provided the participating networks have similar resonance frequencies. This, however, is not necessarily true for cortical networks. Moreover, the communication is slow, as it takes several oscillation cycles to build up the response in the downstream network. Also, only periodic trains of spikes volleys (and not single volleys) with matching intervals can propagate. Here, we present a novel mechanism that alleviates these shortcomings and enables propagation of synchronous spiking across weakly connected networks with not necessarily identical resonance frequencies. In this framework, an individual spike volley can propagate by local amplification through reverberation in a loop between two successive networks, connected by feedforward and feedback connections: the resonance pair. This overcomes the need for activity build-up in downstream networks, causing the volley to propagate distinctly faster and more reliably.


2017 ◽  
Vol 10 (04) ◽  
pp. 710-717
Author(s):  
A. Ahmad ◽  
D. Al Abri ◽  
S. S. Al Busaidi ◽  
M. M. Bait-Suwailam

The authors show that in a Built-In Self-Test (BIST) technique, based on linear-feedback shift registers, when the feedback connections in pseudo-random test-sequence generator and signature analyzer are images of each other and corresponds to primitive characteristic polynomial then behaviors of faults masking remains identical. The simulation results of single stuck-at faults show how the use of such feedback connections in pseudo-random test-sequence generator and signature analyzer yields to mask the same faults.


2020 ◽  
Author(s):  
Haider Al-Tahan ◽  
Yalda Mohsenzadeh

AbstractWhile vision evokes a dense network of feedforward and feedback neural processes in the brain, visual processes are primarily modeled with feedforward hierarchical neural networks, leaving the computational role of feedback processes poorly understood. Here, we developed a generative autoencoder neural network model and adversarially trained it on a categorically diverse data set of images. We hypothesized that the feedback processes in the ventral visual pathway can be represented by reconstruction of the visual information performed by the generative model. We compared representational similarity of the activity patterns in the proposed model with temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) visual brain responses. The proposed generative model identified two segregated neural dynamics in the visual brain. A temporal hierarchy of processes transforming low level visual information into high level semantics in the feedforward sweep, and a temporally later dynamics of inverse processes reconstructing low level visual information from a high level latent representation in the feedback sweep. Our results append to previous studies on neural feedback processes by presenting a new insight into the algorithmic function and the information carried by the feedback processes in the ventral visual pathway.Author summaryIt has been shown that the ventral visual cortex consists of a dense network of regions with feedforward and feedback connections. The feedforward path processes visual inputs along a hierarchy of cortical areas that starts in early visual cortex (an area tuned to low level features e.g. edges/corners) and ends in inferior temporal cortex (an area that responds to higher level categorical contents e.g. faces/objects). Alternatively, the feedback connections modulate neuronal responses in this hierarchy by broadcasting information from higher to lower areas. In recent years, deep neural network models which are trained on object recognition tasks achieved human-level performance and showed similar activation patterns to the visual brain. In this work, we developed a generative neural network model that consists of encoding and decoding sub-networks. By comparing this computational model with the human brain temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) response patterns, we found that the encoder processes resemble the brain feedforward processing dynamics and the decoder shares similarity with the brain feedback processing dynamics. These results provide an algorithmic insight into the spatiotemporal dynamics of feedforward and feedback processes in biological vision.


Entropy ◽  
2020 ◽  
Vol 22 (1) ◽  
pp. 102 ◽  
Author(s):  
Adrian Moldovan ◽  
Angel Caţaron ◽  
Răzvan Andonie

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of feedforward neural networks. The information transfer is measured by the TE of feedback neural connections. Intuitively, TE measures the relevance of a connection in the network and the feedback amplifies this connection. We introduce a backpropagation type training algorithm that uses TE feedback connections to improve its performance.


Sign in / Sign up

Export Citation Format

Share Document