scholarly journals Nonlinear decoding of natural images from large-scale primate retinal ganglion recordings

2020 ◽  
Author(s):  
Young Joon Kim ◽  
Nora Brackbill ◽  
Ella Batty ◽  
JinHyung Lee ◽  
Catalin Mitelut ◽  
...  
Keyword(s):  
Visual Information ◽  
Large Scale ◽  
Ganglion Cells ◽  
State Of The Art ◽  
Fundamental Problem ◽  
The State ◽  
Natural Images ◽  
Retinal Ganglion ◽  
Neural Decoding ◽  
Sensory Stimuli

AbstractDecoding sensory stimuli from neural activity can provide insight into how the nervous system might interpret the physical environment, and facilitates the development of brain-machine interfaces. Nevertheless, the neural decoding problem remains a significant open challenge. Here, we present an efficient nonlinear decoding approach for inferring natural scene stimuli from the spiking activities of retinal ganglion cells (RGCs). Our approach uses neural networks to improve upon existing decoders in both accuracy and scalability. Trained and validated on real retinal spike data from > 1000 simultaneously recorded macaque RGC units, the decoder demonstrates the necessity of nonlinear computations for accurate decoding of the fine structures of visual stimuli. Specifically, high-pass spatial features of natural images can only be decoded using nonlinear techniques, while low-pass features can be extracted equally well by linear and nonlinear methods. Together, these results advance the state of the art in decoding natural stimuli from large populations of neurons.Author summaryNeural decoding is a fundamental problem in computational and statistical neuroscience. There is an enormous literature on this problem, applied to a wide variety of brain areas and nervous systems. Here we focus on the problem of decoding visual information from the retina. The bulk of previous work here has focused on simple linear decoders, applied to modest numbers of simultaneously recorded cells, to decode artificial stimuli. In contrast, here we develop a scalable nonlinear decoding method to decode natural images from the responses of over a thousand simultaneously recorded units, and show that this decoder significantly improves on the state of the art.

scholarly journals Nonlinear Decoding of Natural Images from Large-Scale Primate Retinal Ganglion Recordings

2021 ◽  
pp. 1-32
Author(s):  
Young Joon Kim ◽  
Nora Brackbill ◽  
Eleanor Batty ◽  
JinHyung Lee ◽  
Catalin Mitelut ◽  
...  
Keyword(s):  
Large Scale ◽  
Ganglion Cells ◽  
Natural Images ◽  
Retinal Ganglion ◽  
Neural Decoding ◽  
Sensory Stimuli ◽  
Spatial Features ◽  
Large Populations ◽  
Low Pass ◽  
High Pass

Abstract Decoding sensory stimuli from neural activity can provide insight into how the nervous system might interpret the physical environment, and facilitates the development of brain-machine interfaces. Nevertheless, the neural decoding problem remains a significant open challenge. Here, we present an efficient nonlinear decoding approach for inferring natural scene stimuli from the spiking activities of retinal ganglion cells (RGCs). Our approach uses neural networks to improve on existing decoders in both accuracy and scalability. Trained and validated on real retinal spike data from more than 1000 simultaneously recorded macaque RGC units, the decoder demonstrates the necessity of nonlinear computations for accurate decoding of the fine structures of visual stimuli. Specifically, high-pass spatial features of natural images can only be decoded using nonlinear techniques, while low-pass features can be extracted equally well by linear and nonlinear methods. Together, these results advance the state of the art in decoding natural stimuli from large populations of neurons.

scholarly journals Focal Electrical Stimulation of Human Retinal Ganglion Cells

2020 ◽  
Author(s):  
Sasi Madugula ◽  
Alex R. Gogliettino ◽  
Moosa Zaidi ◽  
Gorish Aggarwal ◽  
Alexandra Kling ◽  
...  
Keyword(s):  
Electrical Stimulation ◽  
Retinal Ganglion Cells ◽  
Visual Information ◽  
Large Scale ◽  
Ganglion Cells ◽  
Cell Types ◽  
Retinal Ganglion ◽  
Single Spike ◽  
Axon Bundle ◽  
Stimulation Of

ABSTRACTElectrical stimulation of retinal ganglion cells (RGCs), which transmit visual information to the brain, is used in retinal implants to treat blindness caused by photoreceptor degeneration. However, the performance of existing clinical implants is limited by indiscriminate stimulation of many cells and cell types. Recent work in isolated macaque retina has shown the ability to precisely evoke spikes in the major RGC types by direct electrical stimulation at safe current levels, with single-cell, single-spike resolution and avoidance of axon bundle activation in many cases. However, these findings have not been verified in the human retina. Here, electrical activation of the major human RGC types was examined using large-scale, multi-electrode recording and stimulation and compared to results from several macaque retinas obtained using the same methods. Electrical stimulation of the major human RGC types closely paralleled results in macaque, with similar somatic and axonal stimulation thresholds, cellular and cell type selectivity of stimulation, avoidance of axon bundle stimulation by calibration, targeting of different cell types based on their distinct electrical signatures, and potential efficacy of real-time stimulus optimization for artificial vision. The results indicate that the macaque retina provides a quantitatively accurate picture of how focal electrical stimulation can be used in future high-resolution implants.

scholarly journals Neural Networks for Efficient Bayesian Decoding of Natural Images from Retinal Neurons

2017 ◽  
Author(s):  
Nikhil Parthasarathy ◽  
Eleanor Batty ◽  
William Falcon ◽  
Thomas Rutten ◽  
Mohit Rajpal ◽  
...  
Keyword(s):  
Neural Networks ◽  
Ganglion Cells ◽  
Natural Images ◽  
Retinal Ganglion ◽  
Natural Scene Statistics ◽  
Neural Signals ◽  
Sensory Stimuli ◽  
Convolutional Autoencoder ◽  
Approximate Bayesian ◽  
Linear Decoding

AbstractDecoding sensory stimuli from neural signals can be used to reveal how we sense our physical environment, and is valuable for the design of brain-machine interfaces. However, existing linear techniques for neural decoding may not fully reveal or exploit the fidelity of the neural signal. Here we develop a new approximate Bayesian method for decoding natural images from the spiking activity of populations of retinal ganglion cells (RGCs). We sidestep known computational challenges with Bayesian inference by exploiting artificial neural networks developed for computer vision, enabling fast nonlinear decoding that incorporates natural scene statistics implicitly. We use a decoder architecture that first linearly reconstructs an image from RGC spikes, then applies a convolutional autoencoder to enhance the image. The resulting decoder, trained on natural images and simulated neural responses, significantly outperforms linear decoding, as well as simple point-wise nonlinear decoding. These results provide a tool for the assessment and optimization of retinal prosthesis technologies, and reveal that the retina may provide a more accurate representation of the visual scene than previously appreciated.

scholarly journals Non-parametric physiological classification of retinal ganglion cells in the mouse retina

2018 ◽  
Author(s):  
Jonathan Jouty ◽  
Gerrit Hilgen ◽  
Evelyne Sernagor ◽  
Matthias H. Hennig
Keyword(s):  
Retinal Ganglion Cells ◽  
Visual Information ◽  
Large Scale ◽  
Distance Measure ◽  
Ganglion Cells ◽  
High Density ◽  
Mouse Retina ◽  
Retinal Ganglion ◽  
Electrode Arrays

Retinal ganglion cells, the sole output neurons of the retina, exhibit surprising diversity. A recent study reported over 30 distinct types in the mouse retina, indicating that the processing of visual information is highly parallelised in the brain. The advent of high density multi-electrode arrays now enables recording from many hundreds to thousands of neurons from a single retina. Here we describe a method for the automatic classification of large-scale retinal recordings using a simple stimulus paradigm and a spike train distance measure as a clustering metric. We evaluate our approach using synthetic spike trains, and demonstrate that major known cell types are identified in high-density recording sessions from the mouse retina with around 1000 retinal ganglion cells. A comparison across different retinas reveals substantial variability between preparations, suggesting pooling data across retinas should be approached with caution. As a parameter-free method, our approach is broadly applicable for cellular physiological classification in all sensory modalities.

scholarly journals Large-scale Semantic Parsing without Question-Answer Pairs

2014 ◽  
Vol 2 ◽  
pp. 377-392 ◽  
Author(s):  
Siva Reddy ◽  
Mirella Lapata ◽  
Mark Steedman
Keyword(s):  
Natural Language ◽  
Large Scale ◽  
Graph Matching ◽  
State Of The Art ◽  
The State ◽  
Semantic Parsing ◽  
Matching Problem ◽  
Weak Supervision ◽  
Benchmark Datasets

In this paper we introduce a novel semantic parsing approach to query Freebase in natural language without requiring manual annotations or question-answer pairs. Our key insight is to represent natural language via semantic graphs whose topology shares many commonalities with Freebase. Given this representation, we conceptualize semantic parsing as a graph matching problem. Our model converts sentences to semantic graphs using CCG and subsequently grounds them to Freebase guided by denotations as a form of weak supervision. Evaluation experiments on a subset of the Free917 and WebQuestions benchmark datasets show our semantic parser improves over the state of the art.

scholarly journals Interspike Interval Based Filtering of Directional Selective Retinal Ganglion Cells Spike Trains

2012 ◽  
Vol 2012 ◽  
pp. 1-17 ◽  
Author(s):  
Aurel Vasile Martiniuc ◽  
Alois Knoll
Keyword(s):  
Retinal Ganglion Cells ◽  
Visual Stimulus ◽  
Visual Information ◽  
Ganglion Cells ◽  
Neuronal Response ◽  
Spike Trains ◽  
Retinal Ganglion ◽  
Stimulus Motion ◽  
Information Rates ◽  
Grating Bars

The information regarding visual stimulus is encoded in spike trains at the output of retina by retinal ganglion cells (RGCs). Among these, the directional selective cells (DSRGC) are signaling the direction of stimulus motion. DSRGCs' spike trains show accentuated periods of short interspike intervals (ISIs) framed by periods of isolated spikes. Here we use two types of visual stimulus, white noise and drifting bars, and show that short ISI spikes of DSRGCs spike trains are more often correlated to their preferred stimulus feature (that is, the direction of stimulus motion) and carry more information than longer ISI spikes. Firstly, our results show that correlation between stimulus and recorded neuronal response is best at short ISI spiking activity and decrease as ISI becomes larger. We then used grating bars stimulus and found that as ISI becomes shorter the directional selectivity is better and information rates are higher. Interestingly, for the less encountered type of DSRGC, known as ON-DSRGC, short ISI distribution and information rates revealed consistent differences when compared with the other directional selective cell type, the ON-OFF DSRGC. However, these findings suggest that ISI-based temporal filtering integrates a mechanism for visual information processing at the output of retina toward higher stages within early visual system.

scholarly journals A method for single-neuron chronic recording from the retina in awake mice

Science ◽  
2018 ◽  
Vol 360 (6396) ◽  
pp. 1447-1451 ◽  
Author(s):  
Guosong Hong ◽  
Tian-Ming Fu ◽  
Mu Qiao ◽  
Robert D. Viveros ◽  
Xiao Yang ◽  
...  
Keyword(s):  
Retinal Ganglion Cells ◽  
Visual Information ◽  
Single Neuron ◽  
Ganglion Cells ◽  
Ex Vivo ◽  
Neural Circuitry ◽  
Retinal Ganglion ◽  
Chronic Recording ◽  
The Brain

The retina, which processes visual information and sends it to the brain, is an excellent model for studying neural circuitry. It has been probed extensively ex vivo but has been refractory to chronic in vivo electrophysiology. We report a nonsurgical method to achieve chronically stable in vivo recordings from single retinal ganglion cells (RGCs) in awake mice. We developed a noncoaxial intravitreal injection scheme in which injected mesh electronics unrolls inside the eye and conformally coats the highly curved retina without compromising normal eye functions. The method allows 16-channel recordings from multiple types of RGCs with stable responses to visual stimuli for at least 2 weeks, and reveals circadian rhythms in RGC responses over multiple day/night cycles.

A Comprehensive Taxonomy of Dynamic Texture Representation

2023 ◽  
Vol 55 (1) ◽  
pp. 1-39
Author(s):  
Thanh Tuan Nguyen ◽  
Thanh Phuong Nguyen
Keyword(s):  
Large Scale ◽  
Environmental Changes ◽  
State Of The Art ◽  
The State ◽  
Future Research ◽  
Research Activities ◽  
Potential Applications ◽  
Benchmark Datasets ◽  
Negative Impacts ◽  
Made In

Representing dynamic textures (DTs) plays an important role in many real implementations in the computer vision community. Due to the turbulent and non-directional motions of DTs along with the negative impacts of different factors (e.g., environmental changes, noise, illumination, etc.), efficiently analyzing DTs has raised considerable challenges for the state-of-the-art approaches. For 20 years, many different techniques have been introduced to handle the above well-known issues for enhancing the performance. Those methods have shown valuable contributions, but the problems have been incompletely dealt with, particularly recognizing DTs on large-scale datasets. In this article, we present a comprehensive taxonomy of DT representation in order to purposefully give a thorough overview of the existing methods along with overall evaluations of their obtained performances. Accordingly, we arrange the methods into six canonical categories. Each of them is then taken in a brief presentation of its principal methodology stream and various related variants. The effectiveness levels of the state-of-the-art methods are then investigated and thoroughly discussed with respect to quantitative and qualitative evaluations in classifying DTs on benchmark datasets. Finally, we point out several potential applications and the remaining challenges that should be addressed in further directions. In comparison with two existing shallow DT surveys (i.e., the first one is out of date as it was made in 2005, while the newer one (published in 2016) is an inadequate overview), we believe that our proposed comprehensive taxonomy not only provides a better view of DT representation for the target readers but also stimulates future research activities.

Precise No-Reference Image Quality Evaluation Based on Distortion Identification

2021 ◽  
Vol 17 (3s) ◽  
pp. 1-21
Author(s):  
Chenggang Yan ◽  
Tong Teng ◽  
Yutao Liu ◽  
Yongbing Zhang ◽  
Haoqian Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Image Quality ◽  
Quality Assessment ◽  
Large Scale ◽  
Quality Evaluation ◽  
Image Quality Assessment ◽  
State Of The Art ◽  
Gaussian White Noise ◽  
The State ◽  
Reference Image

The difficulty of no-reference image quality assessment (NR IQA) often lies in the lack of knowledge about the distortion in the image, which makes quality assessment blind and thus inefficient. To tackle such issue, in this article, we propose a novel scheme for precise NR IQA, which includes two successive steps, i.e., distortion identification and targeted quality evaluation. In the first step, we employ the well-known Inception-ResNet-v2 neural network to train a classifier that classifies the possible distortion in the image into the four most common distortion types, i.e., Gaussian white noise (WN), Gaussian blur (GB), jpeg compression (JPEG), and jpeg2000 compression (JP2K). Specifically, the deep neural network is trained on the large-scale Waterloo Exploration database, which ensures the robustness and high performance of distortion classification. In the second step, after determining the distortion type of the image, we then design a specific approach to quantify the image distortion level, which can estimate the image quality specially and more precisely. Extensive experiments performed on LIVE, TID2013, CSIQ, and Waterloo Exploration databases demonstrate that (1) the accuracy of our distortion classification is higher than that of the state-of-the-art distortion classification methods, and (2) the proposed NR IQA method outperforms the state-of-the-art NR IQA methods in quantifying the image quality.

scholarly journals Coupled CycleGAN: Unsupervised Hashing Network for Cross-Modal Retrieval

2019 ◽  
Vol 33 ◽  
pp. 176-183 ◽  
Author(s):  
Chao Li ◽  
Cheng Deng ◽  
Lei Wang ◽  
De Xie ◽  
Xianglong Liu
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
The State ◽  
Storage Cost ◽  
Common Representation ◽  
Benchmark Datasets ◽  
Query Efficiency ◽  
Hash Codes

In recent years, hashing has attracted more and more attention owing to its superior capacity of low storage cost and high query efficiency in large-scale cross-modal retrieval. Benefiting from deep leaning, continuously compelling results in cross-modal retrieval community have been achieved. However, existing deep cross-modal hashing methods either rely on amounts of labeled information or have no ability to learn an accuracy correlation between different modalities. In this paper, we proposed Unsupervised coupled Cycle generative adversarial Hashing networks (UCH), for cross-modal retrieval, where outer-cycle network is used to learn powerful common representation, and inner-cycle network is explained to generate reliable hash codes. Specifically, our proposed UCH seamlessly couples these two networks with generative adversarial mechanism, which can be optimized simultaneously to learn representation and hash codes. Extensive experiments on three popular benchmark datasets show that the proposed UCH outperforms the state-of-the-art unsupervised cross-modal hashing methods.

Sign in / Sign up

Export Citation Format

Share Document