Robust Working Memory through Short-Term Synaptic Plasticity

Working memory has long been thought to arise from sustained spiking/attractor dynamics. However, recent work has suggested that short-term synaptic plasticity (STSP) may help maintain attractor states over gaps in time with little or no spiking. To determine if STSP endows additional functional advantages, we trained artificial recurrent neural networks (RNNs) with and without STSP to perform an object working memory task. We found that RNNs with and without STSP were both able to maintain memories over distractors presented in the middle of the memory delay. However, RNNs with STSP showed activity that was similar to that seen in the cortex of monkeys performing the same task. By contrast, RNNs without STSP showed activity that was less brain-like. Further, RNNs with STSP were more robust to noise and network degradation than RNNs without STSP. These results show that STSP can not only help maintain working memories, it also makes neural networks more robust.

Emergent Intelligence and Distributed Stochastic Optimization

Emergent intelligence is a property of a system of elements that is not inherent in each element individually. This behavior is based on local communications. This behavior helps to adapt to emerging uncertainties and achieve a global goal. This behavior exists in the natural world. A simplified example of emergent intelligence from the natural world is given. The repetition of natural behavior with the help of simple technical devices, which are limited in resources and cheap in construction, and the use of multi-agent approaches is considered. Distributed algorithms using local communications are considered. Such algorithms are more robust to noise.

Realizing the Effective Detection of Tumor in Magnetic Resonance Imaging using Cluster-Sparse Assisted Super-Resolution

Recently, significant research has been done in Super-Resolution (SR) methods for augmenting the spatial resolution of the Magnetic Resonance (MR) images, which aids the physician in improved disease diagnoses. Single SR methods have drawbacks; they fail to capture self-similarity in non-local patches and are not robust to noise. To exploit the non-local self-similarity and intrinsic sparsity in MR images, this paper proposes the use of Cluster-Sparse Assisted Super-Resolution. This SR method effectively captures similarity in non-locally positioned patches by training on clusters of patches using a self-adaptive dictionary. This method of training also leads to better edge and texture detection. Experiments show that using Cluster-Sparse Assisted Super-Resolution for brain MR images results in enhanced detection of lesions leading to better diagnosis.

Chapter 11. Neuro-Symbolic Semantic Reasoning

Humans have astounding reasoning capabilities. They can learn from very few examples while providing explanations for their decision-making process. In contrast, deep learning techniques–even though robust to noise and very effective in generalizing across several fields including machine vision, natural language understanding, speech recognition, etc. –require large amounts of data and are mostly unable to provide explanations for their decisions. Attaining human-level robust reasoning requires combining sound symbolic reasoning with robust connectionist learning. However, connectionist learning uses low-level representations–such as embeddings–rather than symbolic representations. This challenge constitutes what is referred to as the Neuro-Symbolic gap. A field of study to bridge this gap between the two paradigms has been called neuro-symbolic integration or neuro-symbolic computing. This chapter aims to present approaches that contribute towards bridging the Neuro-Symbolic gap specifically in the Semantic Web field, RDF Schema (RDFS) and EL+ reasoning and to discuss the benefits and shortcomings of neuro-symbolic reasoning.

CLARITY: comparing heterogeneous data using dissimilarity

Integrating datasets from different disciplines is hard because the data are often qualitatively different in meaning, scale and reliability. When two datasets describe the same entities, many scientific questions can be phrased around whether the (dis)similarities between entities are conserved across such different data. Our method, CLARITY, quantifies consistency across datasets, identifies where inconsistencies arise and aids in their interpretation. We illustrate this using three diverse comparisons: gene methylation versus expression, evolution of language sounds versus word use, and country-level economic metrics versus cultural beliefs. The non-parametric approach is robust to noise and differences in scaling, and makes only weak assumptions about how the data were generated. It operates by decomposing similarities into two components: a ‘structural’ component analogous to a clustering, and an underlying ‘relationship’ between those structures. This allows a ‘structural comparison’ between two similarity matrices using their predictability from ‘structure’. Significance is assessed with the help of re-sampling appropriate for each dataset. The software, CLARITY, is available as an R package from github.com/danjlawson/CLARITY .

Comparison of unfolding methods for the inference of runaway electron energy distribution from γ-ray spectroscopic measurements

Unfolding techniques are employed to reconstruct the 1D energy distribution of runaway electrons from Bremsstrahlung hard X-ray spectrum emitted during plasma disruptions in tokamaks. Here we compare four inversion methods: truncated singular value decomposition, which is a linear algebra technique, maximum likelihood expectation maximization, which is an iterative method, and Tikhonov regularization applied to χ 2 and Poisson statistics, which are two minimization approaches. The reconstruction fidelity and the capability of estimating cumulative statistics, such as the mean and maximum energy, have been assessed on both synthetic and experimental spectra. The effect of measurements limitations, such as the low energy cut and few number of counts, on the final reconstruction has also been studied. We find that the iterative method performs best as it better describes the statistics of the experimental data and is more robust to noise in the recorded spectrum.

Relationship Between Deceleration Morphology and Phase Rectified Signal Averaging-Based Parameters During Labor

During labor, uterine contractions trigger the response of the autonomic nervous system (ANS) of the fetus, producing sawtooth-like decelerations in the fetal heart rate (FHR) series. Under chronic hypoxia, ANS is known to regulate FHR differently with respect to healthy fetuses. In this study, we hypothesized that such different ANS regulation might also lead to a change in the FHR deceleration morphology. The hypothesis was tested in an animal model comprising nine normoxic and five chronically hypoxic fetuses that underwent a protocol of umbilical cord occlusions (UCOs). Deceleration morphologies in the fetal inter-beat time interval (FRR) series were modeled using a trapezoid with four parameters, i.e., baseline b, deceleration depth a, UCO response time τu and recovery time τr. Comparing normoxic and hypoxic sheep, we found a clear difference for τu (24.8±9.4 vs. 39.8±9.7 s; p < 0.05), a (268.1±109.5 vs. 373.0±46.0 ms; p < 0.1) and Δτ = τu − τr (13.2±6.9 vs. 23.9±7.5 s; p < 0.05). Therefore, the animal model supported the hypothesis that hypoxic fetuses have a longer response time τu and larger asymmetry Δτ as a response to UCOs. Assessing these morphological parameters during labor is challenging due to non-stationarity, phase desynchronization and noise. For this reason, in the second part of the study, we quantified whether acceleration capacity (AC), deceleration capacity (DC), and deceleration reserve (DR), computed through Phase-Rectified Signal Averaging (PRSA, known to be robust to noise), were correlated with the morphological parameters. DC, AC and DR were correlated with τu, τr and Δτ for a wide range of the PRSA parameter T (Pearson's correlation ρ > 0.8, p < 0.05). In conclusion, deceleration morphologies have been found to differ between normoxic and hypoxic sheep fetuses during UCOs. The same difference can be assessed through PRSA based parameters, further motivating future investigations on the translational potential of this methodology on human data.

Fast and accurate annotation of acoustic signals with deep neural networks

Acoustic signals serve communication within and across species throughout the animal kingdom. Studying the genetics, evolution, and neurobiology of acoustic communication requires annotating acoustic signals: segmenting and identifying individual acoustic elements like syllables or sound pulses. To be useful, annotations need to be accurate, robust to noise, and fast.We here introduce DeepAudioSegmenter (DAS), a method that annotates acoustic signals across species based on a deep-learning derived hierarchical presentation of sound. We demonstrate the accuracy, robustness, and speed of DAS using acoustic signals with diverse characteristics from insects, birds, and mammals. DAS comes with a graphical user interface for annotating song, training the network, and for generating and proofreading annotations. The method can be trained to annotate signals from new species with little manual annotation and can be combined with unsupervised methods to discover novel signal types. DAS annotates song with high throughput and low latency for experimental interventions in realtime. Overall, DAS is a universal, versatile, and accessible tool for annotating acoustic communication signals.

A Randomized Greedy Algorithm for Piecewise Linear Motion Planning

We describe and implement a randomized algorithm that inputs a polyhedron, thought of as the space of states of some automated guided vehicle R, and outputs an explicit system of piecewise linear motion planners for R. The algorithm is designed in such a way that the cardinality of the output is probabilistically close (with parameters chosen by the user) to the minimal possible cardinality.This yields the first automated solution for robust-to-noise robot motion planning in terms of simplicial complexity (SC) techniques, a discretization of Farber’s topological complexity TC. Besides its relevance toward technological applications, our work reveals that, unlike other discrete approaches to TC, the SC model can recast Farber’s invariant without having to introduce costly subdivisions. We develop and implement our algorithm by actually discretizing Macías-Virgós and Mosquera-Lois’ notion of homotopic distance, thus encompassing computer estimations of other sectional category invariants as well, such as the Lusternik-Schnirelmann category of polyhedra.