Multiple-Timescale Neural Networks: Generation of History-Dependent Sequences and Inference Through Autonomous Bifurcations

Sequential transitions between metastable states are ubiquitously observed in the neural system and underlying various cognitive functions such as perception and decision making. Although a number of studies with asymmetric Hebbian connectivity have investigated how such sequences are generated, the focused sequences are simple Markov ones. On the other hand, fine recurrent neural networks trained with supervised machine learning methods can generate complex non-Markov sequences, but these sequences are vulnerable against perturbations and such learning methods are biologically implausible. How stable and complex sequences are generated in the neural system still remains unclear. We have developed a neural network with fast and slow dynamics, which are inspired by the hierarchy of timescales on neural activities in the cortex. The slow dynamics store the history of inputs and outputs and affect the fast dynamics depending on the stored history. We show that the learning rule that requires only local information can form the network generating the complex and robust sequences in the fast dynamics. The slow dynamics work as bifurcation parameters for the fast one, wherein they stabilize the next pattern of the sequence before the current pattern is destabilized depending on the previous patterns. This co-existence period leads to the stable transition between the current and the next pattern in the non-Markov sequence. We further find that timescale balance is critical to the co-existence period. Our study provides a novel mechanism generating robust complex sequences with multiple timescales. Considering the multiple timescales are widely observed, the mechanism advances our understanding of temporal processing in the neural system.

Transformer-Based Deep Learning Models for Well Log Processing and Quality Control by Modelling Global Dependence of the Complex Sequences

A single well from any mature field produces approximately 1.7 million Measurement While Drilling (MWD) data points. We either use cross-correlation and covariance measurement, or Long Short-Term Memory (LSTM) based Deep Learning algorithms to diagnose long sequences of extremely noisy data. LSTM's context size of 200 tokens barely accounts for the entire depth. Proposed work develops application of Transformer-based Deep Learning algorithm to diagnose and predict events in complex sequences of well-log data. Sequential models learn geological patterns and petrophysical trends to detect events across depths of well-log data. However, vanishing gradients, exploding gradients and the limits of convolutional filters, limit the diagnosis of ultra-deep wells in complex subsurface information. Vast number of operations required to detect events between two subsurface points at large separation limits them. Transformers-based Models (TbMs) rely on non-sequential modelling that uses self-attention to relate information from different positions in the sequence of well-log, allowing to create an end-to-end, non-sequential, parallel memory network. We use approximately 21 million data points from 21 wells of Volve for the experiment. LSTMs, in addition to auto-regression (AR), autoregressive moving average (ARMA) and autoregressive integrated moving average (ARIMA) conventionally models the events in the time-series well-logs. However, complex global dependencies to detect events in heterogeneous subsurface are challenging for these sequence models. In the presented work we begin with one meter depth of data from Volve, an oil-field in the North Sea, and then proceed up to 1000 meters. Initially LSTMs and ARIMA models were acceptable, as depth increased beyond a few 100 meters their diagnosis started underperforming and a new methodology was required. TbMs have already outperformed several models in large sequences modelling for natural language processing tasks, thus they are very promising to model well-log data with very large depth separation. We scale features and labels according to the maximum and minimum value present in the training dataset and then use the sliding window to get training and evaluation data pairs from well-logs. Additional subsurface features were able to encode some information in the conventional sequential models, but the result did not compare significantly with the TbMs. TbMs achieved Root Mean Square Error of 0.27 on scale of (0-1) while diagnosing the depth up to 5000 meters. This is the first paper to show successful application of Transformer-based deep learning models for well-log diagnosis. Presented model uses a self-attention mechanism to learn complex dependencies and non-linear events from the well-log data. Moreover, the experimental setting discussed in the paper will act as a generalized framework for data from ultra-deep wells and their extremely heterogeneous subsurface environment.

Calculations on Matrix Transformations Involving an Infinite Tridiagonal Matrix

Given any sequence z=znn≥1 of positive real numbers and any set E of complex sequences, we write Ez for the set of all sequences y=ynn≥1 such that y/z=yn/znn≥1∈E; in particular, sz0 denotes the set of all sequences y such that y/z tends to zero. Here, we consider the infinite tridiagonal matrix Br,s,t˜, obtained from the triangle Br,s,t, by deleting its first row. Then we determine the sets of all positive sequences a=ann≥1 such that EaBr,s,t˜⊂Ea, where E=ℓ∞, c0, or c. These results extend some recent results.

Neural Field Theory of Evoked Response Sequences and Mismatch Negativity With Adaptation

Physiologically based neural field theory of the corticothalamic system is used to calculate the responses evoked by trains of auditory stimuli that correspond to different cortical locations via the tonotopic map. The results are shown to account for standard and deviant evoked responses to frequent and rare stimuli, respectively, in the auditory oddball paradigms widely used in human cognitive studies, and the so-called mismatch negativity between them. It also reproduces a wide range of other effects and variants, including the mechanism by which a change in standard responses relative to deviants can develop through adaptation, different responses when two deviants are presented in a row or a standard is presented after two deviants, relaxation of standard responses back to deviant form after a stimulus-free period, and more complex sequences. Some cases are identified in which adaptation does not account for the whole difference between standard and deviant responses. The results thus provide a systematic means to determine how much of the response is due to adaptation in the system comprising the primary auditory cortex and medial geniculate nucleus, and how much requires involvement of higher-level processing.

New Results on the SSIE with an Operator of the form FΔ⊂E+Fx′ Involving the Spaces of Strongly Summable and Convergent Sequences Using the Cesàro Method

Given any sequence a=(an)n≥1 of positive real numbers and any set E of complex sequences, we can use Ea to represent the set of all sequences y=(yn)n≥1 such that y/a=(yn/an)n≥1∈E. In this paper, we use the spaces w∞, w0 and w of strongly bounded, summable to zero and summable sequences, which are the sets of all sequences y such that n−1∑k=1nykn is bounded and tends to zero, and such that y−le∈w0, for some scalarl . These sets were used in the statistical convergence. Then we deal with the solvability of each of the SSIE FΔ⊂E+Fx′, where E is a linear space of sequences, F=c0, c, ℓ∞, w0, w or w∞, and F′=c0, c or ℓ∞. For instance, the solvability of the SSIE wΔ⊂w0+sxc relies on determining the set of all sequences x=xnn≥1∈U+ that satisfy the following statement. For every sequence y that satisfies the condition limn→∞n−1∑k=1nyk−yk−1−l=0, there are two sequences u and v, with y=u+v such that limn→∞n−1∑k=1nuk=0 and limn→∞vn/xn=L for some scalars l and L.

The Polarization Dependence of 2D IR Cross-Peaks Distinguishes Parallel-Stranded and Antiparallel-Stranded DNA G-quadruplexes

Guanine-rich nucleic acid sequences have a tendency to form four-stranded non-canonical motifs known as G-quadruplexes. These motifs may adopt a wide range of structures characterized by size, strand orientation, guanine base conformation, and fold topology. Using three K+-bound model systems, we show that vibrational coupling between guanine C6=O and ring modes varies between parallel-stranded and antiparallel-stranded G-quadruplexes, and that such structures can be distinguished by comparison of polarization dependent cross-peaks in their two-dimensional infrared (2D IR) spectra. Combined with previously defined vibrational frequency trends, this analysis reveals key features of a 30-nucleotide unimolecular variant of the Bcl-2 proximal promoter that are consistent with its reported structure. This study shows that 2D IR spectroscopy is a convenient method for analyzing G-quadruplex structures that can be applied to complex sequences where traditional high-resolution methods are limited by solubility and disorder.

Localization of Hopfian and Cohopfian Objects in the Categories of A − Mod, AGr(A − Mod) and COMP(AGr(A − Mod))

The aim of this paper is to study the localization of hopfian and cohopfian objects in the categories A-Mod of left A-modules, AGr(A-Mod) of graded left A-modules and COMP(AGr(A-Mod)) of complex sequences associated to graded left A-modules.We have among others the main following results :1. Let M be a noetherian graded left A-module, S a saturated multiplicative part formed by the non-zero homogeneous elements of A verifying the left Ore conditions, N a submodule of M, M_{*} is a noetherian quasi-injective complex sequence associated with M and N_{*} is an essential and completely invariant complex sub\--sequence of M_{*}. Then, S^{-1}(N_{*}) the complex sequence of morphisms of left S^{-1}A\--modules is cohopfian if, and only, if S^{-1}(M_{*}) is cohopfian ;2. let M be a graded left A\--module and S a saturated multiplicative part formed by the non-zero homogeneous elements of A verifying the left Ore conditions. If M_{*} is a hopfian, noetherian and quasi-injective complex sequence associated with M, then the complex sequence of morphisms of left S^{-1}(A)-modules S^{-1}(M_{*}) has the following property :{any epimorphism of sub-complex S^{-1}(N_{*}) of S^{-1}(M_{*}) is an isomorphism } ;3. let M be a graded left A-module, N a graded submodule of M, S a saturated multiplicative part formed by the non-zero homogeneous elements of A verifying the left Ore conditions. M_{*} the quasi-projective complex sequence associated with M and $N_{*}$ a superfluous and completely invariant complex sub\--sequence of $M_{*}$. Then the complex morphism sequence of left $S^{-1}(A)$\--modules $S^{-1}(N_{*})$ is hopfian if, and only if, $S^{-1}(M_{*}/N_{*})$ the complex sequence associated with S^{-1}(M/N) is hopfian.

Extrinsic and Intrinsic Ligaments of the Wrist

Carpal stability depends on the integrity of both intra-articular and intracapsular carpal ligaments. In this review, the role of the radial-sided and ulnar-sided extrinsic and intrinsic ligaments is described, as well as their advanced imaging using magnetic resonance arthrography (MRA) and contrast-enhanced magnetic resonance imaging (MRI) with three-dimensional (3D) scapholunate complex sequences and thin slices. In the last decade, the new concept of a so-called “scapholunate complex” has emerged among hand surgeons, just as the triangular ligament became known as the triangular fibrocartilage complex (TFCC).The scapholunate ligament complex comprises the intrinsic scapholunate (SL), the extrinsic palmar radiocarpal: radioscaphocapitate (RSC), long radiolunate (LRL), short radiolunate (SRL) ligaments, the extrinsic dorsal radiocarpal (DRC) ligament, the dorsal intercarpal (DIC) ligament, as well as the dorsal capsular scapholunate septum (DCSS), a more recently described anatomical structure, and the intrinsic palmar midcarpal scaphotrapeziotrapezoid (STT) ligament complex. The scapholunate (SL) ligament complex is one of the most involved in wrist injuries. Its stability depends on primary (SL ligament) and secondary (RSC, DRC, DIC, STT ligaments) stabilizers.The gold standard for carpal ligament assessment is still diagnostic arthroscopy for many hand surgeons. To avoid surgery as a diagnostic procedure, advanced MRI is needed to detect associated lesions (sprains, midsubstance tears, avulsions and chronic fibrous infiltrations) of the extrinsic, midcarpal and intrinsic wrist ligaments, which are demonstrated in this article using 3D and two-dimensional sequences with thin slices (0.4 and 2 mm thick, respectively).

Behavioral context shapes vocal sequences in two anuran species with different repertoire sizes

Acoustic signals in animals serve to convey context-dependent information to receivers. Birds and mammals combine diverse sounds into complex sequences to communicate, but these sequences largely remain understudied in other taxa. Anuran vocalizations are a prominent feature of their life history, and function in defense of territories and to attract mates. However, despite the spectacular diversity of anurans in tropical regions of the world, vocal diversity and communication strategies remain relatively poorly studied. Specifically, studies of vocal sequences and context-dependent vocal patterns in frogs remain few. Here, we investigated the context-dependent vocal repertoire and the use of vocal sequences by two anuran species belonging to different lineages, both endemic to the hyper-diverse Western Ghats of India. By recording vocal sequences both when frogs were alone and in the presence of a territorial rival, we present evidence that both species modify their vocal repertoire according to context. Specifically, one species appends notes to generate more complex sequences, whereas the other shifts to different note types, resulting in different sequences for different contexts. Thus, despite differences in repertoire size, both frog species are capable of adjusting the temporal sequence of vocalizations to communicate in different contexts. This study highlights the need for further studies of insular frogs, to understand how diversification across these continental islands has influenced the evolution of vocal repertoires, vocal sequence patterns and communication systems.Lay SummaryAnimals employ complex sequences of acoustic signals to communicate in diverse behavioral contexts. Here, we demonstrate that two frog species with different vocal repertoires both modify the sequence of note emissions in the presence of a territorial rival. These patterns demonstrate that anurans are capable of complex shifts in the patterns of their vocalization, to communicate different messages to different receivers. Our findings demonstrate the value of studying behavioral diversity in tropical regions.

Mismatch Negativity and Stimulus-Preceding Negativity in Paradigms of Increasing Auditory Complexity: A Possible Role in Predictive Coding

The auditory mismatch negativity (MMN) has been considered a preattentive index of auditory processing and/or a signature of prediction error computation. This study tries to demonstrate the presence of an MMN to deviant trials included in complex auditory stimuli sequences, and its possible relationship to predictive coding. Additionally, the transfer of information between trials is expected to be represented by stimulus-preceding negativity (SPN), which would possibly fit the predictive coding framework. To accomplish these objectives, the EEG of 31 subjects was recorded during an auditory paradigm in which trials composed of stimulus sequences with increasing or decreasing frequencies were intermingled with deviant trials presenting an unexpected ending. Our results showed the presence of an MMN in response to deviant trials. An SPN appeared during the intertrial interval and its amplitude was reduced in response to deviant trials. The presence of an MMN in complex sequences of sounds and the generation of an SPN component, with different amplitudes in deviant and standard trials, would support the predictive coding framework.