BIOLOGICALLY INSPIRED FILTERS UTILIZING SPECTRAL PROPERTIES OF TOEPLITZ-BLOCK-TOEPLITZ MATRICES

The construction of filters arising from linear neural networks with feed-backward excitatory-inhibitory connections is presented. Spatially invariant coupling between neurons and the distribution of neuron-receptor units in the form of a uniform square grid yield the TBT (Toeplitz-Block-Toeplitz) connection matrix. Utilizing the relationship between spectral properties of such matrices and their generating functions, the method for construction of recurrent linear networks is addressed. By appropriately bounding the generating function, the connection matrix eigenvalues are kept in the desired range allowing for large matrix inverse to be approximated by a convergent power series. Instead of matrix inversion, the single pass convolution with the filter obtained from the network connection weights is applied when solving the network. For the case of inter-neuron coupling in the form of a function that is expandable in a Fourier series in polar angle, the network response filter is shown to be steerable.

Download Full-text

Inversion of Tridiagonal Matrices Using the Dunford-Taylor’s Integral

Symmetry ◽

10.3390/sym13050870 ◽

2021 ◽

Vol 13 (5) ◽

pp. 870

Author(s):

Diego Caratelli ◽

Paolo Emilio Ricci

Keyword(s):

Functional Analysis ◽

Computer Algebra ◽

Toeplitz Matrices ◽

Test Cases ◽

Recursive Procedure ◽

Tridiagonal Matrices ◽

Special Cases ◽

The Matrix ◽

Matrix Eigenvalues ◽

Complex Valued

We show that using Dunford-Taylor’s integral, a classical tool of functional analysis, it is possible to derive an expression for the inverse of a general non-singular complex-valued tridiagonal matrix. The special cases of Jacobi’s symmetric and Toeplitz (in particular symmetric Toeplitz) matrices are included. The proposed method does not require the knowledge of the matrix eigenvalues and relies only on the relevant invariants which are determined, in a computationally effective way, by means of a dedicated recursive procedure. The considered technique has been validated through several test cases with the aid of the computer algebra program Mathematica©.

Download Full-text

Algebraic and Spectral Properties of General Toeplitz Matrices

SIAM Journal on Control and Optimization ◽

10.1137/s0363012900377183 ◽

2002 ◽

Vol 41 (5) ◽

pp. 1413-1439 ◽

Cited By ~ 8

Author(s):

A. Bultheel ◽

P. Carrette

Keyword(s):

Spectral Properties ◽

Toeplitz Matrices

Download Full-text

Echo State Property Linked to an Input: Exploring a Fundamental Characteristic of Recurrent Neural Networks

Neural Computation ◽

10.1162/neco_a_00411 ◽

2013 ◽

Vol 25 (3) ◽

pp. 671-696 ◽

Cited By ~ 69

Author(s):

G. Manjunath ◽

H. Jaeger

Keyword(s):

Neural Networks ◽

Recurrent Neural Networks ◽

Statistical Characteristics ◽

Connection Matrix ◽

Reservoir Computing ◽

Mathematical Methods ◽

Network Response ◽

Network Connection ◽

Memory Content ◽

Definition Of

The echo state property is a key for the design and training of recurrent neural networks within the paradigm of reservoir computing. In intuitive terms, this is a passivity condition: a network having this property, when driven by an input signal, will become entrained by the input and develop an internal response signal. This excited internal dynamics can be seen as a high-dimensional, nonlinear, unique transform of the input with a rich memory content. This view has implications for understanding neural dynamics beyond the field of reservoir computing. Available definitions and theorems concerning the echo state property, however, are of little practical use because they do not relate the network response to temporal or statistical properties of the driving input. Here we present a new definition of the echo state property that directly connects it to such properties. We derive a fundamental 0-1 law: if the input comes from an ergodic source, the network response has the echo state property with probability one or zero, independent of the given network. Furthermore, we give a sufficient condition for the echo state property that connects statistical characteristics of the input to algebraic properties of the network connection matrix. The mathematical methods that we employ are freshly imported from the young field of nonautonomous dynamical systems theory. Since these methods are not yet well known in neural computation research, we introduce them in some detail. As a side story, we hope to demonstrate the eminent usefulness of these methods.

Download Full-text

CONVERGENCE OF A CLASS OF TOEPLITZ TYPE MATRICES

Random Matrices Theory and Application ◽

10.1142/s2010326313500068 ◽

2013 ◽

Vol 02 (03) ◽

pp. 1350006

Author(s):

ARUP BOSE ◽

SREELA GANGOPADHYAY ◽

KOUSHIK SAHA

Keyword(s):

Limit Theorem ◽

Method Of Moments ◽

Spectral Properties ◽

Toeplitz Matrices

We use the method of moments to study the spectral properties in the bulk for finite diagonal large dimensional random and non-random Toeplitz type matrices via the joint convergence of matrices in an appropriate sense. As a consequence we revisit the famous limit theorem of Szegö for non-random symmetric Toeplitz matrices.

Download Full-text