Stochastic Computing Emulation of Memristor Cellular Nonlinear Networks

Cellular Nonlinear Networks (CNN) are a concept introduced in 1988 by Leon Chua and Lin Yang as a bio-inspired architecture capable of massively parallel computation. Since then, CNN have been enhanced by incorporating designs that incorporate memristors to profit from their processing and memory capabilities. In addition, Stochastic Computing (SC) can be used to optimize the quantity of required processing elements; thus it provides a lightweight approximate computing framework, quite accurate and effective, however. In this work, we propose utilization of SC in designing and implementing a memristor-based CNN. As a proof of the proposed concept, an example of application is presented. This application combines Matlab and a FPGA in order to create the CNN. The implemented CNN was then used to perform three different real-time applications on a 512 × 512 gray-scale and a 768 × 512 color image: storage of the image, edge detection, and image sharpening. It has to be pointed out that the same CNN was used for the three different tasks, with the sole change of some programmable parameters. Results show an excellent capability with significant accompanying advantages, such as the low number of needed elements further allowing for a low cost FPGA-based system implementation, something confirming the system’s capacity for real time operation.

Compact Itinerant Microwave Photonics with Superconducting High-Kinetic Inductance Microstrips

Microwave photonics is a remarkably powerful system for quantum simulation and technologies, but its integration in superconducting circuits, superior in many aspects, is constrained by the long wavelengths and impedance mismatches in this platform. We introduce a solution to these difficulties via compact networks of high-kinetic inductance microstrip waveguides and coupling wires with strongly reduced phase velocities. We demonstrate broadband capabilities for superconducting microwave photonics in terms of routing, emulation and generalized linear and nonlinear networks.

Stochastic Computing Emulation of Memristor Cellular Nonlinear Networks

Cellular Nonlinear Networks (CNN) are a concept introduced in 1988 by Leon Chua and Lin Yang as a bio-inspired architecture, capable of massively parallel computation. Later on, CNN have been enhanced by incorporating designs that incorporate memristors to profit from their processing and memory capabilities. In addition, Stochastic Computing (SC) can be used to optimize the quantity of required processing elements; thus it provides a lightweight approximate computing framework, quite accurate and effective, though. In this work, we propose utilization of SC in designing and implementing a memristor-based CNN. As a proof of the proposed concept, an example of application is presented. This application combines Matlab and a FPGA in order to create the CNN. The implemented CNN has then been used to perform three different real-time applications on a 512x512 gray-scale and a 768x512 color image: storage of the image, edge detection, and image sharpening. It has to be pointed out that the same CNN has been used for the three different tasks, with the sole change of some programmable parameters. Results show an excellent capability with significant accompanying advantages, like the low number of needed elements further allowing for a low cost FPGA-based system implementation, something confirming the system’s ability for real time operation.

On the Neural Tangent Kernel of Deep Networks with Orthogonal Initialization

The prevailing thinking is that orthogonal weights are crucial to enforcing dynamical isometry and speeding up training. The increase in learning speed that results from orthogonal initialization in linear networks has been well-proven. However, while the same is believed to also hold for nonlinear networks when the dynamical isometry condition is satisfied, the training dynamics behind this contention have not been thoroughly explored. In this work, we study the dynamics of ultra-wide networks across a range of architectures, including Fully Connected Networks (FCNs) and Convolutional Neural Networks (CNNs) with orthogonal initialization via neural tangent kernel (NTK). Through a series of propositions and lemmas, we prove that two NTKs, one corresponding to Gaussian weights and one to orthogonal weights, are equal when the network width is infinite. Further, during training, the NTK of an orthogonally-initialized infinite-width network should theoretically remain constant. This suggests that the orthogonal initialization cannot speed up training in the NTK (lazy training) regime, contrary to the prevailing thoughts. In order to explore under what circumstances can orthogonality accelerate training, we conduct a thorough empirical investigation outside the NTK regime. We find that when the hyper-parameters are set to achieve a linear regime in nonlinear activation, orthogonal initialization can improve the learning speed with a large learning rate or large depth.

Risk spillover networks in financial system based on information theory

Since the financial system has illustrated an increasingly prominent characteristic of inextricable connections, information theory is gradually utilized to study the financial system. By collecting the daily data of industry index (2005-2020) and region index (2012-2020) listed in China as samples, this paper applies an innovative measure named partial mutual information on mixed embedding to generate directed networks. Based on the analysis of nonlinear relationships among sectors, this paper realizes the accurate construction of “time-varying” financial network from the perspective of risk spillover. The results are presented as follow: (1) interactions can be better understood through the nonlinear networks among distinct sectors, and sectors in the networks could be classified into different types according to their topological properties connected to risk spillover; (2) in the rising stage, information is transmitted rapidly in the network, so the risk is fast diffused and absorbed; (3) in the declining stage, the network topology is more complex and panic sentiments have long term impact leading to more connections; (4) The US market, Japan market and Hongkong market have significant affect on China’s market. The results suggest that this nonlinear measure is an effective approach to develop financial networks and explore the mechanism of risk spillover.

System-Theoretic Methods for Designing Bio-Inspired Mem-Computing Memristor Cellular Nonlinear Networks

The introduction of nano-memristors in electronics may allow to boost the performance of integrated circuits beyond the Moore era, especially in view of their extraordinary capability to process and store data in the very same physical volume. However, recurring to nonlinear system theory is absolutely necessary for the development of a systematic approach to memristive circuit design. In fact, the application of linear system-theoretic techniques is not suitable to explore thoroughly the rich dynamics of resistance switching memories, and designing circuits without a comprehensive picture of the nonlinear behaviour of these devices may lead to the realization of technical systems failing to operate as desired. Converting traditional circuits to memristive equivalents may require the adaptation of classical methods from nonlinear system theory. This paper extends the theory of time- and space-invariant standard cellular nonlinear networks with first-order processing elements for the case where a single non-volatile memristor is inserted in parallel to the capacitor in each cell. A novel nonlinear system-theoretic method allows to draw a comprehensive picture of the dynamical phenomena emerging in the memristive mem-computing array, beautifully illustrated in the so-called Primary Mosaic for the class of uncoupled memristor cellular nonlinear networks. Employing this new analysis tool it is possible to elucidate, with the support of illustrative examples, how to design variability-tolerant bio-inspired cellular nonlinear networks with second-order memristive cells for the execution of computing tasks or of memory operations. The capability of the class of memristor cellular nonlinear networks under focus to store and process information locally, without the need to insert additional memory units in each cell, may allow to increase considerably the spatial resolution of state-of-the-art purely CMOS sensor-processor arrays. This is of great appeal for edge computing applications, especially since the Internet-of-Things industry is currently calling for the realization of miniaturized, lightweight, low-power, and high-speed mem-computers with sensing capability on board.