What can astrocytes compute?

I demonstrate theoretically that calcium waves in astrocytes can compute anything neurons can. A foundational result in neural computation was proving the firing rate model of neurons defines a universal function approximator. In this work I show a similar proof extends to a model of calcium waves in astrocytes, which I confirm in a series of computer simulations. I argue the major limit in astrocyte computation is not their ability to find approximate solutions, but their computational complexity. I suggest some initial experiments that might be used to confirm these predictions.

State-Aware Value Function Approximation with Attention Mechanism for Restless Multi-armed Bandits

The restless multi-armed bandit (RMAB) problem is a generalization of the multi-armed bandit with non-stationary rewards. Its optimal solution is intractable due to exponentially large state and action spaces with respect to the number of arms. Existing approximation approaches, e.g., Whittle's index policy, have difficulty in capturing either temporal or spatial factors such as impacts from other arms. We propose considering both factors using the attention mechanism, which has achieved great success in deep learning. Our state-aware value function approximation solution comprises an attention-based value function approximator and a Bellman equation solver. The attention-based coordination module capture both spatial and temporal factors for arm coordination. The Bellman equation solver utilizes the decoupling structure of RMABs to acquire solutions with significantly reduced computation overheads. In particular, the time complexity of our approximation is linear in the number of arms. Finally, we illustrate the effectiveness and investigate the properties of our proposed method with numerical experiments.

On the Expressibility and Overfitting of Quantum Circuit Learning

Applying quantum processors to model a high-dimensional function approximator is a typical method in quantum machine learning with potential advantage. It is conjectured that the unitarity of quantum circuits provides possible regularization to avoid overfitting. However, it is not clear how the regularization interplays with the expressibility under the limitation of current Noisy-Intermediate Scale Quantum devices. In this article, we perform simulations and theoretical analysis of the quantum circuit learning problem with hardware-efficient ansatz. Thorough numerical simulations show that the expressibility and generalization error scaling of the ansatz saturate when the circuit depth increases, implying the automatic regularization to avoid the overfitting issue in the quantum circuit learning scenario. This observation is supported by the theory on PAC learnability, which proves that VC dimension is upper bounded due to the locality and unitarity of the hardware-efficient ansatz. Our study provides supporting evidence for automatic regularization by unitarity to suppress overfitting and guidelines for possible performance improvement under hardware constraints.

DeepIndices: Remote Sensing Indices Based on Approximation of Functions through Deep-Learning, Application to Uncalibrated Vegetation Images

The form of a remote sensing index is generally empirically defined, whether by choosing specific reflectance bands, equation forms or its coefficients. These spectral indices are used as preprocessing stage before object detection/classification. But no study seems to search for the best form through function approximation in order to optimize the classification and/or segmentation. The objective of this study is to develop a method to find the optimal index, using a statistical approach by gradient descent on different forms of generic equations. From six wavebands images, five equations have been tested, namely: linear, linear ratio, polynomial, universal function approximator and dense morphological. Few techniques in signal processing and image analysis are also deployed within a deep-learning framework. Performances of standard indices and DeepIndices were evaluated using two metrics, the dice (similar to f1-score) and the mean intersection over union (mIoU) scores. The study focuses on a specific multispectral camera used in near-field acquisition of soil and vegetation surfaces. These DeepIndices are built and compared to 89 common vegetation indices using the same vegetation dataset and metrics. As an illustration the most used index for vegetation, NDVI (Normalized Difference Vegetation Indices) offers a mIoU score of 63.98% whereas our best models gives an analytic solution to reconstruct an index with a mIoU of 82.19%. This difference is significant enough to improve the segmentation and robustness of the index from various external factors, as well as the shape of detected elements.

Adaptive Asymptotic Tracking Control for a Class of Uncertain Input-Delayed Systems with Periodic Time-Varying Disturbances

In this paper, the problem of adaptive asymptotic tracking control for a class of uncertain systems with periodic time-varying disturbances and input delay is studied. By combining Fourier series expansion (FSE) with radial basis function neural network (RBFNN), a hybrid function approximator is used to learn the functions with periodic time-varying disturbances. At the same time, the dynamic surface control technique with a nonlinear filter is used to avoid the “complexity explosion” problem in the process of traditional backstepping technology. Ultimately, all closed-loop signals are guaranteed to be semiglobally uniformly bounded, and the given reference signal can be asymptotically tracked by the output signals of system. A simulation example is given to verify the effectiveness of the proposed control scheme.

Chaos and complexity from quantum neural network. A study with diffusion metric in machine learning

In this work, our prime objective is to study the phenomena of quantum chaos and complexity in the machine learning dynamics of Quantum Neural Network (QNN). A Parameterized Quantum Circuits (PQCs) in the hybrid quantum-classical framework is introduced as a universal function approximator to perform optimization with Stochastic Gradient Descent (SGD). We employ a statistical and differential geometric approach to study the learning theory of QNN. The evolution of parametrized unitary operators is correlated with the trajectory of parameters in the Diffusion metric. We establish the parametrized version of Quantum Complexity and Quantum Chaos in terms of physically relevant quantities, which are not only essential in determining the stability, but also essential in providing a very significant lower bound to the generalization capability of QNN. We explicitly prove that when the system executes limit cycles or oscillations in the phase space, the generalization capability of QNN is maximized. Finally, we have determined the generalization capability bound on the variance of parameters of the QNN in a steady state condition using Cauchy Schwartz Inequality.

Learning to Approximate Functions Using Nb-Doped SrTiO3 Memristors

Memristors have attracted interest as neuromorphic computation elements because they show promise in enabling efficient hardware implementations of artificial neurons and synapses. We performed measurements on interface-type memristors to validate their use in neuromorphic hardware. Specifically, we utilized Nb-doped SrTiO3 memristors as synapses in a simulated neural network by arranging them into differential synaptic pairs, with the weight of the connection given by the difference in normalized conductance values between the two paired memristors. This network learned to represent functions through a training process based on a novel supervised learning algorithm, during which discrete voltage pulses were applied to one of the two memristors in each pair. To simulate the fact that both the initial state of the physical memristive devices and the impact of each voltage pulse are unknown we injected noise into the simulation. Nevertheless, discrete updates based on local knowledge were shown to result in robust learning performance. Using this class of memristive devices as the synaptic weight element in a spiking neural network yields, to our knowledge, one of the first models of this kind, capable of learning to be a universal function approximator, and strongly suggests the suitability of these memristors for usage in future computing platforms.

Learning Feedforward Control Using Multiagent Control Approach for Motion Control Systems

Multiagent control system (MACS) has become a promising solution for solving complex control problems. Using the advantages of MACS-based design approaches, a novel solution for advanced control of mechatronic systems has been developed in this paper. The study has aimed at integrating learning control into MACS. Specifically, learning feedforward control (LFFC) is implemented as a pattern for incorporation in MACS. The major novelty of this work is that the feedback control part is realized in a real-time periodic MACS, while the LFFC algorithm is done on-line, asynchronously, and in a separate non-real-time aperiodic MACS. As a result, a MACS-based LFFC design method has been developed. A second-order B-spline neural network (BSN) is used as a function approximator for LFFC whose input-output mapping can be adapted during control and is intended to become equal to the inverse model of the plant. To provide real-time features for the MACS-based LFFC system, the open robot control software (OROCOS) has been employed as development and runtime environment. A case study using a simulated linear motor in the presence of nonlinear cogging and friction force as well as mass variations is used to illustrate the proposed method. A MACS-based LFFC system has been designed and implemented for the simulated plant. The system consists of a setpoint generator, a feedback controller, and a time-index LFFC that can learn on-line. Simulation results have demonstrated the applicability of the design method.

Reinforcement and Imitation Learning Applied to Autonomous Aerial Robot Control

In robotics, the ultimate goal of reinforcement learning is to endow robots with the ability to learn, improve, adapt, and reproduce tasks with dynamically changing constraints based on exploration and autonomous learning. Reinforcement Learning (RL) aims at addressing this problem by enabling a robot to learn behaviors through trial-and-error. With RL, a Neural Network can be trained as a function approximator to directly map states to actuator commands making any predefined control structure not-needed for training. However, the knowledge required to converge these methods is usually built from scratch. Learning may take a long time, not to mention that RL algorithms need a stated reward function. Sometimes, it is not trivial to define one. Often it is easier for a teacher, human or intelligent agent, do demonstrate the desired behavior or how to accomplish a given task. Humans and other animals have a natural ability to learn skills from observation, often from merely seeing these skills’ effects: without direct knowledge of the underlying actions. The same principle exists in Imitation Learning, a practical approach for autonomous systems to acquire control policies when an explicit reward function is unavailable, using supervision provided as demonstrations from an expert, typically a human operator. In this scenario, this work’s primary objective is to design an agent that can successfully imitate a prior acquired control policy using Imitation Learning. The chosen algorithm is GAIL since we consider that it is the proper algorithm to tackle this problem by utilizing expert (state, action) trajectories. As reference expert trajectories, we implement state-of-the-art on and off-policy methods PPO and SAC. Results show that the learned policies for all three methods can solve the task of low-level control of a quadrotor and that all can account for generalization on the original tasks.