Evaluating the Effects of Size and Precision of Training Data on ANN Training Performance for the Prediction of Chaotic Time Series Patterns

In this research, artificial neural networks (ANN) with various architectures are trained to generate the chaotic time series patterns of the Lorenz attractor. The ANN training performance is evaluated based on the size and precision of the training data. The nonlinear Auto-Regressive (NAR) model is trained in open loop mode first. The trained model is then used with closed loop feedback to predict the chaotic time series outputs. The research goal is to use the designed NAR ANN model for the simulation and analysis of Electroencephalogram (EEG) signals in order to study brain activities. A simple ANN topology with a single hidden layer of 3 to 16 neurons and 1 to 4 input delays is used. The training performance is measured by averaged mean square error. It is found that the training performance cannot be improved by solely increasing the training data size. However, the training performance can be improved by increasing the precision of the training data. This provides useful knowledge towards reducing the number of EEG data samples and corresponding acquisition time for prediction.

Predictive Error Compensating Wavelet Neural Network Model for Multivariable Time Series Prediction

Multivariable machine learning (ML) models are increasingly used for time series predictions. However, avoiding the overfitting and underfitting in ML-based time series prediction requires special consideration depending on the size and characteristics of the available training dataset. Predictive error compensating wavelet neural network (PEC-WNN) improves the time series prediction accuracy by enhancing the orthogonal features within a data fusion scheme. In this study, time series prediction performance of the PEC-WNNs have been evaluated on two different problems in comparison to conventional machine learning methods including the long short-term memory (LSTM) network. The results have shown that PECNET provides significantly more accurate predictions. RMSPE error is reduced by more than 60% with respect to other compared ML methods for Lorenz Attractor and wind speed prediction problems.

A Study On Methods for Determining Phase Space Reconstruction Parameters

Several pairs of algorithms were used to determine the phase space reconstruction parameters to analyze the dynamic characteristics of chaotic time series. The reconstructed phase trajectories were compared with the original phase trajectories of the Lorenz attractor, Rössler attractor, and Chens attractor to obtain the optimum method for determining the phase space reconstruction parameters with high precision and efficiency. The research results show that the false nearest neighbor method and the complex auto-correlation method provided the best results. The saturated embedding dimension method based on the saturated correlation dimension method is proposed to calculate the time delay. Different time delays are obtained by changing the embedding dimension parameters of the complex auto-correlation method. The optimum time delay occurs at the point where the time delay is stable. The validity of the method is verified through combing the application of correlation dimension, showing that the proposed method is suitable for the effective determination of the phase space reconstruction parameters.

Parameter Identification of Lorenz System with Incomplete Information: Case of One Known and Two Unknown Functions

In the present paper, which is the continuation of the previous one, the problem of parameter identification of the Lorenz system is solved in assumption that only one of three functions is known at discrete time instants on finite time initial time interval. Two other functions are assumed to be unknown. The regular methods of guess values determination of the unknown parameters are developed. They are based on the Lagrange multiplier and auxiliary parameters approaches. A novel method of initial value problem solution is proposed in which the abovementioned guess values are used for more accurate estimation of the system parameters. It is demonstrated that the proposed IVP method simultaneously solves three different tasks: the problem of function interpolation from its discrete values on the initial time interval; the problem of unknown functions reconstruction on the same time interval, and the problem of extrapolation of all functions on limited time interval. It is also shown that the proposed method reconstructs the Lorenz attractor from limited data volume and data including random components.

Analysis of chaotic behavior of three-dimensional dynamical systems by a B-spline differential quadrature algorithm

A novel approach based on cubic [Formula: see text]-spline functions has been developed to find solutions of three-dimensional chaotic dynamical systems. Interesting dynamical behavior has been illustrated in figures. We observed that dynamical systems depend very sensitively on the initial condition and corresponding behavior has been captured in numerical simulations. The Butterfly effect is used in the development of weather prediction models and has been depicted graphically. This paper deals with the numerical solutions of Lorenz attractor, Chen, Genesio and a combination of Lorenz and Rossler attractors. The computed solutions are quite accurate, consistent and confirm that the cubic [Formula: see text]-spline differential quadrature is a very efficient method to portray complex dynamical behaviors of dynamical systems.

Cluster-based network modeling—From snapshots to complex dynamical systems

We propose a universal method for data-driven modeling of complex nonlinear dynamics from time-resolved snapshot data without prior knowledge. Complex nonlinear dynamics govern many fields of science and engineering. Data-driven dynamic modeling often assumes a low-dimensional subspace or manifold for the state. We liberate ourselves from this assumption by proposing cluster-based network modeling (CNM) bridging machine learning, network science, and statistical physics. CNM describes short- and long-term behavior and is fully automatable, as it does not rely on application-specific knowledge. CNM is demonstrated for the Lorenz attractor, ECG heartbeat signals, Kolmogorov flow, and a high-dimensional actuated turbulent boundary layer. Even the notoriously difficult modeling benchmark of rare events in the Kolmogorov flow is solved. This automatable universal data-driven representation of complex nonlinear dynamics complements and expands network connectivity science and promises new fast-track avenues to understand, estimate, predict, and control complex systems in all scientific fields.

Parallel inference of hierarchical latent dynamics in two-photon calcium imaging of neuronal populations

Dynamic latent variable modelling has provided a powerful tool for understanding how populations of neurons compute. For spiking data, such latent variable modelling can treat the data as a set of point-processes, due to the fact that spiking dynamics occur on a much faster timescale than the computational dynamics being inferred. In contrast, for other experimental techniques, the slow dynamics governing the observed data are similar in timescale to the computational dynamics that researchers want to infer. An example of this is in calcium imaging data, where calcium dynamics can have timescales on the order of hundreds of milliseconds. As such, the successful application of dynamic latent variable modelling to modalities like calcium imaging data will rest on the ability to disentangle the deeper- and shallower-level dynamical systems’ contributions to the data. To-date, no techniques have been developed to directly achieve this. Here we solve this problem by extending recent advances using sequential variational autoencoders for dynamic latent variable modelling of neural data. Our system VaLPACa (Variational Ladders for Parallel Autoencoding of Calcium imaging data) solves the problem of disentangling deeper- and shallower-level dynamics by incorporating a ladder architecture that can infer a hierarchy of dynamical systems. Using some built-in inductive biases for calcium dynamics, we show that we can disentangle calcium flux from the underlying dynamics of neural computation. First, we demonstrate with synthetic calcium data that we can correctly disentangle an underlying Lorenz attractor from calcium dynamics. Next, we show that we can infer appropriate rotational dynamics in spiking data from macaque motor cortex after it has been converted into calcium fluorescence data via a calcium dynamics model. Finally, we show that our method applied to real calcium imaging data from primary visual cortex in mice allows us to infer latent factors that carry salient sensory information about unexpected stimuli. These results demonstrate that variational ladder autoencoders are a promising approach for inferring hierarchical dynamics in experimental settings where the measured variable has its own slow dynamics, such as calcium imaging data. Our new, open-source tool thereby provides the neuroscience community with the ability to apply dynamic latent variable modelling to a wider array of data modalities.