A Function Estimation Approach to Sequential Learning with Neural Networks

1993 ◽  
Vol 5 (6) ◽  
pp. 954-975 ◽  
Author(s):  
Visakan Kadirkamanathan ◽  
Mahesan Niranjan
Keyword(s):  
Function Space ◽  
Time Series Prediction ◽  
Approximation Error ◽  
Lms Algorithm ◽  
Sequential Learning ◽  
Superior Performance ◽  
Function Estimation ◽  
Minimal Network ◽  
Growth Criterion ◽  
Space Approach

In this paper, we investigate the problem of optimal sequential learning, viewed as a problem of estimating an underlying function sequentially rather than estimating a set of parameters of the neural network. First, we arrive at a suboptimal solution to the sequential estimate that can be mapped by a growing gaussian radial basis function (GaRBF) network. This network adds hidden units for each observation. The function space approach in which the estimates are represented as vectors in a function space is used in developing a growth criterion to limit its growth. A simplification of the criterion leads to two joint criteria on the distance of the present pattern and the existing unit centers in the input space and on the approximation error of the network for the given observation to be satisfied together. This network is similar to the resource allocating network (RAN) (Platt 1991a) and hence RAN can be interpreted from a function space approach to sequential learning. Second, we present an enhancement to the RAN. The RAN either allocates a new unit based on the novelty of an observation or adapts the network parameters by the LMS algorithm. The function space interpretation of the RAN lends itself to an enhancement of the RAN in which the extended Kalman filter (EKF) algorithm is used in place of the LMS algorithm. The performance of the RAN and the enhanced network are compared in the experimental tasks of function approximation and time-series prediction demonstrating the superior performance of the enhanced network with fewer number of hidden units. The approach adopted here has led us toward the minimal network required for a sequential learning problem.

Pruning with Replacement on Limited Resource Allocating Networks by F-Projections

1996 ◽  
Vol 8 (4) ◽  
pp. 855-868 ◽  
Author(s):  
Christophe Molina ◽  
Mahesan Niranjan
Keyword(s):  
Ad Hoc ◽  
Theoretical Foundation ◽  
Time Series Prediction ◽  
Limited Resource ◽  
Radial Basis Function Networks ◽  
Function Estimation ◽  
Santa Fe ◽  
Prediction Problem ◽  
Growth Criterion ◽  
Geometric Growth

The principle of F-projection, in sequential function estimation, provides a theoretical foundation for a class of gaussian radial basis function networks known as the resource allocating networks (RAN). The ad hoc rules for adaptively changing the size of RAN architectures can be justified from a geometric growth criterion defined in the function space. In this paper, we show that the same arguments can be used to arrive at a pruning with replacement rule for RAN architectures with a limited number of units. We illustrate the algorithm on the laser time series prediction problem of the Santa Fe competition and show that results similar to those of the winners of the competition can be obtained with pruning and replacement.

A Function Space Approach to Spectral Related Problems

2019 ◽  
Vol 10 (6) ◽  
pp. 1220-1222
Author(s):  
T. Venkatesh ◽  
Karuna Samaje
Keyword(s):  
Function Space ◽  
Space Approach

A numerical method for the approximation of reachable sets of linear control systems

2017 ◽  
Vol 36 (2) ◽  
pp. 423-441 ◽  
Author(s):  
Lizhen Shao ◽  
Fangyuan Zhao ◽  
Guangda Hu
Keyword(s):  
Numerical Method ◽  
Control Systems ◽  
Optimization Problems ◽  
Approximation Error ◽  
Outer Approximation ◽  
Reachable Sets ◽  
Linear Control ◽  
Superior Performance ◽  
Linear Control Systems ◽  
Inner Approximation

Abstract In this article, a numerical method for the approximation of reachable sets of linear control systems is discussed. First a continuous system is transformed into a discrete one with Runge–Kutta methods. Then based on Benson’s outer approximation algorithm for solving multiobjective optimization problems, we propose a variant of Benson’s algorithm to sandwich the reachable set of the discrete system with an inner approximation and an outer approximation. By specifying an approximation error, the quality of the approximations measured in Hausdorff distance can be directly controlled. Furthermore, we use an illustrative example to demonstrate the working of the algorithm. Finally, computational experiments illustrate the superior performance of our proposed algorithm compared to a recent algorithm in the literature.

Feature Extraction and Representation for Distributed Multi-View Human Action Recognition

2013 ◽  
Vol 3 (2) ◽  
pp. 145-154 ◽  
Author(s):  
Jiajia Luo ◽  
Wei Wang ◽  
Hairong Qi
Keyword(s):  
Action Recognition ◽  
Approximation Error ◽  
Human Action Recognition ◽  
Human Action ◽  
Base Station ◽  
Feature Representation ◽  
Superior Performance ◽  
Feature Descriptor ◽  
Testing Stage ◽  
New Feature

Multi-view human action recognition has gained a lot of attention in recent years for its superior performance as compared to single view recognition. In this paper, we propose a new framework for the real-time realization of human action recognition in distributed camera networks (DCNs). We first present a new feature descriptor (Mltp-hist) that is tolerant to illumination change, robust in homogeneous region and computationally efficient. Taking advantage of the proposed Mltp-hist, the noninformative 3-D patches generated from the background can be further removed automatically that effectively highlights the foreground patches. Next, a new feature representation method based on sparse coding is presented to generate the histogram representation of local videos to be transmitted to the base station for classification. Due to the sparse representation of extracted features, the approximation error is reduced. Finally, at the base station, a probability model is produced to fuse the information from various views and a class label is assigned accordingly. Compared to the existing algorithms, the proposed framework has three advantages while having less requirements on memory and bandwidth consumption: 1) no preprocessing is required; 2) communication among cameras is unnecessary; and 3) positions and orientations of cameras do not need to be fixed. We further evaluate the proposed framework on the most popular multi-view action dataset IXMAS. Experimental results indicate that our proposed framework repeatedly achieves state-of-the-art results when various numbers of views are tested. In addition, our approach is tolerant to the various combination of views and benefit from introducing more views at the testing stage. Especially, our results are still satisfactory even when large misalignment exists between the training and testing samples.

Time series prediction model for sequential learning

2007 ◽  
Vol 90 (12) ◽  
pp. 129-139
Author(s):  
Manabu Gouko ◽  
Yoshihiro Sugaya ◽  
Hirotomo Aso
Keyword(s):  
Time Series ◽  
Prediction Model ◽  
Time Series Prediction ◽  
Sequential Learning

scholarly journals Adaptive Parameter Estimation of IIR System-Based WSN Using Multihop Diffusion in Distributed Approach

2020 ◽  
Vol 14 (4) ◽  
pp. 30-41
Author(s):  
Meera Dash ◽  
Trilochan Panigrahi ◽  
Renu Sharma ◽  
Mihir Narayan Mohanty
Keyword(s):  
Impulse Response ◽  
Sensor Node ◽  
Finite Impulse Response ◽  
Lms Algorithm ◽  
Sensor System ◽  
Superior Performance ◽  
Estimation Accuracy ◽  
Infinite Impulse Response ◽  
Communication Overhead ◽  
Least Mean Square

Distributed estimation of parameters in wireless sensor networks is taken into consideration to reduce the communication overhead of the network which makes the sensor system energy efficient. Most of the distributed approaches in literature, the sensor system is modeled with finite impulse response as it is inherently stable. Whereas in real time applications of WSN like target tracking, fast rerouting requires, infinite impulse response system (IIR) is used to model and that has been chosen in this work. It is assumed that every sensor node is equipped with IIR adaptive system. The diffusion least mean square (DLMS) algorithm is used to estimate the parameters of the IIR system where each node in the network cooperates themselves. In a sparse WSN, the performance of a DLMS algorithm reduces as the degree of the node decreases. In order to increase the estimation accuracy with a smaller number of iterations, the sensor node needs to share their information with more neighbors. This is feasible by communicating each node with multi-hop nodes instead of one-hop only. Therefore the parameters of an IIR system is estimated in distributed sparse sensor network using multihop diffusion LMS algorithm. The simulation results exhibit superior performance of the multihop diffusion LMS over non-cooperative and conventional diffusion algorithms.

Sign in / Sign up

Export Citation Format

Share Document