Optimal preconditioned regularization of least mean squares algorithm for robust online learning1

Despite its low computational cost, and steady state behavior, some well known drawbacks of the least means squares (LMS) algorithm are: slow rate of convergence and unstable behaviour for ill conditioned autocorrelation matrices of input signals. Several modified algorithms have been presented with better convergence speed, however most of these algorithms are expensive in terms of computational cost and time, and sometimes deviate from optimal Wiener solution that results in a biased solution of online estimation problem. In this paper, the inverse Cholesky factor of the input autocorrelation matrix is optimized to pre-whiten input signals and improve the robustness of the LMS algorithm. Furthermore, in order to have an unbiased solution, mean squares deviation (MSD) is minimized by improving convergence in misalignment. This is done by regularizing step-size adaptively in each iteration that helps in developing a highly efficient optimal preconditioned regularized LMS (OPRLMS) algorithm with adaptive step-size. Comparison of OPRLMS algorithm with other LMS based algorithms is given for unknown system identification and noise cancelation from ECG signal, that results in preference of the proposed algorithm over the other variants of LMS algorithm.

A Recursive Least-Squares Algorithm for the Identification of Trilinear Forms

High-dimensional system identification problems can be efficiently addressed based on tensor decompositions and modelling. In this paper, we design a recursive least-squares (RLS) algorithm tailored for the identification of trilinear forms, namely RLS-TF. In our framework, the trilinear form is related to the decomposition of a third-order tensor (of rank one). The proposed RLS-TF algorithm acts on the individual components of the global impulse response, thus being efficient in terms of both performance and complexity. Simulation results indicate that the proposed solution outperforms the conventional RLS algorithm (which handles only the global impulse response), but also the previously developed trilinear counterparts based on the least-mean- squares algorithm.

Forming a Hierarchical Choquet Integral with a GA-Based Heuristic Least Square Method

: Identifying the fuzzy measures of the Choquet integral model is an important component in resolving complicated multi-criteria decision-making (MCDM) problems. Previous papers solved the above problem by using various mathematical programming models and regression-based methods. However, when considering complicated MCDM problems (e.g., 10 criteria), the presence of too many parameters might result in unavailable or inconsistent solutions. While k-additive or p-symmetric measures are provided to reduce the number of fuzzy measures, they cannot prevent the problem of identifying the fuzzy measures in a high-dimension situation. Therefore, Sugeno and his colleagues proposed a hierarchical Choquet integral model to overcome the problem, but it required the partition information of the criteria, which usually cannot be obtained in practice. In this paper, we proposed a GA-based heuristic least mean-squares algorithm (HLMS) to construct the hierarchical Choquet integral and overcame the above problems. The genetic algorithm (GA) was used here to determine the input variables of the sub-Choquet integrals automatically, according to the objective of the mean square error (MSE), and calculated the fuzzy measures with the HLMS. Then, we summed these sub-Choquet integrals into the final Choquet integral for the purpose of regression or classification. In addition, we tested our method with four datasets and compared these results with the conventional Choquet integral, logit model, and neural network. On the basis of the results, the proposed model was competitive with respect to other models.