scholarly journals Biologically-Inspired Pulse Signal Processing for Intelligence at the Edge

2021 ◽  
Vol 4 ◽  
Author(s):  
Kan Li ◽  
José C. Príncipe
Keyword(s):  
Machine Learning ◽  
Energy Efficient ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Pulse Signal ◽  
Learning Task ◽  
Keyword Spotting ◽  
Data Types ◽  
Biologically Inspired ◽  
Neural Network Approach

There is an ever-growing mismatch between the proliferation of data-intensive, power-hungry deep learning solutions in the machine learning (ML) community and the need for agile, portable solutions in resource-constrained devices, particularly for intelligence at the edge. In this paper, we present a fundamentally novel approach that leverages data-driven intelligence with biologically-inspired efficiency. The proposed Sparse Embodiment Neural-Statistical Architecture (SENSA) decomposes the learning task into two distinct phases: a training phase and a hardware embedment phase where prototypes are extracted from the trained network and used to construct fast, sparse embodiment for hardware deployment at the edge. Specifically, we propose the Sparse Pulse Automata via Reproducing Kernel (SPARK) method, which first constructs a learning machine in the form of a dynamical system using energy-efficient spike or pulse trains, commonly used in neuroscience and neuromorphic engineering, then extracts a rule-based solution in the form of automata or lookup tables for rapid deployment in edge computing platforms. We propose to use the theoretically-grounded unifying framework of the Reproducing Kernel Hilbert Space (RKHS) to provide interpretable, nonlinear, and nonparametric solutions, compared to the typical neural network approach. In kernel methods, the explicit representation of the data is of secondary nature, allowing the same algorithm to be used for different data types without altering the learning rules. To showcase SPARK’s capabilities, we carried out the first proof-of-concept demonstration on the task of isolated-word automatic speech recognition (ASR) or keyword spotting, benchmarked on the TI-46 digit corpus. Together, these energy-efficient and resource-conscious techniques will bring advanced machine learning solutions closer to the edge.

scholarly journals Hilbert–Schmidt Independence Criterion Regularization Kernel Framework on Symmetric Positive Definite Manifolds

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Xi Liu ◽  
Zengrong Zhan ◽  
Guo Niu
Keyword(s):  
Machine Learning ◽  
Kernel Methods ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Positive Definite ◽  
Kernel Trick ◽  
Symmetric Positive Definite ◽  
Machine Learning Methods ◽  
Classification Tasks ◽  
Independence Criterion

Image recognition tasks involve an increasingly high amount of symmetric positive definite (SPD) matrices data. SPD manifolds exhibit nonlinear geometry, and Euclidean machine learning methods cannot be directly applied to SPD manifolds. The kernel trick of SPD manifolds is based on the concept of projecting data onto a reproducing kernel Hilbert space. Unfortunately, existing kernel methods do not consider the connection of SPD matrices and linear projections. Thus, a framework that uses the correlation between SPD matrices and projections to model the kernel map is proposed herein. To realize this, this paper formulates a Hilbert–Schmidt independence criterion (HSIC) regularization framework based on the kernel trick, where HSIC is usually used to express the interconnectedness of two datasets. The proposed framework allows us to extend the existing kernel methods to new HSIC regularization kernel methods. Additionally, this paper proposes an algorithm called HSIC regularized graph discriminant analysis (HRGDA) for SPD manifolds based on the HSIC regularization framework. The proposed HSIC regularization framework and HRGDA are highly accurate and valid based on experimental results on several classification tasks.

scholarly journals OUTLIER-ROBUST KERNEL HIERARCHICAL-OPTIMIZATION RLS ON A BUDGET WITH AFFINE CONSTRAINTS

2020 ◽  
Author(s):  
Konstantinos Slavakis ◽  
Masahiro Yukawa
Keyword(s):  
Reproducing Kernel ◽  
Side Information ◽  
Reproducing Kernel Hilbert Space ◽  
Learning Task ◽  
Hierarchical Optimization ◽  
Quadratic Loss ◽  
Infinite Dimensional ◽  
Learning Framework ◽  
Affine Constraints ◽  
Personal Use

<div>This paper introduces a non-parametric learning framework to combat outliers in online, multi-output, and nonlinear regression tasks. A hierarchical-optimization problem underpins the learning task: Search in a reproducing kernel Hilbert space (RKHS) for a function that minimizes a sample average $\ell_p$-norm ($1 \leq p \leq 2$) error loss on data contaminated by noise and outliers, subject to side information that takes the form of affine constraints defined as the set of minimizers of a quadratic loss on a finite number of faithful data devoid of noise and outliers. To surmount the computational obstacles inflicted by the choice of loss and the potentially infinite dimensional RKHS, approximations of the $\ell_p$-norm loss, as well as a novel twist of the criterion of approximate linear dependency are devised to keep the computational-complexity footprint of the proposed algorithm bounded over time. Numerical tests on datasets showcase the robust behavior of the advocated framework against different types of outliers, under a low computational load, while satisfying at the same time the affine constraints, in contrast to the state-of-the-art methods which are constraint agnostic.</div><div><br></div><div>-------</div><div><br></div><div>© 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.<br></div>

On extension theorems and their connection to universal consistency in machine learning

2016 ◽  
Vol 14 (06) ◽  
pp. 795-808 ◽  
Author(s):  
Andreas Christmann ◽  
Florian Dumpert ◽  
Dao-Hong Xiang
Keyword(s):  
Machine Learning ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Decision Function ◽  
Theoretical Explanation ◽  
Statistical Machine Learning ◽  
Learning Methods ◽  
Data Set ◽  
Extension Theorems ◽  
Universal Consistency

Statistical machine learning plays an important role in modern statistics and computer science. One main goal of statistical machine learning is to provide universally consistent algorithms, i.e. the estimator converges in probability or in some stronger sense to the Bayes risk or to the Bayes decision function. Kernel methods based on minimizing the regularized risk over a reproducing kernel Hilbert space (RKHS) belong to these statistical machine learning methods. It is in general unknown which kernel yields optimal results for a particular data set or for the unknown probability measure. Hence various kernel learning methods were proposed to choose the kernel and therefore also its RKHS in a data adaptive manner. Nevertheless, many practitioners often use the classical Gaussian RBF kernel or certain Sobolev kernels with good success. The goal of this paper is to offer one possible theoretical explanation for this empirical fact.

Approximation Analysis of Margin-Based Ranking Algorithm

2014 ◽  
Vol 644-650 ◽  
pp. 2286-2289
Author(s):  
Jin Luo
Keyword(s):  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Learning Task ◽  
Least Square ◽  
Ranking Algorithm ◽  
Ranking Data ◽  
Fast Learning ◽  
Data Points ◽  
Approximation Analysis ◽  
Ranking Loss

Ranking data points with respect to a given preference criterion is an example of a preference learning task. In this paper, we investigate the generalization performance of the regularized ranking algorithm associated with least square ranking loss in a reproducing kernel Hilbert space, and use the method of computing hold-out estimates for the proposed algorithm. Based on using the hold-out method, we obtain fast learning rate for this algorithm.

Diabetes prediction using an improved machine learning approach

2021 ◽  
Vol 8 (4) ◽  
pp. 726-735
Author(s):  
S. Lyaqini ◽  
◽  
M. Nachaoui ◽  
Keyword(s):  
Machine Learning ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Stochastic Gradient Descent ◽  
Healthcare Sector ◽  
L1 Norm ◽  
Machine Learning Model ◽  
Machine Learning Approach ◽  
Diabetes Prediction ◽  
Regularized Optimization

This paper deals with a machine-learning model arising from the healthcare sector, namely diabetes progression. The model is reformulated into a regularized optimization problem. The term of the fidelity is the L1 norm and the optimization space of the minimum is constructed by a reproducing kernel Hilbert space (RKSH). The numerical approximation of the model is realized by the Adam method, which shows its success in the numerical experiments (if compared to the stochastic gradient descent (SGD) algorithm).

scholarly journals OUTLIER-ROBUST KERNEL HIERARCHICAL-OPTIMIZATION RLS ON A BUDGET WITH AFFINE CONSTRAINTS

2020 ◽  
Author(s):  
Konstantinos Slavakis ◽  
Masahiro Yukawa
Keyword(s):  
Reproducing Kernel ◽  
Side Information ◽  
Reproducing Kernel Hilbert Space ◽  
Learning Task ◽  
Hierarchical Optimization ◽  
Quadratic Loss ◽  
Infinite Dimensional ◽  
Learning Framework ◽  
Affine Constraints ◽  
Personal Use

<div>This paper introduces a non-parametric learning framework to combat outliers in online, multi-output, and nonlinear regression tasks. A hierarchical-optimization problem underpins the learning task: Search in a reproducing kernel Hilbert space (RKHS) for a function that minimizes a sample average $\ell_p$-norm ($1 \leq p \leq 2$) error loss on data contaminated by noise and outliers, subject to side information that takes the form of affine constraints defined as the set of minimizers of a quadratic loss on a finite number of faithful data devoid of noise and outliers. To surmount the computational obstacles inflicted by the choice of loss and the potentially infinite dimensional RKHS, approximations of the $\ell_p$-norm loss, as well as a novel twist of the criterion of approximate linear dependency are devised to keep the computational-complexity footprint of the proposed algorithm bounded over time. Numerical tests on datasets showcase the robust behavior of the advocated framework against different types of outliers, under a low computational load, while satisfying at the same time the affine constraints, in contrast to the state-of-the-art methods which are constraint agnostic.</div><div><br></div><div>-------</div><div><br></div><div>© 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.<br></div>

scholarly journals OUTLIER-ROBUST KERNEL HIERARCHICAL-OPTIMIZATION RLS ON A BUDGET WITH AFFINE CONSTRAINTS

2021 ◽  
Author(s):  
Konstantinos Slavakis ◽  
Masahiro Yukawa
Keyword(s):  
Reproducing Kernel ◽  
Side Information ◽  
Reproducing Kernel Hilbert Space ◽  
Learning Task ◽  
Hierarchical Optimization ◽  
Quadratic Loss ◽  
Infinite Dimensional ◽  
Learning Framework ◽  
Affine Constraints ◽  
Personal Use

<div>This paper introduces a non-parametric learning framework to combat outliers in online, multi-output, and nonlinear regression tasks. A hierarchical-optimization problem underpins the learning task: Search in a reproducing kernel Hilbert space (RKHS) for a function that minimizes a sample average $\ell_p$-norm ($1 \leq p \leq 2$) error loss on data contaminated by noise and outliers, subject to side information that takes the form of affine constraints defined as the set of minimizers of a quadratic loss on a finite number of faithful data devoid of noise and outliers. To surmount the computational obstacles inflicted by the choice of loss and the potentially infinite dimensional RKHS, approximations of the $\ell_p$-norm loss, as well as a novel twist of the criterion of approximate linear dependency are devised to keep the computational-complexity footprint of the proposed algorithm bounded over time. Numerical tests on datasets showcase the robust behavior of the advocated framework against different types of outliers, under a low computational load, while satisfying at the same time the affine constraints, in contrast to the state-of-the-art methods which are constraint agnostic.</div><div><br></div><div>-------</div><div><br></div><div>© 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.<br></div>

scholarly journals Outlier-Robust Kernel Hierarchical-Optimization RLS On A Budget With Affine Constraints

2020 ◽  
Author(s):  
Konstantinos Slavakis ◽  
Masahiro Yukawa
Keyword(s):  
Reproducing Kernel ◽  
Side Information ◽  
Reproducing Kernel Hilbert Space ◽  
Learning Task ◽  
Hierarchical Optimization ◽  
Quadratic Loss ◽  
Infinite Dimensional ◽  
Learning Framework ◽  
Affine Constraints ◽  
Personal Use

<div>This paper introduces a non-parametric learning framework to combat outliers in online, multi-output, and nonlinear regression tasks. A hierarchical-optimization problem underpins the learning task: Search in a reproducing kernel Hilbert space (RKHS) for a function that minimizes a sample average $\ell_p$-norm ($1 \leq p \leq 2$) error loss on data contaminated by noise and outliers, subject to side information that takes the form of affine constraints defined as the set of minimizers of a quadratic loss on a finite number of faithful data devoid of noise and outliers. To surmount the computational obstacles inflicted by the choice of loss and the potentially infinite dimensional RKHS, approximations of the $\ell_p$-norm loss, as well as a novel twist of the criterion of approximate linear dependency are devised to keep the computational-complexity footprint of the proposed algorithm bounded over time. Numerical tests on datasets showcase the robust behavior of the advocated framework against different types of outliers, under a low computational load, while satisfying at the same time the affine constraints, in contrast to the state-of-the-art methods which are constraint agnostic.</div><div><br></div><div>-------</div><div><br></div><div>© 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.<br></div>

Deep Learning in Disease Diagnosis: Models and Datasets

2020 ◽  
Vol 15 ◽  
Author(s):  
Deeksha Saxena ◽  
Mohammed Haris Siddiqui ◽  
Rajnish Kumar
Keyword(s):  
Biological Sciences ◽  
Machine Learning ◽  
Deep Learning ◽  
Disease Diagnosis ◽  
Learning Models ◽  
Data Types ◽  
Related Data ◽  
Abstract Level ◽  
Experimental Validations ◽  
Selection Of

Background: Deep learning (DL) is an Artificial neural network-driven framework with multiple levels of representation for which non-linear modules combined in such a way that the levels of representation can be enhanced from lower to a much abstract level. Though DL is used widely in almost every field, it has largely brought a breakthrough in biological sciences as it is used in disease diagnosis and clinical trials. DL can be clubbed with machine learning, but at times both are used individually as well. DL seems to be a better platform than machine learning as the former does not require an intermediate feature extraction and works well with larger datasets. DL is one of the most discussed fields among the scientists and researchers these days for diagnosing and solving various biological problems. However, deep learning models need some improvisation and experimental validations to be more productive. Objective: To review the available DL models and datasets that are used in disease diagnosis. Methods: Available DL models and their applications in disease diagnosis were reviewed discussed and tabulated. Types of datasets and some of the popular disease related data sources for DL were highlighted. Results: We have analyzed the frequently used DL methods, data types and discussed some of the recent deep learning models used for solving different biological problems. Conclusion: The review presents useful insights about DL methods, data types, selection of DL models for the disease diagnosis.

Sign in / Sign up

Export Citation Format

Share Document