Convergence analysis of online learning algorithm with two-stage step size

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Weilin Nie ◽  
Cheng Wang
Keyword(s):  
Online Learning ◽  
Optimization Problems ◽  
Learning Algorithm ◽  
Computational Cost ◽  
Reproducing Kernels ◽  
Step Size ◽  
Two Stage ◽  
Logarithmic Term ◽  
Convergence Performance ◽  
Online Learning Algorithm

Abstract Online learning is a classical algorithm for optimization problems. Due to its low computational cost, it has been widely used in many aspects of machine learning and statistical learning. Its convergence performance depends heavily on the step size. In this paper, a two-stage step size is proposed for the unregularized online learning algorithm, based on reproducing Kernels. Theoretically, we prove that, such an algorithm can achieve a nearly min–max convergence rate, up to some logarithmic term, without any capacity condition.

scholarly journals ONLINE LEARNING FOR IMAGE PROCESSING IN NETWORKED SETTING

2017 ◽  
Vol 10 (13) ◽  
pp. 284
Author(s):  
Ankush Rai ◽  
Jagadeesh Kannan R
Keyword(s):  
Machine Learning ◽  
Image Processing ◽  
Online Learning ◽  
Learning Algorithm ◽  
Machine Learning Algorithm ◽  
The Past ◽  
Slow Development ◽  
Online Learning Algorithm

In the past decade development of machine learning algorithm for network settings has witnessed little advancements owing to slow development of technologies for improving bandwidth and latency.  In this study we present a novel online learning algorithm for network based computational operations in image processing setting

Kernel Online Learning Algorithm With Scale Adaptation

2018 ◽  
Vol 65 (11) ◽  
pp. 1788-1792 ◽  
Author(s):  
Shiyuan Wang ◽  
Lujuan Dang ◽  
Badong Chen ◽  
Chengxiu Ling ◽  
Lidan Wang ◽  
...  
Keyword(s):  
Online Learning ◽  
Learning Algorithm ◽  
Scale Adaptation ◽  
Online Learning Algorithm

scholarly journals Online Learning from Data Streams with Varying Feature Spaces

2019 ◽  
Vol 33 ◽  
pp. 3232-3239 ◽  
Author(s):  
Ege Beyazit ◽  
Jeevithan Alagurajah ◽  
Xindong Wu
Keyword(s):  
Online Learning ◽  
Data Streams ◽  
Learning Algorithm ◽  
Feature Space ◽  
Model Complexity ◽  
Risk Minimization ◽  
Minimization Principle ◽  
Empirical Risk ◽  
Feature Spaces ◽  
Online Learning Algorithm

We study the problem of online learning with varying feature spaces. The problem is challenging because, unlike traditional online learning problems, varying feature spaces can introduce new features or stop having some features without following a pattern. Other existing methods such as online streaming feature selection (Wu et al. 2013), online learning from trapezoidal data streams (Zhang et al. 2016), and learning with feature evolvable streams (Hou, Zhang, and Zhou 2017) are not capable to learn from arbitrarily varying feature spaces because they make assumptions about the feature space dynamics. In this paper, we propose a novel online learning algorithm OLVF to learn from data with arbitrarily varying feature spaces. The OLVF algorithm learns to classify the feature spaces and the instances from feature spaces simultaneously. To classify an instance, the algorithm dynamically projects the instance classifier and the training instance onto their shared feature subspace. The feature space classifier predicts the projection confidences for a given feature space. The instance classifier will be updated by following the empirical risk minimization principle and the strength of the constraints will be scaled by the projection confidences. Afterwards, a feature sparsity method is applied to reduce the model complexity. Experiments on 10 datasets with varying feature spaces have been conducted to demonstrate the performance of the proposed OLVF algorithm. Moreover, experiments with trapezoidal data streams on the same datasets have been conducted to show that OLVF performs better than the state-of-the-art learning algorithm (Zhang et al. 2016).

Sign in / Sign up

Export Citation Format

Share Document