Local Algorithms for Pattern Recognition and Dependencies Estimation

1993 ◽  
Vol 5 (6) ◽  
pp. 893-909 ◽  
Author(s):  
V. Vapnik ◽  
L. Bottou
Keyword(s):  
Pattern Recognition ◽  
Convergence Theory ◽  
Learning Problems ◽  
Risk Minimization ◽  
Local Algorithms ◽  
Minimization Principle ◽  
Performance Improvements ◽  
Structural Risk Minimization Principle ◽  
Regression Problems ◽  
Local Risk

In previous publications (Bottou and Vapnik 1992; Vapnik 1992) we described local learning algorithms, which result in performance improvements for real problems. We present here the theoretical framework on which these algorithms are based. First, we present a new statement of certain learning problems, namely the local risk minimization. We review the basic results of the uniform convergence theory of learning, and extend these results to local risk minimization. We also extend the structural risk minimization principle for both pattern recognition problems and regression problems. This extended induction principle is the basis for a new class of algorithms.

Modeling river stage–discharge–sediment rating relation using support vector regression

2012 ◽  
Vol 43 (6) ◽  
pp. 851-861 ◽  
Author(s):  
Sharad K. Jain
Keyword(s):  
Support Vector Regression ◽  
Sediment Concentration ◽  
Support Vector ◽  
Learning Approaches ◽  
Risk Minimization ◽  
Minimization Principle ◽  
River Stage ◽  
Structural Risk Minimization Principle ◽  
Structural Risk ◽  
Regression Problems

A variety of data-driven approaches have been developed in the recent past to capture the properties of hydrological data for improved modeling. These include artificial neural networks (ANNs), fuzzy logic and evolutionary algorithms, amongst others. Of late, kernel-based machine learning approaches have become popular due to their inherent advantages over traditional modeling techniques. In this work, support vector machines (SVMs), a kernel-based learning approach, has been investigated for its suitability to model the relationship between the river stage, discharge, and sediment concentration. SVMs are an approximate implementation of the structural risk minimization principle that aims at minimizing a bound on the generalization error of a model. These have been found to be promising in many areas including hydrology. Application of SVMs to regression problems is known as support vector regression (SVR). This paper presents an application of SVR to model river discharge and sediment concentration rating relation. The results obtained using SVR were compared with those from ANNs and it was found that the SVR approach is better when compared with ANNs.

The Bound of Local Risk Minimization Estimation on Quasi-Probability Space

2013 ◽  
Vol 677 ◽  
pp. 431-435
Author(s):  
Yun Chao Bai ◽  
Ying Chun Guo
Keyword(s):  
Probability Space ◽  
Estimation Problem ◽  
Risk Minimization ◽  
Structural Risk Minimization ◽  
Minimization Principle ◽  
Structural Risk Minimization Principle ◽  
Structural Risk ◽  
Local Risk

the ideas of local risk minimization estimation problem on quasi-probability space is presented; In order to make structural risk minimization principle apply to the problem of local risk minimization estimation, the paper gives and proves the bounds of the bound of local risk minimization estimation on quasi-probability.

A Robust Information Clustering Algorithm

2005 ◽  
Vol 17 (12) ◽  
pp. 2672-2698 ◽  
Author(s):  
Qing Song
Keyword(s):  
Clustering Algorithm ◽  
Risk Minimization ◽  
Minimax Optimization ◽  
Cluster Number ◽  
Minimization Principle ◽  
Nonparametric Approach ◽  
Empirical Risk ◽  
Structural Risk Minimization Principle ◽  
Real Risk ◽  
Data Points

We focus on the scenario of robust information clustering (RIC) based on the minimax optimization of mutual information (MI). The minimization of MI leads to the standard mass-constrained deterministic annealing clustering, which is an empirical risk-minimization algorithm. The maximization of MI works out an upper bound of the empirical risk via the identification of outliers (noisy data points). Furthermore, we estimate the real risk VC-bound and determine an optimal cluster number of the RIC based on the structural risk-minimization principle. One of the main advantages of the minimax optimization of MI is that it is a nonparametric approach, which identifies the outliers through the robust density estimate and forms a simple data clustering algorithm based on the square error of the Euclidean distance.

Application of Genetic Arithmetic and Support Vector Machine in Prediction of Compression Index of Clay

2013 ◽  
Vol 438-439 ◽  
pp. 1167-1170
Author(s):  
Xu Chao Shi ◽  
Ying Fei Gao
Keyword(s):  
Support Vector Machine ◽  
Influence Factors ◽  
Support Vector ◽  
Compression Index ◽  
Risk Minimization ◽  
Minimization Principle ◽  
Structural Risk Minimization Principle ◽  
Svm Model ◽  
Genetic Arithmetic ◽  
Practical Tool

The compression index is an important soil property that is essential to many geotechnical designs. As the determination of the compression index from consolidation tests is relatively time-consuming. Support Vector Machine (SVM) is a statistical learning theory based on a structural risk minimization principle that minimizes both error and weight terms. Considering the fact that parameters in SVM model are difficult to be decided, a genetic SVM was presented in which the parameters in SVM method are optimized by Genetic Algorithm (GA). Taking plasticity index, water content, void ration and density of soil as primary influence factors, the prediction model of compression index based on GA-SVM approach was obtained. The results of this study showed that the GA-SVM approach has the potential to be a practical tool for predicting compression index of soil.

Supervised Factorial Learning

1993 ◽  
Vol 5 (5) ◽  
pp. 750-766 ◽  
Author(s):  
A. Norman Redlich
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Functional Connectivity ◽  
Cellular Automata ◽  
Sensory Processing ◽  
Learning Problems ◽  
Local Algorithms ◽  
Sensory Image ◽  
Nonlinear Technique ◽  
Reversible Cellular Automata

Factorial learning, finding a statistically independent representation of a sensory “image”—a factorial code—is applied here to solve multilayer supervised learning problems that have traditionally required backpropagation. This lends support to Barlow's argument for factorial sensory processing, by demonstrating how it can solve actual pattern recognition problems. Two techniques for supervised factorial learning are explored, one of which gives a novel distributed solution requiring only positive examples. Also, a new nonlinear technique for factorial learning is introduced that uses neural networks based on almost reversible cellular automata. Due to the special functional connectivity of these networks—which resemble some biological microcircuits—learning requires only simple local algorithms. Also, supervised factorial learning is shown to be a viable alternative to backpropagation. One significant advantage is the existence of a measure for the performance of intermediate learning stages.

Sign in / Sign up

Export Citation Format

Share Document