scholarly journals Incremental Online Learning in High Dimensions

2005 ◽  
Vol 17 (12) ◽  
pp. 2602-2634 ◽  
Author(s):  
Sethu Vijayakumar ◽  
Aaron D'Souza ◽  
Stefan Schaal
Keyword(s):  
Linear Models ◽  
Nonlinear Function ◽  
Training Data ◽  
High Dimensional ◽  
Data Sets ◽  
Least Squares Regression ◽  
Computationally Efficient ◽  
Learning Techniques ◽  
Leave One Out ◽  
Nonlinear Function Approximation

Locally weighted projection regression (LWPR) is a new algorithm for incremental nonlinear function approximation in high-dimensional spaces with redundant and irrelevant input dimensions. At its core, it employs nonparametric regression with locally linear models. In order to stay computationally efficient and numerically robust, each local model performs the regression analysis with a small number of univariate regressions in selected directions in input space in the spirit of partial least squares regression. We discuss when and how local learning techniques can successfully work in high-dimensional spaces and review the various techniques for local dimensionality reduction before finally deriving the LWPR algorithm. The properties of LWPR are that it (1) learns rapidly with second-order learning methods based on incremental training, (2) uses statistically sound stochastic leave-one-out cross validation for learning without the need to memorize training data, (3) adjusts its weighting kernels based on only local information in order to minimize the danger of negative interference of incremental learning, (4) has a computational complexity that is linear in the number of inputs, and (5) can deal with a large number of—possibly redundant—inputs, as shown in various empirical evaluations with up to 90 dimensional data sets. For a probabilistic interpretation, predictive variance and confidence intervals are derived. To our knowledge, LWPR is the first truly incremental spatially localized learning method that can successfully and efficiently operate in very high-dimensional spaces.

KANSA METHOD BASED ON THE HAUSDORFF FRACTAL DISTANCE FOR HAUSDORFF DERIVATIVE POISSON EQUATIONS

Fractals ◽  
2018 ◽  
Vol 26 (04) ◽  
pp. 1850084 ◽  
Author(s):  
FAJIE WANG ◽  
WEN CHEN ◽  
CHUANZENG ZHANG ◽  
QINGSONG HUA
Keyword(s):  
Shape Parameter ◽  
Cross Validation ◽  
Optimal Shape ◽  
High Dimensional ◽  
Irregular Domain ◽  
Computationally Efficient ◽  
Poisson Equations ◽  
A New Technique ◽  
Leave One Out ◽  
Kansa Method

This study proposes the radial basis function (RBF) based on the Hausdorff fractal distance and then applies it to develop the Kansa method for the solution of the Hausdorff derivative Poisson equations. The Kansa method is a meshless global technique promising for high-dimensional irregular domain problems. It is, however, noted that the shape parameter of the RBFs can have a significant influence on the accuracy and robustness of the numerical solution. Based on the leave-one-out cross-validation algorithm proposed by Rippa, this study presents a new technique to choose the optimal shape parameter of the RBFs with the Hausdorff fractal distance. Numerical experiments show that the Kansa method based on the Hausdorff fractal distance is highly accurate and computationally efficient for the Hausdorff derivative Poisson equations.

Application of multi-sensor unmanned aerial system for identification of hydrothermal alteration zones

2020 ◽  
Author(s):  
Yosoon Choi ◽  
Jieun Baek ◽  
Jangwon Suh ◽  
Sung-Min Kim
Keyword(s):  
Machine Learning ◽  
Classification Accuracy ◽  
Training Data ◽  
Sensor Data ◽  
Machine Learning Techniques ◽  
Integrated Analysis ◽  
Unmanned Aerial System ◽  
Data Sets ◽  
Learning Techniques ◽  
Hydrothermal Alteration Zones

<p>In this study, we proposed a method to utilize a multi-sensor Unmanned Aerial System (UAS) for exploration of hydrothermal alteration zones. This study selected an area (10m × 20m) composed mainly of the andesite and located on the coast, with wide outcrops and well-developed structural and mineralization elements. Multi-sensor (visible, multispectral, thermal, magnetic) data were acquired in the study area using UAS, and were studied using machine learning techniques. For utilizing the machine learning techniques, we applied the stratified random method to sample 1000 training data in the hydrothermal zone and 1000 training data in the non-hydrothermal zone identified through the field survey. The 2000 training data sets created for supervised learning were first classified into 1500 for training and 500 for testing. Then, 1500 for training were classified into 1200 for training and 300 for validation. The training and validation data for machine learning were generated in five sets to enable cross-validation. Five types of machine learning techniques were applied to the training data sets: k-Nearest Neighbors (k-NN), Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), and Deep Neural Network (DNN). As a result of integrated analysis of multi-sensor data using five types of machine learning techniques, RF and SVM techniques showed high classification accuracy of about 90%. Moreover, performing integrated analysis using multi-sensor data showed relatively higher classification accuracy in all five machine learning techniques than analyzing magnetic sensing data or single optical sensing data only.</p>

scholarly journals Multisubject Learning for Common Spatial Patterns in Motor-Imagery BCI

2011 ◽  
Vol 2011 ◽  
pp. 1-9 ◽  
Author(s):  
Dieter Devlaminck ◽  
Bart Wyns ◽  
Moritz Grosse-Wentrup ◽  
Georges Otte ◽  
Patrick Santens
Keyword(s):  
Motor Imagery ◽  
Spatial Filter ◽  
Training Data ◽  
Machine Learning Techniques ◽  
Data Sets ◽  
Calibration Data ◽  
Common Spatial Pattern ◽  
Computer Interfaces ◽  
Learning Techniques ◽  
The Common

Motor-imagery-based brain-computer interfaces (BCIs) commonly use the common spatial pattern filter (CSP) as preprocessing step before feature extraction and classification. The CSP method is a supervised algorithm and therefore needs subject-specific training data for calibration, which is very time consuming to collect. In order to reduce the amount of calibration data that is needed for a new subject, one can apply multitask (from now on called multisubject) machine learning techniques to the preprocessing phase. Here, the goal of multisubject learning is to learn a spatial filter for a new subject based on its own data and that of other subjects. This paper outlines the details of the multitask CSP algorithm and shows results on two data sets. In certain subjects a clear improvement can be seen, especially when the number of training trials is relatively low.

scholarly journals Enhancing Image Diagnosis by the Implementation of Transfer Classifiers

2019 ◽  
Vol 8 (3) ◽  
pp. 999-1002
Keyword(s):  
Supervised Learning ◽  
Transfer Learning ◽  
Training Data ◽  
Data Sets ◽  
Learning Approaches ◽  
Learning Techniques ◽  
Image Diagnosis ◽  
Sensitivity Problem ◽  
Common Application ◽  
Target Data

Images generated from a variety of sources and foundations today can pose difficulty for a user to interpret similarity in them or analyze them for further use because of their segmentation policies. This unconventionality can generate many errors, because of which the previously used traditional methodologies such as supervised learning techniques less resourceful, which requires huge quantity of labelled training data which mirrors the desired target data. This paper thus puts forward the mechanism of an alternative technique i.e. transfer learning to be used in image diagnosis so that efficiency and accuracy among images can be achieved. This type of mechanism deals with variation in the desired and actual data used for training and the outlier sensitivity, which ultimately enhances the predictions by giving better results in various areas, thus leaving the traditional methodologies behind. The following analysis further discusses about three types of transfer classifiers which can be applied using only small volume of training data sets and their contrast with the traditional method which requires huge quantities of training data having attributes with slight changes. The three different separators were compared amongst them and also together from the traditional methodology being used for a very common application used in our daily life. Also, commonly occurring problems such as the outlier sensitivity problem were taken into consideration and measures were taken to recognise and improvise them. On further research it was observed that the performance of transfer learning exceeds that of the conventional supervised learning approaches being used for small amount of characteristic training data provided reducing the stratification errors to a great extent

scholarly journals Iterative Data Programming for Expanding Text Classification Corpora

2020 ◽  
Vol 34 (08) ◽  
pp. 13332-13337
Author(s):  
Neil Mallinar ◽  
Abhishek Shah ◽  
Tin Kam Ho ◽  
Rajendra Ugrani ◽  
Ayush Gupta
Keyword(s):  
Text Classification ◽  
Training Data ◽  
Data Sets ◽  
Conversational Agents ◽  
Intent Recognition ◽  
Text Data ◽  
Learning Techniques ◽  
Programming Techniques ◽  
Classification Tasks ◽  
Training Examples

Real-world text classification tasks often require many labeled training examples that are expensive to obtain. Recent advancements in machine teaching, specifically the data programming paradigm, facilitate the creation of training data sets quickly via a general framework for building weak models, also known as labeling functions, and denoising them through ensemble learning techniques. We present a fast, simple data programming method for augmenting text data sets by generating neighborhood-based weak models with minimal supervision. Furthermore, our method employs an iterative procedure to identify sparsely distributed examples from large volumes of unlabeled data. The iterative data programming techniques improve newer weak models as more labeled data is confirmed with human-in-loop. We show empirical results on sentence classification tasks, including those from a task of improving intent recognition in conversational agents.

scholarly journals Evaluation of Melanoma Diagnosis using Imbalanced Learning

2018 ◽  
Author(s):  
Lucas Bezerra Maia ◽  
Alan Carlos Lima ◽  
Pedro Thiago Cutrim Santos ◽  
Nigel da Silva Lima ◽  
João Dallyson Sousa De Almeida ◽  
...  
Keyword(s):  
Nearest Neighbor ◽  
Imbalanced Data ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Data Sets ◽  
Imbalanced Learning ◽  
Training Models ◽  
Melanoma Diagnosis ◽  
Learning Techniques ◽  
Detection And Diagnosis

Melanoma is the most lethal type of skin cancer when compared to others, but patients have high recovery rates if the disease is discovered in its early stages. Several approaches to automatic detection and diagnosis have been explored by different authors. Training models with the existing data sets has been a difficult task due to the problem of imbalanced data. This work aims to evaluate the performance of machine learning algorithms combined with imbalanced learning techniques, regarding the task of melanoma diagnosis. Preliminary results have shown that features extracted with ResNet Convolutional Neural Network, along with Random Forest, achieved an improvement of sensibility of approximately 21%, after balancing the training data with Synthetic Minority Oversampling TEchnique (SMOTE) and Edited Nearest Neighbor (ENN) rule.

scholarly journals Improving the resolution of migrated images by approximating the inverse Hessian using deep learning

Geophysics ◽  
2020 ◽  
Vol 85 (4) ◽  
pp. WA173-WA183 ◽  
Author(s):  
Harpreet Kaur ◽  
Nam Pham ◽  
Sergey Fomel
Keyword(s):  
Field Data ◽  
Velocity Model ◽  
Image Resolution ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Data Sets ◽  
Computationally Efficient ◽  
Validation Data ◽  
Network Output

We have estimated migrated images with meaningful amplitudes matching least-squares migrated images by approximating the inverse Hessian using generative adversarial networks (GANs) in a conditional setting. We use the CycleGAN framework and extend it to the conditional CycleGAN such that the mapping from the migrated image to the true reflectivity is subjected to a velocity attribute condition. This algorithm is applied after migration and is computationally efficient. It produces results comparable to iterative inversion but at a significantly reduced cost. In numerical experiments with synthetic and field data sets, the adopted method improves image resolution, attenuates noise, reduces migration artifacts, and enhances reflection amplitudes. We train the network with three different data sets and test on three other data sets, which are not a part of training. Tests on validation data sets verify the effectiveness of the approach. In addition, the field-data example also highlights the effect of the bandwidth of the training data and the quality of the velocity model on the quality of the deep neural network output.

scholarly journals Sparse identification of nonlinear dynamics for model predictive control in the low-data limit

2018 ◽  
Vol 474 (2219) ◽  
pp. 20180335 ◽  
Author(s):  
E. Kaiser ◽  
J. N. Kutz ◽  
S. L. Brunton
Keyword(s):  
Nonlinear Dynamics ◽  
Machine Learning ◽  
Model Predictive Control ◽  
Predictive Control ◽  
Flight Control ◽  
Linear Models ◽  
Training Data ◽  
Data Driven ◽  
Computationally Efficient ◽  
Hiv Model

Data-driven discovery of dynamics via machine learning is pushing the frontiers of modelling and control efforts, providing a tremendous opportunity to extend the reach of model predictive control (MPC). However, many leading methods in machine learning, such as neural networks (NN), require large volumes of training data, may not be interpretable, do not easily include known constraints and symmetries, and may not generalize beyond the attractor where models are trained. These factors limit their use for the online identification of a model in the low-data limit, for example following an abrupt change to the system dynamics. In this work, we extend the recent sparse identification of nonlinear dynamics (SINDY) modelling procedure to include the effects of actuation and demonstrate the ability of these models to enhance the performance of MPC, based on limited, noisy data. SINDY models are parsimonious, identifying the fewest terms in the model needed to explain the data, making them interpretable and generalizable. We show that the resulting SINDY-MPC framework has higher performance, requires significantly less data, and is more computationally efficient and robust to noise than NN models, making it viable for online training and execution in response to rapid system changes. SINDY-MPC also shows improved performance over linear data-driven models, although linear models may provide a stopgap until enough data is available for SINDY. SINDY-MPC is demonstrated on a variety of dynamical systems with different challenges, including the chaotic Lorenz system, a simple model for flight control of an F8 aircraft, and an HIV model incorporating drug treatment.

scholarly journals 32 Cross validation of best linear unbiased predictions of breeding values using an efficient leave-one-out strategy

2020 ◽  
Vol 98 (Supplement_4) ◽  
pp. 10-11
Author(s):  
Jian Cheng ◽  
Rohan Fernando ◽  
Jack C Dekkers
Keyword(s):  
Covariance Matrix ◽  
Random Effects ◽  
Fixed Effects ◽  
Cross Validation ◽  
Mixed Linear Model ◽  
Training Data ◽  
Breeding Value ◽  
Data Sets ◽  
Fixed And Random Effects ◽  
Leave One Out

Abstract Efficient strategies have been developed for leave-one-out cross validation (LOOCV) of predicted phenotypes in a simple model with an overall mean and marker effects or animal genetic effects to evaluate the accuracy of genomic predictions. For such a model, the correlation between the predicted and the observed phenotype is identical to the correlation between the observed phenotype and the estimated breeding value (EBV). When the model is more complex, with multiple fixed and random effects, although the correlation between the observed and predicted phenotype can be obtained efficiently by LOOCV, it is not equal to the correlation between the observed phenotype and EBV, which is the statistic of interest. The objective here was to develop and evaluate an efficient LOOCV method for EBV or for predictions of other random effects under a general mixed linear model. The approach is based on treated all effects in the model, with large variances for fixed effects. Naïve LOOCV requires inverting the (n - 1) x (n - 1) dimensional phenotypic covariance matrix for each of the n (= no. observations) training data sets. Our method efficiently obtains these inverses from the inverse of the phenotypic covariance matrix for all n observations. Naïve LOOCV of EBV by pre-correction of fixed effects using the training data (Naïve LOOCV) and the new efficient LOOCV were compared. The new efficient LOOCV for EBV was 962 times faster than Naïve LOOCV. Prediction accuracies from the two strategies were the same (0.20). Funded by USDA-NIFA grant # 2017-67007-26144.

Sign in / Sign up

Export Citation Format

Share Document