Online Learning Based on Online DCA and Application to Online Classification

2020 ◽  
Vol 32 (4) ◽  
pp. 759-793 ◽  
Author(s):  
Hoai An Le Thi ◽  
Vinh Thanh Ho
Keyword(s):  
Online Learning ◽  
State Of The Art ◽  
Classification Algorithms ◽  
Data Sets ◽  
Prediction Problem ◽  
Dc Algorithm ◽  
Difference Of Convex Functions ◽  
Online Classification ◽  
Learning Techniques ◽  
Difference Of Convex

We investigate an approach based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) for online learning techniques. The prediction problem of an online learner can be formulated as a DC program for which online DCA is applied. We propose the two so-called complete/approximate versions of online DCA scheme and prove their logarithmic/sublinear regrets. Six online DCA-based algorithms are developed for online binary linear classification. Numerical experiments on a variety of benchmark classification data sets show the efficiency of our proposed algorithms in comparison with the state-of-the-art online classification algorithms.

scholarly journals Multi-View Multi-Label Learning with View-Specific Information Extraction

2019 ◽  
Author(s):  
Xuan Wu ◽  
Qing-Guo Chen ◽  
Yao Hu ◽  
Dengbao Wang ◽  
Xiaodong Chang ◽  
...  
Keyword(s):  
Information Extraction ◽  
Real World ◽  
State Of The Art ◽  
Specific Information ◽  
Learning Approach ◽  
Data Sets ◽  
Learning Approaches ◽  
Real World Data ◽  
Learning Techniques ◽  
Shared Information

Multi-view multi-label learning serves an important framework to learn from objects with diverse representations and rich semantics. Existing multi-view multi-label learning techniques focus on exploiting shared subspace for fusing multi-view representations, where helpful view-specific information for discriminative modeling is usually ignored. In this paper, a novel multi-view multi-label learning approach named SIMM is proposed which leverages shared subspace exploitation and view-specific information extraction. For shared subspace exploitation, SIMM jointly minimizes confusion adversarial loss and multi-label loss to utilize shared information from all views. For view-specific information extraction, SIMM enforces an orthogonal constraint w.r.t. the shared subspace to utilize view-specific discriminative information. Extensive experiments on real-world data sets clearly show the favorable performance of SIMM against other state-of-the-art multi-view multi-label learning approaches.

scholarly journals Semi-supervised multi-label classification using an extended graph-based manifold regularization

2022 ◽  
Author(s):  
Ding Li ◽  
Scott Dick
Keyword(s):  
Supervised Learning ◽  
State Of The Art ◽  
Ground Truth ◽  
Classification Algorithms ◽  
Data Sets ◽  
Manifold Regularization ◽  
Regularization Algorithm ◽  
Weighting Strategy ◽  
Extended Graph ◽  
General Vector

AbstractGraph-based algorithms are known to be effective approaches to semi-supervised learning. However, there has been relatively little work on extending these algorithms to the multi-label classification case. We derive an extension of the Manifold Regularization algorithm to multi-label classification, which is significantly simpler than the general Vector Manifold Regularization approach. We then augment our algorithm with a weighting strategy to allow differential influence on a model between instances having ground-truth vs. induced labels. Experiments on four benchmark multi-label data sets show that the resulting algorithm performs better overall compared to the existing semi-supervised multi-label classification algorithms at various levels of label sparsity. Comparisons with state-of-the-art supervised multi-label approaches (which of course are fully labeled) also show that our algorithm outperforms all of them even with a substantial number of unlabeled examples.

scholarly journals SVM-Based Multiple Instance Classification via DC Optimization

Algorithms ◽  
2019 ◽  
Vol 12 (12) ◽  
pp. 249 ◽  
Author(s):  
Annabella Astorino ◽  
Antonio Fuduli ◽  
Giovanni Giallombardo ◽  
Giovanna Miglionico
Keyword(s):  
Binary Classification ◽  
Error Function ◽  
Multiple Instance Learning ◽  
Classification Error ◽  
Support Vector ◽  
Data Sets ◽  
Training Set ◽  
Dc Algorithm ◽  
Difference Of Convex ◽  
Dc Decomposition

A multiple instance learning problem consists of categorizing objects, each represented as a set (bag) of points. Unlike the supervised classification paradigm, where each point of the training set is labeled, the labels are only associated with bags, while the labels of the points inside the bags are unknown. We focus on the binary classification case, where the objective is to discriminate between positive and negative bags using a separating surface. Adopting a support vector machine setting at the training level, the problem of minimizing the classification-error function can be formulated as a nonconvex nonsmooth unconstrained program. We propose a difference-of-convex (DC) decomposition of the nonconvex function, which we face using an appropriate nonsmooth DC algorithm. Some of the numerical results on benchmark data sets are reported.

Maximum-Margin Model for Nearest Prototype Classifiers

2018 ◽  
Vol 22 (4) ◽  
pp. 565-577
Author(s):  
Yoshifumi Kusunoki ◽  
◽  
Chiharu Wakou ◽  
Keiji Tatsumi
Keyword(s):  
Discriminant Function ◽  
Optimization Problem ◽  
Numerical Study ◽  
Classification Performance ◽  
The Other ◽  
Maximum Margin ◽  
Dc Algorithm ◽  
Difference Of Convex Functions ◽  
Dc Program ◽  
Difference Of Convex

In this paper, we study nearest prototype classifiers, which classify data instances into the classes to which their nearest prototypes belong. We propose a maximum-margin model for nearest prototype classifiers. To provide the margin, we define a class-wise discriminant function for instances by the negatives of distances of their nearest prototypes of the class. Then, we define the margin by the minimum of differences between the discriminant function values of instances with respect to the classes they belong to and the values of the other classes. The optimization problem corresponding to the maximum-margin model is a difference of convex functions (DC) program. It is solved using a DC algorithm, which is ak-means-like algorithm, i.e., the members and positions of prototypes are alternately optimized. Through a numerical study, we analyze the effects of hyperparameters of the maximum-margin model, especially considering the classification performance.

Sparse Covariance Matrix Estimation by DCA-Based Algorithms

2017 ◽  
Vol 29 (11) ◽  
pp. 3040-3077 ◽  
Author(s):  
Duy Nhat Phan ◽  
Hoai An Le Thi ◽  
Tao Pham Dinh
Keyword(s):  
Covariance Matrix ◽  
Nonconvex Programming ◽  
Real Data ◽  
Dc Programming ◽  
Covariance Matrix Estimation ◽  
Data Sets ◽  
Matrix Estimation ◽  
Dc Algorithm ◽  
Difference Of Convex Functions ◽  
Novel Approach

This letter proposes a novel approach using the [Formula: see text]-norm regularization for the sparse covariance matrix estimation (SCME) problem. The objective function of SCME problem is composed of a nonconvex part and the [Formula: see text] term, which is discontinuous and difficult to tackle. Appropriate DC (difference of convex functions) approximations of [Formula: see text]-norm are used that result in approximation SCME problems that are still nonconvex. DC programming and DCA (DC algorithm), powerful tools in nonconvex programming framework, are investigated. Two DC formulations are proposed and corresponding DCA schemes developed. Two applications of the SCME problem that are considered are classification via sparse quadratic discriminant analysis and portfolio optimization. A careful empirical experiment is performed through simulated and real data sets to study the performance of the proposed algorithms. Numerical results showed their efficiency and their superiority compared with seven state-of-the-art methods.

scholarly journals Porter 5: state-of-the-art ab initio prediction of protein secondary structure in 3 and 8 classes

2018 ◽  
Author(s):  
Mirko Torrisi ◽  
Manaz Kaleel ◽  
Gianluca Pollastri
Keyword(s):  
Secondary Structure ◽  
Ab Initio ◽  
State Of The Art ◽  
Protein Secondary Structure ◽  
Independent Set ◽  
Machine Learning Techniques ◽  
Supplementary Information ◽  
Data Sets ◽  
Learning Techniques ◽  
Ab Initio Prediction

AbstractMotivationAlthough Secondary Structure Predictors have been developed for more than 60 years, current ab initio methods have still some way to go to reach their theoretical limits. Moreover, the continuous effort towards harnessing ever increasing data sets and more sophisticated, deeper Machine Learning techniques, has not come to an end.ResultsHere we present Porter 5, the last release of one of the best performing ab initio secondary structure predictor. Version 5 achieves 84% accuracy (84% SOV) when tested on 3 classes, and 73% accuracy (82% SOV) on 8 classes, on a large independent set, significantly outperforming all the most recent ab initio predictors we have tested.AvailabilityThe web and standalone versions of Porter5 are available at http://distilldeep.ucd.ie/[email protected] informationSupplementary data are available at Bioinformatics online.

ARTIFICIAL METAPLASTICITY NEURAL NETWORK APPLIED TO CREDIT SCORING

2011 ◽  
Vol 21 (04) ◽  
pp. 311-317 ◽  
Author(s):  
ALEXIS MARCANO-CEDEÑO ◽  
A. MARIN-DE-LA-BARCENA ◽  
J. JIMENEZ-TRILLO ◽  
J. A. PIÑUELA ◽  
D. ANDINA
Keyword(s):  
Neural Network ◽  
Financial Institutions ◽  
State Of The Art ◽  
Credit Scoring ◽  
Error Rates ◽  
Classification Algorithms ◽  
Data Sets ◽  
Training Algorithm ◽  
Low Probability ◽  
German Data

The assessment of the risk of default on credit is important for financial institutions. Different Artificial Neural Networks (ANN) have been suggested to tackle the credit scoring problem, however, the obtained error rates are often high. In the search for the best ANN algorithm for credit scoring, this paper contributes with the application of an ANN Training Algorithm inspired by the neurons' biological property of metaplasticity. This algorithm is especially efficient when few patterns of a class are available, or when information inherent to low probability events is crucial for a successful application, as weight updating is overemphasized in the less frequent activations than in the more frequent ones. Two well-known and readily available such as: Australia and German data sets has been used to test the algorithm. The results obtained by AMMLP shown have been superior to state-of-the-art classification algorithms in credit scoring.

A cross-entropy based stacking method in ensemble learning

2020 ◽  
Vol 39 (3) ◽  
pp. 4677-4688
Author(s):  
Weimin Ding ◽  
Shengli Wu
Keyword(s):  
Ensemble Learning ◽  
Gradient Descent ◽  
State Of The Art ◽  
Classification Problem ◽  
Cross Entropy ◽  
Stochastic Gradient Descent ◽  
Data Sets ◽  
Class Label ◽  
Meta Level ◽  
Learning Techniques

Stacking is one of the major types of ensemble learning techniques in which a set of base classifiers contributes their outputs to the meta-level classifier, and the meta-level classifier combines them so as to produce more accurate classifications. In this paper, we propose a new stacking algorithm that defines the cross-entropy as the loss function for the classification problem. The training process is conducted by using a neural network with the stochastic gradient descent technique. One major characteristic of our method is its treatment of each meta instance as a whole with one optimization model, which is different from some other stacking methods such as stacking with multi-response linear regression and stacking with multi-response model trees. In these methods each meta instance is divided into a set of sub-instances. Multiple models apply to those sub-instances and each for a class label. There is no connection between different models. It is very likely that our treatment is a better choice for finding suitable weights. Experiments with 22 data sets from the UCI machine learning repository show that the proposed stacking approach performs well. It outperforms all three base classifiers, several state-of-the-art stacking algorithms, and some other representative ensemble learning methods on average.

Block Clustering Based on Difference of Convex Functions (DC) Programming and DC Algorithms

2013 ◽  
Vol 25 (10) ◽  
pp. 2776-2807 ◽  
Author(s):  
Hoai Minh Le ◽  
Hoai An Le Thi ◽  
Tao Pham Dinh ◽  
Van Ngai Huynh
Keyword(s):  
Convex Functions ◽  
Optimization Problem ◽  
Dc Programming ◽  
Clustering Problem ◽  
Block Classification ◽  
Dc Algorithm ◽  
Difference Of Convex Functions ◽  
Dc Program ◽  
Difference Of Convex ◽  
Block Clustering

We investigate difference of convex functions (DC) programming and the DC algorithm (DCA) to solve the block clustering problem in the continuous framework, which traditionally requires solving a hard combinatorial optimization problem. DC reformulation techniques and exact penalty in DC programming are developed to build an appropriate equivalent DC program of the block clustering problem. They lead to an elegant and explicit DCA scheme for the resulting DC program. Computational experiments show the robustness and efficiency of the proposed algorithm and its superiority over standard algorithms such as two-mode K-means, two-mode fuzzy clustering, and block classification EM.

scholarly journals Material Recognition Using Deep Learning Techniques

2019 ◽  
Author(s):  
Nampally Tejasri ◽  
Mongkol Ekapanyapong
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Fine Tuning ◽  
Data Sets ◽  
Network Architectures ◽  
Data Set ◽  
Material Recognition ◽  
Learning Techniques ◽  
Computer Vision Systems ◽  
Fully Connected

The classification and recognition of variety of materials that are present in our surroundings be-come an important visual competition have been focused by computer vision systems in the recentyears. Understanding the recognition of the materials in different images that involve a deep learn-ing process made use of the recent development in the field of Artificial Neural Networks broughtthe ability to train various neural network architectures for the extraction of features for this chal-lenging task. In this work, state-of-the-art Convolutional Neural Network (CNN) techniques areused to classify materials and also compare the results obtained by them.The results are gath-ered over two material data sets applying the two popular approaches of Transfer Learning. Theresults showcase that fine-tuning approach achieves very good results compared to the case of ap-proach when the information derived from the layer which is just before the fully connected layeris limited. The results of the comparison indicates the fact that there is an improvement in theperformance and the accuracy of the system particularly in the data set that contains large numberof images.

Sign in / Sign up

Export Citation Format

Share Document