A loss minimization problem

2009 ◽  
Vol 33 (2) ◽  
pp. 67-76
Author(s):  
K. K. Osipenko
Keyword(s):  
Minimization Problem ◽  
Loss Minimization

scholarly journals Using Benson’s Algorithm for Regularization Parameter Tracking

2019 ◽  
Vol 33 ◽  
pp. 3689-3696
Author(s):  
Joachim Giesen ◽  
Sӧren Laue ◽  
Andreas Lӧhne ◽  
Christopher Schneider
Keyword(s):  
Approximate Solution ◽  
Minimization Problem ◽  
Regularization Parameter ◽  
Approximate Solutions ◽  
Loss Minimization ◽  
Real World Data ◽  
Approximation Accuracy ◽  
Parameter Domain ◽  
Parameter Tracking ◽  
Regularization Parameters

Regularized loss minimization, where a statistical model is obtained from minimizing the sum of a loss function and weighted regularization terms, is still in widespread use in machine learning. The statistical performance of the resulting models depends on the choice of weights (regularization parameters) that are typically tuned by cross-validation. For finding the best regularization parameters, the regularized minimization problem needs to be solved for the whole parameter domain. A practically more feasible approach is covering the parameter domain with approximate solutions of the loss minimization problem for some prescribed approximation accuracy. The problem of computing such a covering is known as the approximate solution gamut problem. Existing algorithms for the solution gamut problem suffer from several problems. For instance, they require a grid on the parameter domain whose spacing is difficult to determine in practice, and they are not generic in the sense that they rely on problem specific plug-in functions. Here, we show that a well-known algorithm from vector optimization, namely the Benson algorithm, can be used directly for computing approximate solution gamuts while avoiding the problems of existing algorithms. Experiments for the Elastic Net on real world data sets demonstrate the effectiveness of Benson’s algorithm for regularization parameter tracking.

scholarly journals Reliable Multilabel Classification: Prediction with Partial Abstention

2020 ◽  
Vol 34 (04) ◽  
pp. 5264-5271 ◽  
Author(s):  
Vu-Linh Nguyen ◽  
Eyke Hullermeier
Keyword(s):  
Minimization Problem ◽  
Loss Minimization ◽  
Class Label ◽  
Single Class ◽  
Multilabel Classification ◽  
First Results ◽  
Classification Prediction ◽  
Class Labels ◽  
F Measure

In contrast to conventional (single-label) classification, the setting of multilabel classification (MLC) allows an instance to belong to several classes simultaneously. Thus, instead of selecting a single class label, predictions take the form of a subset of all labels. In this paper, we study an extension of the setting of MLC, in which the learner is allowed to partially abstain from a prediction, that is, to deliver predictions on some but not necessarily all class labels. We propose a formalization of MLC with abstention in terms of a generalized loss minimization problem and present first results for the case of the Hamming loss, rank loss, and F-measure, both theoretical and experimental.

scholarly journals Gradient-based Learning Methods Extended to Smooth Manifolds Applied to Automated Clustering

2020 ◽  
Vol 68 ◽  
pp. 777-816
Author(s):  
Alkis Koudounas ◽  
Simone Fiori
Keyword(s):  
Minimization Problem ◽  
Spectral Clustering ◽  
Learning Algorithms ◽  
Optimization Methods ◽  
Smooth Manifolds ◽  
Loss Minimization ◽  
Parameter Spaces ◽  
Manifold Optimization ◽  
Gradient Based ◽  
Smooth Constraints

Grassmann manifold based sparse spectral clustering is a classification technique that  consists in learning a latent representation of data, formed by a subspace basis, which  is sparse. In order to learn a latent representation, spectral clustering is formulated in  terms of a loss minimization problem over a smooth manifold known as Grassmannian.  Such minimization problem cannot be tackled by one of traditional gradient-based learning  algorithms, which are only suitable to perform optimization in absence of constraints among  parameters. It is, therefore, necessary to develop specific optimization/learning algorithms  that are able to look for a local minimum of a loss function under smooth constraints in  an efficient way. Such need calls for manifold optimization methods. In this paper, we  extend classical gradient-based learning algorithms on   at parameter spaces (from classical  gradient descent to adaptive momentum) to curved spaces (smooth manifolds) by means  of tools from manifold calculus. We compare clustering performances of these methods  and known methods from the scientific literature. The obtained results confirm that the  proposed learning algorithms prove lighter in computational complexity than existing ones  without detriment in clustering efficacy.

An Augmented Iteratively Re-weighted and Refined Least Squares Method for lp Minimization Problem in Inverse Modeling of Potential Field Magnetic Data

2020 ◽  
Vol 1 (3) ◽  
Author(s):  
Maysam Abedi
Keyword(s):  
Magnetic Field ◽  
Magnetic Susceptibility ◽  
Least Squares ◽  
Minimization Problem ◽  
Potential Field ◽  
Field Data ◽  
Least Squares Method ◽  
Porphyry Copper ◽  
Magnetic Data ◽  
Magnetic Field Data

The presented work examines application of an Augmented Iteratively Re-weighted and Refined Least Squares method (AIRRLS) to construct a 3D magnetic susceptibility property from potential field magnetic anomalies. This algorithm replaces an lp minimization problem by a sequence of weighted linear systems in which the retrieved magnetic susceptibility model is successively converged to an optimum solution, while the regularization parameter is the stopping iteration numbers. To avoid the natural tendency of causative magnetic sources to concentrate at shallow depth, a prior depth weighting function is incorporated in the original formulation of the objective function. The speed of lp minimization problem is increased by inserting a pre-conditioner conjugate gradient method (PCCG) to solve the central system of equation in cases of large scale magnetic field data. It is assumed that there is no remanent magnetization since this study focuses on inversion of a geological structure with low magnetic susceptibility property. The method is applied on a multi-source noise-corrupted synthetic magnetic field data to demonstrate its suitability for 3D inversion, and then is applied to a real data pertaining to a geologically plausible porphyry copper unit.  The real case study located in  Semnan province of  Iran  consists  of  an arc-shaped  porphyry  andesite  covered  by  sedimentary  units  which  may  have  potential  of  mineral  occurrences, especially  porphyry copper. It is demonstrated that such structure extends down at depth, and consequently exploratory drilling is highly recommended for acquiring more pieces of information about its potential for ore-bearing mineralization.

Sign in / Sign up

Export Citation Format

Share Document