scholarly journals Multilabel Classification with Partial Abstention: Bayes-Optimal Prediction under Label Independence

2021 ◽  
Vol 72 ◽  
pp. 613-665
Author(s):  
Vu-Linh Nguyen ◽  
Eyke Hüllermeier
Keyword(s):  
Optimization Problem ◽  
Building Blocks ◽  
Loss Functions ◽  
Optimal Prediction ◽  
Expected Loss ◽  
Class Label ◽  
Single Class ◽  
Multilabel Classification ◽  
Formal Framework ◽  
Class Labels

In contrast to conventional (single-label) classification, the setting of multilabel classification (MLC) allows an instance to belong to several classes simultaneously. Thus, instead of selecting a single class label, predictions take the form of a subset of all labels. In this paper, we study an extension of the setting of MLC, in which the learner is allowed to partially abstain from a prediction, that is, to deliver predictions on some but not necessarily all class labels. This option is useful in cases of uncertainty, where the learner does not feel confident enough on the entire label set. Adopting a decision-theoretic perspective, we propose a formal framework of MLC with partial abstention, which builds on two main building blocks: First, the extension of underlying MLC loss functions so as to accommodate abstention in a proper way, and second the problem of optimal prediction, that is, finding the Bayes-optimal prediction minimizing this generalized loss in expectation. It is well known that different (generalized) loss functions may have different risk-minimizing predictions, and finding the Bayes predictor typically comes down to solving a computationally complexity optimization problem. In the most general case, given a prediction of the (conditional) joint distribution of possible labelings, the minimizer of the expected loss needs to be found over a number of candidates which is exponential in the number of class labels. We elaborate on properties of risk minimizers for several commonly used (generalized) MLC loss functions, show them to have a specific structure, and leverage this structure to devise efficient methods for computing Bayes predictors. Experimentally, we show MLC with partial abstention to be effective in the sense of reducing loss when being allowed to abstain.

scholarly journals Reliable Multilabel Classification: Prediction with Partial Abstention

2020 ◽  
Vol 34 (04) ◽  
pp. 5264-5271 ◽  
Author(s):  
Vu-Linh Nguyen ◽  
Eyke Hullermeier
Keyword(s):  
Minimization Problem ◽  
Loss Minimization ◽  
Class Label ◽  
Single Class ◽  
Multilabel Classification ◽  
First Results ◽  
Classification Prediction ◽  
Class Labels ◽  
F Measure

In contrast to conventional (single-label) classification, the setting of multilabel classification (MLC) allows an instance to belong to several classes simultaneously. Thus, instead of selecting a single class label, predictions take the form of a subset of all labels. In this paper, we study an extension of the setting of MLC, in which the learner is allowed to partially abstain from a prediction, that is, to deliver predictions on some but not necessarily all class labels. We propose a formalization of MLC with abstention in terms of a generalized loss minimization problem and present first results for the case of the Hamming loss, rank loss, and F-measure, both theoretical and experimental.

BiLabel-Specific Features for Multi-Label Classification

2021 ◽  
Vol 16 (1) ◽  
pp. 1-23
Author(s):  
Min-Ling Zhang ◽  
Jun-Peng Fang ◽  
Yi-Bo Wang
Keyword(s):  
Predictive Models ◽  
Comparative Studies ◽  
State Of The Art ◽  
Classification Model ◽  
Generation Process ◽  
Prototype Selection ◽  
Class Label ◽  
Benchmark Datasets ◽  
Label Correlations ◽  
Class Labels

In multi-label classification, the task is to induce predictive models which can assign a set of relevant labels for the unseen instance. The strategy of label-specific features has been widely employed in learning from multi-label examples, where the classification model for predicting the relevancy of each class label is induced based on its tailored features rather than the original features. Existing approaches work by generating a group of tailored features for each class label independently, where label correlations are not fully considered in the label-specific features generation process. In this article, we extend existing strategy by proposing a simple yet effective approach based on BiLabel-specific features. Specifically, a group of tailored features is generated for a pair of class labels with heuristic prototype selection and embedding. Thereafter, predictions of classifiers induced by BiLabel-specific features are ensembled to determine the relevancy of each class label for unseen instance. To thoroughly evaluate the BiLabel-specific features strategy, extensive experiments are conducted over a total of 35 benchmark datasets. Comparative studies against state-of-the-art label-specific features techniques clearly validate the superiority of utilizing BiLabel-specific features to yield stronger generalization performance for multi-label classification.

scholarly journals Bayes Estimation of a Two-Parameter Geometric Distribution under Multiply Type II Censoring

2011 ◽  
Vol 2011 ◽  
pp. 1-10
Author(s):  
J. B. Shah ◽  
M. N. Patel
Keyword(s):  
Geometric Distribution ◽  
Correct Choice ◽  
Bayes Estimation ◽  
Loss Functions ◽  
Type Ii ◽  
Expected Loss ◽  
Bayes Estimators ◽  
Linex Loss ◽  
Type Ii Censored Data ◽  
Two Parameter

We derive Bayes estimators of reliability and the parameters of a two- parameter geometric distribution under the general entropy loss, minimum expected loss and linex loss, functions for a noninformative as well as beta prior from multiply Type II censored data. We have studied the robustness of the estimators using simulation and we observed that the Bayes estimators of reliability and the parameters of a two-parameter geometric distribution under all the above loss functions appear to be robust with respect to the correct choice of the hyperparameters a(b) and a wrong choice of the prior parameters b(a) of the beta prior.

scholarly journals Robustness in the Optimization of Risk Measures

2021 ◽  
Author(s):  
Paul Embrechts ◽  
Alexander Schied ◽  
Ruodu Wang
Keyword(s):  
Value At Risk ◽  
Optimization Problem ◽  
Financial Regulation ◽  
Optimization Problems ◽  
Risk Measures ◽  
Risk Measurement ◽  
Loss Functions ◽  
Insurance Regulation ◽  
Convex Risk Measures ◽  
Comparative Advantages

We study issues of robustness in the context of Quantitative Risk Management and Optimization. We develop a general methodology for determining whether a given risk-measurement-related optimization problem is robust, which we call “robustness against optimization.” The new notion is studied for various classes of risk measures and expected utility and loss functions. Motivated by practical issues from financial regulation, special attention is given to the two most widely used risk measures in the industry, Value-at-Risk (VaR) and Expected Shortfall (ES). We establish that for a class of general optimization problems, VaR leads to nonrobust optimizers, whereas convex risk measures generally lead to robust ones. Our results offer extra insight on the ongoing discussion about the comparative advantages of VaR and ES in banking and insurance regulation. Our notion of robustness is conceptually different from the field of robust optimization, to which some interesting links are derived.

A New Adaptive Penalty Method for Constrained Genetic Algorithm and Its Application to Water Distribution Systems

2006 ◽  
Author(s):  
Berge Djebedjian ◽  
Ashraf Yaseen ◽  
Magdy Abou Rayan
Keyword(s):  
Genetic Algorithm ◽  
Penalty Function ◽  
Penalty Method ◽  
Distribution Systems ◽  
Water Distribution ◽  
Optimization Problem ◽  
Water Distribution Systems ◽  
Building Blocks ◽  
Adaptive Penalty ◽  
Adaptive Penalty Method

This paper presents a new adaptive penalty method for genetic algorithms (GA). External penalty functions have been used to convert a constrained optimization problem into an unconstrained problem for GA-based optimization. The success of the genetic algorithm application to the design of water distribution systems depends on the choice of the penalty function. The optimal design of water distribution systems is a constrained non-linear optimization problem. Constraints (for example, the minimum pressure requirements at the nodes) are generally handled within genetic algorithm optimization by introducing a penalty cost function. The optimal solution is found when the pressures at some nodes are close to the minimum required pressure. The goal of an adaptive penalty function is to change the value of the penalty draw-down coefficient during the search allowing exploration of infeasible regions to find optimal building blocks, while preserving the feasibility of the final solution. In this study, a new penalty coefficient strategy is assumed to increase with the total cost at each generation and inversely with the total number of nodes. The application of the computer program to case studies shows that it finds the least cost in a favorable number of function evaluations if not less than that in previous studies and it is computationally much faster when compared with other studies.

scholarly journals Analysis of loss functions for fast single-class classification

2019 ◽  
Vol 62 (1) ◽  
pp. 337-358 ◽  
Author(s):  
Gil Keren ◽  
Sivan Sabato ◽  
Björn Schuller
Keyword(s):  
Loss Functions ◽  
Single Class

Vector space model for patent documents with hierarchical class labels

2012 ◽  
Vol 38 (3) ◽  
pp. 222-233 ◽  
Author(s):  
Yen-Liang Chen ◽  
Yu-Ting Chiu
Keyword(s):  
Feature Selection ◽  
Vector Space ◽  
Selection Process ◽  
Vector Space Model ◽  
Class Label ◽  
Discriminative Ability ◽  
New Approach ◽  
Space Model ◽  
Patent Documents ◽  
Class Labels

A vector space model (VSM) composed of selected important features is a common way to represent documents, including patent documents. Patent documents have some special characteristics that make it difficult to apply traditional feature selection methods directly: (a) it is difficult to find common terms for patent documents in different categories; and (b) the class label of a patent document is hierarchical rather than flat. Hence, in this article we propose a new approach that includes a hierarchical feature selection (HFS) algorithm which can be used to select more representative features with greater discriminative ability to present a set of patent documents with hierarchical class labels. The performance of the proposed method is evaluated through application to two documents sets with 2400 and 9600 patent documents, where we extract candidate terms from their titles and abstracts. The experimental results reveal that a VSM whose features are selected by a proportional selection process gives better coverage, while a VSM whose features are selected with a weighted-summed selection process gives higher accuracy.

scholarly journals Query-Driven Multi-Instance Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 4158-4165
Author(s):  
Yen-Chi Hsu ◽  
Cheng-Yao Hong ◽  
Ming-Sui Lee ◽  
Tyng-Luh Liu
Keyword(s):  
Experimental Results ◽  
Weighted Sum ◽  
Class Label ◽  
Action Classification ◽  
Video Clips ◽  
A New Technique ◽  
Class Labels ◽  
The Given ◽  
Network Component ◽  
Generalized Compatibility

We introduce a query-driven approach (qMIL) to multi-instance learning where the queries aim to uncover the class labels embodied in a given bag of instances. Specifically, it solves a multi-instance multi-label learning (MIML) problem with a more challenging setting than the conventional one. Each MIML bag in our formulation is annotated only with a binary label indicating whether the bag contains the instance of a certain class and the query is specified by the word2vec of a class label/name. To learn a deep-net model for qMIL, we construct a network component that achieves a generalized compatibility measure for query-visual co-embedding and yields proper instance attentions to the given query. The bag representation is then formed as the attention-weighted sum of the instances' weights, and passed to the classification layer at the end of the network. In addition, the qMIL formulation is flexible for extending the network to classify unseen class labels, leading to a new technique to solve the zero-shot MIML task through an iterative querying process. Experimental results on action classification over video clips and three MIML datasets from MNIST, CIFAR10 and Scene are provided to demonstrate the effectiveness of our method.

Multi-contact bipedal robotic locomotion

Robotica ◽  
2015 ◽  
Vol 35 (5) ◽  
pp. 1072-1106 ◽  
Author(s):  
Huihua Zhao ◽  
Ayonga Hereid ◽  
Wen-loong Ma ◽  
Aaron D. Ames
Keyword(s):  
Optimization Problem ◽  
Human Locomotion ◽  
System Model ◽  
Human Model ◽  
Slip Model ◽  
Robotic Locomotion ◽  
Walking Gait ◽  
Trajectory Reconstruction ◽  
Formal Framework ◽  
Reconstruction Strategy

SUMMARYThis paper presents a formal framework for achieving multi-contact bipedal robotic walking, and realizes this methodology experimentally on two robotic platforms: AMBER2 and Assume The Robot Is A Sphere (ATRIAS). Inspired by the key feature encoded in human walking—multi-contact behavior—this approach begins with the analysis of human locomotion and uses it to motivate the construction of a hybrid system model representing a multi-contact robotic walking gait. Human-inspired outputs are extracted from reference locomotion data to characterize the human model or the spring-loaded invert pendulum (SLIP) model, and then employed to develop the human-inspired control and an optimization problem that yields stable multi-domain walking. Through a trajectory reconstruction strategy motivated by the process that generates the walking gait, the mathematical constructions are successfully translated to the two physical robots experimentally.

Impact of PDS Based kNN Classifiers on Kyoto Dataset

2019 ◽  
Vol 6 (2) ◽  
pp. 61-72
Author(s):  
Kailasam Swathi ◽  
Bobba Basaveswara Rao
Keyword(s):  
Correlation Coefficient ◽  
Intrusion Detection Systems ◽  
Computational Time ◽  
Network Intrusion Detection ◽  
Class Label ◽  
Detection Systems ◽  
Network Intrusion ◽  
Network Intrusion Detection Systems ◽  
Significant Difference ◽  
Class Labels

This article compares the performance of different Partial Distance Search-based (PDS) kNN classifiers on a benchmark Kyoto 2006+ dataset for Network Intrusion Detection Systems (NIDS). These PDS classifiers are named based on features indexing. They are: i) Simple PDS kNN, the features are not indexed (SPDS), ii) Variance indexing based kNN (VIPDS), the features are indexed by the variance of the features, and iii) Correlation coefficient indexing-based kNN (CIPDS), the features are indexed by the correlation coefficient of the features with a class label. For comparative study between these classifiers, the computational time and accuracy are considered performance measures. After the experimental study, it is observed that the CIPDS gives better performance in terms of computational time whereas VIPDS shows better accuracy, but not much significant difference when compared with CIPDS. The study suggests to adopt CIPDS when class labels were available without any ambiguity, otherwise it suggested the adoption of VIPDS.

Sign in / Sign up

Export Citation Format

Share Document