Unified Performance Measure for Binary Classification Problems

2020 ◽  
pp. 104-112
Author(s):  
Ana R. Redondo ◽  
Jorge Navarro ◽  
Rubén R. Fernández ◽  
Isaac Martín de Diego ◽  
Javier M. Moguerza ◽  
...  
Keyword(s):  
Binary Classification ◽  
Performance Measure ◽  
Classification Problems

scholarly journals Improving Land Cover Classification Using Genetic Programming for Feature Construction

2021 ◽  
Vol 13 (9) ◽  
pp. 1623
Author(s):  
João E. Batista ◽  
Ana I. R. Cabral ◽  
Maria J. P. Vasconcelos ◽  
Leonardo Vanneschi ◽  
Sara Silva
Keyword(s):  
Land Cover ◽  
Genetic Programming ◽  
Satellite Images ◽  
State Of The Art ◽  
Binary Classification ◽  
Feature Construction ◽  
Classification Problems ◽  
Construction Methods ◽  
Box Models

Genetic programming (GP) is a powerful machine learning (ML) algorithm that can produce readable white-box models. Although successfully used for solving an array of problems in different scientific areas, GP is still not well known in the field of remote sensing. The M3GP algorithm, a variant of the standard GP algorithm, performs feature construction by evolving hyperfeatures from the original ones. In this work, we use the M3GP algorithm on several sets of satellite images over different countries to create hyperfeatures from satellite bands to improve the classification of land cover types. We add the evolved hyperfeatures to the reference datasets and observe a significant improvement of the performance of three state-of-the-art ML algorithms (decision trees, random forests, and XGBoost) on multiclass classifications and no significant effect on the binary classifications. We show that adding the M3GP hyperfeatures to the reference datasets brings better results than adding the well-known spectral indices NDVI, NDWI, and NBR. We also compare the performance of the M3GP hyperfeatures in the binary classification problems with those created by other feature construction methods such as FFX and EFS.

scholarly journals Confidence interval for micro-averaged F1 and macro-averaged F1 scores

2021 ◽  
Author(s):  
Kanae Takahashi ◽  
Kouji Yamamoto ◽  
Aya Kuchiba ◽  
Tatsuki Koyama
Keyword(s):  
Binary Classification ◽  
Classification Problem ◽  
Classification Problems ◽  
Summary Measure ◽  
Medical Field ◽  
Predictive Values ◽  
Binary Classification Problem ◽  
Multi Class Classification ◽  
Sensitivity Specificity ◽  
Measures Of Performance

AbstractA binary classification problem is common in medical field, and we often use sensitivity, specificity, accuracy, negative and positive predictive values as measures of performance of a binary predictor. In computer science, a classifier is usually evaluated with precision (positive predictive value) and recall (sensitivity). As a single summary measure of a classifier’s performance, F1 score, defined as the harmonic mean of precision and recall, is widely used in the context of information retrieval and information extraction evaluation since it possesses favorable characteristics, especially when the prevalence is low. Some statistical methods for inference have been developed for the F1 score in binary classification problems; however, they have not been extended to the problem of multi-class classification. There are three types of F1 scores, and statistical properties of these F1 scores have hardly ever been discussed. We propose methods based on the large sample multivariate central limit theorem for estimating F1 scores with confidence intervals.

Enhancing Vibration-Based Structural Health Monitoring via Edge Computing: A Tiny Machine Learning Perspective

2021 ◽  
Author(s):  
Federica Zonzini ◽  
Francesca Romano ◽  
Antonio Carbone ◽  
Matteo Zauli ◽  
Luca De Marchi
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Structural Health Monitoring ◽  
Health Monitoring ◽  
Network Architecture ◽  
Binary Classification ◽  
Low Cost ◽  
Classification Problems ◽  
Computationally Efficient ◽  
Structural Health

Abstract Despite the outstanding improvements achieved by artificial intelligence in the Structural Health Monitoring (SHM) field, some challenges need to be coped with. Among them, the necessity to reduce the complexity of the models and the data-to-user latency time which are still affecting state-of-the-art solutions. This is due to the continuous forwarding of a huge amount of data to centralized servers, where the inference process is usually executed in a bulky manner. Conversely, the emerging field of Tiny Machine Learning (TinyML), promoted by the recent advancements by the electronic and information engineering community, made sensor-near data inference a tangible, low-cost and computationally efficient alternative. In line with this observation, this work explored the embodiment of the One Class Classifier Neural Network, i.e., a neural network architecture solving binary classification problems for vibration-based SHM scenarios, into a resource-constrained device. To this end, OCCNN has been ported on the Arduino Nano 33 BLE Sense platform and validated with experimental data from the Z24 bridge use case, reaching an average accuracy and precision of 95% and 94%, respectively.

scholarly journals An Agent-Ensemble for Thresholded Multi-Target Classification

2020 ◽  
Vol 10 (4) ◽  
pp. 1376
Author(s):  
Nathan H. Parrish ◽  
Ashley J. Llorens ◽  
Alex E. Driskell
Keyword(s):  
Binary Classification ◽  
Target Type ◽  
Vehicle Classification ◽  
Weighted Likelihood ◽  
Target Classification ◽  
Classification Problems ◽  
Target Class ◽  
Combination Strategy ◽  
Individual Agent ◽  
Ensemble Approach

We propose an ensemble approach for multi-target binary classification, where the target class breaks down into a disparate set of pre-defined target-types. The system goal is to maximize the probability of alerting on targets from any type while excluding background clutter. The agent-classifiers that make up the ensemble are binary classifiers trained to classify between one of the target-types vs. clutter. The agent ensemble approach offers several benefits for multi-target classification including straightforward in-situ tuning of the ensemble to drift in the target population and the ability to give an indication to a human operator of which target-type causes an alert. We propose a combination strategy that sums weighted likelihood ratios of the individual agent-classifiers, where the likelihood ratio is between the target-type for the agent vs. clutter. We show that this combination strategy is optimal under a conditionally non-discriminative assumption. We compare this combiner to the common strategy of selecting the maximum of the normalized agent-scores as the combiner score. We show experimentally that the proposed combiner gives excellent performance on the multi-target binary classification problems of pin-less verification of human faces and vehicle classification using acoustic signatures.

Sign in / Sign up

Export Citation Format

Share Document