scholarly journals Everything has its Price: Foundations of Cost-Sensitive Learning and its Application in Psychology

2021 ◽  
Author(s):  
Philipp Sterner ◽  
David Goretzko ◽  
Florian Pargent
Keyword(s):  
Machine Learning ◽  
False Positive ◽  
Binary Classification ◽  
False Negative ◽  
Classification Algorithms ◽  
Practical Application ◽  
Cost Sensitive Learning ◽  
Misclassification Costs ◽  
Mathematical Foundations ◽  
Open Materials

Psychology has seen an increase in machine learning (ML) methods. In many applications, observations are classified into one of two groups (binary classification). Off-the-shelf classification algorithms assume that the costs of a misclassification (false-positive or false-negative) are equal. Because this is often not reasonable (e.g., in clinical psychology), cost-sensitive learning (CSL) methods can take different cost ratios into account. We present the mathematical foundations and introduce a taxonomy of the most commonly used CSL methods, before demonstrating their application and usefulness on psychological data, i.e., the drug consumption dataset ($N = 1885$) from the UCI Machine Learning Repository. In our example, all demonstrated CSL methods noticeably reduce mean misclassification costs compared to regular ML algorithms. We discuss the necessity for researchers to perform small benchmarks of CSL methods for their own practical application. Thus, our open materials provide R code, demonstrating how CSL methods can be applied within the mlr3 framework (https://osf.io/cvks7/).

Cost-Sensitive Learning in Medicine

2011 ◽  
pp. 57-75
Author(s):  
Alberto Freitas ◽  
Pavel Brazdil ◽  
Altamiro Costa-Pereira
Keyword(s):  
Diagnostic Tests ◽  
False Negative ◽  
Cost Sensitive Learning ◽  
Safety Issues ◽  
Misclassification Costs ◽  
Misclassification Errors ◽  
Cost Sensitive Classification ◽  
Human Costs ◽  
The Cost ◽  
Specific Diagnostic Test

This chapter introduces cost-sensitive learning and its importance in medicine. Health managers and clinicians often need models that try to minimize several types of costs associated with healthcare, including attribute costs (e.g. the cost of a specific diagnostic test) and misclassification costs (e.g. the cost of a false negative test). In fact, as in other professional areas, both diagnostic tests and its associated misclassification errors can have significant financial or human costs, including the use of unnecessary resource and patient safety issues. This chapter presents some concepts related to cost-sensitive learning and cost-sensitive classification and its application to medicine. Different types of costs are also present, with an emphasis on diagnostic tests and misclassification costs. In addition, an overview of research in the area of cost-sensitive learning is given, including current methodological approaches. Finally, current methods for the cost-sensitive evaluation of classifiers are discussed.

scholarly journals The evaluation of binary classification tasks in economical prediction

2010 ◽  
Vol 58 (6) ◽  
pp. 369-378
Author(s):  
Martin Pokorný
Keyword(s):  
False Positive ◽  
Roc Analysis ◽  
Binary Classification ◽  
False Negative ◽  
Optimal Threshold ◽  
False Negatives ◽  
Classifier Performance ◽  
False Negative Error ◽  
Classification Tasks ◽  
Error Costs

In the area of economical classification tasks, the accuracy maximization is often used to evaluate classifier performance. Accuracy maximization (or error rate minimization) suffers from the assumption of equal false positive and false negative error costs. Furthermore, accuracy is not able to express true classifier performance under skewed class distribution. Due to these limitations, the use of accuracy on real tasks is questionable. In a real binary classification task, the difference between the costs of false positive and false negative error is usually critical. To overcome this issue, the Receiver Ope­rating Characteristic (ROC) method in relation to decision-analytic principles can be used. One essential advantage of this method is the possibility of classifier performance visualization by means of a ROC graph. This paper presents concrete examples of binary classification, where the inadequacy of accuracy as the evaluation metric is shown, and on the same examples the ROC method is applied. From the set of possible classification models, the probabilistic classifier with continuous output is under consideration. Mainly two questions are solved. Firstly, the selection of the best classifier from a set of possible classifiers. For example, accuracy metric rates two classifiers almost equiva­lently (87.7 % and 89.3 %), whereas decision analysis (via costs minimization) or ROC analysis reveal differe­nt performance according to target conditions of unequal error costs of false positives and false negatives. Secondly, the setting of an optimal decision threshold at classifier’s output. For example, accuracy maximization finds the optimal threshold at classifier’s output in value of 0.597, but the optimal threshold respecting higher costs of false negatives is discovered by costs minimization or ROC analysis in a value substantially lower (0.477).

scholarly journals Improved machine-learning-based open-water–sea-ice–cloud discrimination over wintertime Antarctic sea ice using MODIS thermal-infrared imagery

2021 ◽  
Vol 15 (3) ◽  
pp. 1551-1565
Author(s):  
Stephan Paul ◽  
Marcus Huntemann
Keyword(s):  
Machine Learning ◽  
Sea Ice ◽  
False Positive ◽  
Cloud Cover ◽  
Near Infrared ◽  
False Negative ◽  
Open Water ◽  
Thermal Infrared ◽  
Polar Regions ◽  
Spatial Coverage

Abstract. The frequent presence of cloud cover in polar regions limits the use of the Moderate Resolution Imaging Spectroradiometer (MODIS) and similar instruments for the investigation and monitoring of sea-ice polynyas compared to passive-microwave-based sensors. The very low thermal contrast between present clouds and the sea-ice surface in combination with the lack of available visible and near-infrared channels during polar nighttime results in deficiencies in the MODIS cloud mask and dependent MODIS data products. This leads to frequent misclassifications of (i) present clouds as sea ice or open water (false negative) and (ii) open-water and/or thin-ice areas as clouds (false positive), which results in an underestimation of actual polynya area and subsequently derived information. Here, we present a novel machine-learning-based approach using a deep neural network that is able to reliably discriminate between clouds, sea-ice, and open-water and/or thin-ice areas in a given swath solely from thermal-infrared MODIS channels and derived additional information. Compared to the reference MODIS sea-ice product for the year 2017, our data result in an overall increase of 20 % in annual swath-based coverage for the Brunt Ice Shelf polynya, attributed to an improved cloud-cover discrimination and the reduction of false-positive classifications. At the same time, the mean annual polynya area decreases by 44 % through the reduction of false-negative classifications of warm clouds as thin ice. Additionally, higher spatial coverage results in an overall better subdaily representation of thin-ice conditions that cannot be reconstructed with current state-of-the-art cloud-cover compensation methods.

scholarly journals Assessment of Neuronal Damage in Brain Slice Cultures Using Machine Learning Based on Spatial Features

2021 ◽  
Vol 15 ◽  
Author(s):  
Urszula Hohmann ◽  
Faramarz Dehghani ◽  
Tim Hohmann
Keyword(s):  
Machine Learning ◽  
Cell Death ◽  
False Positive ◽  
Neuronal Damage ◽  
False Negative ◽  
Neuronal Cell ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Support Vector ◽  
Analysis Tool

Neuronal damage presents a major health issue necessitating extensive research to identify mechanisms of neuronal cell death and potential therapeutic targets. Commonly used models are slice cultures out of different brain regions extracted from mice or rats, excitotoxically, ischemic, or traumatically lesioned and subsequently treated with potential neuroprotective agents. Thereby cell death is regularly assessed by measuring the propidium iodide (PI) uptake or counting of PI-positive nuclei. The applied methods have a limited applicability, either in terms of objectivity and time consumption or regarding its applicability. Consequently, new tools for analysis are needed. Here, we present a framework to mimic manual counting using machine learning algorithms as tools for semantic segmentation of PI-positive dead cells in hippocampal slice cultures. Therefore, we trained a support vector machine (SVM) to classify images into either “high” or “low” neuronal damage and used naïve Bayes, discriminant analysis, random forest, and a multilayer perceptron (MLP) as classifiers for segmentation of dead cells. In our final models, pixel-wise accuracies of up to 0.97 were achieved using the MLP classifier. Furthermore, a SVM-based post-processing step was introduced to differentiate between false-positive and false-negative detections using morphological features. As only very few false-positive objects and thus training data remained when using the final model, this approach only mildly improved the results. A final object splitting step using Hough transformations was used to account for overlap, leading to a recall of up to 97.6% of the manually assigned PI-positive dead cells. Taken together, we present an analysis tool that can help to objectively and reproducibly analyze neuronal damage in brain-derived slice cultures, taking advantage of the morphology of pycnotic cells for segmentation, object splitting, and identification of false positives.

scholarly journals A Multilane Tracking Algorithm Using IPDA with Intensity Feature

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 461
Author(s):  
Behzad Akbari ◽  
Jeyan Thiyagalingam ◽  
Richard Lee ◽  
Thia Kirubarajan
Keyword(s):  
Machine Learning ◽  
False Positive ◽  
Autonomous Vehicles ◽  
False Negative ◽  
Lane Detection ◽  
Multitarget Tracking ◽  
Lighting Conditions ◽  
Novel Approach ◽  
Alternative Approach ◽  
Consistent Manner

Detection of multiple lane markings on road surfaces is an important aspect of autonomous vehicles. Although a number of approaches have been proposed to detect lanes, detecting multiple lane markings, particularly across a large number of frames and under varying lighting conditions, in a consistent manner is still a challenging problem. In this paper, we propose a novel approach for detecting multiple lanes across a large number of frames and under various lighting conditions. Instead of resorting to the conventional approach of processing each frame to detect lanes, we treat the overall problem as a multitarget tracking problem across space and time using the integrated probabilistic data association filter (IPDAF) as our basis filter. We use the intensity of the pixels as an augmented feature to correctly group multiple lane markings using the Hough transform. By representing these extracted lane markings as splines, we then identify a set of control points, which becomes a set of targets to be tracked over a period of time, and thus across a large number of frames. We evaluate our approach on two different fronts, covering both model- and machine-learning-based approaches, using two different datasets, namely the Caltech and TuSimple lane detection datasets, respectively. When tested against model-based approach, the proposed approach can offer as much as 5%, 12%, and 3% improvements on the true positive, false positive, and false positives per frame rates compared to the best alternative approach, respectively. When compared against a state-of-the-art machine learning technique, particularly against a supervised learning method, the proposed approach offers 57%, 31%, 4%, and 9× improvements on the false positive, false negative, accuracy, and frame rates. Furthemore, the proposed approach retains the explainability, or in other words, the cause of actions of the proposed approach can easily be understood or explained.

scholarly journals Cost-Sensitive Learning of Fuzzy Rules for Imbalanced Classification Problems Using FURIA

2014 ◽  
Vol 22 (05) ◽  
pp. 643-675 ◽  
Author(s):  
Ana Palacios ◽  
Krzysztof Trawiński ◽  
Oscar Cordón ◽  
Luciano Sánchez
Keyword(s):  
Machine Learning ◽  
State Of The Art ◽  
Numerical Study ◽  
Fuzzy Rules ◽  
Classification Algorithms ◽  
Classification Problems ◽  
Cost Sensitive Learning ◽  
Imbalanced Classification ◽  
Machine Learning Methods ◽  
Data Level

This paper is intended to verify that cost-sensitive learning is a competitive approach for learning fuzzy rules in certain imbalanced classification problems. It will be shown that there exist cost matrices whose use in combination with a suitable classifier allows for improving the results of some popular data-level techniques. The well known FURIA algorithm is extended to take advantage of this definition. A numerical study is carried out to compare the proposed cost-sensitive FURIA to other state-of-the-art classification algorithms, based on fuzzy rules and on other classical machine learning methods, on 64 different imbalanced datasets.

scholarly journals Malaria Outbreak Detection with Machine Learning Methods

2020 ◽  
Author(s):  
Gurcan Comert ◽  
Negash Begashaw ◽  
Ayse Turhan-Comert
Keyword(s):  
Machine Learning ◽  
Binary Classification ◽  
False Negative ◽  
Machine Learning Techniques ◽  
Outbreak Detection ◽  
Maximum Temperature ◽  
Testing Dataset ◽  
Learning Techniques ◽  
Sample Data ◽  
Malaria Outbreak

AbstractIn this paper, we utilized and compared selected machine learning techniques to detect malaria out-break using observed variables of maximum temperature, minimum temperature, humidity, rainfall amount, positive case, and Plasmodium Falciparum rate. Random decision tree, logistic regression, and Gaussian processes are specially analyzed and adopted to be applied for malaria outbreak detection. The problem is a binary classification with outcomes of outbreak or no outbreak. Sample data provided in the literature from Maharashtra, India is used. Performance of the models are compared with the results from similar studies. Based on the sample data used, we were able to detect the malaria outbreak without any false positive or false negative errors in the testing dataset.

Cost-Sensitive Learning in Medicine

2012 ◽  
pp. 1625-1641
Author(s):  
Alberto Freitas ◽  
Pavel Brazdil ◽  
Altamiro Costa-Pereira
Keyword(s):  
Diagnostic Tests ◽  
False Negative ◽  
Cost Sensitive Learning ◽  
Safety Issues ◽  
Misclassification Costs ◽  
Misclassification Errors ◽  
Cost Sensitive Classification ◽  
Human Costs ◽  
The Cost ◽  
Specific Diagnostic Test

This chapter introduces cost-sensitive learning and its importance in medicine. Health managers and clinicians often need models that try to minimize several types of costs associated with healthcare, including attribute costs (e.g. the cost of a specific diagnostic test) and misclassification costs (e.g. the cost of a false negative test). In fact, as in other professional areas, both diagnostic tests and its associated misclassification errors can have significant financial or human costs, including the use of unnecessary resource and patient safety issues. This chapter presents some concepts related to cost-sensitive learning and cost-sensitive classification and its application to medicine. Different types of costs are also present, with an emphasis on diagnostic tests and misclassification costs. In addition, an overview of research in the area of cost-sensitive learning is given, including current methodological approaches. Finally, current methods for the cost-sensitive evaluation of classifiers are discussed.

Novel machine-learning based cloud mask and its application for Antarctic polynya monitoring using MODIS thermal-infrared imagery

2021 ◽  
Author(s):  
Stephan Paul ◽  
Marcus Huntemann
Keyword(s):  
Machine Learning ◽  
Sea Ice ◽  
False Positive ◽  
Cloud Cover ◽  
Near Infrared ◽  
False Negative ◽  
Open Water ◽  
Thermal Infrared ◽  
Spatial Coverage ◽  
Cloud Mask

<p>The frequent presence of cloud cover in polar regions limits the use of the Moderate-Resolution Imageing Spectroradiometer (MODIS) and similar instruments for the investigation and monitoring of sea-ice polynyas compared to passive-microwave-based sensors. The very low thermal contrast between present clouds and the sea-ice surface in combination with the lack of available visible and near-infrared channels during polar nighttime results in deficiencies in the MODIS cloud mask and dependent MODIS data products. This leads to frequent misclassifications of i) present clouds as sea ice/open water (false-negative) and ii) open-water/thin-ice areas as clouds (false-positive), which results in an underestimation of actual polynya area and subsequent derived information. Here, we present a novel machine-learning based approach using a deep neural network that is able to reliably discriminate between clouds, sea-ice, and open-water/thin-ice areas in a given swath solely from thermal-infrared MODIS channels and derived additional information. Compared to the reference MODIS sea-ice product for the year 2017, our data results in an overall increase of 20% in annual swath-based coverage for the Brunt Ice Shelf polynya, attributed to an improved cloud-cover discrimination and the reduction of false-positive classifications. At the same time, the mean annual polynya area decreases by 44% through the reduction of false-negative classifications of warm clouds as thin ice. Additionally, higher spatial coverage results in an overall better sub-daily representation of thin-ice conditions that cannot be reconstructed with current state-of-the-art cloud-cover compensation methods.</p>

Towards an Adaptive Educational Course on the Mathematical Foundations of Machine Learning

2021 ◽  
Vol 12 (8) ◽  
pp. 431-439
Author(s):  
A. S. Shundeev ◽  
Keyword(s):  
Machine Learning ◽  
Binary Classification ◽  
Classification Problem ◽  
Learning Model ◽  
Pac Learning ◽  
Learning Methods ◽  
Variable Part ◽  
Machine Learning Methods ◽  
Mathematical Foundations ◽  
Educational Course

Today the development of information technology is closely related to the creation and application of machine learning and data analysis methods. In this regard, the need for training specialists in this area is growing. Very often, the study of machine learning methods is combined with the study of a certain programming language and the tools of its specialized library. This approach is undoubtedly justified, because it provides the possibility of accelerated application of the knowledge gained in practice. At the same time, it should be noted that with this approach, it is rather not machine learning methods that are studied, but a certain set of methodological techniques for using the tools of the specialized library. The presented work is devoted to the experience of creating an adaptive educational course on the mathematical foundations of machine learning. This course is aimed at undergraduate and graduate students of mathematical specialties. It is divided into core and variable parts. The obligatory core part is built around the PAC learning model and the binary classification problem. Within the variable part, issues of the weak learning model and the boosting methods are considered. Also a methodology of changing the variable part of the course is discussed.

Sign in / Sign up

Export Citation Format

Share Document