THE ROBUST ESTIMATION OF CLASSIFICATION ERROR RATES

1986 ◽  
pp. 253-260
Author(s):  
JAMES D. KNOKE
Keyword(s):  
Robust Estimation ◽  
Error Rates ◽  
Classification Error

scholarly journals Markovian analysis of cervical cell images.

1976 ◽  
Vol 24 (1) ◽  
pp. 138-144 ◽  
Author(s):  
N J Pressman
Keyword(s):  
Transition Probabilities ◽  
Classification Performance ◽  
Error Rates ◽  
Classification Error ◽  
Correct Classification ◽  
Cell Nuclei ◽  
Step Size ◽  
Cervical Cell ◽  
Texture Parameters ◽  
Markovian Analysis

Markovian analysis is a method to measure optical texture based on gray-level transition probabilities in digitized images. Experiments are described that investigate that classification performance of parameters generated by Markovian analysis. Results using Markov texture parameters show that the selection of a Markov step size strongly affects classification error rates and the number of parameters required to achieve the maximum correct classification rates. Markov texture parameters are shown to achieve high rates of correct classification in discriminating images of normal from abnormal cervical cell nuclei.

Comparison of SPRT and Sequential Bayes Procedures for Classifying Examinees Into Two Categories Using a Computerized Test

1996 ◽  
Vol 21 (4) ◽  
pp. 405-414 ◽  
Author(s):  
Judith A. Spray ◽  
Mark D. Reckase
Keyword(s):  
Accurate Estimate ◽  
Error Rates ◽  
Computerized Testing ◽  
Classification Error ◽  
Ratio Test ◽  
College Selection ◽  
Entry Level ◽  
Testing Procedures ◽  
Test Items ◽  
Sequential Probability

Many testing applications focus on classifying examinees into one of two categories (e.g., pass/fail) rather than on obtaining an accurate estimate of level of ability. Examples of such applications include licensure and certification, college selection, and placement into entry-level or developmental college courses. With the increased availability of computers for the administration and scoring of tests, computerized testing procedures have been developed for efficiently making these classification decisions. The purpose of the research reported in this article was to compare two such procedures, one based on the sequential probability ratio test and the other on sequential Bayes methodology, to determine which required fewer items for classification when the procedures were matched on classification error rates. The results showed that under the conditions studied, the SPRT procedure required fewer test items than the sequential Bayes procedure to achieve the same level of classification accuracy.

scholarly journals Comparison of LDA and SPRT on Clinical Dataset Classifications

2011 ◽  
Vol 4 ◽  
pp. BII.S6935 ◽  
Author(s):  
Chih Lee ◽  
Brittany Nkounkou ◽  
Chun-Hsi Huang
Keyword(s):  
Learning Community ◽  
Prediction Accuracy ◽  
Cross Validation ◽  
Error Rates ◽  
Close Relative ◽  
Classification Error ◽  
Class Label ◽  
Normality Assumption ◽  
Clinical Dataset ◽  
Leave One Out

In this work, we investigate the well-known classification algorithm LDA as well as its close relative SPRT. SPRT affords many theoretical advantages over LDA. It allows specification of desired classification error rates α and β and is expected to be faster in predicting the class label of a new instance. However, SPRT is not as widely used as LDA in the pattern recognition and machine learning community. For this reason, we investigate LDA, SPRT and a modified SPRT (MSPRT) empirically using clinical datasets from Parkinson's disease, colon cancer, and breast cancer. We assume the same normality assumption as LDA and propose variants of the two SPRT algorithms based on the order in which the components of an instance are sampled. Leave-one-out cross-validation is used to assess and compare the performance of the methods. The results indicate that two variants, SPRT-ordered and MSPRT-ordered, are superior to LDA in terms of prediction accuracy. Moreover, on average SPRT-ordered and MSPRT-ordered examine less components than LDA before arriving at a decision. These advantages imply that SPRT-ordered and MSPRT-ordered are the preferred algorithms over LDA when the normality assumption can be justified for a dataset.

scholarly journals SGPNet: A Three-Dimensional Multitask Residual Framework for Segmentation and IDH Genotype Prediction of Gliomas

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Yao Wang ◽  
Yan Wang ◽  
Chunjie Guo ◽  
Shuangquan Zhang ◽  
Lili Yang
Keyword(s):  
Brain Tumor ◽  
Clinical Decision Making ◽  
Three Dimensional ◽  
Clinical Decision ◽  
Error Rates ◽  
The Cancer Genome Atlas ◽  
Classification Error ◽  
Tumor Segmentation ◽  
Brain Tumor Segmentation ◽  
The Status

Glioma is the main type of malignant brain tumor in adults, and the status of isocitrate dehydrogenase (IDH) mutation highly affects the diagnosis, treatment, and prognosis of gliomas. Radiographic medical imaging provides a noninvasive platform for sampling both inter and intralesion heterogeneity of gliomas, and previous research has shown that the IDH genotype can be predicted from the fusion of multimodality radiology images. The features of medical images and IDH genotype are vital for medical treatment; however, it still lacks a multitask framework for the segmentation of the lesion areas of gliomas and the prediction of IDH genotype. In this paper, we propose a novel three-dimensional (3D) multitask deep learning model for segmentation and genotype prediction (SGPNet). The residual units are also introduced into the SGPNet that allows the output blocks to extract hierarchical features for different tasks and facilitate the information propagation. Our model reduces 26.6% classification error rates comparing with previous models on the datasets of Multimodal Brain Tumor Segmentation Challenge (BRATS) 2020 and The Cancer Genome Atlas (TCGA) gliomas’ databases. Furthermore, we first practically investigate the influence of lesion areas on the performance of IDH genotype prediction by setting different groups of learning targets. The experimental results indicate that the information of lesion areas is more important for the IDH genotype prediction. Our framework is effective and generalizable, which can serve as a highly automated tool to be applied in clinical decision making.

scholarly journals Predicting the Reasons of Customer Complaints: A First Step Toward Anticipating Quality Issues of In Vitro Diagnostics Assays with Machine Learning (Preprint)

2018 ◽  
Author(s):  
Stephane Aris-Brosou ◽  
James Kim ◽  
Li Li ◽  
Hui Liu
Keyword(s):  
Product Quality ◽  
Error Rates ◽  
Training Period ◽  
Classification Error ◽  
Diagnostic Systems ◽  
Adaptive Boosting ◽  
Customer Complaints ◽  
In Vitro Diagnostic ◽  
Quality Issues

BACKGROUND Vendors in the health care industry produce diagnostic systems that, through a secured connection, allow them to monitor performance almost in real time. However, challenges exist in analyzing and interpreting large volumes of noisy quality control (QC) data. As a result, some QC shifts may not be detected early enough by the vendor, but lead a customer to complain. OBJECTIVE The aim of this study was to hypothesize that a more proactive response could be designed by utilizing the collected QC data more efficiently. Our aim is therefore to help prevent customer complaints by predicting them based on the QC data collected by in vitro diagnostic systems. METHODS QC data from five select in vitro diagnostic assays were combined with the corresponding database of customer complaints over a period of 90 days. A subset of these data over the last 45 days was also analyzed to assess how the length of the training period affects predictions. We defined a set of features used to train two classifiers, one based on decision trees and the other based on adaptive boosting, and assessed model performance by cross-validation. RESULTS The cross-validations showed classification error rates close to zero for some assays with adaptive boosting when predicting the potential cause of customer complaints. Performance was improved by shortening the training period when the volume of complaints increased. Denoising filters that reduced the number of categories to predict further improved performance, as their application simplified the prediction problem. CONCLUSIONS This novel approach to predicting customer complaints based on QC data may allow the diagnostic industry, the expected end user of our approach, to proactively identify potential product quality issues and fix these before receiving customer complaints. This represents a new step in the direction of using big data toward product quality improvement.

scholarly journals Classification of vulnerability levels using multivariate biomarkers in schizophrenia: a machine-learning approach

2020 ◽  
Author(s):  
Simona Caldani ◽  
François-Benoît Vialatte ◽  
Aurélien Baelde ◽  
Maria Pia Bucci ◽  
Narjes Bendjemaa ◽  
...  
Keyword(s):  
Machine Learning ◽  
Error Rate ◽  
Error Rates ◽  
Supervised Machine Learning ◽  
Classification Error ◽  
Support Vector ◽  
Neurological Soft Signs ◽  
Neurodevelopmental Disease ◽  
Multimodal Features ◽  
Early Intervention In Psychosis

Abstract Background: Schizophrenia is a heterogeneous neurodevelopmental disease involving cognitive and motor impairments. Motor dysfunctions, such as eye movements or neurological soft signs, are proposed as endophenotypic markers. Methods: Supervised machine-learning methods (Support Vector Machines) applied on oculomotor performances using comprehensive testing with prosaccades, antisaccades, memory-guided saccade tasks and smooth pursuit, as well as neurological soft signs assessment, was used to discriminate patients with schizophrenia (SZ, N=53), full siblings of patients (FS, N=45) and healthy volunteers (C, N=48). 80% of patients were used in a training/validation set and 20% on a test set. The discrimination was measured using the classification error (rate of misclassified patients).Results: The most reliable classification was between C and SZ, with only 15% and 12% of error rates for validation and test, whereas the SZ vs. FS classification provided the highest error rates (32% of error rate in both validation and test). Interestingly, neurological soft signs were selected as the best predictor, together with a combination of measures, for the two classifications: C vs. SZ, SZ vs. FS. In addition, memory-guided saccades were consistently selected among the best two multimodal features for the classifications involving the control group (C vs. SZ or FS). Conclusions: Taken together, these results emphasize the importance of neurological soft signs and sensitive oculomotor parameters, especially memory-guided saccades. This classification provides promising avenues for improving early detection of / early intervention in psychosis.

The Performance of the Linear Discriminant Function in Nonoptimal Situations and the Estimation of Classification Error Rates: A Review of Recent Findings

1979 ◽  
Vol 16 (3) ◽  
pp. 370-381 ◽  
Author(s):  
William R. Dillon
Keyword(s):  
Discriminant Function ◽  
Error Rates ◽  
Linear Discriminant Function ◽  
Alternative Methods ◽  
Misclassification Error ◽  
Practical Significance ◽  
Classification Error ◽  
Continuous Variables ◽  
Discrete Variables ◽  
Linear Discriminant

This article is a review of the results, as are available, on the performance of the linear discriminant function in situations where the assumptions of multivariate normality and equal group dispersion structures are violated. Some new results are discussed for the case of classification using discrete variables, and in the case of both binary and continuous variables. In addition, alternative methods which have been proposed, and evaluated, for estimating misclassification error rates are thoroughly reviewed. In all cases, the material is reviewed in terms of practical significance, with particular emphasis on the conditions unfavorable to the performance of each procedure.

NONLINEAR SIGNAL CLASSIFICATION

2002 ◽  
Vol 12 (06) ◽  
pp. 1273-1293 ◽  
Author(s):  
P. E. RAPP ◽  
T. A. A. WATANABE ◽  
P. FAURE ◽  
C. J. CELLUCCI
Keyword(s):  
Error Rate ◽  
Mahalanobis Distance ◽  
Additive Noise ◽  
Stochastic Systems ◽  
Error Rates ◽  
Classification Criteria ◽  
Signal Classification ◽  
Classification Error ◽  
Signal To Noise ◽  
Classification Errors

In this contribution, we show that the incorporation of nonlinear dynamical measures into a multivariate discrimination provides a signal classification system that is robust to additive noise. The signal library was composed of nine groups of signals. Four groups were generated computationally from deterministic systems (van der Pol, Lorenz, Rössler and Hénon). Four groups were generated computationally from different stochastic systems. The ninth group contained inter-decay interval sequences from radioactive cobalt. Two classification criteria (minimum Mahalanobis distance and maximum Bayesian likelihood) were tested. In the absence of additive noise, no errors occurred in a within-library classification. Normally distributed random numbers were added to produce signal to noise ratios of 10, 5 and 0 dB. When the minimum Mahalanobis distance was used as the classification criterion, the corresponding error rates were 2.2%, 4.4% and 20% (Expected Error Rate = 89%). When Bayesian maximum likelihood was the criterion, the error rates were 1.1%, 4.4% and 21% respectively. Using nonlinear measures an effective discrimination can be achieved in cases where spectral measures are known to fail. Most classification errors occurred at low signal to noise ratios when a stochastic signal was misclassified into a different group of stochastic signals. When the within-library classification exercise is limited to the four groups of deterministic signals, no classification errors occurred with clean data, at SNR = 10 dB, or at SNR = 5 dB. A single classification error (Observed Error Rate = 2.5%, Expected Error Rate = 75%) occurred with both classification criteria at SNR = 0 dB.

scholarly journals Adaptive Windowing Framework for Surface Electromyogram-Based Pattern Recognition System for Transradial Amputees

Sensors ◽  
2018 ◽  
Vol 18 (8) ◽  
pp. 2402 ◽  
Author(s):  
Ali Al-Timemy ◽  
Guido Bugmann ◽  
Javier Escudero
Keyword(s):  
Pattern Recognition ◽  
Upper Limb ◽  
Time Domain ◽  
Window Size ◽  
Error Rates ◽  
Classification Error ◽  
Linear Discriminant ◽  
Emg Signal ◽  
Upper Limb Prosthesis ◽  
Upper Limb Prostheses

Electromyogram (EMG)-based Pattern Recognition (PR) systems for upper-limb prosthesis control provide promising ways to enable an intuitive control of the prostheses with multiple degrees of freedom and fast reaction times. However, the lack of robustness of the PR systems may limit their usability. In this paper, a novel adaptive time windowing framework is proposed to enhance the performance of the PR systems by focusing on their windowing and classification steps. The proposed framework estimates the output probabilities of each class and outputs a movement only if a decision with a probability above a certain threshold is achieved. Otherwise (i.e., all probability values are below the threshold), the window size of the EMG signal increases. We demonstrate our framework utilizing EMG datasets collected from nine transradial amputees who performed nine movement classes with Time Domain Power Spectral Descriptors (TD-PSD), Wavelet and Time Domain (TD) feature extraction (FE) methods and a Linear Discriminant Analysis (LDA) classifier. Nonetheless, the concept can be applied to other types of features and classifiers. In addition, the proposed framework is validated with different movement and EMG channel combinations. The results indicate that the proposed framework works well with different FE methods and movement/channel combinations with classification error rates of approximately 13% with TD-PSD FE. Thus, we expect our proposed framework to be a straightforward, yet important, step towards the improvement of the control methods for upper-limb prostheses.

Sign in / Sign up

Export Citation Format

Share Document