scholarly journals Surrogate regret bounds for generalized classification performance metrics

2016 ◽  
Vol 106 (4) ◽  
pp. 549-572 ◽  
Author(s):  
Wojciech Kotłowski ◽  
Krzysztof Dembczyński
Keyword(s):  
Performance Metrics ◽  
Classification Performance ◽  
Regret Bounds

scholarly journals The impact of class imbalance in classification performance metrics based on the binary confusion matrix

2019 ◽  
Vol 91 ◽  
pp. 216-231 ◽  
Author(s):  
Amalia Luque ◽  
Alejandro Carrasco ◽  
Alejandro Martín ◽  
Ana de las Heras
Keyword(s):  
Performance Metrics ◽  
Confusion Matrix ◽  
Class Imbalance ◽  
Classification Performance ◽  
The Impact

scholarly journals Utility of MemTrax and Machine Learning Modeling in Classification of Mild Cognitive Impairment

2020 ◽  
Vol 77 (4) ◽  
pp. 1545-1558
Author(s):  
Michael F. Bergeron ◽  
Sara Landset ◽  
Xianbo Zhou ◽  
Tao Ding ◽  
Taghi M. Khoshgoftaar ◽  
...  
Keyword(s):  
Machine Learning ◽  
Cognitive Impairment ◽  
Mild Cognitive Impairment ◽  
Performance Metrics ◽  
Early Stage ◽  
Characteristic Curve ◽  
Classification Performance ◽  
Cognitive Screening ◽  
Cross Sectional ◽  
Machine Learning Classification

Background: The widespread incidence and prevalence of Alzheimer’s disease and mild cognitive impairment (MCI) has prompted an urgent call for research to validate early detection cognitive screening and assessment. Objective: Our primary research aim was to determine if selected MemTrax performance metrics and relevant demographics and health profile characteristics can be effectively utilized in predictive models developed with machine learning to classify cognitive health (normal versus MCI), as would be indicated by the Montreal Cognitive Assessment (MoCA). Methods: We conducted a cross-sectional study on 259 neurology, memory clinic, and internal medicine adult patients recruited from two hospitals in China. Each patient was given the Chinese-language MoCA and self-administered the continuous recognition MemTrax online episodic memory test on the same day. Predictive classification models were built using machine learning with 10-fold cross validation, and model performance was measured using Area Under the Receiver Operating Characteristic Curve (AUC). Models were built using two MemTrax performance metrics (percent correct, response time), along with the eight common demographic and personal history features. Results: Comparing the learners across selected combinations of MoCA scores and thresholds, Naïve Bayes was generally the top-performing learner with an overall classification performance of 0.9093. Further, among the top three learners, MemTrax-based classification performance overall was superior using just the top-ranked four features (0.9119) compared to using all 10 common features (0.8999). Conclusion: MemTrax performance can be effectively utilized in a machine learning classification predictive model screening application for detecting early stage cognitive impairment.

scholarly journals A Deep Learning Framework for Coronavirus Disease (COVID-19) Detection in X-Ray Images

2020 ◽  
Author(s):  
Tayyip Ozcan
Keyword(s):  
Transfer Learning ◽  
Performance Metrics ◽  
Experimental Studies ◽  
Large Family ◽  
Classification Performance ◽  
The Novel ◽  
X Ray ◽  
Learning Framework ◽  
Classification Feature ◽  
Novel Coronavirus

Abstract Coronavirus, a large family of viruses, causes illness in both humans and animals. The novel coronavirus (COVID-19) came up in Wuhan in December 2019. This deadly COVID-19 pandemic has become very fast-spreading and currently present in several countries worldwide. The timely detection of patients who have COVID-19 is vitally important. To this end, scientists are working on different detection methods.In this paper, a grid search (GS) and pre-trained model aided convolutional neural network (CNN) model is proposed to detect COVID-19 in X-Ray images. In the proposed method, the GS method is employed to optimize the hyperparameters of CNN, which directly affects classification performance. Three pre-trained CNN models (GoogleNet, ResNet18 and ResNet50), which can be used for classification, feature extraction and transfer learning purposes were used for transfer learning in this study. The proposed method was trained using the training and validation subdatasets of the collected dataset and detail evaluations are presented according to different performance metrics. According to the experimental studies, the best results were obtained with the GS and ResNet50 aided model.

scholarly journals Exploring Symmetry of Binary Classification Performance Metrics

Symmetry ◽  
2019 ◽  
Vol 11 (1) ◽  
pp. 47 ◽  
Author(s):  
Amalia Luque ◽  
Alejandro Carrasco ◽  
Alejandro Martín ◽  
Juan Ramón Lama
Keyword(s):  
Performance Metrics ◽  
Binary Classification ◽  
Confusion Matrix ◽  
Full Range ◽  
Classification Performance ◽  
Classification Problems ◽  
Performance Metric ◽  
Selection For ◽  
Proper Performance ◽  
Insight Into

Selecting the proper performance metric constitutes a key issue for most classification problems in the field of machine learning. Although the specialized literature has addressed several topics regarding these metrics, their symmetries have yet to be systematically studied. This research focuses on ten metrics based on a binary confusion matrix and their symmetric behaviour is formally defined under all types of transformations. Through simulated experiments, which cover the full range of datasets and classification results, the symmetric behaviour of these metrics is explored by exposing them to hundreds of simple or combined symmetric transformations. Cross-symmetries among the metrics and statistical symmetries are also explored. The results obtained show that, in all cases, three and only three types of symmetries arise: labelling inversion (between positive and negative classes); scoring inversion (concerning good and bad classifiers); and the combination of these two inversions. Additionally, certain metrics have been shown to be independent of the imbalance in the dataset and two cross-symmetries have been identified. The results regarding their symmetries reveal a deeper insight into the behaviour of various performance metrics and offer an indicator to properly interpret their values and a guide for their selection for certain specific applications.

scholarly journals Panoramic tongue imaging and deep convolutional machine learning model for diabetes diagnosis in humans

2022 ◽  
Vol 12 (1) ◽  
Author(s):  
Saritha Balasubramaniyan ◽  
Vijay Jeyakumar ◽  
Deepa Subramaniam Nachimuthu
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Performance Metrics ◽  
Disease Diagnosis ◽  
Classification Performance ◽  
High Rate ◽  
Diabetes Diagnosis ◽  
Operating Characteristics ◽  
High Glucose Level ◽  
Sensitivity Specificity

AbstractDiabetes is a serious metabolic disorder with high rate of prevalence worldwide; the disease has the characteristics of improper secretion of insulin in pancreas that results in high glucose level in blood. The disease is also associated with other complications such as cardiovascular disease, retinopathy, neuropathy and nephropathy. The development of computer aided decision support system is inevitable field of research for disease diagnosis that will assist clinicians for the early prognosis of diabetes and to facilitate necessary treatment at the earliest. In this research study, a Traditional Chinese Medicine based diabetes diagnosis is presented based on analyzing the extracted features of panoramic tongue images such as color, texture, shape, tooth markings and fur. The feature extraction is done by Convolutional Neural Network (CNN)—ResNet 50 architecture, and the classification is performed by the proposed Deep Radial Basis Function Neural Network (RBFNN) algorithm based on auto encoder learning mechanism. The proposed model is simulated in MATLAB environment and evaluated with performance metrics—accuracy, precision, sensitivity, specificity, F1 score, error rate, and receiver operating characteristics (ROC). On comparing with existing models, the proposed CNN based Deep RBFNN machine learning classifier model outperformed with better classification performance and proving its effectiveness.

Informative Path Planning for Improving Classification Performance through Kinematic Decisions

2014 ◽  
Vol 02 (02) ◽  
pp. 143-156
Author(s):  
Baro Hyun ◽  
Weijia Zhang ◽  
Pierre Kabamba ◽  
Anouck Girard
Keyword(s):  
Path Planning ◽  
Performance Metrics ◽  
Sliding Mode ◽  
Classification Performance ◽  
Planning Problem ◽  
Sensor Performance ◽  
Unmanned Air Vehicle ◽  
Multiple Measurements ◽  
The Relationship ◽  
Path Planning Problem

As an information-gathering agent, unmanned system inevitably relies on its sensing abilities to perceive the world. Such a gathered information is vital for making further significant decisions, thus often operating under the assumption that maximizing the amount of information would result in minimizing the likelihood of committing erroneous decisions. In this work, we question this assumption by carefully examining the relationship between information and the probability of making erroneous decisions, then investigate the implication of the relationship in a Unmanned Air Vehicle (UAV) path planning problem. First, we conduct a functional analysis of two performance metrics (i.e., mutual information and the probability of misclassification) with respect to the sensing abilities. The analysis suggests an effective region of sensor space that can improve the classification performance when multiple measurements are to be taken sequentially. Based on the results of the analysis, we establish sensing strategies to a UAV path planning problem where the sensor performance depends on the relative position (i.e., range and azimuth) of the UAV with respect to the object of interest. Specifically, we use two sliding mode controllers, each of which accounts for a particular sensing strategy, with a hybrid-system switching scheme. We validate our approach with numerical simulation results.

Sign in / Sign up

Export Citation Format

Share Document