scholarly journals Semi-Supervised Learning for Maximizing the Partial AUC

2020 ◽  
Vol 34 (04) ◽  
pp. 4239-4246
Author(s):  
Tomoharu Iwata ◽  
Akinori Fujino ◽  
Naonori Ueda
Keyword(s):  
Supervised Learning ◽  
Binary Classification ◽  
Characteristic Curve ◽  
False Positive Rate ◽  
True Positive Rate ◽  
Unlabeled Data ◽  
Spam Filtering ◽  
Classification Problems ◽  
Partial Area ◽  
Positive Rate

The partial area under a receiver operating characteristic curve (pAUC) is a performance measurement for binary classification problems that summarizes the true positive rate with the specific range of the false positive rate. Obtaining classifiers that achieve high pAUC is important in a wide variety of applications, such as cancer screening and spam filtering. Although many methods have been proposed for maximizing the pAUC, existing methods require many labeled data for training. In this paper, we propose a semi-supervised learning method for maximizing the pAUC, which trains a classifier with a small amount of labeled data and a large amount of unlabeled data. To exploit the unlabeled data, we derive two approximations of the pAUC: the first is calculated from positive and unlabeled data, and the second is calculated from negative and unlabeled data. A classifier is trained by maximizing the weighted sum of the two approximations of the pAUC and the pAUC that is calculated from positive and negative data. With experiments using various datasets, we demonstrate that the proposed method achieves higher test pAUCs than existing methods.

scholarly journals Analysis and Mapping of Rainfall-Induced Landslide Susceptibility in A Luoi District, Thua Thien Hue Province, Vietnam

Water ◽  
2018 ◽  
Vol 11 (1) ◽  
pp. 51 ◽  
Author(s):  
Nguyen Long ◽  
Florimond De Smedt
Keyword(s):  
Landslide Susceptibility ◽  
Factor Model ◽  
Characteristic Curve ◽  
False Positive Rate ◽  
Area Under The Curve ◽  
True Positive Rate ◽  
Landslide Susceptibility Mapping ◽  
Certainty Factor ◽  
Validation Test ◽  
Positive Rate

Rainfall-induced landslides form an important natural threat in Vietnam. The purpose of this study is to explore regional landslide susceptibility mapping in the mountainous district of A Luoi in Thua Thien Hue Province, where data on the occurrence and causes of landslides are very limited. Three methods are applied to examine landslide susceptibility: statistical index, logistic regression and certainty factor. Nine causative factors are considered: elevation, slope, geological strata, fault density, geomorphic landforms, weathering crust, land use, distance to rivers and annual precipitation. The reliability of the landslide susceptibility maps is evaluated by a receiver operating characteristic curve and the area under the curve is used to quantify and compare the prediction accuracy of the models. The certainty factor model performs best. This model is optimized by maximizing the difference between the true positive rate and the false positive rate. The optimal model correctly identifies 84% of the observed landslides. The results are verified with a validation test, whereby the model is calibrated with 75% randomly selected observed landslides, while the remaining 25% of the observed landslides are used for validation. The validation test correctly identifies 81% of the observed landslides in the training set and 73% of the observed landslides in the validation set.

scholarly journals Two-way partial AUC and its properties

2017 ◽  
Vol 28 (1) ◽  
pp. 184-195 ◽  
Author(s):  
Hanfang Yang ◽  
Kun Lu ◽  
Xiang Lyu ◽  
Feifang Hu
Keyword(s):  
Roc Curve ◽  
False Positive ◽  
False Positive Rate ◽  
R Package ◽  
True Positive Rate ◽  
Performance Measure ◽  
True Positive ◽  
Cancer Data ◽  
Partial Area ◽  
Positive Rate

Simultaneous control on true positive rate and false positive rate is of significant importance in the performance evaluation of diagnostic tests. Most of the established literature utilizes partial area under receiver operating characteristic (ROC) curve with restrictions only on false positive rate (FPR), called FPR pAUC, as a performance measure. However, its indirect control on true positive rate (TPR) is conceptually and practically misleading. In this paper, a novel and intuitive performance measure, named as two-way pAUC, is proposed, which directly quantifies partial area under ROC curve with explicit restrictions on both TPR and FPR. To estimate two-way pAUC, we devise a nonparametric estimator. Based on the estimator, a bootstrap-assisted testing method for two-way pAUC comparison is established. Moreover, to evaluate possible covariate effects on two-way pAUC, a regression analysis framework is constructed. Asymptotic normalities of the methods are provided. Advantages of the proposed methods are illustrated by simulation and Wisconsin Breast Cancer Data. We encode the methods as a publicly available R package tpAUC.

scholarly journals MCS-RF: mobile crowdsensing–based air quality estimation with random forest

2018 ◽  
Vol 14 (10) ◽  
pp. 155014771880470 ◽  
Author(s):  
Cheng Feng ◽  
Ye Tian ◽  
Xiangyang Gong ◽  
Xirong Que ◽  
Wendong Wang
Keyword(s):  
Random Forest ◽  
Urban Areas ◽  
Large Scale ◽  
Characteristic Curve ◽  
False Positive Rate ◽  
Estimation Method ◽  
True Positive Rate ◽  
Mobile Crowdsensing ◽  
Fine Grained ◽  
Positive Rate

It is a great challenge to offer a fine-grained and accurate PM2.5 monitoring service in urban areas as required facilities are very expensive and huge. Since PM2.5 has a significant scattering effect on visible light, large-scale user-contributed image data collected by the mobile crowdsensing bring a new opportunity for understanding the urban PM2.5. In this article, we propose a fine-grained PM2.5 estimation method based on random forest with data announced by meteorological departments and collected from smartphone users without any PM2.5 measurement devices. We design and implement a platform to collect data in the real world including the image provided by users. By combining online learning and offline learning, the method based on random forest performs well in terms of time complexity and accuracy. We compare our method with two kinds of baselines: subsets of the whole data sets and six classical models (such as logistic, naive Bayes). Six kinds of evaluation indexes (precision, recall, true-positive rate, false-positive rate, F-measure, and receiver operating characteristic curve area) are used in the evaluation. The experimental results show that our method achieves high accuracy (precision: 0.875, recall: 0.872) on PM2.5 estimation, which outperforms the other methods.

A Smart System of Malware Detection Based on Artificial Immune Network and Deep Belief Network

2021 ◽  
Vol 15 (1) ◽  
pp. 1-25
Author(s):  
Dung Hoang Le ◽  
Nguyen Thanh Vu ◽  
Tuan Dinh Le
Keyword(s):  
Detection Rate ◽  
Binary Classification ◽  
False Positive Rate ◽  
Virus Detection ◽  
Immune Network ◽  
Classification Problems ◽  
Artificial Immune ◽  
Smart System ◽  
Artificial Immune Network ◽  
Positive Rate

This paper proposes a smart system of virus detection that can classify a file as benign or malware with high accuracy detection rate. The approach is based on the aspects of the artificial immune system, in which an artificial immune network is used as a pool to create and develop virus detectors that can detect unknown data. Besides, a deep learning model is also used as the main classifier because of its advantages in binary classification problems. This method can achieve a detection rate of 99.08% on average, with a very low false positive rate.

scholarly journals PRATD: A Phased Remote Access Trojan Detection Method with Double-Sided Features

Electronics ◽  
2020 ◽  
Vol 9 (11) ◽  
pp. 1894
Author(s):  
Chun Guo ◽  
Zihua Song ◽  
Yuan Ping ◽  
Guowei Shen ◽  
Yuhei Cui ◽  
...  
Keyword(s):  
False Positive ◽  
Detection Method ◽  
False Positive Rate ◽  
True Positive Rate ◽  
Remote Access ◽  
Detection Methods ◽  
Security Threats ◽  
True Positive ◽  
Trojan Detection ◽  
Positive Rate

Remote Access Trojan (RAT) is one of the most terrible security threats that organizations face today. At present, two major RAT detection methods are host-based and network-based detection methods. To complement one another’s strengths, this article proposes a phased RATs detection method by combining double-side features (PRATD). In PRATD, both host-side and network-side features are combined to build detection models, which is conducive to distinguishing the RATs from benign programs because that the RATs not only generate traffic on the network but also leave traces on the host at run time. Besides, PRATD trains two different detection models for the two runtime states of RATs for improving the True Positive Rate (TPR). The experiments on the network and host records collected from five kinds of benign programs and 20 famous RATs show that PRATD can effectively detect RATs, it can achieve a TPR as high as 93.609% with a False Positive Rate (FPR) as low as 0.407% for the known RATs, a TPR 81.928% and FPR 0.185% for the unknown RATs, which suggests it is a competitive candidate for RAT detection.

Ascertaining an efficient eligibility cut-off for extended Medicare items for eating disorders

2021 ◽  
pp. 103985622110286
Author(s):  
Tracey Wade ◽  
Jamie-Lee Pennesi ◽  
Yuan Zhou
Keyword(s):  
Eating Disorders ◽  
Eating Disorder ◽  
False Positive Rate ◽  
Area Under The Curve ◽  
Rate Sensitivity ◽  
True Positive Rate ◽  
Eating Disorder Examination Questionnaire ◽  
Eating Disorder Examination ◽  
Positive Rate ◽  
The Relationship

Objective: Currently eligibility for expanded Medicare items for eating disorders (excluding anorexia nervosa) require a score ⩾ 3 on the 22-item Eating Disorder Examination-Questionnaire (EDE-Q). We compared these EDE-Q “cases” with continuous scores on a validated 7-item version of the EDE-Q (EDE-Q7) to identify an EDE-Q7 cut-off commensurate to 3 on the EDE-Q. Methods: We utilised EDE-Q scores of female university students ( N = 337) at risk of developing an eating disorder. We used a receiver operating characteristic (ROC) curve to assess the relationship between the true-positive rate (sensitivity) and the false-positive rate (1-specificity) of cases ⩾ 3. Results: The area under the curve showed outstanding discrimination of 0.94 (95% CI: .92–.97). We examined two specific cut-off points on the EDE-Q7, which included 100% and 87% of true cases, respectively. Conclusion: Given the EDE-Q cut-off for Medicare is used in conjunction with other criteria, we suggest using the more permissive EDE-Q7 cut-off (⩾2.5) to replace use of the EDE-Q cut-off (⩾3) in eligibility assessments.

scholarly journals Improving Ecological Inference by Predicting Individual Ethnicity from Voter Registration Records

2016 ◽  
Vol 24 (2) ◽  
pp. 263-272 ◽  
Author(s):  
Kosuke Imai ◽  
Kabir Khanna
Keyword(s):  
Mean Squared Error ◽  
False Positive Rate ◽  
True Positive Rate ◽  
Voter Registration ◽  
Racial Groups ◽  
Ecological Inference ◽  
Inference Problem ◽  
Individual Level ◽  
Positive Rate ◽  
Election Results

In both political behavior research and voting rights litigation, turnout and vote choice for different racial groups are often inferred using aggregate election results and racial composition. Over the past several decades, many statistical methods have been proposed to address this ecological inference problem. We propose an alternative method to reduce aggregation bias by predicting individual-level ethnicity from voter registration records. Building on the existing methodological literature, we use Bayes's rule to combine the Census Bureau's Surname List with various information from geocoded voter registration records. We evaluate the performance of the proposed methodology using approximately nine million voter registration records from Florida, where self-reported ethnicity is available. We find that it is possible to reduce the false positive rate among Black and Latino voters to 6% and 3%, respectively, while maintaining the true positive rate above 80%. Moreover, we use our predictions to estimate turnout by race and find that our estimates yields substantially less amounts of bias and root mean squared error than standard ecological inference estimates. We provide open-source software to implement the proposed methodology.

Watch For Failing Objects: What Inappropriate Compliance Reveals About Shared Mental Models In Autonomous Cars

2021 ◽  
Vol 65 (1) ◽  
pp. 643-647
Author(s):  
Yosef S. Razin ◽  
Jack Gale ◽  
Jiaojiao Fan ◽  
Jaznae’ Smith ◽  
Karen M. Feigh
Keyword(s):  
Mental Models ◽  
Mental Model ◽  
False Positive Rate ◽  
Ground Truth ◽  
True Positive Rate ◽  
Shared Mental Models ◽  
Shared Mental Model ◽  
Autonomous Cars ◽  
Positive Rate ◽  
Dispositional Factors

This paper evaluates Banks et al.’s Human-AI Shared Mental Model theory by examining how a self-driving vehicle’s hazard assessment facilitates shared mental models. Participants were asked to affirm the vehicle’s assessment of road objects as either hazards or mistakes in real-time as behavioral and subjective measures were collected. The baseline performance of the AI was purposefully low (<50%) to examine how the human’s shared mental model might lead to inappropriate compliance. Results indicated that while the participant true positive rate was high, overall performance was reduced by the large false positive rate, indicating that participants were indeed being influenced by the Al’s faulty assessments, despite full transparency as to the ground-truth. Both performance and compliance were directly affected by frustration, mental, and even physical demands. Dispositional factors such as faith in other people’s cooperativeness and in technology companies were also significant. Thus, our findings strongly supported the theory that shared mental models play a measurable role in performance and compliance, in a complex interplay with trust.

scholarly journals Mixture models reveal multiple positional bias types in RNA-Seq data and lead to accurate transcript concentration estimates

2014 ◽  
Author(s):  
Andreas Tuerk ◽  
Gregor Wiktorin ◽  
Serhat Güler
Keyword(s):  
Probability Distributions ◽  
False Positive Rate ◽  
Synthetic Data ◽  
True Positive Rate ◽  
Rna Seq ◽  
Microarray Quality Control ◽  
Data Set ◽  
Rna Transcripts ◽  
Positive Rate ◽  
Fragment Distribution

Quantification of RNA transcripts with RNA-Seq is inaccurate due to positional fragment bias, which is not represented appropriately by current statistical models of RNA-Seq data. This article introduces the Mix2(rd. "mixquare") model, which uses a mixture of probability distributions to model the transcript specific positional fragment bias. The parameters of the Mix2model can be efficiently trained with the Expectation Maximization (EM) algorithm resulting in simultaneous estimates of the transcript abundances and transcript specific positional biases. Experiments are conducted on synthetic data and the Universal Human Reference (UHR) and Brain (HBR) sample from the Microarray quality control (MAQC) data set. Comparing the correlation between qPCR and FPKM values to state-of-the-art methods Cufflinks and PennSeq we obtain an increase in R2value from 0.44 to 0.6 and from 0.34 to 0.54. In the detection of differential expression between UHR and HBR the true positive rate increases from 0.44 to 0.71 at a false positive rate of 0.1. Finally, the Mix2model is used to investigate biases present in the MAQC data. This reveals 5 dominant biases which deviate from the common assumption of a uniform fragment distribution. The Mix2software is available at http://www.lexogen.com/fileadmin/uploads/bioinfo/mix2model.tgz.

scholarly journals Exploring Author Profiling for Fake News Detection

2021 ◽  
Author(s):  
Shloak Rathod
Keyword(s):  
False Positive Rate ◽  
True Positive Rate ◽  
Supervised Machine Learning ◽  
Online Media ◽  
Fake News ◽  
Writing Style ◽  
Short Term ◽  
Positive Rate ◽  
Rapid Dissemination ◽  
Author Profiling

<div><div><div><p>The proliferation of online media allows for the rapid dissemination of unmoderated news, unfortunately including fake news. The extensive spread of fake news poses a potent threat to both individuals and society. This paper focuses on designing author profiles to detect authors who are primarily engaged in publishing fake news articles. We build on the hypothesis that authors who write fake news repeatedly write only fake news articles, at least in short-term periods. Fake news authors have a distinct writing style compared to real news authors, who naturally want to maintain trustworthiness. We explore the potential to detect fake news authors by designing authors’ profiles based on writing style, sentiment, and co-authorship patterns. We evaluate our approach using a publicly available dataset with over 5000 authors and 20000 articles. For our evaluation, we build and compare different classes of supervised machine learning models. We find that the K-NN model performed the best, and it could detect authors who are prone to writing fake news with an 83% true positive rate with only a 5% false positive rate.</p></div></div></div>

Sign in / Sign up

Export Citation Format

Share Document