scholarly journals Evaluating the Performances of Biomarkers over a Restricted Domain of High Sensitivity

Mathematics ◽  
2021 ◽  
Vol 9 (21) ◽  
pp. 2826
Author(s):  
Manuel Franco ◽  
Juana-María Vivo
Keyword(s):  
Roc Curve ◽  
Performance Metrics ◽  
High Sensitivity ◽  
Roc Curves ◽  
Upper And Lower Bounds ◽  
Area Index ◽  
Colon Cancers ◽  
Partial Area ◽  
Variance Estimates ◽  
Inappropriate Expression

The burgeoning advances in high-throughput technologies have posed a great challenge to the identification of novel biomarkers for diagnosing, by contemporary models and methods, through bioinformatics-driven analysis. Diagnostic performance metrics such as the partial area under the ROC (pAUC) indexes exhibit limitations to analysing genomic data. Among other issues, the inability to differentiate between biomarkers whose ROC curves cross each other with the same pAUC value, the inappropriate expression of non-concave ROC curves, and the lack of a convenient interpretation, restrict their use in practice. Here, we have proposed the fitted partial area index (FpAUC), which is computable through an algorithm valid for any ROC curve shape, as an alternative performance summary for the evaluation of highly sensitive biomarkers. The proposed approach is based on fitter upper and lower bounds of the pAUC in a high-sensitivity region. Through variance estimates, simulations, and case studies for diagnosing leukaemia, and ovarian and colon cancers, we have proven the usefulness of the proposed metric in terms of restoring the interpretation and improving diagnostic accuracy. It is robust and feasible even when the ROC curve shows hooks, and solves performance ties between competitive biomarkers.

P–263 Life Whisperer™, an AI-based algorithm to select non invasively best quality blastocysts for transfer: A multicenter analysis

2021 ◽  
Vol 36 (Supplement_1) ◽  
Author(s):  
P Muño. Espert ◽  
Y Galiana ◽  
L Medrano ◽  
J Ballester ◽  
L Ortega ◽  
...  
Keyword(s):  
Roc Curve ◽  
Inner Cell Mass ◽  
Cell Mass ◽  
High Sensitivity ◽  
Roc Curves ◽  
Time Lapse ◽  
Clinical Pregnancy ◽  
Embryo Selection ◽  
Computer Assisted ◽  
Stage Duration

Abstract Study question Is the AI-based Life Whisperer™ (LW) tool, suitable to evaluate blastocysts quality and predict clinical pregnancy (CP) in couples undergoing ICSI cycles? Summary answer LW blastocyst score is comparable to the scores of other classification methods. This AI model showed high sensitivity and a comparable specificity for CP. What is known already The morphology grading is the most widely used method for the selection and classification of the embryos in clinical practice.However,this evaluation entails intervariability and intravariability decision among the embryologists.Recently, research has been focused on new embryo selection systems based on computer-assisted evaluation such as time-lapse with complex algorithms that allow the recognition of objective parameters of the embryo morphology.The implementation of these technologies requires substantial investments that are not available for all clinics.LW is a new embryo selection method based on AI,where specific hardware is not needed,as it is based on single blastocyst images taken with a routine microscope. Study design, size, duration Between 2017–2020, a total of 513 Day–5 blastocysts, after ICSI, comming from egg donation treatment were included in this retrospective-multicentre study.Day–5 embryos were evaluated with 3 classification methods:Gardner’s blastocyst grade (GB), the computer derived-output Eeva (EV) and LW AI-supported system. The good quality blastocysts were first evaluated using the GB and EV scores and subsequently compared with the LW scores.The sensitivity and specificity of LW was assessed to validate this system as a clinical pregnancy predictor. Participants/materials, setting, methods A total of 513 Day–5 blastocysts, from 134 oocyte donation cycles, were evaluated first by GB score: expansion (1–6), inner cell mass and throphoectoderm (A-C).EV analyses the cell division timing P2 (2cells stage duration) and P3 (3cells stage duration) differentiating three categories:High,Medium and Low(VerMilyea et al.,2014).LW scores ranked 1–10 from a single Day–5 blastocyst HR Image performed on inverted microscope,with a threshold >5 for defining a viable blastocyst.T-test and ROC-curves were used for statistical analysis. Main results and the role of chance The average of LW score obtained from GB higher blastocyst expansion score (≥4) was 7.48±0.09, while the average of LW score obtained from GB lower blastocyst expansion score (<4) was 4.69±0.3 (P < 0.001). The average of LW score yielded from GB good morphology of Inner Cell Mass and trophoectoderm (AA,AB,BA) was 7.98±0.1 while the average of LW score obtained from GB lower quality blastocyst score (BB,BC,CB,CA,AC) was 6.36±0.156 (P < 0.001).The average of LW score resulted from EV High blastocysts was 7.42±0.17, while the average of this obtained from EV low score was 6.43±0.3 (P = 0.009).A correlation between EV and LW score could be assesed, except for the blastocyst that are considered Medium score from EV. Therefore, a strong correlation between GB and LW system, as well GB+EV and LW, was found and an equivalent usability of the LW tool could be confirmed. The analyse of LW score for transferred embryos (N = 156), using ROC curve, showed a high sensitivity (0,928) but a low specificity (0,154) with a threshold of 5. Regarding our data, ROC curve shows that a threshold of 8,46 could enhance the prediction of CPR because in this point the specifity value is higher than 0.5. Limitations, reasons for caution The LW score validation compared to GB and EV methodology was carried out on a small number of embryos.Additionally,not all embryos had been transferred at the time of the analysis.Thus to enhance the accuracy of these data and the specificity of the clinical prediction, a higher sample size is needed. Wider implications of the findings: Blastocyst selection looks equivalent between all systems,but the LW tool is more objective and faster, saving time and costs significantly, without needing substantial hardware investments. Additionally,the LW-system shows almost the highest sensibility and may also improve the specificity by self-learning feeding the AI-system, thus tailoring predictions to each laboratory unique environment. Trial registration number NA

scholarly journals On the Use of Min-Max Combination of Biomarkers to Maximize the Partial Area under the ROC Curve

2019 ◽  
Vol 2019 ◽  
pp. 1-13
Author(s):  
Hua Ma ◽  
Susan Halabi ◽  
Aiyi Liu
Keyword(s):  
Roc Curve ◽  
Area Under The Curve ◽  
Predictive Performance ◽  
Roc Curves ◽  
Covariance Matrices ◽  
Distributed Data ◽  
Multivariate Distributions ◽  
Unequal Variance ◽  
Line Shapes ◽  
Partial Area

Background. Evaluation of diagnostic assays and predictive performance of biomarkers based on the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) are vital in diagnostic and targeted medicine. The partial area under the curve (pAUC) is an alternative metric focusing on a range of practical and clinical relevance of the diagnostic assay. In this article, we adopt and extend the min-max method to the estimation of the pAUC when multiple continuous scaled biomarkers are available and compare the performances of our proposed approach with existing approaches via simulations. Methods. We conducted extensive simulation studies to investigate the performance of different methods for the combination of biomarkers based on their abilities to produce the largest pAUC estimates. Data were generated from different multivariate distributions with equal and unequal variance-covariance matrices. Different shapes of the ROC curves, false positive fraction ranges, and sample size configurations were considered. We obtained the mean and standard deviation of the pAUC estimates through re-substitution and leave-one-pair-out cross-validation. Results. Our results demonstrate that the proposed method provides the largest pAUC estimates under the following three important practical scenarios: (1) multivariate normally distributed data for nondiseased and diseased participants have unequal variance-covariance matrices; or (2) the ROC curves generated from individual biomarker are relative close regardless of the latent normality distributional assumption; or (3) the ROC curves generated from individual biomarker have straight-line shapes. Conclusions. The proposed method is robust and investigators are encouraged to use this approach in the estimation of the pAUC for many practical scenarios.

scholarly journals The Child Behavior Checklist as a Screening Instrument for PTSD in Refugee Children

Children ◽  
2021 ◽  
Vol 8 (6) ◽  
pp. 521
Author(s):  
Ina Nehring ◽  
Heribert Sattel ◽  
Maesa Al-Hallak ◽  
Martin Sack ◽  
Peter Henningsen ◽  
...  
Keyword(s):  
Child Behavior ◽  
Child Behavior Checklist ◽  
High Sensitivity ◽  
Roc Curves ◽  
Structured Interview ◽  
Screening Instrument ◽  
Screening Tools ◽  
Refugee Children ◽  
Post Traumatic Stress ◽  
Sensitivity Specificity

Thousands of refugees who have entered Europe experienced threatening conditions, potentially leading to post traumatic stress disorder (PTSD), which has to be detected and treated early to avoid chronic manifestation, especially in children. We aimed to evaluate and test suitable screening tools to detect PTSD in children. Syrian refugee children aged 4–14 years were examined using the PTSD-semi-structured interview, the Kinder-DIPS, and the Child Behavior Checklist (CBCL). The latter was evaluated as a potential screening tool for PTSD using (i) the CBCL-PTSD subscale and (ii) an alternative subscale consisting of a psychometrically guided selection of items with an appropriate correlation to PTSD and a sufficient prevalence (presence in more than 20% of the cases with PTSD). For both tools we calculated sensitivity, specificity, and a receiver operating characteristic (ROC) curve. Depending on the sum score of the items, the 20-item CBCL-PTSD subscale as used in previous studies yielded a maximal sensitivity of 85% and specificity of 76%. The psychometrically guided item selection resulted in a sensitivity of 85% and a specificity of 83%. The areas under the ROC curves were the same for both tools (0.9). Both subscales may be suitable as screening instrument for PTSD in refugee children, as they reveal a high sensitivity and specificity.

Is Theory Useful for Conflict Prediction? A Response to Beger, Morgan, and Ward

2021 ◽  
pp. 002200272110267
Author(s):  
Robert A. Blair ◽  
Nicholas Sambanis
Keyword(s):  
Civil Wars ◽  
Performance Metrics ◽  
Value Added ◽  
Roc Curves ◽  
Conflict Escalation ◽  
Original Analysis ◽  
Is Theory ◽  
A Minor ◽  
Conflict Prediction ◽  
Technical Question

Beger, Morgan, and Ward (BM&W) call into question the results of our article on forecasting civil wars. They claim that our theoretically-informed model of conflict escalation under-performs more mechanical, inductive alternatives. This claim is false. BM&W’s critiques are misguided or inconsequential, and their conclusions hinge on a minor technical question regarding receiver operating characteristic (ROC) curves: should the curves be smoothed, or should empirical curves be used? BM&W assert that empirical curves should be used and all of their conclusions depend on this subjective modeling choice. We extend our original analysis to show that our theoretically-informed model performs as well as or better than more atheoretical alternatives across a range of performance metrics and robustness specifications. As in our original article, we conclude by encouraging conflict forecasters to treat the value added of theory not as an assumption, but rather as a hypothesis to test.

scholarly journals Comparison of Four Ultrasonography-Based Risk Stratification Systems in Thyroid Nodules with Nondiagnostic/Unsatisfactory Cytology: A Real-World Study

Cancers ◽  
2021 ◽  
Vol 13 (8) ◽  
pp. 1948
Author(s):  
You-Bin Lee ◽  
Young-Lyun Oh ◽  
Jung-Hee Shin ◽  
Sun-Wook Kim ◽  
Jae-Hoon Chung ◽  
...  
Keyword(s):  
Roc Curve ◽  
Thyroid Nodules ◽  
Positive Likelihood Ratio ◽  
Roc Curves ◽  
The Other ◽  
American Thyroid Association ◽  
Data Systems ◽  
Malignancy Rate ◽  
Line P ◽  
The Eu

We compared American Thyroid Association (ATA) guidelines, Korean (K)-Thyroid Imaging, Reporting and Data Systems (TIRADS), EU-TIRADS, and American College of Radiology (ACR) TIRADS in diagnosing malignancy for thyroid nodules with nondiagnostic/unsatisfactory cytology. Among 1143 nondiagnostic/unsatisfactory aspirations from April 2011 to March 2016, malignancy was detected in 39 of 89 excised nodules. The minimum malignancy rate was 7.82% in EU-TIRADS 5 and 1.87–3.00% in EU-TIRADS 3–4. In the other systems, the minimum malignancy rate was 14.29–16.19% in category 5 and ≤3% in the remaining categories. Although the EU-TIRADS category ≥ 5 exhibited the highest positive likelihood ratio (LR) of only 2.214, category ≥ 5 in the other systems yielded the highest positive LR of >5. Receiver operating characteristic (ROC) curves of all systems to predict malignancy were located statistically above the diagonal nondiscrimination line (P for ROC curve: EU-TIRADS, 0.0022; all others, 0.0001). The areas under the ROC curve (AUCs) were not significantly different among the four systems. The ATA guidelines, K-TIRADS, and ACR TIRADS may be useful to guide management for nondiagnostic/unsatisfactory nodules. The EU-TIRADS, although also useful, exhibited inferior performance in predicting malignancy for nondiagnostic/unsatisfactory nodules in Korea, an iodine-sufficient area.

scholarly journals A Novel Hybrid Method for Landslide Susceptibility Mapping-Based GeoDetector and Machine Learning Cluster: A Case of Xiaojin County, China

2021 ◽  
Vol 10 (2) ◽  
pp. 93
Author(s):  
Wei Xie ◽  
Xiaoshuang Li ◽  
Wenbin Jian ◽  
Yang Yang ◽  
Hongwei Liu ◽  
...  
Keyword(s):  
Machine Learning ◽  
Hybrid Method ◽  
Roc Curve ◽  
Landslide Susceptibility ◽  
Susceptibility Mapping ◽  
Assessment Model ◽  
Landslide Susceptibility Mapping ◽  
Support Vector ◽  
Susceptibility Map ◽  
Area Index

Landslide susceptibility mapping (LSM) could be an effective way to prevent landslide hazards and mitigate losses. The choice of conditional factors is crucial to the results of LSM, and the selection of models also plays an important role. In this study, a hybrid method including GeoDetector and machine learning cluster was developed to provide a new perspective on how to address these two issues. We defined redundant factors by quantitatively analyzing the single impact and interactive impact of the factors, which was analyzed by GeoDetector, the effect of this step was examined using mean absolute error (MAE). The machine learning cluster contains four models (artificial neural network (ANN), Bayesian network (BN), logistic regression (LR), and support vector machines (SVM)) and automatically selects the best one for generating LSM. The receiver operating characteristic (ROC) curve, prediction accuracy, and the seed cell area index (SCAI) methods were used to evaluate these methods. The results show that the SVM model had the best performance in the machine learning cluster with the area under the ROC curve of 0.928 and with an accuracy of 83.86%. Therefore, SVM was chosen as the assessment model to map the landslide susceptibility of the study area. The landslide susceptibility map demonstrated fit with landslide inventory, indicated the hybrid method is effective in screening landslide influences and assessing landslide susceptibility.

Diagnostic and prognostic significance of miR-451a in patients with atherosclerosis

Vascular ◽  
2021 ◽  
pp. 170853812110585
Author(s):  
Baizhi Wang ◽  
Xingliang Duan ◽  
Qing Xu ◽  
Yani Li
Keyword(s):  
Operating Characteristic ◽  
Cox Regression ◽  
Prognostic Biomarker ◽  
Prognostic Significance ◽  
High Sensitivity ◽  
Roc Curves ◽  
Cox Regression Analysis ◽  
Chi Squared ◽  
Cardiovascular Endpoint ◽  
Inflammatory Vascular Disease

Objectives Atherosclerosis (AS) is a chronic inflammatory vascular disease. This study aimed to detect the expression level of miR-451a and investigate the diagnostic and prognostic values of miR-451a for AS patients. Methods The relative expression of miR-451a was assessed by qRT-PCR. Comparison of groups was analyzed with the t-test and chi-squared test. Pearson analysis was used to validate the correlation of miR-451 with CRP and CIMT. The receiver operating characteristic (ROC) curves, K-M analysis, and Cox regression analysis were conducted to explore the roles of miR-451a in diagnosing AS patients and predicting outcomes of AS patients. Results The expression of miR-451a was significantly decreased in the serum of AS patients. The results of Pearson analysis showed the expression of miR-451a was negatively correlated with CRP and CIMT. The data of ROC proposed miR-451a could differentiate AS patients from healthy individuals with high sensitivity and specificity. K-M analysis and Cox regression showed miR-451a might be an independent biomarker of suffering cardiovascular endpoint diseases in AS patients. The expression of miR-451a was obviously inhibited in AS patients with cardiovascular endpoint events. Conclusion Deregulation of miR-451a might be associated with the development of AS. MiR-451a might be used as a promising diagnostic and prognostic biomarker for clinical treatment of AS patients.

scholarly journals Comparison of Lipoprotein-Associated Phospholipase A2 and High Sensitive C-Reactive Protein as Determinants of Metabolic Syndrome in Subjects without Coronary Heart Disease: In Search of the Best Predictor

2015 ◽  
Vol 2015 ◽  
pp. 1-7 ◽  
Author(s):  
Mónica Acevedo ◽  
Paola Varleta ◽  
Verónica Kramer ◽  
Giovanna Valentino ◽  
Teresa Quiroga ◽  
...  
Keyword(s):  
Metabolic Syndrome ◽  
Phospholipase A2 ◽  
High Sensitivity ◽  
Roc Curves ◽  
Activity Levels ◽  
Pla2 Activity ◽  
C Reactive Protein ◽  
Cross Sectional ◽  
Reactive Protein ◽  
Cv Disease

High sensitivity C-reactive protein (hsCRP) is a marker of metabolic syndrome (MS) and cardiovascular (CV) disease. Lipoprotein-associated phospholipase A2 (Lp-PLA2) also predicts CV disease. There are no reports comparing these markers as predictors of MS.Methods. Cross-sectional study comparing Lp-PLA2 and hsCRP as predictors of MS in asymptomatic subjects was carried out; 152 subjects without known atherosclerosis participated. Data were collected on demographics, cardiovascular risk factors, anthropometric and biochemical measurements, and hsCRP and Lp-PLA2 activity levels. A logistic regression analysis was performed with each biomarker and receiver operating characteristic (ROC) curves were constructed for MS.Results. Mean age was 46 ± 11 years, and 38% of the subjects had MS. Mean Lp-PLA2 activity was 185 ± 48 nmol/mL/min, and mean hsCRP was 2.1 ± 2.2 mg/L. Subjects with MS had significantly higher levels of Lp-PLA2 (P=0.03) and hsCRP (P<0.0001) than those without MS. ROC curves showed that both markers predicted MS.Conclusion. Lp-PLA2 and hsCRP are elevated in subjects with MS. Both biomarkers were independent and significant predictors for MS, emphasizing the role of inflammation in MS. Further research is necessary to determine if inflammation predicts a higher risk for CV events in MS subjects.

scholarly journals Cardiac biomarkers as prognosticators among SARS-CoV-19 patients in a tertiary hospital in Philippines

2021 ◽  
Vol 42 (Supplement_1) ◽  
Author(s):  
R Ong ◽  
C Chacon ◽  
S Javier
Keyword(s):  
Creatine Kinase ◽  
Logistic Regression ◽  
Hospital Mortality ◽  
Roc Curve ◽  
Predictive Value ◽  
Troponin I ◽  
Binary Logistic Regression ◽  
High Sensitivity ◽  
Cardiac Biomarkers ◽  
Optimal Cutoff

Abstract Background There is overwhelming volume of confirmed cases of COVID-19, despite this numerous knowledge gaps remain in the diagnosis, management, and prognostication of this novel coronavirus infection, making prevention and control a challenge. Methods This retrospective cohort study included patients with real-time reverse transcriptase polymerase chain reaction (rRT-PCR)-confirmed COVID-19. Binary logistic regression was used to determine the association between the cardiac biomarkers and in-hospital mortality. ROC, AUC, and cutoff analyses were used to determine optimal cutoff values for the cardiac biomarkers. Results A total of 90 subjects with a complete panel of cardiac biomarkers out of the 224 rRT-PCR confirmed cases were included. The median age was 57 years (IQR, 47–67 years), majority were males. Sixty-six (77.6%) subjects survived while 19 (22.4%) expired. The most common presenting symptom was fever (75.6%), and the most common comorbidity was hypertension (67.8%). Spearman rho correlation analysis showed moderate positive association of high sensitivity troponin I (hsTnI) with in-hospital mortality (R, 0.434, p = &lt;0.001). Multivariate binary logistic regression analysis showed that creatine kinase and hsTnI were independently associated with in-hospital mortality (OR, 4.103 [95% CI, 1.241–13.563], p=0.021; and OR, 7.899 [95% CI, 2.430–25.675], p=0.001, respectively). ROC curve analysis showed that hsTnI was a good predictor for in-hospital mortality (AUC, 0.829 [95% CI, 0.735–0.923], p = &lt;0.001) and that creatine kinase was a poor predictor (AUC, 0.677 [95% CI, 0.531–0.823], p=0.018). Optimal cutoff point derived from the ROC curve for hsTnI was 0.010 ng/ml (J, 0.574) with a sensitivity of 84% (TPR, 0.842 [95% CI, 0.604–0.966]), specificity of 73% (TNR, 0.732 [95% CI, 0.614–0.386]), and an adjusted negative predictive value of 99% (Known prevalence*adjusted NPV, 0.989), a positive likelihood ratio of 20% (LR+, 3.147 [95% CI, 2.044–4.844]) and a negative likelihood ratio of 30% (LR−, 0.216 [95% CI, 0.076–0.615]). Conclusion High sensitivity troponin I level was a good tool with a very high negative predictive value in significantly predicting in-hospital mortality among rRT-PCR positive COVID-19 patients. FUNDunding Acknowledgement Type of funding sources: None. ROC Curve

Sign in / Sign up

Export Citation Format

Share Document