Conventional administration and scoring procedures suppress the diagnostic accuracy of a performance-based test designed to assess balance ability in lower limb prosthesis users

2019 ◽  
Vol 43 (4) ◽  
pp. 402-408 ◽  
Author(s):  
Andrew Sawers ◽  
Brian J Hafner
Keyword(s):  
Diagnostic Accuracy ◽  
Receiver Operating Characteristic Curve ◽  
Receiver Operating Characteristic ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Practice Effects ◽  
Balance Test ◽  
Walking Test ◽  
History Of ◽  
Operating Characteristic Curve

Background:Practice effects have been observed among performance-based clinical tests administered to prosthesis-users. Their impact on test applications remains unknown.Objective:To determine whether scoring a clinical balance test using conventional procedures that do not accommodate practice effects reduces its diagnostic accuracy relative to scoring it using recommended procedures that do accommodate practice effects.Study Design:Cross-sectional study.Methods:Narrowing Beam Walking Test data from 40 prosthesis users was scored using recommended methods (i.e. average of trials 3–5) and conventional methods applied to other tests (i.e. mean or best of trials 1–3). Area under the receiver operating characteristic curve for each method was compared to 0.50, to determine if it was better than chance at identifying prosthesis-users with a history of falls, and to 0.80, to determine if it surpassed a threshold recommended for diagnostic accuracy.Results:Receiver operating characteristic curve area decreased when the Narrowing Beam Walking Test was scored using conventional rather than recommended procedures. Furthermore, when scored using conventional procedures, the NBWT no longer discriminated between prosthesis-users with and without a history of falls with a probability greater than chance, or exceeded recommended diagnostic thresholds.Conclusion:Scoring the Narrowing Beam Walking Test using conventional procedures that do not accommodate practice effects decreased its diagnostic accuracy among prosthesis-users relative to recommended procedures. Conventional scoring procedures may limit the effectiveness of performance-based tests used to screen for fall risk in prosthesis-users because they do not mitigate practice effects. The influence of practice effects on other tests, and test applications (e.g. clinical evaluation and prediction), is warranted.Clinical relevanceScoring a clinical balance test using conventional procedures that do not mitigate practice effects reduced its diagnostic accuracy. Changing administration and scoring procedures to accommodate practice effects should be considered to improve the diagnostic accuracy of other performance-based balance tests.

Bootstrap-based testing approaches for the assessment of the diagnostic accuracy of biomarkers subject to a limit of detection

2018 ◽  
Vol 28 (5) ◽  
pp. 1564-1578
Author(s):  
Alba M Franco-Pereira ◽  
Christos T Nakas ◽  
Alexander B Leichtle ◽  
M Carmen Pardo
Keyword(s):  
Diagnostic Accuracy ◽  
Receiver Operating Characteristic Curve ◽  
Receiver Operating Characteristic ◽  
Simulation Study ◽  
Operating Characteristic ◽  
Limit Of Detection ◽  
Characteristic Curve ◽  
Operating Characteristic Curve ◽  
Receiver Operating ◽  
Non Parametric

Assessment of the diagnostic accuracy of biomarkers through receiver operating characteristic curve analysis frequently involves a limit of detection imposed by the laboratory analytical system precision. As a consequence, measurements below a certain level are undetectable and ignoring these is known to lead to negatively biased estimates of the area under the receiver operating characteristic curve. In this article, we introduce two receiver operating characteristic curve-based parametric approaches that tackle the issue of correct assessment of diagnostic markers in the presence of a limit of detection. Proposed approaches are simulation-based utilising bootstrap methodology. Non-parametric alternatives that are naively used in the literature do not solve the inherent problem of limit of detection values which are treated as censored observations. However, the latter seems to perform adequately well in our simulation study. Nonparametric bootstrap was consistently used throughout, while other bootstrap alternatives performed similarly in our pilot simulation study. The simulation study involves the comparison of parametric and non-parametric options described here versus alternative strategies that are routinely used in the literature. We apply all methods to a study-setting resembling a chemical quasi-standard situation, where compound tumour biomarkers were searched within a multi-variable set of measurements to discriminate between two groups, namely colorectal cancer and controls. We focus in the assessment of glutamine and methionine.

scholarly journals Bootstrap-based procedures for inference in nonparametric receiver-operating characteristic curve regression analysis

2017 ◽  
Vol 27 (3) ◽  
pp. 740-764 ◽  
Author(s):  
María Xosé Rodríguez-Álvarez ◽  
Javier Roca-Pardiñas ◽  
Carmen Cadarso-Suárez ◽  
Pablo G Tahoces
Keyword(s):  
Diagnostic Accuracy ◽  
Receiver Operating Characteristic Curve ◽  
Receiver Operating Characteristic ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Diagnostic System ◽  
R Package ◽  
Additive Models ◽  
Operating Characteristic Curve ◽  
Receiver Operating

Prior to using a diagnostic test in a routine clinical setting, the rigorous evaluation of its diagnostic accuracy is essential. The receiver-operating characteristic curve is the measure of accuracy most widely used for continuous diagnostic tests. However, the possible impact of extra information about the patient (or even the environment) on diagnostic accuracy also needs to be assessed. In this paper, we focus on an estimator for the covariate-specific receiver-operating characteristic curve based on direct regression modelling and nonparametric smoothing techniques. This approach defines the class of generalised additive models for the receiver-operating characteristic curve. The main aim of the paper is to offer new inferential procedures for testing the effect of covariates on the conditional receiver-operating characteristic curve within the above-mentioned class. Specifically, two different bootstrap-based tests are suggested to check (a) the possible effect of continuous covariates on the receiver-operating characteristic curve and (b) the presence of factor-by-curve interaction terms. The validity of the proposed bootstrap-based procedures is supported by simulations. To facilitate the application of these new procedures in practice, an R-package, known as npROCRegression, is provided and briefly described. Finally, data derived from a computer-aided diagnostic system for the automatic detection of tumour masses in breast cancer is analysed.

scholarly journals Influence of VistaScan image enhancement filters on diagnosis of simulated periapical lesions on intraoral radiographs

2019 ◽  
Vol 48 (2) ◽  
pp. 20180146 ◽  
Author(s):  
Danieli Moura Brasil ◽  
Mayra Cristina Yamasaki ◽  
Gustavo Machado Santaella ◽  
Maria Carolina Zumstein Guido ◽  
Deborah Queiroz Freitas ◽  
...  
Keyword(s):  
Standard Deviation ◽  
Diagnostic Accuracy ◽  
Receiver Operating Characteristic Curve ◽  
Receiver Operating Characteristic ◽  
Image Enhancement ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Periapical Lesions ◽  
Operating Characteristic Curve ◽  
Receiver Operating

Objective: To evaluate whether image enhancement filters of VistaScan system improve the diagnostic accuracy of simulated periapical lesions. Methods: 10 sockets were prepared on bovine ribs to fit a bovine tooth. Bone defects were created and successively enlarged providing four groups (n = 10): Group 0, absence of lesions; Group 1, periapical lesions with 1.6  mm in diameter; Group 2, with 1.8  mm in diameter; and Group 3, with 2.1  mm in diameter. Periapical radiographs were taken using a photostimulable storage phosphor plate and DBSWIN software. VistaScan filters were applied and the images were allocated into seven groups: Nonfiltered, Fine, Caries 1, Caries 2, Endodontic, Periodontal and Noise Reduction. All the 280 images were assessed about the presence or absence of periapical lesions. Pixel intensities standard deviation were compared between nonfiltered and filtered images. Two-Way Analysis of Variance and the post hoc Tukey’s test were used to compare area under the ROC curve, sensitivity and specificity. Results: VistaScan filters showed no significant difference for area under receiver operating characteristic curve (p = 0.124), sensitivity (p = 0.835) and specificity (p = 0.832). Area under receiver operating characteristic curve (p = 0.000) and sensitivity (p = 0.000) in 2.1  mm lesions size were significantly higher than in 1.6  mm and 1.8  mm lesions size. Pixel intensities standard deviation was significantly changed in the filtered images compared to nonfiltered ones (p < 0.01), except for Fine in the bone region (p > 0.05). Conclusion: VistaScan enhancement filters do not influence the diagnostic accuracy of simulated periapical lesions. On the other hand, larger lesions were more frequently detected. The filters change the pixel intensities reducing or intensifying the differences between similar regions.

A validated novel preoperative index to predict the extent of intraperitoneal contamination in patients with acute abdominal pathology: A cohort study

2019 ◽  
Vol 30 (7-8) ◽  
pp. 221-228
Author(s):  
Shahab Hajibandeh ◽  
Shahin Hajibandeh ◽  
Nicholas Hobbs ◽  
Jigar Shah ◽  
Matthew Harris ◽  
...  
Keyword(s):  
Receiver Operating Characteristic Curve ◽  
Receiver Operating Characteristic ◽  
Operating Characteristic ◽  
Validation Cohort ◽  
Characteristic Curve ◽  
Curve Analysis ◽  
Emergency Laparotomy ◽  
Contamination Index ◽  
Operating Characteristic Curve ◽  
Receiver Operating

Aims To investigate whether an intraperitoneal contamination index (ICI) derived from combined preoperative levels of C-reactive protein, lactate, neutrophils, lymphocytes and albumin could predict the extent of intraperitoneal contamination in patients with acute abdominal pathology. Methods Patients aged over 18 who underwent emergency laparotomy for acute abdominal pathology between January 2014 and October 2018 were randomly divided into primary and validation cohorts. The proposed intraperitoneal contamination index was calculated for each patient in each cohort. Receiver operating characteristic curve analysis was performed to determine discrimination of the index and cut-off values of preoperative intraperitoneal contamination index that could predict the extent of intraperitoneal contamination. Results Overall, 468 patients were included in this study; 234 in the primary cohort and 234 in the validation cohort. The analyses identified intraperitoneal contamination index of 24.77 and 24.32 as cut-off values for purulent contamination in the primary cohort (area under the curve (AUC): 0.73, P < 0.0001; sensitivity: 84%, specificity: 60%) and validation cohort (AUC: 0.83, P < 0.0001; sensitivity: 91%, specificity: 69%), respectively. Receiver operating characteristic curve analysis also identified intraperitoneal contamination index of 33.70 and 33.41 as cut-off values for feculent contamination in the primary cohort (AUC: 0.78, P < 0.0001; sensitivity: 87%, specificity: 64%) and validation cohort (AUC: 0.79, P < 0.0001; sensitivity: 86%, specificity: 73%), respectively. Conclusions As a predictive measure which is derived purely from biomarkers, intraperitoneal contamination index may be accurate enough to predict the extent of intraperitoneal contamination in patients with acute abdominal pathology and to facilitate decision-making together with clinical and radiological findings.

scholarly journals Evaluation of factors that predict the success rate of trial of labor after the cesarean section

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Yang Mi ◽  
Pengfei Qu ◽  
Na Guo ◽  
Ruimiao Bai ◽  
Jiayi Gao ◽  
...  
Keyword(s):  
Logistic Regression ◽  
Cesarean Section ◽  
Receiver Operating Characteristic Curve ◽  
Success Rate ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Predictive Ability ◽  
Training Set ◽  
History Of ◽  
Operating Characteristic Curve

Abstract Background For most women who have had a previous cesarean section, vaginal birth after cesarean section (VBAC) is a reasonable and safe choice, but which will increase the risk of adverse outcomes such as uterine rupture. In order to reduce the risk, we evaluated the factors that may affect VBAC and and established a model for predicting the success rate of trial of the labor after cesarean section (TOLAC). Methods All patients who gave birth at Northwest Women’s and Children’s Hospital from January 2016 to December 2018, had a history of cesarean section and voluntarily chose the TOLAC were recruited. Among them, 80% of the population was randomly assigned to the training set, while the remaining 20% were assigned to the external validation set. In the training set, univariate and multivariate logistic regression models were used to identify indicators related to successful TOLAC. A nomogram was constructed based on the results of multiple logistic regression analysis, and the selected variables included in the nomogram were used to predict the probability of successfully obtaining TOLAC. The area under the receiver operating characteristic curve was used to judge the predictive ability of the model. Results A total of 778 pregnant women were included in this study. Among them, 595 (76.48%) successfully underwent TOLAC, whereas 183 (23.52%) failed and switched to cesarean section. In multi-factor logistic regression, parity = 1, pre-pregnancy BMI < 24 kg/m2, cervical score ≥ 5, a history of previous vaginal delivery and neonatal birthweight < 3300 g were associated with the success of TOLAC. The area under the receiver operating characteristic curve in the prediction and validation models was 0.815 (95% CI: 0.762–0.854) and 0.730 (95% CI: 0.652–0.808), respectively, indicating that the nomogram prediction model had medium discriminative power. Conclusion The TOLAC was useful to reducing the cesarean section rate. Being primiparous, not overweight or obese, having a cervical score ≥ 5, a history of previous vaginal delivery or neonatal birthweight < 3300 g were protective indicators. In this study, the validated model had an approving predictive ability.

A conditional approach for the receiver operating characteristic curve construction to evaluate diagnostic test performance in a family-matched case–control design

2021 ◽  
pp. 096228022199595
Author(s):  
Yalda Zarnegarnia ◽  
Shari Messinger
Keyword(s):  
Receiver Operating Characteristic Curve ◽  
Receiver Operating Characteristic ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Case Control ◽  
Paired Data ◽  
Characteristic Curves ◽  
Receiver Operating Characteristic Curves ◽  
Operating Characteristic Curve ◽  
Receiver Operating

Receiver operating characteristic curves are widely used in medical research to illustrate biomarker performance in binary classification, particularly with respect to disease or health status. Study designs that include related subjects, such as siblings, usually have common environmental or genetic factors giving rise to correlated biomarker data. The design could be used to improve detection of biomarkers informative of increased risk, allowing initiation of treatment to stop or slow disease progression. Available methods for receiver operating characteristic construction do not take advantage of correlation inherent in this design to improve biomarker performance. This paper will briefly review some developed methods for receiver operating characteristic curve estimation in settings with correlated data from case–control designs and will discuss the limitations of current methods for analyzing correlated familial paired data. An alternative approach using conditional receiver operating characteristic curves will be demonstrated. The proposed approach will use information about correlation among biomarker values, producing conditional receiver operating characteristic curves that evaluate the ability of a biomarker to discriminate between affected and unaffected subjects in a familial paired design.

Sign in / Sign up

Export Citation Format

Share Document