scholarly journals Inferring sensitivity and specificity of phenotyping algorithms using positive and negative predictive value in validation study in observational health data

2018 ◽  
Vol 3 (4) ◽  
Author(s):  
Mingkai Peng ◽  
Rosa Gini ◽  
Tyler Williamson
Keyword(s):  
Sensitivity And Specificity ◽  
Validation Study ◽  
Validation Studies ◽  
Indirect Method ◽  
Direct Method ◽  
Population Sample ◽  
Simulated Data ◽  
Health Data ◽  
Comparable Accuracy ◽  
Low Prevalence

IntroductionIn observational health data, phenotyping algorithms are needed to process raw information into clinically relevant features. Validation studies traditionally estimate sensitivity and specificity by comparing the phenotyping algorithm with a reference standard on a population sample. There are challenges to conduct validation studies for conditions with low prevalence. Objectives and ApproachWe propose a novel and efficient method for conducting validation studies to indirectly estimate the sensitivity and specificity. We simulated datasets with different levels of disease prevalence and phenotyping algorithms with different sensitivities and specificity. We applied both the traditional (direct) and new (indirect) method on simulated data to estimate the sensitivity and specificity and compare the performance of the two methods. We also designed a gate to exclude true negatives to improve study efficiency on conditions with low prevalence and sensitive analysis was conducted on the imperfect gate. ResultsThe new (indirect) method provided better or comparable accuracy in estimating both sensitivity and specificity compared to the traditional (direct) method. Applying a gate enabled us to conduct validation study in conditions with very low prevalence. An imperfect gate results in the overestimation of sensitivity but has minimal effect on specificity. Conclusion/ImplicationsThe new (indirect) method provides an alternative way to conduct validation studies in observational health data with improvement in estimating accuracy.

scholarly journals Estimation of sensitivity and specificity and calculation of sample size for a validation study with stratified sampling

2020 ◽  
Author(s):  
Kiyoshi Kubota ◽  
Masao Iwagami ◽  
Takuhiro Yamaguchi
Keyword(s):  
Sample Size ◽  
Sensitivity And Specificity ◽  
Validation Study ◽  
Predictive Value ◽  
Validation Studies ◽  
Bootstrap Method ◽  
Stratified Sampling ◽  
Delta Method ◽  
Estimate Sample Size ◽  
The Bootstrap Method

Abstract Background:We propose and evaluate the approximation formulae for the 95% confidence intervals (CIs) of the sensitivity and specificity and a formula to estimate sample size in a validation study with stratified sampling where positive samples satisfying the outcome definition and negative samples that do not are selected with different extraction fractions. Methods:We used the delta method to derive the approximation formulae for estimating the sensitivity and specificity and their CIs. From those formulae, we derived the formula to estimate the size of negative samples required to achieve the intended precision and the formula to estimate the precision for a negative sample size arbitrarily selected by the investigator. We conducted simulation studies in a population where 4% were outcome definition positive, the positive predictive value (PPV)=0.8, and the negative predictive value (NPV)=0.96, 0.98 and 0.99. The size of negative samples, n0, was either selected to make the 95% CI fall within ± 0.1, 0.15 and 0.2 or set arbitrarily as 150, 300 and 600. We assumed a binomial distribution for the positive and negative samples. The coverage of the 95% CIs of the sensitivity and specificity was calculated as the proportion of CIs including the sensitivity and specificity in the population, respectively. For selected studies, the coverage was also estimated by the bootstrap method. The sample size was evaluated by examining whether the observed precision was within the pre-specified value.Results:For the sensitivity, the coverage of the approximated 95% CIs was larger than 0.95 in most studies but in 9 of 18 selected studies derived by the bootstrap method. For the specificity, the coverage of the approximated 95% CIs was approximately 0.93 in most studies, but the coverage was more than 0.95 in all 18 studies derived by the bootstrap method. The calculated size of negative samples yielded precisions within the pre-specified values in most of the studies.Conclusion:The approximation formulae for the 95% CIs of the sensitivity and specificity for stratified validation studies are presented. These formulae will help in conducting and analysing validation studies with stratified sampling.

scholarly journals A Validation Study of a Commercial Wearable Device to Automatically Detect and Estimate Sleep

Biosensors ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 185
Author(s):  
Dean J. Miller ◽  
Gregory D. Roach ◽  
Michele Lastella ◽  
Aaron T. Scanlan ◽  
Clint R. Bellenger ◽  
...  
Keyword(s):  
Eye Movement ◽  
Sensitivity And Specificity ◽  
Validation Study ◽  
Slow Wave Sleep ◽  
Total Sleep Time ◽  
Sleep Time ◽  
Rapid Eye Movement Sleep ◽  
Two Stage ◽  
Manual Adjustment ◽  
Light Sleep

The aims of this study were to: (1) compare actigraphy (ACTICAL) and a commercially available sleep wearable (i.e., WHOOP) under two functionalities (i.e., sleep auto-detection (WHOOP-AUTO) and manual adjustment of sleep (WHOOP-MANUAL)) for two-stage categorisation of sleep (sleep or wake) against polysomnography, and; (2) compare WHOOP-AUTO and WHOOP-MANUAL for four-stage categorisation of sleep (wake, light sleep, slow wave sleep (SWS), or rapid eye movement sleep (REM)) against polysomnography. Six healthy adults (male: n = 3; female: n = 3; age: 23.0 ± 2.2 yr) participated in the nine-night protocol. Fifty-four sleeps assessed by ACTICAL, WHOOP-AUTO and WHOOP-MANUAL were compared to polysomnography using difference testing, Bland–Altman comparisons, and 30-s epoch-by-epoch comparisons. Compared to polysomnography, ACTICAL overestimated total sleep time (37.6 min) and underestimated wake (−37.6 min); WHOOP-AUTO underestimated SWS (−15.5 min); and WHOOP-MANUAL underestimated wake (−16.7 min). For ACTICAL, sensitivity for sleep, specificity for wake and overall agreement were 98%, 60% and 89%, respectively. For WHOOP-AUTO, sensitivity for sleep, wake, and agreement for two-stage and four-stage categorisation of sleep were 90%, 60%, 86% and 63%, respectively. For WHOOP-MANUAL, sensitivity for sleep, wake, and agreement for two-stage and four-stage categorisation of sleep were 97%, 45%, 90% and 62%, respectively. WHOOP-AUTO and WHOOP-MANUAL have a similar sensitivity and specificity to actigraphy for two-stage categorisation of sleep and can be used as a practical alternative to polysomnography for two-stage categorisation of sleep and four-stage categorisation of sleep.

scholarly journals Modeling Linkage Disequilibrium and Identifying Recombination Hotspots Using Single-Nucleotide Polymorphism Data

Genetics ◽  
2003 ◽  
Vol 165 (4) ◽  
pp. 2213-2233 ◽  
Author(s):  
Na Li ◽  
Matthew Stephens
Keyword(s):  
Linkage Disequilibrium ◽  
Recombination Rate ◽  
Population Sample ◽  
Simulated Data ◽  
Region Of Interest ◽  
Population Data ◽  
Recombination Rates ◽  
Single Nucleotide ◽  
Recombination Hotspots ◽  
Genomic Regions

AbstractWe introduce a new statistical model for patterns of linkage disequilibrium (LD) among multiple SNPs in a population sample. The model overcomes limitations of existing approaches to understanding, summarizing, and interpreting LD by (i) relating patterns of LD directly to the underlying recombination process; (ii) considering all loci simultaneously, rather than pairwise; (iii) avoiding the assumption that LD necessarily has a “block-like” structure; and (iv) being computationally tractable for huge genomic regions (up to complete chromosomes). We examine in detail one natural application of the model: estimation of underlying recombination rates from population data. Using simulation, we show that in the case where recombination is assumed constant across the region of interest, recombination rate estimates based on our model are competitive with the very best of current available methods. More importantly, we demonstrate, on real and simulated data, the potential of the model to help identify and quantify fine-scale variation in recombination rate from population data. We also outline how the model could be useful in other contexts, such as in the development of more efficient haplotype-based methods for LD mapping.

scholarly journals Evaluation of automated microvascular flow analysis software AVA 4: a validation study

2021 ◽  
Vol 9 (1) ◽  
Author(s):  
Christian S. Guay ◽  
Mariam Khebir ◽  
T. Shiva Shahiri ◽  
Ariana Szilagyi ◽  
Erin Elizabeth Cole ◽  
...  
Keyword(s):  
General Anesthesia ◽  
Validation Study ◽  
Validation Studies ◽  
Gold Standard ◽  
Flow Analysis ◽  
Automated Analysis ◽  
Altman Analysis ◽  
Analysis Software ◽  
Bland Altman Analysis ◽  
Microvascular Flow

Abstract Background Real-time automated analysis of videos of the microvasculature is an essential step in the development of research protocols and clinical algorithms that incorporate point-of-care microvascular analysis. In response to the call for validation studies of available automated analysis software by the European Society of Intensive Care Medicine, and building on a previous validation study in sheep, we report the first human validation study of AVA 4. Methods Two retrospective perioperative datasets of human microcirculation videos (P1 and P2) and one prospective healthy volunteer dataset (V1) were used in this validation study. Video quality was assessed using the modified Microcirculation Image Quality Selection (MIQS) score. Videos were initially analyzed with (1) AVA software 3.2 by two experienced investigators using the gold standard semi-automated method, followed by an analysis with (2) AVA automated software 4.1. Microvascular variables measured were perfused vessel density (PVD), total vessel density (TVD), and proportion of perfused vessels (PPV). Bland–Altman analysis and intraclass correlation coefficients (ICC) were used to measure agreement between the two methods. Each method’s ability to discriminate between microcirculatory states before and after induction of general anesthesia was assessed using paired t-tests. Results Fifty-two videos from P1, 128 videos from P2 and 26 videos from V1 met inclusion criteria for analysis. Correlational analysis and Bland–Altman analysis revealed poor agreement and no correlation between AVA 4.1 and AVA 3.2. Following the induction of general anesthesia, TVD and PVD measured using AVA 3.2 increased significantly for P1 (p < 0.05) and P2 (p < 0.05). However, these changes could not be replicated with the data generated by AVA 4.1. Conclusions AVA 4.1 is not a suitable tool for research or clinical purposes at this time. Future validation studies of automated microvascular flow analysis software should aim to measure the new software’s agreement with the gold standard, its ability to discriminate between clinical states and the quality thresholds at which its performance becomes unacceptable.

scholarly journals The Preference of Direct or Indirect Methods in Preparing the Statement of Cash Flows in Decision Making: An Academic Perspective

2016 ◽  
Vol 8 (2) ◽  
pp. 206
Author(s):  
Abdullah, S. Hardan ◽  
Majed, A. Qabajeh ◽  
Aymen, M. Alshanti
Keyword(s):  
Decision Making ◽  
Indirect Method ◽  
Direct Method ◽  
Cash Flows ◽  
Decision Makers ◽  
Academic Rank ◽  
Indirect Methods ◽  
Statement Of Cash Flows ◽  
Qualitative Characteristics ◽  
Academic Sector

Two methods are used when reporting cash flows from operating activities: the direct method or the indirect method, both are acceptable from IAS with a preference of direct method. Thus, this paper examines which method of reporting the statement of cash flows provides useful information the decision makers rely on for decision making purposes. To achieve this aim, participants were selected from academic sector represented by universities professors. The study is based on the conceptual framework: qualitative characteristics of accounting information. To be useful, information must be relevant and represents faithfully what it claims to represent. In order to distinguish more useful financial information from those less useful, enhancing qualitative characteristics were examined. Results show that academic professors provide support for direct method of reporting cash flows over indirect method. The study sought to determine the effect of academic rank on these results. Evidence reveals that full and associate professors endorsed the preference of direct method more than assistant professors and lecturers. These results recommend the legislative bodies and entities to adopt the direct method in preparation the statement of cash flows.

scholarly journals Predicting Outcome and Therapy Response in mCRC Patients Using an Indirect Method for CTCs Detection by a Multigene Expression Panel: A Multicentric Prospective Validation Study

2017 ◽  
Vol 18 (6) ◽  
pp. 1265 ◽  
Author(s):  
Yolanda Vidal Insua ◽  
Juan de la Cámara ◽  
Elena Brozos Vázquez ◽  
Ana Fernández ◽  
Francisca Vázquez Rivera ◽  
...  
Keyword(s):  
Validation Study ◽  
Indirect Method ◽  
Therapy Response ◽  
Multigene Expression ◽  
Predicting Outcome

scholarly journals Теплоемкость и магнитокалорический эффект соединений LaFe-=SUB=-11.2-x-=/SUB=-Mn-=SUB=-x-=/SUB=-Co-=SUB=-0.7-=/SUB=-Si-=SUB=-1.1-=/SUB=- (x=0, 0.1, 0.2, 0.3)

2020 ◽  
Vol 62 (5) ◽  
pp. 752
Author(s):  
Н.З. Абдулкадирова ◽  
А.М. Алиев ◽  
А.Г. Гамзатов ◽  
P. Gebara
Keyword(s):  
Heat Capacity ◽  
Magnetic Fields ◽  
Curie Temperature ◽  
Specific Heat ◽  
Magnetocaloric Effect ◽  
Intermetallic Compounds ◽  
Indirect Method ◽  
Direct Method ◽  
Capacity Data ◽  
Value Changes

The specific heat and magnetocaloric effect of the LaFe11.2-хMnxCo0.7Si1.1 intermetallic compounds (x = 0.1, 0.2, 0.3) were measured in the temperature range 80–300 K and in magnetic fields up to 8 T. The magnetocaloric effect (MCE) was estimated using two methods: direct method in cyclic magnetic fields, as well as an indirect method from heat capacity data. It was shown that an increase in the concentration of Mn atoms leads to a shift in the Curie temperature of the TC toward lower temperatures, while the FM value changes slightly.

scholarly journals European Portuguese Version of the Clinical Frailty Scale: Translation, Cultural Adaptation and Validation Study

2021 ◽  
Vol 34 (13) ◽  
Author(s):  
Mário Pereira Pinto ◽  
Sónia Martins ◽  
Edgar Mesquita ◽  
Lia Fernandes
Keyword(s):  
Sensitivity And Specificity ◽  
Validation Study ◽  
Convergent Validity ◽  
Outpatient Clinics ◽  
Clinical Frailty Scale ◽  
Retest Reliability ◽  
European Portuguese ◽  
Tilburg Frailty Indicator ◽  
Test Retest Reliability ◽  
Frailty Indicator

Introduction: This study aims to describe the translation and adaptation of the European Portuguese Clinical Frailty Scale and assess its convergent validity and test-retest reliability.Material and Methods: This validation study included a sample of elderly people admitted in two convalescence units from the National Network of Integrated Continuous Care in Northern Portugal and followed in two outpatient clinics of social solidarity institutions. Convergent validity of the scale was evaluated, against Tilburg Frailty Indicator. Test-retest reliability, sensitivity and specificity were assessed.Results: Overall, 51 patients were included (mean age = 78 years old). The Clinical Frailty Scale identified 43.1% patients with frailty. Kappa values for test-retest reliability (non-frail/frail) was 1.00. The intraclass correlation coefficient for the 9-point total scale was 0.999. A correlation between Clinical Frailty Scale and Tilburg Frailty Indicator was also found (rs = 0.683; p < 0.001). The Cohen’s kappa coefficient was 0.423 in the agreement analysis between these scales. The results for sensitivity and specificity defined that 62.0% of patients were true positives and 81.8% true negatives. The scale accuracy determined by the receiver operating characteristics curve analysis was 0.782.Discussion: This scale showed an excellent test-retest reliability. Robust results on convergent validity were also achieved, with a moderate correlation and agreement with the Tilburg Frailty Indicator, showing good sensitivity and accuracy, as well as high specificity.Conclusion: This version has an excellent test-retest reliability and good convergent validity, and is both a reliable and valid test for application in clinical practice for assessing Portuguese elderly population admitted in convalescence units and outpatient clinics.

Flow Measurement to a Kaplan Turbine Using a Direct Method and an Indirect One

2016 ◽  
Vol 823 ◽  
pp. 361-366
Author(s):  
Ana Maria Budai ◽  
Adrian Cuzmos ◽  
Cristian Fanica ◽  
Damaschin Pepa ◽  
Cosmin Ursoniu ◽  
...  
Keyword(s):  
Direct Measurement ◽  
Flow Rate ◽  
Flow Measurement ◽  
Indirect Method ◽  
Direct Method ◽  
The Other ◽  
Hydropower Plant ◽  
Kaplan Turbine ◽  
A Direct Method

The paper present two different methods for measuring flow in hydropower, one direct and one indirect method. The study consists in comparing the results obtained by measurements made by both methods in a Romanian hydropower plant. The indirect method used is the Winter-Kennedy method. The other method involves direct measurement of the flow rate of water using a specialized outfit in this.

Sign in / Sign up

Export Citation Format

Share Document