Assessment of a generalizable methodology to assess learning from manikin-based simulation technology*

Objective This study combined a learning outcomes-based checklist and salient characteristics derived from wisdom-of-crowds theory to test whether differing groups of judges (diversity maximized versus expertise maximized) would be able to appropriately assess videotaped, manikin-based simulation scenarios. Methods Two groups of 3 judges scored 9 videos of interns managing a simulated cardiac event. The first group had a diverse range of knowledge of simulation procedures, while the second group was more homogeneous in their knowledge and had greater simulation expertise. All judges viewed 3 types of videos (predebriefing, postdebriefing, and 6 month follow-up) in a blinded fashion and provided their scores independently. Intraclass correlation coefficients (ICCs) were used to assess the reliability of judges as related to group membership. Scores from each group of judges were averaged to determine the impact of group on scores. Results Results revealed strong ICCs for both groups of judges (diverse, 0.89; expert, 0.97), with the diverse group of judges having a much wider 95% confidence interval for the ICC. Analysis of variance of the average checklist scores indicated no significant difference between the 2 groups of judges for any of the types of videotapes assessed (F = 0.72, p = .4094). There was, however, a statistically significant difference between the types of videos (F = 14.39, p = .0004), with higher scores at the postdebrief and 6-month follow-up time periods. Conclusions Results obtained in this study provide optimism for assessment procedures in simulation using learning outcomes-based checklists and a small panel of judges.

Download Full-text

Using Pedometers to Estimate Ambulatory Physical Activity in Vietnam

Journal of Physical Activity and Health ◽

10.1123/jpah.8.1.52 ◽

2011 ◽

Vol 8 (1) ◽

pp. 52-61 ◽

Cited By ~ 2

Author(s):

Au Bich Thuy ◽

Leigh Blizzard ◽

Michael Schmidt ◽

Costan Magnussen ◽

Emily Hansen ◽

...

Keyword(s):

Physical Activity ◽

Concurrent Validity ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Population Based ◽

Work Activities ◽

Intraclass Correlation Coefficients ◽

Significant Difference ◽

Day Of The Week ◽

Population Based Survey

Background:Pedometer measurement of physical activity (PA) has been shown to be reliable and valid in industrialized populations, but its applicability in economically developing Vietnam remains untested. This study assessed the feasibility, stability and validity of pedometer estimates of PA in Vietnam.Methods:250 adults from a population-based survey were randomly selected to wear Yamax pedometers and record activities for 7 consecutive days. Stability and concurrent validity were assessed using intraclass correlation coefficients (ICC) and Spearman correlation coefficients.Results:Overall, 97.6% of participants provided at least 1 day of usable recordings, and 76.2% wore pedometers for all 7 days. Only 5.2% of the sample participants were involved in work activities not measurable by pedometer. The number of steps increased with hours of wear. There was no significant difference between weekday and weekend in number of steps, and at least 3 days of recordings were required (ICC of the 3 days of recordings: men 0.96, women 0.97). Steps per hour were moderately correlated (men r = .42, women r = .26) with record estimates of total PA.Conclusions:It is feasible to use pedometers to estimate PA in Vietnam. The measure should involve at least 3 days of recording irrespective of day of the week.

Download Full-text

Deaminase associated single nucleotide variants in blood and saliva-derived exomes from healthy subjects

10.1101/807073 ◽

2019 ◽

Author(s):

Nathan E. Hall ◽

Jared Mamrot ◽

Christopher M.A. Frampton ◽

Prue Read ◽

Edward J. Steele ◽

...

Keyword(s):

Exome Sequencing ◽

Whole Exome Sequencing ◽

Healthy Subjects ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Healthy Individuals ◽

Intraclass Correlation Coefficients ◽

Whole Exome ◽

The Impact ◽

Group 1

AbstractBackgroundDeaminases play an important role in shaping inherited and somatic variants. Disease related SNVs are associated with deaminase mutagenesis and genome instability. Here, we investigate the reproducibility and variance of whole exome SNV calls in blood and saliva of healthy subjects and analyze variants associated with AID, ADAR, APOBEC3G and APOBEC3B deaminase sequence motifs.MethodsSamples from twenty-four healthy Caucasian volunteers, allocated into two groups, underwent whole exome sequencing. Group 1 (n=12) analysis involved one blood and four saliva replicates. A single saliva sample was sequenced for Group 2 subjects (n=12). Overall, a total of 72 whole exome datasets were analyzed. Biological (Group 1 & 2) and technical (Group 1) variance of SNV calls and deaminase metrics were calculated and analyzed using intraclass correlation coefficients. Candidate somatic SNVs were identified and evaluated.ResultsWe report high blood-saliva concordance in germline SNVs from whole exome sequencing. Concordant SNVs, found in all subject replicates, accounted for 97% of SNVs located within the protein coding sequence of genes. Discordant SNVs have a 30% overlap with variants that fail gnomAD quality filters and are less likely to be found in dbSNP. SNV calls and deaminase-associated metrics were found to be reproducible and robust (intraclass correlation coefficients >0.95). No somatic SNVs were conclusively identified when comparing blood and saliva samples.ConclusionsSaliva and blood both provide high quality sources of DNA for whole exome sequencing, with no difference in ability to resolve SNVs and deaminase-associated metrics. We did not identify somatic SNVs when comparing blood and saliva of healthy individuals, and we conclude that more specialized investigative methods are required to comprehensively assess the impact of deaminase activity on genome stability in healthy individuals.

Download Full-text

Inter- and Intratester Reliability of Lower Extremity Circumference Measurements

Journal of Sport Rehabilitation ◽

10.1123/jsr.7.4.300 ◽

1998 ◽

Vol 7 (4) ◽

pp. 300-306 ◽

Cited By ~ 9

Author(s):

Gary L. Harrelson ◽

Deidre Leaver-Dunn ◽

A. Louise Fincher ◽

James D. Leeper

Keyword(s):

Lower Extremity ◽

High Reliability ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Joint Line ◽

Measuring Device ◽

Intraclass Correlation Coefficients ◽

Tape Measure ◽

Significant Difference ◽

Male High School

The purpose of this study was to examine the inter- and intratester reliability of lower extremity circumference measurements obtained by two testers using the same tape measure and two different tape measures. Twenty-one male high school student-athletes participated in this study. Two testers measured lower extremity circumference at three sites using a standard flexible tape measure and a Lufkin tape measure with a Gulick spring-loaded handle attachment. Measurement sites were medial joint line, 20 cm above medial joint line, and 15 cm below medial joint line. Intraclass correlation coefficients were computed for inter- and intratester comparisons for each measuring device and each measurement site. Results indicated high reliability but a significant difference between the two tape measures. These findings indicate that the reliability of lower extremity circumference measurements is not influenced by tester experience and that the Lufkin tape measure with the Gulick handle attachment is the more accurate of the two tape measures.

Download Full-text

Clinical correlates and implications of the reliability of the frailty index in the Canadian Longitudinal Study on Aging

The Journals of Gerontology Series A ◽

10.1093/gerona/glab161 ◽

2021 ◽

Author(s):

Quoc Dinh Nguyen ◽

Erica M Moodie ◽

Mark R Keezer ◽

Christina Wolfson

Keyword(s):

Odds Ratio ◽

Frailty Index ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Community Dwelling ◽

Deficit Accumulation ◽

Intraclass Correlation Coefficients ◽

Reliability Estimates ◽

Predicted Probability ◽

The Impact

Abstract Background Deficit-accumulation frailty indices (FIs) are widely used to characterize frailty. FIs vary in number and composition of items; the impact of this variation on reliability and clinical applicability is unknown. Methods We simulated 12,000 studies using a set of 70 candidate deficits in 12,080 community-dwelling participants 65 years and older. For each study, we varied the number (5, 10, 15, 25, 35, 45) and composition (random selection) of items defining the FI and calculated descriptive and predictive estimates: frailty score, prevalence, frailty cut-off, mortality odds ratio, predicted probability of mortality for FI=0.28 (prevalence threshold), and FI cut-off predicting 10% mortality over the follow-up. We summarized the estimates’ medians and spreads (0.025-0.975 quantiles) by number of items and calculated intraclass correlation coefficients (ICC). Results Medians of frailty scores were 0.11-0.12 with decreasing spreads from 0.04-0.24 to 0.10-0.14 for 5-item and 45-item FIs. The median cut-offs identifying 15% as frail was 0.19-0.20 and stable; the spreads decreased with more items. However, medians and spreads for the prevalence of frailty (medians: 11% to 3%), mortality odds ratio (medians:1.24 to 2.19), predicted probability of mortality (medians: 8% to 17%), and FI cut-off predicting 10% mortality (medians: 0.38 to 0.20) varied markedly. ICC increased from 0.19 (5-item FIs) to 0.84 (45-item FIs). Conclusions Variability in the number and composition of items of individual FIs strongly influences their reliability. Estimates using FIs may not be sufficiently stable for generalizing results or direct application. We propose avenues to improve the development, reporting, and interpretation of FIs.

Download Full-text

Measuring clinical outcomes in children with pediatric acute-onset neuropsychiatric syndrome: data from a 2–5 year follow-up study

BMC Psychiatry ◽

10.1186/s12888-021-03450-5 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Caroline De Visscher ◽

Eva Hesselmark ◽

Daniel Rautio ◽

Ida Gebel Djupedal ◽

Maria Silverberg ◽

...

Keyword(s):

Psychiatric Symptoms ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Acute Onset ◽

Symptom Presentation ◽

Intraclass Correlation Coefficients ◽

Follow Up Study ◽

Functional Measures ◽

Core Symptoms

Abstract Background It is unclear how to best measure the complex symptom presentation of pediatric acute-onset neuropsychiatric syndrome (PANS). Methods Well-characterized participants of a 2–5 year follow-up study (n = 34; 56% male) underwent clinical evaluations and completed scales assessing global symptom severity, functional impairment and specific psychiatric symptoms. We explored inter-correlations between the measures and used intraclass correlation coefficients to evaluate the agreement between clinician-, parent- and child ratings of the same constructs. Results Ratings on symptom-specific measures varied largely between participants. Agreement between informants was excellent on functional scales, fair-to-moderate on global severity scales and mixed on symptom-specific scales. Clinician-rated global and functional measures had stronger inter-correlations with parent- and child-rated functional measures than with symptom-specific measures. Conclusions General instruments assessing global severity and functioning are well suited for the assessment and follow-up of PANS, but should be complemented by symptom-specific scales representative of core symptoms.

Download Full-text

Reliability of Intratester and Intertester Measurements Derived from an Instrumented Ankle Arthrometer

Journal of Sport Rehabilitation ◽

10.1123/jsr.12.3.208 ◽

2003 ◽

Vol 12 (3) ◽

pp. 208-220 ◽

Cited By ~ 33

Author(s):

Tricia J. Hubbard ◽

John E. Kovaleski ◽

Thomas W. Kaminski

Keyword(s):

Sports Medicine ◽

Research Laboratory ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Medicine Research ◽

Intraclass Correlation Coefficients ◽

Type Design ◽

Ankle Arthrometer ◽

Significant Difference ◽

History Of

Context:Measurement reliability is critical when new sports-medicine devices or techniques are developed.Objective:To determine the reliability of laxity measurements obtained from an instrumented ankle arthrometer.Design:Intratester reliability was examined using a test–retest design, and intertester reliability was assessed using the measurements recorded by 2 different examiners on a separate group of participants.Setting:Sports-medicine research laboratory.Participants:40 participants with no history of ankle injury, equally divided across the 2 studies.Measurements:Laxity measurements included anteroposterior (AP) displacement during loading to 125 N. Inversion–eversion (I–E) rotation was tested during loading to 4000 N-mm. The measures were analyzed using intraclass correlation coefficients (ICCs) and dependentttests.Results:Good to excellent ICCs (.80–.99) for intratester and intertester reliability. A significant difference in measures was observed between testers for both AP displacement and I–E rotation.Conclusions:Laxity measurements from an instrumented ankle arthrometer are reliable across test days and examiners

Download Full-text

The Transaxial Orientation Is Superior to Both the Short Axis and Horizontal Long Axis Orientations for Determining Right Ventricular Volume and Ejection Fraction Using Simpson's Method with Cardiac Magnetic Resonance

ISRN Cardiology ◽

10.1155/2013/268697 ◽

2013 ◽

Vol 2013 ◽

pp. 1-9 ◽

Cited By ~ 2

Author(s):

Michael K. Atalay ◽

Kevin J. Chang ◽

David J. Grand ◽

Shawn Haji-Momenian ◽

Jason T. Machan ◽

...

Keyword(s):

Cardiac Magnetic Resonance ◽

Magnetic Resonance ◽

Ejection Fraction ◽

Right Ventricular ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Ventricular Volume ◽

Short Axis ◽

Intraclass Correlation Coefficients ◽

Significant Difference

We sought to determine which of the three orientations is the most reliable and accurate for quantifying right ventricular (RV) volume and ejection fraction (EF) by cardiac magnetic resonance using Simpson’s method. We studied 20 patients using short axis (SA), transaxial (TA), and horizontal long axis (HLA) orientations. Three readers independently traced RV endocardial contours at end-diastole and end-systole for each orientation. End-diastolic volumes (EDVs), end-systolic volumes (ESVs), and EF were calculated and compared with the 3D piecewise smooth subdivision surface (PSSS) method. The intraclass correlation coefficients among the 3 readers for EDV, ESV, and EF were 0.92, 0.82, and 0.42, respectively, for SA, 0.95, 0.92, and 0.67 for TA, and 0.85, 0.93, and 0.69 for HLA. For mean data there was no significant difference between TA and PSSS for EDV (−2.6%, 95% CI: −8.2 to 3.3%), ESV (−5.9%, −15.2 to 4.5%), and EF (1.7%, −1.5 to 4.9%). HLA was accurate for ESV (−8.9%, −18.5 to 1.8%) and EF (−0.7%, −3.8 to 2.5%) but significantly underestimated EDV (−9.8, −16.6 to −2.4%). SA was accurate for EDV (0.5%, −6.0 to 7.5%) but overestimated ESV (10.5%, 0.1 to 21.9%) and had poor interrater reliability for EF. Conclusions. The TA orientation provides the most reliable and accurate measures of EDV, ESV, and EF.

Download Full-text

Pengaruh ukuran sampel dan intraclass correlation coefficients (ICC) terhadap bias estimasi parameter multilevel latent variable modeling: studi dengan simulasi Monte Carlo

Jurnal Penelitian dan Evaluasi Pendidikan ◽

10.21831/pep.v21i1.12895 ◽

2017 ◽

Vol 21 (1) ◽

pp. 34-50

Author(s):

Muhammad Dwirifqi Kharisma Putra ◽

Jahja Umar ◽

Bahrul Hayat ◽

Agung Priyo Utomo

Keyword(s):

Monte Carlo ◽

Markov Chain ◽

Markov Chain Monte Carlo ◽

Sample Size ◽

Latent Variable ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Latent Variable Modeling ◽

Intraclass Correlation Coefficients ◽

The Impact

Studi ini menggunakan simulasi Monte Carlo dilakukan untuk melihat pengaruh ukuran sampel dan intraclass correlation coefficients (ICC) terhadap bias estimasi parameter multilevel latent variable modeling. Kondisi simulasi diciptakan dengan beberapa faktor yang ditetapkan yaitu lima kondisi ICC (0.05, 0.10, 0.15, 0.20, 0.25), jumlah kelompok (30, 50, 100 dan 150), jumlah observasi dalam kelompok (10, 20 dan 50) dan diestimasi menggunakan lima metode estimasi: ML, MLF, MLR, WLSMV dan BAYES. Jumlah kondisi keseluruhan sebanyak 300 kondisi dimana tiap kondisi direplikasi sebanyak 1000 kali dan dianalisis menggunakan software Mplus 7.4. Kriteria bias yang masih dapat diterima adalah < 10%. Hasil penelitian ini menunjukkan bahwa bias yang terjadi dipengaruhi oleh ukuran sampel dan ICC, penelitian ini juga menujukkan bahwa metode estimasi WLSMV dan BAYES berfungsi lebih baik pada berbagai kondisi dibandingkan dengan metode estimasi berbasis ML.Kata kunci: multilevel latent variable modeling, intraclass correlation coefficients, Metode Markov Chain Monte Carlo THE IMPACT OF SAMPLE SIZE AND INTRACLASS CORRELATION COEFFICIENTS (ICC) ON THE BIAS OF PARAMETER ESTIMATION IN MULTILEVEL LATENT VARIABLE MODELING: A MONTE CARLO STUDYAbstractA monte carlo study was conducted to investigate the effect of sample size and intraclass correlation coefficients (ICC) on the bias of parameter estimates in multilevel latent variable modeling. The design factors included (ICC: 0.05, 0.10, 0.15, 0.20, 0.25), number of groups in between level model (NG: 30, 50, 100 and 150), cluster size (CS: 10, 20 and 50) to be estimated with five different estimator: ML, MLF, MLR, WLSMV and BAYES. Factors were interegated into 300 conditions (4 NG 3 CS 5 ICC 5 Estimator). For each condition, replications with convergence problems were exclude until at least 1.000 replications were generated and analyzed using Mplus 7.4, we also consider absolute percent bias <10% to represent an acceptable level of bias. We find that the degree of bias depends on sample size and ICC. We also show that WLSMV and BAYES estimator performed better than ML-based estimator across varying sample sizes and ICC’s conditions.Keywords: multilevel latent variable modeling, intraclass correlation coefficients, Markov Chain Monte Carlo method

Download Full-text

Glycated hemoglobin measurements at three, 12 and 24 months postpartum after gestational diabetes

Clinical & Investigative Medicine ◽

10.25011/cim.v42i4.33343 ◽

2019 ◽

Vol 42 (4) ◽

pp. E37-E41 ◽

Cited By ~ 1

Author(s):

Kristen Barton ◽

Lois Donovan ◽

Isabelle Giroux ◽

David Miller ◽

Michelle Mottola ◽

...

Keyword(s):

Hemoglobin A1c ◽

Glycated Hemoglobin ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

First Year ◽

Intraclass Correlation Coefficients ◽

Lifestyle Choices ◽

Pearson Correlations ◽

Diet And Lifestyle

Purpose: To determine the associations between glycated hemoglobin (A1C) values at three, 12 and 24 months postpartum taken during the Families Defeating Diabetes trial. Methods: The Families Defeating Diabetes trial was a randomized 12 month lifestyle intervention delivered in the first year postpartum. Women were reviewed at three, 6 12 and 24 months for body habitus, diet and lifestyle choices. Glycated hemoglobin levels were measured at three, 12 and 24 months. Results: There were 170 randomization participants: 89 interventional (INT); and 81 control (CON). Of these 170 participants, 50 INT and 47 CON completed 12-month follow-up and 26 INT and 24 CON completed 24-month follow-up. Study outcomes did not differ between the cohorts. Combined intraclass correlation coefficients for reliability of repeated results showed substantial reliability: 0.74 (95% CI 0.63, 0.83) between three and 12 month A1C; and 0.72 (95% CI 0.51, 0.85) for three and 24 month A1C. Pearson correlations for three month vs 12 month A1C were r=0.745 (p

Download Full-text

Reliability and Validity of Electro-Goniometric Range of Motion Measurements in Patients with Hand and Wrist Limitations

The Open Orthopaedics Journal ◽

10.2174/1874325001610010190 ◽

2016 ◽

Vol 10 (1) ◽

pp. 190-205 ◽

Cited By ~ 13

Author(s):

Siamak Bashardoust Tajali ◽

Joy C. MacDermid ◽

Ruby Grewal ◽

Chris Young

Keyword(s):

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Cross Sectional ◽

Intraclass Correlation Coefficients ◽

Significant Difference ◽

Passive Flexion ◽

Quick Dash ◽

And Function ◽

Limited Motion

Study Design: Cross-sectional reliability and validity study. Purpose: 1. To determine intrarater, interrater and inter instrument reliabilities and validity of two digital electro goniometry to measure active wrist/finger range of motions (ROMs) in patients with limited motion. 2. To determine intrarater and interrater reliabilities of digital goniometry to measure torques of PIP passive flexion of the index finger in patients with limited motion. Methods: The study was designed in a randomized block plan on 44 patients (24 women, 20 men) with limited wrist or hand motions. Two experienced raters measured active wrist ROMs, and active and passive index PIP flexion using two digital goniometers. All measures were repeated by one rater 2-5 days after the initial measurements. The reliability measures were analyzed using Intraclass Correlation Coefficients (ICCs) and the construct validity was determined by correlation coefficients analysis between sub measures of scores; patient rated pain and function (PRWE) and quick Disabilities of the Arm, Shoulder and Hand (quick DASH) scores. Results: The intrarater, interrater and inter instrument reliabilities were high in most ROM measures (range 0.64-0.97) for both types of electro-goniometers. The 95% limit of agreements and Bland and Altman plots did not show progressive changes. There was a significant difference in force application between the raters when performing passive ROM measures for PIP index, but the same rater produced consistent force. Most of the NK and J-Tech ROM measures were moderately correlated with the patient rated pain and function scores (range 0.32-0.63).

Download Full-text