scholarly journals Interrater Reliability of AM-PAC “6-Clicks” Basic Mobility and Daily Activity Short Forms

2015 ◽  
Vol 95 (5) ◽  
pp. 758-766 ◽  
Author(s):  
Diane U. Jette ◽  
Mary Stilphen ◽  
Vinoth K. Ranganathan ◽  
Sandra Passek ◽  
Frederick S. Frost ◽  
...  
Keyword(s):  
Daily Activity ◽  
Interrater Reliability ◽  
Physical Therapists ◽  
Short Form ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Occupational Therapists ◽  
Weighted Kappa ◽  
Item Scores ◽  
Mean Differences

BackgroundThe interrater reliability of 2 new inpatient functional short-form measures, Activity Measure for Post-Acute Care (AM-PAC) “6-Clicks” basic mobility and daily activity scores, has yet to be established.ObjectiveThe purpose of this study was to examine the interrater reliability of AM-PAC “6-Clicks” measures.DesignA prospective observational study was conducted.MethodsFour pairs of physical therapists rated basic mobility and 4 pairs of occupational therapists rated daily activity of patients in 1 of 4 hospital services. One therapist in a pair was the primary therapist directing the assessment while the other therapist observed. Each therapist was unaware of the other's AM-PAC “6-Clicks” scores. Reliability was assessed with intraclass correlation coefficients (ICCs), Bland-Altman plots, and weighted kappa.ResultsThe ICCs for the overall reliability of basic mobility and daily activity were .849 (95% confidence interval [CI]=.784, .895) and .783 (95% CI=.696, .847), respectively. The ICCs for the reliability of each pair of raters ranged from .581 (95% CI=.260, .789) to .960 (95% CI=.897, .983) for basic mobility and .316 (95% CI=−.061, .611) to .907 (95% CI=.801, .958) for daily activity. The weighted kappa values for item agreement ranged from .492 (95% CI=.382, .601) to .712 (95% CI=.607, .816) for basic mobility and .251 (95% CI=.057, .445) to .751 (95% CI=.653, .848) for daily activity. Mean differences between raters' scores were near zero.LimitationsRaters were from one health system. Each pair of raters assessed different patients in different services.ConclusionsThe ICCs for AM-PAC “6-Clicks” total scores were very high. Levels of agreement varied across pairs of raters, from large to nearly perfect for physical therapists and from moderate to nearly perfect for occupational therapists. Levels of agreement for individual item scores ranged from small to very large.

Validity and Reliability of 2 Goniometric Mobile Apps: Device, Application, and Examiner Factors

2016 ◽  
Vol 25 (4) ◽  
pp. 371-379 ◽  
Author(s):  
Robert H. Wellmon ◽  
Dawn T. Gulick ◽  
Mark L. Paterson ◽  
Colleen N. Gulick
Keyword(s):  
Outcome Measures ◽  
Concurrent Validity ◽  
Interrater Reliability ◽  
Physical Therapists ◽  
Mobile Apps ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Validity And Reliability ◽  
Measurement Variability ◽  
Number Of Factors

Context:Smartphones are being used in a variety of practice settings to measure joint range of motion (ROM). A number of factors can affect the validity of the measurements generated. However, there are no studies examining smartphone-based goniometer applications focusing on measurement variability and error arising from the electromechanical properties of the device being used.Objective:To examine the concurrent validity and interrater reliability of 2 goniometric mobile applications (Goniometer Records, Goniometer Pro), an inclinometer, and a universal goniometer (UG).Design:Nonexperimental, descriptive validation study.Setting:University laboratory.Participants:3 physical therapists having an average of 25 y of experience.Main Outcome Measures:Three standardized angles (acute, right, obtuse) were constructed to replicate the movement of a hinge joint in the human body. Angular changes were measured and compared across 3 raters who used 3 different devices (UG, inclinometer, and 2 goniometric apps installed on 3 different smartphones: Apple iPhone 5, LG Android, and Samsung SIII Android). Intraclass correlation coefficients (ICCs) and Bland-Altman plots were used to examine interrater reliability and concurrent validity.Results:Interrater reliability for each of the smartphone apps, inclinometer and UG were excellent (ICC = .995–1.000). Concurrent validity was also good (ICC = .998–.999). Based on the Bland-Altman plots, the means of the differences between the devices were low (range = –0.4° to 1.2°).Conclusions:This study identifies the error inherent in measurement that is independent of patient factors and due to the smartphone, the installed apps, and examiner skill. Less than 2° of measurement variability was attributable to those factors alone. The data suggest that 3 smartphones with the 2 installed apps are a viable substitute for using a UG or an inclinometer when measuring angular changes that typically occur when examining ROM and demonstrate the capacity of multiple examiners to accurately use smartphone-based goniometers.

scholarly journals Reliability of the Function in Sitting Test (FIST)

2014 ◽  
Vol 2014 ◽  
pp. 1-6 ◽  
Author(s):  
Sharon L. Gorman ◽  
Monica Rivera ◽  
Lise McCarthy
Keyword(s):  
Interrater Reliability ◽  
Physical Therapists ◽  
Physical Therapist ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Inpatient Rehabilitation ◽  
Neurological Dysfunction ◽  
Retest Reliability ◽  
Reliability Characteristics ◽  
Test Retest Reliability

The function in sitting test (FIST) is a newly developed, performance-based measure examining deficits in seated postural control. The FIST has been shown to be internally consistent and valid in persons with neurological dysfunction but intra- and interrater reliability and test-retest reliability have not been previously described. Seven patients with chronic neurologic dysfunction were tested and videotaped performing the FIST on two consecutive days. Seventeen acute care and inpatient rehabilitation physical therapist raters scored six of the videotaped performance of the FIST on two occasions at least 2 weeks apart. Intraclass correlation coefficients were used to calculate the test-retest and intra- and interrater reliability of the FIST. ICC of 0.97 (95% CI 0.847–0.995) indicated excellent test-retest reliability of the FIST. Intra- and interrater reliability was also excellent with ICCs of 0.99 (95% CI 0.994–0.997) and 0.99 (95% CI 0.988–0.994), respectively. Physical therapists and other rehabilitation professionals can confidently use the FIST in a variety of clinical practice and research settings due to its favorable reliability characteristics. More studies are needed to describe the responsiveness and minimal clinically important level of change in FIST scores to further enhance clinical usefulness of this measure.

scholarly journals Development of an Activity Scale for Individuals With Advanced Parkinson Disease: Reliability and “On-Off” Variability

2000 ◽  
Vol 80 (11) ◽  
pp. 1087-1096 ◽  
Author(s):  
Alice Nieuwboer ◽  
Willy De Weerdt ◽  
René Dom ◽  
Kris Bogaerts ◽  
Godelieve Nuyens
Keyword(s):  
Parkinson Disease ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Weighted Kappa ◽  
Repeated Measurements ◽  
Functional Mobility ◽  
Variable Response ◽  
Phase Measurements ◽  
The Impact

Abstract Background and Purpose. Functional mobility in people with advanced Parkinson disease, some of whom have a variable response to drug treatment, is often difficult to evaluate. The objectives of this study were to investigate the interrater reliability of measurements obtained with a scale designed to measure mobility and to determine the impact of self-rated dyskinesias and fluctuations on the measure. Subjects. Twenty-nine people with Parkinson disease and with disability and considerable disease duration (X̄=11.7 years, SD=4.9, range=6–22) took part in the study. Methods. The subjects' performance on a 10-item scale was videotaped. The videotapes were then scored by 2 independent raters, and the scores were used to determine interrater reliability. The stability of 6 repeated measurements was examined in the home situation, taking into account self-rated fluctuations of motor performance. Results. Weighted Kappa values of agreement (.86–.98) confirmed the reliability between testers. Measurement during the “on” phase (when medication was working optimally) and the “off” phase (when the action of medication was strongly decreased or absent) led to different measurements. Measuring frequently within “on” and “off” phases gave relatively stable measurements for total function, bed transfers, and gait akinesia, the latter during the “off” phase only (intraclass correlation coefficients [ICCs]=.70–.93). However, more modest repeatability applied to transfers from a chair (ICC=.65–.67). Conclusion and Discussion. To ensure valid results in future effect studies, clinical differentiation between “on” and “off” phase measurements is proposed on the basis of patients' own perception of their medication status.

scholarly journals HoNOS–ABI: a reliable outcome measure of neuropsychiatric sequelae to brain injury?

2005 ◽  
Vol 29 (2) ◽  
pp. 53-55 ◽  
Author(s):  
S. Fleminger ◽  
E. Leigh ◽  
P. Eames ◽  
L. Langrell ◽  
R. Nagraj ◽  
...  
Keyword(s):  
Brain Injury ◽  
Interrater Reliability ◽  
Outcome Measure ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Acquired Brain Injury ◽  
Weighted Kappa ◽  
Routine Clinical Practice ◽  
Clinical Implications ◽  
Intraclass Correlation Coefficients

Aims and MethodThe Health of the Nation Outcome Scale for Acquired Brain Injury (HoNOS–ABI) is a relatively new outcome measure designed to assess the neuropsychiatric sequelae of brain damage. This study investigated the interrater reliability of this scale. Fifty patients with traumatic brain injury receiving rehabilitation were each rated twice on the HoNOS–ABI, by two different raters. There were 24 raters in total.ResultsWeighted kappa values ranged from 0.43 to 0.84 and intraclass correlation coefficients from 0.58 to 0.97 for the ten items assessed. This indicated that agreement was moderate to substantial for all items.Clinical ImplicationsThe scales consistently measured the items of interest across different raters. This indicates that HoNOS–ABI is a reliable outcome measure when applied by different raters in routine clinical practice.

Interobserver Reliability Using the Phonetic Level Evaluation With Severely and Profoundly Hearing-Impaired Children

1991 ◽  
Vol 34 (5) ◽  
pp. 989-999 ◽  
Author(s):  
Stephanie Shaw ◽  
Truman E. Coggins
Keyword(s):  
Interrater Reliability ◽  
Interobserver Reliability ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Hearing Impaired ◽  
Intraclass Correlation Coefficients ◽  
Assessment Measure ◽  
Impaired Children ◽  
Speech Assessment ◽  
Hearing Impaired Children

This study examines whether observers reliably categorize selected speech production behaviors in hearing-impaired children. A group of experienced speech-language pathologists was trained to score the elicited imitations of 5 profoundly and 5 severely hearing-impaired subjects using the Phonetic Level Evaluation (Ling, 1976). Interrater reliability was calculated using intraclass correlation coefficients. Overall, the magnitude of the coefficients was found to be considerably below what would be accepted in published behavioral research. Failure to obtain acceptably high levels of reliability suggests that the Phonetic Level Evaluation may not yet be an accurate and objective speech assessment measure for hearing-impaired children.

scholarly journals Development and Relative Validity of a Semiquantitative Food Frequency Questionnaire to Estimate Dietary Intake among a Multi-Ethnic Population in the Malaysian Cohort Project

Nutrients ◽  
2021 ◽  
Vol 13 (4) ◽  
pp. 1163
Author(s):  
Suzana Shahar ◽  
Mohd Razif Shahril ◽  
Noraidatulakma Abdullah ◽  
Boekhtiar Borhanuddin ◽  
Mohd Arman Kamaruddin ◽  
...  
Keyword(s):  
Vitamin C ◽  
Correlation Coefficient ◽  
Food Frequency Questionnaire ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Weighted Kappa ◽  
Dietary Intakes ◽  
Food Frequency ◽  
Relative Validity ◽  
Spearman’S Correlation

Measuring dietary intakes in a multi-ethnic and multicultural setting, such as Malaysia, remains a challenge due to its diversity. This study aims to develop and evaluate the relative validity of an interviewer-administered food frequency questionnaire (FFQ) in assessing the habitual dietary exposure of The Malaysian Cohort (TMC) participants. We developed a nutrient database (with 203 items) based on various food consumption tables, and 803 participants were involved in this study. The output of the FFQ was then validated against three-day 24-h dietary recalls (n = 64). We assessed the relative validity and its agreement using various methods, such as Spearman’s correlation, weighed Kappa, intraclass correlation coefficient (ICC), and Bland–Altman analysis. Spearman’s correlation coefficient ranged from 0.24 (vitamin C) to 0.46 (carbohydrate), and almost all nutrients had correlation coefficients above 0.3, except for vitamin C and sodium. Intraclass correlation coefficients ranged from −0.01 (calcium) to 0.59 (carbohydrates), and weighted Kappa exceeded 0.4 for 50% of nutrients. In short, TMC’s FFQ appears to have good relative validity for the assessment of nutrient intake among its participants, as compared to the three-day 24-h dietary recalls. However, estimates for iron, vitamin A, and vitamin C should be interpreted with caution.

Development and Initial Validation of a Project-Based Rubric to Assess the Systems-Based Practice Competency of Residents in the Clinical Chemistry Rotation of a Pathology Residency

2014 ◽  
Vol 138 (6) ◽  
pp. 809-813
Author(s):  
Carolyn R. Vitek ◽  
Jane C. Dale ◽  
Henry A. Homburger ◽  
Sandra C. Bryant ◽  
Amy K. Saenger ◽  
...  
Keyword(s):  
Critical Thinking ◽  
Interrater Reliability ◽  
Clinical Chemistry ◽  
Core Competencies ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Thinking Skills ◽  
Project Evaluation ◽  
Critical Thinking Skills

Context.— Systems-based practice (SBP) is 1 of 6 core competencies required in all resident training programs accredited by the Accreditation Council for Graduate Medical Education. Reliable methods of assessing resident competency in SBP have not been described in the medical literature. Objective.— To develop and validate an analytic grading rubric to assess pathology residents' analyses of SBP problems in clinical chemistry. Design.— Residents were assigned an SBP project based upon unmet clinical needs in the clinical chemistry laboratories. Using an iterative method, we created an analytic grading rubric based on critical thinking principles. Four faculty raters used the SBP project evaluation rubric to independently grade 11 residents' projects during their clinical chemistry rotations. Interrater reliability and Cronbach α were calculated to determine the reliability and validity of the rubric. Project mean scores and range were also assessed to determine whether the rubric differentiated resident critical thinking skills related to the SBP projects. Results.— Overall project scores ranged from 6.56 to 16.50 out of a possible 20 points. Cronbach α ranged from 0.91 to 0.96, indicating that the 4 rubric categories were internally consistent without significant overlap. Intraclass correlation coefficients ranged from 0.63 to 0.81, indicating moderate to strong interrater reliability. Conclusions.— We report development and statistical analysis of a novel SBP project evaluation rubric. The results indicate the rubric can be used to reliably assess pathology residents' critical thinking skills in SBP.

Development of a Model for the Acquisition and Assessment of Advanced Laparoscopic Suturing Skills Using an Automated Device

2018 ◽  
Vol 25 (3) ◽  
pp. 286-290 ◽  
Author(s):  
Elif Bilgic ◽  
Madoka Takao ◽  
Pepa Kaneva ◽  
Satoshi Endo ◽  
Toshitatsu Takao ◽  
...  
Keyword(s):  
Laparoscopic Surgery ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Instructional Video ◽  
Validity Evidence ◽  
Laparoscopic Suturing ◽  
Intraclass Correlation Coefficients ◽  
Operative Assessment ◽  
Suturing Skills

Background. Needs assessment identified a gap regarding laparoscopic suturing skills targeted in simulation. This study collected validity evidence for an advanced laparoscopic suturing task using an Endo StitchTM device. Methods. Experienced (ES) and novice surgeons (NS) performed continuous suturing after watching an instructional video. Scores were based on time and accuracy, and Global Operative Assessment of Laparoscopic Surgery. Data are shown as medians [25th-75th percentiles] (ES vs NS). Interrater reliability was calculated using intraclass correlation coefficients (confidence interval). Results. Seventeen participants were enrolled. Experienced surgeons had significantly greater task (980 [964-999] vs 666 [391-711], P = .0035) and Global Operative Assessment of Laparoscopic Surgery scores (25 [24-25] vs 14 [12-17], P = .0029). Interrater reliability for time and accuracy were 1.0 and 0.9 (0.74-0.96), respectively. All experienced surgeons agreed that the task was relevant to practice. Conclusion. This study provides validity evidence for the task as a measure of laparoscopic suturing skill using an automated suturing device. It could help trainees acquire the skills they need to better prepare for clinical learning.

scholarly journals A study investigating the validity of an accelerometer in quantification of step count in adult hospital inpatients recovering from critical illness

2019 ◽  
Vol 33 (5) ◽  
pp. 936-942 ◽  
Author(s):  
Jayne Lesley Anderson ◽  
L Samantha Yoward ◽  
Angela J Green
Keyword(s):  
Critical Illness ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Step Count ◽  
Step Counts ◽  
Intraclass Correlation Coefficients ◽  
Count Range ◽  
Mean Differences ◽  
Hospital Inpatients ◽  
Adult Hospital

Objective: To determine the validity of the ActiGraph GT3X accelerometer in step count quantification when compared to observed step count in hospitalised adults recovering from critical illness. Setting: Large National Health Service (NHS) Hospitals Trust. Subjects: In total, 20 hospital ward-based adults (age: mean 62.3, SD 11.5) who had required greater than 48 hours of mechanical ventilation in the intensive care unit. Main measures: Participants walked self-selected distances and speeds as part of a semi-structured movement protocol not exceeding 3 hours. Two ActiGraph GT3X accelerometers were worn, one on the thigh and one on the ankle of the non-dominant leg. Accelerometer-recorded step counts were compared against observed step counts. Results: In total, 31 separate walking episodes were analysed. A mean (SD) of 45.87 (±19.72) steps was calculated for observed step count (range 15–90). Mean differences (95% limits of agreement) of −0.84 steps (−3.88 to 2.2) for the ankle placement and −17.7 steps (−40.63 to 5.25) for the thigh were calculated. Intraclass correlation coefficients (95% confidence intervals) of 0.99 (0.99 to 1.0) and 0.46 (−0.1 to 0.78) were determined for the ankle and thigh, respectively. Placement sites were well tolerated by 95% of participants. Conclusion: An ankle-mounted ActiGraph GT3X accelerometer demonstrates validity in quantification of step count in hospitalised adults recovering from critical illness. A thigh placement was not considered valid.

scholarly journals A Comparative Study of Clinical vs. Digital Exophthalmometry Measurement Methods

2020 ◽  
Vol 2020 ◽  
pp. 1-6
Author(s):  
Tháıs de Sous Pereira ◽  
Cristina Hiromi Kuniyoshi ◽  
Cristiane de Almeida Leite ◽  
Eloisa M. M. S. Gebrim ◽  
Mário L. R. Monteiro ◽  
...  
Keyword(s):  
Ct Scan ◽  
Pearson Correlation ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Measurement Methods ◽  
Gold Standard Method ◽  
Mean Values ◽  
Ct Measurements ◽  
Mean Differences ◽  
No Gold Standard

Background. A number of orbital diseases may be evaluated based on the degree of exophthalmos, but there is still no gold standard method for the measurement of this parameter. In this study we compare two exophthalmometry measurement methods (digital photography and clinical) with regard to reproducibility and the level of correlation and agreement with measurements obtained with Computerized Tomography (CT) measurements. Methods. Seventeen patients with bilateral proptosis and 15 patients with normal orbits diseases were enrolled. Patients underwent orbital CT, Hertel exophthalmometry (HE) and standardized frontal and side facial photographs by a single trained photographer. Exophthalmometry measurements with HE, the digital photographs and axial CT scans were obtained twice by the same examiner and once by another examiner. Pearson correlation coefficient (PCC) was used to assess correlations between methods. Validity between methods was assessed by mean differences, interintraclass correlation coefficients (ICC’s), and Bland–Altman plots. Results. Mean values were significantly higher in the proptosis group (34 orbits) than in the normal group (30 orbits), regardless of the method. Within each group, mean digital exophthalmometry measurements (24.32 ± 5.17 mm and 18.62 ± 3.87 mm) were significantly greater than HE measurements (20.87 ± 2.53 mm and 17.52 ± 2.67 mm) with broader range of standard deviation. Inter-/intraclass correlation coefficients were 0.95/0.93 for clinical, 0.92/0.74 for digital, and 0.91/0.95 for CT measurements. Correlation coefficients between HE and CT scan measurements in both groups of subjects (r = 0.84 and r = 0.91, p<0.05) were greater than those between digital and CT scan measurements (r = 0.61 and r = 0.75, p<0.05). On the Bland–Altman plots, HE showed better agreement to CT measurements compared to the digital photograph method in both groups studied. Conclusions. Although photographic digital exophthalmometry showed strong correlation and agreement with CT scan measurements, it still performs worse than and is not as accurate as clinical Hertel exophthalmometry. This trail is registered with NCT01999790.

Sign in / Sign up

Export Citation Format

Share Document