scholarly journals A systematic review and meta-analysis of measurement properties of objective structured clinical examinations used in physical therapy licensure and a structured review of licensure practices in countries with well-developed regulation systems

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0255696
Author(s):  
Pavlos Bobos ◽  
Dimitra V. Pouliopoulou ◽  
Alexandra Harriss ◽  
Jackie Sadi ◽  
Alison Rushton ◽  
...  
Keyword(s):  
Physical Therapy ◽  
Assessment Tool ◽  
Meta Analysis ◽  
Pearson Correlation ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Measurement Properties ◽  
Clinical Competency ◽  
Professional Degree ◽  
Objective Structured Clinical Examinations

Background The Objective Structured Clinical Examination (OSCE) is a commonly used tool internationally to assess clinical competency. Physical therapy (PT) licensure processes vary internationally. The OSCE is the tool used in Canada to assess clinical competency for PT graduates seeking licensure. Previous studies that examined the measurement properties of OSCEs present contradictory results. Objectives The first objective was to investigate the reliability and validity of OSCEs when administered to PTs during their education or as part of a licensure process. The second objective was to conduct a structured review to report PT educational and licensing components and policies in 17 countries with well-developed PT regulation systems. Methods An electronic search was performed in four databases from inception to 31st March 2021 to identify relevant articles. Two reviewers performed the critical appraisal of the included studies using a validated quality assessment tool. We deployed a random effects meta-analysis on reliability and validity estimates of OSCEs and examined sources of heterogeneity with univariate meta-regressions. We searched websites of professional regulatory bodies and associations for data on educational and licencing components and policies. Educational and licensing components across countries were synthesized descriptively. Results A pooled estimate of Cronbach’s alpha of 0.55, (95% CI: 0.41, 0.67) was determined for OSCEs. The pooled estimate of Intraclass Correlation Coefficient (ICC) between assessors was 0.77 (95% CI: 0.70, 0.83). The pooled estimate of Pearson Correlation between multiple OSCE stations’ scores was 0.27 (95% CI: 0.15, 0.39); and between each station score and the total score was 0.71 (95% CI: 0.61, 0.79). The pooled estimates for kappa Coefficients were 0.75 (95% CI: 0.58, 0.86) and 0.84, (95% CI: 0.72, 0.91) for intra-rater and inter-rater reliability of the standardised patient respectively. From the 17 included countries, Canada (excluding Quebec) was the only country that required both a clinical and written competency exam following graduation from an accredited PT program. Two countries (USA, UAE) required a written competency exam. The remaining 14 countries did not require an additional competency examination after completion of degree requirements from an accredited program. Conclusions We found weak evidence that OSCE examinations items are internally consistent when used to assess PTs. Canada (excluding Quebec) is the only country out of 17 implementing a national clinical competency examination for their PT graduates to achieve licensure after completing professional degree requirements.

scholarly journals Finnish Version of the Eating Assessment Tool (F-EAT-10): A Valid and Reliable Patient-reported Outcome Measure for Dysphagia Evaluation

Dysphagia ◽  
2021 ◽  
Author(s):  
Pia Järvenpää ◽  
Jonna Kuuskoski ◽  
Petra Pietarinen ◽  
Mari Markkanen-Leppänen ◽  
Hanna Freiberg ◽  
...  
Keyword(s):  
Rating Scale ◽  
Outcome Measure ◽  
Assessment Tool ◽  
Pearson Correlation ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Patient Reported Outcome ◽  
Specific Patient ◽  
Patient Reported Outcome Measure ◽  
Patient Reported

AbstractOur aim was to validate a Finnish version of the Eating Assessment Tool (F-EAT-10) for clinical use and to test its reliability and validity in a multicenter nationwide study. Normative data were acquired from 180 non-dysphagic participants (median age 57.0 years, 62.2% female). Dysphagia patients (n = 117, median age 69.7 years, 53.0% female) referred to fiberoptic endoscopic evaluation of swallowing (FEES) completed F-EAT-10 before the examination and after 2 weeks. Patients underwent the 100-ml water swallow test (WST) and FEES was evaluated using the following three scales: the Yale Pharyngeal Residue Severity Rating Scale, Penetration-Aspiration Scale, and the Dysphagia Outcome Severity Scale. An operative cohort of 19 patients (median age 75.8 years, 57.9% female) underwent an endoscopic operation on Zenker’s diverticulum, tight cricopharyngeal muscle diagnosed in videofluorography, or both. Patients completed the F-EAT-10 preoperatively and 3 months postoperatively. The cut-off score for controls was < 3 (sensitivity 94.0%, specificity 96.1%) suggesting that ≥ 3 is abnormal. Re-questionnaires for test–retest reliability analysis were available from 92 FEES patients and 123 controls. The intraclass correlation coefficient was excellent for the total F-EAT-10 score (0.93, 95% confidence interval 0.91–0.95). Pearson correlation coefficients were strong (p < 0.001) for each of the questions and the total score. Internal consistency as assessed by Cronbach’s alpha was excellent (0.95). Some correlations between findings in FEES and 100-ml WST with F-EAT-10 were observed. The change in subjective symptoms of operative patients paralleled the change in F-EAT-10. F-EAT-10 is a reliable, valid, and symptom-specific patient-reported outcome measure for assessing dysphagia among Finnish speakers.

scholarly journals The Patient’s Denture Assessment (Thai version) is a valid and reliable tool for evaluating the outcome of treatment with complete denture

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Sahaprom Namano ◽  
Orapin Komin
Keyword(s):  
Treatment Outcome ◽  
Pearson Correlation ◽  
Treatment Success ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Complete Denture ◽  
Care Clinic ◽  
Geriatric Dentistry ◽  
Significant Difference ◽  
Denture Treatment

Abstract Background Complete tooth losses are still being major problems which resulted in lesser quality of life especially for elderly patients. However, there are still lack of questionnaire to evaluate the treatment outcome from the patient’s aspect. The objective of this study is to evaluate the reliability and validity of the Patient’s Denture Assessment-Thai version (PDA-T), then use this questionnaire to assess patient satisfaction with complete denture treatment outcome also investigates the factors involving their satisfaction. Methods The subjects comprised 120 edentulous adult patients (49 men/71 women; average age 70 years-old) from the Prosthodontic and the Geriatric Dentistry and Special Patients Care Clinic at the Faculty of Dentistry, Chulalongkorn University during 2019 March‒2020 March. The patients were divided into two groups: the group experienced (Exper) (n = 54) with wearing complete dentures, and the non-experienced (NonExper) group (n = 66). The patients used the validated PDA-T to self-assess their treatment at different times. The Exper group completed the questionnaire at t0 (during treatment), t0.5 (2‒8-weeks after t0), and t1 (final follow-up). The NonExper group completed the questionnaire only at t1. Results In the Exper group, Cronbach’s α and average inter-item correlation was 0.95 (range 0.76‒0.95) and 0.47 (range 0.57‒0.83), respectively. The intraclass correlation coefficients (n = 18, 95% confidence interval) were 0.98 overall. The paired t-test (p < 0.05) between t0 and t1 indicated a significant difference between t0 and t1 in every PDA-T topic, and the effect size was 1.71. In the NonExper group, the Pearson correlation analysis indicated no significant correlation between the patients' demographics and masticatory function. Conclusion The reliability and validity of the PDA-T indicate it is a valuable tool for evaluating complete denture treatment. Treatment success affected the patients' satisfaction but was not associated with the type of doctors, genders, ages, or educational level.

Reliability and Validity of the Sedentary Behavior Questionnaire (SBQ) for Adults

2010 ◽  
Vol 7 (6) ◽  
pp. 697-705 ◽  
Author(s):  
Dori E. Rosenberg ◽  
Gregory J. Norman ◽  
Nicole Wagner ◽  
Kevin Patrick ◽  
Karen J. Calfas ◽  
...  
Keyword(s):  
Sedentary Behavior ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Population Surveillance ◽  
Sitting Time ◽  
Measurement Properties ◽  
Sedentary Behaviors ◽  
Partial Correlations ◽  
Overweight Adults

Background:Sedentary behavior is related to obesity, but measures of sedentary behaviors are lacking for adults. The purpose of this study was to examine the reliability and validity of the Sedentary Behavior Questionnaire (SBQ) among overweight adults.Methods:Participants were 49 adults for the 2 week test-retest reliability study (67% female, 53% white, mean age = 20) and 401 overweight women (mean age = 41, 61% white) and 441 overweight men (mean age = 44, 81% white) for the validity study. The SBQ consisted of reports of time spent in 9 sedentary behaviors. Outcomes for validity included accelerometer measured inactivity, sitting time (International Physical Activity Questionnaire), and BMI. Intraclass correlation coefficients (ICCs) assessed reliability and partial correlations assessed validity.Results:ICCs were acceptable for all items and the total scale (range = .51–.93). For men, there were significant relationships of SBQ items with IPAQ sitting time and BMI. For women, there were relationships between the SBQ and accelerometer inactivity minutes, IPAQ sitting time, and BMI.Conclusions:The SBQ has acceptable measurement properties for use among overweight adults. Specific measures of sedentary behavior should be included in studies and population surveillance.

scholarly journals Test-Retest Reliability and Validity of the Six Minute Stepper Test to Evaluate Functional Exercise Capacity in Patients with Ankylosing Spondylitis

2021 ◽  
Author(s):  
Selin Bayram ◽  
Deran Oskay ◽  
Nurten Gizem Tore ◽  
Fulden Sari ◽  
Devrim Can Saraç ◽  
...  
Keyword(s):  
Ankylosing Spondylitis ◽  
Exercise Capacity ◽  
Convergent Validity ◽  
Pearson Correlation ◽  
Physiological Responses ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Rank Test ◽  
Retest Reliability ◽  
Test Retest Reliability

ABSTRACT Objectives The 6-minute stepper test (6MST) is a submaximal test that requires little space to assess exercise capacity compared to the 6-minute walk test (6MWT). The study aims to investigate the test-retest reliability and convergent validity of 6MST and to compare physiological responses, dyspnea, fatigue perception with 6MST and 6MWT in patients with ankylosing spondylitis (AS). Methods To test the convergent validity of 6MST, 65 patients performed both 6MWT and 6MST on the first day and correlation between two tests were assessed with Pearson correlation test. In order to investigate the test-retest reliability of the 6MST, thirty-two of the 65 patients performed 6MST one week later and intraclass correlation coefficients (ICC) were calculated. Dyspnea and fatigue perception were analyzed with using Wilcoxon signed-rank test, physiological responses were analyzed using paired sample t-test. Results Excellent test-retest reliability was observed for 6MST (ICC: 0.988). There was a significant correlation between 6MST and 6MWT (r: 0.725, p&lt;0.001). Dyspnea and leg fatigue perception were significantly higher in 6MST (p&lt;0.05). Physiological responses and fatigue perception were similar in both 6MST and 6MWT (p&gt;0.05). Conclusion This study demonstrated that the 6MST is reliable and valid method to evaluate exercise capacity in patients with AS. 6MST can be used to evaluate exercise capacity of patients with AS.

scholarly journals The Patient’s Denture Assessment (Thai version) is a Valid and Reliable Tool for Evaluating Complete Denture Treatment and Success.

2020 ◽  
Author(s):  
Sahaprom Namano ◽  
Orapin Komin
Keyword(s):  
Pearson Correlation ◽  
Treatment Success ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Complete Denture ◽  
Care Clinic ◽  
Geriatric Dentistry ◽  
Significant Difference ◽  
Denture Treatment

Abstract BACKGROUND. Complete tooth losses are still being major problems which resulted in lesser quality of life especially for elderly patients. However, there are still lack of questionnaire to evaluate the treatment outcome from the patient’s aspect. The objective of this study is to evaluate the reliability and validity of the Patient’s Denture Assessment-Thai version (PDA-T), then use this questionnaire to assess patient satisfaction with complete denture treatment. METHODS. The subjects comprised 120 edentulous adult patients (49 men/71 women; average age 70 years-old) from the Prosthodontic and the Geriatric Dentistry and Special Patients Care Clinic at the Faculty of Dentistry, Chulalongkorn University during 2019 March‒2020 March. The patients were divided into two groups: The group experienced (Exper) (n=54) with wearing complete dentures, and the non-experienced (NonExper) group (n=66). The patients used the validated PDA-T to self-assess their treatment at different times. The Exper group completed the questionnaire at t 0 (during treatment), t 0.5 (2‒8-weeks after t 0 ), and t 1 (final follow-up). The NonExper group completed the questionnaire only at t 1 . RESULTS. In the Exper group, Cronbach’s α and average inter-item correlation was 0.95 (range 0.76‒0.95) and 0.47 (range 0.57‒0.83), respectively. The intraclass correlation coefficients (n = 18, 95% confidence interval) were 0.98 overall. The paired t-test (p < 0.05) between t 0 and t 1 indicated a significant difference between t 0 and t 1 in every PDA-T topic, and the effect size was 1.71. In the NonExper group, the Pearson correlation analysis indicated no significant correlation between the patients' demographics and masticatory function. CONCLUSIONS. The reliability and validity of the PDA-T indicate it is a valuable tool for evaluating complete denture treatment. Treatment success affected the patients' satisfaction, but was not associated with the patients’ prior denture experience, sex, or educational level.

scholarly journals E-ASSESS: Creating an EPA Assessment Tool for Structured Simulated Emergency Scenarios

2020 ◽  
Vol 12 (2) ◽  
pp. 153-158 ◽  
Author(s):  
Caroline Andler ◽  
Sneha Daya ◽  
Katie Kowalek ◽  
Christy Boscardin ◽  
Sandrijn M. van Schaik
Keyword(s):  
Clinical Practice ◽  
Direct Observation ◽  
Assessment Tool ◽  
Pearson Correlation ◽  
Intraclass Correlation ◽  
Professional Activity ◽  
Clinical Tool ◽  
Validity Data ◽  
Simulation Based ◽  
Emergency Scenarios

ABSTRACT Background The entrustable professional activity (EPA) assessment framework allows supervisors to assign entrustment levels to physician trainees for specific activities. Limited opportunity for direct observation of trainees hampers entrustment decisions, in particular for infrequently performed activities. Simulation allows for direct observation, so tools to assess performance of EPAs in simulation could potentially provide additional data to complement clinical assessments. Objective We developed and collected validity evidence for a simulation-based tool grounded in the EPA framework. Methods We developed E-ASSESS (EPA Assessment for Structured Simulated Emergency ScenarioS) to assess performance in 2 EPAs among pediatric residents participating in simulation-based team training in 2017–2018. We collected validity data, applying Messick's unitary view. Three raters used E-ASSESS to assign entrustment levels based on performance in simulation. We compared those ratings to entrustment levels assigned by clinical supervisors (different from the study raters) for the same residents on a separate tool designed for clinical practice. We calculated intraclass correlation (ICC) for each tool and Pearson correlation coefficients to compare ratings between tools. Results Twenty-eight residents participated in the study. The ICC between the 3 raters for entrustment ratings on E-ASSESS ranged from 0.65 to 0.77, while ICC among raters of the clinical tool were 0.59 and 0.57. We found no significant correlations between E-ASSESS ratings and clinical practice ratings for either EPA (r = -0.35 and 0.38, P &gt; .05). Conclusions Assessment following an EPA framework in the simulation context may be useful to provide data points to inform entrustment decisions as part of resident assessment.

scholarly journals The reliability and validity of goniometric elbow measurements in adults: A systematic review of the literature

2018 ◽  
Vol 10 (4) ◽  
pp. 274-284 ◽  
Author(s):  
Suzanne F van Rijn ◽  
Elisa L Zwerus ◽  
Koen LM Koenraadt ◽  
Wilco CH Jacobs ◽  
Michel PJ van den Bekerom ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Interrater Reliability ◽  
Literature Search ◽  
Meta Analysis ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Diagnostic Reliability ◽  
Measuring Tool ◽  
Simple Measuring

Background The universal goniometer is a simple measuring tool. With this review we aimed to investigate the reliability and validity of the universal goniometer in measurements of the adults' elbow. Methods Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines were followed and our study protocol was published online at PROSPERO. A literature search was conducted on relevant studies. Methodological quality was assessed using the Quality Appraisal of Diagnostic Reliability (QAREL) scoring system. Results Out of 697 studies yielded from our literature search, 12 were included. Six studies were rated as high quality. The intrarater reliability intraclass correlation coefficient ranged from 0.45 to 0.99, the interrater reliability ranged from intraclass correlation coefficient 0.53–0.97. One study providing instructions on goniometric alignment did not find a difference in expert versus non-expert examiners. Another study in which examiners were not instructed found a higher interrater reliability in expert examiners. One study investigating the validity of the goniometer in elbow measurements found a maximum standard error of the mean of 11.5° for total range of motion. Discussion Overall, the studies showed high intra- and interrater reliability of the universal goniometer. The reliability of the universal goniometer in non-expert examiners can be increased by clear instructions on goniometric alignment.

scholarly journals Reliability and Validity of 2 Self-Report Measures to Assess Sedentary Behavior in Older Adults

2015 ◽  
Vol 12 (5) ◽  
pp. 727-732 ◽  
Author(s):  
Keith P. Gennuso ◽  
Charles E. Matthews ◽  
Lisa H. Colbert
Keyword(s):  
Physical Activity ◽  
Older Adults ◽  
Sedentary Behavior ◽  
Linear Trend ◽  
Pearson Correlation ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Self Report ◽  
Retest Reliability ◽  
Test Retest Reliability

Background:The purpose of this study was to examine the reliability and validity of 2 currently available physical activity surveys for assessing time spent in sedentary behavior (SB) in older adults.Methods:Fifty-eight adults (≥65 years) completed the Yale Physical Activity Survey for Older Adults (YPAS) and Community Health Activities Model Program for Seniors (CHAMPS) before and after a 10-day period during which they wore an ActiGraph accelerometer (ACC). Intraclass correlation coefficients (ICC) examined test-retest reliability. Overall percent agreement and a kappa statistic examined YPAS validity. Lin’s concordance correlation, Pearson correlation, and Bland-Altman analysis examined CHAMPS validity.Results:Both surveys had moderate test-retest reliability (ICC: YPAS = 0.59 (P < .001), CHAMPS = 0.64 (P < .001)) and significantly underestimated SB time. Agreement between YPAS and ACC was low (κ = −0.0003); however, there was a linear increase (P < .01) in ACC-derived SB time across YPAS response categories. There was poor agreement between ACC-derived SB and CHAMPS (Lin’s r = .005; 95% CI, −0.010 to 0.020), and no linear trend across CHAMPS quartiles (P = .53).Conclusions:Neither of the surveys should be used as the sole measure of SB in a study; though the YPAS has the ability to rank individuals, providing it with some merit for use in correlational SB research.

scholarly journals Reliability and Validity of a Brief Questionnaire to Assess Calcium Intake in Female Collegiate Athletes

2004 ◽  
Vol 14 (2) ◽  
pp. 209-221 ◽  
Author(s):  
Kenneth D. Ward ◽  
Kami Mays Hunt ◽  
Melanie Burstyne Berg ◽  
Deborah A. Slawson ◽  
Christopher M. Vukadinovich ◽  
...  
Keyword(s):  
Calcium Intake ◽  
Assessment Tool ◽  
Intraclass Correlation ◽  
Rapid Assessment ◽  
Reliability And Validity ◽  
Collegiate Athletes ◽  
Assessment Method ◽  
Diet Records ◽  
Rapid Assessment Method ◽  
Female Collegiate Athletes

Calcium intake often is inadequate in female collegiate athletes, increasing the risk for training injuries and future osteoporosis. Thus, a brief and accurate assessment tool to quickly measure calcium intake in athletes is needed. We evaluated the reliability and validity, compared to 6 days of diet records (DRs), of the Rapid Assessment Method (RAM), a self-administered calcium checklist (14). Seventy-six female collegiate athletes (mean age = 18.8 yrs, range = 17–21; 97% Caucasian) were recruited from basketball, cross-country, field hockey, soccer, and volleyball teams. Athletes completed a RAM at the start of the training season to assess calcium intake during the past week. Two weeks later, a second RAM was completed to assess reliability, and athletes began 6 days of diet records (DRs) collection. At completion of DRs, athletes completed a final RAM, corresponding to the same time period as DRs, to assess agreement between the 2 instruments. The RAM demonstrated adequate testretest reliability over 2 weeks (n = 56; Intraclass correlation [ICC] = .54, p <.0001) and adequate agreement with DRs (n = 34; ICC = .41, p = .0067). Calcium intake was below recommended levels, and mean estimates did not differ significantly on the RAM (823±387 mg/d) and DRs (822±330 mg/d; p = .988). Adequacy of calcium intake from both DRs and the RAM was classified as “inadequate” (<1000 mg/d) and “adequate” (≥1000 mg/d). Agreement between the RAM and DRs for adequacy classification was fair (ICC = .30, p = .042), with the RAM identifying 84% of athletes judged to have inadequate calcium intake based on DRs. The RAM briefly and accurately estimates calcium intake in female collegiate athletes compared to DRs.

Sign in / Sign up

Export Citation Format

Share Document