scholarly journals Objective Structured Assessment of Debriefing (OSAD) in simulation-based medical education: Translation and validation of the German version

PLoS ONE ◽  
2020 ◽  
Vol 15 (12) ◽  
pp. e0244816
Author(s):  
Sandra Abegglen ◽  
Andrea Krieg ◽  
Helen Eigenmann ◽  
Robert Greif
Keyword(s):  
Medical Education ◽  
Interrater Reliability ◽  
Evidence Based ◽  
Validity Evidence ◽  
German Version ◽  
Simulation Based ◽  
Expert Input ◽  
Test Retest Reliability ◽  
Composite Reliability

Debriefing is essential for effective learning during simulation-based medical education. To assess the quality of debriefings, reliable and validated tools are necessary. One widely used validated tool is the Objective Structured Assessment of Debriefing (OSAD), which was originally developed in English. The aim of this study was to translate the OSAD into German, and to evaluate the reliability and validity of this German version (G-OSAD) according the ‘Standards of Educational and Psychological Measurement’. In Phase 1, the validity evidence based on content was established by a multistage cross-cultural adaptation translation of the original English OSAD. Additionally, we collected expert input on the adequacy of the content of the G-OSAD to measure debriefing quality. In Phase 2, three trained raters assessed 57 video recorded debriefings to gather validity evidence based on internal structure. Interrater reliability, test-retest reliability, internal consistency, and composite reliability were examined. Finally, we assessed the internal structure by applying confirmatory factorial analysis. The expert input supported the adequacy of the content of the G-OSAD to measure debriefing quality. Interrater reliability (intraclass correlation coefficient) was excellent for the average ratings (three raters: ICC = 0.848; two raters: ICC = 0.790), and good for the single rater (ICC = 0.650). Test-retest reliability was excellent (ICC = 0.976), internal consistency was acceptable (Cronbach’s α = 0.865), and composite reliability was excellent (ω = 0.93). Factor analyses supported the unidimensionality of the G-OSAD, which indicates that these G-OSAD ratings measure debriefing quality as intended. The G-OSAD shows good psychometric qualities to assess debriefing quality, which are comparable to the original OSAD. Thus, this G-OSAD is a tool that has the potential to optimise the quality of debriefings in German-speaking countries.

Validation evidence of the paediatric Objective Structured Assessment of Debriefing (OSAD) tool

2016 ◽  
Vol 2 (3) ◽  
pp. 61-67 ◽  
Author(s):  
Jane Runnacles ◽  
Libby Thomas ◽  
James Korndorffer ◽  
Sonal Arora ◽  
Nick Sevdalis
Keyword(s):  
Assessment Tool ◽  
Learning Experience ◽  
Pearson Correlation ◽  
Phase 1 ◽  
Evidence Based ◽  
Validity Evidence ◽  
Multiple Sources ◽  
Simulation Based ◽  
Expert Input

IntroductionDebriefing is essential to maximise the simulation-based learning experience, but until recently, there was little guidance on an effective paediatric debriefing. A debriefing assessment tool, Objective Structured Assessment of Debriefing (OSAD), has been developed to measure the quality of feedback in paediatric simulation debriefings. This study gathers and evaluates the validity evidence of OSAD with reference to the contemporary hypothesis-driven approach to validity.MethodsExpert input on the paediatric OSAD tool from 10 paediatric simulation facilitators provided validity evidence based on content and feasibility (phase 1). Evidence for internal structure validity was sought by examining reliability of scores from video ratings of 35 postsimulation debriefings; and evidence for validity based on relationship to other variables was sought by comparing results with trainee ratings of the same debriefings (phase 2).ResultsSimulation experts’ scores were significantly positive regarding the content of OSAD and its instructions. OSAD's feasibility was demonstrated with positive comments regarding clarity and application. Inter-rater reliability was demonstrated with intraclass correlations above 0.45 for 6 of the 7 dimensions of OSAD. The internal consistency of OSAD (Cronbach α) was 0.78. Pearson correlation of trainee total score with OSAD total score was 0.82 (p<0.001) demonstrating validity evidence based on relationships to other variables.ConclusionThe paediatric OSAD tool provides a structured approach to debriefing, which is evidence-based, has multiple sources of validity evidence and is relevant to end-users. OSAD may be used to improve the quality of debriefing after paediatric simulations.

scholarly journals TP8.2.26 A new simulation-based course to improve trauma management by junior doctors

2021 ◽  
Vol 108 (Supplement_7) ◽  
Author(s):  
Hannah Schneiders
Keyword(s):  
Medical Education ◽  
Patient Safety ◽  
Small Group ◽  
Unmet Need ◽  
Junior Doctors ◽  
Trauma Management ◽  
Likert Scales ◽  
Sample Group ◽  
Simulation Based

Abstract Aims Trauma calls at small hospitals are often attended by small and/or junior teams, especially overnight. The quality of primary surveys was subjectively observed to be extremely variable, sufficient to cause concern about patient safety. In addition, the introduction of the ‘ward trauma call’ for falls was generating anxiety amongst juniors. I recognised an unmet need and gained permission from the Director of Medical Education to create a half-day ‘Introduction to Trauma’ simulation-based course. Method A sample group of Foundation doctors attended a pilot course. Confidence in aspects of trauma management was assessed using Likert scales. Results Pre-course results indicated doctors were reasonably confident with their A-E assessments (63% moderately or very confident) but lacked confidence in trauma skills such as using a scoop. After the course 100% reported increased confidence about what will happen in a trauma call, and 63% reported increased confidence in their A-E assessments. Increased confidence was also widely reported in trauma skills (e.g. log roll 88%), how to manage a ward trauma call (100%), and where to find further guidance (100%). Conclusions This pilot demonstrated that a small group simulation-based teaching intervention can significantly increase the confidence of foundation doctors in all aspects of their role in trauma management. The most notable increases were in trauma equipment use and in managing ward trauma, suggesting these are areas where foundation doctors lack guidance or experience. The course is now part of trust induction for new foundation doctors.

scholarly journals Development of a Multi-Domain Assessment Tool for Quality Improvement Projects

2017 ◽  
Vol 9 (4) ◽  
pp. 473-478 ◽  
Author(s):  
Glenn Rosenbluth ◽  
Natalie J. Burman ◽  
Sumant R. Ranji ◽  
Christy K. Boscardin
Keyword(s):  
Medical Education ◽  
Quality Improvement ◽  
Assessment Tool ◽  
Medical Profession ◽  
Assessment Tools ◽  
Validity Evidence ◽  
Learner Engagement ◽  
Summative Feedback ◽  
User Friendly

ABSTRACT Background  Improving the quality of health care and education has become a mandate at all levels within the medical profession. While several published quality improvement (QI) assessment tools exist, all have limitations in addressing the range of QI projects undertaken by learners in undergraduate medical education, graduate medical education, and continuing medical education. Objective  We developed and validated a tool to assess QI projects with learner engagement across the educational continuum. Methods  After reviewing existing tools, we interviewed local faculty who taught QI to understand how learners were engaged and what these faculty wanted in an ideal assessment tool. We then developed a list of competencies associated with QI, established items linked to these competencies, revised the items using an iterative process, and collected validity evidence for the tool. Results  The resulting Multi-Domain Assessment of Quality Improvement Projects (MAQIP) rating tool contains 9 items, with criteria that may be completely fulfilled, partially fulfilled, or not fulfilled. Interrater reliability was 0.77. Untrained local faculty were able to use the tool with minimal guidance. Conclusions  The MAQIP is a 9-item, user-friendly tool that can be used to assess QI projects at various stages and to provide formative and summative feedback to learners at all levels.

scholarly journals Development of Chinese Versions of Quality of Life in Late-Stage Dementia and Cognitive Tests for Severe Dementia

2020 ◽  
pp. 172-181
Author(s):  
Suet-Lai Leung ◽  
Hiroyuki Tanaka ◽  
Timothy C.Y. Kwok
Keyword(s):  
Quality Of Life ◽  
Older Adults ◽  
Internal Consistency ◽  
Interrater Reliability ◽  
Severe Dementia ◽  
Good Test ◽  
Retest Reliability ◽  
Chinese Older Adults ◽  
Test Retest Reliability

<b><i>Introduction:</i></b> Valid assessments of quality of life (QoL) and cognition are important in caring for individuals with severe dementia; there is an urgent need for validated assessment tools for specific populations. This study aimed to develop and validate Chinese versions of the Quality of Life in Late-Stage Dementia (QUALID-C) scale and the Cognitive Test for Severe Dementia (CTSD-C) for Chinese older adults. <b><i>Methods:</i></b> This was a cross-sectional validation study comprised of 93 Chinese older adults with severe dementia recruited from 6 residential homes. The content and cultural validity of the QUALID-C and CTSD-C were evaluated by a 7-member expert panel, and interrater reliability, test-retest reliability, internal consistency, concurrent validity, and factorial structure were examined. <b><i>Results:</i></b> The QUALID-C showed acceptable internal consistency (Cronbach α = 0.65), good interrater reliability (intraclass correlation coefficient [ICC] = 0.99), and good test-retest reliability (ICC = 0.96). Principal component analysis yielded 3 factors; the items loaded on the factors were comparable to those in previous studies and suggested the scale’s multidimensionality to measure QoL. The CTSD-C showed satisfactory internal consistency (Cronbach α = 0.862), good interrater reliability (ICC = 0.99), and good test-retest reliability (ICC = 0.958). Principal component analysis yielded 3 factors; the items loaded on factors 1 and 2 resembled the items of the automatic response and attentional control factors of the original study. <b><i>Conclusion:</i></b> The QUALID-C and the CTSD-C are reliable and valid scales to measure the QoL and cognitive functions of Chinese older adults with severe dementia. These assessments can be utilized to evaluate the effectiveness of treatment and future research work.

scholarly journals Translation and validation of the German version of the Young Spine Questionnaire

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Luana Nyirö ◽  
Tobias Potthoff ◽  
Mette Hobaek Siegenthaler ◽  
Fabienne Riner ◽  
Petra Schweinhardt ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Construct Validity ◽  
Point Prevalence ◽  
German Version ◽  
Health Related Quality ◽  
Related Quality ◽  
Health Related ◽  
Pain Frequency ◽  
Test Retest Reliability

Abstract Background Back pain in childhood and adolescence increases the risk for back pain in adulthood, but validated assessment tools are scarce. The aim of this study was to validate the Young Spine Questionnaire (YSQ) in a German version (G-YSQ) in children and adolescents. Methods Children and adolescents between 10 and 16 years (N = 240, 166 females, mean age = 13.05 ± 1.70 years), recruited in chiropractic practices and schools, completed the G-YSQ (translated according to scientific guidelines) and the KIDSCREEN-10 (assessing health-related quality of life) at three time points. Test-retest reliability was determined calculating intraclass correlation coefficients [ICC(3,1)] using start and two week-data. Construct validity was investigated testing a priori hypotheses. To assess responsiveness, the patients additionally filled in the Patient Global Impression of Change (PGIC) after three months and the area under the curve (AUC) of receiver operating curves was calculated. Results The ICC(3,1) was 0.88 for pain intensity and pain frequency, indicating good reliability, 0.68 for week prevalence and 0.60 for point prevalence, indicating moderate reliability. Pain intensity, frequency and prevalence differed between patients and controls (p < 0.001) and, except point prevalence, between older (> 12 years) and younger control participants (p < 0.01). Health-related quality of life of participants with severe pain (in one or several spinal regions) was lower (KIDSCREEN-10, total score: F(4,230) = 7.26, p < 0.001; KIDSCREEN-10, self-rated general health: H(4) = 51.94, p < 0.001) than that of participants without pain or with moderate pain in one spinal region. Thus, altogether these findings indicate construct validity of the G-YSQ. The AUC was 0.69 (95 % CI = 0.57–0.82) and 0.67 (95 % CI = 0.54–0.80) for week and point prevalence, respectively, indicating insufficient responsiveness of the G-YSQ. Conclusions Apart from the question on point prevalence, construct validity and sufficient test-retest reliability was shown for the G-YSQ. However, its responsiveness needs to be improved, possibly by asking for pain frequency during the last week instead of (dichotomous) week prevalence. Trial registration ClinicalTrials.gov, NCT02955342, registered 07/09/2016, https://clinicaltrials.gov/ct2/results?cond=&term=NCT02955342&cntry=CH&state=&city=Zurich&dist=.

scholarly journals Global Health and Graduate Medical Education: A Systematic Review of the Literature

2016 ◽  
Vol 8 (5) ◽  
pp. 685-691 ◽  
Author(s):  
Corey B. Bills ◽  
James Ahn
Keyword(s):  
Systematic Review ◽  
Medical Education ◽  
Global Health ◽  
Graduate Medical Education ◽  
Clinical Benefit ◽  
Reliability And Validity ◽  
Validity Evidence ◽  
Inclusion Criteria ◽  
Graduate Medical

ABSTRACT Background  Global health (GH) interest is increasing in graduate medical education (GME). The popularity of the GH topic has created growth in the GME literature. Objective  The authors aim to provide a systematic review of published approaches to GH in GME. Methods  We searched PubMed using variable keywords to identify articles with abstracts published between January 1975 and January 2015 focusing on GME approaches to GH. Articles meeting inclusion criteria were evaluated for content by authors to ensure relevance. Methodological quality was assessed using the Medical Education Research Study Quality Instrument (MERSQI), which has demonstrated reliability and validity evidence. Results  Overall, 69 articles met initial inclusion criteria. Articles represented research and curricula from a number of specialties and a range of institutions. Many studies reported data from a single institution, lacked randomization and/or evidence of clinical benefit, and had poor reliability and validity evidence. The mean MERSQI score among 42 quantitative articles was 8.87 (2.79). Conclusions  There is significant heterogeneity in GH curricula in GME, with no single strategy for teaching GH to graduate medical learners. The quality of literature is marginal, and the body of work overall does not facilitate assessment of educational or clinical benefit of GH experiences. Improved methods of curriculum evaluation and enhanced publication guidelines would have a positive impact on the quality of research in this area.

Reliability and Validity of a Questionnaire for Assessing Self-Perceived Health-Related Fitness in Spanish Children

2020 ◽  
Vol 23 ◽  
Author(s):  
Carlos Ayán ◽  
Tania Fernández-Villa ◽  
Antía Duro ◽  
Antonio Molina de la Torre
Keyword(s):  
Internal Structure ◽  
Intraclass Correlation ◽  
Perceived Health ◽  
Evidence Based ◽  
Validity Evidence ◽  
Good Test ◽  
Retest Reliability ◽  
Health Related ◽  
Test Retest Reliability ◽  
Health Related Fitness

Abstract There is a need for developing tools aimed at assessing fitness in children, due to its relationship with health. This study is aimed at testing the reliability and the validity of a questionnaire designed for assessing self-perceived health-related fitness in Spanish children. The questionnaire was created based on the model of physical self-concept developed by Fox and Corbin (1989) who conceived four sub-domains: Sport competence, attractive body, strength and physical condition. A total of 283 children (mean age: 10.80 ± 0.69 years; 45.6% girls) answered the questionnaire twice, in order to determine its test-retest reliability. The results obtained in the International Fitness Scale (IFIS) and on a fitness battery were used to determine its validity evidence based on relations to other variables. Exploratory and factorial analyses were performed to check its validity evidence based on internal structure. The obtained results indicated that the questionnaire showed an accurate validity evidence based on internal structure and a very good test-retest reliability, Intraclass correlation coefficient: .88; 95% CI [.84, .90]. The questionnaire established moderate correlations with the IFIS questionnaire (ρ = –.51 to –.68) and the fitness level showed by the children (ρ = –.53). These findings indicate that the questionnaire can be a useful research tool for assessing self-perceived health-related fitness in children.

Sign in / Sign up

Export Citation Format

Share Document