scholarly journals Development and Pilot Testing of a Novel Tool for Evaluating Practical Skills in Hematopathology Residents in Pakistan

2019 ◽  
Vol 11 (4s) ◽  
pp. 177-180
Author(s):  
Bushra Moiz ◽  
Syeda Kauser Ali ◽  
Anila Rashid ◽  
Muhammad Shariq ◽  
Farheen Karim
Keyword(s):  
Direct Observation ◽  
Interrater Reliability ◽  
Faculty Members ◽  
Formative Assessments ◽  
Learner Satisfaction ◽  
Practical Skills ◽  
Assessment Scores ◽  
Evaluation Exercise ◽  
Laboratory Procedures ◽  
Mean Time

ABSTRACT Background The mini-Clinical Evaluation Exercise (mini-CEX) and direct observation of procedural skills (DOPS) are reliable tools for work-based assessment of medical trainees. Tools of this type do not yet exist for evaluation of practical laboratory skills of pathology residents. Objective We developed and piloted a 9-item instrument for direct observation of laboratory skills (DOLS). Methods We used the DOLS tool with 10 hematopathology residents (PGY-1 to PGY-5) from Aga Khan University. Each resident was evaluated by 3 faculty members in the laboratory during 4 separate encounters using the DOLS instrument. We assessed construct validity, interrater reliability and G coefficient, feasibility of using DOLS, and learner satisfaction. Results A total of 120 encounters were observed with a mean score (±1 SD) of 56.7% (±12.44). Assessment scores moderately correlated with the number of laboratory procedures previously performed by participants (r = 0.658 and 0.641; P = .0001) and with PGY level. Interrater reliability ranged between 0.47 and 0.96. Cohen's d was 1.64. Residents accounted for a large component of estimated variance (73%), suggesting DOLS can differentiate residents' laboratory skills; variance associated with assessors was small (0.01%). Residents reported being satisfied with the tool. Mean time (±1 SD) taken for observing and feedback was 17.89 ± 5.89 minutes. Conclusions The new DOLS instrument could provide reliable scores for observing laboratory skills. Residents were satisfied with the tool, and rating times make the tool feasible for formative assessments.

scholarly journals An evaluation of the use of direct observation of procedural skills in the intercollegiate surgical curriculum programme

2014 ◽  
Vol 96 (9) ◽  
pp. e10-e13
Author(s):  
A Mathew ◽  
JD Beard ◽  
M Bussey
Keyword(s):  
Direct Observation ◽  
Learner Satisfaction ◽  
Assessment Framework ◽  
Procedural Skills ◽  
Collaborative Project ◽  
Constructive Feedback ◽  
Workplace Based Assessment ◽  
Evaluation Exercise ◽  
Positive Results ◽  
Case Based

The Intercollegiate Surgical Curriculum Programme (ISCP) ( www.iscp.ac.uk ) is a UK-wide collaborative project involving all the surgical royal colleges and the specialty associations involved in surgical training. Workplace-based assessment (WBA) refers to the assessment of day-to-day working practice 1 and is an integral part of the ISCP assessment framework. The main aim of WBA is to aid learning through constructive feedback. Several WBAs have been adopted by the ISCP including the mini-clinical evaluation exercise (mini-CEX), direct observation of procedural skills (DOPS), case-based discussion (CBD) and procedure-based assessment (PBA). A systematic review of WBAs concluded that there was no clear evidence to show that DOPS can lead to improvements in performance but identified studies that showed largely positive results in terms of learner satisfaction. 2

Simulation of realistic nephrology case scenarios to facilitate intra-professional team learning

2020 ◽  
Vol 81 (7) ◽  
pp. 1-7
Author(s):  
Kathryn Watson ◽  
Nicholas Gosling ◽  
Christopher Broom ◽  
Huon Snelgrove ◽  
Joyce Popoola
Keyword(s):  
Clinical Practice ◽  
Team Learning ◽  
Outcome Data ◽  
Faculty Members ◽  
Practical Skills ◽  
Multiprofessional Team ◽  
Learning Group ◽  
Shift Pattern ◽  
Full Day

Background Learning in the workplace maximises relevance to clinical practice and facilitates the education of the whole multiprofessional team. Provision of structured teaching is becoming increasingly challenging with shift pattern working and staff shortages. This article describes a simulation course designed to facilitate team learning to improve the care of nephrology patients, and presents outcome data over 2 years. Methods A full-day course, using high fidelity manikins, was designed for nephrology specialty trainees and nurse specialists. Nineteen learners (eleven specialty trainees and eight nurse specialists) and nine multidisciplinary team faculty members attended. Evaluation used pre- and post-course assessments, with a 1-year follow-up questionnaire. Results Following the course, improved knowledge scores, 56% to 72% (P<0.05), and confidence scores, 57% to 71% (P<0.005), were demonstrated. Qualitative analysis found ‘intra-disciplinary interaction’, ‘reflection’ and ‘practical skills’ were the greatest enablers of learning. In the 1-year follow-up questionnaire, specialty trainees reported that the course improved clinical practice and helped preparation for consultant roles. Conclusions This course improved knowledge and confidence in managing nephrology scenarios across the multidisciplinary learning group, and the model could be used in other hospital specialties.

From Orientation to Onboarding: A Survey-Based Departmental Improvement Program for New Radiation Oncology Faculty Physicians

2020 ◽  
Vol 16 (4) ◽  
pp. e395-e404
Author(s):  
John J. Cuaron ◽  
Erin F. Gillespie ◽  
Daniel R. Gomez ◽  
Atif J. Khan ◽  
Borys Mychalczak ◽  
...  
Keyword(s):  
Radiation Oncology ◽  
Intervention Program ◽  
Faculty Members ◽  
New Faculty ◽  
Post Intervention ◽  
Improvement Program ◽  
Assessment Scores ◽  
Faculty Orientation ◽  
Physician Assessment ◽  
Academic Center

PURPOSE: To evaluate physician-reported assessments of an established faculty orientation program for new radiation oncology physicians at a large academic center and to prospectively analyze the effects of an onboarding improvement program based on those assessments. MATERIALS AND METHODS: An anonymous survey was designed and distributed to physicians new to the department who received onboarding orientation between 2013 and 2017. Survey questions addressed the comprehensiveness, effectiveness, and utility of various orientation activities. On the basis of the survey results, an improved onboarding program was designed and implemented for nine new faculty members between May 2018 and November 2018. A post-intervention survey querying topics similar to those in the pre-intervention survey was distributed to the new faculty members. Descriptive statistics were generated to compare the pre-intervention and post-intervention groups. RESULTS: The overall rate of survey completion was 85% (17 of 20). The intervention program markedly improved physician assessment of comprehensiveness and effectiveness of the onboarding process. Physicians strongly and consistently identified mentor shadowing, on-the-job training, and other faculty mentorship activities as the most important components of an effective onboarding experience. CONCLUSION: An enhanced, tailored, person-oriented, formal onboarding improvement program significantly increased physician assessment scores of comprehensiveness and effectiveness of the faculty onboarding process. This model can serve as a framework for increasing physician preparedness, encouraging early physician mentorship, and ensuring a universal standard of quality across large practices.

Validation and piloting of direct observation of practical skills tool to assess intubation in the Chilean context

2013 ◽  
Vol 35 (3) ◽  
pp. 231-236 ◽  
Author(s):  
Alejandro E. Delfino ◽  
Madawa Chandratilake ◽  
Fernando R Altermatt ◽  
Ghislaine Echevarria
Keyword(s):  
Direct Observation ◽  
Practical Skills

Establishing Intervention Fidelity of an Oral Motor Intervention for Preterm Infants

2015 ◽  
Vol 34 (2) ◽  
pp. 72-82 ◽  
Author(s):  
Brenda S. Lessen ◽  
Clare A. Morello ◽  
Lori J. Williams
Keyword(s):  
Registered Nurses ◽  
Premature Infant ◽  
Preterm Infants ◽  
Direct Observation ◽  
Interrater Reliability ◽  
Intervention Fidelity ◽  
Specific Intervention ◽  
Convenience Sample ◽  
Oral Motor

AbstractPurpose: To establish intervention fidelity of the premature infant oral motor intervention (PIOMI)Design and sample: A rating tool was developed and tested for interrater reliability before being used to test the fidelity of intervention behaviors. A purposeful convenience sample of three users was recruited to perform the intervention under direct observation.Main outcome variables: Three types of reliabilities were calculated: (1) interrater reliability to test the rating tool, followed by (2) interuser reliability, and (3) test–retest reliabilities to test the intervention behaviors of several registered nurses (RNs) over repeated performances of the intervention.Results: The rating tool demonstrated a high interrater reliability (97.57 percent), indicating its accuracy for systematically rating the specific intervention behaviors. Subsequently, the rating tool was used to determine interuser and test–retest performances of the PIOMI and resulted in high reliabilities (97.59 percent and 97.58 percent, respectively). This demonstrates that the intervention can be reliably delivered as prescribed, supporting intervention fidelity.

Feedback and performance scores for direct observation of procedural skills

2014 ◽  
Vol 96 (7) ◽  
pp. e5-e8 ◽  
Author(s):  
H Burnand ◽  
T Fysh ◽  
J Wheeler ◽  
W Allum
Keyword(s):  
Direct Observation ◽  
Medical Training ◽  
Formative Assessments ◽  
Annual Review ◽  
Procedural Skills ◽  
Postgraduate Medical Training ◽  
Surgical Trainees ◽  
And Performance

Direct observation of procedural skills (DOPS) is one of the four main workplace-based assessments (WBAs) in postgraduate medical training. Surgical trainees of all grades and specialties use the Intercollegiate Surgical Curriculum Programme (ISCP) to make an online record of each DOPS. The DOPS are used alongside other WBAs as formative assessments to aid the evaluation of trainee progression at the annual review of competence progression (ARCP).

scholarly journals Pengembangan Model Kewirausahaan SMK Melalui Koperasi Sekolah

2013 ◽  
Vol 12 (2) ◽  
Author(s):  
Iin Nurbudiyani
Keyword(s):  
Quantitative Data ◽  
Qualitative Data ◽  
Practical Skills ◽  
Development Method ◽  
Assessment Scores ◽  
Entrepreneurial Attitudes ◽  
Attitude Test ◽  
Analysis Technique ◽  
Future Challenges ◽  
Cooperative Models

Entrepreneurship has important roles to play in a nation. Student should be equipped with academic capabilities and practical skills as well as entrepreneurial values and attitudes for future challenges of life. Therefore, a vocational education is strategic to develop through an entrepreneurship model. Purposes of this study are to find out the conditions of school cooperative, models applied to develop learning process for enhancing entrepreneurial attitudes among students, and whether or not the development of learning on the base of school cooperative can enhance the entrepreneurial attitudes among the students of Vocational Schools. The study is conducted by using a research and development method. Data used are quantitative data, such as the assessment scores of attitude tests. The data are supported by qualitative data, such as comment, critique, and recommendation for improvement from material and education experts, of product development. The data are collected using instruments of attitude test, observation, interview and questionnaires. The collected data are analyzed by using a path analysis technique.

scholarly journals Comparing Short Clinical Evaluation Exercise and Direct Observation of Procedural Skills With the Traditional Evaluation Approach on the Clinical Skills of Forensic Medicine Residents

2020 ◽  
Vol 10 (1) ◽  
pp. 27118
Author(s):  
Fares Najari ◽  
Dorsa Najari
Keyword(s):  
Conventional Method ◽  
Direct Observation ◽  
Clinical Evaluation ◽  
Forensic Medicine ◽  
Clinical Skills ◽  
Evaluation Methods ◽  
Control Group ◽  
Procedural Skills ◽  
Evaluation Exercise ◽  
Experimental Group

Background: The development of clinical teaching necessitates the use of novel and appropriate clinical evaluation methods. In the meantime, the use of new evaluation approaches that enhance learning at the same time has been recommended. This study aimed to compare the effect of the two new evaluation methods of direct observation of procedural skills (DOPS) and short clinical evaluation exercise (Mini-CEX) with the conventional evaluation method on clinical skills of forensic medicine residents.Methods: This is a randomized trial performed among forensic medicine residents. Using Cochran's formula, the minimum sample size was calculated to be 25 individuals per group. All the residents were randomly divided into either the experimental or the control groups. After training and evaluation, the experimental group was tested using the DOPS and Mini-CEX methods three times during the course, and the control group was evaluated using the conventional method. The basis for the evaluation of the procedures (physical examination and autopsy) in both groups was the valid and reliable checklists prepared by the researchers. The results were analyzed using descriptive and inferential statistics (e.g., the Chi-square and independent t-test).Results: There was no significant difference between the two groups in terms of demographic variables such as age, sex, and grade point average (GPA) (P>0.06). The means of total scores were significantly higher in the experimental group (using the Mini-CEX and DOPS methods) than the control group (conventional method) (P<0.0002).Conclusion: The use of new evaluation methods of DOPS and Mini-CEX improved the clinical skills of forensic medicine residents. Therefore, it is recommended that resident training centers use these two methods of assessment, together with other methods, to evaluate clinical procedures and boost forensic residents' learning.

Sign in / Sign up

Export Citation Format

Share Document