Inter-rater Reliability of Preceptors on Clinical Pharmacy Competency Evaluation

2020 ◽  
Vol 15 ◽  
Author(s):  
Dixon Thomas ◽  
Sherief Khalifa ◽  
Jayadevan Sreedharan ◽  
Rucha Bond
Keyword(s):  
United Arab Emirates ◽  
Clinical Pharmacy ◽  
Clinical Competence ◽  
Assessment Tool ◽  
Clinical Skills ◽  
Group Discussion ◽  
Pharmacy Students ◽  
Rater Reliability ◽  
Quasi Experimental ◽  
Competency Evaluation

Background:: Clinical competence of pharmacy students is better evaluated at their practice sites. compared to the classroom. A clinical pharmacy competency evaluation rubric like that of the American College of Clinical Pharmacy (ACCP)is an effective assessment tool for clinical skills and can be used to show item reliability. The preceptors should be trained on how to use the rubrics as many inherent factors could influence inter-rater reliability. Objective:: To evaluate inter-rater reliability among preceptors on evaluating clinical competence of pharmacy students, before and after a group discussion intervention. Methods:: In this quasi experimental study in a United Arab Emirates teaching hospital, Seven clinical pharmacy preceptors rated clinical pharmacy competencies of ten recent PharmD graduates referring to their portfolios and preceptorship. Clinical pharmacy competencies were adopted from ACCP and mildly modified to be relevant for the local settings. Results:: Inter-rater reliability (Cronbach's Alpha) among preceptors was reasonable being practitioners at a single site for 2-4 years. At domain level, inter-rater reliability ranged from 0.79 - 0.93 before intervention and 0.94 - 0.99 after intervention. No inter-rater reliability was observed in relation to certain competency elements ranging from 0.31 – 0.61 before intervention, but improved to 0.79 – 0.97 after intervention. Intra-class correlation coefficient improved among all individual preceptors being reliable with each other after group discussion though some had no reliability with each other before group discussion. Conclusion:: Group discussion among preceptors at the training site was found to be effective in improving inter-rater reliability on all elements of the clinical pharmacy competency evaluation. Removing a preceptor from analysis did not affect inter-rater reliability after group discussion.

Jigsaw Approach in Cooperative Learning: Its Effect in Problem Solving Skills for Physics

10.17158/222 ◽  
2012 ◽  
Vol 18 (1) ◽  
Author(s):  
Renan P. Limjuco ◽  
Ma. Teresa M. Gravino
Keyword(s):  
Problem Solving ◽  
Cooperative Learning ◽  
Learning Strategy ◽  
Group Discussion ◽  
Pharmacy Students ◽  
Control Group ◽  
Problem Solving Skills ◽  
School Year ◽  
Quasi Experimental ◽  
Experimental Group

This study was conducted to determine whether or not the jigsaw approach as a cooperative learning strategy can significantly improve the problem solving skills in physics of the First Year Pharmacy students of the University of the Immaculate Conception for School Year 2011-2012. The researchers used the quasi-experimental design and purposively chose one class with forty students to compose the experimental group and another class of the same size to form the control group. The students from the experimental group were supervised through the jigsaw approach for three weeks as they studied Newton’s 2nd Law of Motion. The test scores from the 50-multiple choice-item Physics Questionnaire (α = 0.90) were the data used in the study. Frequency count, percentage, mean, standard deviation, and t-test were the statistical tools employed in the analyses of the data. Findings of the study revealed that the problem solving skills in physics of the Pharmacy students were significantly improved by the jigsaw approach. The researchers also employed a focus group discussion for the triangulation of the method which validated that students who were subjected to jigsaw approach became more proficient, responsible, facilitative, and approachable as learners. The students felt that they needed to study not only for themselves but also for their group mates.

scholarly journals Pharmacy Students’ Attitudes Towards Learning Communication Skills: The Case Of The United Arab Emirates

2015 ◽  
Vol 6 (1) ◽  
pp. 17-22 ◽  
Author(s):  
Tharwat M. El-Sakran ◽  
Sawsan T. M. El-Sakran
Keyword(s):  
Health Care ◽  
Medical Students ◽  
Patient Care ◽  
European Country ◽  
United Arab Emirates ◽  
Communication Skills ◽  
Clinical Pharmacy ◽  
Pharmacy Students ◽  
Students Attitudes ◽  
Skills Learning

Effective communication amongst pharmacists, especially clinical pharmacists, physicians, nurses, patients, patients’ families and relatives and health care personnel is essential. Poor communication does not only lead to frustration and lack of respect among professions but also may compromise patient care if important information is misunderstood, ineffectively conveyed, or left out. This research explores clinical pharmacy students’ and clinical pharmacy graduates’ attitudes towards the relevance and importance of learning communication skills in their pharmacy careers in the United Arab Emirates. For the purposes of this study, Rees, Sheard, and Davies’ (2002) Communication Skills Attitudes Scale (CSAS) is used to measure pharmacy students’ and pharmacy graduates’ attitudes towards communication skills learning. Although this measure was originally designed to measure medical students’ attitudes towards communication skills learning in a European country, the results obtained from our study are not dissimilar to those reported by Rees et al. (2002) in their study. The paper concludes with some practical suggestions for pharmacy experts in charge of pharmacy communication courses. 

scholarly journals Judging clinical competence using structured observation tools: A cautionary tale

2019 ◽  
Vol 47 (6) ◽  
pp. 736-744 ◽  
Author(s):  
Anthony D. Roth ◽  
Pamela Myles-Hooton ◽  
Amanda Branson
Keyword(s):  
Clinical Competence ◽  
Rating Scales ◽  
Assessment Tool ◽  
Cautionary Tale ◽  
Specific Context ◽  
Rater Reliability ◽  
Video Recordings ◽  
Standard Set ◽  
Observation Tools ◽  
Realistic Appraisal

AbstractBackground:One method for appraising the competence with which psychological therapy is delivered is to use a structured assessment tool that rates audio or video recordings of therapist performance against a standard set of criteria.Aims:The present study examines the inter-rater reliability of a well-established instrument (the Cognitive Therapy Scale – Revised) and a newly developed scale for assessing competence in CBT.Method:Six experienced raters working independently and blind to each other’s ratings rated 25 video recordings of therapy being undertaken by CBT therapists in training.Results:Inter-rater reliability was found to be low on both instruments.Conclusions:It is argued that the results represent a realistic appraisal of the accuracy of rating scales, and that the figures often cited for inter-rater reliability are unlikely to be generalizable outside the specific context in which they were achieved. The findings raise concerns about the use of these scales for making summative judgements of clinical competence in both educational and research contexts.

scholarly journals The objective structured clinical examination as an assessment strategy for clinical competence in novice nursing practitioners in Taiwan

BMC Nursing ◽  
2021 ◽  
Vol 20 (1) ◽  
Author(s):  
Sue-Hsien Chen ◽  
Shu-Ching Chen ◽  
Yo-Ping Lai ◽  
Pin-Hsuan Chen ◽  
Kun-Yun Yeh
Keyword(s):  
Interpersonal Communication ◽  
Work Stress ◽  
Clinical Competence ◽  
Professional Competence ◽  
Clinical Skills ◽  
Poor Performance ◽  
Assessment Process ◽  
Clinical Competency ◽  
Professional Assessment ◽  
Quasi Experimental

Abstract Background The conventional written tests and professional assessment have limitation in fair judgement of clinical competence. Because the examiners may not have total objectivity and may lack standardization throughout the assessment process. We sought to design a valid method of competence assessment in medical and nursing specialties. This work was aimed to develop an Objective Structured Clinical Exam (OSCE) to evaluate novice nursing practitioners’ clinical competency, work stress, professional confidence, and career satisfaction. Methods A Quasi-experimental study (pre-post). Fifty-five novice nursing practitioners received the OSCE three-months following their graduation, which consisted of four stations: history taking, physical examination, problem-directed management, interpersonal communication, and the required techniques of related procedures. The examiners had to complete an assessment checklist, and the participants had to complete a pre-post questionnaire (modified from a Nursing Competency Questionnaire, a Stress scale, and Satisfaction with Learning scale). Results Among the novice nursing practitioners, 41 of them (74.5 %) passed the exam with a mean score of 61.38 ± 8.34. There was a significantly higher passing rate among nurses who were working in medical-surgical wards (85.7 %) and the intensive care unit-emergency department (77.8 %) compared to novice nursing practitioners working in other units. All the novice nursing practitioners at Station A had poor performance in assessing patients with a fever. OSCE performance was more associated with educational attainment and work unit, rather than the gender. Finally, the participants showed statistically significant increases in their clinical competency, confidence in their professional competence, satisfaction with the clinical practice, and decreased work stress after the OSCE. Conclusions We found that the OSCE process had a positive educational effect, in providing a meaningful and accurate assessment of the competence of novice nursing practitioners. An appropriate OSCE program is vital for novice nursing practitioners, educators, and administrators. The effective application of OSCEs can help novice nursing practitioners gain confidence in their clinical skills.

scholarly journals Comparison of Group Discussion and Teach Back Selfcare Education Effects on Knowledge, Attitude, and Performance of Hemodialysis Patients

2020 ◽  
Vol 12 (3) ◽  
Author(s):  
Seyed Reza Borzou ◽  
Mehrdad Rasoli ◽  
Zahra Khalili ◽  
Leili Tapak
Keyword(s):  
Assessment Tool ◽  
Group Discussion ◽  
Self Care ◽  
Hemodialysis Patients ◽  
Training Data ◽  
Teaching Materials ◽  
Quasi Experimental ◽  
Care Training ◽  
And Performance ◽  
Education Effects

Background: Hemodialysis patients require self-care training to manage their problems. To understand the effectiveness of different educational methods, it is necessary to evaluate these methods using knowledge, attitude, and practice assessment tool. Objectives: The current study aimed to compare the effects of group discussion and teach-back self-care education on the knowledge, attitude, and performance of hemodialysis patients. Methods: This quasi-experimental study is conducted on 67 patients who were undergoing hemodialysis via fistula for at least 6 months. Using the convenient sampling method, the patients were randomly allocated into two groups: teach-back training (n = 34), and group discussion (n = 33). Both groups were provided with three sessions of self-care training on nutrition, activity, and fistula care by the researcher. Considering the teaching materials, patients in the teach-back group were taught individually to ensure complete comprehension of the information. On the other hand, subjects in the group discussion were exposed to the teaching materials in the form of group training. Data were collected before and one month after providing the training sessions. Results: In this study, the majority of patients in both groups were married men with an average age of 52 years. Following education, knowledge, attitude, and performance increased in both groups. However, the teach-back method (113.88 ± 4.13) had a higher impact on the attitude of patients than group discussion (110.48 ± 5.68) (P = 0.009). Conclusions: Teach-back education increased the knowledge, performance, and attitude of patients. It is, therefore, recommended to use this method for patients with negative attitudes toward treatment and disease.

scholarly journals Dampak Pembelajaran dan Efek Katalitik OSCE pada Mahasiswa Tahun I,II, dan III Fakultas Kedokteran UGM

2014 ◽  
Vol 3 (1) ◽  
pp. 38
Author(s):  
Triyani Triyani ◽  
Gandes Retno Rahayu ◽  
Efrayim Suryadi
Keyword(s):  
Learning Strategy ◽  
Assessment Tool ◽  
Clinical Skills ◽  
Group Discussion ◽  
Sampling Technique ◽  
Cross Sectional ◽  
First Year Students ◽  
Constructive Feedback ◽  
The Impact ◽  
Catalytic Effects

Background: OSCE is one of the assessments used in the Faculty of Medicine of Universitas Gadjah Mada, Yogyakarta since 1993. The uses of the OSCE as assessment tool have been widely proven. However, it also had limitations such as the presence of anxiety of the students and the high complexity of resources. This study aimed at finding out the impact of the OSCE on learning and its catalytic effects on the first, second and third year students of the Faculty of Medicine of Universitas Gadjah Mada.Method: In a qualitative method, data was collected using focus group discussion. There were 25 participants assigned to 3 groups, which were the group of the first year students, that of the second year students and that of the third year students. Samples were drawn using purposive sampling technique. The data was collected once the students have followed the OSCE. Analysis was made using Atlasti verion 6 and intercoding was conducted along with medical and health education experts.Results: The impacts of the OSCE were the learning strategy of the students more focused on clinical skills and their learning motivation increase. The OSCE caused anxiety of the students and they needed constructive feedback. The catalytic effects of the OSCE in the study were: It made the students more aware of the necessity to more diligently learn, to actively look for learning sources, to more creatively learn, to learn in a more integrated and comprehensive manner.Conclusion: The impact of the OSCE on learning was influenced by its organization and students’ characteristics. The positive catalytic effect of the OSCE on the students was that they needed standard OSCE. The study must be continued using other methods for example survey, cross-sectional or mixed methods.

Structured Long Interview and Clinical Examination (SLICE) as Formative Assessment Tool in Clinical Clerkship: Medical Students Perspective

2021 ◽  
pp. 1-16
Author(s):  
Wajiha Shadab ◽  
Amna Ahmed Noor ◽  
Saira Waqqar ◽  
Gul Muhammad Shaikh
Keyword(s):  
Medical Students ◽  
Formative Assessment ◽  
Thematic Analysis ◽  
Assessment Tool ◽  
Clinical Skills ◽  
General Medicine ◽  
Sampling Technique ◽  
Clinical Clerkship ◽  
Primary Investigator ◽  
Skills Learning

Abstract Objective: This study aimed to assess the medical students’ opinions and views on undertaking SLICE as a formative assessment. Methods: This was a qualitative, exploratory study. Purposive sampling technique was used to select final year medical students who have undertaken a formative assessment through SLICE in their clerkship rotation. Total 32 students participated in this study .Four sets of focus group discussions (FGD) were conducted from medical students who had recently gone through their clinical clerkship modules for Pediatrics, General Medicine, General Surgery and Gynecology& Obstetrics. Each recorded FGD was transcribed verbatim. Thematic analysis was conducted manually. Themes were identified from the transcribed data, coded and analyzed. In order to achieve adequate coding and researcher reliability, investigator triangulation was performed. The initial thematic analysis was performed by the primary investigator. Thereafter, two more investigators independently analyzed the data. Before the data was finalized, all the three investigators reached a final consensus upon the themes that had emerged, ensuring triangulation of the analyzed data. Results: A four staged thematic analysis was conducted, in which five major themes and five sub-themes emerged. The main themes being: Purpose, Learning, Timing, Relevancy and Fairness of SLICE. Conclusion: The students generally thought that SLICE was effective in enhancing their clinical skills learning and should be conducted more frequently with minor adjustments. Continuous...

Use of simulation based technology in pre-clinical years improves confidence and satisfaction among medical students

2021 ◽  
pp. 1-19
Author(s):  
Muhammad Bilal Mirza ◽  
Anjiya Sulaiman ◽  
Satwat Hashmi ◽  
Samar Zaki ◽  
Rehana Rehman ◽  
...  
Keyword(s):  
Medical Students ◽  
Medical Curriculum ◽  
Teaching Method ◽  
Clinical Skills ◽  
Descriptive Analysis ◽  
Group Discussion ◽  
Cross Sectional Study ◽  
Learning Objectives ◽  
Student Interest ◽  
Cross Sectional

Abstract Objective: To determine perception of medical students about learning from integrated simulated clinical skills sessions in Respiration and Circulation module of Year 1 undergraduate medical curriculum at Aga Khan University. Subjects & Methods: This cross-sectional study was conducted at the Centre for Innovation in Medical Education, Aga Khan University (AKU) from July 2018 to February 2019. Integrated clinical skills session involves use of a combination of live simulated patient and mechanical simulator to teach clinical skills.  These sessions were conducted in Respiration and Circulation Module of MBBS Year I curriculum after which quantitative data was collected by a questionnaire that assessed usefulness of integrated clinical skills teaching method and confidence gained by students. The responses acquired were assessed on a Likert scale ranging from 1-5, (strongly agree -strongly disagree).  Descriptive analysis was performed. A focused group discussion (FGD) with students and an in-depth interview with a facilitator conducting the clinical skills sessions was arranged. Simple verbatim transcription of FGD recordings and thematic analyses was executed. Results: Quantitative analysis showed that more than 90% of the students believed integrated sessions were effective in acquiring the learning objectives and found them enjoyable and motivating. 80% expressed confidence that they had accomplished learning objectives and felt they have learned practical clinical skills. Qualitative analysis revealed that these sessions enhanced understanding of subject matter and student engagement during sessions. Conclusion: Integrated clinical skills sessions improved student interest, engagement and promoted student confidence. It should be implemented in undergraduate medical teaching curriculum. Continuous...

Undergraduate performance assessment

2017 ◽  
Vol 5 (9) ◽  
pp. 1-7
Author(s):  
Umayya Musharrafieh ◽  
Khalil Ashkar ◽  
Dima Dandashi ◽  
Maya Romani ◽  
Rana Houry ◽  
...  
Keyword(s):  
Medical Students ◽  
Family Medicine ◽  
Assessment Tool ◽  
Clinical Skills ◽  
Faculty Evaluation ◽  
Multiple Choice Questions ◽  
Final Exam ◽  
Monthly Basis ◽  
Medicine Clerkship ◽  
Clinical Evaluations

Introduction: Objective Structured Clinical Examination (OSCE) is considered a useful method of assessing clinical skills besides Multiple Choice Questions (MCQs) and clinical evaluations. Aim: To explore the acceptance of medical students to this assessment tool in medical education and to determine whether the assessment results of MCQs and faculty clinical evaluations agree with the respective OSCE scores of 4th year medical students (Med IV). Methods: performance of a total of 223 Med IV students distributed on academic years 2006-2007, 2007-2008, and 2008-2009 in OSCE, MCQs and faculty evaluations were compared. Out of the total 93 students were asked randomly to fill a questionnaire about their attitudes and acceptance of this tool. The OSCE was conducted every two months for two different groups of medical students who had completed their family medicine rotation, while faculty evaluation based on observation by assessors was submitted on a monthly basis upon the completion of the rotation. The final exam for the family medicine clerkship was performed at the end of the 4thacademic year, and it consisted of MCQsResults: Students highly commended the OSCE as a tool of evaluation by faculty members as it provides a true measure of required clinical skills and communication skills compared to MCQs and faculty evaluation. The study showed a significant positive correlation between the OSCE scores and the clinical evaluation scores while there was no association between the OSCE score and the final exam scores.Conclusion: Student showed high appreciation and acceptance of this type of clinical skills testing. Despite the fact that OSCEs make them more stressed than other modalities of assessment, it remained the preferred one.

Sign in / Sign up

Export Citation Format

Share Document