Development of clinical competence assessment tool for novice physical and occupational therapists—a mixed Delphi study

ABSTRACT This study compares key characteristics and performance of physicians referred to a clinical competence assessment and education program by state medical boards (boards) and hospitals. Physicians referred by boards (400) and by hospitals (102) completed a CPEP clinical competence assessment between July 2002 and June 2010. Key characteristics, self-reported specialty, and average performance rating for each group are reported and compared. Results show that, compared with hospital-referred physicians, board-referred physicians were more likely to be male (75.5% versus 88.3%), older (average age 54.1 versus 50.3 years), and less likely to be currently specialty board certified (80.4% versus 61.8%). On a scale of 1 (best) to 4 (worst), average performance was 2.62 for board referrals and 2.36 for hospital referrals. There were no significant differences between board and hospital referrals in the percentage of physicians who graduated from U.S. and Canadian medical schools. The most common specialties referred differed for boards and hospitals. Conclusion: Characteristics of physicians referred to a clinical competence program by boards and hospitals differ in important respects. The authors consider the potential reasons for these differences and whether boards and hospitals are dealing with different subsets of physicians with different types of performance problems. Further study is warranted.

Download Full-text

Inter-rater Reliability of Preceptors on Clinical Pharmacy Competency Evaluation

Current Drug Therapy ◽

10.2174/1574885515999201209202624 ◽

2020 ◽

Vol 15 ◽

Author(s):

Dixon Thomas ◽

Sherief Khalifa ◽

Jayadevan Sreedharan ◽

Rucha Bond

Keyword(s):

United Arab Emirates ◽

Clinical Pharmacy ◽

Clinical Competence ◽

Assessment Tool ◽

Clinical Skills ◽

Group Discussion ◽

Pharmacy Students ◽

Rater Reliability ◽

Quasi Experimental ◽

Competency Evaluation

Background:: Clinical competence of pharmacy students is better evaluated at their practice sites. compared to the classroom. A clinical pharmacy competency evaluation rubric like that of the American College of Clinical Pharmacy (ACCP)is an effective assessment tool for clinical skills and can be used to show item reliability. The preceptors should be trained on how to use the rubrics as many inherent factors could influence inter-rater reliability. Objective:: To evaluate inter-rater reliability among preceptors on evaluating clinical competence of pharmacy students, before and after a group discussion intervention. Methods:: In this quasi experimental study in a United Arab Emirates teaching hospital, Seven clinical pharmacy preceptors rated clinical pharmacy competencies of ten recent PharmD graduates referring to their portfolios and preceptorship. Clinical pharmacy competencies were adopted from ACCP and mildly modified to be relevant for the local settings. Results:: Inter-rater reliability (Cronbach's Alpha) among preceptors was reasonable being practitioners at a single site for 2-4 years. At domain level, inter-rater reliability ranged from 0.79 - 0.93 before intervention and 0.94 - 0.99 after intervention. No inter-rater reliability was observed in relation to certain competency elements ranging from 0.31 – 0.61 before intervention, but improved to 0.79 – 0.97 after intervention. Intra-class correlation coefficient improved among all individual preceptors being reliable with each other after group discussion though some had no reliability with each other before group discussion. Conclusion:: Group discussion among preceptors at the training site was found to be effective in improving inter-rater reliability on all elements of the clinical pharmacy competency evaluation. Removing a preceptor from analysis did not affect inter-rater reliability after group discussion.

Download Full-text

Developing a consensus-based scoring rubric to enhance practice-based assessment of student nurses' clinical competence: A Delphi study

Nurse Education Today ◽

10.1016/j.nedt.2021.104859 ◽

2021 ◽

Vol 100 ◽

pp. 104859

Author(s):

Ibraheim Almalkawi ◽

Rebecca Jester ◽

Louise Terry

Keyword(s):

Clinical Competence ◽

Delphi Study ◽

Student Nurses ◽

Scoring Rubric

Download Full-text

A Microanalysis of Expert‐student Dialogue Videos: Supporting Preparation and Learning for Clinical Competence Assessment

European Journal Of Dental Education ◽

10.1111/eje.12683 ◽

2021 ◽

Author(s):

Michael G. Botelho ◽

Albert K.M. Chan

Keyword(s):

Clinical Competence ◽

Competence Assessment ◽

Student Dialogue

Download Full-text

How Do We Assess Patient Skills in a Competence-Based Program? Assessment of Patient Competences Using the Spanish Version of the Prolapse and Incontinence Knowledge Questionnaire and Real Practical Cases in Women with Pelvic Floor Disorders

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18052377 ◽

2021 ◽

Vol 18 (5) ◽

pp. 2377

Author(s):

Beatriz Sánchez-Sánchez ◽

Beatriz Arranz-Martín ◽

Beatriz Navarro-Brazález ◽

Fernando Vergara-Pérez ◽

Javier Bailón-Cerezo ◽

...

Keyword(s):

Decision Making ◽

Pelvic Floor ◽

Discriminant Validity ◽

Pelvic Floor Disorders ◽

Assessment Tool ◽

Pelvic Organ ◽

Assessment Tools ◽

Measurement Properties ◽

Competence Assessment ◽

Knowledge Questionnaire

Therapeutic patient education programs must assess the competences that patients achieve. Evaluation in the pedagogical domain ensures that learning has taken place among patients. The Prolapse and Incontinence Knowledge Questionnaire (PIKQ) is a tool for assessing patient knowledge about urinary (UI) and pelvic organ prolapse (POP) conditions. The aim of this study was to translate the Prolapse and Incontinence Knowledge Questionnaire (PIKQ) into Spanish and test its measurement properties, as well as propose real practical cases as a competence assessment tool. The cross-cultural adaptation was conducted by a standardized translation/back-translation method. Measurement properties analysis was performed by assessing the validity, reliability, responsiveness, and interpretability. A total of 275 women were recruited. The discriminant validity showed statistically significant differences in the PIKQ scores between patients and expert groups. Cronbach’s alpha revealed good internal consistency. The test–retest reliability showed excellent correlation with UI and POP scales. Regarding responsiveness, the effect size, and standardized response mean demonstrated excellent values. No floor or ceiling effects were shown. In addition, three “real practical cases” evaluating skills in identifying and analyzing, decision making, and problem-solving were developed and tested. The Spanish PIKQ is a comprehensible, valid, reliable, and responsive tool for the Spanish population. Real practical cases are useful competence assessment tools that are well accepted by women with pelvic floor disorders (PFD), improving their understanding and their decision-making regarding PFD.

Download Full-text

Assessing Housing Accessibility for Older Adults in Japan: A Content Validity Study

Innovation in Aging ◽

10.1093/geroni/igaa057.357 ◽

2020 ◽

Vol 4 (Supplement_1) ◽

pp. 108-108

Author(s):

Rumiko Tsuchiya-Ito ◽

Björn Slaug ◽

Tomonori Sano ◽

Miki Tajima ◽

Sakiko Itoh ◽

...

Keyword(s):

Older Adults ◽

Content Validity ◽

Assessment Tool ◽

Occupational Therapists ◽

Building Design ◽

Validity Study ◽

Housing Enabler ◽

Item List ◽

Existing Housing ◽

Rigorous Procedure

Abstract Scientifically validated tools to assess housing accessibility for older adults in Japan have been lacking. To address this, a rigorous procedure of adapting an existing housing assessment tool—the Housing Enabler, developed in Sweden—for valid use in Japan was conducted. The original tool was translated into the Japanese language, using established translation procedures. In the process, researchers checked the appropriateness of technical terms and adjusted specifications to be in accordance with Japanese standards. An expert panel approach was used to validate the content of the Japanese Housing Enabler. Thirteen certified occupational therapists, architects and care-managers (average experience=14.5 years) participated as experts in the content validity study. They rated each item with regard to relevance for assessing housing accessibility in Japan, on a scale from 1(=Not relevant) to 4(= Highly relevant). They suggested adjustments and additions that they found to be relevant to capture particularities of Japanese housing and building design. After individual ratings, the experts gathered for consensus discussions on suggested revisions of the item list. As a result, the number of items was substantially increased (from 161 to 283). A content validity index (CVI) was calculated for each item (i.e., the proportion of experts rating the relevance as at least 3). Using a recommended threshold of CVI ≥0.78, more than 90% of the items were considered relevant, thus supporting the content validity. However, the large amount of items might jeopardize the feasibility of the instrument. Further studies are needed to evaluate feasibility, criterion-related validity and aspects of reliability.

Download Full-text

Determining when impairment constitutes incapacity for informed consent in schizophrenia research

The British Journal of Psychiatry ◽

10.1192/bjp.bp.106.033324 ◽

2007 ◽

Vol 191 (1) ◽

pp. 38-43 ◽

Cited By ~ 43

Author(s):

Scott Y. H. Kim ◽

Paul S. Appelbaum ◽

Jeffrey Swan ◽

T. Scott Stroup ◽

Joseph P. McEvoy ◽

...

Keyword(s):

Roc Analysis ◽

Operating Characteristic ◽

Assessment Tool ◽

Ethics Committees ◽

Competence Assessment ◽

Evidence Based ◽

Research Ethics Committees ◽

Predictive Values ◽

Risks And Benefits ◽

Capacity Determination

BackgroundAlthough people with schizophrenia display impaired abilities for consent, it is not known how much impairment constitutes incapacity.AimsTo assess a method for determining the categorical capacity status of potential participants in schizophrenia research.MethodExpert-judgement validation of capacity thresholds on the sub-scales of the MacArthur Competence Assessment Tool – Clinical Research (MacCAT–CR) was evaluated using receiver operating characteristic (ROC) analysis in 91 people with severe mental illness and 40 controls.ResultsThe ROC areas under the curve for the understanding, appreciation and reasoning sub-scales of the MacCAT–CR were 0.94 (95% CI 0.88–0.99), 0.85 (95% CI 0.76–0.94) and 0.80 (95% CI 0.70–0.90). These findings yielded negative and positive predictive values of incapacity that can guide the practice of investigators and research ethics committees.ConclusionsBy performing such validation studies for a few categories of research with varying risks and benefits, it might be possible to create evidence-based capacity determination guidelines for most schizophrenia research.

Download Full-text