Tailoring Research Evidence and Theory (TREAT) into Allied Health Journal Clubs: A Hybrid Effectiveness-Implementation Study

BackgroundAllied Health Professionals (AHPs) commonly use journal clubs (JCs) to support Evidence-Based Practice (EBP). There is however little research regarding implementing JCs in the long term, and their impact on EBP use and skills in AHPs. This study investigated the impact of implementing a structured JC format, called “TREAT” that was specifically tailored for each club, on EBP skills, confidence, use and resultant changes in clinical practice over 16 sessions for AHPs in a public health service. The study also investigated AHP’s attendance, adherence, satisfaction and barriers and enablers to implementing the format.MethodsA mixed methods hybrid-effectiveness implementation design was employed, guided by the Knowledge-to-Action cycle. EBP skills, confidence, use, and attitudes were assessed (Adapted Fresno Test, EBPQ, tailored journal club culture questionnaire) at baseline, and after 10 and 16-monthly JC sessions. Satisfaction and impact on clinical practice were explored using questionnaires at the latter two time-points, with free-form responses identifying enablers and barriers to EBP culture and implementation. Data on attendance and adherence to the TREAT format were also collected. ResultsSix JCs comprising a total of 132 unique participants from seven Allied Health professions were assessed across three time points. EBP skills significantly improved on the Adapted Fresno Test after 10-monthly (6.6 points: 95% CI, 0.43 to 12.7) and 16-monthly sessions (7.8 points, 95% CI, 0.85 to 14.7), and on self-reported total EBPQ ratings of confidence at 10-months (4.9 points: 95% CI, 2.2 to 7.5) and 16-months (5.7 points: 95% CI 2.7 to 8.7). Across sessions, 88 AHPs reported adopting new treatments/resources and 64 AHPs reported updating clinical procedures. Mean attendance was 5.7 sessions (SD= 3.8), and average adherence to TREAT components each session was 86% (95% CI, 83 to 89). Most participants recommended the format and reported desire to continue TREAT JCs. Enablers to the JC included using clinically relevant topics and active participation, while reported barriers included limited time to prepare. ConclusionsTREAT JCs can be implemented and sustained by AHPs for 16 monthly-sessions. Participation improved EBP skills and confidence and led to changes in clinical practice. Contextual enablers and barriers should be considered when implementing locally.

Instruments to assess Evidence-Based Practice among healthcare professionals: a systematic review

The study makes it possible to select the most appropriate instruments to evaluate the use of Evidence-Based Practice (EBP) among health professionals. The objective of this study was to assess the measurement properties, summarize and describe the instruments that evaluate the use of EBP in health professionals, currently available through the update of the systematic review. The study was conducted and reported according to recommendations of the PRISMA checklist. A systematic search was conducted in the databases: PubMed, Embase, CINAHL and ERIC. In addition, three groups of search terms: EBP terms; evaluation; cross-cultural adaptation and measurement proprieties. They included studies that showed assessment tools of EBP in healthcare workers in general publication of full-text scientific articles, which tested the measurement properties and publication of an article in English. Searches included published studies from 2006 until July 2020. Evaluation of the methodological quality of the studies was conducted according to the COSMIN initiative. 92 studies were included. Forty new instruments have been identified to assess EBP. From these, most were developed for nursing professionals and physiotherapists. More than 48% of studies have American and Australian English as their native language. Only 28% of the studies included students in the samples. Reliability was considered appropriate (sufficient) in 76% of the instruments. The COSMIN checklist classified 7 (seven) instruments as being suitable for use in the target audience. However, Fresno Test remains the most appropriate instrument for assessing the use of EBP in healthcare professionals. 40 new instruments that assess EBP have been identified. Most are consistent and reliable for measuring the use of EBP in healthcare professionals. The Fresno Test, in a list of seven reliable and valid instruments for analysis, remains the most used and the one that most assesses the domains of EBP.

Formal evidence-based medicine instruction in Turkish undergraduate medical education: an initial evaluation

Background Global and national undergraduate medical education accreditation organizations recommend the inclusion of Evidence-Based Medicine (EBM) instructions into the medical schools’ curricula. Accordingly, some Turkish medical schools have individually developed and implemented EBM training programs, but there is no data of current programs’ effectiveness and students’ learning achievements due to the lack of a validated Turkish language EBM assessment tool. This study evaluates the effect of a newly introduced formal EBM instruction to the curriculum on students’ knowledge and skills by using the recently published Turkish adaptation of the Fresno Test. Methods The study is an experimental investigation using pre- and post-test evaluations. A five-week EBM course was developed according to Kern’s six-step curriculum development approach. A total of 78 students from the third (n = 30), fourth (n = 19) and fifth (n = 29) year of medical school voluntarily consented and were enrolled into the course. Overall, the Cerrahpaşa Medical Faculty had a total of 555, 461, and 400 students enrolled in the third, fourth, and fifth year, respectively. The program has been evaluated based on students’ learning achievements and survey responses. Results The students’ mean pre-test Fresno Test score improved from 49.9 ± 18.2 to 118.9 ± 26.3 post-training. The Cohen’s effect size was 3.04 (95% CI, 2.6–3.5). The overall students’ satisfaction score was 8.66 ± 1.09 on a 1 to 10 scale. Conclusions The program was effective in improving students’ knowledge and skills on EBM. We propose to offer the program as an elective course during the third year of the medical school curriculum based on all data obtained during the program evaluation.

Educational Collaboration Between Russian-Born US Physicians and Russian Oncology Trainees in Evidence-Based Medicine: The Higher School of Oncology

PURPOSE The 2-year Russian oncology residency focuses on diagnosis and treatment of malignancies but lacks evidence-based medicine (EBM) and patient communication skills (PCS) training. To overcome these educational disparities, the 5-year national program, the Higher School of Oncology (HSO), involving Russian expatriate physicians trained in the United States was established. METHODS A retrospective study was conducted. Highly motivated oncology residents were enrolled in the program through the three-step selection process. US-trained Russian expatriate physicians acted as mentors. EBM skills were taught through weekly online journal clubs and clinical case presentations. PCS training included live seminars and simulations after journal clubs. EBM knowledge was assessed using Fresno test among newly enrolled and postgraduate year (PGY) 2-5 HSO residents. PCS were evaluated via simulation exam including two clinical scenarios (maximum score 100 each) among 17 PGY2 HSO residents and seven non-HSO trainees. RESULTS Overall, 54 residents were enrolled over 5 years (8-13 annually); four were released from the program. The mean age was 24 ± 1 years, and 56% were females. Median scores of Fresno test were significantly higher among PGY 2-4 HSO residents compared with newly enrolled participants: 111 (IQR, 71-128) versus 68 (IQR, 42-84), P = .042; moreover, performance correlated with year of program participation (rs = 0.5; P < .0001). PCS assessment score was significantly higher among HSO residents than non-HSO trainees: 71 (IQR, 58-84) versus 15 (IQR, 10-30) for scenario number 1 ( P < .0001) and 78 (IQR, 71-85) versus 22 (IQR, 4-58) for scenario number 2 ( P = .005), respectively. CONCLUSION The involvement of Western-trained expatriates in remote education improves EBM and PCS among oncology trainees from their home country. This strategy can be useful in overcoming global medical education disparities in other specialties and in countries facing similar challenges.

Factor structure and short version of the modified Fresno test to assess the use of the evidence-based practice in physiotherapists

Background The Modified Fresno Test has been used to evaluate the use of the Evidence-Based Physiotherapy (EBP). So far, none of the versions of the Fresno Test were subjected to analysis of the factorial structure. The objective of the study was to describe the exploratory and confirmatory factor structure of the Modified Fresno Test adapted to the Portuguese-Brazilian and analyze the statistical feasibility for the elaboration of a short version. Methods The questionnaire was applied with a convenience sample of 57 physiotherapists, being 36 professionals (13 of these also professors) and 21 students from the last semester of the physiotherapy course. Exploratory Factor Analysis (EFA) was performed by the method of principal components. Confirmatory Factor Analysis (CFA) was performed by the method of maximum likelihood. The total score of the answers in the test and retest was evaluated, totalling 228 observations. Reliability was assessed by means of internal consistency, using Cronbach’s alpha coefficient. Results Reliability was satisfactory (α 0.81) for all questions of the instrument. The coefficient α calculated for the corrected item-total showed values higher than 0.20 except for item 9. Preliminary tests for Exploratory Factor Analysis showed acceptable values with Kaiser-Meyer-Olkin (KMO = 0.80) and Bartlett’s test of sphericity [chi-square (78) = 1149.615, p < 0.001], indicating that the correlations were sufficient for analysis. The analysis revealed the presence of 3 factors (eigenvalues> 1), which explains 60.9% of the instrument’s total variance. In Confirmatory Factor Analysis, none of the indices came close to an acceptable level (≥ 0.90), however, the second model which tested a three-factor structure provided a better fit to the data. From the results of this study the Modified Fresno Test short version was drawn. Conclusion The analysis showed good factor validity and adequate internal consistency for the use of the instrument consisting of 13 questions and 3 factors. This model proved to be better than the original model. The short version consisting of 9 questions may be an appropriate alternative for use in the population of interest.

A Study on How Using an Interactive Multimedia E-Book Improves Teachers’ Ability to Teach Evidence-Based Medicine, Depending on Their Seniority

Background Teaching evidence-based medicine (EBM) is not an easy task. The role of the electronic book (e-book) is a useful supplement to traditional methods for improving skills. Our aim is to use an e-book or PowerPoint to evaluate instructors’ teaching effects. Methods Our study group was introduced to learning evidence-based medicine (EBM) using an interactive e-book available on the Internet, while the control group used a PowerPoint presentation. We adopted the Modified Fresno test to assess EBM skills before and after their learning. EBM teaching sessions via e-book or PowerPoint were 20–30 minutes long, followed by students’ feedback. We adopted the Mann-Whitney U tests to compare teachers’ evaluation of their EBM skills and the students’ assessment of the teachers’ instruction. Results We found no difference of EBM skills between the two groups prior to their experimental learning. Physicians in the study group ranked higher in “Choose a case to explain which kind of research design is used for the study type of the question and explain your choice” (P = 0.011) and “How are the important results expressed in the articles found” (P = 0.023), which was assessed by the Modified Fresno test. Teaching effect was better in the e-book group than in the control group for the items, “I am satisfied with this lesson,” “The teaching was of high quality,” “This was a good teaching method,” and “It aroused my interest in EBM.” No difference was observed between the two groups in the physicians with more than 10 years’ experience. Conclusions The use of interactive e-books in clinical teaching can enhance teachers’ EBM skills and teaching and is thus clinically useful.

Cross-cultural adaptation of the Fresno Test for Turkish language

Objective National and international medical organizations and boards have recognized the importance of Evidence Based Medicine (EBM) and emphasized that EBM training should be included in medical education programs. Although some Turkish medical schools have developed and implemented EBM training programs, no validated Turkish language assessment tool has been available to compare the effectiveness of these training programs to national or international standards. The aim of this study is to cross-culturally adapt the Fresno Test, which is a validated English language tool utilized worldwide in the assessment of EBM training. Methods This study is a cross-sectional validation study, which was performed in two stages: Cross-cultural adaptation of the Fresno Test into Turkish; and evaluation of the psychometric properties, validity, reliability and responsiveness, of the Turkish version of the Fresno Test. Results The content validity of the test was evaluated by experienced physicians in the field of Evidence-Based Medicine, and the content validity index was 1.00. The Cronbach α coefficient was 0.78 on the post-test results. The intraclass correlation (ICC) coefficient and the kappa analysis were calculated to evaluate inter-rater reliability. The ICC coefficients ranged from 0.66 to 0.97 for pre- and post-test results. The Kappa coefficients were 1.00 for all pre-test and post-test questions except one post-test question which was 0.89. The change score of the Fresno Test was used to evaluate responsiveness. The students' score of the Turkish Fresno Test was 49.9 ±18.2 pre-training and 118.9 ±26.3 post-training with a change of 69 points (95% CI, 63.9–74.2). The Cohen’s effect size was 3.04 (95% CI, 2.6–3.5) indicating a very large change in scores. Conclusions The Turkish adapted Fresno Test used to evaluate students’ success and program effectiveness is a valid and reliable measurement tool. It will be of great benefit for the comparison of the effectiveness of Turkish education programs nationally and cross-culturally.