Shifting care from hospital to community, a strategy to integrate care in Singapore: process evaluation of implementation fidelity

Abstract Background RESPOND is a telephone-based falls prevention program for older people who present to a hospital emergency department (ED) with a fall. A randomised controlled trial (RCT) found RESPOND to be effective at reducing the rate of falls and fractures, compared with usual care, but not fall injuries or hospitalisations. This process evaluation aimed to determine whether RESPOND was implemented as planned, and identify implementation barriers and facilitators. Methods A mixed-methods evaluation was conducted alongside the RCT. Evaluation participants were the RESPOND intervention group (n = 263) and the clinicians delivering RESPOND (n = 7). Evaluation data were collected from participant recruitment and intervention records, hospital administrative records, audio-recordings of intervention sessions, and participant questionnaires. The Rochester Participatory Decision-Making Scale (RPAD) was used to evaluate person-centredness (score range 0 (worst) - 9 (best)). Process factors were compared with pre-specified criteria to determine implementation fidelity. Six focus groups were held with participants (n = 41), and interviews were conducted with RESPOND clinicians (n = 6). Quantitative data were analysed descriptively and qualitative data thematically. Barriers and facilitators to implementation were mapped to the ‘Capability, Opportunity, Motivation – Behaviour’ (COM-B) behaviour change framework. Results RESPOND was implemented at a lower dose than the planned 10 h over 6 months, with a median (IQR) of 2.9 h (2.1, 4). The majority (76%) of participants received their first intervention session within 1 month of hospital discharge with a median (IQR) of 18 (12, 30) days. Clinicians delivered the program in a person-centred manner with a median (IQR) RPAD score of 7 (6.5, 7.5) and 87% of questionnaire respondents were satisfied with the program. The reports from participants and clinicians suggested that implementation was facilitated by the use of positive and personally relevant health messages. Complex health and social issues were the main barriers to implementation. Conclusions RESPOND was person-centred and reduced falls and fractures at a substantially lower dose, using fewer resources, than anticipated. However, the low dose delivered may account for the lack of effect on falls injuries and hospitalisations. The results from this evaluation provide detailed information to guide future implementation of RESPOND or similar programs. Trial registration This study was registered with the Australian New Zealand Clinical Trials Registry, number ACTRN12614000336684 (27 March 2014).

Download Full-text

Can implementation failure or intervention failure explain the result of the 3D multimorbidity trial in general practice: mixed-methods process evaluation

BMJ Open ◽

10.1136/bmjopen-2019-031438 ◽

2019 ◽

Vol 9 (11) ◽

pp. e031438 ◽

Cited By ~ 3

Author(s):

Cindy Mann ◽

Ali R G Shaw ◽

Bruce Guthrie ◽

Lesley Wye ◽

Mei-See Man ◽

...

Keyword(s):

Mixed Methods ◽

Process Evaluation ◽

Randomised Trial ◽

Implementation Fidelity ◽

Skills Training ◽

Cluster Randomised Trial ◽

Implementation Barriers ◽

Recommended Care ◽

Intervention Failure ◽

Implementation Failure

ObjectivesDuring a cluster randomised trial, (the 3D study) of an intervention enacting recommended care for people with multimorbidity, including continuity of care and comprehensive biennial reviews, we examined implementation fidelity to interpret the trial outcome and inform future implementation decisions.DesignMixed-methods process evaluation using cross-trial data and a sample of practices, clinicians, administrators and patients. Interviews, focus groups and review observations were analysed thematically and integrated with quantitative data about implementation. Analysis was blind to trial outcomes and examined context, intervention adoption, reach and maintenance, and delivery of reviews to patients.SettingThirty-three UK general practices in three areas.ParticipantsThe trial included 1546 people with multimorbidity. 11 general practitioners, 14 nurses, 7 administrators and 38 patients from 9 of 16 intervention practices were sampled for an interview.ResultsStaff loss, practice size and different administrative strategies influenced implementation fidelity. Practices with whole administrative team involvement and good alignment between the intervention and usual care generally implemented better. Fewer reviews than intended were delivered (49% of patients receiving both intended reviews, 30% partially reviewed). In completed reviews >90% of intended components were delivered, but review observations and interviews with patients and clinicians found variation in style of component delivery, from ‘tick-box’ to patient-centred approaches. Implementation barriers included inadequate skills training to implement patient-centred care planning, but patients reported increased patient-centredness due to comprehensive reviews, extra time and being asked about their health concerns.ConclusionsImplementation failure contributed to lack of impact of the 3D intervention on the trial primary outcome (quality of life), but so did intervention failure since modifiable elements of intervention design were partially responsible. When a decisive distinction between implementation failure and intervention failure cannot be made, identifying potentially modifiable reasons for suboptimal implementation is important to enhance potential for impact and effectiveness of a redesigned intervention.Trial registration numberISRCTN06180958

Download Full-text

Factors affecting adoption, implementation fidelity, and sustainability of the Redesigned Community Health Fund in Tanzania: a mixed methods protocol for process evaluation in the Dodoma region

Global Health Action ◽

10.3402/gha.v8.29648 ◽

2015 ◽

Vol 8 (1) ◽

pp. 29648 ◽

Cited By ~ 7

Author(s):

Albino Kalolo ◽

Ralf Radermacher ◽

Manfred Stoermer ◽

Menoris Meshack ◽

Manuela De Allegri

Keyword(s):

Mixed Methods ◽

Community Health ◽

Process Evaluation ◽

Implementation Fidelity ◽

Factors Affecting ◽

Health Fund

Download Full-text

Assessing implementation fidelity of a community-based infant and young child feeding intervention in Ethiopia identifies delivery challenges that limit reach to communities: a mixed-method process evaluation study

BMC Public Health ◽

10.1186/s12889-015-1650-4 ◽

2015 ◽

Vol 15 (1) ◽

Cited By ~ 34

Author(s):

Sunny S Kim ◽

Disha Ali ◽

Andrew Kennedy ◽

Roman Tesfaye ◽

Amare W Tadesse ◽

...

Keyword(s):

Young Child ◽

Process Evaluation ◽

Mixed Method ◽

Implementation Fidelity ◽

Evaluation Study ◽

Community Based ◽

Child Feeding ◽

Young Child Feeding ◽

Infant And Young Child

Download Full-text

Process evaluation methods, implementation fidelity results and relationship to physical activity and healthy eating in the Faith, Activity, and Nutrition (FAN) study

Evaluation and Program Planning ◽

10.1016/j.evalprogplan.2013.11.003 ◽

2014 ◽

Vol 43 ◽

pp. 93-102 ◽

Cited By ~ 15

Author(s):

Ruth P. Saunders ◽

Sara Wilcox ◽

Meghan Baruth ◽

Marsha Dowda

Keyword(s):

Physical Activity ◽

Process Evaluation ◽

Healthy Eating ◽

Implementation Fidelity ◽

Evaluation Methods

Download Full-text

Text Messages Sent to Household Tuberculosis Contacts in Kampala, Uganda: Process Evaluation (Preprint)

10.2196/preprints.10239 ◽

2018 ◽

Author(s):

Amanda J Meyer ◽

Diana Babirye ◽

Mari Armstrong-Hough ◽

David Mark ◽

Irene Ayakaka ◽

...

Keyword(s):

Process Evaluation ◽

Text Messaging ◽

Text Message ◽

Implementation Fidelity ◽

Text Messages ◽

Household Contacts ◽

Contact Investigation ◽

Study Results ◽

Sms Intervention

BACKGROUND Previous studies have reported the inconsistent effectiveness of text messaging (short message service, SMS) for improving health outcomes, but few have examined to what degree the quality, or “fidelity,” of implementation may explain study results. OBJECTIVE The aim of this study was to determine the fidelity of a one-time text messaging (SMS) intervention to promote the uptake of tuberculosis evaluation services among household contacts of index patients with tuberculosis. METHODS From February to June 2017, we nested a process evaluation of text message (SMS) delivery within the intervention arm of a randomized controlled trial of tuberculosis contact investigation in Kampala, Uganda. Because mobile service providers in Uganda do not provide delivery confirmations, we asked household tuberculosis contacts to confirm the receipt of a one-time tuberculosis-related text message (SMS) by sending a text message (SMS) reply through a toll-free “short code.” Two weeks later, a research officer followed up by telephone to confirm the receipt of the one-time text message (SMS) and administer a survey. We considered participants lost to follow-up after 3 unsuccessful call attempts on 3 separate days over a 1-week period. RESULTS Of 206 consecutive household contacts, 119 had a text message (SMS) initiated from the server. While 33% (39/119) were children aged 5-14 years, including 20% (24/119) girls and 13% (15/119) boys, 18 % (21/119) were adolescents or young adults, including 12% (14/119) young women and 6% (7/119) young men. 50% (59/119) were adults, including 26% (31/119) women and 24% (28/119) men. Of 107 (90%) participants for whom we could ascertain text message (SMS) receipt status, 67% (72/107) confirmed text message (SMS) receipt, including 22% (24/107) by reply text message (SMS) and 45% (48/107) during the follow-up telephone survey. No significant clinical or demographic differences were observed between those who did and did not report receiving the text message (SMS). Furthermore, 52% (56/107) reported ever reading the SMS. The cumulative likelihood of a text message (SMS) reaching its target and being read and retained by a participant was 19%. CONCLUSIONS The fidelity of a one-time text message (SMS) intervention to increase the uptake of household tuberculosis contact investigation and linkage to care was extremely low, a fact only discoverable through detailed process evaluation. This study suggests the need for systematic process monitoring and reporting of implementation fidelity in both research studies and programmatic interventions using mobile communications to improve health.

Download Full-text

Process evaluation of implementation fidelity in a Danish health-promoting school intervention

BMC Public Health ◽

10.1186/s12889-018-6289-5 ◽

2018 ◽

Vol 18 (1) ◽

Cited By ~ 2

Author(s):

Ane Høstgaard Bonde ◽

Nanna Wurr Stjernqvist ◽

Marianne S. Sabinsky ◽

Helle Terkildsen Maindal

Keyword(s):

Process Evaluation ◽

Implementation Fidelity ◽

School Intervention ◽

Danish Health ◽

Health Promoting School ◽

Health Promoting

Download Full-text

Process Evaluation of an Intervention to Increase Provision of Adolescent Vaccines at School Health Centers

Health Education & Behavior ◽

10.1177/1090198114531773 ◽

2014 ◽

Vol 41 (6) ◽

pp. 625-632 ◽

Cited By ~ 4

Author(s):

Shelley D. Golden ◽

Kathryn E. Moracco ◽

Ashley L. Feld ◽

Kea L. Turner ◽

Jessica T. DeFrank ◽

...

Keyword(s):

Process Evaluation ◽

School Health ◽

Vaccine Coverage ◽

Intervention Development ◽

Implementation Fidelity ◽

Health Centers ◽

Human Papillomavirus Vaccine ◽

Vaccination Programs ◽

School Health Centers ◽

Papillomavirus Vaccine

Background. Vaccination programs in school health centers (SHCs) may improve adolescent vaccine coverage. We conducted a process evaluation of an intervention to increase SHC-located vaccination to better understand the feasibility and challenges of such interventions. Method. Four SHCs participated in an intervention to increase provision of recommended vaccines to 2,975 adolescents. We reviewed program materials and SHC staff reports, and interviewed parents to assess implementation fidelity and reactions to materials. Results. Ten percent of parents returned forms with consent to at least one vaccine. Of these, 79% checked the box consenting for “all recommended” vaccines, rather than indicating individual vaccines. SHCs sent supplemental mailings to some parents that clarified (a) vaccination costs or (b) human papillomavirus vaccine recommendation for boys and required parents to reconsent. This process resulted in loss of initial consent, primarily due to nonresponse. In interviews, parents who consented to vaccination indicated that intervention materials were clear and persuasive, but needed greater detail about costs and clinic processes. Conclusions. With limited additional investment, it appears feasible for SHCs to achieve a modest increase in the number of vaccinated adolescents. Providing a checkbox to indicate global consent for all recommended vaccinations, and close collaboration among individuals involved in intervention development, may facilitate vaccination efforts.

Download Full-text

Process evaluation of an environmental and educational nutrition intervention in residential drug-treatment facilities

Public Health Nutrition ◽

10.1017/s1368980012000572 ◽

2012 ◽

Vol 15 (7) ◽

pp. 1159-1167 ◽

Cited By ~ 13

Author(s):

Jennifer A Cowan ◽

Carol M Devine

Keyword(s):

Body Composition ◽

Drug Treatment ◽

Process Evaluation ◽

Food Environment ◽

Implementation Fidelity ◽

Analysis Of Covariance ◽

Dietary Behaviour ◽

Treatment Facilities ◽

Residential Drug Treatment ◽

Drug Treatment Facilities

AbstractObjectiveTo evaluate the implementation of a controlled, 6-week, environmental and educational intervention to improve dietary intake and body composition, and to study the association of implementation fidelity with diet and body composition outcomes.DesignA process evaluation documented participation, dose of nutrition education delivered, participant satisfaction, fidelity and completeness of the food environment intervention implementation, and context through observations and interviews with staff and residents. Intervention sites were scored and categorized as high or low participation and implementation and compared on essential elements of the food environment and on diet and body composition outcomes.SettingSix urban residential drug-treatment facilities in Upstate New York.SubjectsFifty-five primarily black and white men in residential drug-treatment programmes.ResultsParticipants were exposed to 94 % and 69 % of the educational and environmental elements, respectively. High implementation sites were significantly more likely to provide water and 100 % juice, offer fruit or vegetable salad, offer choices of fruits and vegetables, and limit fried foods. Mixed-model analysis of covariance revealed that participants in the high participation and implementation sites reported greater reductions in total energy, percentage of energy from sweets, daily servings of fats, oils and sweets, and BMI over the intervention period. Participants in low participation and implementation sites reported greater reductions in percentage of energy from fat. Differential implementation of environmental elements limited the intervention impact.ConclusionsThese findings document the contribution of changes in eating environments to facilitate dietary behaviour change in community residential substance-abuse settings.

Download Full-text