scholarly journals Assessing fidelity of a community based psychosocial intervention for people with mild dementia within a large randomised controlled trial

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Kirsty Sprange ◽  
Jules Beresford-Dent ◽  
Gail Mountain ◽  
Claire Craig ◽  
Clare Mason ◽  
...  
Keyword(s):  
Data Collection ◽  
Complex Intervention ◽  
Psychosocial Intervention ◽  
Controlled Trial ◽  
Complex Interventions ◽  
Self Report ◽  
Community Based ◽  
Intervention Delivery ◽  
Methodological Challenges ◽  
Fidelity Assessment

Abstract Background Understanding intervention delivery as intended, particularly in complex interventions, should be underpinned by good quality fidelity assessment. We present the findings from a fidelity assessment embedded as part of a trial of a complex community-based psychosocial intervention, Journeying through Dementia (JtD). The intervention was designed to equip individuals with the knowledge and skills to successfully self-manage, maintain independence, and live well with dementia and involves both group and individual sessions. The methodological challenges of developing a conceptual framework for fidelity assessment and creating and applying purposely designed measures derived from this framework are discussed to inform future studies. Methods A conceptual fidelity framework was created out of core components of the intervention (including the intervention manual and training for delivery), associated trial protocols and pre-defined fidelity standards and criteria against which intervention delivery and receipt could be measured. Fidelity data collection tools were designed and piloted for reliability and usability. Data collection in four selected sites (fidelity sites) was via non-participatory observations of the group aspect of the intervention, attendance registers and interventionist (facilitator and supervisor) self-report. Results Interventionists from all four fidelity sites attended intervention training. The majority of group participants at the four sites (71%) received the therapeutic dose of 10 out of 16 sessions. Weekly group meeting attendance (including at ‘out of venue’ sessions) was excellent at 80%. Additionally, all but one individual session was attended by the participants who completed the intervention. It proved feasible to create tools derived from the fidelity framework to assess in-venue group aspects of this complex intervention. Results of fidelity assessment of the observed groups were good with substantial inter-rater reliability between researchers KAPPA 0.68 95% CI (0.58–0.78). Self-report by interventionists concurred with researcher assessments. Conclusions There was good fidelity to training and delivery of the group aspect of the intervention at four sites. However, the methodological challenges of assessing all aspects of this complex intervention could not be overcome due to practicalities, assessment methods and ethical considerations. Questions remain regarding how we can assess fidelity in community-based complex interventions without impacting upon intervention or trial delivery. Trial registration ISRCTN17993825.

Methodological Challenges and Solution Strategies During Implementation of a Midwife-led Multicentre Randomized-controlled Trial (RCT) in Maternity Hospitals

2021 ◽  
Author(s):  
Sabine Striebich ◽  
Elke Mattern ◽  
Theresa Oganowski ◽  
Rainhild Schäfers ◽  
Gertrud M. Ayerle
Keyword(s):  
Complex Intervention ◽  
Controlled Trial ◽  
Complex Interventions ◽  
Quality Standard ◽  
Quality Data ◽  
Technical Failure ◽  
Maternity Hospitals ◽  
Study Teams ◽  
Randomised Controlled ◽  
Delivery Rooms

Abstract Background: RCTs with complex interventions are methodically challenging. Careful planning under everyday conditions in compliance with the relevant international quality standard (ICH-GCP guideline) is crucial. Specific challenges exist for RCTs conducted in delivery rooms due to various factors that cannot be planned beforehand, such as “peak hours” of births and a high work burden for midwives and obstetricians. Moreover, in Germany as well as in other countries, midwives and obstetricians have frequently little experience as investigators in clinical trials. Methods: The randomised controlled trial “BE-UP” tests the effectiveness of an alternative birthing room on the rate of vaginal births and woman-oriented outcomes. In the process of implementing the trial in 17 obstetrical units and in the endeavour to reach the calculated sample size of 3,800 women, the research team encountered a variety of unexpected challenges. The aim is to describe in greater detail the methodical and organisational challenges and to inform about the research team’s strategies to overcome them.Results:The results are presented in five sectors: 1) Selection of and support for cooperating hospitals: they are to be selected according to predefined criteria and strategies to offer continuous support in trial implementation must be mapped out.2) Establishing a process of requesting informed consent: a quality-assured process to inform pregnant women early on must be feasible and effective. 3) Individual, digital real time randomization: besides instructing the maternity teams appropriate measures for technical failure must be provided.4) The standardized birthing room: the complex intervention is to be implemented according to study protocol, yet adapted to the prevailing conditions in the delivery rooms.5) GCP-compliant documentation: midwives and obstetricians is to be instructed in high quality data collection, supported by external monitoring throughout the trial.Conclusion:Since not all potential challenges can be anticipated in the planning of a trial, study teams need to be flexible and react promptly to any problems that threaten recruitment or the implementation of the complex intervention. Thought should be given to the perspectives of midwives and obstetricians as recruiters and how clinic-intern processes could be adapted to correspond with the trial’s requirements.

Abstract 09: Technology Fluency is Not a Barrier to User Adoption of a Mobile Health (mHealth) Wrist-worn Physical Activity (PA) Monitor System: Observations From the Washington, D.C. Cardiovascular (CV) Health and Needs Assessment

Circulation ◽  
2016 ◽  
Vol 133 (suppl_1) ◽  
Author(s):  
Leah Yingling ◽  
Colby Ayers ◽  
Marlene Peters-Lawrence ◽  
Gwenyth Wallen ◽  
Valerie Mitchell ◽  
...  
Keyword(s):  
At Risk ◽  
African American ◽  
Data Collection ◽  
Monitoring System ◽  
Self Report ◽  
Community Based Interventions ◽  
Community Based ◽  
Faith Based ◽  
User Adoption ◽  
Technology Fluency

Background: Emerging mHealth technologies, like wrist worn PA monitors, offer potential for targeting CV health behaviors among at-risk groups in community-based interventions. It is unknown whether technology fluency impedes user adoption of such mHealth interventions. Methods: A CV health evaluation designed using community-based participatory research principles was conducted in African American, faith-based organizations in Washington D.C. wards with the highest obesity rates (NCT01927783). Participants (n=100) underwent a CV health assessment at a community church and were trained to use an mHealth PA monitor for the ensuing 30-day period. Participants wirelessly uploaded PA data weekly to a data collection hub at a participating church and accessed data online. Users were participants with ≥1 days of data; non-users had zero days of data. A validated Computer-Email-Web fluency self-report instrument captured technology fluency, with skill levels from 1 (no fluency) to 5 (high fluency). Results: Eighty-one participants were users (mean age=60, 78% female); 19 were non-users (mean age=57, 84% female). Users were more likely than non-users to report a lower household income (p=0.01). No differences were noted for computer access (94% vs 94%, p=0.2) or cell phone ownership (88% vs 89%, p=0.2). Technology fluency was similar, with all non-users and 95% of users reporting some level of technology fluency for the 17 surveyed skills (Table). Conclusions: In African-American, faith-based communities in at-risk Washington D.C. areas, lower technology fluency does not appear to impede adoption of this mHealth PA-monitoring system, despite lower socioeconomic status among users. This relationship is likely explained by similar technology access among users and non-users. These findings suggest that an mHealth PA monitoring system using a data collection hub may facilitate a future PA intervention for improved CV health in an at-risk African American community, independent of the population’s technology fluency.

scholarly journals Evaluation of intervention fidelity of a complex psychosocial intervention Lifestyle Matters: a randomised controlled trial

BMJ Open ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. e043478
Author(s):  
Kirsty Sprange ◽  
Gail Mountain ◽  
Claire Craig
Keyword(s):  
Randomised Controlled Trial ◽  
Controlled Trial ◽  
Complex Interventions ◽  
Well Being ◽  
Alternative Methods ◽  
Service Development ◽  
Main Challenge ◽  
Fidelity Assessment ◽  
Randomised Controlled ◽  
Mental Well Being

ObjectivesRobust research of complex interventions designed to promote mental well-being in later life is required to inform service development. An essential component is ensuring that such interventions are delivered as intended. We present a detailed description of the design and implementation of a fidelity assessment within a trial of one such intervention (Lifestyle Matters). The findings help to explain the trial results and also inform the design of embedded fidelity assessments within future evaluations of complex interventions.DesignWe conducted a mixed-method fidelity assessment embedded as part of a multicentre pragmatic randomised controlled trial. A conceptual fidelity framework was developed from the Behaviour Change Consortium framework. From this the fidelity assessment was designed. The resulting instrument assessed the following parameters: intervention design, training, supervision; and delivery, receipt and enactment of the intervention.InterventionThe Lifestyle Matters intervention was designed to assist older people to improve and sustain mental well-being through participation in meaningful activity. The aim is to enable participants to engage in both new and neglected activities through a mix of facilitated group meetings and individual sessions.ResultsThe fidelity assessment demonstrated that the intervention was delivered as per protocol for the group component and was tailored to meet individual needs. There was substantial inter-rater agreement for training; and group member performance 0.72; and moderate agreement for facilitator performance 0.55. It was not possible to determine whether small declines seen in facilitator performance were due to facilitator drift or moderating factors such as group dynamics or participant characteristics.ConclusionsThe assessment methods adequately measured criteria identified as being significant indicators of fidelity. Adherence during training, delivery and supervision was good. The subjective nature of identification and rating observed behaviours was the main challenge. Future research should explore alternative methods of assessing fidelity in trials of complex interventions.Trial registration numberISRCTN67209155.

scholarly journals Fidelity Assessment of a Social Work–Led Intervention Among Patients With Firearm Injuries

2020 ◽  
Vol 30 (6) ◽  
pp. 678-687
Author(s):  
Vivian H. Lyons ◽  
Lina R. Benson ◽  
Elizabeth Griffin ◽  
Anthony S. Floyd ◽  
Sharon W. Kiche ◽  
...  
Keyword(s):  
Social Work ◽  
Controlled Trial ◽  
Implementation Fidelity ◽  
Assessment Process ◽  
Ongoing Trial ◽  
Intervention Delivery ◽  
Firearm Injury ◽  
Firearm Injuries ◽  
Fidelity Assessment ◽  
Study Intervention

Purpose: To support future development and refinement of social work–led intervention programs among patients with firearm injuries and to demonstrate how a fidelity assessment can be used to adjust and refine intervention delivery in an ongoing trial. Method: We conducted a fidelity assessment of a randomized controlled trial of a social work–led intervention among patients with a firearm injury. Results: We found that our study intervention was well implemented, meeting 70% of the fidelity assessment score items, however, noted lower fidelity with client-based items. Discussion: As a result of fidelity assessment findings, we refined intervention delivery to improve implementation fidelity including beginning to review cases of all patients each month rather than focusing on patients in crisis. Our fidelity assessment process and findings offer insight into the challenges of implementing an intervention among patients with firearm injuries and highlight the value of monitoring intervention fidelity during an ongoing trial.

scholarly journals The DIAMONDS intervention to support self-management of type 2 diabetes in people with severe mental illness: study protocol for a single-group feasibility study

2021 ◽  
Author(s):  
Jennifer V E Brown ◽  
Ramzi Ajjan ◽  
Sarah Alderson ◽  
Jan R Böhnke ◽  
Claire Carswell ◽  
...  
Keyword(s):  
Type 2 Diabetes ◽  
Mental Illness ◽  
Data Collection ◽  
Severe Mental Illness ◽  
Feasibility Study ◽  
Controlled Trial ◽  
Self Management ◽  
Intervention Delivery ◽  
Intervention Session

AbstractIntroductionThe DIAMONDS programme aims to evaluate a novel supported diabetes self-management intervention for people with severe mental illness (the “DIAMONDS intervention”). The purpose of this study is to test the feasibility of intervention delivery and data collection procedures to inform a definitive randomised controlled trial (RCT).MethodsAdults aged 18 years or over with a diagnosis of type 2 diabetes and severe mental illness (schizophrenia, schizoaffective disorder, or bipolar disorder) will be eligible for inclusion. Individuals with other types of diabetes or non-psychotic mental illness and those lacking capacity to consent will not be eligible. Participants will be recruited from NHS mental health trusts and general practices across the North of England. All participants will receive the DIAMONDS intervention: weekly one-to-one sessions with a trained facilitator (“DIAMONDS Coach”) to support goal setting, action planning, and diabetes education; ongoing self-management supported by a paper-based workbook and optional digital application (app); and monthly peer-support group sessions with other participants. The primary outcomes are: 1. Recruitment rate, measured as proportion of the recruitment target (N=30) achieved at 5 months from start of recruitment, 2. Attrition measured as the proportion of missing outcomes data at the end of the recruitment period (5 months from start of recruitment) for physiological and self-reported data items, 3. Intervention delivery rate recorded as the proportion of planned sessions delivered (measured by the number of completed intervention session logs per participant within 15 weeks of the first intervention session). Secondary outcomes include completeness of data collection at baseline and of process evaluation data at follow-up as well as the feasibility and acceptability of the intervention and of wearing a blinded continuous glucose monitoring device. An intervention fidelity framework will also be developed. Recruitment started in July 2021. The study was prospectively registered: ISRCTN15328700 (12th March 2021).DiscussionThe results of this feasibility study will inform the refinement of the content and delivery of the DIAMONDS intervention, as well as research procedures, including recruitment and data collection, in preparation for the main DIAMONDS RCT.

scholarly journals Methodological challenges and solution strategies during implementation of a midwife-led multicenter randomized controlled trial (RCT) in maternity hospitals

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Sabine Striebich ◽  
Elke Mattern ◽  
Theresa Oganowski ◽  
Rainhild Schäfers ◽  
Gertrud Ayerle
Keyword(s):  
Clinical Trials ◽  
Randomized Controlled Trial ◽  
Complex Intervention ◽  
Controlled Trial ◽  
Complex Interventions ◽  
Quality Data ◽  
Solution Strategies ◽  
Randomized Controlled ◽  
Link Type ◽  
Delivery Rooms

Abstract Background Randomized controlled trials (RCTs), especially multicentric, with complex interventions are methodically challenging. Careful planning under everyday conditions in compliance with the relevant international quality standard (Good Clinical Practice [GCP] guideline) is crucial. Specific challenges exist for RCTs conducted in delivery rooms due to various factors that cannot be planned beforehand. Few published RCTs report challenges and problems in implementing complex interventions in maternity wards. In Germany as well as in other countries, midwives and obstetricians have frequently little experience as investigators in clinical trials. Methods The aim is to describe the key methodological and organizational challenges in conducting a multicenter study in maternity wards and the solution strategies applied to them. In particular, project-related and process-oriented challenges for hospital staff are considered. The exemplarily presented randomized controlled trial “BE-UP” investigates the effectiveness of an alternative design of a birthing room on the rate of vaginal births and women-specific outcomes. Results The results are presented in five sectors: 1) Selection of and support for cooperating hospitals: they are to be selected according to predefined criteria, and strategies to offer continuous support in trial implementation must be mapped out. 2) Establishing a process of requesting informed consent: a quality-assured process to inform pregnant women early on must be feasible and effective. 3) Individual digital real-time randomization: In addition to instructing maternity teams, appropriate measures for technical failure must be provided. 4) The standardized birthing room: The complex intervention is to be implemented according to the study protocol yet adapted to the prevailing conditions in the delivery rooms. 5) GCP-compliant documentation: midwives and obstetricians will be instructed in high-quality data collection, supported by external monitoring throughout the trial. Conclusion Since not all potential challenges can be anticipated in the planning of a trial, study teams need to be flexible and react promptly to any problems that threaten recruitment or the implementation of the complex intervention. Thought should be given to the perspectives of midwives and obstetricians as recruiters and how clinic-intern processes could be adapted to correspond with the trial’s requirements. Trial registration The BE-UP study was registered on 07/03/ 2018 in the German Register for Clinical Trials under Reference No. DRKS00012854 and can also be found on the International Clinical Trials Registry Platform (ICTRP) (see https://apps.who.int/trialsearch/Trial2.aspx?TrialID=DRKS0001285).

Exploring the Influences of Assessment Method, Intervention Steps, Intervention Sessions, and Observation Timing on Treatment Fidelity Estimates

2019 ◽  
Vol 46 (1) ◽  
pp. 3-13 ◽  
Author(s):  
Melissa A. Collier-Meek ◽  
Lisa M. Sanetti ◽  
Lindsay Fallon ◽  
Sandra Chafouleas
Keyword(s):  
Data Collection ◽  
Direct Observation ◽  
Treatment Fidelity ◽  
Assessment Method ◽  
Self Report ◽  
Group Contingency ◽  
Research Reporting ◽  
Fidelity Assessment ◽  
Implementation Assessment ◽  
Assessment Research

Treatment fidelity data are critical to evaluate intervention effectiveness, yet there are only general guidelines regarding treatment fidelity measurement. Initial investigations have found treatment fidelity data collected via direct observation to be more reliable than data collected via permanent product or self-report. However, the comparison of assessment methods is complicated by the intervention steps accounted for, observation timing, and intervention sessions accounted for, which may impact treatment fidelity estimates. In this study, we compared direct observation and permanent product data to evaluate these varied assessment and data collection decisions on treatment fidelity data estimates in three classrooms engaged in a group contingency intervention. Findings revealed that treatment fidelity estimates, in addition to being different across assessment method, are, in fact, different depending on the intervention steps assessed, intervention sessions accounted for, and observation timing. Implications for treatment fidelity assessment research, reporting in intervention research broadly, and implementation assessment in practice are described.

scholarly journals Addressing methodological challenges in implementing the nursing home pain management algorithm randomized controlled trial

2012 ◽  
Vol 9 (5) ◽  
pp. 634-644 ◽  
Author(s):  
Mary Ersek ◽  
Nayak Polissar ◽  
Anna Du Pen ◽  
Anita Jablonski ◽  
Keela Herr ◽  
...  
Keyword(s):  
Older Adults ◽  
Randomized Controlled Trial ◽  
Nursing Home ◽  
Pain Management ◽  
Outcome Measures ◽  
Controlled Trial ◽  
Self Report ◽  
Evidence Based ◽  
Randomized Controlled ◽  
Methodological Challenges

Background Unrelieved pain among nursing home (NH) residents is a well-documented problem. Attempts have been made to enhance pain management for older adults, including those in NHs. Several evidence-based clinical guidelines have been published to assist providers in assessing and managing acute and chronic pain in older adults. Despite the proliferation and dissemination of these practice guidelines, research has shown that intensive systems-level implementation strategies are necessary to change clinical practice and patient outcomes within a health-care setting. One promising approach is the embedding of guidelines into explicit protocols and algorithms to enhance decision making. Purpose The goal of the article is to describe several issues that arose in the design and conduct of a study that compared the effectiveness of pain management algorithms coupled with a comprehensive adoption program versus the effectiveness of education alone in improving evidence-based pain assessment and management practices, decreasing pain and depressive symptoms, and enhancing mobility among NH residents. Methods The study used a cluster-randomized controlled trial (RCT) design in which the individual NH was the unit of randomization. The Roger’s Diffusion of Innovations theory provided the framework for the intervention. Outcome measures were surrogate-reported usual pain, self-reported usual and worst pain, and self-reported pain-related interference with activities, depression, and mobility. Results The final sample consisted of 485 NH residents from 27 NHs. The investigators were able to use a staggered enrollment strategy to recruit and retain facilities. The adaptive randomization procedures were successful in balancing intervention and control sites on key NH characteristics. Several strategies were successfully implemented to enhance the adoption of the algorithm. Limitations/Lessons The investigators encountered several methodological challenges that were inherent to both the design and implementation of the study. The most problematic issue concerned the measurement of outcomes in persons with moderate to severe cognitive impairment. It was difficult to identify valid, reliable, and sensitive outcome measures that could be applied to all NH residents regardless of the ability to self-report. Another challenge was the inability to incorporate advances in implementation science into the ongoing study Conclusions Methodological challenges are inevitable in the conduct of an RCT. The need to optimize internal validity by adhering to the study protocol is compromised by the emergent logistical issues that arise during the course of the study.

Sign in / Sign up

Export Citation Format

Share Document