Bah humbug! Association between sending Christmas cards to trial participants and trial retention: randomised study within a trial conducted simultaneously across eight host trials

Objectives To determine the effectiveness of sending Christmas cards to participants in randomised controlled trials to increase retention rate at follow-ups, and to explore the feasibility of doing a study within a trial (SWAT) across multiple host trials simultaneously. Design Randomised SWAT conducted simultaneously across eight host trials. Setting Eight randomised controlled trials researching various areas including surgery and smoking cessation. Participants 3223 trial participants who were still due at least one follow-up from their host randomised controlled trial. Intervention Participants were randomised (1:1, separately by each host trial) to either received a Christmas card in mid-December 2019 or to not receive a card. Main outcome measure Proportion of participants completing their next follow-up (retention rate) within their host randomised controlled trial. Results 1469 participants (age 16-94 years; 70% (n=1033) female; 96% (813/847) white ethnicity) across the eight host randomised controlled trials were involved in the analysis (cut short owing to covid-19). No evidence was found of a difference in retention rate between the two arms for any of the host trials when analysed separately or when the results were combined (85.3% (639/749) for cards versus 85.4% (615/720) for no card; odds ratio 0.96, 95% confidence interval 0.71 to 1.29; P=0.77). No difference was observed when comparing just participants who were due a follow-up in the 30 days after receiving the card (odds ratio 0.96, 0.42 to 2.21). No evidence of a difference in time to complete the questionnaire was found (hazard ratio 1.01, 95% confidence interval 0.91 to 1.13; P=0.80). These results were robust to post hoc sensitivity analyses. The cost of this intervention was £0.76 (€0.91; $1.02) per participant, and it will have a carbon footprint of approximately 140 g CO 2 equivalent per card. One benefit of this approach was the need to only submit one ethics application. Conclusions Sending Christmas cards to participants in randomised controlled trials does not increase retention. Undertaking a SWAT within multiple randomised controlled trials at the same time is, however, possible. This approach should be used more often to build an evidence base to support selection of recruitment and retention strategies. Although no evidence of a boost to retention was found, embedding a SWAT in multiple host trials simultaneously has been shown to be possible. Study registration SWAT repository https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/FileStore/Filetoupload,846275,en.pdf#search=SWAT%2082 .

Testing approaches to sharing trial results with participants: The Show RESPECT cluster randomised, factorial, mixed methods trial

Background Sharing trial results with participants is an ethical imperative but often does not happen. We tested an Enhanced Webpage versus a Basic Webpage, Mailed Printed Summary versus no Mailed Printed Summary, and Email List Invitation versus no Email List Invitation to see which approach resulted in the highest patient satisfaction with how the results were communicated. Methods and findings We carried out a cluster randomised, 2 by 2 by 2 factorial, nonblinded study within a trial, with semistructured qualitative interviews with some patients (ISRCTN96189403). Each cluster was a UK hospital participating in the ICON8 ovarian cancer trial. Interventions were shared with 384 ICON8 participants who were alive and considered well enough to be contacted, at 43 hospitals. Hospitals were allocated to share results with participants through one of the 8 intervention combinations based on random permutation within blocks of 8, stratified by number of participants. All interventions contained a written plain English summary of the results. The Enhanced Webpage also contained a short video. Both the Enhanced Webpage and Email contained links to further information and support. The Mailed Printed Summary was opt-out. Follow-up questionnaires were sent 1 month after patients had been offered the interventions. Patients’ reported satisfaction was measured using a 5-point scale, analysed by ordinal logistic regression estimating main effects for all 3 interventions, with random effects for site, restricted to those who reported receiving the results and assuming no interaction. Data collection took place in 2018 to 2019. Questionnaires were sent to 275/384 randomly selected participants and returned by 180: 90/142 allocated Basic Webpage, 90/133 Enhanced Webpage; 91/141 no Mailed Printed Summary, 89/134 Mailed Printed Summary; 82/129 no Email List Invitation, 98/146 Email List Invitation. About 3 patients opted out of receiving the Mailed Printed Summary; no patients signed up to the email list. Patients’ satisfaction was greater at sites allocated the Mailed Printed Summary, where 65/81 (80%) were quite or very satisfied compared to sites with no Mailed Printed Summary 39/64(61%), ordinal odds ratio (OR) = 3.15 (1.66 to 5.98, p < 0.001). We found no effect on patient satisfaction from the Enhanced Webpage, OR = 1.47 (0.78 to 2.76, p = 0.235) or Email List Invitation, OR = 1.38 (0.72 to 2.63, p = 0.327). Interviewees described the results as interesting, important, and disappointing (the ICON8 trial found no benefit). Finding out the results made some feel their trial participation had been more worthwhile. Regardless of allocated group, patients who received results generally reported that the information was easy to understand and find, were glad and did not regret finding out the results. The main limitation of our study is the 65% response rate. Conclusions Nearly all respondents wanted to know the results and were glad to receive them. Adding an opt-out Mailed Printed Summary alongside a webpage yielded the highest reported satisfaction. This study provides evidence on how to share results with other similar trial populations. Further research is needed to look at different results scenarios and patient populations. Trial registration ISRCTN: ISRCTN96189403.

Pre-notification and personalisation of text messages to increase questionnaire completion in a smoking cessation pregnancy RCT: an embedded randomised factorial trial

Background: Low completion rates of questionnaires in randomised controlled trials can compromise the reliability of the results, so ways to boost questionnaire completion are often implemented. Although there is evidence to suggest that sending a text message to participants increases completion, there is little evidence around the timing or personalisation of these text messages. Methods: A two-by-two factorial SWAT (study within a trial) was embedded within the MiQuit-3 trial, looking at smoking cessation within pregnant smokers. Participants who reached their 36-week gestational follow-up were randomised to receive a personalised or non-personalised text message, either one week or one day prior to their follow-up. Primary outcomes were completion rate of questionnaire via telephone. Secondary outcomes included: completion rate via any method, time to completion, and number of attempts to contact required. Results In total 194 participants were randomised into the SWAT to receive a text message that was personalised early(n=50), personalised late (n=47), non-personalised early(n=50), or non-personalised late(n=47). There was no evidence that timing of the text message (early: one week before; or late: one day before) had an effect on any of the outcomes. There was evidence that a personalised text message would result in fewer completions compared with a non-personalised text message when data was collected only via the telephone(adjusted OR 0.44, 95% CI 0.22–0.87, p=0.02). However, these results were not significant when looking at completion via any method (adjusted OR 0.61, 95% CI 0.30-1.24, p=0.17). There was no evidence to show that personalisation or not was better for any of the secondary outcomes. Conclusion Timing of the text message does not appear to influence the completion of questionnaires. Personalisation of a text message may be detrimental to questionnaire completion, if data is only collected via the telephone - however, more SWATs should be undertaken in this field.

A qualitative investigation of reasoning behind decisions to decline participation in a research intervention: A study-within-a-trial

The current study-within-a-trial explored individuals’ decisions to decline participation in research trialling a chronic illness-focused therapy (i.e. multiple sclerosis). Four themes were identified from seven semi-structured interviews with participation decliners and were confirmed by the host trial’s Patient & Public Involvement (PPI) panel: acknowledgement of the value of research; ‘fit’ of the study; misinterpretation of participant information; and ‘ignorance is bliss’ – discussed in light of theory and research. This study-within-a-trial extends research on trial recruitment and participation decline; while also suggesting that PPI can be utilised in both a practical and impactful manner.

Randomised study within a trial (SWAT) to evaluate personalised versus standard text message prompts for increasing trial participant response to postal questionnaires (PROMPTS)

Background Use of a person’s name in a text message has been shown to be effective in instigating behaviour change. We evaluated the effectiveness of a personalised text message (including the recipient’s name) versus a standardised text message for prompting a response from trial participants to complete and return postal follow-up questionnaires. Methods Using a randomised study within a trial (SWAT) embedded within the host GRASP (Getting it Right: Addressing Shoulder Pain) trial, participants who provided a mobile telephone number were randomised (1:1) by a central computer system to receive either (1) a personalised text message which included their name or (2) a standard text message. Text messages were sent by the trial office on the same day as the 6-month GRASP follow-up questionnaire. The primary outcome was questionnaire response rate, defined as the proportion of 6-month GRASP follow-up questionnaires returned by participants. Secondary outcomes included time to response, the proportion of participants sent a reminder follow-up questionnaire, and cost. Results Between March 2017 and May 2019 (recruitment period for GRASP trial), 618 participants were randomised to a personalised (n = 309) or standard (n = 309) text message and all were included in the analysis. The overall questionnaire response rate was 87% (n = 537/618); 90% (n = 277/309) of participants responded in the personalised text message group compared to 84% (n = 260/309) in the standard text message group (relative risk (RR) 1.07; 95% CI 1.00 to 1.13). Participants randomised to receive the personalised text message were more likely to return their initial postal questionnaire than those who received the standard text message (n = 185/309; 60% vs. n = 160/309; 52%) (RR 1.16; 95% CI 1.00 to 1.33); this represents an absolute percentage difference between intervention groups of 8%. Post hoc subgroup analysis showed that males under 65 years were the group most likely to return their initial questionnaire if they received a personalised text message. Conclusion Overall, participants who received a personalised text message were more likely to return their questionnaire than those who received the standard text message. Trial registration GRASP Trial ISRCTN16539266; SWAT Repository ID 35

Pre-notification and personalisation of text-messages to retain participants in a smoking cessation pregnancy RCT: an embedded randomised factorial trial

Background: Low response rates in randomised controlled trials can compromise the reliability of the results, so ways to boost retention are often implemented. Although there is evidence to suggest that sending a text message to participants increases retention, there is little evidence around the timing or personalisation of these messages. Methods: A two-by-two factorial SWAT (study within a trial) was embedded within the MiQuit-3 trial, looking at smoking cessation within pregnant smokers. Participants who reached their 36-week gestational follow-up were randomised to receive a personalised or non-personalised text message, either one week or one day prior to the telephone follow-up. Primary outcomes were completion rate of questionnaire via telephone. Secondary outcomes included: completion rate via any method, time to completion, and number of reminders required. Results In total 194 participants were randomised into the SWAT; 50 to personalised early text, 47 to personalised late text, 50 to non-personalised early text, and 47 to non-personalised late text. There was no evidence that timing of the text message (early: one week before; or late: one day before) had an effect on any of the outcomes. There was evidence that a personalised text would result in fewer completions via telephone compared with a non-personalised text (adjusted OR 0.44, 95% CI 0.22–0.87, p=0.02). However, there was no evidence to show that personalisation or not was better for any of the secondary outcomes. Conclusion Timing of the text message does not appear to influence the retention of participants. Personalisation of a text message may be detrimental to retention; however, more SWATs should be undertaken in this field.

Feasibility and diagnostic accuracy of Telephone Administration of an adapted wound heaLing QuestiONnaire for assessment for surgical site infection following abdominal surgery in low and middle-income countries (TALON): protocol for a study within a trial (SWAT)

Background Surgical site infection is the most common complication of abdominal surgery, with a global impact on patients and health systems. There are no tools to identify wound infection that are validated for use in the global setting. The overall aim of the study described in this protocol is to evaluate the feasibility and validity of a remote, digital pathway for wound assessment after hospital discharge for patients in low- and middle-income countries (LMICs). Methods A multi-centre, international, mixed-methods study within a trial, conducted in two stages (TALON-1 and TALON-2). TALON-1 will adapt and translate a universal reporter outcome measurement tool (Bluebelle Wound Healing Questionnaire, WHQ) for use in global surgical research (SWAT store registration: 126) that can be delivered over the telephone. TALON-2 will evaluate a remote wound assessment pathway (including trial retention) and validate the diagnostic accuracy of this adapted WHQ through a prospective cohort study embedded within two global surgery trials. Embedded community engagement and involvement activities will be used to optimise delivery and ensure culturally attuned conduct. TALON-1 and TALON-2 are designed and will be reported in accordance with best practice guidelines for adaptation and validation of outcome measures, and diagnostic test accuracy studies. Discussion Methods to identify surgical site infection after surgery for patients after hospital discharge have the potential to improve patient safety, trial retention, and research efficiency. TALON represents a large, pragmatic, international study co-designed and delivered with LMIC researchers and patients to address an important research gap in global surgery trial methodology.

Conditional versus Non-Conditional Incentives to Maximise Return of Participant Completed Questionnaires in Clinical Trials: A Cluster Randomised Study Within a Trial

BackgroundHigh participant retention enhances the validity of clinical trials. A monetary incentive can increase retention, but it is not known if when it is provided matters. We aimed to determine whether there was a difference in the number follow-up trial questionnaires returned when a monetary incentive was given to participants at recruitment (non-conditional), compared to informing participants at recruitment that the incentive would be given only once a questionnaire had been returned (conditional).MethodA sub-study within the Antivirals for influenza-Like Illness, An rCt of Clinical and Cost effectiveness in primary CarE (ALIC4E) Trial. Matched sites were randomised using computer generated random numbers, to either a non-conditional or conditional incentive. Analyses were conducted according to randomised group irrespective of compliance with two-sided 5% level statistical significance level. The main analysis was regression accounting for site pair, with additional weighted, paired and non-parametric sensitivity analyses. The total number of vouchers distributed, the cost of administration and postage was calculated. ResultsOf the 42 randomised sites (21 to each intervention) only 28 recruited at least one participant with only 10 practice pairs recruiting participants at both constituent sites. Raw diaries return proportions were 0.58 and 0.73 for non-conditional and conditional incentive groups. Regression analysis adjusted for cluster pair showed no significant difference in returns, -0.09, (95% CI, -0.29, 0.10, p=0.336); when weighted there still no clear difference: 0.15 (-0.02, 0.31, p=0.068). There was no clear statistical evidence of a difference in time taken to return questionnaires, nor proportion of pages completed, by intervention group in the main analyses (all p>0.05). The conditional incentive was approximately £23 cheaper per diary returned based upon observed data. Conclusion There was no clear evidence of a difference in participant completed questionnaire returns according to the time at which an incentive is given, nor to completeness of the returned questionnaires, or the time to questionnaire return. There was substantial statistical uncertainty in findings, and some of the sensitivity analysis suggested that a meaningful benefit of a conditional incentive were plausible. The conditional approach cost less in cash terms.

Can patient decision aids reduce decisional conflict in a de-escalation of breast radiotherapy clinical trial? The PRIMETIME Study Within a Trial implemented using a cluster stepped-wedge trial design

Background For patients with early breast cancer considered at very-low risk of local relapse, risks of radiotherapy may outweigh the benefits. Decisions regarding treatment omission can lead to patient uncertainty (decisional conflict), which may be lessened with patient decision aids (PDA). PRIMETIME (ISRCTN 41579286) is a UK-led biomarker-directed study evaluating omission of adjuvant radiotherapy in breast cancer; an embedded Study Within A Trial (SWAT) investigated whether PDA reduces decisional conflict using a cluster stepped-wedge trial design. Methods PDA diagrams and a video explaining risks and benefits of radiotherapy were developed in close collaboration between patient advocates and PRIMETIME trialists. The SWAT used a cluster stepped-wedge trial design, where each cluster represented the radiotherapy centre and referring peripheral centres. All clusters began in the standard information group (patient information and diagrams) and were randomised to cross-over to the enhanced information group (standard information plus video) at 2, 4 or 6 months. Primary endpoint was the decisional conflict scale (0–100, higher scores indicating greater conflict) which was assessed on an individual participant level. Multilevel mixed effects models used a random effect for cluster and a fixed effect for each step to adjust for calendar time and clustering. Robust standard errors were also adjusted for the clustering effect. Results Five hundred twenty-one evaluable questionnaires were returned from 809 eligible patients (64%) in 24 clusters between April 2018 and October 2019. Mean decisional conflict scores in the standard group (N = 184) were 10.88 (SD 11.82) and 8.99 (SD 11.82) in the enhanced group (N = 337), with no statistically significant difference [mean difference − 1.78, 95%CI − 3.82–0.25, p = 0.09]. Compliance with patient information and diagrams was high in both groups although in the enhanced group only 121/337 (36%) reported watching the video. Conclusion The low levels of decisional conflict in PRIMETIME are reassuring and may reflect the high-quality information provision, such that not everyone required the video. This reinforces the importance of working with patients as partners in clinical trials especially in the development of patient-centred information and decision aids.