Including a pen and/or cover letter, containing social incentive text, had no effect on questionnaire response rate: a factorial randomised controlled Study within a Trial

Background: Postal questionnaires are frequently used in randomised controlled trials to collect outcome data on participants; however, poor response can introduce bias, affect generalisability and validity, and reduce statistical power. The objective of this study was to assess whether a pen and/or social incentive text cover letter sent with a postal follow-up questionnaire increased response rates in a trial. Method: A two-by-two factorial randomised controlled trial was embedded within the OTIS host trial. Participants due their 12-month (final) follow-up questionnaire were randomised to be sent: a pen; a social incentive text cover letter; both; or neither. The primary outcome measure was the proportion of participants in each group who returned the questionnaire. Secondary outcomes were: time to return, completeness of the questionnaire, necessity of a reminder letter, and the cost effectiveness. Results: The overall 12-month questionnaire response rate was 721 out of 755 (95.5%). Neither the pen nor social incentive cover letter had a statistically significant effect on response rate: pen 95.2% vs. no pen 95.8%, adjusted OR 0.90 (95% CI 0.45 to 1.80; p=0.77); social incentive cover letter 95.2% vs. no social incentive cover letter 95.8%, adjusted OR 0.84 (95% CI 0.42 to 1.69, p=0.63). No statistically significant differences were observed between either of the intervention groups on time to response, need for a reminder or completeness. Therefore, neither intervention was cost-effective. Conclusions: We found no evidence of a difference in response rates associated with the inclusion of a pen and/or social incentive cover letter with the final follow-up postal questionnaire of the host trial. However, when these results are combined with previous SWATs, the meta-analysis evidence remains that including a pen increases response rates. The social incentive cover letter warrants further investigation to determine effectiveness. Trial registration: ISRCTN22202133 (21st June 2020).

Download Full-text

Including a pen and/or cover letter, containing social incentive text, had no effect on questionnaire response rate: a factorial randomised controlled Study within a Trial

F1000Research ◽

10.12688/f1000research.23767.1 ◽

2020 ◽

Vol 9 ◽

pp. 623

Author(s):

Sophie James ◽

Adwoa Parker ◽

Sarah Cockayne ◽

Sara Rodgers ◽

Caroline Fairhurst ◽

...

Keyword(s):

Statistical Power ◽

Response Rate ◽

Controlled Trial ◽

Response Rates ◽

Outcome Data ◽

Controlled Study ◽

Cover Letter ◽

Questionnaire Response ◽

Randomised Controlled

Download Full-text

A nested randomised controlled trial of a newsletter and Post-it® note did not increase postal questionnaire response rates in a falls prevention trial

F1000Research ◽

10.12688/f1000research.14591.1 ◽

2018 ◽

Vol 7 ◽

pp. 1083

Author(s):

Sara Rodgers ◽

Illary Sbizzera ◽

Sarah Cockayne ◽

Caroline Fairhurst ◽

Sarah E. Lamb ◽

...

Keyword(s):

Randomised Controlled Trials ◽

Statistical Power ◽

Controlled Trial ◽

Meta Analysis ◽

Response Rates ◽

Postal Questionnaire ◽

Falls Prevention ◽

Controlled Trials ◽

Questionnaire Response ◽

Randomised Controlled

Background: Attrition (i.e. when participants do not return the questionnaires) is a problem for many randomised controlled trials. The resultant loss of data leads to a reduction in statistical power and can lead to bias. The aim of this study was to assess whether a pre-notification newsletter and/or a handwritten or printed Post-it® note sticker, as a reminder, increased postal questionnaire response rates for participants of randomised controlled trials. Method: This study was a factorial trial embedded within a trial of a falls-prevention intervention among men and women aged ≥65 years under podiatric care. Participants were randomised into one of six groups: newsletter plus handwritten Post-it®; newsletter plus printed Post-it®; newsletter only; handwritten Post-it® only; printed Post-it® only; or no newsletter or Post-it®. The results were combined with those from previous embedded randomised controlled trials in a meta-analysis. Results: The 12-month response rate was 803/826 (97.2%) (newsletter 95.1%, no newsletter 99.3%, printed Post-it® 97.5%, handwritten Post-it® 97.1%, no Post-it® 97.1%). Pre-notification with a newsletter had a detrimental effect on response rates (adjusted odds ratio (OR), 0.14; 95% CI, 0.04 to 0.48; p<0.01) and time to return the questionnaire (adjusted hazard ratio, 0.86; 95% CI, 0.75 to 0.99; p=0.04). No other statistically significant differences were observed between the intervention groups on response rates, time to response, and the need for a reminder. Conclusions: Post-it® notes have been shown to be ineffective in three embedded trials, whereas the evidence for newsletter reminders is still uncertain.

Download Full-text

A study update newsletter or Post-it® note did not increase postal questionnaire response rates in a falls prevention trial: an embedded randomised factorial trial

F1000Research ◽

10.12688/f1000research.14591.2 ◽

2019 ◽

Vol 7 ◽

pp. 1083

Author(s):

Sara Rodgers ◽

Illary Sbizzera ◽

Sarah Cockayne ◽

Caroline Fairhurst ◽

Sarah E. Lamb ◽

...

Keyword(s):

Randomised Controlled Trials ◽

Statistical Power ◽

Controlled Trial ◽

Response Rates ◽

Postal Questionnaire ◽

Falls Prevention ◽

Controlled Trials ◽

Questionnaire Response ◽

Factorial Trial ◽

Randomised Controlled

Background: Participants not returning data collection questionnaires is a problem for many randomised controlled trials. The resultant loss of data leads to a reduction in statistical power and can result in bias. The aim of this study was to assess whether the use of a study update newsletter and/or a handwritten or printed Post-it® note sticker increased postal questionnaire response rates for participants of a randomised controlled trial. Method: This study was a factorial trial embedded within a host trial of a falls-prevention intervention among men and women aged ≥65 years under podiatric care. Participants were randomised into one of six groups: newsletter plus handwritten Post-it®; newsletter plus printed Post-it®; newsletter only; handwritten Post-it® only; printed Post-it® only; or no newsletter or Post-it®. The results were combined with those from previous embedded randomised controlled trials in meta-analyses. Results: The overall 12-month response rate was 803/826 (97.2%) (newsletter 95.1%, no newsletter 99.3%, printed Post-it® 97.5%, handwritten Post-it® 97.1%, no Post-it® 97.1%). The study update newsletter had a detrimental effect on response rates (adjusted odds ratio 0.14, 95% CI 0.04 to 0.48, p<0.01) and time to return the questionnaire (adjusted hazard ratio 0.86, 95% CI 0.75 to 0.99, p=0.04). No other statistically significant differences were observed between the intervention groups on response rates, time to response, and the need for a reminder. Conclusions: Post-it® notes have been shown to be ineffective in three embedded trials, whereas the evidence for newsletter reminders is still uncertain.

Download Full-text

Does pre-notification increase questionnaire response rates: a randomised controlled trial nested within a systematic review

BMC Medical Research Methodology ◽

10.1186/s12874-021-01462-z ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Benjamin Woolf ◽

Phil Edwards

Keyword(s):

Systematic Review ◽

Controlled Trial ◽

Response Rates ◽

Open Science ◽

Outcome Data ◽

Absolute Difference ◽

Questionnaire Response ◽

Link Type ◽

Science Framework ◽

Randomised Controlled

Abstract Background Missing outcome data can lead to bias in the results of systematic reviews. One way to address missing outcome data is by requesting the data from the trial authors, but non-response is common. One way to potentially improve response rates is by sending study participants advance communication. During the update of a systematic review examining the effect of pre-notification on response rates, study authors needed to be contacted for further information. This study was nested within the systematic review by randomising authors to receive a notification of the upcoming request for information. The objective was to test if pre-notification increased response rates. Methods The participants were study authors included in the systematic review, whose studies were at unclear risk of bias. The intervention was a pre-notification of the request for further information, sent 1 day before the request. The outcome was defined as the proportion of authors who responded to the request for information. Authors were randomised by simple randomisation. Thirty three authors were randomised to the pre-notification arm, and 42 were randomised to the control arm. Authors were blinded to the possibility of an alternative condition. Results All authors randomised were analysed. 14/33 (42.4%) authors in the pre-notification arm had returned responses to the questionnaire, and 18/42 (42.9%) in the control arm. There was no evidence of a difference between these groups (absolute difference = − 0.5, 95% CI (− 23.4 to 22.5%), p = 1). We received no complaints about receiving the pre-notification. Conclusions This study’s results do not support the hypothesis that pre-notification increases response from study authors being contacted for a request for more information. However, the study has a low power, and the results may not generalise to other contexts, methods of administering a pre-notification, or study populations. Trial registration Registration and protocol: This trial is not registered with any trial registry. However, the protocol was posted in advance on the Open Science Framework website and is available on the Open Science Framework website: DOI: 10.17605/OSF.IO/MSV2W or https://osf.io/msv2w/

Download Full-text

Evaluation of the effectiveness of an incentive strategy on the questionnaire response rate in parents of premature babies: a randomised controlled Study Within A Trial (SWAT) nested within SIFT

Trials ◽

10.1186/s13063-021-05515-y ◽

2021 ◽

Vol 22 (1) ◽

Author(s):

Edmund Juszczak ◽

Oliver Hewer ◽

Christopher Partlett ◽

Madeleine Hurd ◽

Vasha Bari ◽

...

Keyword(s):

Response Rate ◽

Response Rates ◽

Absolute Difference ◽

Controlled Study ◽

Multiple Birth ◽

Questionnaire Response ◽

Link Type ◽

Premature Babies ◽

To Receive

Abstract Background Loss to follow-up resulting in missing outcomes compromises the validity of trial results by reducing statistical power, negatively affecting generalisability and undermining assumptions made at analysis, leading to potentially biased and misleading results. Evidence that incentives are effective at improving response rates exists, but there is little evidence regarding the best approach, especially in the field of perinatal medicine. The NIHR-funded SIFT trial follow-up of infants at 2 years of age provided an ideal opportunity to address this remaining uncertainty. Methods Participants: parents of infants from participating neonatal units in the UK and Ireland followed up for SIFT (multicentre RCT investigating two speeds of feeding in babies with gestational age at birth < 32 weeks and/or birthweight < 1500 g). Interventions: parents were randomly allocated to receive incentives (£15 gift voucher) before or after questionnaire return. The objective was to establish whether offering an unconditional incentive in advance or promising an incentive on completion of a questionnaire (conditional) improved the response rate in parents of premature babies. The primary outcome was questionnaire response rate. Permuted block randomisation was performed (variable size blocks), stratified by SIFT allocation (slower/faster feeds) and single/multiple birth. Multiple births were given the same incentives allocation. Parents were unaware that they were in an incentives SWAT; SIFT office staff were not blinded to allocation. Results Parents of 923 infants were randomised: 459 infants allocated to receive incentive before, 464 infants allocated to receive incentive after; analysis was by intention to treat. Allocation to the incentive before completion led to a significantly higher response rate, 83.0% (381/459) compared to the after-completion group, 76.1% (353/464); adjusted absolute difference of 6.8% (95% confidence interval 1.6% to 12.0%). Giving an incentive in advance is the more costly approach, but the mean difference of ~£3 per infant is small given the higher return. Conclusions An unconditional incentive in advance led to a significantly higher response rate compared to the promise of an incentive on completion. Against a backdrop of falling response rates to questionnaires, incentives can be an effective way to increase returns. Trial registration SIFT (ISRCTN76463425). Registered on March 5, 2013.; SWAT registration (SWAT 69 available from http://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/FileStore/Filetoupload,864297,en.pdf). Registered on June 27, 2016.

Download Full-text

Effectiveness of SMS messaging for diarrhoea measurement: a factorial cross-over randomised controlled trial

10.21203/rs.3.rs-24263/v3 ◽

2020 ◽

Author(s):

Ryan Trevor Titus Rego ◽

Samuel Watson ◽

Philbert Ishengoma ◽

Philemon Langat ◽

Hezekiah Pireh Otieno ◽

...

Keyword(s):

Randomised Controlled Trial ◽

Text Messaging ◽

Response Rate ◽

Financial Incentive ◽

Controlled Trial ◽

Response Rates ◽

Lower Response ◽

Lower Response Rate ◽

The Mean ◽

Randomised Controlled

Abstract Background Text messaging systems are used to collect data on symptom prevalence. Using a text messaging system, we evaluated the effects of question load, question frequency, and financial incentive on response rates and reported infant diarrhoea rates in an infant diarrhoea survey. Methods We performed a factorial cross-over randomised controlled trial of an SMS surveying system for infant diarrhoea surveillance with treatments: financial incentive (yes/no), question load (1-question/3-question), and questioning frequency (daily/fortnightly). Participants progressed through all treatment combinations over eight two-week rounds. Data were analysed using multivariable logistic regressions to determine the impacts of the treatments on the response rates and reported diarrhoea rates. Attitudes were explored through qualitative interviews. Results For the 141 participants, the mean response rate was 47%. In terms of percentage point differences (ppd), daily questioning was associated with a lower response rate than fortnightly (-1·2[95%CI:-4·9,2·5]); high (3-question) question loads were associated with a lower response rate than low (1-question) question loads (-7·0[95%CI:-10·8,-3·1]); and financial incentivisation was associated with a higher response rate than no financial incentivisation (6·4[95%CI:2·6,10·2]). The mean two-week diarrhoea rate was 36·4%. Daily questioning was associated with a higher reported diarrhoea rate than fortnightly (29·9[95%CI:22·8,36·9]); with little evidence for impact by incentivisation or question load. Conclusions Close to half of all participants responded to the SMS survey. Daily questioning evoked a statistically higher rate of reported diarrhoea, while financial incentivisation and low (1-question) question loads evoked higher response rates than no incentive and high (3-question) question loads respectively. Trial Registration The protocol was registered on ISRCTN on the 20 th of March 2019 under number ISRCTN11410773 .

Download Full-text

Increasing response rates to follow-up questionnaires in health intervention research: Randomized controlled trial of a gift card prize incentive

Clinical Trials ◽

10.1177/1740774517703320 ◽

2017 ◽

Vol 14 (4) ◽

pp. 381-386 ◽

Cited By ~ 2

Author(s):

Amy J Morgan ◽

Ronald M Rapee ◽

Jordana K Bayer

Keyword(s):

Randomized Controlled Trial ◽

Confidence Interval ◽

Odds Ratio ◽

Response Rate ◽

Controlled Trial ◽

Response Rates ◽

Health Intervention ◽

Randomized Controlled ◽

Gift Card

Background/aims Achieving a high response rate to follow-up questionnaires in randomized controlled trials of interventions is important for study validity. Few studies have tested the value of incentives in increasing response rates to online questionnaires in clinical trials of health interventions. This study evaluated the effect of a gift card prize-draw incentive on response rates to follow-up questionnaires within a trial of an online health intervention. Method The study was embedded in a host randomized controlled trial of an online parenting program for child anxiety. A total of 433 participants were randomly allocated to one of two groups: (1) being informed that they would enter a gift card prize-draw if they completed the final study questionnaire (24-week follow-up) and (2) not informed about the prize-draw. All participants had a 1 in 20 chance of winning an AUD50 gift card after they completed the online questionnaire. Results The odds of the informed group completing the follow-up questionnaire were significantly higher than the uninformed group, (79.6% vs 68.5%, odds ratio = 1.79, 95% confidence interval = 1.15–2.79). This response rate increase of 11.1% (95% confidence interval = 2.8–19.1) occurred in both intervention and control groups in the host randomized controlled trial. The incentive was also effective in increasing questionnaire commencement (84.6% vs 75.9%, odds ratio = 1.74, 95% confidence interval = 1.07–2.84) and reducing the delay in completing the questionnaire (19.9 vs 22.6 days, hazard ratio = 1.34, 95% confidence interval = 1.07–1.67). Conclusion This study adds to evidence for the effectiveness of incentives to increase response rates to follow-up questionnaires in health intervention trials.

Download Full-text