Effectiveness of WhatsApp for measuring migration in follow-up phone surveys - Lessons from a mode experiment in two low-income countries during COVID contact restrictions

Phone surveys have increasingly become important data collection tools in developing countries, particularly in the context of sudden contact restrictions due to the COVID-19 pandemic. Phone surveys offer particular potential for migration scholars aiming to study cross-border migration behavior. Geographic change of location over time complicates the logistics of face-to-face surveys and heavily increases costs. There is, however, limited evidence of the effectiveness of the phone survey modes in different geographic settings more generally, and in migration research more specifically. In this field experiment, we compared the response rates between WhatsApp—a relatively new but increasingly important survey mode—and interactive voice response (IVR) modes, using a sample of 8446 contacts in Senegal and Guinea. At 12%, WhatsApp survey response rates were nearly eight percentage points lower than IVR survey response rates. However, WhatsApp offers higher survey completion rates, substantially lower costs and does not introduce more sample selection bias compared to IVR. We discuss the potential of WhatsApp surveys in low-income contexts and provide practical recommendations for field implementation.

Improving postal survey response using behavioural science: a nested randomised control trial

Background Systematic reviews have identified effective strategies for increasing postal response rates to questionnaires; however, most studies have isolated single techniques, testing the effect of each one individually. Despite providing insight into explanatory mechanisms, this approach lacks ecological validity, given that multiple techniques are often combined in routine practice. Methods We used a two-armed parallel randomised controlled trial (n = 2702), nested within a cross-sectional health survey study, to evaluate whether using a pragmatic combination of behavioural science and evidenced-based techniques (e.g., personalisation, social norms messaging) in a study invitation letter increased response to the survey, when compared with a standard invitation letter. Participants and outcome assessors were blinded to group assignment. We tested this in a sample of women testing positive for human papillomavirus (HPV) at cervical cancer screening in England. Results Overall, 646 participants responded to the survey (response rate [RR] = 23.9%). Logistic regression revealed higher odds of response in the intervention arm (n = 357/1353, RR = 26.4%) compared with the control arm (n = 289/1349, RR = 21.4%), while adjusting for age, deprivation, clinical site, and clinical test result (aOR = 1.30, 95% CI: 1.09–1.55). Conclusion Applying easy-to-implement behavioural science and evidence-based methods to routine invitation letters improved postal response to a health-related survey, whilst adjusting for demographic characteristics. Our findings provide support for the pragmatic adoption of combined techniques in routine research to increase response to postal surveys. Trial registration ISRCTN, ISRCTN15113095. Registered 7 May 2019 – retrospectively registered.

A Meta-Analysis of How Country-Level Factors Affect Web Survey Response Rates

A major challenge in web-based cross-cultural data collection is varying response rates, which can result in low data quality and non-response bias. Country-specific factors such as the political and demographic, economic, and technological factors as well as the socio-cultural environment may have an effect on the response rates to web surveys. This study evaluates web survey response rates using meta-analytical methods based on 110 experimental studies from seven countries. Three dependent variables, so-called effect sizes, are used: the web response rate, the response rate to the comparison survey mode, and the difference between the two response rates. The meta-analysis indicates that four country-specific factors (political and demographic, economic, technological, and socio-cultural) impact the magnitude of web survey response rates. Specifically, web surveys achieve high response rates in countries with high population growth, high internet coverage, and a high survey participation propensity. On the other hand, web surveys are at a disadvantage in countries with a high population age and high cell phone coverage. This study concludes that web surveys can be a reliable alternative to other survey modes due to their consistent response rates and are expected to be used more frequently in national and international settings.

Method Bias Mechanisms and Procedural Remedies

As a crucial method in organizational and social behavior research, self-report surveys must manage method bias. Method biases are distorted scores in survey response, distorted variance in variables, and distorted relational estimates between variables caused by method designs. Studies on method bias have focused on post hoc statistical control, but integrated analyses of the sociopsychological mechanism of method bias are lacking. This review proposes a framework for method bias and offers a relatively complete and detailed review of the sociopsychological and statistical mechanisms of four main types of method bias and their procedural remedies. This review proposes “reduce, remove, and rectify” as a guideline for researchers in survey design to address method bias. Finally, this review presents two directions for future methodology research.

The Effect of the Familiarity of a Survey Sender on Response Outcomes in a Large-Scale Survey of Emergency Medical Services Agencies

Survey response is higher when the request comes from a familiar entity compared to an unknown sender. Little is known about how sender influences response to surveys of organizations. We assessed whether familiarity of the sender influences response outcomes in a survey of emergency medical services agencies. Emergency medical services agencies in one U.S. state were randomly assigned to receive survey emails from either a familiar or unfamiliar sender. Both deployment approaches were subsequently used nationwide, with each state selecting one of the two contact methods. Experimental results showed that requests from the familiar sender achieved higher survey response (54.3%) compared to requests from the unfamiliar sender (36.9%; OR: 2.03; 95% CI: 1.23, 3.33). Similar results were observed in the subsequent nationwide survey; in states where the familiar sender deployed the survey, 62.0% of agencies responded, compared to 51.0% when the survey was sent by the unfamiliar sender (OR: 1.57; 95% CI: 1.47, 1.67). The response difference resulted in nearly 60 additional hours of staff time needed to perform telephone follow-up to nonrespondents. When surveying healthcare organizations, surveyors should recognize that it is more challenging to obtain responses without a pre-established relationship with the organizations.

Improving Postal Survey Response Using Behavioural Science: A Nested Randomised Control Trial

Background Systematic reviews have identified effective strategies for increasing postal response rates to questionnaires; however, most studies have isolated single techniques, testing the effect of each one individually. Despite providing insight into explanatory mechanisms, this approach lacks ecological validity, given that multiple techniques are often combined in routine practice. Methods We used a two-armed randomised controlled trial (N = 2,702), nested within a cross-sectional health survey study, to evaluate whether using a pragmatic combination of behavioural science and evidenced-based techniques (e.g., personalisation, social norms messaging) in a study invitation letter increased response to the survey, when compared with a standard invitation letter. Results Overall, 646 participants responded to the survey (response rate [RR] = 23.9%). Logistic regression revealed higher odds of response in the intervention arm (N = 357/1,353, RR = 26.4%) compared with the control arm (N = 289/1,349, RR = 21.4%), while adjusting for age, deprivation, clinical site, and clinical test result (aOR = 1.30, 95% CI: 1.09–1.55). Conclusion Applying easy-to-implement behavioural science and evidence-based methods to routine invitation letters improved postal response to a health-related survey, whilst adjusting for demographic characteristics. Our findings provide support for the pragmatic and cost-effective adoption of combined techniques in routine research to increase response to postal surveys. Trial registration: ISRCTN, ISRCTN15113095. Registered 7 May 2019 – retrospectively registered: https://doi.org/10.1186/ISRCTN15113095