scholarly journals The Effects of Nonresponse and Sampling Omissions on Estimates on Various Topics in Federal Surveys: Telephone and IVR Surveys of Address-Based Samples

2020 ◽  
Vol 36 (3) ◽  
pp. 631-645
Author(s):  
Floyd J. Fowler ◽  
Philip Brenner ◽  
Anthony M. Roman ◽  
J. Lee Hargraves
Keyword(s):  
Data Collection ◽  
Interactive Voice Response ◽  
Response Rates ◽  
Demographic Characteristics ◽  
Telephone Surveys ◽  
Telephone Interviews ◽  
Survey Questions ◽  
Nonresponse Error ◽  
Collection Strategies ◽  
Voice Response

AbstractWith declining response rates and challenges of using RDD sampling for telephone surveys, collecting data from address-based samples has become more attractive. Two approaches are doing telephone interviews at telephone numbers matched to addresses and asking those at sampled addresses to call into an Interactive Voice Response (IVR) system to answer questions. This study used in-person interviewing to evaluate the effects of nonresponse and problems matching telephone numbers when telephone and IVR were used as the initial modes of data collection. The survey questions were selected from major US federal surveys covering a variety of topics. Both nonresponse and, for telephone, inability to find matches result in important nonresponse error for nearly half the measures across all topics, even after adjustments to fit the known demographic characteristics of the residents. Producing credible estimates requires using supplemental data collection strategies to reduce error from nonresponse.

Redirected Inbound Call Sampling (RICS)

2019 ◽  
Vol 83 (2) ◽  
pp. 386-411
Author(s):  
Burton Levine ◽  
Karol Krotki ◽  
Paul J Lavrakas
Keyword(s):  
Data Collection ◽  
Telephone Survey ◽  
Interactive Voice Response ◽  
Reliability And Validity ◽  
Cost Savings ◽  
The United States ◽  
Dual Frame ◽  
Telephone Surveys ◽  
Fit For Purpose ◽  
Voice Response

Abstract Redirected Inbound Call Sampling (RICS) is an emerging, nonprobability sampling methodology in which calls to nonworking numbers from callers throughout the United States are redirected to a telephone survey recruitment protocol and data collection system. The use of automated, interactive voice response technologies to recruit, screen, and collect data achieves significant cost savings and greatly reduces the time of fielding a survey compared with traditional dual-frame random-digit dialing (DFRDD) surveys that use interviewers for recruitment and data collection. The implementation of RICS that we fielded resulted in respondents who match some demographics of the population about as well as DFRDD telephone surveys. However, we demonstrate some non-ignorable challenges with measurement error in certain types of questions that arise from primacy effects associated with using an interactive voice response system for data collection. We present the results of a RICS study that was designed to better understand the reliability and validity of the data these surveys generate. The investigation presented in this manuscript is a first step to evaluating if RICS can be a fit-for-purpose solution for some survey needs. Our data suggest there is sufficient promise in the RICS methodology to warrant continued development and refinement.

Comparing face-to-face to web data collection: unit response and costs in a national health survey (Preprint)

2020 ◽  
Author(s):  
Elise Braekman ◽  
Stefaan Demarest ◽  
Rana Charafeddine ◽  
Sabine Drieskens ◽  
Finaba Berete ◽  
...  
Keyword(s):  
Data Collection ◽  
Response Rate ◽  
Response Rates ◽  
Demographic Characteristics ◽  
Unit Response ◽  
Cost Advantage ◽  
Web Data ◽  
Demographic Groups ◽  
Considerable Cost ◽  
The Difference

BACKGROUND Potential is seen in web data collection for population health surveys due to a combination of its cost-effectiveness, implementation ease and the increased internet penetration. Nonetheless, web modes may lead to lower and more selective unit response rates than traditional modes and hence may increase bias in the measured indicators. OBJECTIVE This research assesses the unit response and costs of a web versus F2F study. METHODS Alongside the F2F Belgian Health Interview Survey of 2018 (BHIS2018; n gross sample used: 7,698), a web survey (BHISWEB; n gross sample=6,183) is organized. Socio-demographic data on invited individuals is obtained from the national register and census linkages. Unit response rates considering the different sampling probabilities of both surveys are calculated. Logistic regression analyses examine the association between mode system (web vs. F2F) and socio-demographic characteristics on unit non-response. The costs per completed web questionnaire are compared with these for a completed F2F questionnaire. RESULTS The unit response rate is lower in BHISWEB (18.0%) versus BHIS2018 (43.1%). A lower web response is found among all socio-demographic groups, however, the difference is higher among people older than 65, low educated people, people with a non-Belgian nationality, people living alone and these living in Brussels Capital. Not the same socio-demographic characteristics are associated with non-response in both studies. Having another European (OR (95% CI): 1.60 (1.20-2.13)) or a non-European nationality (OR (95% CI): 2.57 (1.79-3.70)) (compared to having the Belgian nationality) and living in the Brussels Capital (95% CI): 1.72 (1.41-2.10)) or Walloon (OR (95% CI): 1.47 (1.15 - 1.87) region (compared to living in the Flemish region) is only in BHISWEB associated with a higher non-response. In BHIS2018 younger people (OR (95% CI): 1.31 (1.11-1.54)) are more likely to be non-respondent than older people, this was not found BHISWEB. In both studies, lower educated people have a higher change to be non-respondent, but this effect is more pronounced in BHISWEB (OR low vs. high education level (95% CI): Web 2.71 (2.21-3.39)); F2F 1.70 (1.48-1.95)). The BHISWEB study has a considerable cost advantage; the total cost per completed questionnaire is almost three times lower (€41) compared to the F2F data collection (€111). CONCLUSIONS The F2F unit response rate is generally higher, yet for certain groups the difference between web versus F2F is more limited. A considerable cost advantage of web collection is found. It is therefore worthwhile to experiment with adaptive mixed-mode designs to optimize financial resources without increasing selection bias; e.g. only inviting socio-demographic groups more eager to participate online for web surveys while remaining to focus on increasing the F2F response rates for other groups. CLINICALTRIAL Studies approved by the Ethics Committee of the University hospital of Ghent

scholarly journals Automated and Interviewer-Administered Mobile Phone Surveys in Burkina Faso: Sociodemographic Differences Among Female Mobile Phone Survey Respondents and Nonrespondents (Preprint)

2020 ◽  
Author(s):  
Abigail R Greenleaf ◽  
Aliou Gadiaga ◽  
Yoonjoung Choi ◽  
Georges Guiella ◽  
Shani Turke ◽  
...  
Keyword(s):  
Burkina Faso ◽  
Cell Phone ◽  
Telephone Interview ◽  
Interactive Voice Response ◽  
Computer Assisted ◽  
Computer Assisted Telephone Interview ◽  
Telephone Interviews ◽  
Phone Survey ◽  
Voice Response

BACKGROUND The remarkable growth of cell phone ownership in low- and middle-income countries has generated significant interest in using cell phones for conducting surveys through computer-assisted telephone interviews, live interviewer-administered surveys, or automated surveys (ie, interactive voice response). OBJECTIVE This study aimed to compare, by mode, the sociodemographic characteristics of cell phone owners who completed a follow-up phone survey with those who did not complete the survey. METHODS The study was based on a nationally representative sample of women aged 15 to 49 years who reported cell phone ownership during a household survey in Burkina Faso in 2016. Female cell phone owners were randomized to participate in a computer-assisted telephone interview or hybrid interactive voice response follow-up phone survey 11 months after baseline interviews. Completion of the phone survey was defined as participants responding to more than 50% of questions in the phone survey. We investigated sociodemographic characteristics associated with cell phone survey completion using multivariable logistic regression models, stratifying the analysis by survey mode and by directly comparing computer-assisted telephone interview and hybrid interactive voice response respondents. RESULTS A total of 1766 women were called for the phone survey between November 5 and 17, 2017. In both the computer-assisted telephone interview and hybrid interactive voice response samples, women in urban communities and women with secondary education or higher were more likely to complete the survey than their rural and less-educated counterparts. Compared directly, women who completed the hybrid interactive voice response survey had higher odds of having a secondary education than those who completed computer-assisted telephone interviews (odds ratio 1.7, 95% CI 1.1-2.6). CONCLUSIONS In Burkina Faso, computer-assisted telephone interviews are the preferred method of conducting cell phone surveys owing to less sample distortion and a higher response rate compared with a hybrid interactive voice response survey.

scholarly journals Data Equivalency of an Interactive Voice Response System for Home Assessment of Back Pain and Function

2007 ◽  
Vol 12 (1) ◽  
pp. 23-30 ◽  
Author(s):  
William S Shaw ◽  
Santosh K Verma
Keyword(s):  
Back Pain ◽  
Data Collection ◽  
Interactive Voice Response ◽  
Push Button ◽  
Working Adults ◽  
Acute Back Pain ◽  
And Function ◽  
Lost To Follow Up ◽  
Voice Response

BACKGROUND: Interactive voice response (IVR) systems that collect survey data using automated, push-button telephone responses may be useful to monitor patients’ pain and function at home; however, its equivalency to other data collection methods has not been studied.OBJECTIVES: To study the data equivalency of IVR measurement of pain and function to live telephone interviewing.METHODS: In a prospective cohort study, 547 working adults (66% male) with acute back pain were recruited at an initial outpatient visit and completed telephone assessments one month later to track outcomes of pain, function, treatment helpfulness and return to work. An IVR system was introduced partway through the study (after the first 227 participants) to reduce the staff time necessary to contact participants by telephone during nonworking hours.RESULTS: Of 368 participants who were subsequently recruited and offered the IVR option, 131 (36%) used IVR, 189 (51%) were contacted by a telephone interviewer after no IVR attempt was made within five days, and 48 (13%) were lost to follow-up. Those with lower income were more likely to use IVR. Analysis of outcome measures showed that IVR respondents reported comparatively lower levels of function and less effective treatment, but not after controlling for differences due to the delay in reaching non-IVR users by telephone (mean: 35.4 versus 29.2 days).CONCLUSIONS: The results provided no evidence of information or selection bias associated with IVR use; however, IVR must be supplemented with other data collection options to maintain high response rates.

scholarly journals Effect of airtime incentives on response and cooperation rates in non-communicable disease interactive voice response surveys: randomised controlled trials in Bangladesh and Uganda

2019 ◽  
Vol 4 (5) ◽  
pp. e001604 ◽  
Author(s):  
Dustin G Gibson ◽  
Adaeze C. Wosu ◽  
George William Pariyo ◽  
Saifuddin Ahmed ◽  
Joseph Ali ◽  
...  
Keyword(s):  
Interactive Voice Response ◽  
Communicable Disease ◽  
Response Rates ◽  
Control Group ◽  
Phone Calls ◽  
Significant Difference ◽  
Registration Number ◽  
Randomised Controlled ◽  
Non Communicable Disease ◽  
Voice Response

BackgroundThe global proliferation of mobile phones offers opportunity for improved non-communicable disease (NCD) data collection by interviewing participants using interactive voice response (IVR) surveys. We assessed whether airtime incentives can improve cooperation and response rates for an NCD IVR survey in Bangladesh and Uganda.MethodsParticipants were randomised to three arms: a) no incentive, b) 1X incentive or c) 2X incentive, where X was set to airtime of 50 Bangladesh Taka (US$0.60) and 5000 Ugandan Shillings (UGX; US$1.35). Adults aged 18 years and older who had a working mobile phone were sampled using random digit dialling. The primary outcomes, cooperation and response rates as defined by the American Association of Public Opinion Research, were analysed using log-binomial regression model.ResultsBetween 14 June and 14 July 2017, 440 262 phone calls were made in Bangladesh. The cooperation and response rates were, respectively, 28.8% (353/1227) and 19.2% (580/3016) in control, 39.2% (370/945) and 23.9% (507/2120) in 50 Taka and 40.0% (362/906) and 24.8% (532/2148) in 100 Taka incentive groups. Cooperation and response rates, respectively, were significantly higher in both the 50 Taka (risk ratio (RR) 1.36, 95% CI 1.21 to 1.53) and (RR 1.24, 95% CI 1.12 to 1.38), and 100 Taka groups (RR 1.39, 95% CI 1.23 to 1.56) and (RR 1.29, 95% CI 1.16 to 1.43), as compared with the controls. In Uganda, 174 157 phone calls were made from 26 March to 22 April 2017. The cooperation and response rates were, respectively, 44.7% (377/844) and 35.2% (552/1570) in control, 57.6% (404/701) and 39.3% (508/1293) in 5000 UGX and 58.8% (421/716) and 40.3% (535/1328) in 10 000 UGX groups. Cooperation and response rates were significantly higher, respectively in the 5000 UGX (RR 1.29, 95% CI 1.17 to 1.42) and (RR 1.12, 95% CI 1.02 to 1.23), and 10 000 UGX groups (RR 1.32, 95% CI 1.19 to 1.45) and (RR 1.15, 95% CI 1.04 to 1.26), as compared with the control group.ConclusionIn two diverse settings, the provision of an airtime incentive significantly improved both the cooperation and response rates of an IVR survey, with no significant difference between the two incentive amounts.Trial registration numberNCT03768323.

scholarly journals Automated and Interviewer-Administered Mobile Phone Surveys in Burkina Faso: Sociodemographic Differences Among Female Mobile Phone Survey Respondents and Nonrespondents

10.2196/17891 ◽  
2020 ◽  
Vol 8 (7) ◽  
pp. e17891
Author(s):  
Abigail R Greenleaf ◽  
Aliou Gadiaga ◽  
Yoonjoung Choi ◽  
Georges Guiella ◽  
Shani Turke ◽  
...  
Keyword(s):  
Burkina Faso ◽  
Cell Phone ◽  
Telephone Interview ◽  
Interactive Voice Response ◽  
Computer Assisted ◽  
Computer Assisted Telephone Interview ◽  
Telephone Interviews ◽  
Phone Survey ◽  
Voice Response

Background The remarkable growth of cell phone ownership in low- and middle-income countries has generated significant interest in using cell phones for conducting surveys through computer-assisted telephone interviews, live interviewer-administered surveys, or automated surveys (ie, interactive voice response). Objective This study aimed to compare, by mode, the sociodemographic characteristics of cell phone owners who completed a follow-up phone survey with those who did not complete the survey. Methods The study was based on a nationally representative sample of women aged 15 to 49 years who reported cell phone ownership during a household survey in Burkina Faso in 2016. Female cell phone owners were randomized to participate in a computer-assisted telephone interview or hybrid interactive voice response follow-up phone survey 11 months after baseline interviews. Completion of the phone survey was defined as participants responding to more than 50% of questions in the phone survey. We investigated sociodemographic characteristics associated with cell phone survey completion using multivariable logistic regression models, stratifying the analysis by survey mode and by directly comparing computer-assisted telephone interview and hybrid interactive voice response respondents. Results A total of 1766 women were called for the phone survey between November 5 and 17, 2017. In both the computer-assisted telephone interview and hybrid interactive voice response samples, women in urban communities and women with secondary education or higher were more likely to complete the survey than their rural and less-educated counterparts. Compared directly, women who completed the hybrid interactive voice response survey had higher odds of having a secondary education than those who completed computer-assisted telephone interviews (odds ratio 1.7, 95% CI 1.1-2.6). Conclusions In Burkina Faso, computer-assisted telephone interviews are the preferred method of conducting cell phone surveys owing to less sample distortion and a higher response rate compared with a hybrid interactive voice response survey.

scholarly journals Score equivalence of paper-, tablet-, and interactive voice response system-based versions of PROMIS, PRO-CTCAE, and numerical rating scales among cancer patients

2021 ◽  
Vol 5 (1) ◽  
Author(s):  
Minji K. Lee ◽  
Timothy J. Beebe ◽  
Kathleen J. Yost ◽  
David T. Eton ◽  
Paul J. Novotny ◽  
...  
Keyword(s):  
Data Collection ◽  
Cancer Patients ◽  
Patient Reported Outcomes ◽  
Missing Values ◽  
Rating Scales ◽  
Interactive Voice Response ◽  
Patient Reported ◽  
Numerical Rating ◽  
Equivalence Margin ◽  
Voice Response

Abstract Background The study tests the effects of data collection modes on patient responses associated with the multi-item measures such as Patient-Reported Outcomes Measurement System (PROMIS®), and single-item measures such as Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE), and Numerical Rating Scale (NRS) measures. Methods Adult cancer patients were recruited from five cancer centers and administered measures of anxiety, depression, fatigue, sleep disturbance, pain intensity, pain interference, ability to participate in social roles and activities, global mental and physical health, and physical function. Patients were randomized to complete the measures on paper (595), interactive voice response (IVR, 596) system, or tablet computer (589). We evaluated differential item functioning (DIF) by method of data collection using the R software package, lordif. For constructs that showed no DIF, we concluded equivalence across modes if the equivalence margin, defined as ± 0.20 × pooled SD, completely surrounds 95% confidence intervals (CI's) for difference in mean score. If the 95% CI fell totally outside the equivalence margin, we concluded systematic score difference by modes. If the 95% CI partly overlaps the equivalence margin, we concluded neither equivalence nor difference. Results For all constructs, no DIF of any kind was found for the three modes. The scores on paper and tablet were more comparable than between IVR and other modes but none of the 95% CI’s were completely outside the equivalence margins, in which we established neither equivalence nor difference. Percentages of missing values were comparable for paper and tablet modes. Percentages of missing values were higher for IVR (2.3% to 6.5% depending on measures) compared to paper and tablet modes (0.7% to 3.3% depending on measures and modes), which was attributed to random technical difficulties experienced in some centers. Conclusion Across all mode comparisons, there were some measures with CI’s not completely contained within the margin of small effect. Two visual modes agreed more than visual-auditory pairs. IVR may induce differences in scores unrelated to constructs being measured in comparison with paper and tablet. The users of the surveys should consider using IVR only when paper and computer administration is not feasible.

Sign in / Sign up

Export Citation Format

Share Document