scholarly journals The Effect of Survey Mode on Data Quality: Disentangling Nonresponse and Measurement Error Bias

2019 ◽  
Vol 35 (1) ◽  
pp. 93-115 ◽  
Author(s):  
Barbara Felderer ◽  
Antje Kirchner ◽  
Frauke Kreuter
Keyword(s):  
Measurement Error ◽  
Data Quality ◽  
Nonresponse Bias ◽  
Web Survey ◽  
Sensitive Questions ◽  
Survey Mode ◽  
Survey Statistics ◽  
Survey Administration ◽  
Error Bias ◽  
The Web

Abstract More and more surveys are conducted online. While web surveys are generally cheaper and tend to have lower measurement error in comparison to other survey modes, especially for sensitive questions, potential advantages might be offset by larger nonresponse bias. This article compares the data quality in a web survey administration to another common mode of survey administration, the telephone. The unique feature of this study is the availability of administrative records for all sampled individuals in combination with a random assignment of survey mode. This specific design allows us to investigate and compare potential bias in survey statistics due to 1) nonresponse error, 2) measurement error, and 3) combined bias of these two error sources and hence, an overall assessment of data quality for two common modes of survey administration, telephone and web. Our results show that overall mean estimates on the web are more biased compared to the telephone mode. Nonresponse and measurement bias tend to reinforce each other in both modes, with nonresponse bias being somewhat more pronounced in the web mode. While measurement error bias tends to be smaller in the web survey implementation, interestingly, our results also show that the web does not consistently outperform the telephone mode for sensitive questions.

scholarly journals Surveying Ourselves

2010 ◽  
Vol 34 (4) ◽  
pp. 448-463 ◽  
Author(s):  
Kristen A. Matteson ◽  
Britta L. Anderson ◽  
Stephanie B. Pinto ◽  
Vrishali Lopes ◽  
Jay Schulkin ◽  
...  
Keyword(s):  
Data Collection ◽  
Data Quality ◽  
Private Practice ◽  
Mixed Mode ◽  
National Sample ◽  
Survey Mode ◽  
Web Based ◽  
Minimal Reduction ◽  
The Web

A survey was distributed, using a sequential mixed-mode approach, to a national sample of obstetrician–gynecologists. Differences between responses to the web-based mode and the on-paper mode were compared to determine if there were systematic differences between respondents. Only two differences in respondents between the two modes were identified. University-based physicians were more likely to complete the web-based mode than private practice physicians. Mail respondents reported a greater volume of endometrial ablations compared to online respondents. The web-based mode had better data quality than the paper-based mailed mode in terms of less missing and inappropriate responses. Together, these findings suggest that, although a few differences were identified, the web-based survey mode attained adequate representativeness and improved data quality. Given the metrics examined for this study, exclusive use of web-based data collection may be appropriate for physician surveys with a minimal reduction in sample coverage and without a reduction in data quality.

scholarly journals Do Sequential Mixed-Mode Surveys Decrease Nonresponse Bias, Measurement Error Bias, and Total Bias? An Experimental Study

2019 ◽  
Vol 7 (4) ◽  
pp. 545-571 ◽  
Author(s):  
Joseph W Sakshaug ◽  
Alexandru Cernat ◽  
Trivellore E Raghunathan
Keyword(s):  
Measurement Error ◽  
Data Collection ◽  
Mixed Mode ◽  
Traffic Accidents ◽  
Nonresponse Bias ◽  
Bias Estimation ◽  
Design Decisions ◽  
Second Mode ◽  
Error Bias

Abstract Mixing multiple modes of survey data collection has become standard practice in survey research. Mixed-mode surveys are faced with a slew of design decisions regarding which types of modes to administer and which sequence to administer them in. Such decisions are largely based on administrative objectives, such as minimizing costs and maximizing response rates. However, just as important to these mixed-mode decisions is their impact on nonresponse bias, measurement error bias, and total bias, which are understudied issues in the mixed-mode literature. In this article, we report on a sequential mixed-mode experiment of young adult drivers randomized to one of two mode sequences: an interviewer-administered (telephone) mode with self-administered (mail) follow-up, or the reverse sequence. Using a mix of direct and indirect bias estimation strategies, we find support for the notion that implementing a second mode of data collection can reduce nonresponse and measurement error bias, but the sequence in which the modes are administered makes a difference: the mail-telephone sequence minimizes bias to a greater extent than the telephone-mail sequence, relative to the starting mode and overall. However, a backfiring effect was found: despite reducing both nonresponse and measurement error bias, switching from mail to telephone increased the total bias in a key estimate of traffic accidents. A discussion of these findings and their implications for survey practice are provided in conclusion.

A Typology of Web Survey Paradata for Assessing Total Survey Error

2018 ◽  
Vol 37 (2) ◽  
pp. 196-213 ◽  
Author(s):  
Colleen A. McClain ◽  
Mick P. Couper ◽  
Andrew L. Hupp ◽  
Florian Keusch ◽  
Gregg Peterson ◽  
...  
Keyword(s):  
Measurement Error ◽  
Case Studies ◽  
Web Survey ◽  
Web Surveys ◽  
New Developments ◽  
Survey Error ◽  
Open Questions ◽  
Error Paradigm ◽  
The Web ◽  
Do So

This article reviews the existing literature on the collection of paradata in web surveys and extends the research in this area beyond the commonly studied measurement error problem to paradata that can be collected for managing and mitigating other important sources of error. To do so, and in keeping with the nature of paradata as process-oriented, we develop a typology of web survey paradata that incorporates information from all steps in the web survey process. We first define web survey paradata and describe general phases of paradata that run parallel to the steps in fielding a typical web survey. Within each phase, we enumerate several errors within the total survey error paradigm that can be examined with paradata, discussing case studies and motivating examples that illustrate innovative uses of paradata across the web survey process. We conclude with a discussion of open questions and opportunities for further work in this area. Overall, we develop this typology keeping technological advancements at the center of our discussion, but with flexibility to continuously incorporate new developments and trends in both technology and study design. Our typology encourages researchers to think about paradata as tools that can be used to investigate a broader range of outcomes than previously studied.

scholarly journals Effects of Incentives and Prenotification on Response Rates and Costs in a National Web Survey of Physicians

2011 ◽  
Vol 34 (4) ◽  
pp. 434-447 ◽  
Author(s):  
Jennifer Dykema ◽  
John Stevenson ◽  
Brendan Day ◽  
Sherrill L. Sellers ◽  
Vence L. Bonham
Keyword(s):  
American Medical Association ◽  
Medical Association ◽  
American Medical ◽  
Cost Effective ◽  
Response Rates ◽  
Nonresponse Bias ◽  
Web Survey ◽  
General Internists ◽  
Mail Surveys ◽  
The Web

Little is known about what strategies are cost-effective in increasing participation among physicians in surveys that are conducted exclusively via the web. To assess the effects of incentives and prenotification on response rates and costs, general internists ( N = 3,550) were randomly selected from the American Medical Association (AMA) Masterfile and assigned to experimental groups that varied in the amount of a promised incentive (none, entry into a $200 lottery, $50, or $100) and prenotification (none, prenotification letter only, or prenotification letter containing a $2 preincentive). Results indicated that the response rates were highest in the groups promised $100 and $50, respectively. While the postal prenotification letter increased response rates, the inclusion of a small token $2 preincentive had no effect on participation. Further, unlike mail surveys of physicians, the $2 preincentive was not cost-effective. Among physicians, larger promised incentives of $50 or $100 are more effective than a nominal preincentive in increasing participation in a web-only survey. Consistent with prior research, there was little evidence of nonresponse bias among the experimental groups.

scholarly journals Paper, E-mail, or Both? Effects of Contact Mode on Participation in a Web Survey of Establishments

2018 ◽  
Vol 37 (6) ◽  
pp. 750-765 ◽  
Author(s):  
Joseph W. Sakshaug ◽  
Basha Vicari ◽  
Mick P. Couper
Keyword(s):  
Response Rate ◽  
Population Based ◽  
High Response Rate ◽  
Nonresponse Bias ◽  
Web Survey ◽  
Participation Rates ◽  
Web Surveys ◽  
Contact Mode ◽  
Mail Address ◽  
E Mail

Identifying strategies that maximize participation rates in population-based web surveys is of critical interest to survey researchers. While much of this interest has focused on surveys of persons and households, there is a growing interest in surveys of establishments. However, there is a lack of experimental evidence on strategies for optimizing participation rates in web surveys of establishments. To address this research gap, we conducted a contact mode experiment in which establishments selected to participate in a web survey were randomized to receive the survey invitation with login details and subsequent reminder using a fully crossed sequence of paper and e-mail contacts. We find that a paper invitation followed by a paper reminder achieves the highest response rate and smallest aggregate nonresponse bias across all-possible paper/e-mail contact sequences, but a close runner-up was the e-mail invitation and paper reminder sequence which achieved a similarly high response rate and low aggregate nonresponse bias at about half the per-respondent cost. Following up undeliverable e-mail invitations with supplementary paper contacts yielded further reductions in nonresponse bias and costs. Finally, for establishments without an available e-mail address, we show that enclosing an e-mail address request form with a prenotification letter is not effective from a response rate, nonresponse bias, and cost perspective.

scholarly journals Going web or staying paper? The use of web-surveys among older people

2020 ◽  
Author(s):  
Susanne Kelfve ◽  
Marie Kivi ◽  
Boo Johansson ◽  
Magnus Lindwall
Keyword(s):  
Older People ◽  
Good Alternative ◽  
Web Survey ◽  
Response Patterns ◽  
Older Individuals ◽  
Web Surveys ◽  
Paper Questionnaire ◽  
Age Range ◽  
Web Questionnaire ◽  
The Web

Abstract Background: Web-surveys are increasingly used in population studies. Yet, web-surveys targeting older individuals are still uncommon for various reasons. However, with younger cohorts approaching older age, the potentials for web-surveys among older people might be improved. In this study, we investigated response patterns in a web-survey targeting older adults and the potential importance of offering a paper questionnaire as an alternative to the web questionnaire. Methods: We analyzed data from three waves of a retirement study, in which a web-push methodology was used and a paper questionnaire was offered as an alternative to the web questionnaire in the last reminder. We mapped the response patterns, compared web- and paper respondents and compared different key outcomes resulting from the sample with and without the paper respondents, both at baseline and after two follow-ups.Results: Paper-respondents, that is, those that did not answer until they got a paper questionnaire with the last reminder, were more likely to be female, retired, single, and to report a lower level of education, higher levels of depression and lower self-reported health, compared to web-respondents. The association between retirement status and depression was only present among web-respondents. The differences between web and paper respondents were stronger in the longitudinal sample (after two follow-ups) than at baseline.Conclusions: We conclude that a web-survey might be a feasible and good alternative in surveys targeting people in the retirement age range. However, without offering a paper questionnaire, a small but important group will likely be missing with potential biased estimates as the result.

Measuring Wages Worldwide

2013 ◽  
pp. 100-119 ◽  
Author(s):  
Stephanie Steinmetz ◽  
Damian Raess ◽  
Kea Tijdens ◽  
Pablo de Pedraza
Keyword(s):  
Developing Countries ◽  
Data Collection ◽  
Demographic Variables ◽  
Web Survey ◽  
Data Collection Tool ◽  
Individual Level ◽  
Estimated Parameters ◽  
The Difference ◽  
The Web ◽  
Post Stratification

This chapter discusses the potentials and constraints of using a volunteer Web survey as a worldwide data collection tool for wages. It provides a detailed description of the bias related to individual-level wages and core socio-demographic and employment-related variables across selected developed and developing countries and evaluates the efficiency of post-stratification weights in adjusting these biases. The results confirm that Web samples are particularly attractive to younger persons, part-timers, and persons working in non-manual occupations. This can be observed across countries, although the strength of the bias differs between them. With respect to the efficiency of post-stratification weights, the results are inconclusive. Whereas it is advisable to implement weights for descriptive purposes of socio-demographic variables, the contrary holds in case of wages. Additionally, weights can have the opposite effect by (moderately) increasing the difference in the estimated parameters between the reference and the Web sample.

Sign in / Sign up

Export Citation Format

Share Document