A Typology of Web Survey Paradata for Assessing Total Survey Error

2018 ◽  
Vol 37 (2) ◽  
pp. 196-213 ◽  
Author(s):  
Colleen A. McClain ◽  
Mick P. Couper ◽  
Andrew L. Hupp ◽  
Florian Keusch ◽  
Gregg Peterson ◽  
...  
Keyword(s):  
Measurement Error ◽  
Case Studies ◽  
Web Survey ◽  
Web Surveys ◽  
New Developments ◽  
Survey Error ◽  
Open Questions ◽  
Error Paradigm ◽  
The Web ◽  
Do So

This article reviews the existing literature on the collection of paradata in web surveys and extends the research in this area beyond the commonly studied measurement error problem to paradata that can be collected for managing and mitigating other important sources of error. To do so, and in keeping with the nature of paradata as process-oriented, we develop a typology of web survey paradata that incorporates information from all steps in the web survey process. We first define web survey paradata and describe general phases of paradata that run parallel to the steps in fielding a typical web survey. Within each phase, we enumerate several errors within the total survey error paradigm that can be examined with paradata, discussing case studies and motivating examples that illustrate innovative uses of paradata across the web survey process. We conclude with a discussion of open questions and opportunities for further work in this area. Overall, we develop this typology keeping technological advancements at the center of our discussion, but with flexibility to continuously incorporate new developments and trends in both technology and study design. Our typology encourages researchers to think about paradata as tools that can be used to investigate a broader range of outcomes than previously studied.

scholarly journals Going web or staying paper? The use of web-surveys among older people

2020 ◽  
Author(s):  
Susanne Kelfve ◽  
Marie Kivi ◽  
Boo Johansson ◽  
Magnus Lindwall
Keyword(s):  
Older People ◽  
Good Alternative ◽  
Web Survey ◽  
Response Patterns ◽  
Older Individuals ◽  
Web Surveys ◽  
Paper Questionnaire ◽  
Age Range ◽  
Web Questionnaire ◽  
The Web

Abstract Background: Web-surveys are increasingly used in population studies. Yet, web-surveys targeting older individuals are still uncommon for various reasons. However, with younger cohorts approaching older age, the potentials for web-surveys among older people might be improved. In this study, we investigated response patterns in a web-survey targeting older adults and the potential importance of offering a paper questionnaire as an alternative to the web questionnaire. Methods: We analyzed data from three waves of a retirement study, in which a web-push methodology was used and a paper questionnaire was offered as an alternative to the web questionnaire in the last reminder. We mapped the response patterns, compared web- and paper respondents and compared different key outcomes resulting from the sample with and without the paper respondents, both at baseline and after two follow-ups.Results: Paper-respondents, that is, those that did not answer until they got a paper questionnaire with the last reminder, were more likely to be female, retired, single, and to report a lower level of education, higher levels of depression and lower self-reported health, compared to web-respondents. The association between retirement status and depression was only present among web-respondents. The differences between web and paper respondents were stronger in the longitudinal sample (after two follow-ups) than at baseline.Conclusions: We conclude that a web-survey might be a feasible and good alternative in surveys targeting people in the retirement age range. However, without offering a paper questionnaire, a small but important group will likely be missing with potential biased estimates as the result.

scholarly journals Going web or staying paper? The use of web-surveys among older people

2020 ◽  
Author(s):  
Susanne Kelfve ◽  
Marie Kivi ◽  
Boo Johansson ◽  
Magnus Lindwall
Keyword(s):  
Older People ◽  
Good Alternative ◽  
Web Survey ◽  
Response Patterns ◽  
Older Individuals ◽  
Web Surveys ◽  
Paper Questionnaire ◽  
Low Educated ◽  
Web Questionnaire ◽  
The Web

Abstract Background: Web-surveys are increasingly used in population studies. Yet, web-surveys targeting older individuals are still uncommon for various reasons. However, with younger cohorts approaching older age, the potentials for web-surveys among older people might be improved. In this study, we investigated response patterns in a web-survey targeting older adults and the potential importance of offering a paper questionnaire as an alternative to the web questionnaire. Methods: We analyzed data from three waves of a retirement study, in which a web-push methodology was used and a paper questionnaire was offered as an alternative to the web questionnaire in the last reminder. We mapped the response patterns, compared web- and paper respondents and compared different key outcomes resulting from the sample with and without the paper respondents, both at baseline and after two follow-ups. Results: Paper-respondents, that is, those that did not answer until they got a paper questionnaire with the last reminder, were more likely to be women, low educated, fully retired, non-married and reported higher depression and lower self-rated health, compared with web-respondents. The association between retirement status and depression was only present among web-respondents. The differences between web and paper respondents were stronger in the longitudinal sample (after two follow-ups) than at baseline. Conclusions: We conclude that a web survey might be a feasible and good alternative in surveys targeting people in the retirement age range. However, without offering a paper questionnaire, a small but important group will likely be missing with potential biased estimates as the result.

scholarly journals Going web or staying paper? The use of web-surveys among older people

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Susanne Kelfve ◽  
Marie Kivi ◽  
Boo Johansson ◽  
Magnus Lindwall
Keyword(s):  
Older People ◽  
Good Alternative ◽  
Web Survey ◽  
Response Patterns ◽  
Older Individuals ◽  
Web Surveys ◽  
Paper Questionnaire ◽  
Age Range ◽  
Web Questionnaire ◽  
The Web

Abstract Background Web-surveys are increasingly used in population studies. Yet, web-surveys targeting older individuals are still uncommon for various reasons. However, with younger cohorts approaching older age, the potentials for web-surveys among older people might be improved. In this study, we investigated response patterns in a web-survey targeting older adults and the potential importance of offering a paper-questionnaire as an alternative to the web-questionnaire. Methods We analyzed data from three waves of a retirement study, in which a web-push methodology was used and a paper questionnaire was offered as an alternative to the web questionnaire in the last reminder. We mapped the response patterns, compared web- and paper respondents and compared different key outcomes resulting from the sample with and without the paper respondents, both at baseline and after two follow-ups. Results Paper-respondents, that is, those that did not answer until they got a paper questionnaire with the last reminder, were more likely to be female, retired, single, and to report a lower level of education, higher levels of depression and lower self-reported health, compared to web-respondents. The association between retirement status and depression was only present among web-respondents. The differences between web and paper respondents were stronger in the longitudinal sample (after two follow-ups) than at baseline. Conclusions We conclude that a web-survey might be a feasible and good alternative in surveys targeting people in the retirement age range. However, without offering a paper-questionnaire, a small but important group will likely be missing with potential biased estimates as the result.

scholarly journals Going web or staying paper? The use of web-surveys among older people

2020 ◽  
Author(s):  
Susanne Kelfve ◽  
Marie Kivi ◽  
Boo Johansson ◽  
Magnus Lindwall
Keyword(s):  
Older People ◽  
Good Alternative ◽  
Web Survey ◽  
Response Patterns ◽  
Older Individuals ◽  
Web Surveys ◽  
Paper Questionnaire ◽  
Low Educated ◽  
Web Questionnaire ◽  
The Web

Abstract Background: Many surveys today use a web-survey as an alternative to more traditional survey modes. Yet, web-surveys targeting older individuals are still uncommon for various reasons. However, with younger cohorts approaching older age, the potentials for web-surveys among older people might be improved. In this study, we investigated response patterns in a web-survey targeting older adults and the potential importance of offering a paper questionnaire as an alternative to the web questionnaire. Methods: We analyzed data from three waves of a web-based retirement study, in which a paper questionnaire was offered as an alternative to the web questionnaire in the last reminder. We mapped the response patterns, compared web- and paper respondents and compared different key outcomes resulting from the sample with and without the paper respondents, both at baseline and after two follow-ups.Results: Paper-respondents, that is, those that did not answer until they got a paper questionnaire with the last reminder, were more likely to be women, low educated, fully retired, non-married and reported higher depression and lower self-rated health, compared with web-respondents. The association between retirement status and depression was only present among web-respondents. The differences between web and paper respondents were stronger in the longitudinal sample (after two follow-ups) than at baseline.Conclusions: We conclude that a web survey might be a feasible and good alternative in surveys targeting people in the retirement age range. However, without offering a paper questionnaire, a small but important group will likely be missing with potential biased estimates as the result.

scholarly journals Sample Representation and Substantive Outcomes Using Web With and Without Incentives Compared to Telephone in an Election Survey

2016 ◽  
Vol 32 (1) ◽  
pp. 165-186 ◽  
Author(s):  
Oliver Lipps ◽  
Nicolas Pekari
Keyword(s):  
Computer Literacy ◽  
Voting Behavior ◽  
Web Survey ◽  
Completion Rates ◽  
Web Surveys ◽  
Unit Costs ◽  
Variable Bias ◽  
Sample Representation ◽  
Different Levels ◽  
The Web

Abstract The objective of this article is to understand how the change of mode from telephone to web affects data quality in terms of sample representation and substantive variable bias. To this end, an experiment, consisting of a web survey with and without a prepaid incentive, was conducted alongside the telephone Swiss election survey. All three designs used identical questionnaires and probability samples drawn from a national register of individuals. First, our findings show that differences in completion rates mostly reflect different levels of coverage in the two modes. Second, incentives in the web survey strongly increase completion rates of all person groups, with the exception of people without Internet access or limited computer literacy. Third, we find voting behavior to be much closer to official figures in the web with the incentive version compared to the two other designs. However, this is partly due to the different sociodemographic compositions of the samples. Other substantive results suggest that the incentive version includes harder-to-reach respondents. Unit costs are much lower in the two web designs compared to the telephone, including when a relatively high incentive is used. We conclude that in countries with high Internet penetration rates such as Switzerland, web surveys are already likely to be highly competitive.

scholarly journals Going web or staying paper? The use of web-surveys among older people

2020 ◽  
Author(s):  
Susanne Kelfve ◽  
Marie Kivi ◽  
Boo Johansson ◽  
Magnus Lindwall
Keyword(s):  
Older People ◽  
Good Alternative ◽  
Web Survey ◽  
Response Patterns ◽  
Older Individuals ◽  
Web Surveys ◽  
Paper Questionnaire ◽  
Low Educated ◽  
Age Range ◽  
The Web

Abstract Background Many surveys today use a web-survey as an alternative to more traditional survey modes. Yet, web-surveys targeting older individuals are still uncommon for various reasons. However, with younger cohorts approaching older age, the potentials for web-surveys among older people might be improved. In this study, we investigated response patterns in a web-survey targeting older adults and the potential importance of offering a paper questionnaire as an alternative to the web questionnaire.Methods We analyzed data from three waves of a web-based retirement study, in which a paper questionnaire was offered as an alternative to the web questionnaire in the last reminder. We mapped the response patterns, compared web- and paper respondents and compared different key outcomes resulting from the sample with and without the paper respondents, both at baseline and after two follow-ups.Results Paper-respondents were more likely to be women, low educated, fully retired, non-married and reported higher depression and lower self-rated health, compared with web-respondents. The association between retirement status and depression was only present among web-respondents. The differences between web and paper respondents were stronger in the longitudinal sample (after two follow-ups) than at baseline.Conclusions We conclude that a web survey might be a feasible and good alternative in surveys targeting people in the retirement age range. However, without offering a paper questionnaire, a small but important group will likely be missing with potential biased estimates as the result.

Improving the Use of Voice Recording in a Smartphone Survey

2019 ◽  
pp. 089443931988870
Author(s):  
Melanie Revilla ◽  
Mick P. Couper
Keyword(s):  
Data Quality ◽  
Web Survey ◽  
Item Nonresponse ◽  
Web Surveys ◽  
Promising Tool ◽  
Tool Evaluation ◽  
Open Questions ◽  
The Voice ◽  
Voice Recording

More and more respondents are answering web surveys using mobile devices. Mobile respondents tend to provide shorter responses to open questions than PC respondents. Using voice recording to answer open-ended questions could increase data quality and help engage groups usually underrepresented in web surveys. Revilla, Couper, Bosch, and Asensio showed that in particular the use of voice recording still presents many challenges, even if it could be a promising tool. This article reports results from a follow-up experiment in which the main goals were to (1) test whether different instructions on how to use the voice recording tool reduce technical and understanding problems, and thereby reduce item nonresponse while preserving data quality and the evaluation of the tool; (2) test whether nonresponse due to context can be reduced by using a filter question, and how this affects data quality and the tool evaluation; and (3) understand which factors affect nonresponse to open-ended questions using voice recording, and if these factors also affect data quality and the evaluation of the tool. The experiment was implemented within a smartphone web survey in Spain focused on Android devices. The results suggest that different instructions did not affect nonresponse to the open questions and had little effect on data quality for those who did answer. Introducing a filter to ensure that people were in a setting that permits voice recording seems useful. Despite efforts to reduce problems, a substantial proportion of respondents are still unwilling or unable to answer open questions using voice recording.

scholarly journals The Effect of Survey Mode on Data Quality: Disentangling Nonresponse and Measurement Error Bias

2019 ◽  
Vol 35 (1) ◽  
pp. 93-115 ◽  
Author(s):  
Barbara Felderer ◽  
Antje Kirchner ◽  
Frauke Kreuter
Keyword(s):  
Measurement Error ◽  
Data Quality ◽  
Nonresponse Bias ◽  
Web Survey ◽  
Sensitive Questions ◽  
Survey Mode ◽  
Survey Statistics ◽  
Survey Administration ◽  
Error Bias ◽  
The Web

Abstract More and more surveys are conducted online. While web surveys are generally cheaper and tend to have lower measurement error in comparison to other survey modes, especially for sensitive questions, potential advantages might be offset by larger nonresponse bias. This article compares the data quality in a web survey administration to another common mode of survey administration, the telephone. The unique feature of this study is the availability of administrative records for all sampled individuals in combination with a random assignment of survey mode. This specific design allows us to investigate and compare potential bias in survey statistics due to 1) nonresponse error, 2) measurement error, and 3) combined bias of these two error sources and hence, an overall assessment of data quality for two common modes of survey administration, telephone and web. Our results show that overall mean estimates on the web are more biased compared to the telephone mode. Nonresponse and measurement bias tend to reinforce each other in both modes, with nonresponse bias being somewhat more pronounced in the web mode. While measurement error bias tends to be smaller in the web survey implementation, interestingly, our results also show that the web does not consistently outperform the telephone mode for sensitive questions.

An Automated Refusal Conversion Strategy for Web Surveys

Field Methods ◽  
2019 ◽  
Vol 31 (4) ◽  
pp. 309-327
Author(s):  
Taylor Lewis ◽  
Mark Gorsak ◽  
Naomi Yount
Keyword(s):  
Primary Reason ◽  
Web Surveys ◽  
Web Based ◽  
Federal Employee ◽  
Opt Out ◽  
The Individual ◽  
Complementary Subset ◽  
E Mail ◽  
The Web ◽  
Do So

This article presents results from an experiment conducted during the web-based 2017 Federal Employee Viewpoint Survey (FEVS) to evaluate an automated refusal conversion strategy whereby a sample of individuals was given the opportunity to opt out from the survey and stop receiving additional e-mail reminders. Before being added to the unsubscribe list, however, the individual was asked to cite the primary reason for choosing not to take the FEVS. A randomly assigned subset was given a last-moment appeal, tailored to the reason provided, at which point the individual could either confirm desiring to opt out or navigate to the start of the survey. Because the complementary subset did not receive the appeal, we are able to report on the efficacy of the strategy in convincing individuals who may not have been initially inclined to participate to do so.

What’s Next?

2011 ◽  
Vol 23 (1) ◽  
pp. 60-63 ◽  
Author(s):  
Peter Vorderer
Keyword(s):  
Cultural Differences ◽  
Factor Model ◽  
New Developments ◽  
Theoretical Concepts ◽  
Open Questions ◽  
Entertainment Products ◽  
Entertainment Theory

This paper points to new developments in the context of entertainment theory. Starting from a background of well-established theories that have been proposed and elaborated mainly by Zillmann and his collaborators since the 1980s, a new two-factor model of entertainment is introduced. This model encompasses “enjoyment” and “appreciation” as two independent factors. In addition, several open questions regarding cultural differences in humans’ responses to entertainment products or the usefulness of various theoretical concepts like “presence,” “identification,” or “transportation” are also discussed. Finally, the question of why media users are seeking entertainment is brought to the forefront, and a possibly relevant need such as the “search for meaningfulness” is mentioned as a possible major candidate for such an explanation.

Sign in / Sign up

Export Citation Format

Share Document