scholarly journals Building on a Sequential Mixed-Mode Research Design in the Monitoring the Future Study

2020 ◽  
Author(s):  
Megan E Patrick ◽  
Mick P Couper ◽  
Bohyun Joy Jang ◽  
Virginia Laetz ◽  
John E Schulenberg ◽  
...  
Keyword(s):  
Mixed Mode ◽  
Survey Design ◽  
Response Rates ◽  
Control Group ◽  
Web Survey ◽  
12Th Grade ◽  
Monitoring The Future ◽  
E Mail ◽  
Mailed Survey ◽  
The Web

Abstract Given the promise of the web push plus e-mail survey design for providing cost-effective and high-quality data (Patrick et al. 2018, 2019) as an alternative to a paper-and-pencil mailed survey design for the longitudinal Monitoring the Future (MTF) study, the current study sought to further enhance the web push condition. The MTF sample is based on US nationally representative samples of 12th grade students surveyed annually. The MTF control group for the current study included participants who completed the in-school baseline survey in the 12th grade and were selected to participate in their first follow-up survey in 2017 via mailed surveys (N = 1,222). A supplementary sample (N = ∼2,450) was assigned to one of the two sequential mixed-mode conditions. Those in condition 1 (N = 1,198), or mail push, were invited to complete mailed surveys and later given a web survey option. Those in condition 2 (N = 1,173), or enhanced web push, were invited to complete a web survey (the same as in the 2014 study, but with the addition of text messages and quick response (QR) codes and the web survey was optimized for mobile devices) and then later given a mailed survey option. Research aims were to examine response rates across conditions, as well as how responses were distributed across mode (paper, web), devices (computer, smartphone, table), and method of accessing the web survey (hand-entered URL, QR code, e-mail link, SMS link). Response rates differed significantly: the MTF control group was 34.2 percent, mail push was 35.4 percent, and enhanced web push was 42.05 percent. The higher response rate in the enhanced web push condition suggests that the additional strategies were effective at bringing in more respondents. Key estimates produced by the enhanced web push condition did not differ from those of the MTF control group.

Response Rates for Mixed-Mode Surveys Using Mail and E-mail/Web

2008 ◽  
Vol 29 (1) ◽  
pp. 99-107 ◽  
Author(s):  
Patrick D. Converse ◽  
Edward W. Wolfe ◽  
Xiaoting Huang ◽  
Frederick L. Oswald
Keyword(s):  
Mixed Mode ◽  
Design Method ◽  
Mail Survey ◽  
Response Rates ◽  
Initial Contact ◽  
Web Survey ◽  
Web Based ◽  
Tailored Design ◽  
E Mail

This study examines response rates for mixed-mode survey implementation involving mail and e-mail/Web components. Using Dillman's Tailored Design Method, 1,500 participants were sent a survey either (a) via mail with a follow-up contact via e-mail that directed them to a Web-based questionnaire or (b) via e-mail that directed them to a Web-based questionnaire with a follow-up contact via mail. Results indicate that these mixed-mode procedures produce moderately high response rates. However, the mail survey tended to be more effective than the e-mail/Web survey, when serving either as the initial contact or as the follow-up contact. These results suggest that survey implementation involving mail followed by e-mail/Web, or even mail-only approaches, may result in larger samples than implementation involving e-mail/Web followed by mail.

Can Phishing Tank Survey Response Rates? Evidence from a Natural Experiment

Field Methods ◽  
2019 ◽  
Vol 31 (4) ◽  
pp. 295-308 ◽  
Author(s):  
Philip S. Brenner
Keyword(s):  
Information Technology ◽  
Private Information ◽  
Natural Experiment ◽  
Response Rates ◽  
Survey Response ◽  
Web Survey ◽  
It Services ◽  
It Department ◽  
University Department ◽  
Mailed Survey

In today’s survey climate, many individuals doubt the legitimacy of survey invitations. Phishing, an Internet-based fraud that tricks users into disclosing private information, has the potential to further erode the perceived legitimacy of e-mailed survey invitations and harm cooperation. However, no study has tested the effect of phishing on response rates. This article reports on a natural experiment examining phishing’s effect on survey response. University faculty and staff received an invitation to participate in an annual web survey on satisfaction with information technology (IT) services followed by a request to participate in a second “survey” ostensibly sent by another university department. However, the second survey invitation was a simulated phishing attack sent by the IT department. Analysis of response rates and the timing of responses from each of the last five years of the legitimate survey suggests that the phishing simulation dramatically reduced response compared to predictions based on previous years.

scholarly journals Timing the Mode Switch in a Sequential Mixed-Mode Survey

2016 ◽  
Vol 35 (2) ◽  
pp. 262-276 ◽  
Author(s):  
James Wagner ◽  
Heather M. Schroeder ◽  
Andrew Piskorowski ◽  
Robert J. Ursano ◽  
Murray B. Stein ◽  
...  
Keyword(s):  
Mixed Mode ◽  
Response Rates ◽  
Design Parameters ◽  
Time Lags ◽  
Negative Consequences ◽  
Mode Switch ◽  
Contact Materials ◽  
Lower Response ◽  
Positive Side ◽  
The Web

Mixed-mode surveys need to determine a number of design parameters that may have a strong influence on costs and errors. In a sequential mixed-mode design with web followed by telephone, one of these decisions is when to switch modes. The web mode is relatively inexpensive but produces lower response rates. The telephone mode complements the web mode in that it is relatively expensive but produces higher response rates. Among the potential negative consequences, delaying the switch from web to telephone may lead to lower response rates if the effectiveness of the prenotification contact materials is reduced by longer time lags, or if the additional e-mail reminders to complete the web survey annoy the sampled person. On the positive side, delaying the switch may decrease the costs of the survey. We evaluate these costs and errors by experimentally testing four different timings (1, 2, 3, or 4 weeks) for the mode switch in a web–telephone survey. This experiment was conducted on the fourth wave of a longitudinal study of the mental health of soldiers in the U.S. Army. We find that the different timings of the switch in the range of 1–4 weeks do not produce differences in final response rates or key estimates but longer delays before switching do lead to lower costs.

scholarly journals The Impact of Pictures on Best-Worst Scaling in Web Surveys

2012 ◽  
Vol 2 (3) ◽  
pp. 79-93
Author(s):  
Marinică Bărbulescu ◽  
Alexandru Cernat
Keyword(s):  
Experimental Design ◽  
Response Rates ◽  
Control Group ◽  
Web Surveys ◽  
Online Surveys ◽  
Best Worst Scaling ◽  
Influence Measurement ◽  
The Impact ◽  
Experimental Group ◽  
The Web

Abstract: Motivation and burden are two of the most important aspects that influence response rates and dropouts in online surveys. As a result, we focus our analyses on how pictures and Best Worst Scaling (BWS), two solutions for each problem, interact in the Web medium. We use an experimental design that compares a BWS with pictures, the experimental group, and BWS without pictures, the control group. Results show that pictures influence measurement of BWS in six out of 16 items. We also observe that Couper's (2001) conclusion that concordant text and images have an accentuation effect while a discordant relationship between the two has an interference impact is partly true in our data. Eight out of the 16 items are at least partially influenced by the concordant/discordant variable while four fully respect this model. We conclude by discussing the impact of our findings and its limitations.

scholarly journals Web mode as part of mixed-mode surveys of the general population

2015 ◽  
Vol 12 (2) ◽  
Author(s):  
Nejc Berzelak ◽  
Vasja Vehovar ◽  
Katja Lozar Manfreda
Keyword(s):  
General Population ◽  
Mixed Mode ◽  
Response Rate ◽  
Cost Savings ◽  
Response Rates ◽  
Web Surveys ◽  
Face To Face ◽  
Survey Modes ◽  
Survey Designs ◽  
The Web

Lower data collection costs make web surveys a promising alternative to conventional face-to-face and telephone surveys. A transition to the new mode has already been widely initiated in commercial research, but web surveys remains limited in academic and official research projects that typically require probability samples and high response rates. Various design approaches for coping with the problems of sampling frames, incomplete Internet use, and nonresponse in web surveys have been proposed. Mixed-mode designs and incentives are two common strategies to reach Internet non-users and increase the response rates in web surveys. However, such survey designs can substantially increase the costs, the complexity of administration and the possibility of uncontrolled measurement effects. This paper presents and demonstrates an approach to the evaluation of various survey designs with simultaneous consideration of the errors and costs. It focuses on the designs involving the web mode and discusses their potential to replace traditional modes for probability surveys of the general population. The main idea of this approach is that part of the cost savings enabled by the web mode can be allocated to incentives and complementary survey modes to compensate for the Internet non-coverage and the higher nonresponse. The described approach is demonstrated in an experimental case study that compares the performance of mixed-mode designs with the web mode and prepaid cash incentive with that of an official survey conducted using the face-to-face and telephone modes. The results show that the mixed-mode designs with the web mode and incentives can greatly increase the response rate, which even surpasses that of the conventional survey modes, but still offer substantial cost savings. However, the results also show that higher response rate does not necessary translate to higher data quality, especially when the main aim is to obtain estimates that are highly comparable with those of the reference survey.

Increasing Response Rates among Coaches: The Role of Prenotification Methods

2002 ◽  
Vol 16 (3) ◽  
pp. 230-238 ◽  
Author(s):  
Aubrey Kent ◽  
Brian Turner
Keyword(s):  
Significant Influence ◽  
Response Rate ◽  
Cost Effective ◽  
Response Rates ◽  
Return Rate ◽  
Head Coaches ◽  
The Internet ◽  
Control Group ◽  
E Mail

This study determined whether, in a population of intercollegiate head coaches, prenotification had a significant influence on the return rate of mailed questionnaires. Acknowledging the growing use of e-mail and the Internet for survey distribution, the current study evaluated the effectiveness of e-mail as a prenotification technique, Response rates of Intercollegiate Head Coaches to mailed questionnaires from two separate samples were analyzed. Comparisons were made based upon the categorization of coaches into groups of prenotification by e-mail, formal letter, and a non-prenotified control group. Results indicated that prenotification of the survey recipients significantly increased response rates, with the group receiving e-mail prenotification having the highest response rate among the three groups. In addition to being cost effective for researchers, e-mail prenotification was an effective way to increase both the number and variety of contacts with survey recipients.

scholarly journals Effects of Incentives and Prenotification on Response Rates and Costs in a National Web Survey of Physicians

2011 ◽  
Vol 34 (4) ◽  
pp. 434-447 ◽  
Author(s):  
Jennifer Dykema ◽  
John Stevenson ◽  
Brendan Day ◽  
Sherrill L. Sellers ◽  
Vence L. Bonham
Keyword(s):  
American Medical Association ◽  
Medical Association ◽  
American Medical ◽  
Cost Effective ◽  
Response Rates ◽  
Nonresponse Bias ◽  
Web Survey ◽  
General Internists ◽  
Mail Surveys ◽  
The Web

Little is known about what strategies are cost-effective in increasing participation among physicians in surveys that are conducted exclusively via the web. To assess the effects of incentives and prenotification on response rates and costs, general internists ( N = 3,550) were randomly selected from the American Medical Association (AMA) Masterfile and assigned to experimental groups that varied in the amount of a promised incentive (none, entry into a $200 lottery, $50, or $100) and prenotification (none, prenotification letter only, or prenotification letter containing a $2 preincentive). Results indicated that the response rates were highest in the groups promised $100 and $50, respectively. While the postal prenotification letter increased response rates, the inclusion of a small token $2 preincentive had no effect on participation. Further, unlike mail surveys of physicians, the $2 preincentive was not cost-effective. Among physicians, larger promised incentives of $50 or $100 are more effective than a nominal preincentive in increasing participation in a web-only survey. Consistent with prior research, there was little evidence of nonresponse bias among the experimental groups.

Sign in / Sign up

Export Citation Format

Share Document