Challenges of Virtual RDS for Recruitment of Sexual Minority Women for a Behavioral Health Study

Respondent-driven sampling (RDS) is an approach commonly used to recruit nonprobability samples of rare and hard-to-find populations. The purpose of this study was to explore the utility of phone- and web-based RDS methodology to sample sexual minority women (SMW) for participation in a telephone survey. Key features included (i) utilizing a national probability survey sample to select seeds; (ii) web-based recruitment with emailed coupons; and (iii) virtual processes for orienting, screening, and scheduling potential participants for computer-assisted telephone interviews. Rather than resulting in a large diverse sample of SMW, only a small group of randomly selected women completed the survey and agreed to recruit their peers, and very few women recruited even one participant. Only seeds from the most recent of two waves of the probability study generated new SMW recruits. Three RDS attempts to recruit SMW over several years and findings from brief qualitative interviews revealed four key challenges to successful phone- and web-based RDS with this population. First, population-based sampling precludes sampling based on participant characteristics that are often used in RDS. Second, methods that distance prospective participants from the research team may impede development of relationships, investment in the study, and motivation to participate. Third, recruitment for telephone surveys may be impeded by multiple burdens on seeds and recruits (e.g., survey length, understanding the study and RDS process). Finally, many seeds from a population-based sample may be needed, which is not generally feasible when working with a limited pool of potential seeds. This method may yield short recruitment chains, which would not meet key RDS assumptions for approximation of a probability sample. In conclusion, potential challenges to using RDS in studies with SMW, particularly those using virtual approaches, should be considered.

Careless Responding and Insufficient Effort Responding

Careless responding, also known as insufficient effort responding, refers to survey/test respondents providing random, inattentive, or inconsistent answers to question items due to lack of effort in conforming to instructions, interpreting items, and/or providing accurate responses. Researchers often use these two terms interchangeably to describe deviant behaviors in survey/test responding that threaten data quality. Careless responding threatens the validity of research findings by bringing in random and systematic errors. Specifically, careless responding can reduce measurement reliability, while under specific circumstances it can also inflate the substantive relations between variables. Numerous factors can explain why careless responding happens (or does not happen), such as individual difference characteristics (e.g., conscientiousness), survey characteristics (e.g., survey length), and transient psychological states (e.g., positive and negative affect). To identify potential careless responding, researchers can use procedural detection methods and post hoc statistical methods. For example, researchers can insert detection items (e.g., infrequency items, instructed response items) into the questionnaire, monitor participants’ response time, and compute statistical indices, such as psychometric antonym/synonym, Mahalanobis distance, individual reliability, individual response variability, and model fit statistics. Application of multiple detection methods would be better able to capture careless responding given convergent evidence. Comparison of results based on data with and without careless respondents can help evaluate the degree to which the data are influenced by careless responding. To handle data contaminated by careless responding, researchers may choose to filter out identified careless respondents, recode careless responses as missing data, or include careless responding as a control variable in the analysis. To prevent careless responding, researchers have tried utilizing various deterrence methods developed from motivational and social interaction theories. These methods include giving warning, rewarding, or educational messages, proctoring the process of responding, and designing user-friendly surveys. Interest in careless responding has been growing not only in business and management but also in other related disciplines. Future research and practice on careless responding in the business and management areas can also benefit from findings in other related disciplines.

Development of the CDQL: a comprehensive quality of life measure for patients with contact dermatitis (Preprint)

BACKGROUND There is limited measurement and reporting of quality of life (QoL) outcomes in contact dermatitis (CD). OBJECTIVE The purpose of this study is to develop a standardized Contact Dermatitis Quality of Life index (CDQL). METHODS A list of 81 topics was compiled from review of QoL measures used previously in CD research. Two rounds of online Delphi surveys were sent to registrants of the 2018 American Contact Dermatitis Society meeting, asking that physicians rank the relevance of topics for measuring QoL in CD using a 4-point scale. Items met consensus for inclusion if at least 78% ranked them as relevant or very relevant and their median score was ≥3.25. RESULTS Thirty-four physicians completed the initial survey and 17 completed the follow-up survey. A total of 22 topics met consensus for inclusion in the CDQL, addressing symptoms, emotions, functions of daily living, social and physical functions, work/school functions, and treatment. CONCLUSIONS The study was limited by few open-ended questions in the initial survey, lack of direct patient feedback, and long survey length, which likely contributed to lower survey participation. The CDQL is a comprehensive, CD-specific QoL measure developed on the basis of expert consensus via a modified Delphi process.

Rushing to the end: Participants’ perceptions of demotivating aspects of online surveys

More and more social science studies are now acquiring data through the internet, reaching participantsonline. Some participants start out engaged and motivated to participate, but progressively slide into“rushing behaviors”. We inquired experts in survey responding about when, in online studies, theywould feel a desire for rushing (defined as speeding with no concerns about the quality of responses).This qualitative approach uncovered Repetition, Survey length and No interest in topic as the threemain features that would motivate these participants to rush in surveys. Subsequent inquiry of thesame participants indicated that repetition concerns the type of questions made (more than stimuli ortask), the execution of the same task more than 5-6 times, or for more than 6 minutes. Survey lengthconcerns a preference for shorter surveys, as well as the subjective experience in which length exceedspreviously set expectations (i.e., longer than announced), contributing to rushing by effectivelylowering the hourly pay rate as the survey increases in length. Interest in topic was reported to beconsistently low, despite not being the main reason to quit the survey. However, a change in expectedlevel of interest in the middle of the survey is reported as a factor that will promote rushing behaviors.We discuss these data as informative regarding how pre-tests of surveys can benefit from theseparticipants’ expertise.

Three-Form Split Questionnaire Design for Panel Surveys

Longitudinal or panel surveys are effective tools for measuring individual level changes in the outcome variables and their correlates. One drawback of these studies is dropout or nonresponse, potentially leading to biased results. One of the main reasons for dropout is the burden of repeatedly responding to long questionnaires. Advancements in survey administration methodology and multiple imputation software now make it possible for planned missing data designs to be implemented for improving the data quality through a reduction in survey length. Many papers have discussed implementing a planned missing data study using a split questionnaire design in the cross-sectional setting, but development of these designs in a longitudinal study has been limited. Using simulations and data from the Health and Retirement Study (HRS), we compare the performance of several methods for administering a split questionnaire design in the longitudinal setting. The results suggest that the optimal design depends on the data structure and estimand of interest. These factors must be taken into account when designing a longitudinal study with planned missing data.

Time-lapse electrical resistivity investigation of seawater intrusion in coastal aquifer of Ibeno, Southeastern Nigeria

Saltwater intrusion, erosion, and periodic flooding during severe storms are some of the critical problems bedeviling the precarious coastal environment of Nigeria. A time-lapse study aimed at monitoring the dynamics of saltwater intrusion within the coastland surrounding Ibeno, southeastern Nigeria is presented. Three (3) vertical electrical soundings (VES) and four time-lapse electrical resistivity tomography (ERT) profiles with minimum and maximum electrode spacing of 5 m and 40 m and profile lengths of 120 m and 150 m respectively were acquired in 2016 and 2017 using the Wenner and Schlumberger arrays. Data acquired from the field surveys were modeled with WINRESIST and RES2DINV software to generate 1D and 2D resistivity images of the subsurface. The VES results with a maximum survey length of AB = 300 m revealed a four-layer KH curve. The 2D geo-electrical sections delineated contrast between the high resistivity values obtained for the saturated freshwater zones and the low resistivity values obtained for the saturated saltwater zone. Expectedly, the intrusion was dominant nearer the coastline than inland. The salinity changes are believed to have been caused by both incremental tides passing through highly porous materials in the active-salinity change area and over-pumping of groundwater. The time-lapse resistivity measurements showed significant, active lateral salinity changes rapidly intruding the freshwater aquifers. The study demonstrates the application of time-lapse 2D ERT and vertical electrical sounding in identifying the spatial and temporal changes of saltwater intrusion in the coastal aquifer.

Reducing the Length of the Multidimensional Scale of Perceived Social Support

The Multidimensional Scale of Perceived Social Support (MSPSS) is a widely used 12-item measure that assesses perceived social support from three sources: friends, family, and significant others. Previously published psychometric properties indicate that a shorter version of this scale may adequately assess perceived social support and reduce participant burden. The current studies sought to develop such a reduced scale across two studies. Study 1 examined a sample of spouses of US military personnel ( N = 5,436) randomly separated into exploratory and confirmatory samples. In the exploratory sample, we developed a 6-item reduced MSPSS using multidimensional item response theory. In the confirmatory sample, the reduced MSPSS fit the hypothesized structure and was highly correlated with the full MSPSS. Study 2 administered the full and reduced MSPSS separately within a sample of undergraduate students ( N = 188). The reduced MSPSS had high correlations with the full measure ( r = .90) and fit the hypothesized factor structure. Across both studies, correlations with related constructs were similar between the reduced and full MSPSS, demonstrating almost no loss of construct validity. Overall, the reduced MSPSS captured perceived social support with little loss of information. This reduced scale may be useful for minimizing survey length and participant burden.

How long do respondents think online surveys should be? New evidence from two online panels in Germany

In recent years, the number of surveys, especially online surveys, has increased dramatically. Due to the absence of interviewers in this survey mode (who can motivate the respondents to continue answering), some researchers and practitioners argue that online surveys should not be longer than 20 min. However, so far, there has been little research investigating how long respondents think that online surveys should or could be. In this study, we therefore asked respondents of two online panels in Germany (one probability-based panel and one nonprobability panel) about their opinions on the ideal and maximum lengths of surveys. We also investigated whether socio-demographic, personality-related, and survey-related variables were associated with the ideal and maximum lengths reported by respondents. Finally, we compared the stated and observed survey lengths to evaluate the extent to which respondents are able to accurately estimate survey length. Our results suggest that the ideal length of an online survey is between 10 and 15 min and the maximum length is between 20 and 28 min, depending on the measure of central tendency (mean or median) used and the panel. Moreover, we found significant effects of socio-demographics (gender, age, education, and number of persons in household), of personality traits, and survey-related questions (whether the respondents liked the survey, found it easy, and answered from a PC) on at least one of the dependent variables (ideal or maximum lengths). Finally, we found only small differences (less than two min) between stated and observed lengths.

Writing Organizational Survey Items That Predict What Matters in Organizations

Organizational surveys are essential tools for gathering data in 21st-century organizations. This chapter provides a practitioner-oriented guide to developing survey content and items. In this chapter, the authors highlight the need to first determine highly specific survey project goals and deduce broad survey content domains based on those goals. They advise practitioners to build close relationships with employees and line managers to develop short survey tools that are organizationally relevant and predictive of organizational outcomes of interest (i.e., customer satisfaction, employee retention). Based on extant research evidence around item and scale development, they discuss response formats, survey length considerations, respondent literacy issues, and cultural as well as language considerations relevant to survey development. They conclude with ethical considerations and a brief summary.

Associations of Mail Survey Length and Layout With Response Rates

We assess the association between survey layout and response rates (RRs) in the 2017 Medicare Advantage Consumer Assessment of Healthcare Providers and Systems mail survey. Among 438 Medicare Advantage plans surveyed by six vendors, there was latitude in survey layout, and plans could add up to 12 supplemental items. Regression models predicted survey response from survey characteristics (page count, number of supplemental items, and survey attractiveness), and beneficiary sociodemographics. Beneficiary-age-by-survey-characteristic interactions assessed whether survey characteristics were more strongly related to RRs among older beneficiaries. We found that surveys with more supplemental items and less attractive layouts had lower adjusted odds of response. RRs were more sensitive to format among older beneficiaries. The difference in adjusted RRs for the most favorable versus the least favorable survey design was 14.5%. For a 65-year-old, this difference was 13.6%; for an 80-year-old, it was 21.0%. These findings suggest that even within a relatively standardized survey, formatting can substantially influence RRs.