Context effects in Internet surveys

2012 ◽  
Author(s):  
Jolene D. Smyth ◽  
Don A. Dillman ◽  
Leah Melani Christian
Keyword(s):  
Context Effects ◽  
Response Scale ◽  
Survey Mode ◽  
Response Options ◽  
Telephone Surveys ◽  
Internet Surveys ◽  
Response Scales ◽  
Definition Of

This article first presents a definition of context effects that eliminates from consideration factors beyond the control of survey researchers yet is sufficiently broad to incorporate diverse but related sources of survey context. It then examines four types of context effects that have been documented in mail and telephone surveys with an eye towards identifying new concerns which have arisen or may arise as a result of conducting Internet surveys. The four sources of context effects discussed are: the survey mode used to pose questions to respondents, the order in which questions are asked, the ordering of response options, and the choice of response scale. In addition to reviewing previous research, the results of new context experiments are reported in which response scales across Internet and telephone modes are manipulated.

The Advent of Internet Surveys for Political Research: A Comparison of Telephone and Internet Samples

2003 ◽  
Vol 11 (1) ◽  
pp. 1-22 ◽  
Author(s):  
Robert P. Berrens ◽  
Alok K. Bohara ◽  
Hank Jenkins-Smith ◽  
Carol Silva ◽  
David L. Weimer
Keyword(s):  
Global Climate ◽  
Science Research ◽  
Knowledge Networks ◽  
The Internet ◽  
Survey Mode ◽  
Telephone Surveys ◽  
Internet Surveys ◽  
Interviewer Bias ◽  
National Probability Sample ◽  
Harris Interactive

The Internet offers a number of advantages as a survey mode: low marginal cost per completed response, capabilities for providing respondents with large quantities of information, speed, and elimination of interviewer bias. Those seeking these advantages confront the problem of representativeness both in terms of coverage of the population and capabilities for drawing random samples. Two major strategies have been pursued commercially to develop the Internet as a survey mode. One strategy, used by Harris Interactive, involves assembling a large panel of willing respondents who can be sampled. Another strategy, used by Knowledge Networks, involves using random digit dialing (RDD) telephone methods to recruit households to a panel of Web-TV enabled respondents. Do these panels adequately deal with the problem of representativeness to be useful in political science research? The authors address this question with results from parallel surveys on global climate change and the Kyoto Protocol administered by telephone to a national probability sample and by Internet to samples of the Harris Interactive and Knowledge Networks panels. Knowledge and opinion questions generally show statistically significant but substantively modest difference across the modes. With inclusion of standard demographic controls, typical relational models of interest to political scientists produce similar estimates of parameters across modes. It thus appears that, with appropriate weighting, samples from these panels are sufficiently representative of the U.S. population to be reasonable alternatives in many applications to samples gathered through RDD telephone surveys.

scholarly journals Psychometric validation and refinement of the Interoception Sensory Questionnaire (ISQ) in adolescents and adults on the autism spectrum

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Evan Suzman ◽  
Zachary J. Williams ◽  
Jacob I. Feldman ◽  
Michelle Failla ◽  
Carissa J. Cascio ◽  
...  
Keyword(s):  
Autism Spectrum ◽  
The Body ◽  
Self Report ◽  
Psychometric Validation ◽  
Response Scale ◽  
Response Options ◽  
Response Choice ◽  
Autistic People ◽  
Adolescents And Adults ◽  
Sensory Questionnaire

Abstract Background Individuals on the autism spectrum are reported to display alterations in interoception, the sense of the internal state of the body. The Interoception Sensory Questionnaire (ISQ) is a 20-item self-report measure of interoception specifically intended to measure this construct in autistic people. The psychometrics of the ISQ, however, have not previously been evaluated in a large sample of autistic individuals. Methods Using confirmatory factor analysis, we evaluated the latent structure of the ISQ in a large online sample of adults on the autism spectrum and found that the unidimensional model fit the data poorly. Using misspecification analysis to identify areas of local misfit and item response theory to investigate the appropriateness of the seven-point response scale, we removed redundant items and collapsed the response options to put forth a novel eight-item, five-response choice ISQ. Results The revised, five-response choice ISQ (ISQ-8) showed much improved fit while maintaining high internal reliability. Differential item functioning (DIF) analyses indicated that the items of the ISQ-8 were answered in comparable ways by autistic adolescents and adults and across multiple other sociodemographic groups. Limitations Our results were limited by the fact that we did not collect data for typically developing controls, preventing the analysis of DIF by diagnostic status. Additionally, while this study proposes a new 5-response scale for the ISQ-8, our data were not collected using this method; thus, the psychometric properties for the revised version of this instrument require further investigation. Conclusion The ISQ-8 shows promise as a reliable and valid measure of interoception in adolescents and adults on the autism spectrum, but additional work is needed to examine its psychometrics in this population. A free online score calculator has been created to facilitate the use of ISQ-8 latent trait scores for further studies of autistic adolescents and adults (available at https://asdmeasures.shinyapps.io/ISQ_score/).

Satisfaction with daily activities after retirement in Europe

2014 ◽  
Vol 35 (6) ◽  
pp. 1146-1170 ◽  
Author(s):  
ERIC BONSANG ◽  
ARTHUR VAN SOEST
Keyword(s):  
European Countries ◽  
Daily Activities ◽  
Response Scale ◽  
Central European ◽  
Demographic Groups ◽  
Anchoring Vignettes ◽  
Response Scales ◽  
Mediterranean Countries ◽  
Determinants Of Satisfaction ◽  
Country Differences

ABSTRACTThis paper analyses the determinants of satisfaction with daily activities among retirees aged 65 years or older in 11 European countries. We use data from the Survey of Health, Ageing and Retirement in Europe on self-assessed satisfaction with daily activities and anchoring vignettes to correct for potential differences in response scales across countries and socio-demographic groups. On average, retired Europeans appear to be satisfied with their daily activities, but there are large differences across countries: respondents from Northern countries tend to be more satisfied than individuals from Central European or Mediterranean countries. Our analysis shows that correcting for response scale differentials alters the country ranking for satisfaction with daily activities but hardly affects the conclusions on the factors driving within-country differences between socio-demographic groups.

THE COMPARATIVE FRAME OF REFERENCE IN SELF-RATED HEALTH QUESTIONS MATTERS WHEN PREDICTING DIFFICULTY WITH ACTIVITIES OF DAILY LIVING

2015 ◽  
pp. 1-6
Author(s):  
C. SIORDIA ◽  
H.T. NGUYEN
Keyword(s):  
Activities Of Daily Living ◽  
Daily Living ◽  
Cross Sectional Study ◽  
Community Dwelling ◽  
Frame Of Reference ◽  
Response Scale ◽  
Cross Sectional ◽  
Response Options ◽  
Self Rated Health ◽  
Level Of Difficulty

Background: Within the assessment of frailty, self-rated health (SRH) questions remain frequently used in survey research attempting to quantify a subjective and global measure of health. Studies have largely ignored the fact that SRH questions may differ in their ability to predict level of difficulty with performing activities of daily living (ADLs)—a variance partially influenced by whether a comparative frame of reference in used in the SRH question. Specific Aim: Investigate if a Comparative-SRH (C-SRH) question with response options ranging on an adjectival scale ranging from 0 to 3; and a Non-Comparative-SRH (NC-SRH) question with an adjectival response scale from range 0 to 5; differ in their ability to predict level of difficulty in performing ADLs after accounting for demographic, psychosocial, and related health factors. Setting & Design: Cross-sectional study of community-dwelling adults (n=275; x age=68; female=54%; x BMI=31) from North Carolina participating in the Hispanic Aging Survey (HAS) were used in multivariable linear regressions to predict a granular measure of level of difficulty performing ADLs. Results: Only C-SRH has a statistically significant relationship with ADLs—where each increase in C-SRH (i.e., more positively self-rated health) is associated with a decrease in ADL level of difficult. Conclusions: Using a comparative frame of reference in SRH questions may have important implications when attempting to understand the statistical relationship between self-rated global health and physical function in the assessment of frailty in older adults.

Use of the Lithium Response Scale with an Outpatient Psychiatric Sample

1980 ◽  
Vol 47 (1) ◽  
pp. 69-70 ◽  
Author(s):  
Bruce M. Burdick ◽  
Cooper B. Holmes
Keyword(s):  
Lithium Therapy ◽  
Response Scale ◽  
Response Scales ◽  
Lithium Response ◽  
Low Ability

Two potentially useful MMPI Lithium Response Scales showed low ability to predict antidepressant responses to lithium therapy in an outpatient sample of 20.

Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples

2011 ◽  
Vol 75 (4) ◽  
pp. 709-747 ◽  
Author(s):  
David S. Yeager ◽  
Jon A. Krosnick ◽  
LinChiat Chang ◽  
Harold S. Javitz ◽  
Matthew S. Levendusky ◽  
...  
Keyword(s):  
Telephone Surveys ◽  
Internet Surveys

Effects of Unbalanced Response Scales on Judgments of Social Stimuli

1963 ◽  
Vol 12 (2) ◽  
pp. 403-414 ◽  
Author(s):  
Walter Weiss
Keyword(s):  
College Students ◽  
General Purpose ◽  
Response Scale ◽  
Social Stimuli ◽  
Response Scales ◽  
Social Prestige ◽  
The Social ◽  
Physical Stimuli ◽  
The Prestige

The general purpose of the research was to determine if scale judgments of certain social stimuli would be differentially affected by the distribution of scale categories constituting the response scale. Previous research (Weiss & Hodgson, 1963) had demonstrated such an effect on judgments of neutral, physical stimuli. Two experiments were therefore conducted in which 72 and 248 college students judged the social prestige of 40 occupations. Ss used one of four types of scales: a 3-category balanced scale (+, 0, −), a 7-category balanced scale (+3 to −3), a 5-category unbalanced minus scale (+, 0, −1, −2, −3), or a 5-category unbalanced plus scale (+3, +2, +1, 0, −). The non-discriminating plus and minus categories were to be used for occupations that were above or below the average in prestige, regardless of how far above or below they might be. The zero category designated those average in prestige. And the discriminating plus or minus categories represented degrees of prestige above or below the average. The data revealed significant effects of the scales on the typical social prestige of occupations assigned to the ‘average’ category of judgment. Relative to the balanced scales, the unbalanced ones induced a shift in the prestige value of the ‘average’ category in the direction of the single, non-discriminating category. Relative to the 7-category balanced scale, the 3-category one produced a shift in the direction of the non-discriminating plus side. The data were interpreted primarily in terms of relationships between implicit category standards, latitudes of categories, and the distribution of the judged stimuli.

Relations Between Variables and Trends Over Time in Rdd Telephone and Nonprobability Sample Internet Surveys

2020 ◽  
Vol 8 (1) ◽  
pp. 37-61
Author(s):  
Josh Pasek ◽  
Jon A Krosnick
Keyword(s):  
Data Streams ◽  
Behavioral Intentions ◽  
Lower Cost ◽  
The Internet ◽  
Telephone Surveys ◽  
Internet Surveys ◽  
Trends Over Time ◽  
Nonprobability Sample ◽  
The Impact ◽  
Over Time

Abstract Survey researchers today can choose between relatively higher-cost probability sample telephone surveys and lower-cost surveys of nonprobability samples of potential respondents who complete questionnaires via the internet. Previous studies generally indicated that the former yield more accurate distributions of variables, but little work to date has explored the impact of mode and sampling on associations between variables and trends over time. The current study did so using parallel surveys conducted in 2010 focused on opinions, events, behavioral intentions, and behaviors involving that year’s Decennial Census. A few comparisons indicated that the two data streams yielded similar results, but the two methods frequently yielded different results, often strikingly so, and the results yielded by the probability samples seem likely to be the more accurate ones.

scholarly journals Construction and validation of an Educational Content Validation Instrument in Health

2018 ◽  
Vol 71 (suppl 4) ◽  
pp. 1635-1641 ◽  
Author(s):  
Sarah de Sá Leite ◽  
Aline Cruz Esmeraldo Áfio ◽  
Luciana Vieira de Carvalho ◽  
Jacqueline Mota da Silva ◽  
Paulo César de Almeida ◽  
...  
Keyword(s):  
Health Professionals ◽  
Conceptual Structure ◽  
Content Validation ◽  
Methodological Study ◽  
Scale Selection ◽  
Response Scale ◽  
Educational Content ◽  
Good Reliability ◽  
Good Internal Consistency ◽  
Definition Of

ABSTRACT Objective: to construct and validate the Educational Content Validation Instrument in Health. Method: methodological study that includes the establishment of the conceptual structure; definition of objectives and population; construction of items and response scale; selection and arrangement of items; instrument structuring; opinion of experts; pre-test and content validation. Results: an instrument with 15 items was constructed and, after expert evaluation, eighteen items were obtained, divided into three domains: objectives (four), structure/presentation (nine), and relevance (two). Six items were modified since they presented a percentage of agreement below 0.8. Items of the total instrument presented good internal consistency (0.877) regarding domains. Conclusion: an Educational Content Validation Instrument in Health was elaborated and validated, presenting good reliability, and may contribute to the practice of researchers and health professionals in the development of educational content.

Sign in / Sign up

Export Citation Format

Share Document