Effect of Response Format and Product Involvement on Questionnaire Reliability

1983 ◽  
Vol 52 (2) ◽  
pp. 407-411 ◽  
Author(s):  
Richard H. Evans

This study examined the effect of response format and product involvement on reliability. According to the results, reliability varied over the four response formats and product-involvement levels. An inverse pattern existed between product involvement and reliability.

2019 ◽  
Vol 11 (5) ◽  
pp. 688-696
Author(s):  
Maria Iankilevitch ◽  
Lindsey A. Cary ◽  
Jessica D. Remedios ◽  
Alison L. Chasteen

Due to their awareness of multiraciality and their perceptions of race categories as fluid, multiracial individuals may be unique in how they racially categorize multiracial faces. Yet race categorization research has largely overlooked how multiracial individuals categorize other mixed-race people. We therefore asked Asian, White, and multiracial individuals to categorize Asian-White faces using an open-ended response format, which more closely mirrors real-world race categorizations than forced-choice response formats. Our results showed that perceivers from all three racial groups tended to categorize Asian-White faces as monoracial Asian, White, or Hispanic. However, multiracial perceivers categorized the Asian-White faces as multiracial more often than monoracial perceivers did. Our findings suggest that multiracial individuals may approach racial categorization differently from either monoracial majority or minority group members. Furthermore, our results illustrate possible difficulties multiracial people may face when trying to identify other multiracial in-group members.


2021 ◽  
Author(s):  
Swaha Pattanaik ◽  
Mike John ◽  
Seungwon Chung ◽  
San Keller

Abstract PurposeWe compared measurement properties of 5-point and 11-point response formats for Orofacial Esthetic Scale (OES) items to determine whether collapsing the format would degrade OES score precision.MethodsData were collected from a consecutive sample of adult dental patients from HealthPartners dental clinics in Minnesota (N=2,078). We fitted an Item Response Theory (IRT) model to the 11-point scale and six, derived 5-point scales. We compared all response formats using test (or scale) information, correlation between the IRT scores, Cronbach’s alpha estimates for each scaling format, correlations based on the observed scores for the seven OES items and the eighth global item, and the relationship of observed and IRT scores to an external criterion using orofacial appearance (OA) indicators from the Oral Health Impact Profile (OHIP).ResultsThe correlations among scores based on the different response formats were uniformly high for observed (0.97-0.99) and IRT scores (0.96-0.99); as were correlations of both observed and IRT scores and the OHIP measure of OA (0.65-0.69). Cronbach’s alpha based on any of the 5-point formats (α = 0.95) was nearly the same as that based on the 11-point format (α = 0.96). The weighted total information area for five of six, 5-point derived formats was 98% of that for the 11-point scale ConclusionsOur results support the use of scores based on a 5-point response format for OES items. The measurement properties of scores based on a 5-point response format are comparable to those of scores based on the 11-point format.


Author(s):  
Jan Karem Höhne

AbstractMany study designs in social science research rely on repeated measurements implying that the same respondents are asked the same (or nearly the same) questions at least twice. An assumption made by such study designs is that respondents second answer does not depend on their first answer. However, if respondents recall their initial answer and base their second answer on it memory effects may affect the survey outcome. In this study, I investigate respondents’ recall ability and memory effects within the same survey and randomly assign respondents to a device type (PC or smartphone) and a response format (response scale or text field) for reporting their previous answer. While the results reveal no differences regarding device types, they reveal differences regarding response formats. Respondents’ recall ability is higher when they are provided with the response scale again than when they are only provided with a text field (without displaying the response scale again). The same finding applies to the size of estimated memory effects. This study provides evidence that the size of memory effects may have been overestimated in previous studies.


1998 ◽  
Vol 24 (3) ◽  
Author(s):  
Nadene Hanekom ◽  
Delene Visser

This study examines the degree of acquiescence present when the item and response formats of a summated rating scale are varied. It is often recommended that acquiescence response bias in rating scales may be controlled by using both positively and negatively worded items. Such items are generally worded in the Likert-type format of statements. The purpose of the study was to establish whether items in question format would result in a smaller degree of acquiescence than items worded as statements. the response format was also varied (five- and seven-point options) to determine whether this would influence the reliability and degree of acquiescence in the scales. A twenty-item Locus of Control (LC) questionnaire was used, but each item was complemented by its opposite, resulting in 40 items. The subjects, divided randomly into two groups, were second year students who had to complete four versions of the questionnaire, plus a shortened version of Bass's scale for measuring acquiescence. The LC version were questions or statements each combined with a five- or seven-point respons format. Partial counterbalancing was introduced by testing on two separate occasions, presenting the tests to the two groups in the opposite order. The degree of acquiescence was assessed by correlating the items with their opposite, and by correlating scores on each version with scores on the acquiescence questionnaire. No major difference were found between the various item and response format in relation to acquiescence. Opsomming Hierdie ondersoek is uitgevoer om te bepaal of die mate van instemmingsgeneigdheid deur die item- en responsformaat van 'n gesommeerde selfbeoordelingskaal beinvloed word. Daar word dikwels aanbeveel dat die gebruik van positief- sowel as negatiefbewoorde items in 'n vraelys instemmingsgeneigdheid beperk. Suike items word gewoonlik in die tradisionele Likertformaat as stellings geformuleer. Die doel van die ondersoek was om te bepaal of items in vraagformaat minder instemmingsgeneigdheid as die stellingformaat tot gevolg het. Die responsformaat is ook gevarieer (vyf- en sewepunt opsies) om te bepaal of die betroubaarheid en graad van instemmingsgeneigdheid daardeur beinvloed word. 'n Twintigitem Lokus van Beheer (LvB) vraelys is vir hierdie doel gebruik, maar vir eike item is "n inverse gekonstrueer, sodat die uiteindelike vraelys uit 40 items bestaan het. Die steekproef wat ewekansig in twee groepe verdeel is, was tweedejaarstudente wat elk vier weergawes van die vraelys asook "n verkorte weergawe van Bass se mstemrningsgeneigdheidsvraelys moes voltooi. Die LvB-weergawes was vrae of stellings wat elk met 'n vyf- of sewepuntresponsformaat gekombineer is. Gedeeltelike teenbalansering is toegepas deur twee toetsgeleenthede waarin die vraelys aan die onderskeie groepe in die teenoorgestelde volgorde geadministreer is, te gebruik. Die teenwoordigheid van instemmingsgeneigdheid is bepaal deur die items met hul inverses te korreleer, asook om eike vraelys met die instemmmgsgeneigdheidsvraelys te korreleer. Geen noemenswaardige verskille is tussen die verskillende item- en responsformaat, met betrekking tot instemmingsgeneigdheid, gevind nie.


Author(s):  
Jiajin Tong ◽  
Robert M. Bickmeier ◽  
Steven G. Rogelberg

The present research compares and contrasts frequency versus agreement response formats, two approaches to measuring job burnout and work engagement. Construct-based and measurement-based arguments for the superiority of the frequency response format in measuring burnout/engagement are provided, demonstrating that frequency-based measurements will explain relatively more variance in outcome variables. Fair comparison, time order counterbalance, and multiple measuring waves justify the comparison and reduce common method errors of self-report measures. Sample 1 (N = 242) was composed of employees from multiple organizations, while the participants in Sample 2 (N = 281) were employees from one company. Relative importance analysis showed that frequency outperforms the agreement response format in measuring burnout and engagement in both samples. These findings suggest that the frequency response format provides a more valuable method of detecting the dynamic nature of burnout/engagement, which offers methodological guidance for future research involving dynamic constructs. These findings can lead to improvements in the measurement of the dynamic experiences of burnout and engagement. This is one of the first studies to provide evidence whether the dynamic nature of the constructs would have any bearing on the response formats.


2016 ◽  
Vol 46 (4) ◽  
pp. 477-490 ◽  
Author(s):  
Aaron Adibe Agbo

The present study examined the competing factor structures, validity, and reliability of the state and trait versions of the International Positive and Negative Affect Schedule – Short Form and the effect of the number of response categories on the scale qualities among Nigerians. Undergraduates (N-1510) completed a trait version of the scale with 5-point and 7-point response formats and a state version with a 7-point response format. Confirmatory factor analysis was employed alongside other techniques to gauge the psychometric properties of the scale. The obtained factor structures for the trait and state versions were in line with the theoretical assumptions of the scale and previous findings. A correlated two-factor model provided the best fit for the trait version, while an orthogonal two-factor model provided the best fit for the state version. The scale performed well with the 7-point response format, but it performed poorly with the 5-point response format, suggesting that the behaviour of the scale depends on the number of response categories. The findings, limitations of the study, and suggestions for further studies are discussed.


2018 ◽  
Author(s):  
Miroslav Sirota ◽  
Marie Juanchich

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular since it reliably predicts reasoning performance, decision-making and beliefs. Across studies, the response format of CRT items sometimes differs, assuming construct equivalence of the tests with open-ended vs. multiple choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the non-equivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered an open-ended, a two- or a four-option response format of stem-equivalent CRT items and completed tasks on belief bias, denominator neglect and paranormal beliefs (benchmark indicators of predictive validity) as well as actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the number of correct responses, the number of intuitive responses (with the exception of the two-option version being higher than the other tests) and in the correlational patterns with the indicators of predictive and construct validity. All three test versions were similarly reliable but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps to build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons.


2016 ◽  
Author(s):  
Michelle A Harris ◽  
Brent Donnellan ◽  
Kali Trzesniewski

This paper introduces the Lifespan Self-Esteem scale (LSE; a 4-item measure with a 5-point smilie face response format), a short measure of global self-esteem suitable for use with populations drawn from across the lifespan. Existing measures of global self-esteem cannot be used across multiple developmental periods due to changes in item content, response formats, and other scale characteristics, and have not been validated for children under the age of 8. The LSE is essentially unidimensional, internally consistent, and converges with existing measures across ages 5 to 93 (N=2,761).


2021 ◽  
pp. 088626052110642
Author(s):  
RaeAnn E. Anderson ◽  
Kelly Cuccolo

Objective: This study examined the impact of a dichotomous versus scaled response format on prevalence rates of sexual violence perpetration and victimization, thus conceptually replicating Hamby et al., 2006 and extending those findings to the context of sexual violence. Methods: Two samples were recruited from Amazon Mechanical Turk and were randomly assigned to either a dichotomous or scaled response format of the same questionnaire. Sample 1 was used to examine perpetration and received a perpetration specific version of the Post-Refusal Sexual Persistence Scale (PRSPS). Sample 2 was used to examine victimization and received a victimization specific version of the PRSPS. Results: Men and women who received the scaled response format reported significantly more incidents of sexual perpetration, and for each type of tactic studied, on the order of 1.7-9x more frequently. The association between response format condition and prevalence rates of sexual violence victimization was significant only for men (23.0 vs 39.4%) and for the tactic of verbal coercion (30.1 vs 41.5%), with the scaled response format producing greater responding. Conclusions: The response format of sexual violence items can significantly alter prevalence rates of sexual violence perpetration, with scaled response formats producing greater endorsements than dichotomous formats. Response format also appears to impact prevalence rates of sexual violence victimization, particularly for men.


Author(s):  
Vera Toepoel ◽  
Brenda Vermeeren ◽  
Baran Metin

Studies of the processes underlying question answering in surveys suggest that the choice of (layout for) response categories can have a significant effect on respondent answers. In recent years, the use of pictures, such as emojis or stars, is often used in online communication. It is unclear if pictorial answer categories can replace traditional verbal formats as measurement instruments in surveys. In this article we investigate different versions of a Likert-scale to see if they generate similar results and user experiences. Data comes from the non-probability based Flitspanel in the Netherlands. The hearts and stars designs received lower average scores compared to the other formats. Smileys produced average answer scores in line with traditional radio buttons. Respondents evaluated the smiley design most positively. Grid designs were evaluated more negatively. People wanting to compare survey outcomes should be aware of these effects and only compare results when similar response formats are used.


Sign in / Sign up

Export Citation Format

Share Document