Approaches to Handling Common Response Styles and Issues in Educational Surveys

2017 ◽  
Author(s):  
Chester Chun Seng Kam ◽  
Xitao Fan
Keyword(s):  
Survey Data ◽  
Social Desirability ◽  
Negative Impact ◽  
Response Styles ◽  
Correlation Technique ◽  
General Tendency ◽  
Extreme Response ◽  
Careless Responding ◽  
Common Response ◽  
Different Response

Survey has been a widely used data collection method for a variety of purposes in educational research. Although response styles have the potential to contaminate survey results, educational researchers often do little to control for such negative effects. Under discussion are five common response issues, their impact on survey data, and the methods that may be used to minimize the negative impact of these response issues on survey data. The five response issues in question are acquiescence (including disacquiescence), careless responding, extreme response, social desirability, and item-keying effect. Acquiescence (disacquiescence) refers to a respondent’s general tendency to agree (or disagree) with an item regardless of its content. This response style can distort item and construct correlations, compromising the results of factor analytic and correlational findings. Careless responding refers to a respondent’s tendency to pay insufficient attention to item content before responding, which can also lead to a biased estimation of relationships. Extreme response refers to the tendency of selecting extreme response options (e.g., strongly agree or strongly disagree) over middle options (e.g., neutral). Social desirability refers to a respondent’s tendency to rate him- or herself in an overly positive light. Finally, item-keying effect refers to a respondent’s differential responses to regular-keyed and reverse-keyed items. This effect often creates the illusion that items with opposite keying directions measure distinct constructs even when they may not. A growing amount of research has been done on how to control for the negative impact of these response styles, although the research may be limited and uneven for different response issues. A variety of approaches and methods exist for handling these response issues in research practice. Different response issues may require considerations at different stages of research. For example, effective handling of acquiescence response may require steps in both survey construction (e.g., including a hidden measure of acquiescence) and survey data analytic treatment (partial correlation technique), while controlling for item-keying effect may require more sophisticated modeling techniques (e.g., multitrait-multimethod confirmatory factor analysis).

scholarly journals Standardization in psychological research.

2010 ◽  
Vol 3 (1) ◽  
pp. 88-96 ◽  
Author(s):  
Ronald Fischer ◽  
Taciano L. Milfont
Keyword(s):  
Social Desirability ◽  
Cultural Psychology ◽  
Statistical Tests ◽  
Psychological Research ◽  
Cross Cultural ◽  
Response Styles ◽  
Theoretical Interpretation ◽  
Simple Step ◽  
Different Response ◽  
Standardization Methods

The term standardization has been used in a number of different ways in psychological research, mainly in relation to standardization of procedure, standardization of interpretation and standardization of scores. The current paper will discuss the standardization of scores in more detail. Standardization of scores is a common praxis in settings where researchers are concerned with different response styles, issues of faking or social desirability. In these contexts, scores are transformed to increase validity prior to data analysis. In this paper, we will outline a broad taxonomy of standardization methods, will discuss when and how scores can be standardized, and what statistical tests are available after the transformation. Simple step-by-step procedures and examples of syntax files for SPSS are provided. Applications for personality, organizational and cross-cultural psychology will be discussed. Limitations of these techniques are discussed, especially in terms of theoretical interpretation of the transformed scores and use of such scores with multivariate statistics.

Multidimensional Modeling of Traits and Response Styles

2017 ◽  
Vol 33 (5) ◽  
pp. 352-364 ◽  
Author(s):  
Eunike Wetzel ◽  
Claus H. Carstensen
Keyword(s):  
Personality Traits ◽  
Latent Trait ◽  
Response Style ◽  
Response Styles ◽  
Extreme Response ◽  
Item Responses ◽  
A Posteriori Estimates ◽  
Sum Scores ◽  
Different Response ◽  
A Minor

Abstract. Response styles can influence item responses in addition to a respondent’s latent trait level. A common concern is that comparisons between individuals based on sum scores may be rendered invalid by response style effects. This paper investigates a multidimensional approach to modeling traits and response styles simultaneously. Models incorporating different response styles as well as personality traits (Big Five facets) were compared regarding model fit. Relationships between traits and response styles were investigated and different approaches to modeling extreme response style (ERS) were compared regarding their effects on trait estimates. All multidimensional models showed a better fit than the unidimensional models, indicating that response styles influenced item responses with ERS showing the largest incremental variance explanation. ERS and midpoint response style were mainly trait-independent whereas acquiescence and disacquiescence were strongly related to several personality traits. Expected a posteriori estimates of participants’ trait levels did not differ substantially between two-dimensional and unidimensional models when a set of heterogeneous items was used to model ERS. A minor adjustment of trait estimates occurred when the same items were used to model ERS and the trait, though the ERS dimension in this approach only reflected scale-specific ERS, rather than a general ERS tendency.

Cultural Differences in Careless Responding

2019 ◽  
Vol 50 (3) ◽  
pp. 336-357 ◽  
Author(s):  
Ina Grau ◽  
Christine Ebbeler ◽  
Rainer Banse
Keyword(s):  
Social Desirability ◽  
Gender Inequality ◽  
Cultural Dimensions ◽  
Response Style ◽  
Response Styles ◽  
Strongly Correlated ◽  
Country Level ◽  
Careless Responding ◽  
The Individual ◽  
Big 5 Personality

Careless responding (CR) in surveys has been identified as a serious threat for the validity of survey data. It occurs when participants respond without following the instructions or reading the content of items. The purpose of this article is to assess and explain the amount of CR at the individual and country level. In our study, individual CR (measured by indices using a short Big 5 personality questionnaire) correlated with education level and personality traits. In addition, using a sample of 8,320 participants from 34 countries, CR at the country level was investigated and it was strongly correlated with the cultural dimensions: human development, individualism, gender inequality, and power distance. CR can be seen as a powerful predictor of differences between countries. Finally, a comparison between CR and response styles (extreme and midpoint answers, acquiescence, and socially desirable responses) was conducted. CR and response styles showed some overlap, but loaded on different factors. CR is compatible with extreme answers, midpoint answers, and acquiescence, but different from social desirability. Social desirability is the only response style which requires careful reading of items.

Do Individual Response Styles Matter?

2013 ◽  
Vol 34 (2) ◽  
pp. 69-81 ◽  
Author(s):  
Eunike Wetzel ◽  
Jan R. Böhnke ◽  
Claus H. Carstensen ◽  
Matthias Ziegler ◽  
Fritz Ostendorf
Keyword(s):  
Differential Item Functioning ◽  
Response Style ◽  
Response Styles ◽  
Individual Response ◽  
Latent Classes ◽  
Complete Sample ◽  
Men And Women ◽  
Item Functioning ◽  
Extreme Response ◽  
Different Response

The occurrence of differential item functioning (DIF) for gender indicates that an instrument may not be functioning equivalently for men and women. Aside from DIF effects, item responses in personality questionnaires can also be influenced by response styles. This study analyzes the German NEO-PI-R regarding its differential item functioning for men and women while taking response styles into account. To this end, mixed Rasch models were estimated first to identify latent classes that differed in their response style. These latent classes were identified as extreme response style (ERS) and nonextreme response style (NERS). Then, DIF analyses were conducted separately for the different response styles and compared with DIF results for the complete sample. Several items especially on Neuroticism, Agreeableness, and Conscientiousness facets showed gender-DIF and thus function differentially between men and women. DIF results differed mainly in size between the complete sample and the response style subsamples, though DIF classification was overall consistent between ERS, NERS, and the complete sample.

Response Biases

2016 ◽  
pp. 349-363 ◽  
Author(s):  
Eunike Wetzel ◽  
Jan R. Böhnke ◽  
Anna Brown
Keyword(s):  
Cultural Differences ◽  
Test Construction ◽  
Response Styles ◽  
Cross Cultural Differences ◽  
Extreme Response ◽  
Trait Level ◽  
Item Responses ◽  
Item Content ◽  
General Response ◽  
Response Biases

Response biases comprise a variety of systematic tendencies of responding to questionnaire items. Response biases exert an influence on item responses in addition to any constructs that the questionnaire is designed to measure and can therefore potentially bias the corresponding trait level estimates. This chapter addresses general response biases that are independent of item content, including response styles (e.g., extreme response style, acquiescence) and rater biases (halo effect, leniency/severity bias), as well as response biases that are related to item content and depend strongly on the context (socially desirable responding). The chapter summarizes research on correlates of response biases and research on inter-individual and cross-cultural differences in engaging in response styles and rater biases. It describes different methods that can be applied at the test construction stage to prevent or minimize the occurrence of response biases. Finally, it depicts methods developed for correcting for the effects of response biases.

scholarly journals Expected effects of the US tax reform on other countries: global and local survey evidence

2020 ◽  
Vol 27 (6) ◽  
pp. 1608-1630
Author(s):  
Dorine Boumans ◽  
Clemens Fuest ◽  
Carla Krolage ◽  
Klaus Wohlrabe
Keyword(s):  
Survey Data ◽  
Negative Impact ◽  
Manufacturing Companies ◽  
The Us ◽  
The Usa ◽  
Business Survey ◽  
Economic Ties ◽  
Global And Local ◽  
Jobs Act ◽  
The Impact

Abstract The Tax Cuts and Jobs Act constitutes the largest change to the US tax system since the 1980s and thoroughly alters the way in which multinational companies are taxed. Current assessments on the reform’s international impact vary widely. This article sheds light on the tax reform’s expected effects on other countries. We first use representative German business survey data to analyze the impact of the reform on German firms. Many firms with substantial US revenues or capacities in the USA intend to expand US investment in response to the reform, in particular large firms and manufacturing companies. The effects on investment in Germany are ambiguous: While some firms substitute between investment locations, others expand in both countries. We subsequently extend our analysis to a global level using worldwide survey data. The results suggest a negative impact on tax revenues and investment in countries with close economic ties to the USA.

Elite Perceptions of the Judiciary in East Central and South East Europe

2013 ◽  
Vol 21 (1) ◽  
pp. 70-102
Author(s):  
Åse B. Grødeland ◽  
Aadne Aasland
Keyword(s):  
Coping Strategies ◽  
Survey Data ◽  
Negative Impact ◽  
Public Attitudes ◽  
Eu Accession ◽  
East Central ◽  
East Europe ◽  
South East Europe ◽  
Accession Process ◽  
The Eu

This article challenges the notion that the EU accession process has reduced the scope for informal practice in the judiciary in post-communist states. Elite survey data suggest that such practice is widespread – although primarily used in response to transition, informal practice has to some extent been carried over from communism and is largely used out of habit. Efforts to reduce the negative impact of informal practice in the judiciary should therefore not only seek to enhance its independence and capacity, but also address (1) public attitudes towards the law and the judiciary as such; (2) the coping strategies applied when interacting with the judiciary; and (3) the judges’ and prosecutors’ responses to such strategies.

Heritability of Acquiescence Bias and Item Keying Response Style Associated With the HEXACO Personality Scale

2013 ◽  
Vol 16 (4) ◽  
pp. 790-798 ◽  
Author(s):  
Chester Kam ◽  
Julie Aitken Schermer ◽  
Juliette Harris ◽  
Philip A. Vernon
Keyword(s):  
Environmental Effects ◽  
Response Style ◽  
Response Styles ◽  
Future Research ◽  
Shared Environment ◽  
Personality Measure ◽  
Personality Scale ◽  
Actual Content ◽  
Common Response ◽  
Response Behaviors

The current research investigates the heritability of two of the most common response styles: acquiescence bias (tendency to agree or disagree with survey items regardless of the items’ actual content) and item keying (differential responding related to the use of regular- and reverse-keyed items). We estimated response styles from a common personality measure (HEXACO) and examined the heritability of each with univariate genetics analyses. The results show item keying effect was heritable but acquiescence bias was not. Neither response style was strongly influenced by the shared environment of the twins. Unique environmental effects were found to be substantial for response styles. The current findings have important implications for future research of response behaviors that are often overlooked by behavioral geneticists.

Sign in / Sign up

Export Citation Format

Share Document