On the Construct-Related Validity of Implicit Trait Policies

2021 ◽  
pp. 089020702110569
Author(s):  
Jan-Philipp Freudenstein ◽  
Patrick Mussel ◽  
Stefan Krumm
Keyword(s):  
Measurement Error ◽  
Personality Traits ◽  
Domain Knowledge ◽  
Implicit Beliefs ◽  
Situational Judgment ◽  
Situational Judgment Tests ◽  
General Domain ◽  
Personality Research

In response to recent calls to incorporate Implicit Trait Policies (ITPs) into personality research, the current study examined the construct-related validity of ITP measures. ITPs are defined as implicit beliefs about the effectiveness of behaviors that reflect a certain trait. They are assessed by utilizing the methodology of Situational Judgment Tests. We empirically examined ( N = 339) several underlying key assumptions of ITP theory, including trait-specificity, the relation to personality traits, their context-independence, and the relation to general domain knowledge. Overall, our results showed little support for these assumptions. Although we found some confirmation for expected correlations between ITPs and personality traits, most of the observed variance in ITP measures was either method-specific or due to measurement error. We conclude that the herein examined ITP measures lack construct-related validity and discuss implications for ITP theory and assessment.

It's Time To Examine the Nomological Net of Job Knowledge

2016 ◽  
Vol 9 (1) ◽  
pp. 51-55 ◽  
Author(s):  
W. Jackeline Torres ◽  
Margaret E. Beier
Keyword(s):  
Negative Feedback ◽  
Positive Feedback ◽  
Domain Knowledge ◽  
Work Life ◽  
Situational Judgment ◽  
Situational Judgment Tests ◽  
General Domain ◽  
Job Knowledge

Lievens and Motowidlo (2016) argue compellingly that situational judgment tests (SJTs) measure job-relevant general domain knowledge, conceptualized as implicit trait policies (ITPs). ITPs are defined as a person's knowledge about the utility of expressing certain traits. They develop through the feedback a person receives when acting in accordance with their trait profiles in different environments (work, life, leisure). Positive feedback reinforces the knowledge that behavior in accordance with one's own traits is appropriate, and negative feedback reinforces the knowledge that an approach that differs from one's trait tendencies may be more effective. As such, ITPs represent a person's knowledge about the effectiveness of behaviors across a variety of contexts.

In Defense of the Situation: An Interactionist Explanation for Performance on Situational Judgment Tests

2016 ◽  
Vol 9 (1) ◽  
pp. 23-28 ◽  
Author(s):  
Alexandra M. Harris ◽  
Lane E. Siedor ◽  
Yi Fan ◽  
Benjamin Listyg ◽  
Nathan T. Carter
Keyword(s):  
Domain Knowledge ◽  
Strong Support ◽  
Situational Judgment Test ◽  
Situational Judgment ◽  
Situational Judgment Tests ◽  
General Domain ◽  
Situational Strength ◽  
Activation Theory ◽  
Trait Activation ◽  
Key Pathways

Whereas Lievens and Motowidlo (2016) propose a model of situational judgment test (SJT) performance that removes the “situation” in favor of conceptualizing SJTs as a measure of general domain knowledge, we argue that the expression of general domain knowledge is in fact contingent on situational judgment. As we explain, the evidence cited by Lievens and Motowidlo against a situational component does not inherently exclude the importance of situations from SJTs and does overlook the strong support for a person–situation interaction explanation of behavior. Based on the interactionist literature—in particular, the trait activation theory (TAT) and situational strength literatures—we propose a model that both maintains the key pathways and definitions posited by Lievens and Motowidlo and integrates the situational component of SJTs.

Why Some Situational Judgment Tests Fail To Predict Job Performance (and Others Succeed)

2016 ◽  
Vol 9 (1) ◽  
pp. 71-77 ◽  
Author(s):  
Deborah L. Whetzel ◽  
Matthew C. Reeder
Keyword(s):  
Job Performance ◽  
Validation Studies ◽  
Domain Knowledge ◽  
Validity Evidence ◽  
Situational Judgment ◽  
Situational Judgment Tests ◽  
General Domain ◽  
Design Considerations

Situational judgment tests (SJTs) occasionally fail to predict job performance in criterion-related validation studies, often despite much effort to follow scholarly recipes for their development. This commentary provides some plausible explanations for why this may occur as well as some tips for SJT development. In most cases, we frame the issue from an implicit trait policy (ITP) perspective (Motowidlo, Hooper, & Jackson, 2006a, 2006b) and the measurement of general domain knowledge. In other instances, we believe that the issue does not have a direct tie to the ITP concept, but our experience suggests that the issue is of sufficient importance to include in this response. The first two issues involve challenges gathering validity evidence to support the use of SJTs, and the remaining issues deal more directly with SJT design considerations.

Both General Domain Knowledge and Situation Assessment Are Needed To Better Understand How SJTs Work

2016 ◽  
Vol 9 (1) ◽  
pp. 43-47 ◽  
Author(s):  
Jinyan Fan ◽  
Melissa Stuhlman ◽  
Lijun Chen ◽  
Qingxiong Weng
Keyword(s):  
Domain Knowledge ◽  
Situation Assessment ◽  
Situational Judgment ◽  
Situational Judgment Tests ◽  
General Domain ◽  
Strong Case

Although Lievens and Motowidlo (2016) made a strong case for reconceptualizing situational judgment tests (SJTs) as measures of general domain knowledge, we disagree with their view that the judgment or assessment of the situation itself is not important. We contend that situation assessment is an integral yet ignored factor in SJTs and that both general domain knowledge and situation assessment are needed to better understand how SJTs work.

Reinvigorating the Concept of a Situation in Situational Judgment Tests

2016 ◽  
Vol 9 (1) ◽  
pp. 38-42 ◽  
Author(s):  
Nicolas A. Brown ◽  
Ashley Bell Jones ◽  
David G. Serfass ◽  
Ryne A. Sherman
Keyword(s):  
Predictive Validity ◽  
Domain Knowledge ◽  
Predictive Power ◽  
Personality Measures ◽  
Situational Judgment ◽  
Situational Judgment Tests ◽  
General Domain ◽  
Per Se

What is the role of the situation in situational judgment tests (SJTs)? Lievens and Motowidlo (2016) assert that SJTs are somewhat of a misnomer because they do not actually measure how individuals would behave in a given situation per se. According to these researchers, SJTs assess general domain knowledge—whether potential employees recognize the “utility of expressing certain traits” (p. 4). As a result, SJTs map onto personality measures, which are a summary of behavior across time and situations. SJTs provide predictive validity in part because they tap into personality. However, rather than renaming SJTs, it is possible to reintroduce the concept of a situation to provide even greater predictive power. Thus, the goals of this commentary are to (a) clarify what constitutes a situation, (b) describe what SJTs might actually measure, and (c) set forth a path for a taxonomy of workplace situations.

scholarly journals Situational Judgment Tests: From Measures of Situational Judgment to Measures of General Domain Knowledge

2015 ◽  
Vol 9 (1) ◽  
pp. 3-22 ◽  
Author(s):  
Filip Lievens ◽  
Stephan J. Motowidlo
Keyword(s):  
Domain Knowledge ◽  
Choice Response ◽  
Actual Behavior ◽  
Response Format ◽  
Scientific Interest ◽  
Situational Judgment ◽  
Situational Judgment Tests ◽  
Selection Procedures ◽  
General Domain ◽  
Work Samples

Situational judgment tests (SJTs) are typically conceptualized as contextualized selection procedures that capture candidate responses to a set of relevant job situations as a basis for prediction. SJTs share their sample-based and contextualized approach with work samples and assessment center exercises, although they differ from these other simulations by presenting the situations in a low-fidelity (e.g., written) format. In addition, SJTs do not require candidates to respond through actual behavior because they capture candidates’ situational judgment via a multiple-choice response format. Accordingly, SJTs have also been labeled low-fidelity simulations. This SJT paradigm has been very successful: In the last 2 decades, scientific interest in SJTs has grown, and they have made rapid inroads in practice as attractive, versatile, and valid selection procedures. Contrary to their popularity and the voluminous research on their criterion-related validity, however, there has been little attention to developing a theory of why SJTs work. Similarly, in SJT development, often little emphasis is placed on measuring clear and explicit constructs. Therefore, Landy (2007) referred to SJTs as “psychometric alchemy” (p. 418).

SJTs as Measures of General Domain Knowledge for Multimedia Formats: Do Actions Speak Louder Than Words?

2016 ◽  
Vol 9 (1) ◽  
pp. 77-83 ◽  
Author(s):  
Bobby Naemi ◽  
Michelle Martin-Raugh ◽  
Harrison Kell
Keyword(s):  
Domain Knowledge ◽  
Prosocial Behaviors ◽  
General Knowledge ◽  
Situational Judgment ◽  
Situational Judgment Tests ◽  
General Domain ◽  
Paper And Pencil ◽  
Single Response

Lievens and Motowidlo (2016) present a case for situational judgment tests (SJTs) to be conceptualized as measures of general domain knowledge, which the authors define as knowledge of the effectiveness of general domains such as integrity, conscientiousness, and prosocial behaviors in different jobs. This argument comes from work rooted in the use of SJTs as measures of implicit trait policies (Motowidlo & Beier, 2010; Motowidlo, Hooper, & Jackson, 2006), measured with a format described as a “single response SJT” (Kell, Motowidlo, Martin, Stotts, & Moreno, 2014; Motowidlo, Crook, Kell, & Naemi, 2009). Given evidence that SJTs can be used as measures of general domain knowledge, the focal article concludes with a suggestion that general knowledge can be measured not only by traditional text-based or paper-and-pencil SJTs but also through varying alternate formats, including multimedia SJTs and interactive SJTs.

Situational Judgment Tests as an Alternative Measure for Personality Assessment

2018 ◽  
Vol 34 (5) ◽  
pp. 328-335 ◽  
Author(s):  
Patrick Mussel ◽  
Thomas Gatzka ◽  
Johannes Hewig
Keyword(s):  
Personality Traits ◽  
Personality Assessment ◽  
Work Performance ◽  
Well Being ◽  
Self Report ◽  
Alternative Measure ◽  
Applied Psychology ◽  
Situational Judgment Test ◽  
Situational Judgment ◽  
Situational Judgment Tests

Abstract. Across many domains of applied psychology, personality traits are related to important outcomes such as well-being, psychological disorders, work performance, and academic achievement. However, self-reports, the most common approach to personality assessment, have certain limitations and disadvantages, such as being prone to faking. We investigated whether situational judgment tests, an established assessment technique to predict job performance, might serve as an alternative measure for the assessment of personality. Our results show that a situational judgment test specifically developed to assess narrow personality traits may possess high levels of construct validity. Additionally, our results indicate that the situational judgment was equivalent to a self-report personality measure with regard to predicting a number of theoretically related criteria. We conclude that situational judgment tests may serve as an alternative method for the assessment of personality and discuss potential theoretical and applied drawbacks.

Clearly Defined Constructs and Specific Situations Are the Currency of SJTs

2016 ◽  
Vol 9 (1) ◽  
pp. 34-38 ◽  
Author(s):  
Lijun Chen ◽  
Jinyan Fan ◽  
Lu Zheng ◽  
Elissa Hack
Keyword(s):  
Domain Knowledge ◽  
Situational Judgment Test ◽  
Situational Judgment ◽  
General Domain

Although we echo Lievens and Motowidlo's (2016) view that situational judgment test (SJT) research should subscribe to the construct-driven approach, we disagree with their argument on two counts. First, we question whether measuring general domain knowledge represents the only way to advance SJT research. Second, we question whether it is appropriate to downplay the importance of situations in SJTs. In this commentary, we first briefly review construct-driven SJT studies and then share our own experience in developing an SJT for integrity in China using the construct-driven approach. Based on the review and reflection, we come to two major conclusions: (a) construct-driven SJT research has progressed well so far without the reconceptualization of SJTs as measures of general domain knowledge, and (b) specific situations are an important feature of SJTs that should not yet be dismissed.

Sign in / Sign up

Export Citation Format

Share Document