With the availability of a personality relevant situation taxonomy (Ten Berge & De Raad, 2002), a burning question ahead is whether systematic inclusion of situation references in personality questionnaires does improve their descriptive and predictive capacities. In this paper, the reliability question was addressed, and we investigated for which factors of the Big Five the interjudge‐agreement would increase upon adding situational information. In this study we excluded the fifth factor, Intellectual Autonomy. Two studies were performed, involving a first test in Study 1, and a replication with three tests through varying sets of items in Study 2. In Study 1, trait descriptive adjectives representing four factors were administered twice (N = 182), one time without and one time with situational information. In Study 2 (N = 152), we varied both the trait‐adjectives and the situational trait‐sentences from set to set. All four tests indicated that the interjudge‐agreement for Extraversion generally decreased upon adding situation information, for Agreeableness and Conscientiousness it did not essentially change, and the interjudge‐agreement for Emotional Stability substantially increased. Some explanations and implications are discussed. Copyright © 2008 John Wiley & Sons, Ltd.