Why Does Construct Validity Matter in Measuring Implementation Fidelity? A Methodological Case Study

In reading intervention research, implementation fidelity is assumed to be positively related to student outcomes, but the methods used to measure fidelity are often treated as an afterthought. Fidelity has been conceptualized and measured in many different ways, suggesting a lack of construct validity. One aspect of construct validity is the fidelity index of a measure. This methodological case study examined how different decisions in fidelity indices influence relative rank ordering of individuals on the construct of interest and influence our perception of the relation between the construct and intervention outcomes. Data for this study came from a large State-funded project to implement multi-tiered systems of support for early reading instruction. Analyses were conducted to determine whether the different fidelity indices are stable in relative rank ordering participants and if fidelity indices of dosage and adherence data influence researcher decisions on model building within a multilevel modeling framework. Results indicated that the fidelity indices resulted in different relations to outcomes with the most commonly used fidelity indices for both dosage and adherence being the worst performing. The choice of index to use should receive considerable thought during the design phase of an intervention study.

The General Factor of Psychopathology in the Adolescent Brain Cognitive Development (ABCD) Study: A Comparison of Alternative Modeling Approaches

Many models of psychopathology include a single general factor of psychopathology (GFP) or “ p factor” to account for covariation across symptoms. The Adolescent Brain Cognitive Development (ABCD) Study provides a rich opportunity to study the development of the GFP. However, a variety of approaches for modeling the GFP have emerged, raising questions about how modeling choices affect estimated GFP scores. We used the ABCD baseline assessment (ages 9–10 years old; N = 11,875) of the parent-rated Child Behavior Checklist (CBCL) to examine the implications of modeling the GFP using items versus scales, using a priori CBCL scales versus data-driven dimensions, and using bifactor, higher order, or single-factor models. Children’s rank-ordering on the GFP was stable across models, and GFP scores were similarly related to criterion variables. Results suggest that although theoretical debates about modeling the GFP continue, the practical implications of these choices for rank-ordering children and assessing external associations will often be modest.

Rating Scale Data and its Utility: Additional Evidence to be Explored in the Medical Education Continuum

The purposes of this study are to assess the utility of self-appraisals and ratings of program directors (PDs) and to introduce new ways to use self-assessed and rating scale data. The data for this study was collected from graduates of our school who were enrolled in different residency programs around the country. The interns and PDs completed a similar set of questions. The correlation between the ratings of the two group was .21. The Cronbach’s alpha of interns and PD surveys were .89 and .97, respectively. The interns consistently rated themselves lower compared to ratings the PDs assigned them. The two groups agreed on the areas of strengths and weaknesses based on their mean ratings and rank-ordering of competencies. This study proposes that lowest mean ratings of measures that appear at the bottom in the rank-ordering be considered as areas that deserve special attention. The results of this study brought validity evidence to the utility of self-appraisals and PD’s ratings of interns.

Who Am I? Let Me Think: Assessing the Considered Self-Concept

My Multiple Selves (MMS) is an open-ended self-concept measure that leads respondents to systematically explore who they are before selecting and rank-ordering their most important selves. A sample of college students ( n = 204) completed either the MMS or the frequently used Twenty Statements Test (TST). As predicted, results demonstrated that MMS responses included elements underdetected by the TST: other people, school crowd labels, aspirations for the future, and undesirable selves. Furthermore, the MMS revealed that when other people are included in self, they are rank ordered by participants as the most important element. Shortcomings of the MMS are discussed, and directions for future research outlined.

Ranking the Regimes in Aristotle's Politics: The Four-Principles Approach

There is a long-standing debate over which constitution Aristotle regards as best in the Politics. I attempt to clarify his view by reconstructing four principles he uses to assess constitutions, in both ideal and more ordinary circumstances: (i) the supremacy-of-virtue principle, (ii) the more-virtuous-citizens-are-better-than-fewer principle, (iii) the equality principle, and (iv) the stability principle. I apply these principles to defend a rank ordering of constitutions, which situates the ideal aristocracy of books 7 and 8 at the top, and tyranny, along with unmixed forms of democracy and oligarchy, at the bottom.

Competition

This chapter discusses the unavoidability of competition, even among those who renounce the more common forms of competitions for honor, wealth, and so forth. It starts with a discussion of the Garden of Eden and conditions of zero scarcity therein, with but one scarce item: the forbidden fruit. Then the chapter goes on to positional goods, rank-ordering people, which leads to a discussion of seating arrangements, musical chairs, and the failure of King Arthur to resolve the matter with a round table. It then provides a treatment of Christian attempts to avoid honor competitions by elevating humility to the status of one of the chief virtues. But that ends in humility contests and we are back to square one. The chapter ends with a sublime text from Gregory of Tours about a truly humble miracle-working young monk and the efforts to keep him humble despite his miracle-working powers.