The External Validity of College Student Subject Pools in Experimental Research: A Cross-Sample Comparison of Treatment Effect Heterogeneity

Are student subject experiment pools comparable across institutions? Despite repeated concerns over the “college sophomore problem,” many experiment-based studies still rely on student subject pools due to their convenience and accessibility. In this paper, I investigate whether student subject pools are comparable across universities by examining how respondents across three student subject pools at distinct educational institutions perform on the same survey experiment about crisis bargaining between states. I argue that, due to selection biases inherent in university matriculation and the self-selection of students into experimental protocols, respondents across these subject pools will exhibit key demographic differences. I also examine whether respondents across these subject pools think similarly about international politics and respond comparably to experimental treatments. I find that, while there are significant demographic differences across subject pools, subjects across institutions respond similarly to experimental treatments—with the key exception of information regarding the regime type of a state. Furthermore, there is little evidence that these demographic differences impact conditional average treatment effects across subgroups. These findings carry critical implications for the use of student samples across political science and within international relations more specifically, particularly regarding the current replication crisis in the discipline.

The Impact of Concussion Education on the Knowledge and Perceived Expertise of Novice Health Care Professionals

Context: Concussion legislation mandates that health care providers have experience in concussion management. Unfortunately, standards for current continuing and clinician education are ill defined. Objective: (1) Determine if a didactic-based educational intervention would increase knowledge and perceived expertise and (2) examine the correlations between the variables of knowledge, experience, and perceived expertise. Design: Prospective cohort study, level II. Patients or Other Participants: Novice health care providers were divided into 2 groups: college sophomore athletic training students (n = 16) and college sophomore medical dietetics students (n = 19). Setting: Classroom setting. Intervention(s): Both groups were administered a knowledge questionnaire before the intervention (Time 1) and again 30 weeks later (Time 3). The athletic training student group completed a didactic intervention and completed the questionnaire at the end of the quarter ∼15 weeks later (Time 2). Main Outcome Measure(s): The main outcome measure was a 34-item questionnaire designed to examine knowledge, experience, and perceived expertise using true-false items, scenarios, Likert-scaled items, and open-ended questions derived from existing evidence and current literature. The scores from the knowledge, perceived expertise, and experience items served as dependent variables. Results: No statistically significant interaction between groups existed on knowledge scores after the didactic intervention (P = .10). Statistically significant interactions existed between group and time for both perceived expertise (F1,33 = 86.38, P ≤ .001) and experience (F1,33 = 14.2, P ≤ .001) with the athletic training student group demonstrating significant increases in scores over time. There was a statistically significant correlation between the number of concussions evaluated and perceived expertise (r2 = 0.630, P ≤ .001). Conclusions: Educators need to implement the best educational techniques to maximize knowledge attainment and perceived expertise. While it appears that clinical experience may supersede didactic education, a combination of both will encourage higher-level thinking and implementation.

Academic Involvement and the Nontraditional Student

This study expands upon previous research into academic involvement, which focused on youthful full-time university students, by surveying 213 adult part-time students. Two academic involvement measures were generally inversely related to absenteeism, although unrelated to final grade. However, those scoring high on these measures tend to be women and to have an internal locus of control, a need for achievement, and a strong Protestant work ethic. This research suggests that academic involvement is a valuable concept whose generalizability extends beyond the “college sophomore.”