Test scores’ robustness to scaling: The scale_transformation command

2021 ◽  
Vol 21 (3) ◽  
pp. 756-771
Author(s):  
Andres Yi Chang
Keyword(s):  
Achievement Gap ◽  
Test Scores ◽  
Large Datasets ◽  
Achievement Gaps ◽  
Social Scientists ◽  
Arbitrary Choice ◽  
Scale Transformations ◽  
Over Time

Social scientists frequently rely on the cardinal comparability of test scores to assess achievement gaps between population subgroups and their evolution over time. This approach has been criticized because of the ordinal nature of test scores and the sensitivity of results to order-preserving transformations that are theoretically plausible. Bond and Lang (2013, Review of Economics and Statistics 95: 1468–1479) document the sensitivity of measured ability to scaling choices and develop a method to assess the robustness of changes in ability over time to scaling choices. In this article, I present the scale_transformation command, which expands the Bond and Lang (2013) method to more general cases and optimizes their algorithm to work with large datasets. The command assesses the robustness of an achievement gap between two subgroups to any arbitrary choice of scale by finding bounds for the original gap estimation. Additionally, it finds scale transformations that are very likely and unlikely to benchmark against the results obtained. Finally, it also allows the user to measure how much gap growth coefficients change when including controls in their specifications.

Are Achievement Gap Estimates Biased by Differential Student Test Effort? Putting an Important Policy Metric to the Test

2018 ◽  
Vol 120 (12) ◽  
pp. 1-26
Author(s):  
James Soland
Keyword(s):  
Achievement Gap ◽  
White Students ◽  
Test Scores ◽  
Achievement Gaps ◽  
Content Mastery ◽  
Race And Gender ◽  
Subgroup Differences ◽  
And Gender ◽  
Background Context ◽  
Differential Test

Background/Context Achievement gaps motivate a range of practices and policies aimed at closing those gaps. Most gaps studies assume that differences in observed test scores across subgroups are measuring differences in content mastery. For such an assumption to hold, students in the subgroups being compared need to be giving similar effort on the test. Studies already show that low test effort is prevalent and biases observed test scores downward. What research does not demonstrate is whether test effort differs by subgroup and, therefore, biases estimates of achievement gaps. Purpose This study examines whether test effort differs by student subgroup, including by race and gender. The sensitivity of achievement gap estimates to any differences in test effort is also considered. Research Design A behavioral proxy for test effort called “rapid guessing” was used. Rapid guessing occurs when students answer a test item so fast, they could not have understood its content. Rates of rapid guessing were compared across subgroups. Then, achievement gaps were estimated unconditional and conditional on measures of rapid guessing. Findings Test effort differs substantially by subgroup, with males rapidly guessing nearly twice as often as females in later grades, and Black students rapidly guessing more often than White students. However, these differences in rapid guessing generally do not impact substantive interpretations of achievement gaps, though basic conclusions about male–female gaps and changes in gaps as students progress through school may change when models account for test effort. Conclusions Although the bias introduced into achievement gap estimates by differential test effort is hard to quantify, results provide an important reminder that test scores reflect achievement only to the extent that students are willing and able to demonstrate what they have learned. Understanding why there are subgroup differences in test effort would likely be useful to educators and is worthy of additional study.

Closing Achievement Gaps: Examining the Roles of School Background and Process

2021 ◽  
pp. 001312452198944
Author(s):  
Huang Wu ◽  
Jianping Shen ◽  
Jessaca Spybrook ◽  
Xingyuan Gao
Keyword(s):  
Achievement Gap ◽  
Teacher Collaboration ◽  
White Students ◽  
Science Achievement ◽  
Linear Models ◽  
Achievement Gaps ◽  
School Level ◽  
Academic Learning ◽  
Closing Achievement Gaps

The purpose of this study was to examine the role of school background and school process in closing achievement gaps between White and non-White students in science. To answer the research questions, a series of two-level hierarchical linear models (HLM) was performed on the fourth-grade U.S. portion of the 2015 Trends in International Mathematics and Science Study (TIMSS) data. Results indicate that (a) the science achievement gap between White and non-White students is 0.21 standard deviation, holding student and school background constant; (b) the science achievement gap varies across schools; (c) none of the school background variables are associated with the achievement gap in a school; and (d) school emphasis on student academic learning is not only associated with higher school-level science achievement, but also with a narrower science achievement gap between White and non-White students. However, teacher collaboration is not associated with school-level science achievement but is associated with a larger achievement gap. Implications, limitations, and recommendations for further research are discussed.

Development of standardized regression-based formulas to assess meaningful cognitive change in early Parkinson’s disease

2020 ◽  
Author(s):  
Hannah L Combs ◽  
Kate A Wyman-Chick ◽  
Lauren O Erickson ◽  
Michele K York
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Test Scores ◽  
Prediction Models ◽  
Early Stage ◽  
Cognitive Change ◽  
Cognitive Test ◽  
Healthy Controls ◽  
Change Over Time ◽  
Over Time

Abstract Objective Longitudinal assessment of cognitive and emotional functioning in patients with Parkinson’s disease (PD) is helpful in tracking progression of the disease, developing treatment plans, evaluating outcomes, and educating patients and families. Determining whether change over time is meaningful in neurodegenerative conditions, such as PD, can be difficult as repeat assessment of neuropsychological functioning is impacted by factors outside of cognitive change. Regression-based prediction formulas are one method by which clinicians and researchers can determine whether an observed change is meaningful. The purpose of the current study was to develop and validate regression-based prediction models of cognitive and emotional test scores for participants with early-stage idiopathic PD and healthy controls (HC) enrolled in the Parkinson’s Progression Markers Initiative (PPMI). Methods Participants with de novo PD and HC were identified retrospectively from the PPMI archival database. Data from baseline testing and 12-month follow-up were utilized in this study. In total, 688 total participants were included in the present study (NPD = 508; NHC = 185). Subjects from both groups were randomly divided into development (70%) and validation (30%) subsets. Results Early-stage idiopathic PD patients and healthy controls were similar at baseline. Regression-based models were developed for all cognitive and self-report mood measures within both populations. Within the validation subset, the predicted and observed cognitive test scores did not significantly differ, except for semantic fluency. Conclusions The prediction models can serve as useful tools for researchers and clinicians to study clinically meaningful cognitive and mood change over time in PD.

scholarly journals The Foundational Lawlessness of the Law Itself: Racial Criminalization & the Punitive Roots of Punishment in America

Daedalus ◽  
2022 ◽  
Vol 151 (1) ◽  
pp. 107-120
Author(s):  
Khalil Gibran Muhammad
Keyword(s):  
Social Construction ◽  
Ethnic Studies ◽  
Brute Force ◽  
Behavioral Models ◽  
Social Scientists ◽  
Basic Assumptions ◽  
The Social ◽  
Legal Structures ◽  
The Law ◽  
Over Time

Abstract Empirical researchers and criminal justice practitioners have generally set aside history in exchange for behavioral models and methodologies that focus primarily on crime itself as the most measurable and verifiable driver of American punitiveness. There are innumerable legal and political questions that have arisen out of these approaches. Everything from the social construction of illegality to the politicization of punishment to the stigmatization of physical identities and social statuses have long called into question the legal structures that underpin what counts as crime and how punishment is distributed. And yet, until quite recently, the question of what history has to offer has mostly been left to historians, historically minded social scientists, critical race and ethnic studies scholars, community and prison-based activists, investigative journalists, and rights advocates. What is at stake is precisely the foundational lawlessness of the law itself. At all times, a White outlaw culture that rewarded brute force and strength of arms against racialized others unsettles basic assumptions about how we are to understand criminalization and punitiveness over time: that is, who has counted as a criminal and to what end has the state used violence or punishment?

Book Burning—American Style

2019 ◽  
pp. 146-169
Author(s):  
Kathy Peiss
Keyword(s):  
Public Libraries ◽  
Entire Body ◽  
Military Government ◽  
Social Scientists ◽  
American Military ◽  
American Style ◽  
Book World ◽  
The Military ◽  
The Relationship ◽  
Over Time

The American military government in Germany faced a particular problem of mass acquisitions tied to postwar occupation policy. The Allies had agreed to purge Nazism from the German book world. The military confiscated countless volumes, sequestering and even destroying them. Bookstores and publishers had been forced to surrender these works. Over time this became an operation to make an entire body of published works inaccessible and unreadable. Communications experts, social scientists, progressive educators, and librarians applied their expertise to achieve this goal. However, when Order No. 4 was issued, requiring the confiscation and destruction of all Nazi material, including books in public libraries, many Americans accused the military of engaging in book burning. The episode reveals tensions over the relationship between reading, freedom, democracy, and the wartime state.

Verbal reasoning test scores and their stability over time

2000 ◽  
Vol 42 (2) ◽  
pp. 167-174 ◽  
Author(s):  
Alison F. Primrose ◽  
Mary Fuller ◽  
Michael Littledyke
Keyword(s):  
Test Scores ◽  
Verbal Reasoning ◽  
Reasoning Test ◽  
Over Time

Road Traffic Accidents in Turkey: Some Historical and Sociological Perspectives

1999 ◽  
Vol 20 ◽  
pp. 107-123
Author(s):  
Theo Nichols ◽  
Fatih Güngör
Keyword(s):  
Traffic Accidents ◽  
Road Traffic ◽  
Risk Compensation ◽  
Road Traffic Accidents ◽  
Public Concern ◽  
Social Scientists ◽  
Systematic Analysis ◽  
Turkish Case ◽  
Over Time

Although road traffic accidents are a matter of considerable public concern in Turkey, they have attracted little systematic analysis from social scientists. This paper seeks to examine accident trends from 1955 to 1995, in terms of changes in fatalities per vehicle as well as fatalities per head of population. It then critically surveys the adequacy and applicability to the Turkish case of theories of motorization and accidents and of risk compensation and homeostasis. Finally, the paper speculates on how accident variation over time in Turkey may relate to wider social and economic processes.

Sign in / Sign up

Export Citation Format

Share Document