NIH peer review: Criterion scores completely account for racial disparities in overall impact scores

Previous research has found that funding disparities are driven by applications’ final impact scores and that only a portion of the black/white funding gap can be explained by bibliometrics and topic choice. Using National Institutes of Health R01 applications for council years 2014–2016, we examine assigned reviewers’ preliminary overall impact and criterion scores to evaluate whether racial disparities in impact scores can be explained by application and applicant characteristics. We hypothesize that differences in commensuration—the process of combining criterion scores into overall impact scores—disadvantage black applicants. Using multilevel models and matching on key variables including career stage, gender, and area of science, we find little evidence for racial disparities emerging in the process of combining preliminary criterion scores into preliminary overall impact scores. Instead, preliminary criterion scores fully account for racial disparities—yet do not explain all of the variability—in preliminary overall impact scores.

Download Full-text

National Institutes of Health Institute and Center Award Rates and Funding Disparities

10.1101/2020.12.27.424490 ◽

2020 ◽

Author(s):

Michael S Lauer

Keyword(s):

African American ◽

Peer Review ◽

National Institutes Of Health ◽

Regression Analyses ◽

Probit Regression ◽

Health Institute ◽

Principal Investigators ◽

Peer Reviewers ◽

Funding Disparities ◽

Topic Choice

A previous report found an association of topic choice with race-based funding disparities among R01 applications submitted to the National Institutes of Health ("NIH") between 2011-2015. The report noted that applications submitted by African American or Black ("AAB") Principal Investigators ("PIs") skewed toward a small number of topics that were less likely to be funded (or "awarded"). It was suggested that the lower award rates may be related to biases of peer reviewers against topics preferred by AAB PIs. However, the previous report did not account for differential funding ecologies among NIH Institutes and Centers ("ICs"). In a re-analysis, I find that 10% of 148 algorithmically-designated topics account for 50% of applications submitted by AAB PIs. These applications on "AAB Preferred" topics are indeed funded at lower rates than applications on other topics, but their peer review outcomes are similar. The lower rate of funding for applications focused on AAB Preferred topics is likely primarily due to their assignment to ICs with lower award rates. In probit regression analyses, I find that topic choice does partially explain race-based funding disparities, but IC-specific award rates explain the disparities to an even greater degree.

Download Full-text

Associations of topic-specific peer review outcomes and institute and center award rates with funding disparities at the National Institutes of Health

eLife ◽

10.7554/elife.67173 ◽

2021 ◽

Vol 10 ◽

Author(s):

Michael S Lauer ◽

Jamie Doyle ◽

Joy Wang ◽

Deepshikha Roychowdhury

Keyword(s):

African American ◽

Peer Review ◽

National Institutes Of Health ◽

Principal Investigators ◽

Peer Reviewers ◽

Funding Disparities ◽

Topic Choice

A previous report found an association of topic choice with race-based funding disparities among R01 applications submitted to the National Institutes of Health (‘NIH’) between 2011 and 2015. Applications submitted by African American or Black (‘AAB’) Principal Investigators (‘PIs’) skewed toward a small number of topics that were less likely to be funded (or ‘awarded’). It was suggested that lower award rates may be related to topic-related biases of peer reviewers. However, the report did not account for differential funding ecologies among NIH Institutes and Centers (‘ICs’). In a re-analysis, we find that 10% of 148 topics account for 50% of applications submitted by AAB PIs. These applications on ‘AAB Preferred’ topics were funded at lower rates, but peer review outcomes were similar. The lower rate of funding for these topics was primarily due to their assignment to ICs with lower award rates, not to peer-reviewer preferences.

Download Full-text