ABSTRACTObjectivesTo determine the presence of a set of pre-specified traditional and progressive criteria used to assess scientists for promotion and tenure in faculties of biomedical sciences among universities worldwide.DesignCross-sectional study.SettingNot applicable.Participants170 randomly selected universities from the Leiden Ranking of world universities list were considered.Main outcome measuresTwo independent reviewers searched for all guidelines applied when assessing scientists for promotion and tenure for institutions with biomedical faculties. Where faculty-level guidelines were not available, institution-level guidelines were sought. Available documents were reviewed and the presence of 5 traditional (e.g., number of publications) and 7 progressive (e.g., data sharing) criteria was noted in guidelines for assessing assistant professors, associate professors, professors, and the granting of tenure.ResultsA total of 146 institutions had faculties of biomedical sciences with 92 having eligible guidelines available to review. Traditional criteria were more commonly reported than progressive criteria (t(82)= 15.1, p= .001). Traditional criteria mentioned peer-reviewed publications, authorship order, journal impact, grant funding, and national or international reputation in 95%, 37%, 28%, 67%, and 48% of the guidelines, respectively. Conversely, among progressive criteria only citations (any mention in 26%) and accommodations for extenuating circumstances (37%) were relatively commonly mentioned; while there was rare mention of alternative metrics for sharing research (2%) and data sharing (1%), and 3 criteria (publishing in open access mediums, registering research, and adhering to reporting guidelines) were not found in any institution reviewed. We observed notable differences across continents on whether guidelines are accessible or not (Australia 100%, North America 97%, Europe 50%, Asia 58%, South America 17%), and more subtle differences on the use of specific criteria.ConclusionsThis study demonstrates that the current evaluation of scientists emphasizes traditional criteria as opposed to progressive criteria. This may reinforce research practices that are known to be problematic while insufficiently supporting the conduct of better-quality research and open science. Institutions should consider incentivizing progressive criteria.RegistrationOpen Science Framework (https://osf.io/26ucp/)What is already known on this topicAcademics tailor their research practices based on the evaluation criteria applied within their academic institution.Ensuring that biomedical researchers are incentivized by adhering to best practice guidelines for research is essential given the clinical implications of this work.While changes to the criteria used to assess professors and confer tenure have been recommended, a systematic assessment of promotion and tenure criteria being applied worldwide has not been conducted.What this study addsAcross countries, university guidelines focus on rewarding traditional research criteria (peer-reviewed publications, authorship order, journal impact, grant funding, and national or international reputation).The minimum requirements for promotion and tenure criteria are predominantly objective in nature, although several of them are inadequate measures to assess the impact of researchers.Developing and evaluating more appropriate, progressive indicators of research may facilitate changes in the evaluation practices for rewarding researchers.
Creating SPACE to evolve academic assessment
