The use of bibliometrics in assessments of social scientists

The paper is focused on practical advice for the use of bibliometrics in research assessment in the social sciences. Guidelines are presented from three official sources of advice with a particular focus on individual-level assessments of applications for positions, promotions, and external funding. General problems with applying bibliometrics in evaluations of the social sciences are also discussed, as well as the specific problems with using the Journal Impact Factor and the H-Index. The conclusion is not that bibliometrics should be avoided in research assessment of social scientists. Used with care and competence, bibliometrics can be a valuable extra source of information, but not replace judgement in research evaluation.

Peer Review Metrics and their influence on the Journal Impact

The manuscript processing timeline, a necessary facet of the publishing process, varies from journal to journal, and its influence on the journal impact needs to be studied. The current research looks into the correlation between the ‘Peer Review Metrics’ (submission to first editorial decision; submission to first post-review decision and submission to accept) and the ‘Journal Impact Data’ (2-year Impact Factor; 5-year Impact Factor; Immediacy Index; Eigenfactor Score and Article Influence Score). The data related to ‘Peer Review Metrics’ (submission to first editorial decision; submission to first post-review decision and submission to accept) and ‘Journal Impact Data’ (2-year Impact Factor; 5-year Impact Factor; Immediacy Index; Eigenfactor Score and Article Influence Score) were downloaded from the ‘Nature Research’ journals website https://www.nature.com/nature-portfolio/about/journal-metrics . Accordingly, correlations were drawn between the ‘Peer Review Metrics’ and the ‘Journal Impact Data’. If the time from ‘submission to first editorial decision’ decreases, the ‘Journal Impact Data’ increases and vice versa. However, an increase or decrease in the time from ‘submission to first editorial decision’ does not affect the ‘Eigenfactor Score’ of the journal and vice versa. An increase or decrease in the time from ‘submission to first post-review decision’ does not affect any ‘Journal Impact Data’ and vice versa. If the time from ‘submission to acceptance’ increases, the ‘Journal Impact Data’ (2-year Impact Factor, 5-year Impact Factor, Immediacy Index and Article Influence Score) also increases, and if the time from ‘submission to acceptance’ decreases, so will the ‘Journal Impact Data’. However, an increase or decrease in the time from ‘submission to acceptance’ does not affect the ‘Eigenfactor Score’ of the journal and vice versa. The study will act as a ready reference tool for the scholars to select the most appropriate submitting platforms for their scholarly endeavours. Furthermore, the performance and evaluative indicators responsible for a journal’s overall research performance can also be understood from a micro-analytical view, which will help the researchers select appropriate journals for their future scholarly submissions. Lengthy publication timelines are a big problem for the researchers because they are not able to get the credit for their research on time. Since the study validates a relationship between the ‘Peer Review Metrics’ and ‘Journal Impact Data’, the findings will be of great help in making an appropriate journal’s choice. The study can be an eye opener for the journal administrators who vocalise a speed-up publication process by enhancing certain areas of publication timeline. The study is the first of its kind that correlates the ‘Peer Review Metrics’ of the journals and the ‘Journal Impact Data’. The study’s findings are limited to the data retrieved from the ‘Nature Research’ journals and cannot be generalised to the full score of journals. The study can be extended across other publishers to generalise the findings. Even the articles’ early access availability concerning ‘Peer Review Metrics’ of the journals and the ‘Journal Impact Data’ can be studied.

Top 50 medical journals from Balkan countries: A bibliometric analysis, 2000–2020

Background: Science Citation Index Expanded (SCIE) is one of the most important indexes that medical journals aspire to be covered by. Currently, SCIE indexes 14,840 peer-reviewed journals across 178 disciplines. Among these journals are 3445 medical journals, divided into more than 40 subject categories. Objectives: To reveal the impact and contribution of medical journals from Balkan countries through the Journal Impact Factor of those journals, the number of articles published by them, and the number of times those articles have been cited. Methods: Balkan countries are countries that fall or fully or partly within the Balkan peninsula. All medical journals from those countries listed in the SCIE were ranked based on cumulative citations between 2000 and 2020. Among them, the top 50 journals in terms of cumulative citations were chosen for the study, which analysed the data on 129,259 research articles and reviews that covered 27 different subject categories within the broad field of medicine. The countries were Bosnia & Herzegovina, Croatia, Greece, North Macedonia, Romania, Serbia, Slovenia, and Turkey Results: The top 50 journals included those published from eight Balkan countries. Turkey had the most journals (21) in the Web of Science (WoS) and Greece had 13 but, when ranked in terms of the number of journals in WoS per million people, Croatia topped the list, with 1.22 journals per million of its population, followed by Greece (1.21 journals). The top-cited journals were Anticancer Research (206,226 citations), International Journal of Oncology (171,654), Oncology Reports (157,467), Molecular Medicine Reports (82,009), and Oncology Letters (69,161). Oncology was the most cited subject category and Croatia, the country with maximum interaction with other Balkan countries, that is, papers in Croatian journals cited journals published from the maximum number of Balkan counties. Conclusion: The study provides insights into the last two decades of progress in academic publishing and in the performances of medical journals published from Balkan countries.

Scientific Publishing Differences between Sexes in Cardiology, Cardiac, and General Surgery

Background Research and scientific publications are important for the advancement of science but also for the individual career development. While women have become the majority of students in medicine we do not know about female presence and scientific activity in cardiac surgery. We thus aimed to compare scientific output of women and men in German University departments of cardiac surgery with cardiology departments focusing on the same organ and surgical departments not addressing the heart (general surgery) of 34 universities in Germany. Methods Personnel working at University departments was identified on the institutions internet homepage in 2014. Publications in 2011 to 2013 on PubMed were identified. Author and manuscript characteristics were determined. Results A lower proportion of women were working in cardiac surgery (25%) compared with cardiology (35%) and general (32%) surgery independent of executive function or academic degree. Scientifically active women published fewer manuscripts per capita than men both, in total and as first author. Additionally, the mean and the cumulated journal impact factor of the journals chosen was lower for women compared with men in cardiology but not in cardiac and general surgery. Conclusion We conclude that the differences in scientific activity between women and men are more pronounced in cardiology compared with general and cardiac surgery, indicating that a higher proportion of women in a field does not result in reduced differences between sexes. The low number of women together with the lower number of manuscripts in cardiac surgery may render the appointment of women as directors challenging.

PP118 A Survival Analysis Of The Lag Times In The Publication Of Network Meta-Analyses

IntroductionThe use of inconsistent and outdated information may significantly compromise healthcare decision-making. We aimed to assess the extent of lag times in the publication and indexing of network meta-analyses (NMAs).MethodsSearches for NMAs on drug interventions were performed in PubMed (May 2020). Lag times were measured as the time between the last systematic search and the date of the article's submission, acceptance, online publication, indexing, and Medical Subject Heading (MeSH) allocation. Correlations between lag times and time trends were calculated by means of Spearman's rank correlation coefficient. Time-to-event analyses were performed considering independent variables such as geographical origin, journal impact factor, Scopus CiteScore, and open access status.ResultsWe included 1,245 NMAs. The median time from last search to article submission and publication was 6.8 months and 11.6 months, respectively. Only five percent of authors updated their literature searches after submission. There was a very slight decreasing historical trend for acceptance (r =−0.087; p = 0.01), online publication (r =−0.08; p = 0.008), and indexing lag times (r =−0.080; p = 0.007). Journal impact factor influenced the MeSH allocation process (log-rank p = 0.02). Slight differences were observed for acceptance, online publication, and indexing lag times when comparing open access and subscription journals.ConclusionsAuthors need to update their literature searches before submission to reduce evidence production time. Peer reviewers and editors should ensure that authors comply with NMA standards and encourage the development of living meta-analyses.

Medidas insuficientes para un cambio en la evalua-ción de la investigación en España: glosando las nuevas directrices de la ANECA

En una carta publicada hace seis meses (Delgado-López-Cózar, et al., 2021), pocas semanas después de que la Agencia Española de Evaluación se adhiriera a la declaración dora (dora, 2012), hacíamos un llamamiento a las autoridades científicas españolas para que abandonaran las políticas de evaluación de la investigación basadas en un uso excesivo e indiscriminado de los indicadores bibliométricos —especialmente el Journal Impact Factor (jif)—, para valorar el desempeño individual de los académicos. En concreto se les animaba a «que suscriban y cumplan con la dora y sigan las recomendaciones en el uso de indicadores bibliométricos señalados en el Manifiesto Leiden» (Hicks et al., 2015). Una mesa redonda organizada en el mes de julio con participación de distintos agentes del sistema español de i+d+i coincidía en la necesidad de afrontar ese cambio de forma urgente.

Informed consent in pragmatic trials: results from a survey of trials published 2014–2019

ObjectivesTo describe reporting of informed consent in pragmatic trials, justifications for waivers of consent and reporting of alternative approaches to standard written consent. To identify factors associated with (1) not reporting and (2) not obtaining consent.MethodsSurvey of primary trial reports, published 2014–2019, identified using an electronic search filter for pragmatic trials implemented in MEDLINE, and registered in ClinicalTrials.gov.ResultsAmong 1988 trials, 132 (6.6%) did not include a statement about participant consent, 1691 (85.0%) reported consent had been obtained, 139 (7.0%) reported a waiver and 26 (1.3%) reported consent for one aspect (eg, data collection) but a waiver for another (eg, intervention). Of the 165 trials reporting a waiver, 76 (46.1%) provided a justification. Few (53, 2.9%) explicitly reported use of alternative approaches to consent. In multivariable logistic regression analyses, lower journal impact factor (p=0.001) and cluster randomisation (p<0.0001) were significantly associated with not reporting on consent, while trial recency, cluster randomisation, higher-income country settings, health services research and explicit labelling as pragmatic were significantly associated with not obtaining consent (all p<0.0001).DiscussionNot obtaining consent seems to be increasing and is associated with the use of cluster randomisation and pragmatic aims, but neither cluster randomisation nor pragmatism are currently accepted justifications for waivers of consent. Rather than considering either standard written informed consent or waivers of consent, researchers and research ethics committees could consider alternative consent approaches that may facilitate the conduct of pragmatic trials while preserving patient autonomy and the public’s trust in research.

Introducing a bibliometric index based on factor analysis

This work applies a factor analysis with VARIMAX rotation to develop a bibliometric indicator, named the Weighted Factor Index, in order to derive a new classification for journals belonging to a certain category, alternative to the one provided by the Journal Impact Factor. For this, 16 metrics from three different databases (Web of Science, Scopus and SCImago Journal Rank) are considered. The Weighed Factor Index entails the advantage of incorporating and summarizing information from all the indicators; so as to test its performance, it was applied to rank journals belonging to the category Information Science & Library Science.

Characteristics and Research Techniques Associated with the Journal Impact Factor and Other Key Metrics in Pharmacology Journals

In the present age, there is intense pressure on researchers to publish their research in ‘high-impact factor’ journals. It would be interesting to understand the trend of research publications in the field of pharmacology by exploring the characteristics of research articles, including research techniques, in relation to the journal’s key bibliometrics, particularly journal impact factor (JIF), the seemingly most mentioned metric. This study aimed to determine the characteristics and research techniques in relation to research articles in pharmacology journals with higher or lower JIF values. A cross-sectional study was conducted on primary research journals under the ‘Pharmacology and Pharmacy’ category. Analysis of 768 original research articles across 32 journals (with an average JIF of 2.565 ± 0.887) demonstrated that research studies involving molecular techniques, in vivo experiments on animals, and bioinformatics and computational modeling were significantly associated with a higher JIF value of the journal in which such contributions were published. Our analysis suggests that research studies involving such techniques/approaches are more likely to be published in higher-ranked pharmacology journals.