scholarly journals Can altmetrics reflect societal impact considerations?: Exploring the potential of altmetrics in the context of a sustainability science research center

2020 ◽  
pp. 1-18 ◽  
Author(s):  
Omar Kassab ◽  
Lutz Bornmann ◽  
Robin Haunschild
Keyword(s):  
Sustainability Science ◽  
Expert Panel ◽  
Research Evaluation ◽  
Research Output ◽  
Science Research ◽  
Research Proposal ◽  
Research Center ◽  
Societal Impact ◽  
Publicly Funded ◽  
Ex Ante

Societal impact considerations play an increasingly important role in research evaluation. In particular, in the context of publicly funded research, proposal templates commonly include sections to outline strategies for achieving broader impact. Both the assessment of the strategies and the later evaluation of their success are associated with challenges in their own right. Ever since their introduction, altmetrics have been discussed as a remedy for assessing the societal impact of research output. On the basis of data from a research center in Switzerland, this study explores their potential for this purpose. The study is based on the papers (and the corresponding metrics) published by about 200 either accepted or rejected applicants for funding by the Competence Center Environment and Sustainability (CCES). The results of the study seem to indicate that altmetrics are not suitable for reflecting the societal impact of research that was considered: The metrics do not correlate with the ex ante considerations of an expert panel.

scholarly journals Does public outreach impede research performance? Exploring the ‘researcher’s dilemma’ in a sustainability research center

2019 ◽  
Vol 46 (5) ◽  
pp. 710-720 ◽  
Author(s):  
Omar Kassab
Keyword(s):  
General Public ◽  
Evaluation System ◽  
Career Advancement ◽  
Sustainability Science ◽  
Research Performance ◽  
Science Research ◽  
Research Center ◽  
Public Outreach ◽  
Sustainability Research ◽  
Broad Consensus

Abstract Researchers and universities are increasingly urged to communicate their findings to the general public. Despite the broad consensus about the necessity of this task, researchers are still reluctant to engage in public outreach activities. One major reason is that while being somewhat time consuming, engagement in public outreach is not adequately reflected in the metrics that are relevant for career advancement. The study at hand examines to what extent this dilemma is empirically justified. A series of statistical analyses are carried out on the basis of data from a sustainability science research center in Switzerland. The study comes to the conclusion that research performance is overall positively associated to engagement in public outreach activities. This insight has implications for the academic incentive and evaluation system.

scholarly journals Introducing and testing an advanced quantitative methodological approach for the evaluation of research centers: a case study on sustainability science

2019 ◽  
Vol 29 (2) ◽  
pp. 135-149 ◽  
Author(s):  
Omar Kassab ◽  
Rüdiger Mutz ◽  
Hans-Dieter Daniel
Keyword(s):  
Sustainability Science ◽  
Research Evaluation ◽  
Methodological Approach ◽  
Research Performance ◽  
Three Dimensions ◽  
Research Center ◽  
Research Centers ◽  
Evaluation Approach ◽  
Ex Post ◽  
Global University

Abstract With the growing complexity of societal and scientific problems, research centers have emerged to facilitate the conduct of research beyond disciplinary and institutional boundaries. While they have become firmly established in the global university landscape, research centers raise some critical questions for research evaluation. Existing evaluation approaches designed to assess universities, departments, projects, or individual researchers fail to capture some of the core characteristics of research centers and their participants, including the diversity of the involved researchers, at what point in time they join and leave the research center, or the intensity of their participation. In addressing these aspects, this article introduces an advanced approach for the ex post evaluation of research centers. It builds on a quasi-experimental within-group design, bibliometric analyses, and multilevel statistics to assess average and individual causal effects of research center affiliation on participants along three dimensions of research performance. The evaluation approach is tested with archival data from a center in the field of sustainability science. Against a widely held belief, we find that participation in research centers entails no disadvantages for researchers as far as their research performance is concerned. However, individual trajectories varied strongly.

scholarly journals Making sense of altmetrics: The perceived threats and opportunities for academic identity

2019 ◽  
Vol 46 (4) ◽  
pp. 479-489 ◽  
Author(s):  
Áine Regan ◽  
Maeve Henchion
Keyword(s):  
Knowledge Production ◽  
Research Policy ◽  
Research Evaluation ◽  
Research Output ◽  
Academic Identity ◽  
Research Impact ◽  
Analytical Framework ◽  
Publicly Funded ◽  
Making Sense ◽  
Quantitative Indicators

Abstract How research impact is defined and evaluated is much-debated at research policy level. Offering one avenue for capturing societal research impact, altmetrics are proposed as quantitative indicators providing a measure of the reach and attention that a research output, such as a peer-reviewed paper, is receiving online. Eighty publicly-funded food researchers participated in an online mixed-methods engagement study. The analytical framework of sensemaking was used to explore participants’ views of altmetrics as a threat or opportunity for their perceived professional identities. The identities important to our participants included ensuring rigour and quality in knowledge production; communicating and engaging with non-academic audiences; and bringing about tangible and meaningful changes in society. While an appetite for changes to research evaluation was apparent in our study, altmetrics was perceived to introduce a number of different threats as well as opportunities to the academic identity, which will influence its potential uptake and use.

Towards complexity-sensitive book metrics for scholarly monographs in national databases for research output

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Linda Sīle ◽  
Raf Guns ◽  
Alesia A. Zuccala ◽  
Tim C.E. Engels
Keyword(s):  
Information Science ◽  
Research Evaluation ◽  
Research Output ◽  
Content Type ◽  
Social Sciences And Humanities ◽  
Work Related ◽  
Bibliographic Records ◽  
Bibliographic Data ◽  
The One ◽  
National Databases

PurposeThis study investigates an approach to book metrics for research evaluation that takes into account the complexity of scholarly monographs. This approach is based on work sets – unique scholarly works and their within-work related bibliographic entities – for scholarly monographs in national databases for research output.Design/methodology/approachThis study examines bibliographic records on scholarly monographs acquired from four European databases (VABB in Flanders, Belgium; CROSBI in Croatia; CRISTIN in Norway; COBISS in Slovenia). Following a data enrichment process using metadata from OCLC WorldCat and Amazon Goodreads, the authors identify work sets and the corresponding ISBNs. Next, on the basis of the number of ISBNs per work set and the presence in WorldCat, they design a typology of scholarly monographs: Globally visible single-expression works, Globally visible multi-expression works, Miscellaneous and Globally invisible works.FindingsThe findings show that the concept “work set” and the proposed typology can aid the identification of influential scholarly monographs in the social sciences and humanities (i.e. the Globally visible multi-expression works).Practical implicationsIn light of the findings, the authors outline requirements for the bibliographic control of scholarly monographs in national databases for research output that facilitate the use of the approach proposed here.Originality/valueThe authors use insights from library and information science (LIS) to construct complexity-sensitive book metrics. In doing so, the authors, on the one hand, propose a solution to a problem in research evaluation and, on the other hand, bring to attention the need for a dialogue between LIS and neighbouring communities that work with bibliographic data.

scholarly journals Research Productivity and Citation Impact of Indian institutes of Science Education and Research

2021 ◽  
Vol 41 (6) ◽  
pp. 455-462
Author(s):  
Jyotshna Sahoo ◽  
Sudam Charan Sahu ◽  
Basudev Mohanty
Keyword(s):  
Science Education ◽  
Research Productivity ◽  
Basic Science ◽  
Research Output ◽  
Citation Impact ◽  
Science Research ◽  
Publication Patterns ◽  
Research Outcomes ◽  
Basic Science Research ◽  
Research Publications

The paper’s main objective is to investigate the trends of basic science research in India using a combination of qualitative and quantitative approaches. It examines the publication patterns and impact of research productivity of five basic science institutions, i.e., “Indian Institutes of Science Education and Research” (IISER), namely IISER Kolkata, IISER Pune, IISER Mohali, IISER Bhopal, and IISER Thiruvananthapuram. The research output indexed in the SCOPUS bibliographic database of these five established IISERs was obtained from 2015 to 2019. A total number of 7329 research publications were analysed using various scientometric dimensions. This paper makes a concerted effort to present a comprehensive picture of the assessment of research outcomes at the five older IISERs, which are ostensibly India’s most active and prominent basic science research institutions. The findings reveal that these institutions are accountable for important research outcomes, such as a high number of citations, preferences towards open access (OA) publications, a rise in research publication year over year, a strong author network, a high degree of collaboration, and a high impact in terms of other scientometrics indicators. This paper discusses the findings of the research publications on the position of IISERs in basic sciences research and draws some conclusions about their effects.

scholarly journals Problematizing Digital Research Evaluation using DOIs in Practice-Based Arts, Humanities and Social Science Research

F1000Research ◽  
2015 ◽  
Vol 4 ◽  
pp. 193 ◽  
Author(s):  
Muriel Swijghuisen Reigersberg
Keyword(s):  
Social Science ◽  
Impact Analysis ◽  
Research Evaluation ◽  
Social Science Research ◽  
Research Integrity ◽  
Science Research ◽  
Careful Consideration ◽  
Research Data ◽  
Impact Measurement ◽  
Digital Research

This paper explores emerging practices in research data management in the arts, humanities and social sciences (AHSS). It will do so vis-à-vis current citation conventions and impact measurement for research in AHSS. Case study findings on research data inventoried at Goldsmiths’, University of London will be presented. Goldsmiths is a UK research-intensive higher education institution which specialises in arts, humanities and social science research. The paper’s aim is to raise awareness of the subject-specific needs of AHSS scholars to help inform the design of future digital tools for impact analysis in AHSS. Firstly, I shall explore the definition of research data and how it is currently understood by AHSS researchers. I will show why many researchers choose not to engage with digital dissemination techniques and ORCID. This discussion must necessarily include the idea that practice-based and applied AHSS research are processes which are not easily captured in numerical ‘sets’ and cannot be labelled electronically without giving careful consideration to what a group or data item ‘represents’ as part of the academic enquiry, and therefore how it should be cited and analysed as part of any impact assessment. Then, the paper will explore: the role of the monograph and arts catalogue in AHSS scholarship; how citation practices and digital impact measurement in AHSS currently operate in relation to authorship and how digital identifiers may hypothetically impact on metrics, intellectual property (IP), copyright and research integrity issues in AHSS. I will also show that, if we are to be truly interdisciplinary, as research funders and strategic thinkers say we should, it is necessary to revise the way we think about digital research dissemination. This will involve breaking down the boundaries between AHSS and other types of research.

Sign in / Sign up

Export Citation Format

Share Document