Introduction: critical reflections on research impact

The Uk ◽

The Impact ◽

Critical Reflections

This chapter briefly explains what we mean by ‘the impact agenda’ and what the UK approach to research impact assessment involves. This chapter also makes the case for why an empirical investigation of the recent changes associated with research impact assessment is required and provides key definitions and an overview of the rest of the book.

A Study on Methodologies for Research Impact Assessment: Responses of the UK Research Councils to the Warry Report

Industry and Higher Education ◽

10.5367/ihe.2010.0006 ◽

2010 ◽

Vol 24 (5) ◽

pp. 393-397 ◽

Cited By ~ 1

Author(s):

André Luiz de Campos

Keyword(s):

Impact Assessment ◽

Research Funding ◽

Research Impact ◽

Policy Makers ◽

Impact Studies ◽

The Uk ◽

Research Councils ◽

The experience of the UK Research Councils in assessing the impacts of their research funding is discussed, including a report on the findings of research which reviewed the impact studies implemented by the Research Councils. The response of the Councils to the challenge of demonstrating the impacts of their funding and the main methodologies used are presented and the implications of both for the Research Councils and policy makers elsewhere are outlined.

The Impact Agenda

10.1332/policypress/9781447339854.001.0001 ◽

2020 ◽

Author(s):

Katherine Smith ◽

Justyna Bandola-Gill ◽

Nasar Meer ◽

Ellen Stewart ◽

Richard Watermeyer

Keyword(s):

Impact Assessment ◽

Academic Research ◽

Research Impact ◽

Empirical Examination ◽

Societal Benefit ◽

Wide Range ◽

Academic Practices ◽

The Uk ◽

As international interest in promoting and assessing the impact of research grows, this book examines the ensuing controversies, consequences and challenges. It places a particular emphasis on learning from experiences in the UK, since this is the country at the forefront of a range of new approaches to incentivising, monitoring and rewarding research impact achievements. The book aims to understand the origins and rationale for these changes and to critically assess their consequences for academic practice. Combining a review of existing literature with a range of new qualitative data (from interviews, focus groups and documentary analysis), The Impact Agenda is unique in providing a comprehensive, cross-disciplinary empirical examination of the ways in which various forms of research impact assessment are shaping academic practices. Although the primary focus of the book is on the UK, the book also considers the different approaches that other countries with an interest in research impact are taking (notably Australia, Canada and the Netherlands). While noting the benefits that the increasing emphasis on outward facing work is bringing, the book draws attention to a wide range of challenges and controversies associated with research impact assessment and, in particular, with the UK’s chosen approach. It concludes by using the insights in the book to propose an alternative, more theoretically robust approach to incentivising and rewarding efforts to undertake and use academic research for societal benefit.

The rise of research impact

The Impact Agenda ◽

10.1332/policypress/9781447339854.003.0002 ◽

2020 ◽

pp. 13-26

Author(s):

Katherine E. Smith ◽

Justyna Bandola-Gill ◽

Nasar Meer ◽

Ellen Stewart ◽

Richard Watermeyer

Keyword(s):

Impact Assessment ◽

Research Impact ◽

The Uk ◽

Historical Antecedents

This chapter charts the origins and emergence of the research impact agenda in the UK, noting some of its historical antecedents and international influences. It explores the apparent motivations and rationales that have underpinned the various dimensions of research impact assessment, and the associated expectations that different actors appear to have. The chapter includes a visual summary of key developments in the evolution of research impact assessment and incentivisation in the UK.

Evaluating cancer research impact: lessons and examples from existing reviews on approaches to research impact assessment

Health Research Policy and Systems ◽

10.1186/s12961-020-00658-x ◽

2021 ◽

Vol 19 (1) ◽

Author(s):

Catherine R. Hanna ◽

Kathleen A. Boyd ◽

Robert J. Jones

Keyword(s):

Cancer Research ◽

Impact Assessment ◽

Impact Evaluation ◽

Grey Literature ◽

Real Life ◽

Research Impact ◽

Review Articles ◽

Literature Reviews ◽

Abstract Background Performing cancer research relies on substantial financial investment, and contributions in time and effort from patients. It is therefore important that this research has real life impacts which are properly evaluated. The optimal approach to cancer research impact evaluation is not clear. The aim of this study was to undertake a systematic review of review articles that describe approaches to impact assessment, and to identify examples of cancer research impact evaluation within these reviews. Methods In total, 11 publication databases and the grey literature were searched to identify review articles addressing the topic of approaches to research impact assessment. Information was extracted on methods for data collection and analysis, impact categories and frameworks used for the purposes of evaluation. Empirical examples of impact assessments of cancer research were identified from these literature reviews. Approaches used in these examples were appraised, with a reflection on which methods would be suited to cancer research impact evaluation going forward. Results In total, 40 literature reviews were identified. Important methods to collect and analyse data for impact assessments were surveys, interviews and documentary analysis. Key categories of impact spanning the reviews were summarised, and a list of frameworks commonly used for impact assessment was generated. The Payback Framework was most often described. Fourteen examples of impact evaluation for cancer research were identified. They ranged from those assessing the impact of a national, charity-funded portfolio of cancer research to the clinical practice impact of a single trial. A set of recommendations for approaching cancer research impact assessment was generated. Conclusions Impact evaluation can demonstrate if and why conducting cancer research is worthwhile. Using a mixed methods, multi-category assessment organised within a framework, will provide a robust evaluation, but the ability to perform this type of assessment may be constrained by time and resources. Whichever approach is used, easily measured, but inappropriate metrics should be avoided. Going forward, dissemination of the results of cancer research impact assessments will allow the cancer research community to learn how to conduct these evaluations.

Engaging with research impact assessment for an environmental science case study

Nature Communications ◽

10.1038/s41467-019-12020-z ◽

2019 ◽

Vol 10 (1) ◽

Cited By ~ 2

Author(s):

Kirstie A. Fryirs ◽

Gary J. Brierley ◽

Thom Dixon

Keyword(s):

Impact Assessment ◽

Environmental Science ◽

Academic Research ◽

Research Impact ◽

Research Field ◽

Rating Systems ◽

Impact Pathway ◽

Abstract Impact assessment is embedded in many national and international research rating systems. Most applications use the Research Impact Pathway to track inputs, activities, outputs and outcomes of an invention or initiative to assess impact beyond scholarly contributions to an academic research field (i.e., benefits to environment, society, economy and culture). Existing approaches emphasise easy to attribute ‘hard’ impacts, and fail to include a range of ‘soft’ impacts that are less easy to attribute, yet are often a dominant part of the impact mix. Here, we develop an inclusive 3-part impact mapping approach. We demonstrate its application using an environmental initiative.

Missiological research: Making impact or missing the mark?

Missiology An International Review ◽

10.1177/00918296211002137 ◽

2021 ◽

pp. 009182962110021

Author(s):

Hae-Won Kim

Keyword(s):

Impact Assessment ◽

Research Impact ◽

Societal Impact ◽

Conceptual Frameworks ◽

Research To Practice ◽

Research Knowledge ◽

Academic Impact ◽

Key Concepts

This article explores ways of conceptualizing research impact and its assessment in the context of missiological research. Can it be assumed that there is a link between missiological research knowledge and research impact on mission practice and practitioners? First, the author defines and discusses some key concepts – research impact, impact assessment, academic impact and societal impact – as well as conceptual frameworks of and approaches to research impact assessment. The author then begins to conceptualize a framework for linking research to practice in missiological research which can be further developed into a framework for research impact assessment.

Evidence-based Nomenclature and Taxonomy of Research Impact Indicators

Journal of Data and Information Science ◽

10.2478/jdis-2020-0018 ◽

2020 ◽

Vol 5 (3) ◽

pp. 33-56 ◽

Cited By ~ 1

Author(s):

Mudassar Arsalan ◽

Omar Mubin ◽

Abdullah Al Mahmud

Keyword(s):

Policy Development ◽

Research Impact ◽

Development Economic ◽

Evidence Based ◽

Novel Approach ◽

Scientific Disciplines ◽

Impact Indicators ◽

Environment Sustainability ◽

AbstractPurposeThis study aims to classify research impact indicators based on their characteristics and scope. A concept of evidence-based nomenclature of research impact (RI) indicator has been introduced for generalization and transformation of scope.Design/methodology/approchLiterature was collected related to the research impact assessment. It was categorized in conceptual and applied case studies. One hundred and nineteen indicators were selected to prepare classification and nomenclature. The nomenclature was developed based on the principle—“every indicator is a contextual-function to explain the impact”. Every indicator was disintegrated into three parts, i.e. Function, Domain, and Target Areas.FindingsThe main functions of research impact indicators express improvement (63%), recognition (23%), and creation/development (14%). The focus of research impact indicators in literature is more towards the academic domain (59%) whereas the environment/sustainability domain is least considered (4%). As a result, research impact related to the research aspects is felt the most (29%). Other target areas include system and services, methods and procedures, networking, planning, policy development, economic aspects and commercialisation, etc.Research limitationsThis research applied to 119 research impact indicators. However, the inclusion of additional indicators may change the result.Practical implicationsThe plausible effect of nomenclature is a better organization of indicators with appropriate tags of functions, domains, and target areas. This approach also provides a framework of indicator generalization and transformation. Therefore, similar indicators can be applied in other fields and target areas with modifications.Originality/valueThe development of nomenclature for research impact indicators is a novel approach in scientometrics. It is developed on the same line as presented in other scientific disciplines, where fundamental objects need to classify on common standards such as biology and chemistry.

Differential research impact in cancer practice guidelines’ evidence base: lessons from ESMO, NICE and SIGN

ESMO Open ◽

10.1136/esmoopen-2017-000258 ◽

2018 ◽

Vol 3 (1) ◽

pp. e000258 ◽

Cited By ~ 8

Author(s):

Elena Pallari ◽

Anthony W Fox ◽

Grant Lewison

Keyword(s):

Practice Guidelines ◽

Western Europe ◽

Statistical Tests ◽

Geographic Location ◽

Basic Research ◽

Research Impact ◽

Evidence Base ◽

Effective Decision Making ◽

The Uk ◽

BackgroundThis is an appraisal of the impact of cited research evidence underpinning the development of cancer clinical practice guidelines (CPGs) by the professional bodies of the European Society for Medical Oncology (ESMO), the National Institute for Health and Care Excellence (NICE) and the Scottish Intercollegiate Guidelines Network (SIGN).MethodsA total of 101 CPGs were identified from ESMO, NICE and SIGN websites across 13 cancer sites. Their 9486 cited references were downloaded from the Web of Science Clarivate Group database, analysed on Excel (2016) using Visual Basic Application macros and imported onto SPSS (V.24.0) for statistical tests.ResultsESMO CPGs mostly cited research from Western Europe, while the NICE and SIGN ones from the UK, Canada, Australia and Scandinavian countries. The ESMO CPGs cited more recent and basic research (eg, drugs treatment), in comparison with NICE and SIGN CPGs where older and more clinical research (eg, surgery) papers were referenced. This chronological difference in the evidence base is also in line with that ESMO has a shorter gap between the publication of the research and its citation on the CPGs. It was demonstrated that ESMO CPGs report more chemotherapy research, while the NICE and SIGN CPGs report more surgery, with the results being statistically significant.ConclusionsWe showed that ESMO, NICE and SIGN differ in their evidence base of CPGs. Healthcare professionals should be aware of this heterogeneity in effective decision-making of tailored treatments to patients, irrespective of geographic location across Europe.

Academic life in the impact vanguard: the view from knowledge exchange organisations

The Impact Agenda ◽

10.1332/policypress/9781447339854.003.0007 ◽

2020 ◽

pp. 137-160

Author(s):

Katherine E. Smith ◽

Justyna Bandola-Gill ◽

Nasar Meer ◽

Ellen Stewart ◽

Richard Watermeyer

Keyword(s):

Knowledge Translation ◽

Knowledge Exchange ◽

Research Impact ◽

Interview Data ◽

Academic Life ◽

Types Of Knowledge ◽

Knowledge Brokerage ◽

Experimental Subjects ◽

The Uk ◽

This chapter focuses on academics working in university-based groups that have been charged with, and funded to achieve, knowledge translation and research impact. These are, we suggest, academics working at the vanguard of the impact agenda, who we might consider as experimental subjects from whom we can learn. This chapter includes a summary of the types of knowledge brokerage roles and organisations that have been created in the UK and the perceived and stated rationales for these new roles and organisations, and an analysis of interview data providing insights into the perspectives of academics working within two such groups.