scholarly journals Evaluating cancer research impact: lessons and examples from existing reviews on approaches to research impact assessment

2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Catherine R. Hanna ◽  
Kathleen A. Boyd ◽  
Robert J. Jones
Keyword(s):  
Cancer Research ◽  
Impact Assessment ◽  
Impact Evaluation ◽  
Grey Literature ◽  
Real Life ◽  
Research Impact ◽  
Review Articles ◽  
Research Impact Assessment ◽  
Literature Reviews ◽  
The Impact

Abstract Background Performing cancer research relies on substantial financial investment, and contributions in time and effort from patients. It is therefore important that this research has real life impacts which are properly evaluated. The optimal approach to cancer research impact evaluation is not clear. The aim of this study was to undertake a systematic review of review articles that describe approaches to impact assessment, and to identify examples of cancer research impact evaluation within these reviews. Methods In total, 11 publication databases and the grey literature were searched to identify review articles addressing the topic of approaches to research impact assessment. Information was extracted on methods for data collection and analysis, impact categories and frameworks used for the purposes of evaluation. Empirical examples of impact assessments of cancer research were identified from these literature reviews. Approaches used in these examples were appraised, with a reflection on which methods would be suited to cancer research  impact evaluation going forward. Results In total, 40 literature reviews were identified. Important methods to collect and analyse data for impact assessments were surveys, interviews and documentary analysis. Key categories of impact spanning the reviews were summarised, and a list of frameworks commonly used for impact assessment was generated. The Payback Framework was most often described. Fourteen examples of impact evaluation for cancer research were identified. They ranged from those assessing the impact of a national, charity-funded portfolio of cancer research to the clinical practice impact of a single trial. A set of recommendations for approaching cancer research impact assessment was generated. Conclusions Impact evaluation can demonstrate if and why conducting cancer research  is worthwhile. Using a mixed methods, multi-category assessment organised within a framework, will provide a robust evaluation, but the ability to perform this type of assessment may be constrained by time and resources. Whichever approach is used, easily measured, but inappropriate metrics should be avoided. Going forward, dissemination of the results of cancer research impact assessments will allow the cancer research community to learn how to conduct these evaluations.

Introduction: critical reflections on research impact

2020 ◽  
pp. 1-12
Author(s):  
Katherine E. Smith ◽  
Justyna Bandola-Gill ◽  
Nasar Meer ◽  
Ellen Stewart ◽  
Richard Watermeyer
Keyword(s):  
Impact Assessment ◽  
Empirical Investigation ◽  
Research Impact ◽  
Research Impact Assessment ◽  
The Uk ◽  
The Impact ◽  
Critical Reflections

This chapter briefly explains what we mean by ‘the impact agenda’ and what the UK approach to research impact assessment involves. This chapter also makes the case for why an empirical investigation of the recent changes associated with research impact assessment is required and provides key definitions and an overview of the rest of the book.

A Study on Methodologies for Research Impact Assessment: Responses of the UK Research Councils to the Warry Report

2010 ◽  
Vol 24 (5) ◽  
pp. 393-397 ◽  
Author(s):  
André Luiz de Campos
Keyword(s):  
Impact Assessment ◽  
Research Funding ◽  
Research Impact ◽  
Policy Makers ◽  
Impact Studies ◽  
Research Impact Assessment ◽  
The Uk ◽  
Research Councils ◽  
The Impact

The experience of the UK Research Councils in assessing the impacts of their research funding is discussed, including a report on the findings of research which reviewed the impact studies implemented by the Research Councils. The response of the Councils to the challenge of demonstrating the impacts of their funding and the main methodologies used are presented and the implications of both for the Research Councils and policy makers elsewhere are outlined.

The Impact Agenda

2020 ◽  
Author(s):  
Katherine Smith ◽  
Justyna Bandola-Gill ◽  
Nasar Meer ◽  
Ellen Stewart ◽  
Richard Watermeyer
Keyword(s):  
Impact Assessment ◽  
Academic Research ◽  
Research Impact ◽  
Empirical Examination ◽  
Societal Benefit ◽  
Wide Range ◽  
Research Impact Assessment ◽  
Academic Practices ◽  
The Uk ◽  
The Impact

As international interest in promoting and assessing the impact of research grows, this book examines the ensuing controversies, consequences and challenges. It places a particular emphasis on learning from experiences in the UK, since this is the country at the forefront of a range of new approaches to incentivising, monitoring and rewarding research impact achievements. The book aims to understand the origins and rationale for these changes and to critically assess their consequences for academic practice. Combining a review of existing literature with a range of new qualitative data (from interviews, focus groups and documentary analysis), The Impact Agenda is unique in providing a comprehensive, cross-disciplinary empirical examination of the ways in which various forms of research impact assessment are shaping academic practices. Although the primary focus of the book is on the UK, the book also considers the different approaches that other countries with an interest in research impact are taking (notably Australia, Canada and the Netherlands). While noting the benefits that the increasing emphasis on outward facing work is bringing, the book draws attention to a wide range of challenges and controversies associated with research impact assessment and, in particular, with the UK’s chosen approach. It concludes by using the insights in the book to propose an alternative, more theoretically robust approach to incentivising and rewarding efforts to undertake and use academic research for societal benefit.

scholarly journals Engaging with research impact assessment for an environmental science case study

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Kirstie A. Fryirs ◽  
Gary J. Brierley ◽  
Thom Dixon
Keyword(s):  
Impact Assessment ◽  
Environmental Science ◽  
Academic Research ◽  
Research Impact ◽  
Research Field ◽  
Rating Systems ◽  
Research Impact Assessment ◽  
Impact Pathway ◽  
The Impact

Abstract Impact assessment is embedded in many national and international research rating systems. Most applications use the Research Impact Pathway to track inputs, activities, outputs and outcomes of an invention or initiative to assess impact beyond scholarly contributions to an academic research field (i.e., benefits to environment, society, economy and culture). Existing approaches emphasise easy to attribute ‘hard’ impacts, and fail to include a range of ‘soft’ impacts that are less easy to attribute, yet are often a dominant part of the impact mix. Here, we develop an inclusive 3-part impact mapping approach. We demonstrate its application using an environmental initiative.

Missiological research: Making impact or missing the mark?

2021 ◽  
pp. 009182962110021
Author(s):  
Hae-Won Kim
Keyword(s):  
Impact Assessment ◽  
Research Impact ◽  
Societal Impact ◽  
Conceptual Frameworks ◽  
Research To Practice ◽  
Research Knowledge ◽  
Academic Impact ◽  
Research Impact Assessment ◽  
Key Concepts

This article explores ways of conceptualizing research impact and its assessment in the context of missiological research. Can it be assumed that there is a link between missiological research knowledge and research impact on mission practice and practitioners? First, the author defines and discusses some key concepts – research impact, impact assessment, academic impact and societal impact – as well as conceptual frameworks of and approaches to research impact assessment. The author then begins to conceptualize a framework for linking research to practice in missiological research which can be further developed into a framework for research impact assessment.

scholarly journals Adding value to the use of Health Impact Assessment and social return on investment technologies

2020 ◽  
Vol 30 (Supplement_5) ◽  
Author(s):  
L Green ◽  
K Ashton ◽  
M Dyakova ◽  
L Parry-Williams
Keyword(s):  
Impact Assessment ◽  
Return On Investment ◽  
Health Impact Assessment ◽  
Grey Literature ◽  
Health Impact ◽  
Social Value ◽  
Decision Makers ◽  
Structured Interviews ◽  
Social Return On Investment ◽  
The Impact

Abstract Health Impact Assessment (HIA) and Social Return on Investment (SROI) are beneficial public health methodologies that assess potential effects on health including social, economic and environmental factors and have synergies in their approaches. This paper explores how HIA and SROI can complement each other to capture and account for the impact and social value of an assessed intervention or policy. A scoping review of academic and grey literature was undertaken to identify case studies published between January 1996 and April 2019 where HIA and SROI methods have been used to complement each other. Semi-structured interviews were carried out with nine international experts from a range of regulatory/legislative contexts to gain a better understanding of past experiences and expertise of both HIA and SROI. A thematic analysis was undertaken on the data collected. The review identified two published reports which outline when HIA and SROI have both been used to assess the same intervention. Interview results suggest that both methods have strengths as standalone processes i.e. HIAs are well-structured in their approach, assessing health in its broadest context and SROI can add value by monetizing social value as well as capturing social/environmental impact. Similarities of the two methods were identified i.e. a strong emphasis on stakeholder engagement and common shared principles. When questioned how the two methods could complement each other in practice, the results indicate the benefits of using HIA to explore initial impact, and as a platform on which to build SROI to monetarize social value. HIA and SROI methodologies have cross-over. The research suggests potential benefits when used in tandem or combining the methods to assess impact and account for health and social value. Innovative work is now being carried out in Wales to understand the implications of this in practice and to understand how the results of the two methods could be used by decision-makers. Key messages HIA and SROI methods can be used in tandem to capture both the health impact and social value of policies and proposed interventions. HIA and SROI when used together can provide valuable information to inform decision makers around the health impact and social value of proposed policies and interventions.

scholarly journals Evidence-based Nomenclature and Taxonomy of Research Impact Indicators

2020 ◽  
Vol 5 (3) ◽  
pp. 33-56 ◽  
Author(s):  
Mudassar Arsalan ◽  
Omar Mubin ◽  
Abdullah Al Mahmud
Keyword(s):  
Policy Development ◽  
Research Impact ◽  
Development Economic ◽  
Evidence Based ◽  
Novel Approach ◽  
Scientific Disciplines ◽  
Research Impact Assessment ◽  
Impact Indicators ◽  
Environment Sustainability ◽  
The Impact

AbstractPurposeThis study aims to classify research impact indicators based on their characteristics and scope. A concept of evidence-based nomenclature of research impact (RI) indicator has been introduced for generalization and transformation of scope.Design/methodology/approchLiterature was collected related to the research impact assessment. It was categorized in conceptual and applied case studies. One hundred and nineteen indicators were selected to prepare classification and nomenclature. The nomenclature was developed based on the principle—“every indicator is a contextual-function to explain the impact”. Every indicator was disintegrated into three parts, i.e. Function, Domain, and Target Areas.FindingsThe main functions of research impact indicators express improvement (63%), recognition (23%), and creation/development (14%). The focus of research impact indicators in literature is more towards the academic domain (59%) whereas the environment/sustainability domain is least considered (4%). As a result, research impact related to the research aspects is felt the most (29%). Other target areas include system and services, methods and procedures, networking, planning, policy development, economic aspects and commercialisation, etc.Research limitationsThis research applied to 119 research impact indicators. However, the inclusion of additional indicators may change the result.Practical implicationsThe plausible effect of nomenclature is a better organization of indicators with appropriate tags of functions, domains, and target areas. This approach also provides a framework of indicator generalization and transformation. Therefore, similar indicators can be applied in other fields and target areas with modifications.Originality/valueThe development of nomenclature for research impact indicators is a novel approach in scientometrics. It is developed on the same line as presented in other scientific disciplines, where fundamental objects need to classify on common standards such as biology and chemistry.

scholarly journals Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations

2020 ◽  
Author(s):  
Brian M Belcher ◽  
Karl Hughes
Keyword(s):  
Impact Assessment ◽  
Agricultural Research ◽  
Conservation Status ◽  
Research Impact ◽  
Research Field ◽  
Research Quality ◽  
Research Approach ◽  
Quasi Experimental ◽  
Systems Transformation ◽  
The Impact

Abstract Researchers and research organizations are under increasing pressure to demonstrate that their work contributes to positive change and helps solve pressing societal challenges. There is a simultaneous trend towards more engaged transdisciplinary research that is complexity-aware and appreciates that change happens through systems transformation, not only through technological innovation. Appropriate evaluation approaches are needed to evidence research impact and generate learning for continual improvement. This is challenging in any research field, but especially for research that crosses disciplinary boundaries and intervenes in complex systems. Moreover, evaluation challenges at the project scale are compounded at the programme scale. The Forest, Trees and Agroforestry (FTA) research programme serves as an example of this evolution in research approach and the resulting evaluation challenges. FTA research is responding to the demand for greater impact with more engaged research following multiple pathways. However, research impact assessment in the CGIAR (Consultative Group on International Agricultural Research) was developed in a technology-centric context where counterfactual approaches of causal inference (experimental and quasi-experimental) predominate. Relying solely on such approaches is inappropriate for evaluating research contributions that target policy and institutional change and systems transformation. Instead, we propose a multifaceted, multi-scale, theory-based evaluation approach. This includes nested project- and programme-scale theories of change (ToCs); research quality assessment; theory-based outcome evaluations to empirically test ToCs and assess policy, institutional, and practice influence; experimental and quasi-experimental impact of FTA-informed ‘large n’ innovations; ex ante impact assessment to estimate potential impacts at scale; and logically and plausibly linking programme-level outcomes to secondary data on development and conservation status.

scholarly journals 14 Starting a collaborative research impact assessment: a tale of two research funders

2019 ◽  
Author(s):  
Kay Lakin ◽  
Sanjay Thakrar ◽  
Claire Vaughan ◽  
Sabine Best ◽  
Laura Chatland ◽  
...  
Keyword(s):  
Impact Assessment ◽  
Collaborative Research ◽  
Research Impact ◽  
Research Impact Assessment
Sign in / Sign up

Export Citation Format

Share Document