Missiological research: Making impact or missing the mark?

This article explores ways of conceptualizing research impact and its assessment in the context of missiological research. Can it be assumed that there is a link between missiological research knowledge and research impact on mission practice and practitioners? First, the author defines and discusses some key concepts – research impact, impact assessment, academic impact and societal impact – as well as conceptual frameworks of and approaches to research impact assessment. The author then begins to conceptualize a framework for linking research to practice in missiological research which can be further developed into a framework for research impact assessment.

Evaluating cancer research impact: lessons and examples from existing reviews on approaches to research impact assessment

Background Performing cancer research relies on substantial financial investment, and contributions in time and effort from patients. It is therefore important that this research has real life impacts which are properly evaluated. The optimal approach to cancer research impact evaluation is not clear. The aim of this study was to undertake a systematic review of review articles that describe approaches to impact assessment, and to identify examples of cancer research impact evaluation within these reviews. Methods In total, 11 publication databases and the grey literature were searched to identify review articles addressing the topic of approaches to research impact assessment. Information was extracted on methods for data collection and analysis, impact categories and frameworks used for the purposes of evaluation. Empirical examples of impact assessments of cancer research were identified from these literature reviews. Approaches used in these examples were appraised, with a reflection on which methods would be suited to cancer research impact evaluation going forward. Results In total, 40 literature reviews were identified. Important methods to collect and analyse data for impact assessments were surveys, interviews and documentary analysis. Key categories of impact spanning the reviews were summarised, and a list of frameworks commonly used for impact assessment was generated. The Payback Framework was most often described. Fourteen examples of impact evaluation for cancer research were identified. They ranged from those assessing the impact of a national, charity-funded portfolio of cancer research to the clinical practice impact of a single trial. A set of recommendations for approaching cancer research impact assessment was generated. Conclusions Impact evaluation can demonstrate if and why conducting cancer research is worthwhile. Using a mixed methods, multi-category assessment organised within a framework, will provide a robust evaluation, but the ability to perform this type of assessment may be constrained by time and resources. Whichever approach is used, easily measured, but inappropriate metrics should be avoided. Going forward, dissemination of the results of cancer research impact assessments will allow the cancer research community to learn how to conduct these evaluations.

Evidence-based Nomenclature and Taxonomy of Research Impact Indicators

PurposeThis study aims to classify research impact indicators based on their characteristics and scope. A concept of evidence-based nomenclature of research impact (RI) indicator has been introduced for generalization and transformation of scope.Design/methodology/approchLiterature was collected related to the research impact assessment. It was categorized in conceptual and applied case studies. One hundred and nineteen indicators were selected to prepare classification and nomenclature. The nomenclature was developed based on the principle—“every indicator is a contextual-function to explain the impact”. Every indicator was disintegrated into three parts, i.e. Function, Domain, and Target Areas.FindingsThe main functions of research impact indicators express improvement (63%), recognition (23%), and creation/development (14%). The focus of research impact indicators in literature is more towards the academic domain (59%) whereas the environment/sustainability domain is least considered (4%). As a result, research impact related to the research aspects is felt the most (29%). Other target areas include system and services, methods and procedures, networking, planning, policy development, economic aspects and commercialisation, etc.Research limitationsThis research applied to 119 research impact indicators. However, the inclusion of additional indicators may change the result.Practical implicationsThe plausible effect of nomenclature is a better organization of indicators with appropriate tags of functions, domains, and target areas. This approach also provides a framework of indicator generalization and transformation. Therefore, similar indicators can be applied in other fields and target areas with modifications.Originality/valueThe development of nomenclature for research impact indicators is a novel approach in scientometrics. It is developed on the same line as presented in other scientific disciplines, where fundamental objects need to classify on common standards such as biology and chemistry.

Introduction: critical reflections on research impact

This chapter briefly explains what we mean by ‘the impact agenda’ and what the UK approach to research impact assessment involves. This chapter also makes the case for why an empirical investigation of the recent changes associated with research impact assessment is required and provides key definitions and an overview of the rest of the book.

The rise of research impact

This chapter charts the origins and emergence of the research impact agenda in the UK, noting some of its historical antecedents and international influences. It explores the apparent motivations and rationales that have underpinned the various dimensions of research impact assessment, and the associated expectations that different actors appear to have. The chapter includes a visual summary of key developments in the evolution of research impact assessment and incentivisation in the UK.

The Impact Agenda

As international interest in promoting and assessing the impact of research grows, this book examines the ensuing controversies, consequences and challenges. It places a particular emphasis on learning from experiences in the UK, since this is the country at the forefront of a range of new approaches to incentivising, monitoring and rewarding research impact achievements. The book aims to understand the origins and rationale for these changes and to critically assess their consequences for academic practice. Combining a review of existing literature with a range of new qualitative data (from interviews, focus groups and documentary analysis), The Impact Agenda is unique in providing a comprehensive, cross-disciplinary empirical examination of the ways in which various forms of research impact assessment are shaping academic practices. Although the primary focus of the book is on the UK, the book also considers the different approaches that other countries with an interest in research impact are taking (notably Australia, Canada and the Netherlands). While noting the benefits that the increasing emphasis on outward facing work is bringing, the book draws attention to a wide range of challenges and controversies associated with research impact assessment and, in particular, with the UK’s chosen approach. It concludes by using the insights in the book to propose an alternative, more theoretically robust approach to incentivising and rewarding efforts to undertake and use academic research for societal benefit.

The dilemmas and uncertainties in assessing the societal impact of research

In the 2000s, many European Union countries have established research impact assessment practices as part of the scrutiny of the third mission of universities. The Finnish research evaluation system has widely adopted a societal impact criterion. In this study, the question is what guiding principles does impact assessment rely on. This point of view is based on the experiences and opinions of experts interviewed for the study. Four implicit principles of impact assessment which guide assessment goals and practical implementations were found. The guiding principles have several social and methodological dilemmas due to liminal interpretations between social interests and academic endeavor. This study recommends that the evaluation scholars should consider the actual social purpose vis-à-vis consistent methodological approaches to assessment before applying ‘all-round’ solutions to scholarly fields.