There is more than what meets the eye: university preparation for the socio-economic impact requirement in research assessment exercise 2020 in Hong Kong

PurposeThis paper aims to examine how Hong Kong universities have responded to a newly included assessment element of socio-economic impact in a government-implemented research evaluation system – Research Assessment Exercise (RAE) 2020 – within the context of tightening audits and forceful knowledge economy objectives.Design/methodology/approachThis paper reports an institutional case study of the institutional-level response to the RAE 2020 impact requirement at a top-ranked comprehensive university in Hong Kong. A qualitative inquiry approach was adopted. The data sources mainly include university documents related to the RAE 2020 socio-economic impact policy, interview data with nine RAE-eligible academics at the case university, documents on the RAE exercises issued by the University Grants Committee (UGC) and field notes taken during the RAE information sessions.FindingsThe institutionalisation process of the RAE socio-economic impact agenda could be considered as establishing an indicator-oriented reward and recognition regime for knowledge transfer and knowledge exchange (KT/KE). Overall, two major institutional strategies were identified in operating the RAE 2020 impact agenda at the case university: (1) launching various policy initiatives: driven by the RAE-defined socio-economic impact; (2) incorporating socio-economic impact into faculty evaluation: premised upon the 16 KT performance indicators laid down by the UGC.Originality/valueThis article adds to the theoretical debate on the local reproduction of the global in studies of neoliberalism in higher education by describing a Hong Kong case study, supported by empirical data, of an actual university's responses to the newly included impact requirement in RAE 2020. More specifically, this study reveals that (1) the policy for socio-economic impact might be designed in a neutral or even benevolent manner, but has taken on a neoliberal and managerial dimension in its actual implementation; and (2) the neoliberal discourse underpinning the university's operation can be accounted for and explicated by the local factors embedded in the specific academic environment.

Just an artifact? The concordance between peer review and bibliometrics in economics and statistics in the Italian research assessment exercise

During the Italian research assessment exercise (2004–2010), the governmental agency (ANVUR) in charge of its realization performed an experiment on the concordance between peer review and bibliometrics at an individual article level. The computed concordances were at most weak for science, technology, engineering and mathematics. The only exception was the moderate concordance found for the area of economics and statistics. In this paper, the disclosed raw data of the experiment are used to shed light on the anomalous results obtained for economics and statistics. In particular, the data permit to document that the protocol of the experiment adopted for economics and statistics was different from the one used in the other areas. Indeed, in economics and statistics a same group of scholars developed the bibliometric ranking of journals for evaluating articles, managed peer reviews and formed the consensus groups for deciding the final scores of articles after having received the referee’s reports. This paper shows that the highest level of concordance in economics and statistics was an artifact mainly due to the role played by consensus groups in boosting the agreement between bibliometrics and peer review. Peer Review https://publons.com/publon/10.1162/qss_a_00172

Impact of research assessment exercises on research approaches and foci of accounting disciplines in Australia

PurposeThe overall aim of this paper was to examine the impact of the Australian research assessment exercise on the research approaches (positivist/non-positivist) favoured by accounting disciplines in Australia. Our key research question examined how the outputs and foci of research in elite accounting disciplines changed over a 16-year period. Our analysis was informed by Bourdieu's notions of academic elitism and symbolic violence.Design/methodology/approachWe analysed all papers published in 20 major accounting journals across a 16-year period by Australian accounting disciplines that were highly rated in the research assessment exercise. We also compared our results from this group against two case study accounting disciplines that were not rated as “world class”.FindingsOur key finding is that the introduction of a research assessment exercise in Australia has resulted in research outputs of elite accounting disciplines over this period being increasingly focused on positivist rather than non-positivist research. Our findings evidence a narrowing of accounting disciplines' research agendas and foci across the period.Research limitations/implicationsOur findings highlight a considerable narrowing of the research agenda and paradigms in accounting disciplines that is not in the public interest. Our findings also have implications for the literature on academic elitism. The narrowing of the research agenda and greater foci on positivist research exhibited in our findings demonstrates the role of dominant elites in controlling the research agenda through a research assessment exercise.Practical implicationsA practical implication is that proper research, regardless of the approach used, must be appropriately recognised and accepted by Accounting Disciplines, not ostracised or discouraged. Research implications are the breadth of accounting research should be celebrated and concentration eschewed. Australian accounting discipline leaders should not fall for the illusion that the only good research is that which is published in a small number of North American positivist journals.Originality/valueOur findings provide insights into Bourdieu's work through demonstrating how dominant players have successfully exploited an external regulatory mechanism, a research assessment exercise, to strengthen their position within a field and exert control over the research agendas of accounting disciplines. Previous work by Bourdieu has not directly examined how actors utilise these outside forces as instruments for shaping their own field.

The production of research elites. Research performance assessment in the United Kingdom

The production of research elites yields non-intended stratification effects. The contribution illustrates this by drawing on the British Research Assessment Exercise/Research Excellence Framework. Building on data from the three most recent assessments (RAE 2001, RAE 2008, REF 2014), a field and capital theoretical framework examines the unequal distribution of symbolic, social, and economic resources via panel membership, research staff, and research funding. The distribution of these resources is correlated to RAE/REF rank groups. The contribution concludes that the elite (re )produced by research performance assessments in Britain is not (solely) based on “research excellence”, but on previous allocations of symbolic, social, and economic resources.

Ranking Research: Rankings as Devices in Higher Education Governance

The paper provides a brief review of the literature on rankings in the higher education sector. It distinguishes literature on standardization effects and on stratification effects. The main section of the paper presents different rankings that are produced from the data of the Research Assessment Exercise and the Research Excellence Framework in the UK. Among them are rankings produced directly by the RAE/REF, and rankings produced by media outlets drawing on the RAE/REF data. Informed by these case examples, the closing section discusses the performativity of rankings. It looks into the different implications and effects rankings and the associated performance assessments have on higher education institutions, their personnel, and the personnel's practices.

The visible hand of research performance assessment

Far from allowing a governance of universities by the invisible hand of market forces, research performance assessments do not just measure differences in research quality, but yield themselves visible symptoms in terms of a stratification and standardization of disciplines. The article illustrates this with a case study of UK history departments and their assessment by the Research Assessment Exercise (RAE) and the Research Excellence Framework (REF), drawing on data from the three most recent assessments (RAE 2001, 2008, REF 2014). Symptoms of stratification are documented by the distribution of memberships in assessment panels, of research active staff, and of external research grants. Symptoms of a standardization are documented by the publications submitted to the assessments. The main finding is that the RAEs/REF and the selective allocation of funds they inform consecrate and reproduce a disciplinary center that, in contrast to the periphery, is well-endowed with grants and research staff, decides in panels over the quality standards of the field, and publishes a high number of articles in high-impact journals. This selectivity is oriented toward previous distributions of resources and a standardized notion of “excellence” rather than research performance.

The Real-Time REF Review: A Pilot Study to Examine the Feasibility of a Longitudinal Evaluation of Perceptions and Attitudes Towards REF 2021

The UK first introduced a national research assessment exercise in 1986, and methods of assessment continue to evolve. Following the 2016 Stern Review and further rounds of technical consultation, the UK higher education community is now preparing for the next Research Excellence Framework – REF 2021.Despite its importance in shaping UK research cultures, there is limited systematic and nuanced evidence about how academics across the sector view the REF, and which aspects are viewed favourably or unfavourably. The aims of this pilot study were twofold: first, it was designed to gather initial data to address this evidence gap; second, it was aimed at testing the feasibility of conducting a longitudinal study into academic and managerial attitudes towards the REF. We argue that further research to better understand the effects of the REF on research cultures, institutions, and individuals should be part of the evidence used to inform the development of future iterations of the exerciseThe Real Time REF Review Pilot Study was developed and delivered by a research team from Cardiff University and the University of Sheffield, in collaboration with Research England.