Substantiality: A Construct Indicating Research Excellence to Measure University Research Performance

Purpose The adequacy of research performance of universities or research institutes have often been evaluated and understood in two axes: “quantity” (i.e. size or volume) and “quality” (i.e. what we define here as a measure of excellence that is considered theoretically independent of size or volume, such as clarity in diamond grading). The purpose of this article is, however, to introduce a third construct named “substantiality” (“ATSUMI” in Japanese) of research performance and to demonstrate its importance in evaluating/understanding research universities. Design/methodology/approach We take a two-step approach to demonstrate the effectiveness of the proposed construct by showing that (1) some characteristics of research universities are not well captured by the conventional constructs (“quantity” and “quality”)-based indicators, and (2) the “substantiality” indicators can capture them. Furthermore, by suggesting that “substantiality” indicators appear linked to the reputation that appeared in university reputation rankings by simple statistical analysis, we reveal additional benefits of the construct. Findings We propose a new construct named “substantiality” for measuring research performance. We show that indicators based on “substantiality” can capture important characteristics of research institutes. “Substantiality” indicators demonstrate their “predictive powers” on research reputation. Research limitations The concept of “substantiality” originated from IGO game; therefore the ease/difficulty of accepting the concept is culturally dependent. In other words, while it is easily accepted by people from Japan and other East Asian countries and regions, it might be difficult for researchers from other cultural regions to accept it. Practical implications There is no simple solution to the challenge of evaluating research universities’ research performance. It is vital to combine different types of indicators to understand the excellence of research institutes. Substantiality indicators could be part of such a combination of indicators. Originality/value The authors propose a new construct named substantiality for measuring research performance. They show that indicators based on this construct can capture the important characteristics of research institutes.

James W. Carey and Communication Research: Reputation at the University’s Margins

James W. Carey, by the time of his death in 2006, was a towering figure in communication research in the United States. In this book, Jefferson Pooley provides a critical introduction to Carey’s work, tracing the evolution of his media theorizing from his graduate school years through to the publication, in 1989, of his landmark Communication as Culture. The book is an attempt to understand the unusual—if also undeniable—significance that Carey holds for so many communication scholars, as well as making his work accessible to advanced undergraduate and postgraduate students.

Recognition and reward in the academy

Purpose The publication oeuvre of a researcher carries great value when academic careers are assessed, and being recognised as a successful candidate is usually equated with being a productive author. Yet, how publications are valued in the context of evaluating careers is so far an understudied topic. The paper aims to discuss these issues. Design/methodology/approach Through a content analysis of assessment reports in three disciplines – biomedicine, economics and history – this paper analyses how externalities are used to evaluate publication oeuvres. Externalities are defined as features such as reviews and bibliometric indicators, which can be assessed without evaluating the epistemological claims made in the actual text. Findings All three fields emphasise similar aspects when assessing: authorship, publication prestige, temporality of research, reputation within the field and boundary keeping. Yet, how these facets of quality are evaluated, and the means through which they are assessed differs between disciplines. Moreover, research fields orient themselves according to different temporal horizons, i.e. history looks to the past and economics to the future when research is evaluated. Research limitations/implications The complexities involved in the process of evaluating candidates are also reflected in the findings, and while the comparative approach taken effectively highlights domain specific differences it may also hide counter-narratives, and subtle intradisciplinary discussion on quality. Originality/value This study offers a novel perspective on how publications are valued when assessing academic careers. Especially striking is how research across different fields is evaluated through different time horizons. This finding is significant in the debate on more overarching and formal systems of research evaluation.