Measuring Research

2018 ◽  
Author(s):  
Cassidy R. Sugimoto ◽  
Vincent Larivière
Keyword(s):  
Peer Review ◽  
Research Evaluation ◽  
Research Output ◽  
Scholarly Communication ◽  
Academic Disciplines ◽  
Publish Or Perish ◽  
Scholarly Output ◽  
Emerging Issues ◽  
The Many ◽  
The Impact

Policy makers, academic administrators, scholars, and members of the public are clamoring for indicators of the value and reach of research. The question of how to quantify the impact and importance of research and scholarly output, from the publication of books and journal articles to the indexing of citations and tweets, is a critical one in predicting innovation, and in deciding what sorts of research is supported and whom is hired to carry it out. There is a wide set of data and tools available for measuring research, but they are often used in crude ways, and each have their own limitations and internal logics. Measuring Research: What Everyone Needs to Know® will provide, for the first time, an accessible account of the methods used to gather and analyze data on research output and impact. Following a brief history of scholarly communication and its measurement — from traditional peer review to crowdsourced review on the social web — the book will look at the classification of knowledge and academic disciplines, the differences between citations and references, the role of peer review, national research evaluation exercises, the tools used to measure research, the many different types of measurement indicators, and how to measure interdisciplinarity. The book also addresses emerging issues within scholarly communication, including whether or not measurement promotes a "publish or perish" culture, fraud in research, or "citation cartels." It will also look at the stakeholders behind these analytical tools, the adverse effects of these quantifications, and the future of research measurement.

Changing Face of Scholarly Communication and Its Impact on Library and Information Centres

2019 ◽  
pp. 870-883
Author(s):  
G. Saroja
Keyword(s):  
Social Networking Sites ◽  
Research Output ◽  
Time Lag ◽  
Scholarly Communication ◽  
Electronic Journals ◽  
Institutional Repositories ◽  
The Social ◽  
History Of ◽  
Research Findings ◽  
The Impact

Scholarly communication involves publishing the research findings by academics and researchers in order to share and make available the academic or research output to the global community of researchers. Emergence of Internet and World Wide Web has brought revolutionary changes in the process of scholarly communication. Increasing price of serial publications, time lag in the publication and readership and other associated problems were addressed by the electronic journals and open access initiatives. Other models like – Consortia and Institutional Repositories have evolved as a cost saving models and improving communication. The social networking sites on the Internet are also promoting scholarly communication to a great extent. In the light of the changing technological environment this chapter depicts the history of scholarly publishing and reviews the changes that took place in the process of scholarly communication. Further, the impact of the changing models on Library and Information Centres (LICs) is examined.

scholarly journals Publish-and-Flourish: Using Blockchain Platform to Enable Cooperative Scholarly Communication

Publications ◽  
2019 ◽  
Vol 7 (2) ◽  
pp. 33
Author(s):  
Emilija Stojmenova Duh ◽  
Andrej Duh ◽  
Uroš Droftina ◽  
Tim Kos ◽  
Urban Duh ◽  
...  
Keyword(s):  
Peer Review ◽  
Review Process ◽  
User Behavior ◽  
Peer Review Process ◽  
Cooperative Behavior ◽  
Scholarly Communication ◽  
Research Paper ◽  
Strategic Games ◽  
Publish Or Perish ◽  
Smart Contract

Scholarly communication is today immersed in publish-or-perish culture that propels non-cooperative behavior in the sense of strategic games played by researchers. Here we introduce and describe a blockchain based platform for decentralized scholarly communication. The design of the platform rests on community driven publishing reviewing processes and implements cryptoeconomic incentives that promote cooperative user behavior. The key to achieve cooperation in blockchain based scholarly communication is to transform today’s static research paper into a modifiable research paper under continuous peer review process. We introduce and discuss the implementation of a modifiable research paper as a smart contract on the blockchain.

scholarly journals Mixed Method Study Examines Undergraduate Student Researchers’ Knowledge and Perceptions About Scholarly Communication Practices

2017 ◽  
Vol 12 (3) ◽  
pp. 159
Author(s):  
Melissa Goertzen
Keyword(s):  
Data Management ◽  
Peer Review ◽  
Undergraduate Students ◽  
Undergraduate Student ◽  
Review Process ◽  
Peer Review Process ◽  
Scholarly Communication ◽  
Student Researchers ◽  
Communication Practices ◽  
The Impact

A Review of: Riehle, C. F., & Hensley, M. K. (2017). What do undergraduate students know about scholarly communication?: A mixed methods study. Portal: Libraries and the Academy, 17(1), 145–178. http://dx.doi.org/10.1353/pla.2017.0009 Abstract Objective – To examine undergraduate student researchers’ perception and understanding of scholarly communication practices and issues. Design – Mixed method study involving a survey and semi-structured interviews. Setting – Two major undergraduate universities in the Midwest region of the United States. Subjects – Undergraduate students who participated in or had completed undergraduate research experiences with faculty mentors. Method – The method was first approved by Institutional Review Board offices at both campuses involved in the study. Then, students received invitations to participate in a survey via email (Campus 1 = 221 students; Campus 2 = 345 students). Identical online surveys ran separately on each campus; both remained open for a period of three weeks. All respondents received a reminder email one week before the survey closed. Participants answered twelve questions related to demographics and scholarly communication practices. The survey examined knowledge and experience across five areas: the peer review process, author and publisher rights, publication and access models, impact of research, and data management. All students who completed the survey were entered in a drawing for a $50 Amazon card. The response rates were 34.8% (Campus 1) and 18.6% (Campus 2). Surveys on both campuses were administered using different software: campus 1 utilized Qualtrics survey software while campus 2 used an institution-specific survey software. Data sets were normed and merged later in the study to enable comparison and identify broad themes. Survey respondents were also invited to participate in a 15 to 20 minute follow-up interview and were compensated with a $20 Amazon gift card. The interviews consisted of four open-ended questions that further examined students’ knowledge of scholarly communication practices. The researchers coded interview transcripts and identified themes. Qualitative software was used to analyze the surveys and assess coder agreement. Finally, connections and anomalies between survey and interview results were explored. Main Results – Quantitative and qualitative data collected during the study indicate that students were most confident in their understanding of the peer-review process and data management but felt less confident in their knowledge of author and publisher rights, publication and access models, and determining the impact of scholarly research publication. In addition, they value instruction related to scholarly communication topics like the peer-review process, publication models, and data management. However, few students feel confident in their current level of knowledge or ability surrounding the previously mentioned topics. Study findings suggest that this knowledge gap is based on a lack of training or discussion of scholarly communication topics in relation to students’ research activities. Results also suggest that undergraduate students have difficulty articulating their rights as authors and their scholarly communication practices. In many cases, skill sets like data management are learned through trial and error while students progress through the research process. In some cases, faculty mentors have misperceptions and assumptions about undergraduate students’ knowledge and abilities regarding scholarly communication practices. This can create challenges for undergraduate students as they attempt to make informed decisions about research activities based on a limited foundation of experience or information. Finally, results indicate that undergraduate student researchers do not currently view the library as a place to learn about scholarly communication practices. The authors suggest that by forming strategic relationships with undergraduate research program directors, faculty, and graduate student mentors, librarians are in a prime position to incorporate scholarly communication practices into information literacy sessions or provide point-of-need coaching. Conclusion – The researchers conclude that academic libraries are in a unique position to support overarching research, teaching, and learning goals within the academic community. By developing programs that support information literacy and scholarly communication, libraries demonstrate value and align goals with teaching and learning priorities within the higher education community as a whole. Through this work, librarians support students as knowledge creators and advocate for training that emphasizes data literacy, copyright and authors’ rights, and the impact of research within specific disciplines.

scholarly journals Open-access mega-journals

2017 ◽  
Vol 73 (2) ◽  
pp. 263-283 ◽  
Author(s):  
Valerie Spezi ◽  
Simon Wakeling ◽  
Stephen Pinfield ◽  
Claire Creaser ◽  
Jenny Fry ◽  
...  
Keyword(s):  
Open Access ◽  
Peer Review ◽  
Large Scale ◽  
Scholarly Communication ◽  
Academic Community ◽  
Scholarly Publishing ◽  
Content Type ◽  
Academic Literature ◽  
Article Processing Charges ◽  
The Impact

Purpose Open-access mega-journals (OAMJs) represent an increasingly important part of the scholarly communication landscape. OAMJs, such as PLOS ONE, are large scale, broad scope journals that operate an open access business model (normally based on article-processing charges), and which employ a novel form of peer review, focussing on scientific “soundness” and eschewing judgement of novelty or importance. The purpose of this paper is to examine the discourses relating to OAMJs, and their place within scholarly publishing, and considers attitudes towards mega-journals within the academic community. Design/methodology/approach This paper presents a review of the literature of OAMJs structured around four defining characteristics: scale, disciplinary scope, peer review policy, and economic model. The existing scholarly literature was augmented by searches of more informal outputs, such as blogs and e-mail discussion lists, to capture the debate in its entirety. Findings While the academic literature relating specifically to OAMJs is relatively sparse, discussion in other fora is detailed and animated, with debates ranging from the sustainability and ethics of the mega-journal model, to the impact of soundness-only peer review on article quality and discoverability, and the potential for OAMJs to represent a paradigm-shifting development in scholarly publishing. Originality/value This paper represents the first comprehensive review of the mega-journal phenomenon, drawing not only on the published academic literature, but also grey, professional and informal sources. The paper advances a number of ways in which the role of OAMJs in the scholarly communication environment can be conceptualised.

scholarly journals OpenUP’s Open Science Pilots. Designing innovative peer review and dissemination pilot studies

2016 ◽  
Author(s):  
Michela Vignoli ◽  
Oliver Zendel ◽  
Matthias Schörghuber
Keyword(s):  
Peer Review ◽  
Science Communication ◽  
Research Evaluation ◽  
Open Science ◽  
The Public ◽  
Fair Process ◽  
Constructive Feedback ◽  
Research Communities ◽  
Specific Implementation ◽  
The Impact

The review-disseminate-assess cycle is a multifaceted process involving different stakeholders: researchers, publishers, research institutions and funders, private companies, industry, and citizens. The H2020 project OpenUP aspires to bring all these stakeholders into an open dialogue to consensually identify and spread the review-disseminate-assess mechanisms advancing evolving practices of RRI in an Open Science context. OpenUP will actively engage research communities and implement a series of hands-on pilots to validate OpenUP’s proposed (open) peer review, innovative dissemination, and impact indicator frameworks. The pilots will be carried out in close cooperation with selected, devoted research communities from four scientific areas: arts and humanities, social sciences, life sciences, and energy. This poster visualises UpenUP’s pilot design exemplified by two of the seven Open Science Pilots to be conducted by the project: 1) Open Peer Review for Conferences, and 2) Addressing and Reaching Businesses and the Public with Research Output.The first pilot will evaluate the feasibility and acceptance of open peer reviewing in a conference setting. The specific implementation of the applied schema will be determined by the results from the state of the art study as well as the user questionnaire answers from other work packages of OpenUP. In comparison to the traditional way (double-blind evaluation of submitted papers by assigned reviewers chosen by the conference organisers) the new schema should allow for a more open and fair process as well as give additional incentives to reviewers. The actual pilot study will be conducted at a medium sized conference in consultation with the conference organisers. Follow-up questionnaires and interviews will show if the stakeholders preferred the new process and can provide constructive feedback for improvements and policy decisions.The second pilot will test existing and potential alternative forms, formats, and channels of open science communication, and explore how the targeted audiences, in particular businesses and the public, can be best reached via these channels. In a preparatory phase the team will map open science communication formats/channels and their targeted audiences. The team will conduct a workshop to elicit the targeted stakeholders’ requirements and expectations towards a useful and appealing communication of scientific contents. This will be the basis of the second pilot presented here. It will actively involve one or more energy research community projects beyond the OpenUP consortium. The goals are to test the previously established communication standards and channels for the energy area, and evaluate the impact and resonance at the targeted audiences.By actively involving research communities and other relevant stakeholders into the pilots, OpenUP will not only evaluate the practicability of the identified open peer review, innovative dissemination, and impact measurement methods in particular settings. It also intends to create and disseminate success stories, best/good practices, and policy recommendations, which will help further communities to implement working Open Science approaches in their research evaluation and communication strategies.

scholarly journals The C in ORCID: from authorship to contributorship

2020 ◽  
Author(s):  
Gabriela Mejias ◽  
Paloma Marín-Arraiza
Keyword(s):  
Peer Review ◽  
Scholarly Communication ◽  
Global Scale ◽  
Digital Infrastructure ◽  
Current Implementation ◽  
Different Types ◽  
Research Scholarship ◽  
The Impact ◽  
Share Information ◽  
Research Resources

ORCID is part of the wider digital infrastructure needed for all those who participate in research, scholarship and innovation to share information on a global scale. As part of our commitment with openness, we enable transparent and trustworthy connections between researchers, their contributions, and activities by providing an identifier for individuals to use with their name as they engage in research. Since its foundation in 2013, ORCID has aimed to enable and improve recognition for a broader range of contributions than textual publications. Its current worktype list includes  43 different types of contributions. In combination with acknowledging further contributions, ORCID also considers research resources (infrastructure, collection, equipment and service) to provide a better understanding of the impact of the work undertaken by researchers using them. This webinar will explore how ORCID supports the creation of a permanent, and unambiguous record of research and scholarly communication by enabling reliable attribution of contributors and their activities. The session will focus on the development and current implementation of ORCID tools to acknowledge and encourage contributions such as preprints and peer review activities, as well as the work being done to support contributor roles (CRediT taxonomy).    The session will also showcase an analysis of the current community adoption of these features and aims to trigger a discussion on the future of the scholarly record.

scholarly journals Blessed Are the Gatekeepers: A Longitudinal Study of the Editorial Boards of The Accounting Review

2009 ◽  
Vol 24 (3) ◽  
pp. 299-318 ◽  
Author(s):  
Timothy J. Fogarty ◽  
Chih-Hsien Liao
Keyword(s):  
Longitudinal Study ◽  
Peer Review ◽  
Academic Disciplines ◽  
Editorial Boards ◽  
The People ◽  
Editorial Review ◽  
Object Of Study ◽  
The Impact ◽  
Over Time

ABSTRACT: All academic disciplines must establish procedures to protect the quality of their knowledge. This usually entails a process of peer review whose function is to make judgments about the value of scholarship. The impact of these judgments is to elevate some scholarship to the highest plateaus of prominence, to prevent other work from publication, and to place the remainder in a system of stratification. The people who perform this function have considerable power to shape the literature, and therefore form a worthwhile object of study. This paper considers the editorial review board of The Accounting Review at three points in time over a 20-year period. The results suggest that institutional concentration of the gatekeepers has modestly decreased over time. However, other differences suggest that the scholarly representativeness of the group has diminished.

scholarly journals A 20-year evaluation of PLAAS research outputs: Impact on the scholarly domain and in social media

2019 ◽  
Vol 115 (7/8) ◽  
Author(s):  
Gillian Kerchhoff ◽  
Michelle Kahn ◽  
Mary Nassimbeni
Keyword(s):  
Social Media ◽  
Research Output ◽  
Grey Literature ◽  
Scholarly Communication ◽  
Research Process ◽  
The Body ◽  
Western Cape ◽  
Unique Identifier ◽  
Mainstream Media ◽  
The Impact

Patterns and methods of scholarly communication have changed with the growth in information technology, particularly the Internet and the social web. The changes have necessitated a broader definition of scholarly communication and the role of social media in the research process. We sought to record the body of work that the Institute for Poverty, Land and Agrarian Studies (PLAAS), a research institute at the University of the Western Cape, produced over a 20-year period (1995–2015) – the first two decades of its existence – and to measure its visibility and impact using bibliometrics and altmetrics. A survey was also carried out to investigate to what extent PLAAS researchers knew and used social media in their research practice. Scopus and Google Scholar were used as citation indices and Altmetric.com provided Altmetric scores – a measure of impact through social and mainstream media. The full list of PLAAS outputs showed a composition of 54% grey literature and 46% journal articles and monographs. Given that over half of PLAAS research outputs were in the form of grey literature, and therefore not indexed in traditional bibliometric databases, we suggest that alternative metrics be used in conjunction with bibliometrics, to measure the impact of a body of work on the scholarly domain. Although the bibliometrics in this study were a useful quantitative indicator of the impact of PLAAS research, this study was inconclusive with regard to determining the impact of the research output via altmetrics, partly because not any of the grey literature, nor any author from PLAAS, had a unique identifier, thus making it difficult to track and find quantitative indicators. Nonetheless, the potential benefit for PLAAS of using altmetrics was demonstrated in selected case studies of the output of three PLAAS researchers active on social media platforms.

scholarly journals Outside the Four Corners: Exploring Non­-Traditional Scholarly Communication

2015 ◽  
Vol 6 (2) ◽  
Author(s):  
K. Jane Burpee ◽  
Bobby Glushko ◽  
Lisa Goddard ◽  
Inba Kehoe ◽  
Patricia Moore
Keyword(s):  
Scholarly Communication ◽  
University Libraries ◽  
Journal Articles ◽  
Research Libraries ◽  
Association Of Research Libraries ◽  
Scholarly Output ◽  
Canadian Association ◽  
Four Corners ◽  
The Impact ◽  
Means Of Production

Traditional outputs of scholarly communication, such as monographs and journal articles are being supplemented by new forms of scholarship, particularly in fields such as digital humanities. Canadian university libraries have long played a role supporting the creation, distribution, and preservation of scholarly objects. That support must be extended to include new formats and modes of scholarly work, such as digital portfolios, non-linear narratives, social media, scholarly video journals, etc. As the means of production and forms of scholarly output diversify, libraries will need to understand the impact of these digital shifts and identify areas where library efforts can have the most influence. This article examines developing areas of non-traditional scholarly communication and discusses implications for members of the Canadian Association of Research Libraries (CARL).

Changing Face of Scholarly Communication and Its Impact on Library and Information Centres

2017 ◽  
pp. 100-117 ◽  
Author(s):  
G. Saroja
Keyword(s):  
Social Networking Sites ◽  
Research Output ◽  
Time Lag ◽  
Scholarly Communication ◽  
Electronic Journals ◽  
Institutional Repositories ◽  
The Social ◽  
History Of ◽  
Research Findings ◽  
The Impact

Scholarly communication involves publishing the research findings by academics and researchers in order to share and make available the academic or research output to the global community of researchers. Emergence of Internet and World Wide Web has brought revolutionary changes in the process of scholarly communication. Increasing price of serial publications, time lag in the publication and readership and other associated problems were addressed by the electronic journals and open access initiatives. Other models like – Consortia and Institutional Repositories have evolved as a cost saving models and improving communication. The social networking sites on the Internet are also promoting scholarly communication to a great extent. In the light of the changing technological environment this chapter depicts the history of scholarly publishing and reviews the changes that took place in the process of scholarly communication. Further, the impact of the changing models on Library and Information Centres (LICs) is examined.

Sign in / Sign up

Export Citation Format

Share Document