scholarly journals A comparison of systematic reviews and guideline-based systematic reviews in medical studies

2021 ◽  
Author(s):  
Alexander Schniedermann
Keyword(s):  
Systematic Reviews ◽  
Social Dynamics ◽  
Research Evaluation ◽  
Citation Impact ◽  
Scientific Output ◽  
Research Quality ◽  
Academic Quality ◽  
Reporting Standard ◽  
Using Data ◽  
Prolific Authors

AbstractThe question of how citation impact relates to academic quality accompanies every decade in bibliometric research. Although experts have employed more complex conceptions of research quality for responsible evaluation, detailed analyses of how impact relates to dimensions such as methodological rigor are lacking. But the increasing number of formal guidelines for biomedical research offer not only the potential to understand the social dynamics of standardization, but also their relations to scientific rewards. By using data from Web of Science and PubMed, this study focuses on systematic reviews from biomedicine and compares this genre with those systematic reviews that applied the PRISMA reporting standard. Besides providing an overview about growth and location, it was found that the latter, more standardized type of systematic review accumulates more citations. It is argued that instead of reinforcing the traditional conception that higher impact represents higher quality, highly prolific authors could be more inclined to develop and apply new standards than more average researchers. In addition, research evaluation would benefit from a more nuanced conception of scientific output which respects the intellectual role of various document types.

scholarly journals How Do Editors Select Papers, and How Good are They at Doing It?

2011 ◽  
Vol 11 (1) ◽  
Author(s):  
Robert Hofmeister ◽  
Matthias Krapf
Keyword(s):  
Research Evaluation ◽  
Evaluation Process ◽  
Research Quality ◽  
Using Data ◽  
Number Of Citations

Abstract Using data on the B.E. Journals that rank articles into four quality tiers, this paper examines the accuracy of the research evaluation process in economics. We find that submissions by authors with strong publication records and authors affiliated with highly-ranked institutions are significantly more likely to be published in higher tiers. Citation success as measured by RePEc statistics also depends heavily on the overall research records of the authors. Finally and most importantly, we measure how successful the B.E. Journals’ editors and their reviewers have been at assigning articles to quality tiers. While, on average, they are able to distinguish more influential from less influential manuscripts, we also observe many assignments that are not compatible with the belief that research quality is reflected by the number of citations.

scholarly journals Extending Geodemographics Using Data Primitives: A Review and a Methodological Proposal

2021 ◽  
Vol 10 (6) ◽  
pp. 386
Author(s):  
Jennie Gray ◽  
Lisa Buckner ◽  
Alexis Comber
Keyword(s):  
Social Dynamics ◽  
Feature Space ◽  
Social Processes ◽  
Multiple Time ◽  
Current State ◽  
Cluster Label ◽  
Static Data ◽  
The Social ◽  
Using Data

This paper reviews geodemographic classifications and developments in contemporary classifications. It develops a critique of current approaches and identifiea a number of key limitations. These include the problems associated with the geodemographic cluster label (few cluster members are typical or have the same properties as the cluster centre) and the failure of the static label to describe anything about the underlying neighbourhood processes and dynamics. To address these limitations, this paper proposed a data primitives approach. Data primitives are the fundamental dimensions or measurements that capture the processes of interest. They can be used to describe the current state of an area in a multivariate feature space, and states can be compared over multiple time periods for which data are available, through for example a change vector approach. In this way, emergent social processes, which may be too weak to result in a change in a cluster label, but are nonetheless important signals, can be captured. As states are updated (for example, as new data become available), inferences about different social processes can be made, as well as classification updates if required. State changes can also be used to determine neighbourhood trajectories and to predict or infer future states. A list of data primitives was suggested from a review of the mechanisms driving a number of neighbourhood-level social processes, with the aim of improving the wider understanding of the interaction of complex neighbourhood processes and their effects. A small case study was provided to illustrate the approach. In this way, the methods outlined in this paper suggest a more nuanced approach to geodemographic research, away from a focus on classifications and static data, towards approaches that capture the social dynamics experienced by neighbourhoods.

scholarly journals Internally Incentivized Interdisciplinarity: Organizational Restructuring of Research and Emerging Tensions

Minerva ◽  
2021 ◽  
Author(s):  
Mikko Salmela ◽  
Miles MacLeod ◽  
Johan Munck af Rosenschöld
Keyword(s):  
Interdisciplinary Research ◽  
Interdisciplinary Collaboration ◽  
Scientific Output ◽  
Green Technology ◽  
Academic Careers ◽  
Organizational Restructuring ◽  
Structured Interviews ◽  
Using Data ◽  
The University ◽  
Internal Funding

AbstractInterdisciplinarity is widely considered necessary to solving many contemporary problems, and new funding structures and instruments have been created to encourage interdisciplinary research at universities. In this article, we study a small technical university specializing in green technology which implemented a strategy aimed at promoting and developing interdisciplinary collaboration. It did so by reallocating its internal research funds for at least five years to “research platforms” that required researchers from at least two of the three schools within the university to participate. Using data from semi-structured interviews from researchers in three of these platforms, we identify specific tensions that the strategy has generated in this case: (1) in the allocation of platform resources, (2) in the division of labor and disciplinary relations, (3) in choices over scientific output and academic careers. We further show how the particular platform format exacerbates the identified tensions in our case. We suggest that certain features of the current platform policy incentivize shallow interdisciplinary interactions, highlighting potential limits on the value of attempting to push for interdisciplinarity through internal funding.

Does Interviewer Religious Dress Affect Survey Responses? Evidence from Morocco

2014 ◽  
Vol 7 (4) ◽  
pp. 734-760 ◽  
Author(s):  
Lindsay J. Benstead
Keyword(s):  
Missing Data ◽  
Social Dynamics ◽  
Representative Survey ◽  
Sensitive Questions ◽  
Interviewer Effects ◽  
Nationally Representative ◽  
And Gender ◽  
Using Data ◽  
Religious Dress ◽  
Survey Responses

AbstractFew studies examine religiosity-of-interviewer effects, despite recent expansion of surveying in the Muslim world. Using data from a nationally-representative survey of 800 Moroccans conducted in 2007, this study investigates whether and why interviewer religiosity and gender affect responses to religiously-sensitive questions. Interviewer dress affects responses to four of six items, but effects are larger and more consistent for religious respondents, in support of power relations theory. Religious Moroccans provide less pious responses to secular-appearing interviewers, whom they may link to the secular state, and more religious answers to interviewers wearing hijab, in order to safeguard their reputation in a society that values piety. Interviewer traits do not affect the probability of item-missing data. Religiosity-of-interviewer effects depend on interviewer gender for questions about dress choice, a gendered issue closely related to interviewer dress. Interviewer gender and dress should be coded and controlled for to reduce bias and better understand social dynamics.

Social Norms and the Relationship Between Cigarette Use and Religiosity Among Adolescents in the United States

2010 ◽  
Vol 38 (1) ◽  
pp. 39-48 ◽  
Author(s):  
Jan Gryczynski ◽  
Brian W. Ward
Keyword(s):  
Social Dynamics ◽  
Reference Group ◽  
The United States ◽  
Negative Association ◽  
Protective Effects ◽  
Cigarette Use ◽  
Learning Networks ◽  
The Social ◽  
Using Data ◽  
Social Learning Processes

This study investigated the social dynamics that underlie the negative association between religiosity and cigarette use among U.S. adolescents. Using data from the 2007 National Survey on Drug Use and Health, the authors used a theory-based conceptual model (vicarious learning networks [VLN]) to examine the role that key reference group norms play in the religiosity—smoking relationship. This relationship is partially mediated by parents’ and close friends’ perceived disapproval for smoking. However, religiosity maintains a strong negative association with smoking. Consistent with the VLN model, cigarette use varied substantively based on reference group normative configurations. To the extent that the protective effects of religiosity arise from its influence in structuring the social milieu, some of religiosity’s benefits could potentially be leveraged through interventions that promote healthy norms among reference groups within the social network. The VLN model may be a useful tool for conceptualizing the transmission of health behavior through social learning processes.

scholarly journals A Mixed-Methods Approach to Synthesizing Evidence on Mediators of Intervention Effects

2011 ◽  
Vol 33 (7) ◽  
pp. 870-900 ◽  
Author(s):  
Jennifer Leeman ◽  
YunKyung Chang ◽  
Corrine I. Voils ◽  
Jamie L. Crandell ◽  
Margarete Sandelowski
Keyword(s):  
Mixed Methods ◽  
Systematic Reviews ◽  
Behavioral Change ◽  
Observational Studies ◽  
Intervention Studies ◽  
Meta Analysis ◽  
Intervention Effects ◽  
Mediation Effects ◽  
Mixed Methods Approach ◽  
Using Data

Greater understanding of the mechanisms (mediators) by which behavioral-change interventions work is critical to developing theory and refining interventions. Although systematic reviews have been advocated as a method for exploring mediators, this is rarely done. One challenge is that intervention researchers typically test only two paths of the mediational model: the effect of the intervention on mediators and on outcomes. The authors addressed this challenge by drawing information not only from intervention studies but also from observational studies that provide data on associations between potential mediators and outcomes. They also reviewed qualitative studies of participants’ perceptions of why and how interventions worked. Using data from intervention ( n = 37) and quantitative observational studies ( n = 55), the authors conducted a meta-analysis of the mediation effects of eight variables. Qualitative findings ( n = 6) contributed to more in-depth explanations for findings. The methods used have potential to contribute to understanding of core mechanisms of behavioral-change interventions.

Evaluating the Research Quality of Education Journals in China: Implications for Increasing Global Impact in Peripheral Countries

2017 ◽  
Vol 87 (3) ◽  
pp. 583-618 ◽  
Author(s):  
Juanjuan Zhao ◽  
Gulbahar H. Beckett ◽  
Lihshing Leigh Wang
Keyword(s):  
Education Research ◽  
Citation Impact ◽  
Academic Research ◽  
Research Quality ◽  
Journal Articles ◽  
Global Knowledge ◽  
Academic Journal ◽  
Western Countries ◽  
Research Journal ◽  
Research Scholarship

There has been a rapid growth of academic research and publishing in non-Western countries. However, academic journal articles in these peripheral countries suffer from low citation impact and limited global recognition. This critical review systematically analyzed 1,096 education research journal articles that were published in China in a 10-year span using a multistage stratified cluster and random sampling method and a validated rubric for assessing research quality. Our findings reveal that the vast majority of the articles lacked rigor, with insufficient or nonsystematic literature reviews, incomplete descriptions of research design, and inadequately grounded recommendations for translating research into practice. Acknowledging the differences in publishing cultures in the center-periphery divide, we argue that education research publications in non-Western countries should try to meet Western publishing standards in order to participate in global knowledge production and research vitality. Implications for emerging countries that strive to transform their research scholarship are discussed.

Research Evaluation of Financial Research – Evidence from a Survey

2020 ◽  
Vol 53 (3) ◽  
pp. 383-419
Author(s):  
Enrico Miersch
Keyword(s):  
Research Evaluation ◽  
Team Work ◽  
Research Quality ◽  
Team Collaboration ◽  
Seminar Series ◽  
Publication Success ◽  
Project Characteristics ◽  
New Variables ◽  
Financial Research

I explore the role of team collaboration in financial research based on a survey of conference participants. Precisely, I investigate whether various team and project characteristics are associated with research quality. The use of different quality proxies, i. e., normalized total citations, Journal Impact Factor, and publication success in top 10 finance journals, and the origin of the data allow me to not only analyse working papers spread throughout the entire quality spectrum but also to research new variables, e. g., team work quality and communication intensity. I document a positive relation between research quality and the following project characteristics: authors’ scholarly capability, working paper’s presentation at top tier conferences and research seminar series.

scholarly journals RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials

2015 ◽  
Vol 23 (1) ◽  
pp. 193-201 ◽  
Author(s):  
Iain J Marshall ◽  
Joël Kuiper ◽  
Byron C Wallace
Keyword(s):  
Machine Learning ◽  
Clinical Trials ◽  
Systematic Reviews ◽  
Risk Of Bias ◽  
Reasonable Accuracy ◽  
Bias Assessment ◽  
Cochrane Database ◽  
The Difference ◽  
Using Data ◽  
Unclear Risk

Abstract Objective To develop and evaluate RobotReviewer, a machine learning (ML) system that automatically assesses bias in clinical trials. From a (PDF-formatted) trial report, the system should determine risks of bias for the domains defined by the Cochrane Risk of Bias (RoB) tool, and extract supporting text for these judgments. Methods We algorithmically annotated 12,808 trial PDFs using data from the Cochrane Database of Systematic Reviews (CDSR). Trials were labeled as being at low or high/unclear risk of bias for each domain, and sentences were labeled as being informative or not. This dataset was used to train a multi-task ML model. We estimated the accuracy of ML judgments versus humans by comparing trials with two or more independent RoB assessments in the CDSR. Twenty blinded experienced reviewers rated the relevance of supporting text, comparing ML output with equivalent (human-extracted) text from the CDSR. Results By retrieving the top 3 candidate sentences per document (top3 recall), the best ML text was rated more relevant than text from the CDSR, but not significantly (60.4% ML text rated ‘highly relevant' v 56.5% of text from reviews; difference +3.9%, [−3.2% to +10.9%]). Model RoB judgments were less accurate than those from published reviews, though the difference was <10% (overall accuracy 71.0% with ML v 78.3% with CDSR). Conclusion Risk of bias assessment may be automated with reasonable accuracy. Automatically identified text supporting bias assessment is of equal quality to the manually identified text in the CDSR. This technology could substantially reduce reviewer workload and expedite evidence syntheses.

Sign in / Sign up

Export Citation Format

Share Document