scholarly journals Journal ratings as predictors of articles quality in Arts, Humanities and Social Sciences: an analysis based on the Italian Research Evaluation Exercise

F1000Research ◽  
2015 ◽  
Vol 4 ◽  
pp. 196 ◽  
Author(s):  
Andrea Bonaccorsi ◽  
Tindaro Cicero ◽  
Antonio Ferrara ◽  
Marco Malgarini
Keyword(s):  
Large Scale ◽  
Research Evaluation ◽  
Probit Model ◽  
Journal Ranking ◽  
Research Assessment ◽  
Peer Reviews ◽  
Arts And Humanities ◽  
Political Sciences ◽  
Evaluation Exercise ◽  
Ranking Journal

The aim of this paper is to understand whether the probability of receiving positive peer reviews is influenced by having published in an independently assessed, high-ranking journal: we eventually interpret a positive relationship among peer evaluation and journal ranking as evidence that journal ratings are good predictors of article quality. The analysis is based on a large dataset of over 11,500 research articles published in Italy in the period 2004-2010 in the areas of architecture, arts and humanities, history and philosophy, law, sociology and political sciences. These articles received a score by a large number of externally appointed referees in the context of the Italian research assessment exercise (VQR); similarly, journal scores were assigned in a panel-based independent assessment, which involved all academic journals in which Italian scholars have published, carried out under a different procedure. The score of an article is compared with that of the journal it is published in: more specifically, we first estimate an ordered probit model, assessing the probability for a paper of receiving a higher score, the higher the score of the journal; in a second step, we concentrate on the top papers, evaluating the probability of a paper receiving an excellent score having been published in a top-rated journal. In doing so, we control for a number of characteristics of the paper and its author, including the language of publication, the scientific field and its size, the age of the author and the academic status. We add to the literature on journal classification by providing for the first time a large scale test of the robustness of expert-based classification.

Handbook for Scientometrics: Science and Technology Development Indicators

2020 ◽  
Author(s):  
Mark Akoev ◽  
◽  
Valentina Markusova ◽  
Olga Moskaleva ◽  
Vladimir Pislyakov ◽  
...  
Keyword(s):  
Technology Development ◽  
Citation Index ◽  
Research Evaluation ◽  
Research Management ◽  
Research Assessment ◽  
Research Institutions ◽  
Scientometric Indicators ◽  
Development Indicators ◽  
Reasonable Approach ◽  
Eugene Garfield

The Second edition Russian Scientometric Handbook is designed to provide an overview of the field of scientometrics. The Handbook describes the history of creation of the breakthrough concept of citation indexing by Dr. Eugene Garfield, and development of the first multidisciplinary scholarly citation index, the Science Citation Index. Application of scientometric tools and methods in research management and resource allocation is discussed. Authors survey various scientometric indicators relevant to individual researchers, journals, research institutions and whole countries. Authors explore new types of indicators, such as altmetrics, relationship between scientometric indicators and the nature of scientific communication, and various methods of visualizing scientometric information. Possibilities and limitations of various scientometric techniques are examined. Authors highlight the need for an informed and reasonable approach to the use of quantitative indicators for research assessment. The Handbook includes the first Russian translations of three articles by Dr. Eugene Garfield. The Handbook is intended for use by researchers, science analysts, universities and research institutions administrators, libraries and information centers staff, graduate students, and the general reader interested in scientometrics and research evaluation.

scholarly journals How does research evaluation impact educational research? Exploring intended and unintended consequences of research assessment in the United Kingdom, 1986–2014

2017 ◽  
Vol 16 (6) ◽  
pp. 820-842 ◽  
Author(s):  
Marcelo Marques ◽  
Justin JW Powell ◽  
Mike Zapp ◽  
Gert Biesta
Keyword(s):  
Peer Review ◽  
Educational Research ◽  
Evaluation System ◽  
Research Evaluation ◽  
Academic Staff ◽  
Policy Instruments ◽  
Unintended Consequences ◽  
Research Assessment ◽  
Research Article ◽  
Departments Of Education

Research evaluation systems in many countries aim to improve the quality of higher education. Among the first of such systems, the UK’s Research Assessment Exercise (RAE) dating from 1986 is now the Research Excellence Framework (REF). Highly institutionalised, it transforms research to be more accountable. While numerous studies describe the system’s effects at different levels, this longitudinal analysis examines the gradual institutionalisation and (un)intended consequences of the system from 1986 to 2014. First, we analyse historically RAE/REF’s rationale, formalisation, standardisation, and transparency, framing it as a strong research evaluation system. Second, we locate the multidisciplinary field of education, analysing the submission behaviour (staff, outputs, funding) of departments of education over time to find decreases in the number of academic staff whose research was submitted for peer review assessment; the research article as the preferred publication format; the rise of quantitative analysis; and a high and stable concentration of funding among a small number of departments. Policy instruments invoke varied responses, with such reactivity demonstrated by (1) the increasing submission selectivity in the number of staff whose publications were submitted for peer review as a form of reverse engineering, and (2) the rise of the research article as the preferred output as a self-fulfilling prophecy. The funding concentration demonstrates a largely intended consequence that exacerbates disparities between departments of education. These findings emphasise how research assessment impacts the structural organisation and cognitive development of educational research in the UK.

scholarly journals CREATING STRENGTH IN AGE: HARNESSING THE POWER OF ARTS AND HUMANITIES NETWORKS

2019 ◽  
Vol 3 (Supplement_1) ◽  
pp. S29-S29
Author(s):  
Gay P Hanna ◽  
Pamela Saunders ◽  
Niyati Dhokai
Keyword(s):  
Older People ◽  
Performing Arts ◽  
Research Evaluation ◽  
Medical Center ◽  
Well Being ◽  
Value Systems ◽  
National Endowment ◽  
Medical Systems ◽  
Arts And Humanities ◽  
Health And Well Being

Abstract Arts and Humanities networks harness social capital in the service of older populations creating strength in age. This symposium will feature presentations in aging, arts, education, health and humanities exemplifying enormous and often underutilized resources readily available to engage older people across the spectrum of aging to combat decline and frailty at cognitive and physiological levels. Presenters will describe innovative partnership projects such as Sound Health, an initiative developed by the National Institutes of Health and the John F. Kennedy Center for the Performing Arts to expand knowledge and understanding of how listening, performing, or creating music could be harnessed for health and well-being; hybrid arts and humanities in health programs based within medical systems such as the Center for Performing Arts in Medicine at Texas Medical Center: Houston Methodist promoting research/evaluation of arts inventions to improve overall quality of patient care; and, MedStar Georgetown Lombardi Arts and Humanities Program providing a continuum of support for older patients and their caregivers from diagnoses through treatment processes. A Georgetown University case study will be presented on how arts, ethics and humanities are necessary and ideal components of an interdisciplinary master’s degree program in aging studies to ensure understanding a diverse and inter-generational cohort and student’s cultural value systems. The symposium will conclude with a presentation from the National Endowment of the Arts describing program service infrastructures across the country supporting arts engagement of older people, their families and caregivers focusing on lifelong learning; health and well-being; and age friendly design

scholarly journals GTX.Digest.VCF: an online NGS data interpretation system based on intelligent gene ranking and large-scale text mining

2019 ◽  
Vol 12 (S8) ◽  
Author(s):  
Yanhuang Jiang ◽  
Chengkun Wu ◽  
Yanghui Zhang ◽  
Shaowei Zhang ◽  
Shuojun Yu ◽  
...  
Keyword(s):  
Text Mining ◽  
Large Scale ◽  
Research Evaluation ◽  
Rapid Development ◽  
Academic Research ◽  
Data Interpretation ◽  
Biological Data ◽  
Sequencing Data ◽  
Web Portal ◽  
Interpretation System

Abstract Background An important task in the interpretation of sequencing data is to highlight pathogenic genes (or detrimental variants) in the field of Mendelian diseases. It is still challenging despite the recent rapid development of genomics and bioinformatics. A typical interpretation workflow includes annotation, filtration, manual inspection and literature review. Those steps are time-consuming and error-prone in the absence of systematic support. Therefore, we developed GTX.Digest.VCF, an online DNA sequencing interpretation system, which prioritizes genes and variants for novel disease-gene relation discovery and integrates text mining results to provide literature evidence for the discovery. Its phenotype-driven ranking and biological data mining approach significantly speed up the whole interpretation process. Results The GTX.Digest.VCF system is freely available as a web portal at http://vcf.gtxlab.com for academic research. Evaluation on the DDD project dataset demonstrates an accuracy of 77% (235 out of 305 cases) for top-50 genes and an accuracy of 41.6% (127 out of 305 cases) for top-5 genes. Conclusions GTX.Digest.VCF provides an intelligent web portal for genomics data interpretation via the integration of bioinformatics tools, distributed parallel computing, biomedical text mining. It can facilitate the application of genomic analytics in clinical research and practices.

Old habits, new dynamics: a case study of Web 2.0 in collaborative humanities and arts research

2011 ◽  
Vol 5 (2) ◽  
pp. 147-157
Author(s):  
Dan Goren
Keyword(s):  
Web 2.0 ◽  
Digital Technology ◽  
Large Scale ◽  
Web 2.0 Technologies ◽  
Research Project ◽  
Research Practices ◽  
Research Environment ◽  
Arts And Humanities ◽  
Virtual Research Environment

Whilst the application of online multimedia digital technology within arts and humanities research has burgeoned over the last decade, the practice of openly conducting collaborative and in particular discursive research publicly online remains one of the most unfamiliar and conceptually problematic areas for many academics in the field. Based on user surveys, blog posts, and forum discussions, this article provides both an account and assessment of Web 2.0 technologies in use on a large-scale arts and humanities research project. Examining usage by and impressions of both the project team and the wider community of users, it investigates both the advantages gained and problems faced through the use of a virtual research environment (VRE). It also pays special attention to the use of video and its implications for research practices.

Reply to the letter on Ancaiani et al. ‘Evaluating Scientific research in Italy: The 2004–10 research evaluation exercise’

2017 ◽  
Vol 26 (4) ◽  
pp. 358-360 ◽  
Author(s):  
Sergio Benedetto ◽  
Tindaro Cicero ◽  
Marco Malgarini ◽  
Carmen Nappi
Keyword(s):  
Research Evaluation ◽  
Scientific Research ◽  
Evaluation Exercise

Workflows Científicos com Apoio de Bases de Conhecimento em Tempo Real

2020 ◽  
Author(s):  
Victor S. Bursztyn ◽  
Jonas Dias ◽  
Marta Mattoso
Keyword(s):  
Knowledge Base ◽  
Domain Knowledge ◽  
Large Scale ◽  
Workflow Engine ◽  
Workflow Execution ◽  
Human In The Loop ◽  
Domain Specific ◽  
Political Sciences ◽  
Provenance Data ◽  
Domain Specific Knowledge

One major challenge in large-scale experiments is the analytical capacity to contrast ongoing results with domain knowledge. We approach this challenge by constructing a domain-specific knowledge base, which is queried during workflow execution. We introduce K-Chiron, an integrated solution that combines a state-of-the-art automatic knowledge base construction (KBC) system to Chiron, a well-established workflow engine. In this work we experiment in the context of Political Sciences to show how KBC may be used to improve human-in-the-loop (HIL) support in scientific experiments. While HIL in traditional domain expert supervision is done offline, in K-Chiron it is done online, i.e. at runtime. We achieve results in less laborious ways, to the point of enabling a breed of experiments that could be unfeasible with traditional HIL. Finally, we show how provenance data could be leveraged with KBC to enable further experimentation in more dynamic settings.

scholarly journals Robustifying Scholia: paving the way for knowledge discovery and research assessment through Wikidata

2019 ◽  
Vol 5 ◽  
Author(s):  
Lane Rasberry ◽  
Egon Willighagen ◽  
Finn Nielsen ◽  
Daniel Mietchen
Keyword(s):  
Knowledge Discovery ◽  
Large Scale ◽  
Knowledge Workers ◽  
Open Data ◽  
Linked Open Data ◽  
Data Curation ◽  
Research Assessment ◽  
Knowledge Graph ◽  
Incomplete Datasets ◽  
Active Contributor

Knowledge workers like researchers, students, journalists, research evaluators or funders need tools to explore what is known, how it was discovered, who made which contributions, and where the scholarly record has gaps. Existing tools and services of this kind are not available as Linked Open Data, but Wikidata is. It has the technology, active contributor base, and content to build a large-scale knowledge graph for scholarship, also known as WikiCite. Scholia visualizes this graph in an exploratory interface with profiles and links to the literature. However, it is just a working prototype. This project aims to "robustify Scholia" with back-end development and testing based on pilot corpora. The main objective at this stage is to attain stability in challenging cases such as server throttling and handling of large or incomplete datasets. Further goals include integrating Scholia with data curation and manuscript writing workflows, serving more languages, generating usage stats, and documentation.

scholarly journals METHODICAL APPROACHES TO THE ASSESSMENT OF THE IMPACT OF THE QUALITY OF TRANSFORMATION PROCESSES ON THE DEVELOPMENT OF THE FINANCIAL SECTOR

2021 ◽  
Vol 1 (36) ◽  
pp. 203-209
Author(s):  
О. Baranovskyi ◽  
I. Boiarko ◽  
T. Zlunitsyna
Keyword(s):  
National Economy ◽  
Large Scale ◽  
Financial Sector ◽  
Research Assessment ◽  
Logical Scheme ◽  
Rating Agencies ◽  
Integral Assessment ◽  
The Impact ◽  
Transformation Processes

The existing methods of separate central banks, regulators of financial markets, international financial and credit institutions, rating agencies and consulting companies, which can be used partially to assess the quality of transformational processes taking place in the financial sector, are analyzed. An adequate methodical toolkit is proposed, which will allow to assess the potential of the domestic financial sector, to identify the patterns of transformation processes, and also to assess the impact of their quality on the functioning of financial intermediaries and to propose appropriate mechanisms for ensuring their further development. The structural-logical scheme of formation of criteria and system of indicators of the quality of transformational processes in the financial sector of the country’s economy is developed. Investigated general approaches are considered to be useful for developing a methodology for the integral assessment of the quality of transformation processes in the financial sector of the economy by specifying the content of the main stages and methodological approaches to the goals and object of the research. Assessment of the quality of transformation processes in the financial sector of the national economy is an ambiguous process. This is due to the fact that, firstly, in this regard it can be considered as the impact of transformations on the quality of objects, and as the quality of the transformations in particular. Secondly, transformation processes in the financial sector of the national economy can be positive and negative; general and specific; large-scale and minor; systematic and dotted; intensive and extensive; priority and non-priority; stimulating and moderating; divergent; resource, technological, institutional, organizational, quality, behavioral; timely and spatial; painful, conflict and non-conflict; and also they are made with different expenses. Thirdly, both expected and actual results of transformation processes in the financial sector should be assessed.

scholarly journals Comparative Analysis of the Bibliographic Data Sources Dimensions and Scopus: An Approach at the Country and Institutional Levels

2021 ◽  
Vol 5 ◽  
Author(s):  
Vicente P. Guerrero-Bote ◽  
Zaida Chinchilla-Rodríguez ◽  
Abraham Mendoza ◽  
Félix de Moya-Anegón
Keyword(s):  
Comparative Analysis ◽  
Large Scale ◽  
Research Evaluation ◽  
Large Fraction ◽  
Data Sources ◽  
Global Level ◽  
Institutional Level ◽  
Scientific Databases ◽  
Bibliographic Data ◽  
Document Level

This paper presents a large-scale document-level comparison of two major bibliographic data sources: Scopus and Dimensions. The focus is on the differences in their coverage of documents at two levels of aggregation: by country and by institution. The main goal is to analyze whether Dimensions offers as good new opportunities for bibliometric analysis at the country and institutional levels as it does at the global level. Differences in the completeness and accuracy of citation links are also studied. The results allow a profile of Dimensions to be drawn in terms of its coverage by country and institution. Dimensions’ coverage is more than 25% greater than Scopus which is consistent with previous studies. However, the main finding of this study is the lack of affiliation data in a large fraction of Dimensions documents. We found that close to half of all documents in Dimensions are not associated with any country of affiliation while the proportion of documents without this data in Scopus is much lower. This situation mainly affects the possibilities that Dimensions can offer as instruments for carrying out bibliometric analyses at the country and institutional level. Both of these aspects are highly pragmatic considerations for information retrieval and the design of policies for the use of scientific databases in research evaluation.

Sign in / Sign up

Export Citation Format

Share Document