scholarly journals Are replication rates the same across academic fields? Community forecasts from the DARPA SCORE programme

2020 ◽  
Vol 7 (7) ◽  
pp. 200566
Author(s):  
Michael Gordon ◽  
Domenico Viganola ◽  
Michael Bishop ◽  
Yiling Chen ◽  
Anna Dreber ◽  
...  
Keyword(s):  
Large Scale ◽  
Academic Discipline ◽  
Empirical Studies ◽  
Survey Response ◽  
Replication Rate ◽  
Replication Studies ◽  
Research Fields ◽  
The Social ◽  
Replication Crisis ◽  
Independent Replication

The Defense Advanced Research Projects Agency (DARPA) programme ‘Systematizing Confidence in Open Research and Evidence' (SCORE) aims to generate confidence scores for a large number of research claims from empirical studies in the social and behavioural sciences. The confidence scores will provide a quantitative assessment of how likely a claim will hold up in an independent replication. To create the scores, we follow earlier approaches and use prediction markets and surveys to forecast replication outcomes. Based on an initial set of forecasts for the overall replication rate in SCORE and its dependence on the academic discipline and the time of publication, we show that participants expect replication rates to increase over time. Moreover, they expect replication rates to differ between fields, with the highest replication rate in economics (average survey response 58%), and the lowest in psychology and in education (average survey response of 42% for both fields). These results reveal insights into the academic community's views of the replication crisis, including for research fields for which no large-scale replication studies have been undertaken yet.

scholarly journals Researcher Degrees of Freedom and a Lack of Transparency Contribute to Unreliable Results of Nonparametric Statistical Analyses Across SPSS, SAS, Stata, and R

2020 ◽  
Author(s):  
Cooper Hodges ◽  
Hannah Michelle Lindsey ◽  
Paula Johnson ◽  
Bryant M Stone ◽  
James carter
Keyword(s):  
Degrees Of Freedom ◽  
Statistical Procedure ◽  
Statistical Analyses ◽  
Computational Error ◽  
Statistical Software ◽  
Replication Studies ◽  
The Social ◽  
Replication Crisis ◽  
Hand Calculation ◽  
Nonparametric Statistical

The replication crisis within the social and behavioral sciences has called into question the consistency of research methodology. A lack of attention to minor details in replication studies may limit researchers’ abilities to reproduce the results. One such overlooked detail is the statistical programs used to analyze the data. In the current investigation, we compared the results of several nonparametric analyses and measures of normality conducted on a large sample of data in SPSS, SAS, Stata, and R with results obtained through hand-calculation using the raw computational formulas. Multiple inconsistencies were found in the results produced between statistical packages due to algorithmic variation, computational error, and lack of clarity and/or specificity in the statistical output generated. We also highlight similar inconsistencies in supplementary analyses conducted on subsets of the data, which reflect realistic sample sizes. These inconsistencies were largely due to algorithmic variations used within packages when the analyses are performed on data from small- or medium-sized samples. We discuss how such inconsistencies may influence the conclusions drawn from the results of statistical analyses depending on the statistical software used, and we urge researchers to analyze their data across multiple packages, report details regarding the statistical procedure used for data analysis and consider these details when conducting direct replications studies.

scholarly journals Does health informatics have a replication crisis?

2018 ◽  
Vol 25 (8) ◽  
pp. 963-968 ◽  
Author(s):  
Enrico Coiera ◽  
Elske Ammenwerth ◽  
Andrew Georgiou ◽  
Farah Magrabi
Keyword(s):  
Health Informatics ◽  
Context Effects ◽  
Evidence Based ◽  
Medical Sciences ◽  
Replication Studies ◽  
Research Fields ◽  
Replication Crisis ◽  
The Cost ◽  
The Impact

Abstract Objective Many research fields, including psychology and basic medical sciences, struggle with poor reproducibility of reported studies. Biomedical and health informatics is unlikely to be immune to these challenges. This paper explores replication in informatics and the unique challenges the discipline faces. Methods Narrative review of recent literature on research replication challenges. Results While there is growing interest in re-analysis of existing data, experimental replication studies appear uncommon in informatics. Context effects are a particular challenge as they make ensuring replication fidelity difficult, and the same intervention will never quite reproduce the same result in different settings. Replication studies take many forms, trading-off testing validity of past findings against testing generalizability. Exact and partial replication designs emphasize testing validity while quasi and conceptual studies test generalizability of an underlying model or hypothesis with different methods or in a different setting. Conclusions The cost of poor replication is a weakening in the quality of published research and the evidence-based foundation of health informatics. The benefits of replication include increased rigor in research, and the development of evaluation methods that distinguish the impact of context and the nonreproducibility of research. Taking replication seriously is essential if biomedical and health informatics is to be an evidence-based discipline.

Spatially integrated and socially segregated: The effects of mixed-income neighbourhoods on social well-being

Urban Studies ◽  
2017 ◽  
Vol 55 (9) ◽  
pp. 1859-1874 ◽  
Author(s):  
Amie Thurber ◽  
Claire Riehle Bohmann ◽  
Craig Anne Heflinger
Keyword(s):  
Large Scale ◽  
Empirical Studies ◽  
Well Being ◽  
The United States ◽  
Spatial Integration ◽  
Policy And Practice ◽  
Social Lives ◽  
Urban Neighbourhoods ◽  
The Social ◽  
Mixed Income

Mixed-income development is the latest in a long line of policy interventions to improve outcomes in distressed urban neighbourhoods, particularly as an alternative to large scale public housing projects. Such developments are inherently integrationist, and have profound effects on the social lives and well-being of residents. After situating mixed-income developments within current residential demographic trends with regard to race and income, this article provides a comprehensive review of the 22 empirical studies addressing social integration in mixed-income developments in the United States, focusing on understanding the effects of spatial integration on social well-being. We discuss policy and practice implications to optimise the social well-being of residents, as well as strategies to improve scholarship in these settings.

scholarly journals Ethical ambiguity in the academic periphery: Global versus local science in explaining plagiarism

2021 ◽  
Author(s):  
Katerina Guba ◽  
Angelika Tsivinskaya
Keyword(s):  
Academic Discipline ◽  
Empirical Studies ◽  
Academic Misconduct ◽  
Lower Percentage ◽  
Plagiarism Detection ◽  
The Past ◽  
Research Fields ◽  
Average Share ◽  
Detection Software ◽  
Doctoral Dissertations

The past decade has seen extensive research carried out on the systematic causes of academic misconduct. Simultaneously, less attention has been paid to the variation in academic pathologies between research fields, as most empirical studies focus on one particular discipline. We propose that academic discipline is one of several systematic factors that might contribute to academic misbehavior. Drawing on a neo-institutional approach, we argue that on the academic periphery, the norm of textual originality has not drawn equal support across different research fields depending on its level of internationalization. Using plagiarism detection software, we analyzed 2,405 doctoral dissertations randomly selected from all dissertations defended in Russia between 2006 and 2016. We measured the globalization of each academic discipline by calculating the share of publications indexed in the global citation database in relation to overall output. Our results showed that, with an average share of detected borrowings of over 19%, the incidence of plagiarism on the academic periphery is remarkably higher than in Western countries. Overall, disciplines closely follow the pattern of higher globalization associated with a lower percentage of borrowed text. We also found that unauthorized borrowing is less prevalent at research-oriented institutions supporting global ethical standards. Our findings suggest that it might be misleading to measure the prevalence of academic misconduct on the academic periphery without paying attention to variations at the disciplinary level.

scholarly journals What does the literature say about business groups and pyramidal ownership in the period of 1960-2018?

2019 ◽  
Author(s):  
Silvia Amélia Mendonça Flores ◽  
Wesley Vieira da Silva ◽  
Igor Bernardi Sonza
Keyword(s):  
Large Scale ◽  
Web Of Science ◽  
Social Role ◽  
Business Groups ◽  
Political Connections ◽  
The Political ◽  
Research Fields ◽  
The Social ◽  
Pyramidal Ownership ◽  
And Control

This paper aims to analyze the results of the research in pyramidal structures within the scope of the business groups, based on a systematic literature review. The research was conducted on two large-scale journals databases (Web of Science and Scopus), using VOSviewer, HistCite™, and Iramuteq software. The textual corpus is consisting of 65 articles and 137 authors and co-authors. Bae et al. (2002) and Almeida and Wolfenzon (2006) are the most influential for the research fields. We infer as a conceptual framework that searches in pyramidal structures are contained in the field of business groups since they represent a form of organization and representation of ownership and control. We identify as a theoretical gap the analysis of the political connections and the social role. Thus, the contributions are in the sense of presenting a panorama on the themes, supporting future researches.

scholarly journals Experts as facilitators for the implementation of social media in the library? A social network approach

2014 ◽  
Vol 32 (3) ◽  
pp. 529-545 ◽  
Author(s):  
Hadewijch Vanwynsberghe ◽  
Elke Boudry ◽  
Ruben Vanderlinde ◽  
Pieter Verdegem
Keyword(s):  
Social Media ◽  
Social Network ◽  
Large Scale ◽  
Public Libraries ◽  
Empirical Studies ◽  
Social Capital Theory ◽  
Capital Theory ◽  
Content Type ◽  
The Social ◽  
Social Network Approach

Purpose – Based on the social capital theory, the authors assume that personal and professional experts are both relevant to people's competence development. However, to date, there is little empirical evidence of how professional experts can support, or impede, people in learning how to deal with social media. The purpose of this paper is to examine the role and position of social media experts in the distribution of information on social media within the library as organization. Design/methodology/approach – The paper draws upon social network and qualitative methods, within three public libraries located in Belgium. Findings – The findings suggest that as the most central actors, social media experts in a library play a significant role in either supporting or constraining the distribution of information on social media. Research limitations/implications – While the sample size was chosen to conduct a mixed methods study that would explore how the position of a social media expert in an organization such as the library facilitates or prevents the exchange of social media information, the authors acknowledge the need for large-scale empirical studies that can substantiate the findings in larger and more diverse samples. Originality/value – This unique study explores how the role and position social media experts in Belgian public libraries can support, or impede, librarians in learning how to deal with social media. This study is useful for other public libraries who want to implement social media, establish a social media policy and/or provide social media training.

scholarly journals How open science can benefit bilingualism research: A lesson in six tales

2021 ◽  
Author(s):  
Rodrigo Dal Ben ◽  
Melanie Brouillard ◽  
Ana Maria Gonzalez-Barrero ◽  
Hilary Killam ◽  
Lena V. Kremin ◽  
...  
Keyword(s):  
Social Sciences ◽  
Large Scale ◽  
Open Science ◽  
Recent Discussion ◽  
Science Practices ◽  
The Social ◽  
Replication Crisis ◽  
Do So

Bilingualism is hard to define, measure, and study. Sparked by the so-called replication crisis in the social sciences, a recent discussion on the advantages of open science is gaining momentum. Here we join this debate to argue that bilingualism research would greatly benefit from embracing open science. We do so in a unique way, by presenting six fictional stories that illustrate how open science practices — sharing preprints, materials, code, and data; pre-registering studies; and joining large-scale collaborations — can strengthen bilingualism research and further improve its quality.

scholarly journals What does the literature say about business groups and pyramidal ownership in the period of 1960-2018?

2019 ◽  
Author(s):  
Silvia Amélia Mendonça Flores ◽  
Wesley Vieira da Silva ◽  
Igor Bernardi Sonza
Keyword(s):  
Large Scale ◽  
Web Of Science ◽  
Social Role ◽  
Business Groups ◽  
Political Connections ◽  
The Political ◽  
Research Fields ◽  
The Social ◽  
Pyramidal Ownership ◽  
And Control

This paper aims to analyze the results of the research in pyramidal structures within the scope of the business groups, based on a systematic literature review. The research was conducted on two large-scale journals databases (Web of Science and Scopus), using VOSviewer, HistCite™, and Iramuteq software. The textual corpus is consisting of 65 articles and 137 authors and co-authors. Bae et al. (2002) and Almeida and Wolfenzon (2006) are the most influential for the research fields. We infer as a conceptual framework that searches in pyramidal structures are contained in the field of business groups since they represent a form of organization and representation of ownership and control. We identify as a theoretical gap the analysis of the political connections and the social role. Thus, the contributions are in the sense of presenting a panorama on the themes, supporting future researches.

scholarly journals The Rubik’s cube of postcolonialism: Theory's syncretism and challenges in postcolonial studies

2017 ◽  
Vol 69 (2-3) ◽  
pp. 309-331
Author(s):  
Natasa Jovanovic
Keyword(s):  
International Relations ◽  
Social Life ◽  
Large Scale ◽  
Academic Discipline ◽  
Postcolonial Studies ◽  
Modern World ◽  
Critical Examination ◽  
Question Mark ◽  
The Social ◽  
Medium Level

The paper examines the genesis of postcolonialism in various (mutually conditioned) forms: at the conceptual, humanistic, theoretical and disciplinary level. With the contextualization of the work of the first authors who put the question mark on the established and dominant western-centric perception of global divisions, we will (de)construct various historical and paradigmatic influences on the development of postcolonialism. A special emphasis is put on the position of postcolonialism within the so-called Great Debates in the academic discipline of International Relations. Also, we consider the possibility of development of postcolonialism as a theory on the medium level that has a multiple utility for International Relations. A critical examination of the initial assumptions of postcolonialism as inherently processual, reflexive and subversive, will open up the issues of the contemporary challenges of the social life of former colonies and their relationships with other actors on the international scene. One of the major issues (which can be set as a hypothesis) is how to use the advantage of the epistemological and theoretical postulates of postcolonialism in the research of the modern world in which the orientalist rhetoric largely survives, but due to the rise of terrorism and large-scale migration from the Middle East, the political and social reality is changed?

Multiple-source Record Linkage in a Rural Industrial Community, 1680–1820

1994 ◽  
Vol 6 (3) ◽  
pp. 133-142 ◽  
Author(s):  
Steve King
Keyword(s):  
Early Modern ◽  
Large Scale ◽  
Social Economic ◽  
Life Cycles ◽  
Multiple Source ◽  
Demographic Analysis ◽  
Ordinary People ◽  
Industrial Community ◽  
The Social ◽  
Broad Outline

Re-creating the social, economic and demographic life-cycles of ordinary people is one way in which historians might engage with the complex continuities and changes which underlay the development of early modern communities. Little, however, has been written on the ways in which historians might deploy computers, rather than card indexes, to the task of identifying such life cycles from the jumble of the sources generated by local and national administration. This article suggests that multiple-source linkage is central to historical and demographic analysis, and reviews, in broad outline, some of the procedures adopted in a study which aims at large scale life cycle reconstruction.

Sign in / Sign up

Export Citation Format

Share Document