scholarly journals An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014–2017)

2020 ◽  
Vol 7 (2) ◽  
pp. 190806 ◽  
Author(s):  
Tom E. Hardwicke ◽  
Joshua D. Wallach ◽  
Mallory C. Kidwell ◽  
Theiss Bendixen ◽  
Sophia Crüwell ◽  
...  
Keyword(s):  
Social Sciences ◽  
Conflicts Of Interest ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Social Science Research ◽  
Science Research ◽  
Research Quality ◽  
Research Practices ◽  
Related Research ◽  
The Social

Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.

scholarly journals An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017)

2019 ◽  
Author(s):  
Tom Elis Hardwicke ◽  
Joshua D Wallach ◽  
Mallory Kidwell ◽  
Theiss Bendixen ◽  
Sophia Crüwell ◽  
...  
Keyword(s):  
Social Sciences ◽  
Conflicts Of Interest ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Social Science Research ◽  
Science Research ◽  
Research Quality ◽  
Research Practices ◽  
Related Research ◽  
The Social

Serious concerns about research quality have catalyzed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency, and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014-2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]), or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.

A Tale of Two Cultures

2012 ◽  
Author(s):  
Gary Goertz ◽  
James Mahoney
Keyword(s):  
Social Sciences ◽  
Quantitative Research ◽  
Social Science Research ◽  
Measurement Data ◽  
Science Research ◽  
Quantitative Research Methods ◽  
Qualitative And Quantitative ◽  
The Social ◽  
Scientific Worldview ◽  
Different Cultures

Some in the social sciences argue that the same logic applies to both qualitative and quantitative research methods. This book demonstrates that these two paradigms constitute different cultures, each internally coherent yet marked by contrasting norms, practices, and toolkits. The book identifies and discusses major differences between these two traditions that touch nearly every aspect of social science research, including design, goals, causal effects and models, concepts and measurement, data analysis, and case selection. Although focused on the differences between qualitative and quantitative research, the book also seeks to promote toleration, exchange, and learning by enabling scholars to think beyond their own culture and see an alternative scientific worldview. The book is written in an easily accessible style and features a host of real-world examples to illustrate methodological points.

scholarly journals Has the Credibility of the Social Sciences Been Credibly Destroyed? Reanalyzing the “Many Analysts, One Data Set” Project

2021 ◽  
Vol 7 ◽  
pp. 237802312110244
Author(s):  
Katrin Auspurg ◽  
Josef Brüderl
Keyword(s):  
Social Sciences ◽  
Causal Reasoning ◽  
Research Question ◽  
Social Science Research ◽  
Science Research ◽  
Skin Tone ◽  
Data Set ◽  
The Social ◽  
The Many ◽  
Definition Of

In 2018, Silberzahn, Uhlmann, Nosek, and colleagues published an article in which 29 teams analyzed the same research question with the same data: Are soccer referees more likely to give red cards to players with dark skin tone than light skin tone? The results obtained by the teams differed extensively. Many concluded from this widely noted exercise that the social sciences are not rigorous enough to provide definitive answers. In this article, we investigate why results diverged so much. We argue that the main reason was an unclear research question: Teams differed in their interpretation of the research question and therefore used diverse research designs and model specifications. We show by reanalyzing the data that with a clear research question, a precise definition of the parameter of interest, and theory-guided causal reasoning, results vary only within a narrow range. The broad conclusion of our reanalysis is that social science research needs to be more precise in its “estimands” to become credible.

Statistical and measurement pitfalls in the use of meta-regression in meta-analysis

2017 ◽  
Vol 22 (5) ◽  
pp. 469-476 ◽  
Author(s):  
Frank L. Schmidt
Keyword(s):  
Social Sciences ◽  
Organizational Behavior ◽  
Meta Analysis ◽  
Social Science Research ◽  
Science Research ◽  
Content Type ◽  
Marketing Social ◽  
Meta Regression ◽  
Meta Analyses ◽  
Almost All

Purpose Meta-regression is widely used and misused today in meta-analyses in psychology, organizational behavior, marketing, management, and other social sciences, as an approach to the identification and calibration of moderators, with most users being unaware of serious problems in its use. The purpose of this paper is to describe nine serious methodological problems that plague applications of meta-regression. Design/methodology/approach This paper is methodological in nature and is based on well-established principles of measurement and statistics. These principles are used to illuminate the potential pitfalls in typical applications of meta-regression. Findings The analysis in this paper demonstrates that many of the nine statistical and measurement pitfalls in the use of meta-regression are nearly universal in applications in the literature, leading to the conclusion that few meta-regressions in the literature today are trustworthy. A second conclusion is that in almost all cases, hierarchical subgrouping of studies is superior to meta-regression as a method of identifying and calibrating moderators. Finally, a third conclusion is that, contrary to popular belief among researchers, the process of accurately identifying and calibrating moderators, even with the best available methods, is complex, difficult, and data demanding. Practical implications This paper provides useful guidance to meta-analytic researchers that will improve the practice of moderator identification and calibration in social science research literatures. Social implications Today, many important decisions are made on the basis of the results of meta-analyses. These include decisions in medicine, pharmacology, applied psychology, management, marketing, social policy, and other social sciences. The guidance provided in this paper will improve the quality of such decisions by improving the accuracy and trustworthiness of meta-analytic results. Originality/value This paper is original and valuable in that there is no similar listing and discussion of the pitfalls in the use of meta-regression in the literature, and there is currently a widespread lack of knowledge of these problems among meta-analytic researchers in all disciplines.

scholarly journals Realizing the transformative potential of conservation through the social sciences, arts and humanities

2018 ◽  
Author(s):  
Nathan Bennett ◽  
Robin Roth
Keyword(s):  
Social Sciences ◽  
Social Science Research ◽  
Science Research ◽  
Status Quo ◽  
Conservation Practices ◽  
Arts And Humanities ◽  
Transformative Potential ◽  
The Arts ◽  
The Social ◽  
Conservation Actions

Conservation actions most often occur in peopled seascapes and landscapes. As a result, conservation decisions cannot rely solely on evidence from the natural sciences, but must also be guided by the social sciences, the arts and the humanities. However, we are concerned that too much of the current attention is on research that serves an instrumental purpose, by which we mean that the social sciences are used to justify and promote status quo conservation practices. The reasons for engaging the social sciences, as well as the arts and the humanities, go well beyond making conservation more effective. In this editorial, we briefly reflect on how expanding the types of social science research and the contributions of the arts and the humanities can help to achieve the transformative potential of conservation.

Environment and Health Research in China: The State of the Field

2013 ◽  
Vol 214 ◽  
pp. 255-282 ◽  
Author(s):  
Jennifer Holdaway
Keyword(s):  
Social Sciences ◽  
Civil Society ◽  
Social Science ◽  
International Collaboration ◽  
Health Research ◽  
Social Science Research ◽  
Science Research ◽  
Environment And Health ◽  
The Social ◽  
Recent Developments

AbstractIn the context of this symposium, this article reviews social science research in the emerging field of environment and health in China, with a particular focus on the impacts of pollution. It begins with a discussion of the particular nature of China's environment-related health problems, distinguishing the different challenges presented by diseases of poverty, affluence and transition. It then reviews recent developments in policy and civil society with regard to environment and health, and the extent to which work in the social sciences has advanced our knowledge of these and of state–society interactions. The article concludes with some reflections on the need for and challenges of interdisciplinary and international collaboration in this area.

Literature Review

2021 ◽  
pp. 158-162
Author(s):  
Mathieu Ouimet ◽  
Pierre-Olivier Bédard
Keyword(s):  
Social Sciences ◽  
Literature Review ◽  
Social Science ◽  
Research Question ◽  
Relevant Literature ◽  
Social Science Research ◽  
Science Research ◽  
The Social ◽  
Academic Researchers ◽  
Review Protocol

This chapter highlights literature review. Reviewing the published literature is one of the key activities of social science research, as a way to position one’s academic contribution, but also to get a bird’s eye view of what the relevant literature says on a given topic or research question. Many guides have been created to assist academic researchers and students in conducting a literature review, but there is no consensus on the most appropriate method to do so. One of the reasons for this lack of consensus is the plurality of epistemological attitudes that coexist in the social sciences. Before initiating a literature review, the researcher should start by clarifying the need for and the purpose of the review. Once this has been clarified, the actual review protocol, tools, and databases to be used will need to be determined to strike a balance between the scope of the study and the depth of the review.

Sign in / Sign up

Export Citation Format

Share Document