scholarly journals Reproducible Research in R: A Tutorial on How to Do the Same Thing More Than Once

Psych ◽  
2021 ◽  
Vol 3 (4) ◽  
pp. 836-867
Author(s):  
Aaron Peikert ◽  
Caspar J. van Lissa ◽  
Andreas M. Brandmaier

Computational reproducibility is the ability to obtain identical results from the same data with the same computer code. It is a building block for transparent and cumulative science because it enables the originator and other researchers, on other computers and later in time, to reproduce and thus understand how results came about, while avoiding a variety of errors that may lead to erroneous reporting of statistical and computational results. In this tutorial, we demonstrate how the R package repro supports researchers in creating fully computationally reproducible research projects with tools from the software engineering community. Building upon this notion of fully automated reproducibility, we present several applications including the preregistration of research plans with code (Preregistration as Code, PAC). PAC eschews all ambiguity of traditional preregistration and offers several more advantages. Making technical advancements that serve reproducibility more widely accessible for researchers holds the potential to innovate the research process and to help it become more productive, credible, and reliable.

2021 ◽  
Author(s):  
Aaron Peikert ◽  
Caspar J. Van Lissa ◽  
Andreas Markus Brandmaier

Reproducibility has long been considered integral to the scientific method. Something is called reproducible when an independent person obtains the same results from the same data. Until recently, detailed descriptions of methods and analyses were the primary instrument for ensuring scientific reproducibility. Technological advancements now enable scientists to achieve a more comprehensive standard; one in which any individual can be granted access to a digital research repository, and reproduce the analyses from the raw data to the final report including all relevant statistical analyses with a single command. This method has far-reaching implications for scientific archiving, reproducibility and replication, scientific productivity, and the credibility and reliability of scientific findings. One obstacle preventing the widespread adoption of this method is that the underlying technological advancements are complicated to use. This paper introduces `repro`, an R-package, which guides researchers in the installation and use of the tools required for making a research project reproducible. Finally, we suggest the use of the proposed tools for the preregistration of study plans as reproducible computer code (preregistration as code; PAC). Since computer code represents the planned analyses exactly as they will be executed, it is more precise than natural language descriptions of those analyses, which merely complement the PAC as a more readable summary. PAC circumvents the shortcomings of ambiguous preregistrations that may give researchers undesired degrees of freedom. Hence, reproducibility made convenient with automation has a wide range of applications to accelerate scientific progress.


Author(s):  
Honghai LI ◽  
Jun CAI

The transformation of China's design innovation industry has highlighted the importance of design research. The design research process in practice can be regarded as the process of knowledge production. The design 3.0 mode based on knowledge production MODE2 has been shown in the Chinese design innovation industry. On this cognition, this paper establishes a map with two dimensions of how knowledge integration occurs in practice based design research, which are the design knowledge transfer and contextual transformation of design knowledge. We use this map to carry out the analysis of design research cases. Through the analysis, we define four typical practice based design research models from the viewpoint of knowledge integration. This method and the proposed model can provide a theoretical basis and a path for better management design research projects.


2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Christine E. Laustsen ◽  
Albert Westergren ◽  
Pia Petersson ◽  
Maria Haak

Abstract Background Researchers have shown an increased interest in involving professionals from outside academia in research projects. Professionals are often involved in research on ageing and health when the purpose is to address the gap between research and practice. However, there is a need to acquire more knowledge about what the involvement might lead to by exploring researchers’ experiences of involving professionals in research on ageing and health and developing conceptual areas. Therefore, the aim of this study was to identify conceptual areas of professionals’ involvement in research on ageing and health, from the perspective of researchers themselves. Methods Group concept mapping, a participatory and mixed method, was used to conceptualize areas. Researchers with experience of involving professionals in research projects on ageing and health participated in qualitative data collection through brainstorming sessions (n = 26), and by sorting statements (n = 27). They then took part in quantitative data collection, where they rated statements according to how much a statement strengthened research (n = 26) and strengthened practice (n = 24). Data were analysed using multidimensional scaling analysis and hierarchical cluster analysis. In addition, a qualitative analysis of the latent meaning of the cluster map was conducted. Results Analysis of the sorting stage generated five clusters illustrating conceptual areas of professionals’ involvement in research projects on ageing and health. The five clusters are as follows: complex collaboration throughout the research process; adaptation of research to different stakeholders, mutual learning through partnership; applicable and sustainable knowledge; legitimate research on ageing and health. The qualitative latent meaning of the cluster map showed two themes: the process of involvement and the outcome of involvement. A positive strong correlation (0.87) was found between the rating of strengthened research and practice. Conclusions This study reveals conceptual areas on a comprehensive and illustrative map which contributes to the understanding of professionals’ involvement in research on ageing and health. A conceptual basis for further studies is offered, where the aim is to investigate the processes and outcomes entailed in involving professionals in research on ageing and health. The study also contributes to the development of instruments and theories for optimizing the involvement of professionals in research.


2021 ◽  
Vol 20 ◽  
pp. 160940692110161
Author(s):  
Syahirah Abdul Rahman ◽  
Lauren Tuckerman ◽  
Tim Vorley ◽  
Cristian Gherhes

The onset of the COVID-19 pandemic has seen the implementation of unprecedented social distancing measures, restricting social interaction and with it the possibility for conducting face-to-face qualitative research. This paper provides lessons from a series of qualitative research projects that were adapted during the COVID-19 pandemic to ensure their continuation and completion. By reflecting on our experiences and discussing the opportunities and challenges presented by crises to the use of a number of qualitative research methods, we provide a series of insights and lessons for proactively building resilience into the qualitative research process. We show that reflexivity, responsiveness, adaptability, and flexibility ensured continuity in the research projects and highlighted distinct advantages to using digital methods, providing lessons beyond the COVID-19 context. The paper concludes with reflections on research resilience and adaptation during crises.


Data Science ◽  
2021 ◽  
pp. 1-21
Author(s):  
Caspar J. Van Lissa ◽  
Andreas M. Brandmaier ◽  
Loek Brinkman ◽  
Anna-Lena Lamprecht ◽  
Aaron Peikert ◽  
...  

Adopting open science principles can be challenging, requiring conceptual education and training in the use of new tools. This paper introduces the Workflow for Open Reproducible Code in Science (WORCS): A step-by-step procedure that researchers can follow to make a research project open and reproducible. This workflow intends to lower the threshold for adoption of open science principles. It is based on established best practices, and can be used either in parallel to, or in absence of, top-down requirements by journals, institutions, and funding bodies. To facilitate widespread adoption, the WORCS principles have been implemented in the R package worcs, which offers an RStudio project template and utility functions for specific workflow steps. This paper introduces the conceptual workflow, discusses how it meets different standards for open science, and addresses the functionality provided by the R implementation, worcs. This paper is primarily targeted towards scholars conducting research projects in R, conducting research that involves academic prose, analysis code, and tabular data. However, the workflow is flexible enough to accommodate other scenarios, and offers a starting point for customized solutions. The source code for the R package and manuscript, and a list of examplesof WORCS projects, are available at https://github.com/cjvanlissa/worcs.


2016 ◽  
Vol 45 (1) ◽  
pp. 55-69 ◽  
Author(s):  
Sebastian Warnholz ◽  
Timo Schmid

The demand for reliable regional estimates from sample surveys has been substantially grown over the last decades. Small area estimation provides statistical methods to produce reliable predictions when the sample sizes in regions are too small to apply direct estimators. Model- and design-based simulations are used to gain insights into the quality of the introduced methods. In this article we present a framework which may help to guarantee the reproducibility of simulation studies in articles and during research. The introduced R-package saeSim is adjusted to provide a simulation environment for the special case of small area estimation. The package may allow the prospective researcher during the research process to produce simulation studies with a minimal eort of coding.


2021 ◽  
Author(s):  
Bernadette Fritzsch ◽  
Daniel Nüst

<p>Open Science has established itself as a movement across all scientific disciplines in recent years. It supports good practices in science and research that lead to more robust, comprehensible, and reusable results. The aim is to improve the transparency and quality of scientific results so that more trust is achieved, both in the sciences themselves and in society. Transparency requires that uncertainties and assumptions are made explicit and disclosed openly. <br>Currently, the Open Science movement is largely driven by grassroots initiatives and small scale projects. We discuss some examples that have taken on different facets of the topic:</p><ul><li>The software developed and used in the research process is playing an increasingly important role. The Research Software Engineers (RSE) communities have therefore organized themselves in national and international initiatives to increase the quality of research software.</li> <li>Evaluating reproducibility of scientific articles as part of peer review requires proper creditation and incentives for both authors and specialised reviewers to spend extra efforts to facilitate workflow execution. The Reproducible AGILE initiative has established a reproducibility review at a major community conference in GIScience.</li> <li>Technological advances for more reproducible scholarly communication beyond PDFs, such as containerisation, exist, but are often inaccessible to domain experts who are not programmers. Targeting geoscience and geography, the project Opening Reproducible Research (o2r) develops infrastructure to support publication of research compendia, which capture data, software (incl. execution environment), text, and interactive figures and maps.</li> </ul><p>At the core of scientific work lie replicability and reproducibility. Even if different scientific communities use these terms differently, the recognition that these aspects need more attention is commonly shared and individual communities can learn a lot from each other. Networking is therefore of great importance. The newly founded initiative German Reproducibility Network (GRN) wants to be a platform for such networking and targets all of the above initiatives. GRN is embedded in a growing network of similar initiatives, e.g. in the UK, Switzerland and Australia. Its goals include </p><ul><li>Support of local open science groups</li> <li>Connecting local or topic-centered initiatives for the exchange of experiences</li> <li>Attracting facilities for the goals of Open Science </li> <li>Cultivate contacts to funding organizations, publishers and other actors in the scientific landscape</li> </ul><p>In particular, the GRN aims to promote the dissemination of best practices through various formats of further education, in order to sensitize particularly early career researchers to the topic. By providing a platform for networking, local and domain-specific groups should be able to learn from one another, strengthen one another, and shape policies at a local level.</p><p>We present the GRN in order to address the existing local initiatives and to win them for membership in the GRN or sibling networks in other countries.</p>


2010 ◽  
Vol 09 (01) ◽  
pp. A05 ◽  
Author(s):  
Victoria Stodden

From contributions of astronomy data and DNA sequences to disease treatment research, scientific activity by non-scientists is a real and emergent phenomenon, and raising policy questions. This involvement in science can be understood as an issue of access to publications, code, and data that facilitates public engagement in the research process, thus appropriate policy to support the associated welfare enhancing benefits is essential. Current legal barriers to citizen participation can be alleviated by scientists’ use of the “Reproducible Research Standard,” thus making the literature, data, and code associated with scientific results accessible. The enterprise of science is undergoing deep and fundamental changes, particularly in how scientists obtain results and share their work: the promise of open research dissemination held by the Internet is gradually being fulfilled by scientists. Contributions to science from beyond the ivory tower are forcing a rethinking of traditional models of knowledge generation, evaluation, and communication. The notion of a scientific “peer” is blurred with the advent of lay contributions to science raising questions regarding the concepts of peer-review and recognition. New collaborative models are emerging around both open scientific software and the generation of scientific discoveries that bear a similarity to open innovation models in other settings. Public engagement in science can be understood as an issue of access to knowledge for public involvement in the research process, facilitated by appropriate policy to support the welfare enhancing benefits deriving from citizen-science.


Sign in / Sign up

Export Citation Format

Share Document