scholarly journals Operationalizing the Replication Standard

2018 ◽  
Vol 13 (1) ◽  
pp. 114-124 ◽  
Author(s):  
Thu-Mai Lewis Christian ◽  
Sophia Lafferty-Hess ◽  
William G Jacoby ◽  
Thomas Carsey

In response to widespread concerns about the integrity of research published in scholarly journals, several initiatives have emerged that are promoting research transparency through access to data underlying published scientific findings. Journal editors, in particular, have made a commitment to research transparency by issuing data policies that require authors to submit their data, code, and documentation to data repositories to allow for public access to the data. In the case of the American Journal of Political Science (AJPS) Data Replication Policy, the data also must undergo an independent verification process in which materials are reviewed for quality as a condition of final manuscript publication and acceptance. Aware of the specialized expertise of the data archives, AJPS called upon the Odum Institute Data Archive to provide a data review service that performs data curation and verification of replication datasets. This article presents a case study of the collaboration between AJPS and the Odum Institute Data Archive to develop a workflow that bridges manuscript publication and data review processes. The case study describes the challenges and the successes of the workflow integration, and offers lessons learned that may be applied by other data archives that are considering expanding their services to include data curation and verification services to support reproducible research.

2018 ◽  
Author(s):  
Thu-Mai Lewis Christian ◽  
Sophia Lafferty-Hess ◽  
William G. Jacoby ◽  
Thomas M. Carsey

In response to widespread concerns about the integrity of research published in scholarly journals, several initiatives have emerged that are promoting research transparency through access to data underlying published scientific findings. Journal editors, in particular, have made a commitment to research transparency by issuing data policies that require authors to submit their data, code, and documentation to data repositories to allow for public access to the data. In the case of the American Journal of Political Science (AJPS) Data Replication Policy, the data also must undergo an independent verification process in which materials are reviewed for quality as a condition of final manuscript publication and acceptance.Aware of the specialized expertise of the data archives, AJPS called upon the Odum Institute Data Archive to provide a data review service that performs data curation and verification of replication datasets. This article presents a case study of the collaboration between AJPS and the Odum Institute Data Archive to develop a workflow that bridges manuscript publication and data review processes. The case study describes the challenges and the successes of the workflow integration, and offers lessons learned that may be applied by other data archives that are considering expanding their services to include data curation and verification services to support reproducible research.


2019 ◽  
pp. 339-368
Author(s):  
Niamh Darcy ◽  
Sriyanjit Perera ◽  
Grades Stanley ◽  
Susan Rumisha ◽  
Kelvin Assenga ◽  
...  

In 2009, the Tanzanian Ministry of Health, Community Development, Gender, Elderly and Children (MoHCDGEC) counted over 10 different health facility lists managed by donors, government ministries, agencies and implementing partners. These function-specific lists were not integrated or linked. The ministry's Health Sector Strategic Plan included the development of an authoritative source for all health facility information, called the Master Facility List (MFL). During development, the ministry adopted the term Health Facility Registry (HFR), an online tool providing public access to a database about all officially recognized health facilities (public and private). The MFL, which includes the health facility list at any specific point in time can be exported from the HFR. This chapter presents the Tanzanian case study describing the work and lessons learned in building the HFR—focusing on software development, introducing geographic positioning systems and harmonizing MFL data. MoHCDGEC launched the HFR public portal in September 2015.


PLoS ONE ◽  
2021 ◽  
Vol 16 (5) ◽  
pp. e0244122
Author(s):  
Dario Righelli ◽  
Claudia Angelini

During last years “irreproducibility” became a general problem in omics data analysis due to the use of sophisticated and poorly described computational procedures. For avoiding misleading results, it is necessary to inspect and reproduce the entire data analysis as a unified product. Reproducible Research (RR) provides general guidelines for public access to the analytic data and related analysis code combined with natural language documentation, allowing third-parties to reproduce the findings. We developed easyreporting, a novel R/Bioconductor package, to facilitate the implementation of an RR layer inside reports/tools. We describe the main functionalities and illustrate the organization of an analysis report using a typical case study concerning the analysis of RNA-seq data. Then, we show how to use easyreporting in other projects to trace R functions automatically. This latter feature helps developers to implement procedures that automatically keep track of the analysis steps. Easyreporting can be useful in supporting the reproducibility of any data analysis project and shows great advantages for the implementation of R packages and GUIs. It turns out to be very helpful in bioinformatics, where the complexity of the analyses makes it extremely difficult to trace all the steps and parameters used in the study.


2008 ◽  
Vol 3 (4) ◽  
pp. 65
Author(s):  
Gaby Haddow

A Review of: Perry, Carol Marie. “Archiving of Publicly Funded Research Data: A Survey of Canadian Researchers.” Government Information Quarterly 25 (2008): 133-48. Objective – To assess researchers’ attitudes and behaviours in relation to archiving research data and to determine researchers’ views about policies relating to data archiving. Design – Survey. Setting – Canadian universities. Subjects – Social sciences and humanities researchers from universities across Canada. Methods – A questionnaire comprising 15 questions was mailed to 175 researchers randomly sampled from a publicly available list of 5,821 individuals who had received grants and awards from the Social Sciences and Humanities Research Council of Canada (SSHRC). From this sample, 75 (43.4%) responded within the five week time-frame stipulated. The questionnaire was constructed using four existing surveys and asked researchers for information about: geographical location, years of research experience, research funding sources, current plans to archive research data, awareness of archiving policies, attitude to mandated research data archiving, effect of mandatory data archiving policies on grant-seeking, attitude to making archived research data accessible, and use of research data collected by others. The questionnaire also included space for respondents to make comments. Responses to each question were analyzed, followed by a series of cross-analyses to investigate associations between findings. These cross-analyses include: whether attitudes to making data accessible differed according to length of experience, support for a national archive initiative, or agreement with a mandatory policy; and whether plans to archive research data were associated with awareness of policy, agreement with mandatory policy, or funding from government agencies other than SSHRC. Some cross-analyses were conducted between responses to questions and the comments provided by respondents. Most of the questionnaire responses were analyzed as percentages in two categories only. For example, length of service responses were presented as “up to 20 years” and “more than 20 years,” and for responses relating to agreement with mandatory archiving “no” and “undecided” responses were combined into one category and compared with “yes” responses. Main Results Plans to archive research data Only 41.3% of the respondent sample had current plans to archive their research data; plans that included filing materials in their office and destroying materials. A small proportion of this group (18.7%) planned to archive their research data in an established repository. Respondents who planned to archive their research data were more likely to have received funding from other government agencies (64.7%) than respondents who had not received other government funding (23.5%). Examination of the comments made by respondents indicated those who planned to archive their data were more likely to make comments on this issue. Knowledge of data archiving policies Respondents’ awareness of data archiving policies, particularly that of the SSHRC, an Organisation for Economic Co-operation and Development (OECD) declaration, and that of their own university, was low. Less than a third of the sample (28%) was aware of the SSHRC policy and less than 10% of the OECD declaration or their own university’s policy. A sizeable proportion of the sample (65.4%) stated they were “uncertain” whether their university had a data archiving policy. Of the respondents who planned to archive their data only 42.4% were aware of the SSHRC policy, and a high percentage of respondents with no plans to archive their data (83.3%) were also unaware of this policy. When asked if they agreed with a mandatory policy of data archiving being linked to grant funding the sample group was almost evenly split into agree (33.3%), don’t agree (28%), and uncertain (36%). Only 46.9% of the sample who planned to archive their data agreed with the policy. Support for data archives There was strong support for a data archive at the home institution by respondents who agreed with mandatory archiving of research data (84%). However, when asked about allowing public access to this data the results were evenly divided across most of the sample group (66 respondents). Full open access to data was supported by 44% of this group and 44% thought access should be at the discretion of the depositing researcher. Agreement with open access to data was also associated with researchers who agreed with mandatory archiving policy (70.8%) and researchers with over 20 years experience (68%). The enforcement of a mandatory archiving policy would not alter most respondents’ future grant-seeking behaviour (86.7%). Support for a national data archive was solid across the sample group, with only 17.3% indicating they did not support an initiative of this type. Almost all respondents who agreed with public access to research data supported a national data archive (96.5%) and of the respondents who did not support a national archive 90.9% were undecided about data access or wanted restrictions in place. When asked about use of research data collected by others, 48% had used such data in the past and 49.3% had not analyzed research data collected by others. Conclusion – While support for a national data archives strategy is strong, researchers’ attitudes and behaviour towards data archiving is less supportive and their plans to archive research data would not, in most cases, meet standards of archiving practice. The notion of open access to research data has less support, with researchers noting barriers relating to confidentiality, ethics, effect on academic career, and cost of archiving. Funding of data archiving raises particular concerns and the author recommends that SSHRC policy is clarified in this regard. Further, the author notes that differences between institutional requirements and national policies relating to research data require attention so that archiving policy objectives are coordinated. The decisions made in moving towards a national data archiving policy will enable Canada to contribute to the wider international discussion of standards for research data archives.


Author(s):  
Niamh Darcy ◽  
Sriyanjit Perera ◽  
Grades Stanley ◽  
Susan Rumisha ◽  
Kelvin Assenga ◽  
...  

In 2009, the Tanzanian Ministry of Health, Community Development, Gender, Elderly and Children (MoHCDGEC) counted over 10 different health facility lists managed by donors, government ministries, agencies and implementing partners. These function-specific lists were not integrated or linked. The ministry's Health Sector Strategic Plan included the development of an authoritative source for all health facility information, called the Master Facility List (MFL). During development, the ministry adopted the term Health Facility Registry (HFR), an online tool providing public access to a database about all officially recognized health facilities (public and private). The MFL, which includes the health facility list at any specific point in time can be exported from the HFR. This chapter presents the Tanzanian case study describing the work and lessons learned in building the HFR—focusing on software development, introducing geographic positioning systems and harmonizing MFL data. MoHCDGEC launched the HFR public portal in September 2015.


2020 ◽  
Author(s):  
Dario Righelli ◽  
Claudia Angelini

AbstractDuring last years “irreproducibility” became a general problem in omics data analysis due to the use of sophisticated and poorly described computational procedures. For avoiding misleading results, it is necessary to inspect and reproduce the entire data analysis as a unified product. Reproducible Research (RR) provides general guidelines for public access to the analytic data and related analysis code combined with natural language documentation, allowing third-parties to reproduce the findings. We developed easyreporting, a novel R/Bioconductor package, to facilitate the implementation of an RR layer inside reports/tools without requiring any knowledge of the R Markdown language. We describe the main functionalities and illustrate how to create an analysis report using a typical case study concerning the analysis of RNA-seq data. Then, we also show how to trace R functions automatically. Thanks to this latter feature, easyreporting results beneficial for developers to implement procedures that automatically keep track of the analysis steps within Graphical User Interfaces (GUIs). Easyreporting can be useful in supporting the reproducibility of any data analysis project and the implementation of GUIs. It turns out to be very helpful in bioinformatics, where the complexity of the analyses makes it extremely difficult to trace all the steps and parameters used in the study.


2011 ◽  
Vol 15 (1) ◽  
Author(s):  
Michael L. Fetters ◽  
Tova Garcia Duby

Faculty development programs are critical to the implementation and support of curriculum innovation. In this case study, the authors present lessons learned from ten years of experience in faculty development programs created to support innovation in technology enhanced learning. Stages of curriculum innovation are matched to stages of faculty development, and important lessons for success as well as current challenges are delineated and discussed.


Author(s):  
Kaye Chalwell ◽  
Therese Cumming

Radical subject acceleration, or moving students through a subject area faster than is typical, including skipping grades, is a widely accepted approach to support students who are gifted and talented. This is done in order to match the student’s cognitive level and learning needs. This case study explored radical subject acceleration for gifted students by focusing on one school’s response to the learning needs of a ten year old mathematically gifted student. It provides insight into the challenges, accommodations and approach to radical subject acceleration in an Australian school. It explored the processes and decisions made to ensure that a gifted student’s learning needs were met and identified salient issues for radical subject acceleration. Lessons learned from this case study may be helpful for schools considering radical acceleration.


Sign in / Sign up

Export Citation Format

Share Document