scholarly journals Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition

2018 ◽  
Vol 5 (8) ◽  
pp. 180448 ◽  
Author(s):  
Tom E. Hardwicke ◽  
Maya B. Mathur ◽  
Kyle MacDonald ◽  
Gustav Nilsonne ◽  
George C. Banks ◽  
...  

Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data (‘analytic reproducibility’). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition . Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/174, 78% post-policy), although not all data appeared reusable (23/104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly, there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.

2018 ◽  
Author(s):  
Tom Elis Hardwicke ◽  
Maya B Mathur ◽  
Kyle Earl MacDonald ◽  
Gustav Nilsonne ◽  
George Christopher Banks ◽  
...  

Access to data is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data (“analytic reproducibility”). To investigate, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/174, 78% post-policy), although not all data appeared reusable (23/104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification, and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.


2021 ◽  
pp. 002203452110202
Author(s):  
F. Schwendicke ◽  
J. Krois

Data are a key resource for modern societies and expected to improve quality, accessibility, affordability, safety, and equity of health care. Dental care and research are currently transforming into what we term data dentistry, with 3 main applications: 1) medical data analysis uses deep learning, allowing one to master unprecedented amounts of data (language, speech, imagery) and put them to productive use. 2) Data-enriched clinical care integrates data from individual (e.g., demographic, social, clinical and omics data, consumer data), setting (e.g., geospatial, environmental, provider-related data), and systems level (payer or regulatory data to characterize input, throughput, output, and outcomes of health care) to provide a comprehensive and continuous real-time assessment of biologic perturbations, individual behaviors, and context. Such care may contribute to a deeper understanding of health and disease and a more precise, personalized, predictive, and preventive care. 3) Data for research include open research data and data sharing, allowing one to appraise, benchmark, pool, replicate, and reuse data. Concerns and confidence into data-driven applications, stakeholders’ and system’s capabilities, and lack of data standardization and harmonization currently limit the development and implementation of data dentistry. Aspects of bias and data-user interaction require attention. Action items for the dental community circle around increasing data availability, refinement, and usage; demonstrating safety, value, and usefulness of applications; educating the dental workforce and consumers; providing performant and standardized infrastructure and processes; and incentivizing and adopting open data and data sharing.


2019 ◽  
Vol 10 (20) ◽  
pp. 17 ◽  
Author(s):  
Mattia Previtali ◽  
Riccardo Valente

<p>The open data paradigm is changing the research approach in many fields such as remote sensing and the social sciences. This is supported by governmental decisions and policies that are boosting the open data wave, and in this context archaeology is also affected by this new trend. In many countries, archaeological data are still protected or only limited access is allowed. However, the strong political and economic support for the publication of government data as open data will change the accessibility and disciplinary expertise in the archaeological field too. In order to maximize the impact of data, their technical openness is of primary importance. Indeed, since a spreadsheet is more usable than a PDF of a table, the availability of digital archaeological data, which is structured using standardised approaches, is of primary importance for the real usability of published data. In this context, the main aim of this paper is to present a workflow for archaeological data sharing as open data with a large level of technical usability and interoperability. Primary data is mainly acquired through the use of digital techniques (e.g. digital cameras and terrestrial laser scanning). The processing of this raw data is performed with commercial software for scan registration and image processing, allowing for a simple and semi-automated workflow. Outputs obtained from this step are then processed in modelling and drawing environments to generate digital models, both 2D and 3D. These crude geometrical data are then enriched with further information to generate a Geographic Information System (GIS) which is finally published as open data using Open Geospatial Consortium (OGC) standards to maximise interoperability.</p><p><strong>Highlights:</strong></p><ul><li><p>Open data will change the accessibility and disciplinary expertise in the archaeological field.</p></li><li><p>The main aim of this paper is to present a workflow for archaeological data sharing as open data with a large level of interoperability.</p></li><li><p>Digital acquisition techniques are used to document archaeological excavations and a Geographic Information System (GIS) is generated that is published as open data.</p></li></ul>


Author(s):  
D. P. Misra ◽  
Alka Mishra

This chapter analyzes the impact that an open data policy can have on the citizens of India. Especially in a scenario where government accountability and transparency has become the buzzword for good governance and further look at whether the availability of open data can become an agent for socio-economic change in India. What kind of change it can bring to India which has its own complexities when it comes to socio economic issues and whether the steps taken by the government are up to the mark to address these complexities through data sharing. In order to understand the changes which may occur for the good or the bad, the chapter looks at specific examples where the open data platform have been utilized in India and what impact they have had on the Indian society and how the citizens have responded to it.


2018 ◽  
Vol 74 (5) ◽  
pp. 1053-1073 ◽  
Author(s):  
Wolfgang Zenk-Möltgen ◽  
Esra Akdeniz ◽  
Alexia Katsanidou ◽  
Verena Naßhoven ◽  
Ebru Balaban

Purpose Open data and data sharing should improve transparency of research. The purpose of this paper is to investigate how different institutional and individual factors affect the data sharing behavior of authors of research articles in sociology and political science. Design/methodology/approach Desktop research analyzed attributes of sociology and political science journals (n=262) from their websites. A second data set of articles (n=1,011; published 2012-2014) was derived from ten of the main journals (five from each discipline) and stated data sharing was examined. A survey of the authors used the Theory of Planned Behavior to examine motivations, behavioral control, and perceived norms for sharing data. Statistical tests (Spearman’s ρ, χ2) examined correlations and associations. Findings Although many journals have a data policy for their authors (78 percent in sociology, 44 percent in political science), only around half of the empirical articles stated that the data were available, and for only 37 percent of the articles could the data be accessed. Journals with higher impact factors, those with a stated data policy, and younger journals were more likely to offer data availability. Of the authors surveyed, 446 responded (44 percent). Statistical analysis indicated that authors’ attitudes, reported past behavior, social norms, and perceived behavioral control affected their intentions to share data. Research limitations/implications Less than 50 percent of the authors contacted provided responses to the survey. Results indicate that data sharing would improve if journals had explicit data sharing policies but authors also need support from other institutions (their universities, funding councils, and professional associations) to improve data management skills and infrastructures. Originality/value This paper builds on previous similar research in sociology and political science and explains some of the barriers to data sharing in social sciences by combining journal policies, published articles, and authors’ responses to a survey.


2021 ◽  
pp. 374-383
Author(s):  
Branka Mraović

This paper aims to shed light on how students and young employees in Croatia assess their education for open data and what is their opinion on the compliance of the central Open Data Portal with the needs of young people as well as how they evaluate open data policy related to the young people in Croatia. This research highlights the lack of technical knowledge as a serious obstacle to the productive use of open data. As many as 56% of respondents from companies that have undergone digital transformation believe that they do not have enough knowledge to participate in open data projects, and the same scepticism is expressed by 59.6% of non-technical respondents and 45.7% of students. The data presented in this paper is part of a broader empirical research on the impact of digitalization on the transformation of the Croatian economy, carried out by the author in late 2018 on a sample of 51 young employees from 10 companies in the city of Zagreb and 70 students from 16 technical and non-technical Faculties of Zagreb University.


2019 ◽  
Author(s):  
John Towse ◽  
David Alexander Ellis ◽  
Andrea Towse

Open data-sharing is a valuable practice that ought to enhance the impact, reach and transparency of a research project. While widely advocated by many researchers and mandated by some journals and funding agencies, little is known about detailed practices across psychological science. In a pre-registered study, we show that overall, few research papers directly link to available data in many, though not all, journals. Most importantly, even where open data can be identified, the majority of these lacked completeness and reusability - conclusions that closely mirror those reported outside of Psychology. Exploring the reasons behind these findings, we offer seven specific recommendations for engineering and incentivizing improved practices, so that the potential of open data can be better realized across psychology and social science more generally.


PLoS ONE ◽  
2021 ◽  
Vol 16 (5) ◽  
pp. e0250887
Author(s):  
Luke A. McGuinness ◽  
Athena L. Sheppard

Objective To determine whether medRxiv data availability statements describe open or closed data—that is, whether the data used in the study is openly available without restriction—and to examine if this changes on publication based on journal data-sharing policy. Additionally, to examine whether data availability statements are sufficient to capture code availability declarations. Design Observational study, following a pre-registered protocol, of preprints posted on the medRxiv repository between 25th June 2019 and 1st May 2020 and their published counterparts. Main outcome measures Distribution of preprinted data availability statements across nine categories, determined by a prespecified classification system. Change in the percentage of data availability statements describing open data between the preprinted and published versions of the same record, stratified by journal sharing policy. Number of code availability declarations reported in the full-text preprint which were not captured in the corresponding data availability statement. Results 3938 medRxiv preprints with an applicable data availability statement were included in our sample, of which 911 (23.1%) were categorized as describing open data. 379 (9.6%) preprints were subsequently published, and of these published articles, only 155 contained an applicable data availability statement. Similar to the preprint stage, a minority (59 (38.1%)) of these published data availability statements described open data. Of the 151 records eligible for the comparison between preprinted and published stages, 57 (37.7%) were published in journals which mandated open data sharing. Data availability statements more frequently described open data on publication when the journal mandated data sharing (open at preprint: 33.3%, open at publication: 61.4%) compared to when the journal did not mandate data sharing (open at preprint: 20.2%, open at publication: 22.3%). Conclusion Requiring that authors submit a data availability statement is a good first step, but is insufficient to ensure data availability. Strict editorial policies that mandate data sharing (where appropriate) as a condition of publication appear to be effective in making research data available. We would strongly encourage all journal editors to examine whether their data availability policies are sufficiently stringent and consistently enforced.


Author(s):  
John N. Towse ◽  
David A Ellis ◽  
Andrea S Towse

Abstract Open data-sharing is a valuable practice that ought to enhance the impact, reach, and transparency of a research project. While widely advocated by many researchers and mandated by some journals and funding agencies, little is known about detailed practices across psychological science. In a pre-registered study, we show that overall, few research papers directly link to available data in many, though not all, journals. Most importantly, even where open data can be identified, the majority of these lacked completeness and reusability—conclusions that closely mirror those reported outside of Psychology. Exploring the reasons behind these findings, we offer seven specific recommendations for engineering and incentivizing improved practices, so that the potential of open data can be better realized across psychology and social science more generally.


2014 ◽  
Vol 38 (6) ◽  
pp. 709-722 ◽  
Author(s):  
Wolfgang Zenk-Möltgen ◽  
Greta Lepthien

Purpose – Data sharing is key for replication and re-use in empirical research. Scientific journals can play a central role by establishing data policies and providing technologies. The purpose of this paper is to analyses the factors which influence data sharing by investigating journal data policies and the behaviour of authors in sociology. Design/methodology/approach – The web sites of 140 sociology journals were consulted to check their data policy. The results are compared with similar studies from political science and economics. A broad selection of articles published in five selected journals over a period of two years are examined to determine whether authors really cite and share their data and the factors which are related to this. Findings – Although only a few sociology journals have explicit data policies, most journals make reference to a common policy supplied by their association of publishers. Among the journals selected, relatively few articles provide data citations and even fewer make data available – this is true both for journals with and without a data policy. But authors writing for journals with higher impact factors and with data policies are more likely to cite data and to make it really accessible. Originality/value – No study of journal data policies has been undertaken to date for the domain of sociology. A comparison of authors’ behaviours regarding data availability, data citation, and data accessibility for journals with or without a data policy provides useful information about the factors which improve data sharing.


Sign in / Sign up

Export Citation Format

Share Document