Peer Review #2 of "Heritability estimation of osteoarthritis in the pig-tailed macaque (Macaca nemestrina) with a look toward future data collection (v0.1)"

Tony Davies and a number of others consider collecting supplementary spectroscopic data. Like Eurospec, the plan is to use such supplementary data not only to enhance the published paper, but also to aid thorough peer-review by allowing reviewers access to the full data rather than, as Tony puts it, “low-resolution images of data”. I’m sure you will be interested in a look at the future through this column.

Download Full-text

Analysis of environmental and biologic methyl parathion data to improve future data collection.

Environmental Health Perspectives ◽

10.1289/ehp.02110s61071 ◽

2002 ◽

Vol 110 (suppl 6) ◽

pp. 1071-1074 ◽

Cited By ~ 10

Author(s):

Rubina Imtiaz ◽

Gilbert Haugh

Keyword(s):

Data Collection ◽

Methyl Parathion ◽

Future Data

Download Full-text

Peer Review #1 of "Comparison of mobile and clinical EEG sensors through resting state simultaneous data collection (v0.1)"

10.7287/peerj.8969v0.1/reviews/1 ◽

2020 ◽

Keyword(s):

Data Collection ◽

Peer Review ◽

Resting State ◽

Clinical Eeg

Download Full-text

Registered reports: an early example and analysis

PeerJ ◽

10.7717/peerj.6232 ◽

2019 ◽

Vol 7 ◽

pp. e6232 ◽

Cited By ~ 3

Author(s):

Richard Wiseman ◽

Caroline Watt ◽

Diana Kornbrot

Keyword(s):

Data Collection ◽

Peer Review ◽

Circumstantial Evidence ◽

Focused Attention ◽

Behavioral Sciences ◽

Positive Role ◽

Questionable Research Practices ◽

Methodological Rigor ◽

Replication Crisis ◽

Crisis In Psychology

The recent ‘replication crisis’ in psychology has focused attention on ways of increasing methodological rigor within the behavioral sciences. Part of this work has involved promoting ‘Registered Reports’, wherein journals peer review papers prior to data collection and publication. Although this approach is usually seen as a relatively recent development, we note that a prototype of this publishing model was initiated in the mid-1970s by parapsychologist Martin Johnson in the European Journal of Parapsychology (EJP). A retrospective and observational comparison of Registered and non-Registered Reports published in the EJP during a seventeen-year period provides circumstantial evidence to suggest that the approach helped to reduce questionable research practices. This paper aims both to bring Johnson’s pioneering work to a wider audience, and to investigate the positive role that Registered Reports may play in helping to promote higher methodological and statistical standards.

Download Full-text

Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability

Advances in Methods and Practices in Psychological Science ◽

10.1177/2515245920958687 ◽

2020 ◽

Vol 3 (3) ◽

pp. 309-331 ◽

Cited By ~ 2

Author(s):

Charles R. Ebersole ◽

Maya B. Mathur ◽

Erica Baranski ◽

Diane-Jo Bart-Plange ◽

Nicholas R. Buttrick ◽

...

Keyword(s):

Data Collection ◽

Peer Review ◽

Total Sample ◽

Median Number ◽

Open Science ◽

Original Study ◽

Effect Sizes ◽

Replication Studies ◽

Expert Review ◽

Science Collaboration

Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically significant effect ( p < .05). Commenters suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these RP:P studies failed to replicate the original effects. We revised the replication protocols and received formal peer review prior to conducting new replication studies. We administered the RP:P and revised protocols in multiple laboratories (median number of laboratories per original study = 6.5, range = 3–9; median total sample = 1,279.5, range = 276–3,512) for high-powered tests of each original finding with both protocols. Overall, following the preregistered analysis plan, we found that the revised protocols produced effect sizes similar to those of the RP:P protocols (Δ r = .002 or .014, depending on analytic approach). The median effect size for the revised protocols ( r = .05) was similar to that of the RP:P protocols ( r = .04) and the original RP:P replications ( r = .11), and smaller than that of the original studies ( r = .37). Analysis of the cumulative evidence across the original studies and the corresponding three replication attempts provided very precise estimates of the 10 tested effects and indicated that their effect sizes (median r = .07, range = .00–.15) were 78% smaller, on average, than the original effect sizes (median r = .37, range = .19–.50).

Download Full-text

Distributed Data Collection for the Next Generation ATLAS EventIndex Project

EPJ Web of Conferences ◽

10.1051/epjconf/201921404010 ◽

2019 ◽

Vol 214 ◽

pp. 04010

Author(s):

Álvaro Fernández Casaní ◽

Dario Barberis ◽

Javier Sánchez ◽

Carlos García Montoro ◽

Santiago González de la Hoz ◽

...

Keyword(s):

Data Collection ◽

Current Approach ◽

Distributed Data ◽

Future Data ◽

Data Files ◽

Complete Catalogue ◽

And Performance ◽

Processing And Storage ◽

Storage Technologies ◽

And Storage

The ATLAS EventIndex currently runs in production in order to build a complete catalogue of events for experiments with large amounts of data. The current approach is to index all final produced data files at CERN Tier0, and at hundreds of grid sites, with a distributed data collection architecture using Object Stores to temporarily maintain the conveyed information, with references to them sent with a Messaging System. The final backend of all the indexed data is a central Hadoop infrastructure at CERN; an Oracle relational database is used for faster access to a subset of this information. In the future of ATLAS, instead of files, the event should be the atomic information unit for metadata, in order to accommodate future data processing and storage technologies. Files will no longer be static quantities, possibly dynamically aggregating data, and also allowing event-level granularity processing in heavily parallel computing environments. It also simplifies the handling of loss and or extension of data. In this sense the EventIndex may evolve towards a generalized whiteboard, with the ability to build collections and virtual datasets for end users. This proceedings describes the current Distributed Data Collection Architecture of the ATLAS EventIndex project, with details of the Producer, Consumer and Supervisor entities, and the protocol and information temporarily stored in the ObjectStore. It also shows the data flow rates and performance achieved since the new Object Store as temporary store approach was put in production in July 2017. We review the challenges imposed by the expected increasing rates that will reach 35 billion new real events per year in Run 3, and 100 billion new real events per year in Run 4. For simulated events the numbers are even higher, with 100 billion events/year in run 3, and 300 billion events/year in run 4. We also outline the challenges we face in order to accommodate future use cases in the EventIndex.

Download Full-text