Systematic review of guidelines for internal validity in the design, conduct and analysis of preclinical biomedical experiments involving laboratory animals

Over the last two decades, awareness of the negative repercussions of flaws in the planning, conduct and reporting of preclinical research involving experimental animals has been growing. Several initiatives have set out to increase transparency and internal validity of preclinical studies, mostly publishing expert consensus and experience. While many of the points raised in these various guidelines are identical or similar, they differ in detail and rigour. Most of them focus on reporting, only few of them cover the planning and conduct of studies. The aim of this systematic review is to identify existing experimental design, conduct, analysis and reporting guidelines relating to preclinical animal research. A systematic search in PubMed, Embase and Web of Science retrieved 13 863 unique results. After screening these on title and abstract, 613 papers entered the full-text assessment stage, from which 60 papers were retained. From these, we extracted unique 58 recommendations on the planning, conduct and reporting of preclinical animal studies. Sample size calculations, adequate statistical methods, concealed and randomised allocation of animals to treatment, blinded outcome assessment and recording of animal flow through the experiment were recommended in more than half of the publications. While we consider these recommendations to be valuable, there is a striking lack of experimental evidence on their importance and relative effect on experiments and effect sizes.

Download Full-text

Quality of Reporting in Preclinical Urethral Tissue Engineering Studies: A Systematic Review to Assess Adherence to the ARRIVE Guidelines

Animals ◽

10.3390/ani11082456 ◽

2021 ◽

Vol 11 (8) ◽

pp. 2456

Author(s):

Tariq O. Abbas ◽

Abubakr Elawad ◽

Abdul Kareem Pullattayil S. ◽

Cristian Pablo Pennisi

Keyword(s):

Systematic Review ◽

Tissue Engineering ◽

Animal Studies ◽

Therapeutic Option ◽

Laboratory Animals ◽

Size Estimation ◽

Ethical Guidelines ◽

Preclinical Research ◽

Quality Of Reporting

Preclinical research within the area of urethral tissue engineering has not yet been successfully translated into an efficient therapeutic option for patients. This gap could be attributed, in part, to inadequate design and reporting of the studies employing laboratory animals. In this study, a systematic review was conducted to investigate the quality of reporting in preclinical studies utilizing tissue engineering approaches for urethral repair. The scope was on studies performed in rabbits, published between January 2014 and March 2020. Quality assessment of the data was conducted according to the Animal Research: Reporting of in Vivo Experiments (ARRIVE) guidelines by the scoring of a 38-item checklist in different categories. A total of 28 articles that fulfilled the eligibility criteria were included in the study. The range of ARRIVE score was from 0 to 100, taking into consideration having reported the item in question or not. The mean checklist score was 53%. The items that attained the highest scores included the number of animals utilized, the size of control and experimental groups, and the definition of experimental outcomes. The least frequently reported items included the data regarding the experimental procedure, housing and husbandry, determination and justification of the number of animals, and reporting of adverse events. Surprisingly, full disclosure about ethical guidelines and animal protocol approval was missing in 54% of the studies. No paper stated the sample size estimation. Overall, our study found that a large number of studies display inadequate reporting of fundamental information and that the quality of reporting improved marginally over the study period. We encourage a comprehensive implementation of the ARRIVE guidelines in animal studies exploring tissue engineering for urethral repair, not only to facilitate effective translation of preclinical research findings into clinical therapies, but also to ensure compliance with ethical principles and to minimize unnecessary animal studies.

Download Full-text

Protocol for a systematic review of guidelines for rigour in the design, conduct and analysis of biomedical experiments involving laboratory animals

BMJ Open Science ◽

10.1136/bmjos-2018-000004 ◽

2018 ◽

Vol 2 (1) ◽

pp. e000004 ◽

Cited By ~ 1

Author(s):

Jan Vollert ◽

Esther Schenker ◽

Malcolm Macleod ◽

Anton Bespalov ◽

Hanno Wuerbel ◽

...

Keyword(s):

Systematic Review ◽

Biomedical Research ◽

Animal Studies ◽

English Language ◽

Internal Validity ◽

Laboratory Animals ◽

Validity And Reliability ◽

Standardized Reporting ◽

Two Phases ◽

Google Search

ObjectiveWithin the last years, there has been growing awareness of the negative repercussions of unstandardized planning, conduct and reporting of preclinical and biomedical research. Several initiatives have set the aim of increasing validity and reliability in reporting of studies and publications, and publishers have formed similar groups. Additionally, several groups of experts across the biomedical spectrum have published experience and opinion-based guidelines and guidance on potential standardized reporting. While all these guidelines cover reporting of experiments, an important step prior to this should be rigours planning and conduction of studies. The aim of this systematic review is to identify and harmonize existing experimental design, conduct and analysis guidelines relating to internal validity and reproducibility of preclinical animal research. The review will also identify literature describing risks of bias pertaining to the design, conduct and analysis of preclinical biomedical research.Search strategyPubMed, Embase and Web of Science will be searched systematically to identify guidelines published in English language in peer-reviewed journals before January 2018 (box 1). All articles or systematic reviews in English language that describe or review guidelines on the internal validity and reproducibility of animal studies will be included. Google search for guidelines published on the websites of major funders and professional organisations can be found in (Box 2).Screening and annotationUnique references will be screened in two phases: screening for eligibility based on title and abstract, followed by screening for definitive inclusion based on full text. Screening will be performed in SyRF (http://syrf.org.uk). Each reference will be randomly presented to two independent reviewers. Disagreements between reviewers will be resolved by additional screening of the reference by a third, senior researcher.Data management and reportingAll data, including extracted text and guidelines, will be stored in the SyRF platform. Elements of the included guidelines will be identified using a standardized extraction form. Reporting will follow the PRISMA guidelines as far as applicable.

Download Full-text

Astrocyte Biomarkers in Alzheimer Disease: A Systematic Review and Meta-analysis

Neurology ◽

10.1212/wnl.0000000000012109 ◽

2021 ◽

pp. 10.1212/WNL.0000000000012109

Author(s):

Bruna Bellaver ◽

João Pedro Ferrari-Souza ◽

Lucas Uglione da Ros ◽

Stephen F Carter ◽

Elena Rodriguez-Vieitez ◽

...

Keyword(s):

Systematic Review ◽

Web Of Science ◽

Meta Analysis ◽

Effect Sizes ◽

Significant Progress ◽

Exclusion Criteria ◽

Mao B ◽

Positron Emission ◽

Registration Number ◽

Mean Differences

Objective:To perform a systematic review and meta-analysis to determine whether fluid and imaging astrocyte biomarkers are altered in Alzheimer's disease (AD).Methods:PubMed and Web of Science databases were searched for articles reporting fluid or imaging astrocyte biomarkers in AD. Pooled effect sizes were determined with mean differences (SMD) using the Hedge’s G method with random-effects to determine biomarker performance. Adapted questions from QUADAS-2 were applied for quality assessment. A protocol for this study has been previously registered in PROSPERO (registration number: CRD42020192304).Results:The initial search identified 1,425 articles. After exclusion criteria were applied, 33 articles (a total of 3,204 individuals) measuring levels of GFAP, S100B, YKL-40 and AQP4 in the blood and cerebrospinal fluid (CSF), as well as MAO-B, indexed by positron emission tomography 11C-deuterium-L-deprenyl ([11C]-DED), were included. GFAP (SMD = 0.94; 95% CI = 0.71-1.18) and YKL-40 (SMD = 0.76; CI 95% = 0.63-0.89) levels in the CSF, S100B levels in the blood (SMD = 2.91; CI 95% = 1.01-4.8) were found significantly increased in AD patients.Conclusions:Despite significant progress, applications of astrocyte biomarkers in AD remain in their early days. The meta-analysis demonstrated that astrocyte biomarkers are consistently altered in AD and supports further investigation for their inclusion in the AD clinical research framework for observational and interventional studies.

Download Full-text

Trigeminal neuralgia and genetics: A systematic review

Molecular Pain ◽

10.1177/17448069211016139 ◽

2021 ◽

Vol 17 ◽

pp. 174480692110161

Author(s):

Mari Aaroe Mannerak ◽

Aslan Lashkarivand ◽

Per Kristian Eide

Keyword(s):

Systematic Review ◽

Animal Models ◽

Trigeminal Neuralgia ◽

Animal Studies ◽

Genetic Factors ◽

Web Of Science ◽

Cochrane Library ◽

Trigeminal Pain ◽

Familial Form

Trigeminal neuralgia (TN) is a severe facial pain disease of unknown cause and unclear genetic background. To examine the existing knowledge about genetics in TN, we performed a systematic study asking about the prevalence of familial trigeminal neuralgia, and which genes that have been identified in human TN studies and in animal models of trigeminal pain. MedLine, Embase, Cochrane Library and Web of Science were searched from inception to January 2021. 71 studies were included in the systematic review. Currently, few studies provide information about the prevalence of familial TN; the available evidence indicates that about 1–2% of TN cases have the familial form. The available human studies propose the following genes to be possible contributors to development of TN: CACNA1A, CACNA1H, CACNA1F, KCNK1, TRAK1, SCN9A, SCN8A, SCN3A, SCN10A, SCN5A, NTRK1, GABRG1, MPZ gene, MAOA gene and SLC6A4. Their role in familial TN still needs to be addressed. The experimental animal studies suggest an emerging role of genetics in trigeminal pain, though the animal models may be more relevant for trigeminal neuropathic pain than TN per se. In summary, this systematic review suggests a more important role of genetic factors in TN pathogenesis than previously assumed.

Download Full-text

Translational Intracerebral Hemorrhage Research: Has Current Neuroprotection Research ARRIVEd at a Standard for Experimental Design and Reporting?

Translational Stroke Research ◽

10.1007/s12975-020-00824-x ◽

2020 ◽

Vol 11 (6) ◽

pp. 1203-1213 ◽

Cited By ~ 2

Author(s):

Lane J. Liddle ◽

Shivani Ralhan ◽

Daniel L. Ward ◽

Frederick Colbourne

Keyword(s):

Experimental Design ◽

Intracerebral Hemorrhage ◽

Animal Studies ◽

Effect Sizes ◽

Healthy Male ◽

Preclinical Research ◽

Exclusion Criteria ◽

Current Review ◽

Pubmed Database ◽

Design Characteristics

Abstract One major aim of preclinical intracerebral hemorrhage (ICH) research is to develop and test potential neuroprotectants. Published guidelines for experimental design and reporting stress the importance of clearly and completely reporting results and methodological details to ensure reproducibility and maximize information availability. The current review has two objectives: first, to characterize current ICH neuroprotection research and, second, to analyze aspects of translational design in preclinical ICH studies. Translational design is the adoption and reporting of experimental design characteristics that are thought to be clinically relevant and critical to reproducibility in animal studies (e.g., conducting and reporting experiments according to the STAIR and ARRIVE guidelines, respectively). Given that ICH has no current neuroprotective treatments and an ongoing reproducibility crisis in preclinical research, translational design should be considered by investigators. We conducted a systematic review of ICH research from 2015 to 2019 using the PubMed database. Our search returned 281 published manuscripts studying putative neuroprotectants in animal models. Contemporary ICH research predominantly uses young, healthy male rodents. The collagenase model is the most commonly used. Reporting of group sizes, blinding, and randomization are almost unanimous, but group size calculations, mortality and exclusion criteria, and animal model characteristics are infrequently reported. Overall, current ICH neuroprotection research somewhat aligns with experimental design and reporting guidelines. However, there are areas for improvement. Because failure to consider translational design is associated with inflation of effect sizes (and possibly hindered reproducibility), we suggest that researchers, editors, and publishers collaboratively consider enhanced adherence to published guidelines.

Download Full-text

Relationship between gestational acrylamide exposure and offspring’s growth: a systematic review and meta-analysis of cohort studies

Public Health Nutrition ◽

10.1017/s1368980019005123 ◽

2020 ◽

Vol 23 (10) ◽

pp. 1791-1799

Author(s):

Yongle Zhan ◽

Ying Xiao ◽

Tianjia Guan ◽

Shuyang Zhang ◽

Yu Jiang

Keyword(s):

Systematic Review ◽

Cohort Studies ◽

Web Of Science ◽

Meta Analysis ◽

Maternal Diet ◽

Effect Sizes ◽

Current Evidence ◽

Random Effects Model ◽

Exposure Group ◽

Dietary Strategy

AbstractObjective:To estimate the current evidence regarding the association between gestational acrylamide (AA) exposure and offspring’s growth.Design:Systematic review and meta-analysis.Setting:A systematic literature search for relevant publications was conducted using PubMed, Medline, Embase, Web of Science databases from inception to 26 April 2019. The standardised mean difference (SMD) or OR with 95 % CI was selected as the effect sizes and was calculated using a random effects model.Results:Five cohort studies including 54 728 participants were identified. Offspring’s birth weight was significantly lower in high AA exposure group than in low AA exposure group (SMD –0·05, 95 % CI –0·09, –0·02, P = 0·005). There was also an association between maternal AA exposure and small for gestational age (OR 1·14, 95 % CI 1·06, 1·23, P < 0·001). In addition, pooled ORs suggested that children had a high risk of developing overweight/obesity in the future in maternal high AA exposure group (OR 1·14, 95 % CI 1·08, 1·21, P < 0·001 at age 3; OR 1·13, 95 % CI 1·07, 1·19, P < 0·001 at age 5; OR 1·09, 95 % CI 1·02, 1·16, P = 0·020 at age 8).Conclusions:These findings have important implications for conducting health education, providing guidance on maternal diet and developing an appropriate dietary strategy for pregnant women to reduce dietary AA exposure.

Download Full-text

Outcome heterogeneity and bias in acute experimental spinal cord injury

Neurology ◽

10.1212/wnl.0000000000007718 ◽

2019 ◽

Vol 93 (1) ◽

pp. e40-e51 ◽

Cited By ~ 8

Author(s):

Ralf Watzlawick ◽

Ana Antonic ◽

Emily S. Sena ◽

Marcel A. Kopp ◽

Julian Rind ◽

...

Keyword(s):

Spinal Cord ◽

Spinal Cord Injury ◽

Internal Validity ◽

Effect Sizes ◽

Response To Treatment ◽

Preclinical Research ◽

Neurologic Recovery ◽

Experimental Spinal Cord Injury ◽

Cord Injury ◽

The Impact

ObjectiveTo determine whether and to what degree bias and underestimated variability undermine the predictive value of preclinical research for clinical translation.MethodsWe investigated experimental spinal cord injury (SCI) studies for outcome heterogeneity and the impact of bias. Data from 549 preclinical SCI studies including 9,535 animals were analyzed with meta-regression to assess the effect of various study characteristics and the quality of neurologic recovery.ResultsOverall, the included interventions reported a neurobehavioral outcome improvement of 26.3% (95% confidence interval 24.3–28.4). Response to treatment was dependent on experimental modeling paradigms (neurobehavioral score, site of injury, and animal species). Applying multiple outcome measures was consistently associated with smaller effect sizes compared with studies applying only 1 outcome measure. More than half of the studies (51.2%) did not report blinded assessment, constituting a likely source of evaluation bias, with an overstated effect size of 7.2%. Assessment of publication bias, which extrapolates to identify likely missing data, suggested that between 2% and 41% of experiments remain unpublished. Inclusion of these theoretical missing studies suggested an overestimation of efficacy, reducing the effect sizes by between 0.9% and 14.3%.ConclusionsWe provide empirical evidence of prevalent bias in the design and reporting of experimental SCI studies, resulting in overestimation of the effectiveness. Bias compromises the internal validity and jeopardizes the successful translation of SCI therapies from the bench to bedside.

Download Full-text

Polyphenols as Prebiotics in the Management of High-Fat Diet-Induced Obesity: A Systematic Review of Animal Studies

Foods ◽

10.3390/foods10020299 ◽

2021 ◽

Vol 10 (2) ◽

pp. 299

Author(s):

Mohanambal Moorthy ◽

Usha Sundralingam ◽

Uma D. Palanisamy

Keyword(s):

Systematic Review ◽

Body Weight ◽

High Fat Diet ◽

Animal Studies ◽

Literature Search ◽

Web Of Science ◽

Preclinical Studies ◽

High Fat ◽

Diet Induced Obesity

Obesity is a disease growing at an alarming rate and numerous preclinical studies have proven the role of polyphenols in managing this disease. This systematic review explores the prebiotic effect of polyphenols in the management of obesity among animals fed on a high-fat diet. A literature search was carried out in PubMed, Scopus, CINAHL, Web of Science, and Embase databases following the PRISMA guidelines. Forty-four studies reported a significant reduction in obesity-related parameters. Most notably, 83% of the studies showed a decrease in either body weight/visceral adiposity/plasma triacylglyceride. Furthermore, 42 studies reported a significant improvement in gut microbiota (GM), significantly affecting the genera Akkermansia, Bacteroides, Blautia, Roseburia, Bifidobacteria, Lactobacillus, Alistipes, and Desulfovibrio. Polyphenols’ anti-obesity, anti-hyperglycaemic, and anti-inflammatory properties were associated with their ability to modulate GM. This review supports the notion of polyphenols as effective prebiotics in ameliorating HFD-induced metabolic derangements in animal models.

Download Full-text

Substance use disorders and suicidality in youth: A systematic review and meta-analysis with a focus on the direction of the association

PLoS ONE ◽

10.1371/journal.pone.0255799 ◽

2021 ◽

Vol 16 (8) ◽

pp. e0255799

Author(s):

Charlie Rioux ◽

Anne-Sophie Huet ◽

Natalie Castellanos-Ryan ◽

Laurianne Fortier ◽

Myriam Le Blanc ◽

...

Keyword(s):

Mental Health ◽

Systematic Review ◽

Substance Use ◽

Psychiatric Disorder ◽

Web Of Science ◽

Meta Analysis ◽

Effect Sizes ◽

Secondary Substance ◽

Bidirectional Associations ◽

Meta Analyses

Background Reviews and meta-analyses suggest that substance use and suicidality (i.e., suicidal ideations and attempts) are associated in youth, but the direction of this association remains unclear. Theoretically, the secondary psychiatric disorder hypothesis (SPDH) posits that substance use leads to suicidality, while the secondary substance use disorder hypothesis (SSUDH) posits that suicidality leads to substance use. To clarify these associations, this meta-analysis systematically reviewed studies that examined the prospective associations between SUDs and suicidality in youth (age 25 and younger) and compared results according to the direction of the association. Methods Web of Science, Embase, PsycINFO, PubMed, Medline and ProQuest Dissertations & Theses Global were searched from inception to March 8, 2020, and 55 effect sizes from 23 samples were included and analyzed using a three-level meta-analysis. Results SUDs significantly predicted subsequent suicidality (OR = 2.16, 95%CI 1.57–2.97), suicidality significantly predicted subsequent SUDs (OR = 2.16, 95%CI 1.53–3.04), and these effect sizes did not differ (p = 0.49). Conclusions Considering that 65% of reviewed studies only examined the SPDH, this review highlights that more attention should be given to the SSUDH, and that studies should examine bidirectional associations between SUDs and suicidality across time. Clinically, because SUDs and suicidality were found to influence each other, results suggest that mental health and SUDs should ideally be detected and treated early, and that co-occurring disorders should be assessed and treated concomitantly.

Download Full-text

Laboratory animals search filter for different literature databases: PubMed, Embase, Web of Science and PsycINFO

Laboratory Animals ◽

10.1177/00236772211045485 ◽

2021 ◽

pp. 002367722110454

Author(s):

Stevie van der Mierden ◽

Carlijn R Hooijmans ◽

Alice HJ Tillema ◽

Simone Rehn ◽

André Bleich ◽

...

Keyword(s):

Random Sample ◽

Systematic Reviews ◽

Animal Studies ◽

Web Of Science ◽

Real Life ◽

Animal Research ◽

Laboratory Animals ◽

Primary Study ◽

Search Filter ◽

Search Filters

Systematic reviews are important tools in animal research, but the ever-increasing number of studies makes retrieval of all relevant publications challenging. Search filters aid in retrieving as many animal studies as possible. In this paper we provide updated and expanded versions of the SYRCLE animal filters for PubMed and Embase. We provide the Embase filter for both Embase.com and via Ovid. Furthermore, we provide new animal search filters for Web of Science (WoS) and APA PsycINFO via psycnet.apa.org and via Ovid. Compared with previous versions, the new filters retrieved 0.5–47.1% (19 references for PubMed, 837 for WoS) more references in a real-life example. All filters retrieved additional references, comprising multiple relevant reviews. A random sample from WoS found at least one potentially relevant primary study. These animal search filters facilitate identifying as many animal studies as possible while minimising the number of non-animal studies.

Download Full-text