Systematic comparison of ranking aggregation methods for gene lists in experimental results

2022 ◽  
Author(s):  
Bo Wang ◽  
Andy Law ◽  
Tim Regan ◽  
Nicholas Parkinson ◽  
Joby Cole ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Simulated Data ◽  
Real Data ◽  
Biomedical Science ◽  
Aggregation Method ◽  
Aggregation Methods ◽  
The Common ◽  
Ranking Aggregation ◽  
Meta Analyses ◽  
Answer Ranking

A common experimental output in biomedical science is a list of genes implicated in a given biological process or disease. The results of a group of studies answering the same, or similar, questions can be combined by meta-analysis to find a consensus or a more reliable answer. Ranking aggregation methods can be used to combine gene lists from various sources in meta-analyses. Evaluating a ranking aggregation method on a specific type of dataset before using it is required to support the reliability of the result since the property of a dataset can influence the performance of an algorithm. Evaluation of aggregation methods is usually based on a simulated database especially for the algorithms designed for gene lists because of the lack of a known truth for real data. However, simulated datasets tend to be too small compared to experimental data and neglect key features, including heterogeneity of quality, relevance and the inclusion of unranked lists. In this study, a group of existing methods and their variations which are suitable for meta-analysis of gene lists are compared using simulated and real data. Simulated data was used to explore the performance of the aggregation methods as a function of emulating the common scenarios of real genomics data, with various heterogeneity of quality, noise level, and a mix of unranked and ranked data using 20000 possible entities. In addition to the evaluation with simulated data, a comparison using real genomic data on the SARS-CoV-2 virus, cancer (NSCLC), and bacteria (macrophage apoptosis) was performed. We summarise our evaluation results in terms of a simple flowchart to select a ranking aggregation method for genomics data.

scholarly journals Coordinate Based Random Effect Size meta-analysis of neuroimaging studies

2016 ◽  
Author(s):  
CR Tench ◽  
Radu Tanasescu ◽  
WJ Cottam ◽  
CS Constantinescu ◽  
DP Auer
Keyword(s):  
Effect Size ◽  
Error Control ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Random Effect ◽  
Simulated Data ◽  
Real Data ◽  
Type 1 Error ◽  
Meta Analyses

1AbstractLow power in neuroimaging studies can make them difficult to interpret, and Coordinate based meta‐ analysis (CBMA) may go some way to mitigating this issue. CBMA has been used in many analyses to detect where published functional MRI or voxel-based morphometry studies testing similar hypotheses report significant summary results (coordinates) consistently. Only the reported coordinates and possibly t statistics are analysed, and statistical significance of clusters is determined by coordinate density.Here a method of performing coordinate based random effect size meta-analysis and meta-regression is introduced. The algorithm (ClusterZ) analyses both coordinates and reported t statistic or Z score, standardised by the number of subjects. Statistical significance is determined not by coordinate density, but by a random effects meta-analyses of reported effects performed cluster-wise using standard statistical methods and taking account of censoring inherent in the published summary results. Type 1 error control is achieved using the false cluster discovery rate (FCDR), which is based on the false discovery rate. This controls both the family wise error rate under the null hypothesis that coordinates are randomly drawn from a standard stereotaxic space, and the proportion of significant clusters that are expected under the null. Such control is vital to avoid propagating and even amplifying the very issues motivating the meta-analysis in the first place. ClusterZ is demonstrated on both numerically simulated data and on real data from reports of grey matter loss in multiple sclerosis (MS) and syndromes suggestive of MS, and of painful stimulus in healthy controls. The software implementation is available to download and use freely.

scholarly journals Hyperbolastic Models from a Stochastic Differential Equation Point of View

Mathematics ◽  
2021 ◽  
Vol 9 (16) ◽  
pp. 1835
Author(s):  
Antonio Barrera ◽  
Patricia Román-Román ◽  
Francisco Torres-Ruiz
Keyword(s):  
Differential Equation ◽  
Simulated Data ◽  
Real Data ◽  
Point Of View ◽  
Likelihood Method ◽  
Numerical Resolution ◽  
The Family ◽  
The Common ◽  
Linear Differential ◽  
Mean Function

A joint and unified vision of stochastic diffusion models associated with the family of hyperbolastic curves is presented. The motivation behind this approach stems from the fact that all hyperbolastic curves verify a linear differential equation of the Malthusian type. By virtue of this, and by adding a multiplicative noise to said ordinary differential equation, a diffusion process may be associated with each curve whose mean function is said curve. The inference in the resulting processes is presented jointly, as well as the strategies developed to obtain the initial solutions necessary for the numerical resolution of the system of equations resulting from the application of the maximum likelihood method. The common perspective presented is especially useful for the implementation of the necessary procedures for fitting the models to real data. Some examples based on simulated data support the suitability of the development described in the present paper.

scholarly journals Coordinate based meta-analysis of networks in neuroimaging studies

2018 ◽  
Author(s):  
CR Tench ◽  
Radu Tanasescu ◽  
CS Constantinescu ◽  
DP Auer ◽  
WJ Cottam
Keyword(s):  
Spatial Clustering ◽  
Meta Analysis ◽  
Simulated Data ◽  
Real Data ◽  
Network Effect ◽  
Effect Sizes ◽  
Multivariate Normal ◽  
Z Scores ◽  
Independent Effects ◽  
Multiple Clusters

AbstractMeta-analysis of published neuroimaging results is commonly performed using coordinate based meta-analysis (CBMA). Most commonly CBMA algorithms detect spatial clustering of reported coordinates across multiple studies by assuming that results relating to the common hypothesis fall in similar anatomical locations. The null hypothesis is that studies report uncorrelated results, which is simulated by random coordinates. It is assumed that multiple clusters are independent yet it is likely that multiple results reported per study are not, and in fact represent a network effect. Here the multiple reported effect sizes (reported peak Z scores) are assumed multivariate normal, and maximum likelihood used to estimate the parameters of the covariance matrix. The hypothesis is that the effect sizes are correlated. The parameters are covariance of effect size, considered as edges of a network, while clusters are considered as nodes. In this way coordinate based meta-analysis of networks (CBMAN) estimates a network of reported meta-effects, rather than multiple independent effects (clusters).CBMAN uses only the same data as CBMA, yet produces extra information in terms of the correlation between clusters. Here it is validated on numerically simulated data, and demonstrated on real data used previously to demonstrate CBMA. The CBMA and CBMAN clusters are similar, despite the very different hypothesis.

Large-scale randomized evidence: Trials and meta-analyses of trials

2020 ◽  
pp. 51-66
Author(s):  
Colin Baigent ◽  
Richard Peto ◽  
Richard Gray ◽  
Natalie Staplin ◽  
Sarah Parish ◽  
...  
Keyword(s):  
Clinical Trials ◽  
Large Scale ◽  
Meta Analysis ◽  
False Negative ◽  
Premature Deaths ◽  
Common Causes ◽  
The Common ◽  
Meta Analyses ◽  
Positive Results

Clinical trials generally need to be able to detect or to refute realistically moderate (but still worthwhile) differences between treatments in long-term disease outcome. Large-scale randomized evidence should be able to detect such effects, but medium-sized trials or medium-sized meta-analyses can, and often do, yield false-negative or exaggeratedly positive results. Hundreds of thousands of premature deaths each year could be avoided by seeking appropriately large-scale randomized evidence about various widely practicable treatments for the common causes of death, and by disseminating this evidence appropriately. This chapter takes a look at the use of large-scale randomized evidence—produced from trials and meta-analysis of trials—and how this data should be handled in order to produce accurate result.

scholarly journals Causal inference and adjustment for reference-arm risk in indirect treatment comparison meta-analysis

2020 ◽  
Vol 9 (10) ◽  
pp. 737-750
Author(s):  
Elyse Swallow ◽  
Oscar Patterson-Lomba ◽  
Rajeev Ayyagari ◽  
Corey Pelletier ◽  
Rina Mehta ◽  
...  
Keyword(s):  
Causal Inference ◽  
Treatment Effects ◽  
Meta Analysis ◽  
Real Data ◽  
Indirect Treatment Comparison ◽  
Common Reference ◽  
Treatment Comparison ◽  
Improve Model ◽  
Indirect Treatment ◽  
Meta Analyses

Aim: To illustrate that bias associated with indirect treatment comparison and network meta-analyses can be reduced by adjusting for outcomes on common reference arms. Materials & methods: Approaches to adjusting for reference-arm effects are presented within a causal inference framework. Bayesian and Frequentist approaches are applied to three real data examples. Results: Reference-arm adjustment can significantly impact estimated treatment differences, improve model fit and align indirectly estimated treatment effects with those observed in randomized trials. Reference-arm adjustment can possibly reverse the direction of estimated treatment effects. Conclusion: Accumulating theoretical and empirical evidence underscores the importance of adjusting for reference-arm outcomes in indirect treatment comparison and network meta-analyses to make full use of data and reduce the risk of bias in estimated treatments effects.

scholarly journals History and Spread of Viruses (COVID-19) and Associated Demographic and Clinical Parameters – A systematic review and meta-analysis

2021 ◽  
Vol 33 (1) ◽  
pp. 9-24
Author(s):  
Swambhavi Awasthi ◽  
Sunil Sharma ◽  
Saurav Attri ◽  
Sakshi Malik Attri ◽  
Rajesh Sharawat ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Random Effect ◽  
Random Effect Model ◽  
Fixed Effect Model ◽  
Primary Objective ◽  
Asymptomatic Patients ◽  
Effect Model ◽  
Co Morbidity ◽  
The Common ◽  
Meta Analyses

COVID-19 made a huge impact on the world due to its rapid transmission and no treatments being available for it. The virus affected more people and spread to various countries than what was predicted when COVID-19 initially began spreading. There have been numerous pandemics and epidemics in the 21st century yet COVID-19 has affected more people and spread widely. The primary objective of the study was to explore history, spread and associated parameters of existing viruses especially COVID-19. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline was followed for a systematic search to identify eligible published articles. Clinical data, regarding COVID-19 patients, was obtained from previously published articles. The main cause of COVID-19 spreading rapidly was noted to be due to a high percentage of asymptomatic patients, transmission being air-borne, and the lack of knowledge and preventative measures being implemented when the virus began spreading. The common co-morbidity that found in patients was Diabetes Mellitus, Hypertension, and Coronary Heart Disease. The common symptoms, found through the Meta-analysis, that the patients faced included cough (55.4%), fever (68.4%), fatigue (20.3%), and shortness of breath (18.1%). The proportion of asymptotic positive cases was measured 58.3% (95%CI: 24.7% – 87.9%) while mortality proportion was found to be 6.7% (fixed-effect model) and 13.4% (random-effect model). The Meta-analysis indicated that a higher percentage of males were affected by COVID-19 than females and more patients are found to be asymptomatic. Moreover, the mortality rate of patients that have had COVID-19 was found to be low. 

scholarly journals Low statistical power in biomedical science: a review of three human research domains

2017 ◽  
Vol 4 (2) ◽  
pp. 160254 ◽  
Author(s):  
Estelle Dumas-Mallet ◽  
Katherine S. Button ◽  
Thomas Boraud ◽  
Francois Gonon ◽  
Marcus R. Munafò
Keyword(s):  
Effect Size ◽  
Statistical Power ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Average Power ◽  
Biomedical Science ◽  
Significant Finding ◽  
Biomedical Sciences ◽  
True Effect Size ◽  
Meta Analyses

Studies with low statistical power increase the likelihood that a statistically significant finding represents a false positive result. We conducted a review of meta-analyses of studies investigating the association of biological, environmental or cognitive parameters with neurological, psychiatric and somatic diseases, excluding treatment studies, in order to estimate the average statistical power across these domains. Taking the effect size indicated by a meta-analysis as the best estimate of the likely true effect size, and assuming a threshold for declaring statistical significance of 5%, we found that approximately 50% of studies have statistical power in the 0–10% or 11–20% range, well below the minimum of 80% that is often considered conventional. Studies with low statistical power appear to be common in the biomedical sciences, at least in the specific subject areas captured by our search strategy. However, we also observe evidence that this depends in part on research methodology, with candidate gene studies showing very low average power and studies using cognitive/behavioural measures showing high average power. This warrants further investigation.

scholarly journals Assessing Functional Relations in Single-Case Designs

2014 ◽  
Vol 38 (6) ◽  
pp. 878-913 ◽  
Author(s):  
Rumen Manolov ◽  
Vicenta Sierra ◽  
Antonio Solanas ◽  
Juan Botella
Keyword(s):  
Visual Analysis ◽  
Single Case ◽  
Meta Analysis ◽  
Functional Relation ◽  
Simulated Data ◽  
Evidence Based Practices ◽  
Functional Relations ◽  
Data Variability ◽  
Single Case Designs ◽  
Meta Analyses

In the context of the evidence-based practices movement, the emphasis on computing effect sizes and combining them via meta-analysis does not preclude the demonstration of functional relations. For the latter aim, we propose to augment the visual analysis to add consistency to the decisions made on the existence of a functional relation without losing sight of the need for a methodological evaluation of what stimuli and reinforcement or punishment are used to control the behavior. Four options for quantification are reviewed, illustrated, and tested with simulated data. These quantifications include comparing the projected baseline with the actual treatment measurements, on the basis of either parametric or nonparametric statistics. The simulated data used to test the quantifications include nine data patterns in terms of the presence and type of effect and comprise ABAB and multiple-baseline designs. Although none of the techniques is completely flawless in terms of detecting a functional relation only when it is present but not when it is absent, an option based on projecting split-middle trend and considering data variability as in exploratory data analysis proves to be the best performer for most data patterns. We suggest that the information on whether a functional relation has been demonstrated should be included in meta-analyses. It is also possible to use as a weight the inverse of the data variability measure used in the quantification for assessing the functional relation. We offer an easy to use code for open-source software for implementing some of the quantifications.

scholarly journals Neural effects of antidepressant medication and psychological treatments: a quantitative synthesis across three meta-analyses

2021 ◽  
pp. 1-5
Author(s):  
Camilla L. Nord ◽  
Lisa Feldman Barrett ◽  
Kristen A. Lindquist ◽  
Yina Ma ◽  
Lindsey Marwood ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Antidepressant Medication ◽  
Neural Activation ◽  
Negative Valence ◽  
Psychological Therapies ◽  
Quantitative Synthesis ◽  
The Common ◽  
Using Data ◽  
Meta Analyses ◽  
Affect Processing

Background Influential theories predict that antidepressant medication and psychological therapies evoke distinct neural changes. Aims To test the convergence and divergence of antidepressant- and psychotherapy-evoked neural changes, and their overlap with the brain's affect network. Method We employed a quantitative synthesis of three meta-analyses (n = 4206). First, we assessed the common and distinct neural changes evoked by antidepressant medication and psychotherapy, by contrasting two comparable meta-analyses reporting the neural effects of these treatments. Both meta-analyses included patients with affective disorders, including major depressive disorder, generalised anxiety disorder and panic disorder. The majority were assessed using negative-valence tasks during neuroimaging. Next, we assessed whether the neural changes evoked by antidepressants and psychotherapy overlapped with the brain's affect network, using data from a third meta-analysis of affect-based neural activation. Results Neural changes from psychotherapy and antidepressant medication did not significantly converge on any region. Antidepressants evoked neural changes in the amygdala, whereas psychotherapy evoked anatomically distinct changes in the medial prefrontal cortex. Both psychotherapy- and antidepressant-related changes separately converged on regions of the affect network. Conclusions This supports the notion of treatment-specific brain effects of antidepressants and psychotherapy. Both treatments induce changes in the affect network, but our results suggest that their effects on affect processing occur via distinct proximal neurocognitive mechanisms of action.

scholarly journals A Systematic Review and Meta-Analysis of the Relationship Between Social Dominance Status and Common Behavioral Phenotypes in Male Laboratory Mice

2021 ◽  
Vol 14 ◽  
Author(s):  
Justin A. Varholick ◽  
Jeremy D. Bailoo ◽  
Ashley Jenkins ◽  
Bernhard Voelkl ◽  
Hanno Würbel
Keyword(s):  
Systematic Review ◽  
Learned Helplessness ◽  
Social Dominance ◽  
Forced Swim Test ◽  
Meta Analysis ◽  
Assessment Method ◽  
Biomedical Science ◽  
Laboratory Mice ◽  
Dominance Status ◽  
Meta Analyses

Background: Social dominance status (e.g., dominant or subordinate) is often associated with individual differences in behavior and physiology but is largely neglected in experimental designs and statistical analysis plans in biomedical animal research. In fact, the extent to which social dominance status affects common experimental outcomes is virtually unknown. Given the pervasive use of laboratory mice and culminating evidence of issues with reproducibility, understanding the role of social dominance status on common behavioral measures used in research may be of paramount importance.Methods: To determine whether social dominance status—one facet of the social environment—contributes in a systematic way to standard measures of behavior in biomedical science, we conducted a systematic review of the existing literature searching the databases of PubMed, Embase, and Web of Science. Experiments were divided into several domains of behavior: exploration, anxiety, learned helplessness, cognition, social, and sensory behavior. Meta-analyses between experiments were conducted for the open field, elevated plus-maze, and Porsolt forced swim test.Results: Of the 696 publications identified, a total of 55 experiments from 20 published studies met our pre-specified criteria. Study characteristics and reported results were highly heterogeneous across studies. A systematic review and meta-analyses, where possible, with these studies revealed little evidence for systematic phenotypic differences between dominant and subordinate male mice.Conclusion: This finding contradicts the notion that social dominance status impacts behavior in significant ways, although the lack of an observed relationship may be attributable to study heterogeneity concerning strain, group-size, age, housing and husbandry conditions, and dominance assessment method. Therefore, further research considering these secondary sources of variation may be necessary to determine if social dominance generally impacts treatment effects in substantive ways.

Sign in / Sign up

Export Citation Format

Share Document