scholarly journals g versus c: comparing individual and collective intelligence across two meta-analyses

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Luke I. Rowe ◽  
John Hattie ◽  
Robert Hester
Keyword(s):  
Statistical Power ◽  
Group Performance ◽  
Collective Intelligence ◽  
Meta Analysis ◽  
Performance Criterion ◽  
General Factor ◽  
General Mental Ability ◽  
Meta Analyses ◽  
Primary Predictor ◽  
Better Than

AbstractCollective intelligence (CI) is said to manifest in a group’s domain general mental ability. It can be measured across a battery of group IQ tests and statistically reduced to a latent factor called the “c-factor.” Advocates have found the c-factor predicts group performance better than individual IQ. We test this claim by meta-analyzing correlations between the c-factor and nine group performance criterion tasks generated by eight independent samples (N = 857 groups). Results indicated a moderate correlation, r, of .26 (95% CI .10, .40). All but four studies comprising five independent samples (N = 366 groups) failed to control for the intelligence of individual members using individual IQ scores or their statistically reduced equivalent (i.e., the g-factor). A meta-analysis of this subset of studies found the average IQ of the groups’ members had little to no correlation with group performance (r = .06, 95% CI −.08, .20). Around 80% of studies did not have enough statistical power to reliably detect correlations between the primary predictor variables and the criterion tasks. Though some of our findings are consistent with claims that a general factor of group performance may exist and relate positively to group performance, limitations suggest alternative explanations cannot be dismissed. We caution against prematurely embracing notions of the c-factor unless it can be independently and robustly replicated and demonstrated to be incrementally valid beyond the g-factor in group performance contexts.

The Validity of Specific Cognitive Abilities for the Prediction of Training Success in Germany

2014 ◽  
Vol 13 (3) ◽  
pp. 123-133 ◽  
Author(s):  
Wiebke Goertz ◽  
Ute R. Hülsheger ◽  
Günter W. Maier
Keyword(s):  
Cognitive Abilities ◽  
Personnel Selection ◽  
Meta Analysis ◽  
Mental Ability ◽  
Occupational Category ◽  
General Mental Ability ◽  
Training Success ◽  
Specific Cognitive Abilities ◽  
Better Than

General mental ability (GMA) has long been considered one of the best predictors of training success and considerably better than specific cognitive abilities (SCAs). Recently, however, researchers have provided evidence that SCAs may be of similar importance for training success, a finding supporting personnel selection based on job-related requirements. The present meta-analysis therefore seeks to assess validities of SCAs for training success in various occupations in a sample of German primary studies. Our meta-analysis (k = 72) revealed operational validities between ρ = .18 and ρ = .26 for different SCAs. Furthermore, results varied by occupational category, supporting a job-specific benefit of SCAs.

Power-Enhanced Funnel Plots for Meta-Analysis

2020 ◽  
Vol 228 (1) ◽  
pp. 43-49 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Relevant Information ◽  
Selective Reporting ◽  
Graphical Display ◽  
Funnel Plot ◽  
Evidential Value ◽  
Graphical Displays ◽  
Funnel Plots ◽  
Meta Analyses

Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.

A Bayesian Network Meta-analysis of Add-on Drug Therapies Specific for Pulmonary Arterial Hypertension

2019 ◽  
Vol 54 (5) ◽  
pp. 423-433 ◽  
Author(s):  
Maja Petrovič ◽  
Igor Locatelli
Keyword(s):  
Pulmonary Arterial Hypertension ◽  
Combination Therapy ◽  
Arterial Hypertension ◽  
Meta Analysis ◽  
Phosphodiesterase Type 5 Inhibitors ◽  
Number Of Patients ◽  
All Cause Mortality ◽  
Pulmonary Arterial ◽  
Meta Analyses ◽  
Better Than

Background: Recently published meta-analyses did not discriminate between drug agents used for initial and sequential combination therapy. Objective: To assess the comparative efficacy of drugs specific for the treatment of pulmonary arterial hypertension (PAH) as add-on therapies based on 6-minute walk distance (6MWD), all-cause mortality, and discontinuation due to adverse events (AEs). Methods: EMBASE, PubMed, Cochrane Library, and ClinicalTrials.gov were searched until December 9, 2018, for the randomized, placebo-controlled clinical trials (RCTs) conducted on primarily adult patients diagnosed with PAH. Data extracted from applicable RCTs were as follows: for 6MWD mean change from baseline, the total number of patients, and the number of patients with events, per treatment. Network meta-analysis (NMA) was conducted in a Bayesian framework. Results: A total of 16 RCTs were eligible for analysis, with 4112 patients. Add-on therapy with tadalafil or inhaled treprostinil performed better than endothelin receptor antagonists alone [27 m; 95% credible interval (CrI): (11, 43); and 19 m; 95% CrI: (10, 27); respectively]. Add-on therapy with macitentan or bosentan performed better than phosphodiesterase type 5 inhibitors alone [26 m; 95% CrI: (6.4, 45); and 22 m; 95% CrI: (5.1, 38); respectively]. Differences in all-cause mortality and discontinuation due to AEs were nonsignificant. Conclusion and Relevance: Our NMA evaluated efficacy and safety of add-on therapies in patients with PAH. None of the previous meta-analyses evaluated RCTs focusing solely on patients pretreated with another PAH-specific drug therapy. Our results support guideline recommendations on combination therapy in PAH patients and add the quantitative perspective on which sequential therapy demonstrated the greatest effect size.

scholarly journals Quantifying the advantages of conducting a prospective meta-analysis (PMA): a case study of early childhood obesity prevention

Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
A. L. Seidler ◽  
◽  
K. E. Hunter ◽  
D. Espinoza ◽  
S. Mihrshahi ◽  
...  
Keyword(s):  
Early Childhood ◽  
Childhood Obesity ◽  
External Validity ◽  
Statistical Power ◽  
Meta Analysis ◽  
Intervention Design ◽  
Intervention Effects ◽  
Early Childhood Obesity ◽  
Limited Power ◽  
Meta Analyses

Abstract Background For prospective meta-analyses (PMAs), eligible studies are identified, and the PMA hypotheses, selection criteria, and analysis methods are pre-specified before the results of any of the studies are known. This reduces publication bias and selective outcome reporting and provides a unique opportunity for outcome standardisation/harmonisation. We conducted a world-first PMA of four trials investigating interventions to prevent early childhood obesity. The aims of this study were to quantitatively analyse the effects of prospective planning on variations across trials, outcome harmonisation, and the power to detect intervention effects, and to derive recommendations for future PMA. Methods We examined intervention design, participant characteristics, and outcomes collected across the four trials included in the EPOCH PMA using their registration records, protocol publications, and variable lists. The outcomes that trials planned to collect prior to inclusion in the PMA were compared to the outcomes that trials collected after PMA inclusion. We analysed the proportion of matching outcome definitions across trials, the number of outcomes per trial, and how collaboration increased the statistical power to detect intervention effects. Results The included trials varied in intervention design and participants, this improved external validity and the ability to perform subgroup analyses for the meta-analysis. While individual trials had limited power to detect the main intervention effect (BMI z-score), synthesising data substantially increased statistical power. Prospective planning led to an increase in the number of collected outcome categories (e.g. weight, child’s diet, sleep), and greater outcome harmonisation. Prior to PMA inclusion, only 18% of outcome categories were included in all trials. After PMA inclusion, this increased to 91% of outcome categories. However, while trials mostly collected the same outcome categories after PMA inclusion, some inconsistencies in how the outcomes were measured remained (such as measuring physical activity by hours of outside play versus using an activity monitor). Conclusion Prospective planning led to greater outcome harmonisation and greater power to detect intervention effects, while maintaining acceptable variation in trial designs and populations, which improved external validity. Recommendations for future PMA include more detailed harmonisation of outcome measures and careful pre-specification of analyses to avoid research waste by unnecessary over-collection of data.

scholarly journals Acupuncture for the Relief of Chronic Pain: A Synthesis of Systematic Reviews

Medicina ◽  
2019 ◽  
Vol 56 (1) ◽  
pp. 6
Author(s):  
Carole A. Paley ◽  
Mark I. Johnson
Keyword(s):  
Chronic Pain ◽  
Systematic Reviews ◽  
Statistical Power ◽  
English Language ◽  
Meta Analysis ◽  
Risk Of Bias ◽  
Supporting Evidence ◽  
Pharmacological Interventions ◽  
Eligibility Criteria ◽  
Meta Analyses

Background and Objectives: It is estimated that 28 million people in the UK live with chronic pain. A biopsychosocial approach to chronic pain is recommended which combines pharmacological interventions with behavioural and non-pharmacological treatments. Acupuncture represents one of a number of non-pharmacological interventions for pain. In the current climate of difficult commissioning decisions and constantly changing national guidance, the quest for strong supporting evidence has never been more important. Although hundreds of systematic reviews (SRs) and meta-analyses have been conducted, most have been inconclusive, and this has created uncertainty in clinical policy and practice. There is a need to bring all the evidence together for different pain conditions. The aim of this review is to synthesise SRs of RCTs evaluating the clinical efficacy of acupuncture to alleviate chronic pain and to consider the quality and adequacy of the evidence, including RCT design. Materials and Methods: Electronic databases were searched for English language SRs and meta-analyses on acupuncture for chronic pain. The SRs were scrutinised for methodology, risk of bias and judgement of efficacy. Results: A total of 177 reviews of acupuncture from 1989 to 2019 met our eligibility criteria. The majority of SRs found that RCTs of acupuncture had methodological shortcomings, including inadequate statistical power with a high risk of bias. Heterogeneity between RCTs was such that meta-analysis was often inappropriate. Conclusions: The large quantity of RCTs on acupuncture for chronic pain contained within systematic reviews provide evidence that is conflicting and inconclusive, due in part to recurring methodological shortcomings of RCTs. We suggest that an enriched enrolment with randomised withdrawal design may overcome some of these methodological shortcomings. It is essential that the quality of evidence is improved so that healthcare providers and commissioners can make informed choices on the interventions which can legitimately be provided to patients living with chronic pain.

scholarly journals DOES SIZE MATTER? A CRITICAL REVIEW OF META-ANALYSIS IN AGRONOMY

2019 ◽  
Vol 55 (2) ◽  
pp. 200-229 ◽  
Author(s):  
TIMOTHY J. KRUPNIK ◽  
JENS A. ANDERSSON ◽  
LEONARD RUSINAMHODZI ◽  
MARC CORBEELS ◽  
CAROL SHENNAN ◽  
...  
Keyword(s):  
Statistical Power ◽  
Agricultural Research ◽  
Meta Analysis ◽  
Research Question ◽  
Farming Systems ◽  
Conservation Agriculture ◽  
Research Results ◽  
Analytical Results ◽  
Meta Analyses ◽  
Size Matter

SUMMARYIntended to test broad hypotheses and arrive at unifying conclusions, meta-analysis is the process of extracting, assembling, and analyzing large quantities of data from multiple publications to increase statistical power and uncover explanatory patterns. This paper describes the ways in which meta-analysis has been applied to support claims and counter-claims regarding two topics widely debated in agricultural research, namely organic agriculture (OA) and conservation agriculture (CA). We describe the origins of debate for each topic and assess prominent meta-analyses considering data-selection criteria, research question framing, and the interpretation and extrapolation of meta-analytical results. Meta-analyses of OA and CA are also examined in the context of the political economy of development-oriented agricultural research. Does size matter? We suggest that it does, although somewhat ironically. While meta-analysis aims to pool all relevant studies and generate comprehensive databases from which broad insights can be drawn, our case studies suggest that the organization of many meta-analyses may affect the generalizability and usefulness of research results. The politicized nature of debates over OA and CA also appear to affect the divergent ways in which meta-analytical results may be interpreted and extrapolated in struggles over the legitimacy of both practices. Rather than resolving scientific contestation, these factors appear to contribute to the ongoing debate. Meta-analysis is nonetheless becoming increasingly popular with agricultural researchers attracted by the power for the statistical inference offered by large datasets. This paper consequently offers three suggestions for how scientists and readers of scientific literature can more carefully evaluate meta-analyses. First, the ways in which papers and data are collected should be critically assessed. Second, the justification of research questions, framing of farming systems, and the scales at which research results are extrapolated and discussed should be carefully evaluated. Third, when applied to strongly politicized topics situated in an arena of scientific debate, as is the case with OA and CA, more conservative interpretations of meta-analytical results that recognize the socially and politically embedded nature of agricultural research is are needed.

scholarly journals Low statistical power in biomedical science: a review of three human research domains

2017 ◽  
Vol 4 (2) ◽  
pp. 160254 ◽  
Author(s):  
Estelle Dumas-Mallet ◽  
Katherine S. Button ◽  
Thomas Boraud ◽  
Francois Gonon ◽  
Marcus R. Munafò
Keyword(s):  
Effect Size ◽  
Statistical Power ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Average Power ◽  
Biomedical Science ◽  
Significant Finding ◽  
Biomedical Sciences ◽  
True Effect Size ◽  
Meta Analyses

Studies with low statistical power increase the likelihood that a statistically significant finding represents a false positive result. We conducted a review of meta-analyses of studies investigating the association of biological, environmental or cognitive parameters with neurological, psychiatric and somatic diseases, excluding treatment studies, in order to estimate the average statistical power across these domains. Taking the effect size indicated by a meta-analysis as the best estimate of the likely true effect size, and assuming a threshold for declaring statistical significance of 5%, we found that approximately 50% of studies have statistical power in the 0–10% or 11–20% range, well below the minimum of 80% that is often considered conventional. Studies with low statistical power appear to be common in the biomedical sciences, at least in the specific subject areas captured by our search strategy. However, we also observe evidence that this depends in part on research methodology, with candidate gene studies showing very low average power and studies using cognitive/behavioural measures showing high average power. This warrants further investigation.

scholarly journals DEMOCRACY ISN'T THAT SMART (BUT WE CAN MAKE IT SMARTER): ON LANDEMORE'S DEMOCRATIC REASON

Episteme ◽  
2016 ◽  
Vol 14 (2) ◽  
pp. 161-175 ◽  
Author(s):  
Aaron Ancell
Keyword(s):  
Decision Procedure ◽  
Group Performance ◽  
Collective Intelligence ◽  
Decision Procedures ◽  
Recent Book ◽  
Cognitive Diversity ◽  
Individual Ability ◽  
The Individual ◽  
Average Individual ◽  
Better Than

AbstractIn her recent book, Democratic Reason, Hélène Landemore argues that, when evaluated epistemically, “a democratic decision procedure is likely to be a better decision procedure than any non-democratic decision procedures, such as a council of experts or a benevolent dictator” (p. 3). Landemore's argument rests heavily on studies of collective intelligence done by Lu Hong and Scott Page. These studies purport to show that cognitive diversity – differences in how people solve problems – is actually more important to overall group performance than average individual ability – how smart the individual members are. Landemore's argument aims to extrapolate from these results to the conclusion that democracy is epistemically better than any non-democratic rival. I argue here that Hong and Page's results actually undermine, rather than support, this conclusion. More specifically, I argue that the results do not show that democracy is better than any non-democratic alternative, and that in fact, they suggest the opposite – that at least some non-democratic alternatives are likely to epistemically outperform democracy.

scholarly journals Current Evidence Does Support the Use of KT to Treat Chronic Knee Pain in Short Term: A Systematic Review and Meta-Analysis

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Wen-hao Luo ◽  
Ye Li
Keyword(s):  
Knee Pain ◽  
Meta Analysis ◽  
Exercise Program ◽  
Current Evidence ◽  
Chronic Knee Pain ◽  
Randomized Controlled Studies ◽  
Significant Difference ◽  
The Mean ◽  
Meta Analyses ◽  
Better Than

Objective. To demonstrate whether KT is better than placebo taping, nonelastic taping, or no taping in reducing pain. Methods. PubMed, Embase, Web of Science, the Cochrane Central Library, and ClinicalTrials.gov were systematically searched up to 20 October 2020 for randomized controlled studies that used KT to treat chronic knee pain according to PRISMA guidelines. We extracted the mean differences and SD in pretreatment and posttreatment for selected outcomes measured in the experimental and control groups for subsequent meta-analyses. Results. In total, 8 studies involving 416 participants fulfilled the inclusion criteria. Our results indicated that KT is better than other tapings (placebo taping or nonelastic taping) in the early four weeks. The mean difference was −1.44 (95% CI: −2.04–−0.84, I2 = 49%, P ≤ 0.01 ). Treatment methods which were performed for more than six weeks (0.16 (95% CI: −0.35–0.68, I2 = 0%, P = 0.53 )) show no significant difference in reducing pain. In studies in which visual analogue scale was measured, a positive effect was observed for KT combined with exercise program training (−3.27 (95% CI: −3.69–2.85, I2 = 0%, P < 0.05 )). Conclusion. KT exhibited significant but temporary pain reduction.

scholarly journals P-P39 Perioperative Interventions to Reduce Pancreatic Fistula following Pancreaticoduodenectomy: A Systematic Review and Meta-Analysis

2021 ◽  
Vol 108 (Supplement_9) ◽  
Author(s):  
James Halle-Smith ◽  
Rupaly Pande ◽  
Lewis Hall ◽  
James Hodson ◽  
Keith J Roberts ◽  
...  
Keyword(s):  
Systematic Review ◽  
Pancreatic Fistula ◽  
Statistical Power ◽  
Fundamental Problem ◽  
Meta Analysis ◽  
New Directions ◽  
Randomised Controlled ◽  
Meta Analyses ◽  
Post Hoc ◽  
Pancreatic Duct Drainage

Abstract Background Many studies evaluate interventions to reduce POPF following PD, but often report conflicting results. Previous meta-analyses have generally included non-randomised trials and not considered novel interventions.  Aim To evaluate interventions to reduce postoperative pancreatic fistula (POPF) following pancreatoduodenectomy (PD) with level 1 data. Methods A systematic review and meta-analysis assessed randomised controlled trials (RCTs) evaluating interventions to reduce All-POPF or clinically relevant (CR)-POPF after PD. A post-hoc analysis of negative RCTs assessed whether these had appropriate levels of statistical power. Results Among 22 interventions (n = 7,512 patients, 55 studies), 12 were assessed by multiple studies, and subject to meta-analysis. Of these, external pancreatic duct drainage was the only intervention found to be associated with significantly reduced rates of CR- and all-POPF. In addition, Ulinastatin was associated with significantly reduced rates of CR-POPF, whilst invagination (vs duct to mucosa) pancreatojejunostomy was associated with significantly reduced rates of all-POPF. Review of negative RCTs found the majority to be underpowered, with post-hoc power calculations indicating that interventions would need to reduce the POPF rate to ≤ 1% in order to achieve 80% power in 16/34 (All-POPF) and 19/25 (CR-POPF) studies, respectively.   Conclusions Meta-analysis supports a role for several interventions to reduce POPF after PD, although data is often inconsistent and/or based on small trials. Systematic review identifies other interventions which may benefit from further study. However, underpowered trials appear to be a fundamental problem, inherently more so with CR-POPF. Larger trials, or new directions for research are required to further understanding in this field. 

Sign in / Sign up

Export Citation Format

Share Document