Estimation of sampling error variance in the meta-analysis of correlations: Use of average correlation in the homogeneous case.

An extensive comparison of the sample-weighted method (Hunter & Schmidt, 1990), and a newer unweighted method (Osburn & Callender, 1992) of meta-analysis is presented using actual data. Several of the advantages of the unweighted method predicted by Osburn and Callendar’s simulation research did not always hold in actual application. Specifically, the unweighted method did not always produce larger estimates of observed variance, credibility intervals, and confidence intervals than the sample-weighted method when large sample outliers are present. Also, Osburn and Callender’s research on mean sampling variance formulae did not generalize to meta-analysis using the average correlation estimator to measure sample error variance. Finally, results show that while both methods may generate similar parameter and variance estimates in primary meta-analysis, they may lead researchers to reach different substantive conclusions in the analysis of moderators.

Download Full-text

Meta-analysis of Advertising Elasticity Estimates: How to Correct for the Sampling Error Bias

Vikalpa The Journal for Decision Makers ◽

10.1177/0256090919980205 ◽

1998 ◽

Vol 23 (2) ◽

pp. 29-36

Author(s):

Subir Bandyopadhyay

Keyword(s):

Causal Relationship ◽

Sampling Error ◽

Meta Analysis ◽

Error Variance ◽

Statistical Technique ◽

Social Scientists ◽

Moderator Variables ◽

Extreme Caution ◽

Research Findings ◽

Error Bias

Meta-analysis is a powerful statistical technique that allows social scientists to cumulate research findings across studies. Marketing researchers have used meta-analysis to measure the effectiveness of marketing mix strategies⁄ e.g., advertising, pricing, and promotion. Typically, researchers determine an average elasticity as a measure of the strategy to influence sales, and try to demonstrate causal relationship between the elasticity and some moderator variables. However, extreme caution should be exercised before making any causal inference from meta-analytic results. For example, the variance of an elasticity may be biased due to the sampling error in each estimate. Hence, the sampling error variance must be measured and accounted for before attempting to unravel any causal relationship between the elasticity and moderator variables. In this study by Bandyopadhyay⁄ a meta-analysis of advertising elasticity is done to demonstrate how to correct for the sampling error variance and measure the effect of moderator variables on elasticities. The results, according to the author, can help brand managers make useful inferences about the overall advertising effectiveness.

Download Full-text

Reliability of Computerized Neurocognitive Tests for Concussion Assessment: A Meta-Analysis

Journal of Athletic Training ◽

10.4085/1062-6050-52.6.03 ◽

2017 ◽

Vol 52 (9) ◽

pp. 826-833 ◽

Cited By ~ 24

Author(s):

James L. Farnsworth ◽

Lucas Dargo ◽

Brian G. Ragan ◽

Minsoo Kang

Keyword(s):

Correlation Coefficient ◽

Intraclass Correlation Coefficient ◽

Effect Size ◽

Sampling Error ◽

Meta Analysis ◽

Intraclass Correlation ◽

Cognitive Testing ◽

Test Duration ◽

Neurocognitive Tests ◽

The Impact

Objective: Although widely used, computerized neurocognitive tests (CNTs) have been criticized because of low reliability and poor sensitivity. A systematic review was published summarizing the reliability of Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) scores; however, this was limited to a single CNT. Expansion of the previous review to include additional CNTs and a meta-analysis is needed. Therefore, our purpose was to analyze reliability data for CNTs using meta-analysis and examine moderating factors that may influence reliability. Data Sources: A systematic literature search (key terms: reliability, computerized neurocognitive test, concussion) of electronic databases (MEDLINE, PubMed, Google Scholar, and SPORTDiscus) was conducted to identify relevant studies. Study Selection: Studies were included if they met all of the following criteria: used a test-retest design, involved at least 1 CNT, provided sufficient statistical data to allow for effect-size calculation, and were published in English. Data Extraction: Two independent reviewers investigated each article to assess inclusion criteria. Eighteen studies involving 2674 participants were retained. Intraclass correlation coefficients were extracted to calculate effect sizes and determine overall reliability. The Fisher Z transformation adjusted for sampling error associated with averaging correlations. Moderator analyses were conducted to evaluate the effects of the length of the test-retest interval, intraclass correlation coefficient model selection, participant demographics, and study design on reliability. Heterogeneity was evaluated using the Cochran Q statistic. Data Synthesis: The proportion of acceptable outcomes was greatest for the Axon Sports CogState Test (75%) and lowest for the ImPACT (25%). Moderator analyses indicated that the type of intraclass correlation coefficient model used significantly influenced effect-size estimates, accounting for 17% of the variation in reliability. Conclusions: The Axon Sports CogState Test, which has a higher proportion of acceptable outcomes and shorter test duration relative to other CNTs, may be a reliable option; however, future studies are needed to compare the diagnostic accuracy of these instruments.

Download Full-text

A Systematic Review and Meta-Analysis of Depression among Farming Populations Worldwide

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph17249376 ◽

2020 ◽

Vol 17 (24) ◽

pp. 9376

Author(s):

Briana N. M. Hagen ◽

Charlotte B. Winder ◽

Jared Wootten ◽

Carrie K. McMullen ◽

Andria Jones-Bitton

Keyword(s):

Systematic Review ◽

Sampling Error ◽

Meta Analysis ◽

Group Analysis ◽

The United States ◽

Self Report ◽

Design Elements ◽

Prevalence Of Depression ◽

Migrant Farm ◽

Substantial Heterogeneity

A systematic review and meta-analysis were conducted to determine the overall prevalence of depression among farming populations globally, and explore any heterogeneity present. Eligible studies were primary research articles published in English, which involved the collection of data for the purpose of determining the prevalence of depression among a farming population. Four relevant databases were searched in January 2019. Potential for bias was assessed using a modified Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool. From 7662 records, 72 articles were deemed relevant and had data extracted. Of these, 45 utilized the Center for Epidemiologic Studies—Depression Revised scale (CES-D/DR) to quantify depression, 42 of which were conducted in the United States (U.S.). As a result, meta-analyses were restricted to this geographic location. Substantial heterogeneity was seen in the initial whole-group analysis (I2 = 97%), and while sub-group exploration suggested a significantly higher prevalence of depression among migrant farm workers (26%, 95% CI = 21–31%) than in studies examining a non-migrant farming population (12%, 95% CI = 8–17%), substantial heterogeneity remained (I2 = 96%), indicating that the majority of between study variation was due to factors other than sampling error. Additionally, the majority of studies (81%) in migrant farm worker populations were published since 2010, while only 21% of studies in non-migrant farming populations were published in this timeframe. It is possible with recent efforts to de-stigmatize mental illness, participants in more recent studies may be more likely to self-report depressive symptoms. Hence, while it appears that migrant farmworker populations may have an elevated prevalence of depression, it is also apparent that little research in the U.S. has been done to evaluate depression among non-migrant farming populations in recent years. Perhaps a reporting bias may account for some of the difference between the two populations. A research gap also appears to exist in estimating the prevalence of depression among farming populations outside of the US. Assessment for bias at the study level revealed challenges in reporting of key study design elements, as well as potential for selection bias in the majority of studies.

Download Full-text

Ensemble-Based Targeted Observation Method Applied to Radar Radial Velocity Observations on Idealized Supercell Low-Level Rotation Forecasts: A Proof of Concept

Monthly Weather Review ◽

10.1175/mwr-d-19-0197.1 ◽

2020 ◽

Vol 148 (3) ◽

pp. 877-890 ◽

Cited By ~ 1

Author(s):

Christopher A. Kerr ◽

Xuguang Wang

Keyword(s):

Decision Making ◽

Radial Velocity ◽

Variance Reduction ◽

Sampling Error ◽

Error Variance ◽

Low Level ◽

Optimal Ensemble ◽

Targeted Observation ◽

Background Forecast ◽

Full Volume

Abstract The potential future installation of a multifunction phased-array radar (MPAR) network will provide capabilities of case-specific adaptive scanning. Knowing the impacts adaptive scanning may have on short-term forecasts will influence scanning strategy decision-making in hopes to produce the most optimal ensemble forecast while also benefiting human severe weather warning decision-making. An ensemble-based targeted observation algorithm is applied to an observing system simulation experiment (OSSE) where the impacts of synthetic idealized supercell radial velocity observations are estimated before the observations are “collected” and assimilated. The forecast metric of interest is the low-level rotation forecast metric (0–1-km updraft helicity), a surrogate for tornado prediction. It is found that the ensemble-based targeted observation approach can reasonably estimate the true error variance reduction when an effective method that treats sampling error is applied, the period of model forecast is associated with less degrees of nonlinearity, and the observation information content relative to the background forecast is larger. In some scenarios, a subset of a full-volume scan assimilation produces better forecasts than all observations within the full volume. Assimilating the full-volume scan increases the number of potential spurious correlations arising between the forecast metric and radial velocity observation induced state perturbations, which may degrade the forecast metric accuracy.

Download Full-text

Comparing Adjoint- and Ensemble-Sensitivity Analysis with Applications to Observation Targeting

Monthly Weather Review ◽

10.1175/2007mwr1904.1 ◽

2007 ◽

Vol 135 (12) ◽

pp. 4117-4134 ◽

Cited By ~ 118

Author(s):

Brian Ancell ◽

Gregory J. Hakim

Keyword(s):

Variance Reduction ◽

Initial Conditions ◽

Sampling Error ◽

Forecast Error ◽

Single Point ◽

Error Variance ◽

Initial Time ◽

Forecast Errors ◽

Adjoint Sensitivity ◽

Time Optimal

Abstract The sensitivity of numerical weather forecasts to small changes in initial conditions is estimated using ensemble samples of analysis and forecast errors. Ensemble sensitivity is defined here by linear regression of analysis errors onto a given forecast metric. It is shown that ensemble sensitivity is proportional to the projection of the analysis-error covariance onto the adjoint-sensitivity field. Furthermore, the ensemble-sensitivity approach proposed here involves a small calculation that is easy to implement. Ensemble- and adjoint-based sensitivity fields are compared for a representative wintertime flow pattern near the west coast of North America for a 90-member ensemble of independent initial conditions derived from an ensemble Kalman filter. The forecast metric is taken for simplicity to be the 24-h forecast of sea level pressure at a single point in western Washington State. Results show that adjoint and ensemble sensitivities are very different in terms of location, scale, and magnitude. Adjoint-sensitivity fields reveal mesoscale lower-tropospheric structures that tilt strongly upshear, whereas ensemble-sensitivity fields emphasize synoptic-scale features that tilt modestly throughout the troposphere and are associated with significant weather features at the initial time. Optimal locations for targeting can easily be determined from ensemble sensitivity, and results indicate that the primary targeting locations are located away from regions of greatest adjoint and ensemble sensitivity. It is shown that this method of targeting is similar to previous ensemble-based methods that estimate forecast-error variance reduction, but easily allows for the application of statistical confidence measures to deal with sampling error.

Download Full-text

Student Ratings of Instruction and Student Achievement: A Meta-analysis of Multisection Validity Studies

Review of Educational Research ◽

10.3102/00346543051003281 ◽

1981 ◽

Vol 51 (3) ◽

pp. 281-309 ◽

Cited By ~ 407

Author(s):

Peter A. Cohen

Keyword(s):

Student Achievement ◽

Teaching Effectiveness ◽

Meta Analysis ◽

Strong Support ◽

Student Ratings ◽

Full Time ◽

Average Correlation ◽

Student Ratings Of Instruction ◽

Validity Studies ◽

The Relationship

The present study used meta-analytic methodology to synthesize research on the relationship between student ratings of instruction and student achievement. The data for the meta-analysis came from 41 independent validity studies reporting on 68 separate multisection courses relating student ratings to student achievement. The average correlation between an overall instructor rating and student achievement was .43; the average correlation between an overall course rating and student achievement was .47. While large effect sizes were also found for more specific rating dimensions such as Skill and Structure, other dimensions showed more modest relationships with student achievement. A hierarchical multiple regression analysis showed that rating/achievement correlations were larger for full-time faculty when students knew their final grades before rating instructors and when an external evaluator graded students’ achievement tests. The results of the meta-analysis provide strong support for the validity of student ratings as measures of teaching effectiveness.

Download Full-text

Kajian Meta-Analisis Hubungan antara Self Esteem dan Kesejahteraan Psikologis

Buletin Psikologi ◽

10.22146/buletinpsikologi.9382 ◽

2017 ◽

Vol 25 (1) ◽

Author(s):

Yeni Triwahyuningsih

Keyword(s):

Sampling Error ◽

Meta Analysis ◽

Well Being ◽

Total Sample ◽

Self Esteem ◽

Psychological Wellbeing ◽

Psychological Well Being ◽

Empirically Supported ◽

Different Cultures ◽

The Relationship

In the last decade,the research on the relationship between self esteem and psychological well - being has increased. The wellbeing that distinguishes between hedonic and eudaimonic ideology is widely used in research and has been empirically supported by experts from different cultures. The results of the study about correlation between self-esteem and psychological wellbeing showed varying results. The purpose of this study was to determine the relationship between self esteem and psychological wellbeing through a meta-analysis study. The total study used was 24. Meta-analysis was performed based on sampling error. The results of the meta-analysis show generally that between self-esteem and psychological well-being is low. Correlation based on sampling error is 0.269, withi n the 95% confidence interval limit. The limited number of studies in the study may be a weakness. The accuracy of meta-analysis depends on the total sample used

Download Full-text

META-ANALYSIS FOR EVALUATING THE EFFICIENCY OF GENOMIC SELECTION IN CEREALS

Journal of Basic and Applied Genetics ◽

10.35407/bag.2020.31.01.03 ◽

2020 ◽

Vol 31 (1) ◽

pp. 23-32

Author(s):

M. A. Rueda Calderón ◽

M. Balzarini ◽

C. Bruno

Keyword(s):

Systematic Review ◽

Genomic Selection ◽

Statistical Approach ◽

Predictive Accuracy ◽

Meta Analysis ◽

Predictive Ability ◽

Genomic Data ◽

Forest Plot ◽

Average Correlation ◽

Best Linear Unbiased

Genomic selection (GS) is used to predict the merit of a genotype with respect to a quantitative trait from molecular or genomic data. Statistically, GS requires fitting a regression model with multiple predictors associated with the molecular markers (MM) states. The model is calibrated in a population with phenotypic and genomic data. The abundance and correlation of MM information make model estimation challenging. For that reason there are diverse strategies to adjust the model: based on best linear unbiased predictors (BLUP), Bayesian regressions and machine learning methods. The correlation between the observed phenotype and the predicted genetic merit by the fitted model provides a measure of the efficiency (predictive ability) of the GS. The objective of this work was to perform a metaanalysis on the efficiency of GS in cereals. A systematic review of related GS studies and a meta-analysis, in wheat and maize, was carried out to obtain a global measure of GS efficiency under different scenarios (MM quantity and statistical models used in GS). The meta-analysis indicated an average correlation coefficient of 0.61 between observed and predicted genetic merits. There were no significant differences in the efficiency of the GS based on BLUP (RR-BLUP and GBLUP), the most common statistical approach. The increase of MM data, make GS efficiency do not vary widely. Key words: Systematic review; Random effects model; Forest plot; Predictive accuracy.

Download Full-text

Harga Diri dan Adiksi Internet: Tinjauan Meta-Analisis

Buletin Psikologi ◽

10.22146/buletinpsikologi.40306 ◽

2019 ◽

Vol 27 (1) ◽

pp. 70 ◽

Cited By ~ 1

Author(s):

Wahyu Rahardjo

Keyword(s):

Social Relationships ◽

Internet Addiction ◽

Deviant Behavior ◽

Sampling Error ◽

Meta Analysis ◽

Self Esteem ◽

The Internet ◽

Men And Women ◽

Analysis Study ◽

True Correlation

The aim of this meta-analysis study is to figure out the true correlation between self-esteem and internet addiction. This meta-analysis uses 159 studies from 40 scientific articles from the year of 2005-2018 and involved in 120.825 participants. Correction for the two artifacts studied in this meta-analysis first is sampling error, and the second one is measurement error. The results support the hypothesis and show some similar findings whereas the true correlations from the groups confirm previous researches that self-esteem has a negative correlation to internet addiction. The strongest correlation found in adolescence group followed by men and women, all participant, also students and college students groups. However, these findings show that the internet accommodates individuals with negative self-esteem to build online social relationships and fulfilling their communication and pleasure needs and makes them easier committed to deviant behavior such as internet addiction.

Download Full-text