scholarly journals Designing Studies and Evaluating Research Results: Type M and Type S Errors for Pearson Correlation Coefficient

2020 ◽  
Author(s):  
Giulia Bertoldo ◽  
Claudio Zandonella Callegher ◽  
Gianmarco Altoè
Keyword(s):  
Correlation Coefficient ◽  
Effect Size ◽  
Statistical Power ◽  
Pearson Correlation ◽  
Statistical Significance ◽  
Design Analysis ◽  
Pearson Correlation Coefficient ◽  
Size Estimation ◽  
Present Contribution ◽  
Effect Size Estimation

It is widely appreciated that many studies in psychological science suffer from low statistical power. One of the consequences of analyzing underpowered studies with thresholds of statistical significance, is a high risk of finding exaggerated effect size estimates, in the right or in the wrong direction. These inferential risks can be directly quantified in terms of Type M (magnitude) error and Type S (sign) error, which directly communicate the consequences of design choices on effect size estimation. Given a study design, Type M error is the factor by which a statistically significant effect is on average exaggerated. Type S error is the probability to find a statistically significant result in the opposite direction to the plausible one. Ideally, these errors should be considered during a prospective design analysis in the design phase of a study to determine the appropriate sample size. However, they can also be considered when evaluating studies’ results in a retrospective design analysis. In the present contribution we aim to facilitate the considerations of these errors in the research practice in psychology. For this reason we illustrate how to consider Type M and Type S errors in a design analysis using one of the most common effect size measures in psychology: Pearson correlation coefficient. We provide various examples and make the R functions freely available to enable researchers to perform design analysis for their research projects.

scholarly journals Managing risk in ski resorts: Environmental factors affecting actual and estimated speed on signposted groomed slopes in a cohort of adult recreational alpine skiers

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0256349
Author(s):  
Luis Carus ◽  
Isabel Castillo
Keyword(s):  
Wind Speed ◽  
Correlation Coefficient ◽  
Environmental Conditions ◽  
Pearson Correlation ◽  
Statistical Significance ◽  
Linear Regression Analysis ◽  
Maximum Speed ◽  
Pearson Correlation Coefficient ◽  
Ski Resorts

Background Certain weather conditions are clearly harmful, increasing the risk of injury of winter sports participants substantially. The objective of this study was to investigate actual speeds of skiers on signposted groomed slopes and to measure their skill to accurately estimate them with regard to environmental conditions such as visibility, sky cover, snow quality, wind and temperature. Methods The data were obtained from a sample of 421 adult recreational skiers taking ski courses. The Pearson correlation coefficient was used to explore the relationship between actual and estimated speed for all participants. Multiple linear regression analysis was used to measure the effect of environmental conditions on both the skiers’ actual speeds and their errors of estimation. Values of 0.05 or less were considered to indicate statistical significance. Results The Pearson correlation coefficient between estimated and actual speed was 0.90 (P < 0.001). Skiers underestimated their actual speed on average by 13.06 km/h or 24.1%. Visibility, quality of snow and wind speed were shown to significantly affect both actual maximum speed and estimated speed. Good visibility, grippy snow and calm wind were associated with both the highest actual maximum speed and the lowest ability to estimate it. Conclusion Certain environmental conditions are associated with the actual speed at which skiers travel and with their ability to estimate it. Visibility, quality of snow and wind speed seem to influence both actual speed and the ability to estimate it while sky cover and temperature do not. A reinforced understanding of skiing speed on signposted groomed slopes is useful to gain insight into crashes and the mechanisms of resulting injuries, to evaluate means of protection and to devise successful prevention policies in ski resorts.

scholarly journals Improving effect size estimation and statistical power with multi-echo fMRI and its impact on understanding the neural systems supporting mentalizing

2015 ◽  
Author(s):  
Michael V. Lombardo ◽  
Bonnie Auyeung ◽  
Rosemary J. Holt ◽  
Jack Waldman ◽  
Amber N. V. Ruigrok ◽  
...  
Keyword(s):  
Effect Size ◽  
Statistical Power ◽  
Small Sample ◽  
Cognitive Domain ◽  
Group Level ◽  
Size Estimation ◽  
Sample Sizes ◽  
Brain Organization ◽  
Effect Size Estimation ◽  
Size Estimates

AbstractFunctional magnetic resonance imaging (fMRI) research is routinely criticized for being statistically underpowered due to characteristically small sample sizes and much larger sample sizes are being increasingly recommended. Additionally, various sources of artifact inherent in fMRI data can have detrimental impact on effect size estimates and statistical power. Here we show how specific removal of non-BOLD artifacts can improve effect size estimation and statistical power in task-fMRI contexts, with particular application to the social-cognitive domain of mentalizing/theory of mind. Non-BOLD variability identification and removal is achieved in a biophysical and statistically principled manner by combining multi-echo fMRI acquisition and independent components analysis (ME-ICA). Group-level effect size estimates on two different mentalizing tasks were enhanced by ME-ICA at a median rate of 24% in regions canonically associated with mentalizing, while much more substantial boosts (40-149%) were observed in non-canonical cerebellar areas. This effect size boosting is primarily a consequence of reduction of non-BOLD noise at the subject-level, which then translates into consequent reductions in between-subject variance at the group-level. Power simulations demonstrate that enhanced effect size enables highly-powered studies at traditional sample sizes. Cerebellar effects observed after applying ME-ICA may be unobservable with conventional imaging at traditional sample sizes. Thus, ME-ICA allows for principled design-agnostic non-BOLD artifact removal that can substantially improve effect size estimates and statistical power in task-fMRI contexts. ME-ICA could help issues regarding statistical power and non-BOLD noise and enable potential for novel discovery of aspects of brain organization that are currently under-appreciated and not well understood.

scholarly journals Improving effect size estimation and statistical power with multi-echo fMRI and its impact on understanding the neural systems supporting mentalizing

NeuroImage ◽  
2016 ◽  
Vol 142 ◽  
pp. 55-66 ◽  
Author(s):  
Michael V. Lombardo ◽  
Bonnie Auyeung ◽  
Rosemary J. Holt ◽  
Jack Waldman ◽  
Amber N.V. Ruigrok ◽  
...  
Keyword(s):  
Effect Size ◽  
Statistical Power ◽  
Neural Systems ◽  
Size Estimation ◽  
Effect Size Estimation

scholarly journals Another Assessment of Fat Degeneration of Retracted Supraspinatus Muscle

2018 ◽  
Vol 21 (4) ◽  
pp. 200-206
Author(s):  
Yeon Seok Jeong ◽  
Jae Kwang Yum ◽  
Sang Yoon Park
Keyword(s):  
Rotator Cuff ◽  
Correlation Coefficient ◽  
Rotator Cuff Tear ◽  
Pearson Correlation ◽  
Statistical Significance ◽  
Maximum Diameter ◽  
Pearson Correlation Coefficient ◽  
Supraspinatus Muscle ◽  
Cuff Tear ◽  
Goutallier Classification

BACKGROUND: The purpose of this study was to assess the relevance of preoperative magnetic resonance imaging (MRI) evaluation by occupation ratio (OR) at maximum diameter of supraspinatus muscle.METHODS: Patients from the Inje University Sanggye Paik Hospital who received rotator cuff repair and underwent pre- and postoperative MRI were selected as subjects of this study. On T1-weighted MRIs, OR of fat and muscle at Y-shaped view, OR at a location on supraspinatus muscle where its diameter was maximum on coronal view, and pre- and postoperative Goutallier Classification and changes in the tangent sign were measured. Statistical significance of postoperative OR was assessed regarding time from symptom onset to surgery, size of rotator cuff tear, preoperative OR, and the difference between ORs measured at maximum diameter of supraspinatus muscle and Y-shaped view.RESULTS: Preoperative OR at Y-shaped view was 52.28 ± 8.57 (32.5?65.3). Preoperative OR difference between maximum diameter and Y-shaped view was 13.76 ± 10.51 (2.38?42.04), and Pearson correlation coefficient was 0.604 (p=0.001). Postoperative OR at Y-shaped view was 63.77 ± 9.35 (37.3?76.1). Pearson correlation coefficient of pre- and postoperative Goutallier Classification was ?0.579 (p=0.002) and Pearson correlation coefficient of the postoperative difference between ORs measured at maximum diameter of supraspinatus muscle and Y-shaped view was ?0.386 (p=0.047).CONCLUSIONS: Fatty degeneration of supraspinatus muscle in rotator cuff tear patients should be evaluated not only in the conventional Y-shaped view, but also at location of maximum diameter of supraspinatus muscle to establish patients' therapeutic plan.

scholarly journals Examining reproducibility in psychology: A hybrid method for combining a statistically significant original study and a replication

2017 ◽  
Author(s):  
Robbie Cornelis Maria van Aert ◽  
Marcel A. L. M. van Assen
Keyword(s):  
Hybrid Method ◽  
Effect Size ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Open Science ◽  
Original Study ◽  
High Rate ◽  
Size Estimation ◽  
Effect Size Estimation ◽  
Science Collaboration

The unrealistic high rate of positive results within psychology increased the attention for replication research. Researchers who conduct a replication and want to statistically combine the results of their replication with a statistically significant original study encounter problems when using traditional meta-analysis techniques. The original study’s effect size is most probably overestimated because of it being statistically significant and this bias is not taken into consideration in traditional meta-analysis. We developed a hybrid method that does take statistical significance of the original study into account and enables (a) accurate effect size estimation, (b) estimation of a confidence interval, and (c) testing of the null hypothesis of no effect. We analytically approximate the performance of the hybrid method and describe its good statistical properties. Applying the hybrid method to the data of the Reproducibility Project Psychology (Open Science Collaboration, 2015) demonstrated that the conclusions based on the hybrid method are often in line with those of the replication, suggesting that many published psychological studies have smaller effect sizes than reported in the original study and that some effects may be even absent. We offer hands-on guidelines for how to statistically combine an original study and replication, and developed a web-based application (https://rvanaert.shinyapps.io/hybrid) for applying the hybrid method.

Sign in / Sign up

Export Citation Format

Share Document