scholarly journals A Strategy for Using Bias and RMSE as Outcomes in Monte Carlo Studies in Statistics

2019 ◽  
Vol 17 (2) ◽  
Author(s):  
Michael Harwell
Keyword(s):  
Monte Carlo ◽  
Root Mean Square Error ◽  
Mean Square Error ◽  
Monte Carlo Study ◽  
Error Rates ◽  
Type I ◽  
Type Ii ◽  
Mean Square ◽  
Type Ii Error ◽  
Monte Carlo Studies

To help ensure important patterns of bias and accuracy are detected in Monte Carlo studies in statistics this paper proposes conditioning bias and root mean square error (RMSE) measures on estimated Type I and Type II error rates. A small Monte Carlo study is used to illustrate this argument.

Type I and type II error rates for quantitative trait loci (QTL) mapping studies using recombinant inbred mouse strains

1996 ◽  
Vol 26 (2) ◽  
pp. 149-160 ◽  
Author(s):  
J. K. Belknap ◽  
S. R. Mitchell ◽  
L. A. O'Toole ◽  
M. L. Helms ◽  
J. C. Crabbe
Keyword(s):  
Quantitative Trait ◽  
Recombinant Inbred ◽  
Inbred Mouse ◽  
Error Rates ◽  
Mouse Strains ◽  
Type I ◽  
Inbred Mouse Strains ◽  
Type Ii ◽  
Type Ii Error ◽  
Recombinant Inbred Mouse

Predictive value of phase II clinical trials in pancreatic cancer: Rethinking the road to progress.

2013 ◽  
Vol 31 (15_suppl) ◽  
pp. 4036-4036 ◽  
Author(s):  
Daniel M. Halperin ◽  
J. Jack Lee ◽  
James C. Yao
Keyword(s):  
Clinical Trials ◽  
Phase Ii ◽  
Predictive Value ◽  
Error Rates ◽  
Phase Iii ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Phase Ii Trials ◽  
Fda Approval

4036 Background: Few new therapies for pancreatic adenocarcinoma (PC) have been approved by the Food and Drug Administration (FDA) or recommended by the National Comprehensive Cancer Network (NCCN), reflecting frequent failures in phase III trials. We hypothesize that the high failure rate in large trials is due to a low predictive value for “positive” phase II studies. Methods: Given a median time from initiation of clinical trials to FDA approval of 6.3 years, we conducted a systematic search of the clinicaltrials.gov database for phase II interventional trials of antineoplastic therapy in PC initiated from 1999-2004. We reviewed drug labels and NCCN guidelines for FDA approval and guideline recommendations. Results: We identified 70 phase II trials that met our inclusion criteria. Forty-five evaluated compounds without preexisting FDA approval, 23 evaluated drugs approved in other diseases, and 2 evaluated cellular therapies. With a median follow-up of 12.5 years, none of these drugs gained FDA approval in PC. Four trials, all combining chemotherapy with radiation, eventually resulted in NCCN recommendations. Forty-two of the trials have been published. Of 16 studies providing pre-specified type I error rates, these rates were ≥0.1 in 8 studies, 0.05 in 6 studies and <0.025 in 2 studies. Of 21 studies specifying type II error rates, 7 used >0.1, 10 used 0.1, and 4 used <0.1. Published studies reported a median enrollment of 47 subjects. Fourteen trials reported utilizing a randomized design. Conclusions: The low rate of phase II trials resulting in eventual regulatory approval of therapies for PC reflects the challenge of conquering a tough disease as well as deficiencies in the statistical designs. New strategies are necessary to quantify and improve odds of success in drug development. Statistical parameters of individual or coupled phase II trials should be tailored to achieve the desired predictive value prior to initiating pivotal phase III studies. Positive predictive value of a phase II study assuming a 1%, 2%, or 5% prior probability of success and 10% type II error rate. [Table: see text]

Summarizing Monte Carlo Results in Methodological Research

1992 ◽  
Vol 17 (4) ◽  
pp. 297-313 ◽  
Author(s):  
Michael R. Harwell
Keyword(s):  
Monte Carlo ◽  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Type I Error Rates ◽  
Power Of A Test ◽  
Methodological Research ◽  
Assumption Violations ◽  
Monte Carlo Studies ◽  
Effective Use

Monte Carlo studies provide information that can assist researchers in selecting a statistical test when underlying assumptions of the test are violated. Effective use of this literature is hampered by the lack of an overarching theory to guide the interpretation of Monte Carlo studies. The problem is exacerbated by the impressionistic nature of the studies, which can lead different readers to different conclusions. These shortcomings can be addressed using meta-analytic methods to integrate the results of Monte Carlo studies. Quantitative summaries of the effects of assumption violations on the Type I error rate and power of a test can assist researchers in selecting the best test for their data. Such summaries can also be used to evaluate the validity of previously published statistical results. This article provides a methodological framework for quantitatively integrating Type I error rates and power values for Monte Carlo studies. An example is provided using Monte Carlo studies of Bartlett’s (1937) test of equality of variances. The importance of relating meta-analytic results to exact statistical theory is emphasized.

The High Cost of Complexity in Experimental Design and Data Analysis: Type I and Type II Error Rates in Multiway ANOVA

2002 ◽  
Vol 28 (4) ◽  
pp. 515-530 ◽  
Author(s):  
Rachel A. Smith ◽  
Timothy R. Levine ◽  
Kenneth A. Lachlan ◽  
Thomas A. Fediuk
Keyword(s):  
Data Analysis ◽  
Experimental Design ◽  
Error Rates ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error

A New and Simpler Approximation for ANOVA Under Variance Heterogeneity

1994 ◽  
Vol 19 (2) ◽  
pp. 91-101 ◽  
Author(s):  
Ralph A. Alexander ◽  
Diane M. Govern
Keyword(s):  
Type I Error ◽  
Order Approximation ◽  
Error Rates ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Tail Probabilities ◽  
Type I Error Rates ◽  
Heterogeneity Of Variance ◽  
The Face

A new approximation is proposed for testing the equality of k independent means in the face of heterogeneity of variance. Monte Carlo simulations show that the new procedure has Type I error rates that are very nearly nominal and Type II error rates that are quite close to those produced by James’s (1951) second-order approximation. In addition, it is computationally the simplest approximation yet to appear, and it is easily applied to Scheffé (1959) -type multiple contrasts and to the calculation of approximate tail probabilities.

Sign in / Sign up

Export Citation Format

Share Document