scholarly journals Invited Commentary: Comparing the Independent Segments Procedure with Group Sequential Designs

2021 ◽  
Author(s):  
Daniel Lakens
Keyword(s):  
Sequential Analysis ◽  
Sequential Designs ◽  
Equivalence Test ◽  
Group Sequential ◽  
Psychological Science ◽  
Type 1 Error ◽  
Software Packages ◽  
Type 2 Errors

Psychological science would become more efficient if researchers implemented sequential designs where feasible. Miller and Ulrich (2020) propose an independent segments procedure where data can be analyzed at a prespecified number of equally spaced looks while controlling the Type 1 error rate. Such procedures already exist in the sequential analysis literature, and in this commentary I reflect on whether psychologist should choose to adopt these existing procedure instead. I believe limitations in the independent segments procedure make it relatively unattractive. Being forced to stop for futility based on a bound not chosen to control Type 2 errors, or reject a smallest effect size of interest in an equivalence test, limit the inferences one can make. Having to use a prespecified number of equally spaced looks is logistically inconvenient. And not having the flexibility to choose α and β spending functions limit the possibility to design efficient studies based on the goal and limitations of the researcher. Recent software packages such as rpact (Wassmer & Pahlke, 2019) make sequential designs equally easy to perform as the independent segments procedure. While learning new statistical methods always takes time, I believe psychological scientists should start on a path that will not limit them in the flexibility and inferences their statistical procedure provides.

scholarly journals Robustness of testing procedures for confirmatory subpopulation analyses based on a continuous biomarker

2018 ◽  
Vol 28 (6) ◽  
pp. 1879-1892 ◽  
Author(s):  
Alexandra Christine Graf ◽  
Gernot Wassmer ◽  
Tim Friede ◽  
Roland Gerard Gera ◽  
Martin Posch
Keyword(s):  
Error Rate ◽  
Dependence Structure ◽  
Sequential Designs ◽  
Group Sequential ◽  
Testing Procedures ◽  
Prognostic Effect ◽  
Type 1 Error ◽  
The Family ◽  
Sequential Trials

With the advent of personalized medicine, clinical trials studying treatment effects in subpopulations are receiving increasing attention. The objectives of such studies are, besides demonstrating a treatment effect in the overall population, to identify subpopulations, based on biomarkers, where the treatment has a beneficial effect. Continuous biomarkers are often dichotomized using a threshold to define two subpopulations with low and high biomarker levels. If there is insufficient information on the dependence structure of the outcome on the biomarker, several thresholds may be investigated. The nested structure of such subpopulations is similar to the structure in group sequential trials. Therefore, it has been proposed to use the corresponding critical boundaries to test such nested subpopulations. We show that for biomarkers with a prognostic effect that is not adjusted for in the statistical model, the variability of the outcome may vary across subpopulations which may lead to an inflation of the family-wise type 1 error rate. Using simulations we quantify the potential inflation of testing procedures based on group sequential designs. Furthermore, alternative hypotheses tests that control the family-wise type 1 error rate under minimal assumptions are proposed. The methodological approaches are illustrated by a trial in depression.

Three essays on the commercialization of university discoveries

2020 ◽  
Author(s):  
◽  
Hao Cheng
Keyword(s):  
Selection Process ◽  
Empirical Studies ◽  
Project Selection ◽  
University Of Missouri ◽  
Type 1 Error ◽  
Ex Post ◽  
The University ◽  
Type 2 Error

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] Universities commercialize their discoveries at an increasing pace in order to maximize their economic impact and generate additional funding for research. They form technology transfer offices (TTOs) to evaluate the commercial value of university inventions and choose the most promising ones to patent and commercialize. Uncertainties and asymmetric information in project selection make the TTO choices difficult and can cause both type 1 error (forgo valuable discoveries) and type 2 error (select low-value discoveries). In this dissertation, I examine the TTO's project selection process and the factors that influence the choice of academic inventions for patenting and commercialization, the type 1 error committed, and the final licensing outcome. The dissertation contains three essays. In the first essay, I analyze project selection under uncertainty when both the quality of the proposed project and the motives of the applicant are uncertain. Some inventors may have an incentive to disguise the true quality and commercial value of their discoveries in order to conform to organizational expectations of disclosure while retaining rights to potentially pursue commercialization of their discoveries outside the organization's boundaries for their own benefit. Inventors may equally, ex post, lose interest to the commercialization of their invention due to competing job demands. I develop a model to examine the decision process of a university TTO responsible for the commercialization of academic inventions under such circumstances. The model describes the conditions that prompt Type 1 and Type 2 errors and allows for inferences for minimizing each. Little is known about the factors that make project selection effective or the opposite and there has been limited empirical analysis in this area. The few empirical studies that are available, examine the sources of type 2 error but there is no empirical work that analyzes type 1 error and the contributing factors. Research on type 1 error encounters two main difficulties. First, it is difficult to ascertain the decision process and second, it is challenging to approximate the counterfactual. Using data from the TTO of the University of Missouri, in the second essay I study the factors that influence the project selection process of the TTO in and the ex post type 1 error realized. In most cases, universities pursue commercialization of their inventions through licensing. There have been a few empirical studies that have researched the factors that affect licensing and their relative importance. In the third essay, I examine the characteristics of university inventions that are licensed using almost 10 years of data on several hundred of inventions, their characteristics, and the licensing status.

scholarly journals Bias, precision and timeliness of historical (background) rate comparison methods for vaccine safety monitoring: an empirical multi-database analysis

2021 ◽  
Author(s):  
Xintong Li ◽  
Lana YH Lai ◽  
Anna Ostropolets ◽  
Faaizah Arshad ◽  
Eng Hooi Tan ◽  
...  
Keyword(s):  
Vaccine Safety ◽  
Negative Control ◽  
Historical Background ◽  
Background Rate ◽  
Type 1 Error ◽  
The Cost ◽  
Type 2 Error ◽  
Safety Signals

Using real-world data and past vaccination data, we conducted a large-scale experiment to quantify bias, precision and timeliness of different study designs to estimate historical background (expected) compared to post-vaccination (observed) rates of safety events for several vaccines. We used negative (not causally related) and positive control outcomes. The latter were synthetically generated true safety signals with incident rate ratios ranging from 1.5 to 4. Observed vs. expected analysis using within-database historical background rates is a sensitive but unspecific method for the identification of potential vaccine safety signals. Despite good discrimination, most analyses showed a tendency to overestimate risks, with 20%-100% type 1 error, but low (0% to 20%) type 2 error in the large databases included in our study. Efforts to improve the comparability of background and post-vaccine rates, including age-sex adjustment and anchoring background rates around a visit, reduced type 1 error and improved precision but residual systematic error persisted. Additionally, empirical calibration dramatically reduced type 1 to nominal but came at the cost of increasing type 2 error. Our study found that within-database background rate comparison is a sensitive but unspecific method to identify vaccine safety signals. The method is positively biased, with low (<=20%) type 2 error, and 20% to 100% of negative control outcomes were incorrectly identified as safety signals due to type 1 error. Age-sex adjustment and anchoring background rate estimates around a healthcare visit are useful strategies to reduce false positives, with little impact on type 2 error. Sufficient sensitivity was reached for the identification of safety signals by month 1-2 for vaccines with quick uptake (e.g., seasonal influenza), but much later (up to month 9) for vaccines with slower uptake (e.g., varicella-zoster or papillomavirus). Finally, we reported that empirical calibration using negative control outcomes reduces type 1 error to nominal at the cost of increasing type 2 error.

Profile Tolerances Modeling: A Unified Framework for Representing Geometric Variations for Line Profiles

2015 ◽  
Author(s):  
Jiao Chen ◽  
Yuan Li ◽  
Jianfeng Yu ◽  
Wenbin Tang
Keyword(s):  
Line Profile ◽  
Assessment Method ◽  
Maximum Deviation ◽  
Unified Framework ◽  
Tolerance Zone ◽  
Type 1 Error ◽  
Split Point ◽  
Type 2 Error

Tolerance modeling is the most basic issue in Computer Aided Tolerancing (CAT). It will negatively influence the performance of subsequent activities such as tolerance analysis to a great extent if the resultant model cannot accurately represent variations in tolerance zone. According to ASME Y14.5M Standard [1], there is a class of profile tolerances for lines and surfaces which should also be interpreted correctly. Aim at this class of tolerances, the paper proposes a unified framework called DOFAS for representing them which composed of three parts: a basic DOF (Degrees of Freedom) model for interpreting geometric variations for profiles, an assessment method for filtering out and rejecting those profiles cannot be accurately represented and a split algorithm for splitting rejected profiles into sub profiles to make their variations interpretable. The scope of discussion in this paper is restricted to the line profiles; we will focus on the surface profiles in forthcoming papers. From the DOF model, two types of errors result from the rotations of the features are identified and formulized. One type of the errors is the result of the misalignment between profile boundary and tolerance zone boundary (noted as type 1); and if the feature itself exceeds the range of tolerance zone the other type of errors will form (noted as type 2). Specifically, it is required that the boundary points of the line profile should align with the corresponding boundary lines of the tolerance zone and an arbitrary point of the line profile should lie within the tolerance zone when line profile rotates in the tolerance zone. To make DOF model as accurate as possible, an assessment method and a split algorithm are developed to evaluate and eliminate these two type errors. It is clear that not all the line features carry the two type errors; as such the assessment method is used as a filter for checking and reserving such features that are consistent with the error conditions. In general, feature with simple geometry is error-free and selected by the filter whereas feature with complex geometry is rejected. According to the two type errors, two sub-procedures of the assessment process are introduced. The first one mathematically is a scheme of solving the maximum deviation of rotation trajectories of profile boundary, so as to neglect the type 1 error if it approaches to zero. The other one is to solve the maximum deviation of trajectories of all points of the feature: type 2 error can be ignored when the retrieved maximum deviation is not greater than prescribed threshold, so that the feature will always stay within the tolerance zone. For such features rejected by the filter which are inconsistent with the error conditions, the split algorithm, which is spread into the three cases of occurrence of type 1 error, occurrence of type 2 error and concurrence of two type errors, is developed to ease their errors. By utilizing and analyzing the geometric and kinematic properties of the feature, the split point is recognized and obtained accordingly. Two sub-features are retrieved from the split point and then substituted into the DOFAS framework recursively until all split features can be represented in desired resolution. The split algorithm is efficient and self-adapting lies in the fact that the rules applied can ensure high convergence rate and expected results. Finally, the implementation with two examples indicates that the DOFAS framework is capable of representing profile tolerances with enhanced accuracy thus supports the feasibility of the proposed approach.

Statistical Power in Psychiatric Research

1986 ◽  
Vol 20 (2) ◽  
pp. 189-200 ◽  
Author(s):  
Kevin D. Bird ◽  
Wayne Hall
Keyword(s):  
Sample Size ◽  
Error Rate ◽  
Statistical Power ◽  
Error Rates ◽  
Psychiatric Research ◽  
Type 1 Error ◽  
Type 2 Error ◽  
Power Analyses

Statistical power is neglected in much psychiatric research, with the consequence that many studies do not provide a reasonable chance of detecting differences between groups if they exist in the population. This paper attempts to improve current practice by providing an introduction to the essential quantities required for performing a power analysis (sample size, effect size, type 1 and type 2 error rates). We provide simplified tables for estimating the sample size required to detect a specified size of effect with a type 1 error rate of α and a type 2 error rate of β, and for estimating the power provided by a given sample size for detecting a specified size of effect with a type 1 error rate of α. We show how to modify these tables to perform power analyses for multiple comparisons in univariate and some multivariate designs. Power analyses for each of these types of design are illustrated by examples.

Errors

2017 ◽  
Author(s):  
Mark Harrison
Keyword(s):  
Emergency Medicine ◽  
Type 2 Errors

This chapter describes types of errors as applied to Emergency Medicine, and in particular the Primary FRCEM examination. The chapter outlines the key details of type 1 errors and type 2 errors. This chapter is laid out exactly following the RCEM syllabus, to allow easy reference and consolidation of learning.

THE EFFECTS OF L1–L2 PHONOLOGICAL MAPPINGS ON L2 PHONOLOGICAL SENSITIVITY

2020 ◽  
Vol 42 (5) ◽  
pp. 1041-1076
Author(s):  
Jeong-eun Kim ◽  
Yejin Cho ◽  
Youngsun Cho ◽  
Yeonjung Hong ◽  
Seohyun Kim ◽  
...  
Keyword(s):  
Response Times ◽  
Lower Sensitivity ◽  
English Sentence ◽  
Real Time Processing ◽  
Phonological Knowledge ◽  
Phonological Sensitivity ◽  
Type 2 Errors ◽  
English I

AbstractThis study examines the effects of asymmetrical mappings of L2 sounds to L1 sounds on real-time processing of L2 phonology. L1-Korean participants completed a self-paced listening (SPL) task paired with a picture verification (PV) task, in which an English sentence was presented word by word along with a picture that matched or mismatched the sentence. In the critical region, an L2 vowel was deliberately replaced with the wrong vowel for two types of English vowel pairs: Type 1: English vowel pairs showing a one-to-one mapping to Korean counterparts (e.g., English: /i/ and /æ/ to Korean /i/ and /æ/, respectively); and Type 2: English vowel pairs showing a two-to-one mapping to a Korean counterpart (e.g., English /i/ and /ɪ/ to Korean /i/). We analyzed response times (RTs) and PV accuracy. Longer RTs were observed for Type 1 errors than Type 2 errors, indicating lower sensitivity to L2 vowels with two-to-one mapping to an L1 vowel. Also, PV accuracy was lower for the sentences containing Type 2 errors. These results suggest that asymmetrical L2-L1 sound mapping can affect learners’ processing of L2 phonological knowledge, which in turn can negatively affect their comprehension.

Bioassessment of stream ecosystems enduring a decade of simulated degradation: lessons for the real world

2012 ◽  
Vol 69 (4) ◽  
pp. 784-796 ◽  
Author(s):  
Robert C. Bailey ◽  
Garry Scrimgeour ◽  
David Coté ◽  
Dan Kehler ◽  
Simon Linke ◽  
...  
Keyword(s):  
Suspended Sediments ◽  
Error Rates ◽  
Reference Condition ◽  
Benthic Macroinvertebrate ◽  
Taxon Richness ◽  
Macroinvertebrate Communities ◽  
Starting Condition ◽  
Type 2 Errors

The effects on benthic macroinvertebrate communities of simulated degradation of streams enabled evaluation of the effects of starting condition, type of degradation, and biota descriptor on the type 1 and type 2 error rates of bioassessment. Benthic macroinvertebrate communities from five reference streams in the Fraser River basin (British Columbia, Canada) were used as the starting conditions of replicated simulations of the effects of suspended sediments in three temporal patterns (none, one-time severe, constant moderate). The dynamics of the simulated stream communities and the type 1 and type 2 errors associated with bioassessments, as described by (i) taxon richness, (ii) EPT (Ephemeroptera, Plecoptera, Trichoptera) richness, (iii) proportion of EPT individuals in the community, (iv) difference in composition from the median reference community (MCDist), (v) Simpson’s diversity, and (vi) Simpson’s equitability, depended on the combination of starting condition, simpact treatment, and the biota descriptor. To reduce type 1 and type 2 errors in bioassessments using the reference condition approach, bioassessment programs should include (i) matching of test and expected reference communities and refinement of the definition of reference condition and (ii) several biota descriptors that include measures of richness, tolerance, and community composition.

RADDLE MARKINGS AND PRESENCE OF SPERMATOZOA IN VAGINAL SMEARS AS PREDICTORS OF EWES LAMBING TO MATINGS AT THE SYNCHRONIZED ESTRUS

1984 ◽  
Vol 64 (3) ◽  
pp. 563-568
Author(s):  
J. N. B. SHRESTHA ◽  
P. S. FISER ◽  
L. AINSWORTH
Keyword(s):  
Key Words ◽  
Substantial Improvement ◽  
Vaginal Smears ◽  
Type 1 Error ◽  
Ewe Lambs ◽  
Type 2 Error ◽  
Sexually Mature

A comparison was made of raddle markings and presence of spermatozoa in vaginal smears as a means of predicting which ewes will subsequently lamb. Data were obtained from 308 ewe lambs (6–7.5 mo old) and 464 sexually mature ewes (26–28 mo old) mated to experienced rams at the synchronized estrus induced by treatment with fluorogestone-acetate-impregnated intravaginal sponges and i.m. injection of pregnant mares' serum gonadotrophin. Ewes raddled during 12 h intervals from 0 to 72 h after sponge removal were recorded. Vaginal smears taken 48 and 72 h after sponge removal were examined microscopically for presence of spermatozoa. Of the ewes raddled, 41% of the ewe lambs and 86% of mature ewes, lambed. Corresponding results based on presence of spermatozoa in combined vaginal smears data were 51 and 89%. In ewe lambs, 96% of the ewes with no raddle markings and 100% of the ewes with no spermatozoa in vaginal smears were not pregnant. Corresponding results for mature ewes were 56 and 95%, respectively. This study shows that the presence of spermatozoa in vaginal smears 48 and 72 h after induction of a synchronized estrus improves the accuracy of predicting ewes that subsequently lamb to matings at the synchronized estrus over that based on ewes raddled. This improvement results from a reduction in the number of ewe lambs predicted as bred that failed to lamb (Type 1 error), and a substantial improvement in the percentage of mature ewes predicted as non-pregnant that subsequently lambed (Type 2 error). Key words: Raddle markings, spermatozoa, vaginal smears, synchronized estrus, lambing, ewes

Sign in / Sign up

Export Citation Format

Share Document