Planned missing data in early literacy interventions: A replication study with an additional gold standard

Introduction In a digital early literacy intervention RCT, children born late preterm fell behind peers when in a control condition, but outperformed them when assigned to the intervention. Results did however not replicate previous findings. Replication is often complicated by resource quality. Gold Standard measures are generally time-intensive and costly, while they closely align with, and are more sensitive to changes in, early literacy and language performance. A planned missing data approach, leaving these gold standard measures incomplete, might aid in addressing the origin(s) of non-replication. Methods Participants after consent were 695 p Dutch primary school pupils of normal and late preterm birth. The high-quality measures, in additional to simpler but complete measures, were intentionally administered to a random subsample of children. Five definitions of gold standard alignment were evaluated. Results Two out of five gold standard levels improved precision compared to the original results. The lowest gold standard level did not lead to improvement: precision was actually diminished. In two gold standard definitions, an alphabetical factor and a writing-only factor the model estimates were comparable to the original results. Only the most precise definition of the gold standard level replicated the original results. Conclusion Gold standard measures could only be used to improve model efficiency in RCT-designs under sufficiently high convergent validity.

Download Full-text

Reflection on modern methods: planned missing data designs for epidemiological research

International Journal of Epidemiology ◽

10.1093/ije/dyaa042 ◽

2020 ◽

Vol 49 (5) ◽

pp. 1702-1711 ◽

Cited By ~ 1

Author(s):

Charlie Rioux ◽

Antoine Lewin ◽

Omolola A Odejimi ◽

Todd D Little

Keyword(s):

Missing Data ◽

Gold Standard ◽

Epidemiological Research ◽

Missing Data Treatments ◽

Planned Missing Data ◽

Missing Completely At Random ◽

Modern Methods ◽

Missing Items ◽

Data Design ◽

Planned Missing

Abstract Taking advantage of the ability of modern missing data treatments in epidemiological research (e.g. multiple imputation) to recover power while avoiding bias in the presence of data that is missing completely at random, planned missing data designs allow researchers to deliberately incorporate missing data into a research design. A planned missing data design may be done by randomly assigning participants to have missing items in a questionnaire (multiform design) or missing occasions of measurement in a longitudinal study (wave-missing design), or by administering an expensive gold-standard measure to a random subset of participants while the whole sample is administered a cheaper measure (two-method design). Although not common in epidemiology, these designs have been recommended for decades by methodologists for their benefits—notably that data collection costs are minimized and participant burden is reduced, which can increase validity. This paper describes the multiform, wave-missing and two-method designs, including their benefits, their impact on bias and power, and other factors that must be taken into consideration when implementing them in an epidemiological study design.

Download Full-text

Making the Most of Your Research Budget: Efficiency of a Three-Method Measurement Design With Planned Missing Data

Assessment ◽

10.1177/1073191118798050 ◽

2018 ◽

Vol 27 (5) ◽

pp. 903-920 ◽

Cited By ~ 1

Author(s):

Mario Lawes ◽

Martin Schultze ◽

Michael Eid

Keyword(s):

Cost Effectiveness ◽

Missing Data ◽

Research Design ◽

Gold Standard ◽

Statistical Power ◽

Measurement Model ◽

Additional Measure ◽

Planned Missing Data ◽

The Cost ◽

Planned Missing

Planned missing data (PMD) designs are an elegant way to incorporate expensive gold standard methods (e.g., biomarker) and cheaper but systematically biased methods (e.g., questionnaires) in research designs while ensuring high statistical power and low research costs. This article outlines a PMD design with one expensive gold standard and two cheap but biased methods (three-method measurement [3-MM] design). The cost effectiveness of different 3-MM-PMD designs is investigated and compared with the cost effectiveness of corresponding same-price two-method measurement designs using a simulation study. The results underline that PMD designs yield higher statistical power compared with complete data designs in a wide variety of conditions. Adding a second cheap method to the measurement model (i.e., using a 3-MM-PMD design) can increase the statistical power of the research design even further while keeping costs constant, when the additional measure is inexpensive, shares only small amounts of bias variance with the initial cheap measure, and when the gold standard measure is highly expensive compared with the cheap measures. Recommendations as well as a computer program for finding the optimal research design are provided.

Download Full-text

Planned missing data designs in psychological research.

Psychological Methods ◽

10.1037/1082-989x.11.4.323 ◽

2006 ◽

Vol 11 (4) ◽

pp. 323-343 ◽

Cited By ~ 238

Author(s):

John W. Graham ◽

Bonnie J. Taylor ◽

Allison E. Olchowski ◽

Patricio E. Cumsille

Keyword(s):

Missing Data ◽

Psychological Research ◽

Planned Missing Data ◽

Planned Missing

Download Full-text

Planned Missing Data Designs for Developmental Researchers

Child Development Perspectives ◽

10.1111/cdep.12043 ◽

2013 ◽

Vol 7 (4) ◽

pp. 199-204 ◽

Cited By ~ 93

Author(s):

Todd D. Little ◽

Mijke Rhemtulla

Keyword(s):

Missing Data ◽

Planned Missing Data ◽

Planned Missing

Download Full-text

Early Literacy Intervention for Preschoolers Who Need Tier 3 Support

Topics in Early Childhood Special Education ◽

10.1177/0271121416642454 ◽

2016 ◽

Vol 36 (4) ◽

pp. 205-217 ◽

Cited By ~ 3

Author(s):

Ruth A. Kaminski ◽

Kelly A. Powell-Smith

Keyword(s):

Early Literacy ◽

Phonemic Awareness ◽

Single Case ◽

Literacy Skills ◽

Literacy Intervention ◽

Early Literacy Skills ◽

Literacy Interventions ◽

Early Literacy Intervention ◽

Short And Long Term ◽

Tier 3

Phonemic awareness has been consistently identified as an essential skill for as well as an important predictor of later reading achievement. Children who lack these early literacy skills at kindergarten entry are more likely to demonstrate both short- and long-term reading difficulties. Despite the importance of providing intervention early, there is a paucity of research on Tier 3 early literacy interventions in preschool. A single-case multiple baseline across subjects design was used to examine the effects of a Tier 3 phonemic awareness intervention with preschool children who were identified as needing Tier 3 support in early literacy skills. The intervention was conducted individually with children, 5 to 10 min a day over an 8-week period. The results show gains in phonemic awareness for all children; however, the intervention was clearly more effective for some students than others. Factors that may have affected children’s learning are discussed.

Download Full-text