Current Practices in Missing Data Handling for Interrupted Time Series Studies Performed on Individual-Level Data: A Scoping Review in Health Research

Background: Disasters and other community-wide events can introduce significant interruptions and trauma to impacted communities. Children and young people can be disproportionately affected with additional educational disruptions. With the increasing threat of climate change, establishing a timely and adaptable framework to evaluate the impact of disasters on academic achievement is needed. However, analytical challenges are posed by the availability issue of individual-level data. Methods: A new method, Bayesian hierarchical meta-regression, was developed to evaluate the impact of the 2014 Hazelwood mine fire (a six-week fire event in Australia) using only aggregated school-level data from the standardised National Assessment Program-Literacy and Numeracy (NAPLAN) test. NAPLAN results and school characteristics (2008-2018) from 69 primary/secondary schools with different levels of mine fire-related smoke exposure were used to estimate the impact of the event. Using an interrupted time-series design, the model estimated immediate effects and post-interruption trend differences with full Bayesian statistical inference. Results: Major academic interruptions across NAPLAN domains were evident in high exposure schools in the year post-mine fire (highest in Writing: 11.09 [95%CI: 3.16-18.93], lowest in Reading: 8.34 [95%CI: 1.07-15.51]). The interruption was comparable to a three to four-month delay in educational attainment and had not fully recovered after several years. Conclusions: Considerable academic delays were found as a result of a mine fire, highlighting the need to provide educational and community-based supports in response to future events. Importantly, this work provides a statistical method using readily available aggregated data to assess the educational impacts in response to other disasters

Download Full-text

Missing Data in Research on Youth and Family Programs

Psychological Reports ◽

10.1177/00332941211026851 ◽

2021 ◽

pp. 003329412110268

Author(s):

Jaime Ballard ◽

Adeya Richmond ◽

Suzanne van den Hoogenhof ◽

Lynne Borden ◽

Daniel Francis Perkins

Keyword(s):

Missing Data ◽

Statistical Methods ◽

Self Report ◽

Program Participation ◽

Multilevel Data ◽

Individual Level ◽

Level Data ◽

Before And After ◽

Post Test ◽

The Individual

Background Multilevel data can be missing at the individual level or at a nested level, such as family, classroom, or program site. Increased knowledge of higher-level missing data is necessary to develop evaluation design and statistical methods to address it. Methods Participants included 9,514 individuals participating in 47 youth and family programs nationwide who completed multiple self-report measures before and after program participation. Data were marked as missing or not missing at the item, scale, and wave levels for both individuals and program sites. Results Site-level missing data represented a substantial portion of missing data, ranging from 0–46% of missing data at pre-test and 35–71% of missing data at post-test. Youth were the most likely to be missing data, although site-level data did not differ by the age of participants served. In this dataset youth had the most surveys to complete, so their missing data could be due to survey fatigue. Conclusions Much of the missing data for individuals can be explained by the site not administering those questions or scales. These results suggest a need for statistical methods that account for site-level missing data, and for research design methods to reduce the prevalence of site-level missing data or reduce its impact. Researchers can generate buy-in with sites during the community collaboration stage, assessing problematic items for revision or removal and need for ongoing site support, particularly at post-test. We recommend that researchers conducting multilevel data report the amount and mechanism of missing data at each level.

Download Full-text

Is Suicidality Increased in Covid-19 Pandemic? A Scoping Review and Quality Assessment

10.47363/jmhc/2021(4)170 ◽

2021 ◽

pp. 1-8

Author(s):

Kimberly Virginin Cruz Correia da Silva ◽

Keyword(s):

Scoping Review ◽

Web Of Science ◽

Significant Risk ◽

Stressful Situation ◽

Confounding Variable ◽

Individual Level ◽

Level Data ◽

Study Population ◽

Coronavirus Infection ◽

Risk Of Exposure

Background: There are emerging concerns that the COVID-19 pandemic may specifically increase suicide. Methods: Scoping Review in the MEDLINE/PubMed, SCOPUS, Web of Science, PsycINFO, Science Direct databases and in the medRxiv, bioRxiv and PsyArXiv preprint servers, using the descriptors “Covid-19”, “coronavirus infection”, “coronavirus”, “2019-nCoV”, “2019 new coronavirus disease”, “SARS-CoV-2”, “Suicide”, “General Public” and “Mental Health”. Results: A total of 62 studies were included in this review, where 10 studies were reported to have been conducted between March and May 2021; 39 in 2020; 4 in 2019; 3 in 2018; 1 in 2015; 2 in 2014; 2 in 2010 and 1 in 2004, all were conducted via online platforms. Limitations: We have interpreted our study findings in the context of the overall significant risk of exposure to suicide in our study population, while recognizing that individual level data of exposure to COVID-19 is a significant confounding variable. Conclusions: Being one of the first reviews in this context, the findings are anticipated to be helpful to predict the possible solutions for reducing the number of suicides in and facilitate further studies on strategies of how to alleviate such a stressful situation in COVID-19.

Download Full-text

Design characteristics and statistical methods used in interrupted time series studies evaluating public health interventions: a review

Journal of Clinical Epidemiology ◽

10.1016/j.jclinepi.2020.02.006 ◽

2020 ◽

Vol 122 ◽

pp. 1-11 ◽

Cited By ~ 5

Author(s):

Simon L. Turner ◽

Amalia Karahalios ◽

Andrew B. Forbes ◽

Monica Taljaard ◽

Jeremy M. Grimshaw ◽

...

Keyword(s):

Public Health ◽

Time Series ◽

Statistical Methods ◽

Interrupted Time Series ◽

Health Interventions ◽

Public Health Interventions ◽

Time Series Studies ◽

Design Characteristics

Download Full-text

A reanalysis of cluster randomized trials showed interrupted time-series studies were valuable in health system evaluation

Journal of Clinical Epidemiology ◽

10.1016/j.jclinepi.2014.10.003 ◽

2015 ◽

Vol 68 (3) ◽

pp. 324-333 ◽

Cited By ~ 42

Author(s):

Atle Fretheim ◽

Fang Zhang ◽

Dennis Ross-Degnan ◽

Andrew D. Oxman ◽

Helen Cheyne ◽

...

Keyword(s):

Time Series ◽

Health System ◽

Randomized Trials ◽

Interrupted Time Series ◽

System Evaluation ◽

Cluster Randomized Trials ◽

Time Series Studies ◽

Cluster Randomized

Download Full-text

The use of segmented regression in analysing interrupted time series studies: an example in pre-hospital ambulance care

Implementation Science ◽

10.1186/1748-5908-9-77 ◽

2014 ◽

Vol 9 (1) ◽

Cited By ~ 75

Author(s):

Monica Taljaard ◽

Joanne E McKenzie ◽

Craig R Ramsay ◽

Jeremy M Grimshaw

Keyword(s):

Time Series ◽

Interrupted Time Series ◽

Segmented Regression ◽

Time Series Studies

Download Full-text

Evaluation of statistical methods used in the analysis of interrupted time series studies: a simulation study

10.1101/2020.10.12.20211706 ◽

2020 ◽

Author(s):

Simon L Turner ◽

Andrew B Forbes ◽

Amalia Karahalios ◽

Monica Taljaard ◽

Joanne E McKenzie

Keyword(s):

Time Series ◽

Statistical Methods ◽

Simulated Data ◽

Population Level ◽

Interrupted Time Series ◽

Ordinary Least Squares ◽

In Series ◽

Time Series Studies ◽

Unbiased Estimates ◽

Biased Estimates

AbstractInterrupted time series (ITS) studies are frequently used to evaluate the effects of population-level interventions or exposures. To our knowledge, no studies have compared the performance of different statistical methods for this design. We simulated data to compare the performance of a set of statistical methods under a range of scenarios which included different level and slope changes, varying lengths of series and magnitudes of autocorrelation. We also examined the performance of the Durbin-Watson (DW) test for detecting autocorrelation. All methods yielded unbiased estimates of the level and slope changes over all scenarios. The magnitude of autocorrelation was underestimated by all methods, however, restricted maximum likelihood (REML) yielded the least biased estimates. Underestimation of autocorrelation led to standard errors that were too small and coverage less than the nominal 95%. All methods performed better with longer time series, except for ordinary least squares (OLS) in the presence of autocorrelation and Newey-West for high values of autocorrelation. The DW test for the presence of autocorrelation performed poorly except for long series and large autocorrelation. From the methods evaluated, OLS was the preferred method in series with fewer than 12 points, while in longer series, REML was preferred. The DW test should not be relied upon to detect autocorrelation, except when the series is long. Care is needed when interpreting results from all methods, given confidence intervals will generally be too narrow. Further research is required to develop better performing methods for ITS, especially for short series.

Download Full-text