scholarly journals Comparison of Six Statistical Methods for Interrupted time Series Studies: Empirical Evaluation of 190 Published Series

2020 ◽  
Author(s):  
Simon Turner ◽  
Amalia Karahalios ◽  
Andrew Forbes ◽  
Monica Taljaard ◽  
Jeremy Grimshaw ◽  
...  
Keyword(s):  
Time Series ◽  
Statistical Method ◽  
Statistical Methods ◽  
Statistical Significance ◽  
Empirical Evaluation ◽  
Interrupted Time Series ◽  
Series Data ◽  
Standard Errors ◽  
P Values ◽  
The Impact

Abstract Background The Interrupted Time Series (ITS) is a quasi-experimental design commonly used in public health to evaluate the impact of interventions or exposures. Multiple statistical methods are available to analyse data from ITS studies, but no empirical investigation has examined how the different methods compare when applied to real-world datasets. MethodsA random sample of 200 ITS studies identified in a previous methods review were included. Time series data from each of these studies was sought. Each dataset was re-analysed using six statistical methods. Point and confidence interval estimates for level and slope changes, standard errors, p-values and estimates of autocorrelation were compared between methods. ResultsFrom the 200 ITS studies, including 230 time series, 190 datasets were obtained. We found that the choice of statistical method can importantly affect the level and slope change point estimates, their standard errors, width of confidence intervals and p-values. Statistical significance (categorised at the 5% level) often differed across the pairwise comparisons of methods, ranging from 4% to 25% disagreement. Estimates of autocorrelation differed depending on the method used and the length of the series. ConclusionsThe choice of statistical method in ITS studies can lead to substantially different conclusions about the impact of the interruption. Pre-specification of the statistical method is encouraged, and naive conclusions based on statistical significance should be avoided.

scholarly journals Comparison of six statistical methods for interrupted time series studies: empirical evaluation of 190 published series

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Simon L. Turner ◽  
Amalia Karahalios ◽  
Andrew B. Forbes ◽  
Monica Taljaard ◽  
Jeremy M. Grimshaw ◽  
...  
Keyword(s):  
Time Series ◽  
Statistical Method ◽  
Statistical Methods ◽  
Time Series Data ◽  
Statistical Significance ◽  
Empirical Evaluation ◽  
Interrupted Time Series ◽  
Series Data ◽  
Standard Errors ◽  
The Impact

Abstract Background The Interrupted Time Series (ITS) is a quasi-experimental design commonly used in public health to evaluate the impact of interventions or exposures. Multiple statistical methods are available to analyse data from ITS studies, but no empirical investigation has examined how the different methods compare when applied to real-world datasets. Methods A random sample of 200 ITS studies identified in a previous methods review were included. Time series data from each of these studies was sought. Each dataset was re-analysed using six statistical methods. Point and confidence interval estimates for level and slope changes, standard errors, p-values and estimates of autocorrelation were compared between methods. Results From the 200 ITS studies, including 230 time series, 190 datasets were obtained. We found that the choice of statistical method can importantly affect the level and slope change point estimates, their standard errors, width of confidence intervals and p-values. Statistical significance (categorised at the 5% level) often differed across the pairwise comparisons of methods, ranging from 4 to 25% disagreement. Estimates of autocorrelation differed depending on the method used and the length of the series. Conclusions The choice of statistical method in ITS studies can lead to substantially different conclusions about the impact of the interruption. Pre-specification of the statistical method is encouraged, and naive conclusions based on statistical significance should be avoided.

scholarly journals Design characteristics and statistical methods used in interrupted time series studies evaluating public health interventions: protocol for a review

BMJ Open ◽  
2019 ◽  
Vol 9 (1) ◽  
pp. e024096 ◽  
Author(s):  
Simon L Turner ◽  
Amalia Karahalios ◽  
Andrew B Forbes ◽  
Monica Taljaard ◽  
Jeremy M Grimshaw ◽  
...  
Keyword(s):  
Public Health ◽  
Time Series ◽  
Statistical Methods ◽  
Time Series Data ◽  
Interrupted Time Series ◽  
Health Interventions ◽  
Series Data ◽  
Public Health Interventions ◽  
Design Characteristics ◽  
The Impact

IntroductionAn interrupted time series (ITS) design is an important observational design used to examine the effects of an intervention or exposure. This design has particular utility in public health where it may be impracticable or infeasible to use a randomised trial to evaluate health system-wide policies, or examine the impact of exposures (such as earthquakes). There have been relatively few studies examining the design characteristics and statistical methods used to analyse ITS designs. Further, there is a lack of guidance to inform the design and analysis of ITS studies.This is the first study in a larger project that aims to provide tools and guidance for researchers in the design and analysis of ITS studies. The objectives of this study are to (1) examine and report the design characteristics and statistical methods used in a random sample of contemporary ITS studies examining public health interventions or exposures that impact on health-related outcomes, and (2) create a repository of time series data extracted from ITS studies. Results from this study will inform the remainder of the project which will investigate the performance of a range of commonly used statistical methods, and create a repository of input parameters required for sample size calculation.Methods and analysisWe will collate 200 ITS studies evaluating public health interventions or the impact of exposures. ITS studies will be identified from a search of the bibliometric database PubMed between the years 2013 and 2017, combined with stratified random sampling. From eligible studies, we will extract study characteristics, details of the statistical models and estimation methods, effect metrics and parameter estimates. Further, we will extract the time series data when available. We will use systematic review methods in the screening, application of inclusion and exclusion criteria, and extraction of data. Descriptive statistics will be used to summarise the data.Ethics and disseminationEthics approval is not required since information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. A repository of data extracted from the published ITS studies will be made publicly available.

The Impact of a National Antimicrobial Stewardship Program on Antibiotic Prescribing in Primary Care: An Interrupted Time Series Analysis

2018 ◽  
Vol 69 (2) ◽  
pp. 227-232 ◽  
Author(s):  
Violeta Balinskaite ◽  
Alan P Johnson ◽  
Alison Holmes ◽  
Paul Aylin
Keyword(s):  
Primary Care ◽  
Time Series ◽  
Broad Spectrum ◽  
Interrupted Time Series ◽  
Series Data ◽  
Antibiotic Prescribing ◽  
Segmented Regression ◽  
Broad Spectrum Antibiotic ◽  
Stewardship Program ◽  
The Impact

Abstract Background The Quality Premium was introduced in 2015 to financially reward local commissioners of healthcare in England for targeted reductions in antibiotic prescribing in primary care. Methods We used a national antibiotic prescribing dataset from April 2013 until February 2017 to examine the number of antibiotic items prescribed, the total number of antibiotic items prescribed per STAR-PU (specific therapeutic group age/sex-related prescribing units), the number of broad-spectrum antibiotic items prescribed, and broad-spectrum antibiotic items prescribed, expressed as a percentage of the total number of antibiotic items. To evaluate the impact of the Quality Premium on antibiotic prescribing, we used a segmented regression analysis of interrupted time series data. Results During the study period, over 140 million antibiotic items were prescribed in primary care. Following the introduction of the Quality Premium, antibiotic items prescribed decreased by 8.2%, representing 5933563 fewer antibiotic items prescribed during the 23 post-intervention months, as compared with the expected numbers based on the trend in the pre-intervention period. After adjusting for the age and sex distribution in the population, the segmented regression model also showed a significant relative decrease in antibiotic items prescribed per STAR-PU. A similar effect was found for broad-spectrum antibiotics (comprising 10.1% of total antibiotic prescribing), with an 18.9% reduction in prescribing. Conclusions This study shows that the introduction of financial incentives for local commissioners of healthcare to improve the quality of prescribing was associated with a significant reduction in both total and broad-spectrum antibiotic prescribing in primary care in England.

Abstract 19225: Impact of Change in Resuscitation Guidelines on National Out-of-hospital Cardiac Arrest Outcomes: Fulfilled Expectations?

Circulation ◽  
2015 ◽  
Vol 132 (suppl_3) ◽  
Author(s):  
Shaker M Eid ◽  
Aiham Albaeni ◽  
Rebeca Rios ◽  
May Baydoun ◽  
Bolanle Akinyele ◽  
...  
Keyword(s):  
Time Series ◽  
Cardiac Arrest ◽  
Time Series Data ◽  
Statistical Significance ◽  
Interrupted Time Series ◽  
The United States ◽  
Series Data ◽  
National Database ◽  
Resuscitation Guidelines ◽  
Hospital Cardiac Arrest

Background: The intent of the 5-yearly Resuscitation Guidelines is to improve outcomes. Previous studies have yielded conflicting reports of a beneficial impact of the 2005 guidelines on out-of-hospital cardiac arrest (OHCA) survival. Using a national database, we examined survival before and after the introduction of both the 2005 and 2010 guidelines. Methods: We used the 2000 through 2012 National Inpatient Sample database to select patients ≥18 years admitted to hospitals in the United States with non-traumatic OHCA (ICD-9 CM codes 427.5 & 427.41). A quasi-experimental (interrupted time series) design was used to compare monthly survival trends. Outcomes for OHCA were compared pre- and post- 2005 and 2010 resuscitation guidelines release as follows: 01/2000-09/2005 vs. 10/2005-9/2010 and 10/2005-9/2010 vs. 10/2010-12/2012. Segmented regression analyses of interrupted time series data were performed to examine changes in survival to hospital discharge. Results: For the pre- and post- guidelines periods, 81600, 69139 and 36556 patients respectively survived to hospital admission following OHCA. Subsequent to the release of the 2005 guidelines, there was a statistically significant worsening in survival trends (β= -0.089, 95% CI -0.163 – -0.016, p =0.018) until the release of the 2010 guidelines when a sharp increase in survival was noted which persisted for the period of study (β= 0.054, 95% CI -0.143 – 0.251, p =0.588) but did not achieve statistical significance (Figure). Conclusion: National clinical guidelines developed to impact outcomes must include mechanisms to assess whether benefit actually occurs. The worsening in OHCA survival following the 2005 guidelines is thought provoking but the improvement following the release of the 2010 guidelines is reassuring and worthy of perpetuation.

scholarly journals Does Monetary Support Increase the Number of Scientific Papers? An Interrupted Time Series Analysis

2018 ◽  
Vol 3 (1) ◽  
pp. 19-39 ◽  
Author(s):  
Yaşar Tonta
Keyword(s):  
Time Series ◽  
Time Series Analysis ◽  
Interrupted Time Series ◽  
Support Program ◽  
Series Data ◽  
Control Group ◽  
Interrupted Time Series Analysis ◽  
Series Analysis ◽  
The Impact

Abstract Purpose One of the main indicators of scientific production is the number of papers published in scholarly journals. Turkey ranks 18th place in the world based on the number of scholarly publications. The objective of this paper is to find out if the monetary support program initiated in 1993 by the Turkish Scientific and Technological Research Council (TÜBİTAK) to incentivize researchers and increase the number, impact, and quality of international publications has been effective in doing so. Design/methodology/approach We analyzed some 390,000 publications with Turkish affiliations listed in the Web of Science (WoS) database between 1976 and 2015 along with about 157,000 supported ones between 1997 and 2015. We used the interrupted time series (ITS) analysis technique (also known as “quasi-experimental time series analysis” or “intervention analysis”) to test if TÜBİTAK’s support program helped increase the number of publications. We defined ARIMA (1,1,0) model for ITS data and observed the impact of TÜBİTAK’s support program in 1994, 1997, and 2003 (after one, four and 10 years of its start, respectively). The majority of publications (93%) were full papers (articles), which were used as the experimental group while other types of contributions functioned as the control group. We also carried out a multiple regression analysis. Findings TÜBİTAK’s support program has had negligible effect on the increase of the number of papers with Turkish affiliations. Yet, the number of other types of contributions continued to increase even though they were not well supported, suggesting that TÜBİTAK’s support program is probably not the main factor causing the increase in the number of papers with Turkish affiliations. Research limitations Interrupted time series analysis shows if the “intervention” has had any significant effect on the dependent variable but it does not explain what caused the increase in the number of papers if it was not the intervention. Moreover, except the “intervention”, other “event(s)” that might affect the time series data (e.g., increase in the number of research personnel over the years) should not occur during the period of analysis, a prerequisite that is beyond the control of the researcher. Practical implications TÜBİTAK’s “cash-for-publication” program did not seem to have direct impact on the increase of the number of papers published by Turkish authors, suggesting that small amounts of payments are not much of an incentive for authors to publish more. It might perhaps be a better strategy to concentrate limited resources on a few high impact projects rather than to disperse them to thousands of authors as “micropayments.” Originality/value Based on 25 years’ worth of payments data, this is perhaps one of the first large-scale studies showing that “cash-for-publication” policies or “piece rates” paid to researchers tend to have little or no effect on the increase of researchers’ productivity. The main finding of this paper has some implications for countries wherein publication subsidies are used as an incentive to increase the number and quality of papers published in international journals. They should be prepared to consider reviewing their existing support programs (based usually on bibliometric measures such as journal impact factors) and revising their reward policies.

scholarly journals Massive social protests amid the pandemic in selected Colombian cities: Did they increase COVID-19 cases?

2021 ◽  
Author(s):  
Jose Moreno-Montoya ◽  
Laura A Rodriguez Villamizar ◽  
Alvaro Javier Idrovo
Keyword(s):  
Time Series ◽  
Time Series Data ◽  
Moving Average ◽  
Interrupted Time Series ◽  
Series Data ◽  
Arima Models ◽  
Interrupted Time Series Analysis ◽  
Social Protests ◽  
Show Evidence ◽  
The Impact

Background. Since April 28, 2021, in Colombia there are social protests with numerous demonstrations in various cities. This occurs whereas the country faces the third wave of the COVID-19 pandemic. The aim of this study was to assess the effect of social protests on the number and trend of the confirmed COVID-19 cases in some selected Colombian cities where social protests had more intensity. Methods. We performed and interrupted time-series analysis (ITSA) and Autoregressive Integrated Moving Average (ARIMA) models, based on the confirmed COVID-19 cases in Colombia, between March 1 and May 15, 2021, for the cities of Bogota, Cali, Barranquilla, Medellin, and Bucaramanga. The ITSA models estimated the impact of social demonstrations on the number and trend of cases for each city by using Newey-West standard errors and ARIMA models assessed the overall pattern of the series and effect of the intervention. We considered May 2, 2021, as the intervention date for the analysis, five days after social demonstrations started in the country. Findings. During the study period the number of cases by city was 1,014,815 for Bogota, 192,320 for Cali, 175,269 for Barranquilla, 311,904 for Medellin, and 62,512 for Bucaramanga. Heterogeneous results were found among cities. Only for the cities of Cali and Barranquilla statistically significant changes in trend of the number of cases were obtained after the intervention: positive in the first city, negative in the second one. None ARIMA models show evidence of abrupt changes in the trend of the series for any city and intervention effect was only positive for Bucaramanga. Interpretation. The findings confer solid evidence that social protests had an heterogenous effect on the number and trend of COVID-19 cases. Divergent effects might be related to the epidemiologic time of the pandemic and the characteristics of the social protests. Assessing the effect of social protests within a pandemic is complex and there are several methodological limitations. Further analyses are required with longer time-series data.

scholarly journals Evaluation of statistical methods used in the analysis of interrupted time series studies: a simulation study

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Simon L. Turner ◽  
Andrew B. Forbes ◽  
Amalia Karahalios ◽  
Monica Taljaard ◽  
Joanne E. McKenzie
Keyword(s):  
Time Series ◽  
Statistical Methods ◽  
Population Level ◽  
Interrupted Time Series ◽  
Ordinary Least Squares ◽  
Standard Errors ◽  
In Series ◽  
Time Series Studies ◽  
Unbiased Estimates ◽  
Biased Estimates

Abstract Background Interrupted time series (ITS) studies are frequently used to evaluate the effects of population-level interventions or exposures. However, examination of the performance of statistical methods for this design has received relatively little attention. Methods We simulated continuous data to compare the performance of a set of statistical methods under a range of scenarios which included different level and slope changes, varying lengths of series and magnitudes of lag-1 autocorrelation. We also examined the performance of the Durbin-Watson (DW) test for detecting autocorrelation. Results All methods yielded unbiased estimates of the level and slope changes over all scenarios. The magnitude of autocorrelation was underestimated by all methods, however, restricted maximum likelihood (REML) yielded the least biased estimates. Underestimation of autocorrelation led to standard errors that were too small and coverage less than the nominal 95%. All methods performed better with longer time series, except for ordinary least squares (OLS) in the presence of autocorrelation and Newey-West for high values of autocorrelation. The DW test for the presence of autocorrelation performed poorly except for long series and large autocorrelation. Conclusions From the methods evaluated, OLS was the preferred method in series with fewer than 12 points, while in longer series, REML was preferred. The DW test should not be relied upon to detect autocorrelation, except when the series is long. Care is needed when interpreting results from all methods, given confidence intervals will generally be too narrow. Further research is required to develop better performing methods for ITS, especially for short series.

EFFECTS OF TREND AND PERIODICITY ON THE CORRELATION DIMENSION AND THE LYAPUNOV EXPONENTS

2008 ◽  
Vol 18 (12) ◽  
pp. 3679-3687 ◽  
Author(s):  
AYDIN A. CECEN ◽  
CAHIT ERKAL
Keyword(s):  
Time Series ◽  
Lyapunov Exponent ◽  
Lyapunov Exponents ◽  
Correlation Dimension ◽  
Time Series Data ◽  
Logistic Map ◽  
Model Identification ◽  
Series Data ◽  
Largest Lyapunov Exponent ◽  
The Impact

We present a critical remark on the pitfalls of calculating the correlation dimension and the largest Lyapunov exponent from time series data when trend and periodicity exist. We consider a special case where a time series Zi can be expressed as the sum of two subsystems so that Zi = Xi + Yi and at least one of the subsystems is deterministic. We show that if the trend and periodicity are not properly removed, correlation dimension and Lyapunov exponent estimations yield misleading results, which can severely compromise the results of diagnostic tests and model identification. We also establish an analytic relationship between the largest Lyapunov exponents of the subsystems and that of the whole system. In addition, the impact of a periodic parameter perturbation on the Lyapunov exponent for the logistic map and the Lorenz system is discussed.

scholarly journals Correction to: Measuring the impact of the Afordable Care Act Medicaid expansion on access to primary care using an interrupted time series approach

2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Elizabeth A. Brown ◽  
Brandi M. White ◽  
Walter J. Jones ◽  
Mulugeta Gebregziabher ◽  
Kit N. Simpson
Keyword(s):  
Primary Care ◽  
Time Series ◽  
Interrupted Time Series ◽  
Medicaid Expansion ◽  
Time Series Approach ◽  
Access To Primary Care ◽  
The Impact

An amendment to this paper has been published and can be accessed via the original article.

Sign in / Sign up

Export Citation Format

Share Document