scholarly journals The reporting of progression criteria in protocols of pilot trials designed to assess the feasibility of main trials is insufficient: a meta-epidemiological study

2019 ◽  
Vol 5 (1) ◽  
Author(s):  
Lawrence Mbuagbaw ◽  
Sarah Daisy Kosa ◽  
Daeria O. Lawson ◽  
Rosa Stalteri ◽  
Oluwatobi R. Olaiya ◽  
...  
Keyword(s):  
Sample Size ◽  
Primary Outcome ◽  
Linear Models ◽  
Feasibility Studies ◽  
Main Trial ◽  
Generalised Linear Models ◽  
Definitive Trial ◽  
Research Protocols ◽  
Pilot Trials ◽  
Pubmed Search

Abstract Introduction Pilot and feasibility trials are conducted to determine feasibility or to collect information that would inform the design of a larger definitive trial. Clear progression criteria are required to determine if a definitive or main trial is feasible and how it should be designed. We sought to determine how often progression criteria are reported and the associated factors. Methods We conducted a methodological review of protocols for pilot randomised trials published in three journals that publish research protocols (BMJ Open, Trials, Pilot and Feasibility Studies), using a PubMed search (2013–2017). We extracted bibliometric information including the country in which the study was conducted, source of funding, type of intervention, use of a primary feasibility outcome, sample size reporting, and justification. We used generalised linear models to determine the factors associated with reporting progression criteria. Results Our search retrieved 276 articles, of which 49 were not eligible. We included 227 articles. Overall, 45/227 (19.8%; 95% confidence interval [CI] 14.8–25.6) reported progression criteria. Protocols published in more recent years were significantly associated with higher odds of reporting progression criteria (adjusted odds ratio [aOR] 1.40; 95% CI 1.03–1.92; p = 0.034). Pilot trials from Europe (aOR 0.19; 95% CI 0.08–0.48; p < 0.001) and the rest of the world (aOR 0.05; 95% CI 0.01–0.18; p < 0.003) compared to North America were significantly associated with lower odds of reporting progression criteria. Journal, source of funding, sample size, intervention type, and having a primary outcome related to feasibility were not significantly associated with reporting progression criteria. Conclusion Progression criteria are not often explicitly stated in protocols of pilot trials leaving room for varied interpretation of findings. The development of formal guidance for progression criteria in protocols of pilot trials is warranted.

scholarly journals Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back!

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
M. Lewis ◽  
K. Bromley ◽  
C. J. Sutton ◽  
G. McCray ◽  
H. L. Myers ◽  
...  
Keyword(s):  
Hypothesis Testing ◽  
Sample Size ◽  
Clinical Outcomes ◽  
Treatment Fidelity ◽  
Outcome Evaluation ◽  
Main Trial ◽  
Process Outcomes ◽  
Pilot Trials ◽  
Green Zone ◽  
Acceptable Outcome

Abstract Background The current CONSORT guidelines for reporting pilot trials do not recommend hypothesis testing of clinical outcomes on the basis that a pilot trial is under-powered to detect such differences and this is the aim of the main trial. It states that primary evaluation should focus on descriptive analysis of feasibility/process outcomes (e.g. recruitment, adherence, treatment fidelity). Whilst the argument for not testing clinical outcomes is justifiable, the same does not necessarily apply to feasibility/process outcomes, where differences may be large and detectable with small samples. Moreover, there remains much ambiguity around sample size for pilot trials. Methods Many pilot trials adopt a ‘traffic light’ system for evaluating progression to the main trial determined by a set of criteria set up a priori. We construct a hypothesis testing approach for binary feasibility outcomes focused around this system that tests against being in the RED zone (unacceptable outcome) based on an expectation of being in the GREEN zone (acceptable outcome) and choose the sample size to give high power to reject being in the RED zone if the GREEN zone holds true. Pilot point estimates falling in the RED zone will be statistically non-significant and in the GREEN zone will be significant; the AMBER zone designates potentially acceptable outcome and statistical tests may be significant or non-significant. Results For example, in relation to treatment fidelity, if we assume the upper boundary of the RED zone is 50% and the lower boundary of the GREEN zone is 75% (designating unacceptable and acceptable treatment fidelity, respectively), the sample size required for analysis given 90% power and one-sided 5% alpha would be around n = 34 (intervention group alone). Observed treatment fidelity in the range of 0–17 participants (0–50%) will fall into the RED zone and be statistically non-significant, 18–25 (51–74%) fall into AMBER and may or may not be significant and 26–34 (75–100%) fall into GREEN and will be significant indicating acceptable fidelity. Discussion In general, several key process outcomes are assessed for progression to a main trial; a composite approach would require appraising the rules of progression across all these outcomes. This methodology provides a formal framework for hypothesis testing and sample size indication around process outcome evaluation for pilot RCTs.

scholarly journals Determining sample size for progression criteria for pragmatic pilot RCTs: The Hypothesis test Strikes Back!

2020 ◽  
Author(s):  
Martyn Lewis ◽  
Kieran Bromley ◽  
Christopher J Sutton ◽  
Gareth McCray ◽  
Helen Lucy Myers ◽  
...  
Keyword(s):  
Hypothesis Testing ◽  
Sample Size ◽  
Clinical Outcomes ◽  
Treatment Fidelity ◽  
Outcome Evaluation ◽  
Main Trial ◽  
Process Outcomes ◽  
Pilot Trials ◽  
Green Zone ◽  
Acceptable Outcome

Abstract BackgroundThe current CONSORT guidelines for reporting pilot trials do not recommend hypothesis testing of clinical outcomes on the basis that a pilot trial is under-powered to detect such differences and this is the aim of the main trial. It states that primary evaluation should focus on descriptive analysis of feasibility/process outcomes (e.g. recruitment, adherence, treatment fidelity). Whilst the argument for not testing clinical outcomes is justifiable, the same does not necessarily apply to feasibility/process outcomes, where differences may be large and detectable with small samples. Moreover, there remains much ambiguity around sample size for pilot trials. MethodsMany pilot trials adopt a ‘traffic light’ system for evaluating progression to the main trial determined by a set of criteria set up a priori. We construct a hypothesis-testing approach for binary feasibility outcomes focused around this system that tests against being in the RED zone (unacceptable outcome) based on an expectation of being in the GREEN zone (acceptable outcome) and choose the sample size to give high power to reject being in the RED zone if the GREEN zone holds true. Pilot point estimates falling in the RED zone will be statistically non-significant and in the GREEN zone will be significant; the AMBER zone designates potentially acceptable outcome and statistical tests may be significant or non-significant.ResultsFor example, in relation to treatment fidelity, if we assume the upper boundary of the RED zone is 50% and the lower boundary of the GREEN zone is 75% (designating unacceptable and acceptable treatment fidelity, respectively), the sample size required for analysis given 90% power and one-sided 5% alpha would be around n=35 (intervention group alone). Observed treatment fidelity in the range of 0-17 participants (0-50%) will fall into the RED zone and be statistically non-significant; 18-26 (51-74%) fall into AMBER and may or may not be significant; 27-35 (75-100%) fall into GREEN and will be significant indicating acceptable fidelity.DiscussionIn general, several key process outcomes are assessed for progression to a main trial; a composite approach would require appraising the rules of progression across all these outcomes. This methodology provides a formal framework for hypothesis-testing and sample size indication around process outcome evaluation for pilot RCTs.

scholarly journals Determining sample size for progression criteria for pragmatic pilot RCTs: The Hypothesis test Strikes Back!

2021 ◽  
Author(s):  
Martyn Lewis ◽  
Kieran Bromley ◽  
Christopher J Sutton ◽  
Gareth McCray ◽  
Helen Lucy Myers ◽  
...  
Keyword(s):  
Hypothesis Testing ◽  
Sample Size ◽  
Clinical Outcomes ◽  
Treatment Fidelity ◽  
Outcome Evaluation ◽  
Main Trial ◽  
Process Outcomes ◽  
Pilot Trials ◽  
Green Zone ◽  
Acceptable Outcome

Abstract Background The current CONSORT guidelines for reporting pilot trials do not recommend hypothesis testing of clinical outcomes on the basis that a pilot trial is under-powered to detect such differences and this is the aim of the main trial. It states that primary evaluation should focus on descriptive analysis of feasibility/process outcomes (e.g. recruitment, adherence, treatment fidelity). Whilst the argument for not testing clinical outcomes is justifiable, the same does not necessarily apply to feasibility/process outcomes, where differences may be large and detectable with small samples. Moreover, there remains much ambiguity around sample size for pilot trials. Methods Many pilot trials adopt a ‘traffic light’ system for evaluating progression to the main trial determined by a set of criteria set up a priori. We construct a hypothesis-testing approach for binary feasibility outcomes focused around this system that tests against being in the RED zone (unacceptable outcome) based on an expectation of being in the GREEN zone (acceptable outcome) and choose the sample size to give high power to reject being in the RED zone if the GREEN zone holds true. Pilot point estimates falling in the RED zone will be statistically non-significant and in the GREEN zone will be significant; the AMBER zone designates potentially acceptable outcome and statistical tests may be significant or non-significant.Results For example, in relation to treatment fidelity, if we assume the upper boundary of the RED zone is 50% and the lower boundary of the GREEN zone is 75% (designating unacceptable and acceptable treatment fidelity, respectively), the sample size required for analysis given 90% power and one-sided 5% alpha would be around n=34 (intervention group alone). Observed treatment fidelity in the range of 0-17 participants (0-50%) will fall into the RED zone and be statistically non-significant; 18-25 (51-74%) fall into AMBER and may or may not be significant; 26-34 (75-100%) fall into GREEN and will be significant indicating acceptable fidelity.Discussion In general, several key process outcomes are assessed for progression to a main trial; a composite approach would require appraising the rules of progression across all these outcomes. This methodology provides a formal framework for hypothesis-testing and sample size indication around process outcome evaluation for pilot RCTs.

scholarly journals Comparison of Approaches to Testing Equality of Expectations Among Samples from Poisson and Negative Binomial Distribution

2018 ◽  
Vol 66 (4) ◽  
pp. 1025-1034
Author(s):  
Martin Tejkal ◽  
Zuzana Hübnerová
Keyword(s):  
Maximum Likelihood ◽  
Sample Size ◽  
Negative Binomial Distribution ◽  
Binomial Distribution ◽  
Linear Models ◽  
Negative Binomial ◽  
Test Statistics ◽  
Power Functions ◽  
Generalised Linear Models ◽  
The Relationship

The paper deals with testing of the hypothesis of equality of expectations among p samples from Poisson or negative binomial distribution. a comparison of two main approaches is carried out. The first approach is based on transforming the samples from either Poisson or negative binomial distribution in order to achieve normality or variance stability, and then testing the hypothesis of equality of expectations via the F‑test. In the second approach, test statistics coming from the theory of maximum likelihood appearing in generalised linear models framework, specially designed for testing the hypothesis among samples from the respective distributions (Poisson or negative binomial), are used. The comparison is done graphically, by plotting the simulated power functions of the test of the hypothesis of equality of expectations, when first or second approach was used. Additionally, the relationship between the power functions obtained via the respective approaches and sample sizes is studied by evaluating the respective power functions as functions of a sample size numerically.

Pilot and feasibility trials in traditional Chinese medicine: a literature survey

2020 ◽  
Author(s):  
Guowei Li ◽  
Darong Wu ◽  
Xuejiao Chen ◽  
Jie Zeng ◽  
Ziyi Li ◽  
...  
Keyword(s):  
Chinese Medicine ◽  
Traditional Chinese Medicine ◽  
Guideline Adherence ◽  
Reporting Quality ◽  
Feasibility Studies ◽  
Substantial Improvement ◽  
Definitive Trial ◽  
Pilot Trials ◽  
The Impact

Abstract Background The guidelines for pilot and feasibility studies were published in 2016. Little is known about the guideline adherence of TCM (Traditional Chinese Medicine) pilot trials or whether the guidelines can significantly enhance the quality of implementation and reporting of TCM pilot trials. We aimed to investigate the guideline adherence, assess the impact of guidelines on TCM pilot trials, and discuss potentially undetected challenges for TCM pilot trials, by conducting a liteature survey.Methods We systematically searched MEDLINE, EMBASE and CNKI to retreive TCM pilot trials. We randomly chose 50 pilot trials from the eligible studies for analyses. The CONSORT extension to pilot and feasibility studies was used as a framework to assess the methodology and reporting quality of the studies.Results The included stuides had a guideline adherence level ranging from 4% to 96%, where the lowest adherence was found in the item 6c (prespecified criteria used to judge progression to future definitive trial). The guidance published in 2016 seemed to exert minimal effect on guideline adherence in TCM pilot trials. The unidentified issues related to TCM pilot trials from the guidelines included blinding, lack of standard formula of interventions, difficulty in comparison for effect assessment of interventions, and difficulty in bias control.Conclusions The current practice in TCM pilot trials requried substantial improvement in the literature. Further endeavors are needed for training and dissemination of guideline adherence, and development of more detailed methodology in the field of TCM pilot trials.

scholarly journals Determining sample size for progression criteria for pragmatic pilot RCTs: The Hypothesis test Strikes Back!

2020 ◽  
Author(s):  
Martyn Lewis ◽  
Kieran Bromley ◽  
Christopher J Sutton ◽  
Gareth McCray ◽  
Helen Lucy Myers ◽  
...  
Keyword(s):  
Hypothesis Testing ◽  
Sample Size ◽  
Clinical Outcomes ◽  
Treatment Fidelity ◽  
Outcome Evaluation ◽  
Main Trial ◽  
Process Outcomes ◽  
Pilot Trials ◽  
Green Zone ◽  
Acceptable Outcome

Abstract Background The current CONSORT guidelines for reporting pilot trials do not recommend hypothesis testing of clinical outcomes on the basis that a pilot trial is under-powered to detect such differences and this is the aim of the main trial. It states that primary evaluation should focus on descriptive analysis of feasibility/process outcomes (e.g. recruitment, adherence, treatment fidelity). Whilst the argument for not testing clinical outcomes is justifiable, the same does not necessarily apply to feasibility/process outcomes, where differences may be large and detectable with small samples. Moreover, there remains much ambiguity around sample size for pilot trials. Methods Many pilot trials adopt a ‘traffic light’ system for evaluating progression to the main trial determined by a set of criteria set up a priori. We construct a hypothesis-testing approach focused around this system that tests against being in the RED zone (unacceptable outcome) based on an expectation of being in the GREEN zone (acceptable outcome) and choose the sample size to give high power to reject being in the RED zone if the GREEN zone holds true. Pilot point estimates falling in the RED zone will be statistically non-significant and in the GREEN zone will be significant; the AMBER zone designates potentially acceptable outcome and statistical tests may be significant or non-significant.Results For example, in relation to treatment fidelity, if we assume the upper boundary of the RED zone is 50% and the lower boundary of the GREEN zone is 75% (designating unacceptable and acceptable treatment fidelity, respectively), the sample size required for analysis given 90% power and one-sided 5% alpha would be around n=30 (intervention group alone). Observed treatment fidelity in the range of 0-15 participants (0-50%) will fall into the RED zone and be statistically non-significant; 16-22 (51-74%) fall into AMBER and may or may not be significant; 23-30 (75-100%) fall into GREEN and will be significant indicating acceptable fidelity.Discussion In general, several key process outcomes are assessed for progression to a main trial; a composite approach would require appraising the rules of progression across all these outcomes. This methodology provides a formal framework for hypothesis-testing and sample size indication around process outcome evaluation for pilot RCTs.

scholarly journals Using ActionADE to create information continuity to reduce re-exposures to harmful medications: study protocol for a randomized controlled trial

Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Jeffrey P. Hau ◽  
Penelope M. A. Brasher ◽  
Amber Cragg ◽  
Serena Small ◽  
Maeve Wickham ◽  
...  
Keyword(s):  
Health Information ◽  
Adverse Drug Event ◽  
Primary Outcome ◽  
Adverse Drug Events ◽  
Primary Objective ◽  
Drug Event ◽  
Controlled Trials ◽  
Drug Reactions ◽  
Main Trial ◽  
Randomized Controlled

Abstract Background Repeat exposures to culprit medications are a common cause of preventable adverse drug events. Health information technologies have the potential to reduce repeat adverse drug events by improving information continuity. However, they rarely interoperate to ensure providers can view adverse drug events documented in other systems. We designed ActionADE to enable rapid documentation of adverse drug events and communication of standardized information across health sectors by integrating with legacy systems. We will leverage ActionADE’s implementation to conduct two parallel, randomized trials: patients with adverse drug reactions in the main trial and those diagnosed with non-adherence in a secondary trial. Primary objective of the main trial is to evaluate the effects of providing information continuity about adverse drug reactions on culprit medication re-dispensations over 12 months. Primary objective of the secondary trial is to evaluate the effect of providing information continuity on adherence over 12 months. Methods We will conduct two parallel group, triple-blind randomized controlled trials in participating hospitals in British Columbia, Canada. We will enroll adults presenting to hospital with an adverse drug event to prescribed outpatient medication. Clinicians will document the adverse drug event in ActionADE. The software will use an algorithm to determine patient eligibility and allocate eligible patients to experimental or control. In the experimental arm, ActionADE will transmit information to PharmaNet, where adverse drug event information will be displayed in community pharmacies when re-dispensations are attempted. In the control arm, ActionADE will retain information in the local record. We will enroll 3600 adults with an adverse drug reaction into the main trial. The main trial’s primary outcome is re-dispensation of a culprit or same-class medication within 12 months; the secondary trial’s primary outcome will be adherence to culprit medication. Secondary outcomes include health services utilization and mortality. Discussion These studies have the potential to guide policy decisions and investments needed to drive health information technology integrations to prevent repeat adverse drug events. We present an example of how a health information technology implementation can be leveraged to conduct pragmatic randomized controlled trials. Trial registration ClinicalTrials.gov NCT04568668, NCT04574648. Registered on 1 October 2020.

scholarly journals Progression from external pilot to definitive randomised controlled trial: a methodological review of progression criteria reporting

BMJ Open ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. e048178
Author(s):  
Katie Mellor ◽  
Saskia Eddy ◽  
Nicholas Peckham ◽  
Christine M Bond ◽  
Michael J Campbell ◽  
...  
Keyword(s):  
Randomised Controlled Trial ◽  
Controlled Trial ◽  
A Priori ◽  
Pilot Trial ◽  
Public Library ◽  
Feasibility Studies ◽  
Methodological Review ◽  
Pilot Trials ◽  
Trial Protocols ◽  
Randomised Controlled

ObjectivesPrespecified progression criteria can inform the decision to progress from an external randomised pilot trial to a definitive randomised controlled trial. We assessed the characteristics of progression criteria reported in external randomised pilot trial protocols and results publications, including whether progression criteria were specified a priori and mentioned in prepublication peer reviewer reports.Study designMethodological review.MethodsWe searched four journals through PubMed: British Medical Journal Open, Pilot and Feasibility Studies, Trials and Public Library of Science One. Eligible publications reported external randomised pilot trial protocols or results, were published between January 2018 and December 2019 and reported progression criteria. We double data extracted 25% of the included publications. Here we report the progression criteria characteristics.ResultsWe included 160 publications (123 protocols and 37 completed trials). Recruitment and retention were the most frequent indicators contributing to progression criteria. Progression criteria were mostly reported as distinct thresholds (eg, achieving a specific target; 133/160, 83%). Less than a third of the planned and completed pilot trials that included qualitative research reported how these findings would contribute towards progression criteria (34/108, 31%). The publications seldom stated who established the progression criteria (12/160, 7.5%) or provided rationale or justification for progression criteria (44/160, 28%). Most completed pilot trials reported the intention to proceed to a definitive trial (30/37, 81%), but less than half strictly met all of their progression criteria (17/37, 46%). Prepublication peer reviewer reports were available for 153/160 publications (96%). Peer reviewer reports for 86/153 (56%) publications mentioned progression criteria, with peer reviewers of 35 publications commenting that progression criteria appeared not to be specified.ConclusionsMany external randomised pilot trial publications did not adequately report or propose prespecified progression criteria to inform whether to proceed to a future definitive randomised controlled trial.

Nonnull asymptotic distributions of three classic criteria in generalised linear models

Biometrika ◽  
1994 ◽  
Vol 81 (4) ◽  
pp. 709-720 ◽  
Author(s):  
GAUSS M. CORDEIRO ◽  
DENISE A. BOTTER ◽  
SILVIA L. DE PAULA FERRARI
Keyword(s):  
Linear Models ◽  
Asymptotic Distributions ◽  
Generalised Linear Models

Basilar Artery Occlusion Chinese Endovascular Trial: Protocol for a prospective randomized controlled study

2021 ◽  
pp. 174749302110409
Author(s):  
Chuanhui Li ◽  
Chuanjie Wu ◽  
Longfei Wu ◽  
Wenbo Zhao ◽  
Jian Chen ◽  
...  
Keyword(s):  
Sample Size ◽  
Basilar Artery ◽  
Symptom Onset ◽  
Primary Outcome ◽  
Mechanical Thrombectomy ◽  
Infarct Volume ◽  
Artery Occlusion ◽  
Basilar Artery Occlusion ◽  
Controlled Study ◽  
Randomized Controlled

Rationale There are no randomized trials examining the best treatment for acute basilar artery occlusion in the 6–24-hour time window. Aims To assess the safety and efficacy of thrombectomy for stroke due to basilar artery occlusion in patients randomized within 6–24 h from symptom onset or time last seen well. Sample size For an estimated difference of 20% in proportions of the primary outcome between the two groups, 318 patients will be included for 5% significance and 90% power with a planned interim analysis after two-thirds of the sample size (212 patients) have achieved the 90 days follow-up. Methods and design A prospective, multi-center, randomized, controlled, open-label and blinded-endpoint trial. The randomization employs a 1:1 ratio of mechanical thrombectomy with the detachable Solitaire thrombectomy device and best medical therapy (BMT) vs. BMT alone. Study outcomes The primary outcome will be the proportion of patients achieving modified Rankin Scale (mRS) 0–3 at 90 days. Key secondary outcomes are: dramatic early favorable response, dichotomized mRS score (0–2 vs. 3–6 and 0–4 vs. 5–6) at 90 days, ordinal (shift) mRS analysis at 90 days, infarct volume at 24 h, vessel recanalization at 24 h in both treatment arms, and successful recanalization in the thrombectomy arm according to the modified thrombolysis in cerebral infarction (mTICI) classification defined as mTICI 2 b or 3. Safety variables are mortality at 90 days, symptomatic intracranial hemorrhage rates at 24 h, and procedure-related complications. Discussion Results from this trial will indicate whether mechanical thrombectomy is superior to medical management alone in achieving favorable outcomes in subjects with acute stroke caused by basilar artery occlusion presenting within 6–24 h from symptom onset. Trial registration: URL: http://www.clinicaltrials.gov . ClinicalTrials.gov Identifier: NCT02737189.

Sign in / Sign up

Export Citation Format

Share Document