Abstract
Reports of "positive" results in early phase trials as presented at ASH presumably herald therapeutic advances, or at a minimum, a larger, potentially confirmatory, randomized trial. However, the predictive value of an ASH abstract reporting positive results in AML for subsequent clinical utility seems low (Estey 2006, ASH). Furthermore, not all results presented at ASH are published in peer-reviewed journals, and selectively publishing positive results leads to publication bias. Moreover, truly negative studies may be scientifically more rigorous and accurate than positive studies given the unequivocal findings. The extent of publication bias is unknown as is the frequency with which positive or negative abstracts lead to subsequent investigation in phase III and the reasons why positive phase II studies might not progress to phase III.
We downloaded all 2013 - 2015 ASH abstracts (N = 17,251) and evaluated all abstracts reporting phase II clinical trials (N = 371) of novel drugs and therapeutic regimens presented at ASH in these years, covering investigational treatments of MM, CLL, AML, DLBCL, MDS, NHL, ALL, CML, MCL, SLL, other lymphomas and POEMS. We first scored abstracts "positive", "negative" or "inconclusive". Criteria for a positive abstract were words/phrases such as "encouraging", "promising", "could represent a novel therapeutic option" and "warrants investigation in a randomized trial". Negative abstracts included terms such as "does not support further research" and "demonstrates no clinical activity". The remainder were scored as inconclusive. Using this approach, we scored 296/371 (80%) abstracts as positive, 37/371 (10%) as negative, and 38/371 (10%) as inconclusive.
292/371 abstracts (79%) were published in peer-reviewed journals. The abstract conclusion (positive, negative or inconclusive) was not associated with publication in a peer-reviewed journal. Most frequently, studies were published in Blood (34/292 [11.6%]) and British Journal of Haematology (39/292 [13.4%]) . In Blood, 91% (31/34) of the studies were positive. British Journal of Haematology published significantly more negative studies than Blood (26%, Fisher Exact p = 0.02). Abstracts reporting studies with larger sample sizes tended to be published more often (p = 0.066).
Differences exist between the abstract conclusion and later peer-reviewed publications. Of positive ASH abstracts, 6% changed to a negative conclusion in the peer-reviewed publication. Similarly, 6.5% of the initial negative abstract later reversed to a positive conclusion.
53% of positive abstracts did not lead to phase III studies, as registered on clinicaltrials.gov. Subsequently, regimens described in positive peer-reviewed publications did not proceed to phase III research in 48%. To explore why, we sent questionnaires to the first and/or last authors of positive studies not prompting phase III trials. 52% responded. Failure of positive phase II trials to proceed to phase III was due to the decision by the pharmaceutical company to halt clinical investigation (44%), lack of any intent to study the drug in phase III in the first place (40%), insufficient funding (35%), insufficient efficacy (despite the "positive" abstract; 33%) and safety concerns (4%) (Figure). Additional reasons for not proceeding to phase III were the availability of newer regimens, the rarity of the disease, or when regulatory approval had already been obtained after phase II.
In conclusion, "positive" and "negative" ASH abstracts are published as full papers equally often, although the positive ones may be published more often in journals with higher "impact factors". More than half of the regimens presented in positive ASH abstracts remain unevaluated in randomized phase III trials. A separate problem is the likely tendency to disproportionately submit (and/or accept) positive, rather than negative, studies to ASH in the first place. We believe our findings raise issues in clinical research that may not be in the best interest of patients. This demands more consideration than it currently receives.
Figure 1 Figure 1.
Disclosures
Ossenkoppele: Astellas: Consultancy, Honoraria; Agios: Consultancy, Honoraria; Abbvie, AGIOS, BMS/Celgene Astellas,AMGEN, Gilead,Servier,JAZZ,Servier Novartis: Consultancy, Honoraria; BMS/Celgene: Consultancy, Honoraria; Jazz: Consultancy, Honoraria; Servier: Consultancy, Honoraria; Gilead: Consultancy, Honoraria. Rowe: Biosight Inc.: Consultancy.
Most ASH Abstracts Reporting Phase II Studies Lead to Peer-Reviewed Publications, but Less Than 50% of "Positive" Abstracts Lead to Phase III Investigations: An Analysis of 371 Abstracts 2013 - 2015
