Four points regarding reproducibility and external statistical validity: a comment on Walter et al.
Walter et al. (2021) present phase 1–2–3 trial data that show two doses of the BNT162b2 (Pfizer–BioN-Tech) Covid-19 vaccine were safe and effective in children aged 5–11 years. Given that millions of children in this age group are receiving the paediatric Pfizer COVID-19 vaccine, that there are potential risks, and that the balance of benefits over potential risks is more limited in children compared to adults due to low rates of serious disease (ATAGI 2021), gold standards ought to be applied to supporting data in terms of placebo-controlled disease endpoint efficacy trials, safety databases large enough to detect adverse events, and appropriate data sharing to enable reproduction and scrutiny of results. Four points are worthy of attention regarding the reproducibility and external statistical validity of the analysis reported in Walter et al. (2021). ‘External validity’ refers to the extent to which conclusions drawn from the data (and statistical tests thereof) are likely to correspond to, or be generalisable to, the real world (Campbell 1957). ‘Reproducibility’ refers to the ability of independent researchers to draw the same conclusions from the data (Kass et al. 2016).