National Database Research in Head and Neck Reconstructive Surgery: A Call for Increased Transparency and Reproducibility

Objective To reproduce a published study comparing outcomes of patients who underwent microvascular reconstruction by plastic surgeons and otolaryngologists and to examine how case selection and methodology using the National Surgical Quality Improvement Program (NSQIP) data set can affect results and conclusions. Study Design Cross-sectional analysis of US national database. Setting American College of Surgeons National Surgical Quality Improvement Program (NSQIP) from 2005 to 2017. Subjects and Methods A recently published study that used the NSQIP database to compare outcomes after head and neck free tissue transfer between plastic surgeons and otolaryngologists was reproduced. Different approaches to case selection and statistical analysis were evaluated and their effects on statistical significance and study conclusions were compared. Results When all cases of free tissue transfer, captured in NSQIP between 2005 and 2017, were compared between plastic surgery and otolaryngology, plastic surgery patients appeared to have lower rates of complications and length of stay. However, a more in-depth analysis demonstrated that these results were confounded by older and sicker otolaryngology patients. A second analysis of the same NSQIP data, limited to only head and neck oncologic reconstructions, demonstrated that otolaryngology patients had fewer complications on univariate and multivariable analysis. Conclusion We demonstrated how case selection and analysis can significantly affect results. It is incumbent upon researchers who use NSQIP and other publicly available data sets to fully detail their methodology to allow other researchers to reproduce and evaluate their work and for the journal editorial process to carefully evaluate the methodology and conclusions of their contributing authors.