A Tool for Quantifying the Quality of Online Health Information on Student Health Center Websites (Preprint)

BACKGROUND The Internet has become a major source of health information especially for adolescents and young adults. Unfortunately, inaccurate, incomplete or outdated health information is widespread online. Often adolescents and young adults turn to authoritative websites such as the student health center (SHC) website of the university they are attending to obtain reliable health information. Although most on-campus SHC clinics comply with the American College Health Association (ACHA) standards, their websites are not subject to any standards or code of conduct. In the absence of quality standards or guidelines, the monitoring and compliance processes do not exist for SHC websites either. As such, there is no oversight on the health information published on the SHC websites by any central governing body. OBJECTIVE Our objective is to enable researchers to monitor online information quality at scale. We have created a tool that can efficiently quantify the quality of information posted on SHC websites about a health topic. Specifically, this quantitative tool provides information on quality, such as reading ease, coverage of the topic, and the degree of fact-based objective information. METHODS Our cross-functional team has designed and developed an open-source software, QMOHI: Quantitative Measures of Online Health Information, using the Agile software development methodology. The QMOHI tool finds the SHC website and gathers information on the specific health topic of interest from a prespecified list of university websites. Based on the retrieved text, the tool computes eight different quality metrics. The QMOHI tool is a fully automated tool that is designed to be scalable, generalizable, and robust. RESULTS The first empirical evaluation shows that the QMOHI tool is highly scalable and substantially more efficient than the manual approach of assessing online information quality. The second experimental results demonstrate QMOHI’s ability to work effectively with starkly different health topics (COVID, Cancer, LARC, and Condom) and with narrowly focused topics (hormonal IUD and copper IUD); thereby establishing the generalizability and versatility of the tool. The results from the last experiment demonstrate that QMOHI is not vulnerable to typical structural changes that SHC websites may undergo (e.g. URL changes) over a long period of time. QMOHI is able to support longitudinal studies by being robust to such website changes. CONCLUSIONS QMOHI allows public health researchers and practitioners to conduct large-scale studies of SHC websites that were previously too time intensive. The capability to generalize broadly or focus narrowly allows for wide applications of QMOHI, equipping researchers to study both mainstream and underexplored health topics. QMOHI’s ability to robustly analyze SHC websites periodically facilitates longitudinal investigations and monitor SHC progress. QMOHI serves as a launching pad for our future work that aims to develop a broadly applicable public health tool for online health information studies with potential applications far beyond SHC websites.