Abstract
Background
Polycystic ovary syndrome (PCOS) is characterized by hyperandrogenemia, oligo-anovulation, and numerous ovarian cysts. Hospital electronic medical records provide an avenue for investigating polycystic ovary morphology commonly seen in PCOS at a large scale. The purpose of this study was to develop and evaluate the performance of two machine learning text algorithms, for classification of polycystic ovary morphology (PCOM) in pelvic ultrasounds.
Methods
Pelvic ultrasound reports from patients at Boston Medical Center between October 1, 2003 and December 12, 2016 were included for analysis, which resulted in 39,093 ultrasound reports from 25,535 unique women. Following the 2003 Rotterdam Consensus Criteria for polycystic ovary syndrome, 2000 randomly selected ultrasounds were expert labeled for PCOM status as present, absent, or unidentifiable (not able to be determined from text alone). An ovary was marked as having PCOM if there was mention of numerous peripheral follicles or if the volume was greater than 10 ml in the absence of a dominant follicle or other confounding pathology. Half of the labeled data was used to develop and refine the algorithms, and the other half was used as a test set for evaluating its accuracy.
Results
On the evaluation set of 1000 random US reports, the accuracy of the classifiers were 97.6% (95% CI: 96.5, 98.5%) and 96.1% (94.7, 97.2%). Both models were more adept at identifying PCOM-absent ultrasounds than either PCOM-unidentifiable or PCOM-present ultrasounds. The two classifiers estimated prevalence of PCOM within the whole set of 39,093 ultrasounds to be 44% PCOM-absent, 32% PCOM-unidentifiable, and 24% PCOM-present.
Conclusions
Although accuracy measured on the test set and inter-rater agreement between the two classifiers (Cohen’s Kappa = 0.988) was high, a major limitation of our approach is that it uses the ultrasound report text as a proxy and does not directly count follicles from the ultrasound images themselves.
Comparison of Anti-müllerian Hormone (AMH) and Hormonal Assays for Phenotypic Classification of Polycystic Ovary Syndrome
