Generating publicly available progress notes of authentic quality: A crowdsourcing-based approach (Preprint)

BACKGROUND Sharing progress notes as a common social capital is essential in research and education, but the content of progress notes is sensitive and needs to be kept confidential. Publishing actual progress notes are difficult due to privacy concerns. OBJECTIVE This study aims to generate a large repository of pseudo-progress notes of authentic quality. We focused on two requirements for authentic quality: the validity and consistency of the data, from the perspective of medical practice, and the empirical and semantic characteristics of progress notes, such as shorthand styles used for reporting changes in a patient's physical status, long narrative sentences detailing patient anxiety, and interprofessional communications. METHODS We proposed a practical framework that consists of a simulation of the notes and evaluation of the simulated notes. The framework utilized two human cognitive traits: (1) the ability to use imitation to simulate objects with diverse characteristics without background knowledge and (2) the use of comparison as a strategy for deep thinking. This enabled crowd workers to generate a large number of progress notes. Our framework involved three steps. In step 1, crowd workers imitated actual progress notes decomposed into subject data (S), object data (O), and assessment and plan (A/P). These imitated texts were then shuffled and recomposed in S, O, and A/P in order to create simulated progress notes. In step 2, crowd workers identified the characteristics of actual progress notes based on comparisons between actual and dummy progress notes. These characteristics were clustered based on their similarities. Each cluster exhibited the empirical and semantic characteristics of the actual progress notes. Finally, in step 3, the texts from step 1 that exhibited the identified characteristics from step 2 were evaluated as quality-guaranteed progress notes that met the two requirements. All data were preprocessed to protect patient privacy. RESULTS Step 1: By recomposing the 700 imitated texts, 9,856 simulated progress notes were generated. Step 2: 3,938 differences between actual progress notes and dummy progress notes were identified. After clustering, 166 characteristics were evaluated to be appropriate as empirical and semantic characteristics of the actual progress notes. Step 3: 500 crowd workers demonstrated that 83.0% of the simulated progress notes satisfied at least one of the characteristics obtained in step 2. The crowd workers' artificially-reproduced progress notes were evaluated to determine the most realistic, based on four metrics: disease, morpheme, readability, and reality. CONCLUSIONS Our results demonstrated that crowd workers could generate and evaluate highly professional documents. We have made our large repository of high-quality crowdsourced progress notes publicly available, and we encourage their use in the development of medical education and research.