Streamlining statistical reproducibility: NHLBI ORCHID clinical trial results reproduction

Reproducibility in medical research has been a long-standing issue. More recently, the COVID-19 pandemic has publicly underlined this fact as the retraction of several studies reached out to general media audiences. A significant number of these retractions occurred after in-depth scrutiny of the methodology and results by the scientific community. Consequently, these retractions have undermined confidence in the peer-review process, which is not considered sufficiently reliable to generate trust in the published results. This partly stems from opacity in published results, the practical implementation of the statistical analysis often remaining undisclosed. We present a workflow that uses a combination of informatics tools to foster statistical reproducibility: an open-source programming language, Jupyter Notebook, cloud-based data repository, and an application programming interface can streamline an analysis and help to kick-start new analyses. We illustrate this principle by (1) reproducing the results of the ORCHID clinical trial, which evaluated the efficacy of hydroxychloroquine in COVID-19 patients, and (2) expanding on the analyses conducted in the original trial by investigating the association of premedication with biological laboratory results. Such workflows will be encouraged for future publications from National Heart, Lung, and Blood Institute-funded studies.

Patients’, pharmacists’, and prescribers’ attitude toward using blockchain and machine learning in a proposed ePrescription system: online survey

Objective To evaluate the attitudes of the parties involved in the system toward the new features and measure the potential benefits of introducing the use of blockchain and machine learning (ML) to strengthen the in-place methods for safely prescribing medication. The proposed blockchain will strengthen the security and privacy of the patient’s prescription information shared in the network. Once the ePrescription is submitted, it is only available in read-only mode. This will ensure there is no alteration to the ePrescription information after submission. In addition, the blockchain will provide an improved tracking mechanism to ensure the originality of the ePrescription and that a prescriber can only submit an ePrescription with the patient’s authorization. Lastly, before submitting an ePrescription, an ML algorithm will be used to detect any anomalies (eg, missing fields, misplaced information, or wrong dosage) in the ePrescription to ensure the safety of the prescribed medication for the patient. Methods The survey contains questions about the features introduced in the proposed ePrescription system to evaluate the security, privacy, reliability, and availability of the ePrescription information in the system. The study population is comprised of 284 respondents in the patient group, 39 respondents in the pharmacist group, and 27 respondents in the prescriber group, all of whom met the inclusion criteria. The response rate was 80% (226/284) in the patient group, 87% (34/39) in the pharmacist group, and 96% (26/27) in the prescriber group. Key Findings The vast majority of the respondents in all groups had a positive attitude toward the proposed ePrescription system’s security and privacy using blockchain technology, with 72% (163/226) in the patient group, 70.5% (24/34) in the pharmacist group, and 73% (19/26) in the prescriber group. Moreover, the majority of the respondents in the pharmacist (70%, 24/34) and prescriber (85%, 22/26) groups had a positive attitude toward using ML algorithms to generate alerts regarding prescribed medication to enhance the safety of medication prescribing and prevent medication errors. Conclusion Our survey showed that a vast majority of respondents in all groups had positive attitudes toward using blockchain and ML algorithms to safely prescribe medications. However, a need for minor improvements regarding the proposed features was identified, and a post-implementation user study is needed to evaluate the proposed ePrescription system in depth.

Sleep apnea phenotyping and relationship to disease in a large clinical biobank

Objective Sleep apnea is associated with a broad range of pathophysiology. While electronic health record (EHR) information has the potential for revealing relationships between sleep apnea and associated risk factors and outcomes, practical challenges hinder its use. Our objectives were to develop a sleep apnea phenotyping algorithm that improves the precision of EHR case/control information using natural language processing (NLP); identify novel associations between sleep apnea and comorbidities in a large clinical biobank; and investigate the relationship between polysomnography statistics and comorbid disease using NLP phenotyping. Materials and Methods We performed clinical chart reviews on 300 participants putatively diagnosed with sleep apnea and applied International Classification of Sleep Disorders criteria to classify true cases and noncases. We evaluated 2 NLP and diagnosis code-only methods for their abilities to maximize phenotyping precision. The lead algorithm was used to identify incident and cross-sectional associations between sleep apnea and common comorbidities using 4876 NLP-defined sleep apnea cases and 3× matched controls. Results The optimal NLP phenotyping strategy had improved model precision (≥0.943) compared to the use of one diagnosis code (≤0.733). Of the tested diseases, 170 disorders had significant incidence odds ratios (ORs) between cases and controls, 8 of which were confirmed using polysomnography (n = 4544), and 281 disorders had significant prevalence OR between sleep apnea cases versus controls, 41 of which were confirmed using polysomnography data. Discussion and Conclusion An NLP-informed algorithm can improve the accuracy of case-control sleep apnea ascertainment and thus improve the performance of phenome-wide, genetic, and other EHR analyses of a highly prevalent disorder.

Generalizable EHR-R-REDCap pipeline for a national multi-institutional rare tumor patient registry

Objective To develop a clinical informatics pipeline designed to capture large-scale structured Electronic Health Record (EHR) data for a national patient registry. Materials and Methods The EHR-R-REDCap pipeline is implemented using R statistical software to remap and import structured EHR data into the Research Electronic Data Capture (REDCap)-based multi-institutional Merkel Cell Carcinoma (MCC) Patient Registry using an adaptable data dictionary. Results Clinical laboratory data were extracted from EPIC Clarity across several participating institutions. Laboratory values (Labs) were transformed, remapped, and imported into the MCC registry using the EHR labs abstraction (eLAB) pipeline. Forty-nine clinical tests encompassing 482 450 results were imported into the registry for 1109 enrolled MCC patients. Data-quality assessment revealed highly accurate, valid labs. Univariate modeling was performed for labs at baseline on overall survival (N = 176) using this clinical informatics pipeline. Conclusion We demonstrate feasibility of the facile eLAB workflow. EHR data are successfully transformed and bulk-loaded/imported into a REDCap-based national registry to execute real-world data analysis and interoperability.

Prenatal Telemedicine During COVID-19: Patterns of Use and Barriers to Access

Lay Summary To protect pregnant patients from infection during the COVID pandemic, maternity care providers turned to video and phone visits (“telemedicine”) to provide as much prenatal care as possible. To evaluate this change in our prenatal care program, we surveyed 164 pregnant people who had participated in a virtual prenatal visit about their care. Participants reported both positive and negative experiences, ranging from appreciation for having a safer option than in-person visits during the pandemic, to problems due to poor internet connection, lack of privacy, and lack of access to necessary equipment. Although 77.4% of respondents indicated they would recommend telemedicine to a friend, our program evaluation highlights the fact that the ability to participate in virtual care is not equally distributed. Unless steps are taken to address this problem, relying on telemedicine for a significant portion of prenatal care could result in widening disparities in prenatal care and outcomes. Policymakers and healthcare systems which provide telemedicine must address issues of access to technology and connectivity to avoid adding to maternal health disparities. Objective To evaluate patient experience with a prenatal telemedicine visit and identify barriers to accessing telemedicine among rural pregnant people in northern New England during the beginning of the COVID-19 pandemic. Materials and Methods We conducted a post-visit electronic survey of pregnant people who successfully participated in a prenatal telemedicine visit at a rural academic medical center in Northern New England. Nineteen questions were included in five domains; 1) engagement with prenatal care; 2) barriers to telemedicine and in person healthcare; 3) experience of prenatal care; 4) remote pregnancy surveillance tools; 5) sources of COVID-19 information. Results Responses were obtained from 164 pregnant people. Forty percent of participants had participated in an audio-only telemedicine visit, and 60% in a video telemedicine visit. The visit was easy or somewhat easy for 79% of respondents and somewhat difficult or difficult for 6.8%. The most common barrier to accessing telemedicine was poor internet or phone connectivity, followed by childcare responsibilities, lack of equipment and lack of privacy. Participants also engaged in additional remote prenatal care including phone calls with registered nurses (7.6%), communication with the obstetrics team through a secure health messaging portal (21.1%) and home health monitoring (76.3%). Discussion and Conclusions In this survey evaluating the experience of pregnant people participating in a prenatal telemedicine visit during the COVID-19 pandemic, respondents had a positive experience with telemedicine overall, but also identified significant barriers to participation including issues with connectivity and lack of equipment for the visit. Most participants used telemedicine in combination with other tools for remote self-care.

Dark Clouds and Silver Linings: Impact of COVID-19 on Internet Users’ Privacy

Objectives To examine the impact of COVID-19 pandemic on the extent of potential violations of Internet users’ privacy. Materials and Methods We conducted a longitudinal study of the data sharing practices of the top 1,000 websites in the US between April 9th and August 27th, 2020. We fitted a conditional latent growth curve model on the data to examine the longitudinal trajectory of the third-party data sharing over the 21 weeks period of the study and examine how website characteristics affect this trajectory. We denote websites that asked for permission before placing cookies on users’ browsers as "privacy-respecting". Results As the weekly number of COVID-19 deaths increased by 1,000, the average number of third parties increased by 0.26 [95%CI, 0.15 to 0.37] P<.001 units in the next week. This effect was more pronounced for websites with higher traffic as they increased their third parties by an additional 0.41 [95% CI, 0.18 to 0.64]; P<.001 units per week. However, privacy respecting websites that experienced a surge in traffic reduced their third parties by 1.01 [95% CI, -2.01 to 0]; P = 0.05 units per week in response to every 1,000 COVID-19 deaths in the preceding week. Discussion While in general websites shared their users’ data with more third parties as COVID-19 progressed in the US, websites’ expected traffic and respect for users’ privacy significantly affect such trajectory. Conclusions Attention should also be paid to the impact of the pandemic on elevating online privacy threats, and the variation in third-party tracking among different types of websites. Lay Summary As the COVID-19 pandemic progressed in the country, the demand for online services surged. As the level of Internet use increased, websites’ opportunity to track and monetize users’ data increased with it. In this research, we examine the extent to which websites increased the number of third-parties with which they share their user’ data and how such practices were moderated by a website’s level of respect for users’ privacy and traffic surge. We find that while the number of third parties increased over time, the websites with higher respect for privacy tend to decrease the number of their parties only if they also experience a significant increase in their traffic.

Patient Generated Health Data Earn a Seat at the Table:Clinical Adoption During the Covid-19 Transition to Telemedicine

Background Patient generated health data (PGHD) have not achieved widespread clinical adoption. However, the COVID-induced shift to telemedicine may have created opportunities for PGHD as surrogates for vital signs collected in-person. We assessed whether this shift was associated with greater ambulatory care PGHD use. Methods We conducted an interrupted time series analysis of physician enrollment in, and patient-initiated vital sign transmission of non-COVID associated PGHD through, a national PGHD platform (Validic). Results Ten health systems, 4,695 physicians, and 51,320 patients were included. We found a significant increase in physician enrollment (slope change of 0.86/week, P=.02). Platform application programming interface calls continued their pre-COVID upward trend, despite large reductions in overall encounters. Discussion These findings suggest significantly greater pandemic-associated clinical demand for PGHD, and patient supply disproportionate to encounter rates. Conclusion Increasing clinical use and ongoing efforts to reduce barriers, could help seize current adoption momentum to realize PGHD’s potential value. Lay Summary Patient generated health data (PGHD) - health-related data created and recorded by or from patients outside of the clinical setting to help address a health concern—have not yet achieved widespread adoption in routine clinical care. The COVID-19 pandemic precipitated a rapid transition of outpatient encounters to telemedicine in which healthcare providers lacked access to vital signs routinely collected during in-person visits. We conducted an analysis to determine whether the transition to telemedicine increased patient transmission of, and provider adoption of vital sign-related PGHD as surrogates for their in-person equivalents. We found that the number of healthcare providers enrolling on a national PGHD platform increased significantly following the transition to telemedicine, and that the amount of PGHD transmission continued the upward trajectory that it was already experiencing, substantially outpacing the dramatic decline in overall encounters that occurred early in the pandemic. While adoption challenges persist, including questions about accuracy of PGHD, liability, reimbursement, and the potential for exacerbating disparities, these findings suggest an increasing willingness of patients and healthcare providers to use vital sign-related PGHD to supplement telemedicine encounters. Increasing clinical use and ongoing efforts to reduce barriers, could help seize current adoption momentum to realize PGHD’s potential value.

Implementing a digital system for contact tracing and case investigation during COVID-19 pandemic in San Francisco: a qualitative study

Lay summary During the COVID-19 pandemic, many health departments implemented digital systems to collect, store, and share data for case investigation and contact tracing (CICT). In San Francisco, much of the contact tracing workforce was entirely remote and had little to no public health experience. Given this unique situation, we wanted to understand their experience with the digital system to inform future implementation of digital systems for public health responses. This case study describes how CICT workers in San Francisco experienced and used the digital system and how it could be improved. We conducted semi-structured 90-minute interviews and a short survey with a sample of 37 CICT workers, and found that, overall, the digital system was easy to learn and improved workers’ experience of data management during the pandemic. The digital system was also important in fostering a supportive and collaborative work environment. We found that the system could be improved to better support culturally sensitive care and highlight the importance of digital systems in ensuring equitable public health responses.

A scoping review of knowledge authoring tools used for developing computerized clinical decision support systems

Objective Clinical Knowledge Authoring Tools (CKATs) are integral to the computerized Clinical Decision Support (CDS) development life cycle. CKATs enable authors to generate accurate, complete, and reliable digital knowledge artifacts in a relatively efficient and affordable manner. This scoping review aims to compare knowledge authoring tools and derive the common features of CKATs. Materials and Methods We performed a keyword-based literature search, followed by a snowball search, to identify peer-reviewed publications describing the development or use of CKATs. We used PubMed and Embase search engines to perform the initial search (n = 1579). After removing duplicate articles, nonrelevant manuscripts, and not peer-reviewed publication, we identified 47 eligible studies describing 33 unique CKATs. The reviewed CKATs were further assessed, and salient characteristics were extracted and grouped as common CKAT features. Results Among the identified CKATs, 55% use an open source platform, 70% provide an application programming interface for CDS system integration, and 79% provide features to validate/test the knowledge. The majority of the reviewed CKATs describe the flow of information, offer a graphical user interface for knowledge authors, and provide intellisense coding features (94%, 97%, and 97%, respectively). The composed list of criteria for CKAT included topics such as simulating the clinical setting, validating the knowledge, standardized clinical models and vocabulary, and domain independence. None of the reviewed CKATs met all common criteria. Conclusion Our scoping review highlights the key specifications for a CKAT. The CKAT specification proposed in this review can guide CDS authors in developing more targeted CKATs.

Using a certified electronic health record technology platform to screen, test and refer patients with prediabetes

The objective of this study was to determine if certified electronic health record technology (CEHRT) can be used to identify and refer patients with prediabetes to lifestyle change programs (LCPs) recognized by the National Diabetes Prevention Program (DPP). This pilot utilized a prediabetes registry, patient portal, and clinical decision support to increase referrals. Data from 36 primary care providers showed 4930 patients were eligible for DPP LCP, 293 referrals were generated, compared to 20 referrals in the baseline period, and 116 patients enrolled. Referral to enrollment conversion rates were 41% in the study period and 69% in the post-study 1-year period. CEHRT functionalities can support systematic identification and management of prediabetes. The referral rate increased 7-fold compared to the baseline period, with high referral to enrollment conversion rates. CEHRT coupled with active provider engagement can serve as a tool to identify prediabetes patients and facilitate LCP referrals and enrollment.