Video-Based Pose Estimation for Gait Analysis in Stroke Survivors during Clinical Assessments: A Proof-of-Concept Study

Recent advancements in deep learning have produced significant progress in markerless human pose estimation, making it possible to estimate human kinematics from single camera videos without the need for reflective markers and specialized labs equipped with motion capture systems. Such algorithms have the potential to enable the quantification of clinical metrics from videos recorded with a handheld camera. Here we used DeepLabCut, an open-source framework for markerless pose estimation, to fine-tune a deep network to track 5 body keypoints (hip, knee, ankle, heel, and toe) in 82 below-waist videos of 8 patients with stroke performing overground walking during clinical assessments. We trained the pose estimation model by labeling the keypoints in 2 frames per video and then trained a convolutional neural network to estimate 5 clinically relevant gait parameters (cadence, double support time, swing time, stance time, and walking speed) from the trajectory of these keypoints. These results were then compared to those obtained from a clinical system for gait analysis (GAITRite®, CIR Systems). Absolute accuracy (mean error) and precision (standard deviation of error) for swing, stance, and double support time were within 0.04 ± 0.11 s; Pearson’s correlation with the reference system was moderate for swing times (<i>r</i> = 0.4–0.66), but stronger for stance and double support time (<i>r</i> = 0.93–0.95). Cadence mean error was −0.25 steps/min ± 3.9 steps/min (<i>r</i> = 0.97), while walking speed mean error was −0.02 ± 0.11 m/s (<i>r</i> = 0.92). These preliminary results suggest that single camera videos and pose estimation models based on deep networks could be used to quantify clinically relevant gait metrics in individuals poststroke, even while using assistive devices in uncontrolled environments. Such development opens the door to applications for gait analysis both inside and outside of clinical settings, without the need of sophisticated equipment.

A Smartphone Application as an Exploratory Endpoint in a Phase 3 Parkinson’s Disease Clinical Trial: A Pilot Study

<b><i>Background:</i></b> Smartphones can generate objective measures of Parkinson’s disease (PD) and supplement traditional in-person rating scales. However, smartphone use in clinical trials has been limited. <b><i>Objective:</i></b> This study aimed to determine the feasibility of introducing a smartphone research application into a PD clinical trial and to evaluate the resulting measures. <b><i>Methods:</i></b> A smartphone application was introduced part-way into a phase 3 randomized clinical trial of inosine. The application included finger tapping, gait, and cognition tests, and participants were asked to complete an assessment battery at home and in clinic alongside the Movement Disorder Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). <b><i>Results:</i></b> Of 236 eligible participants in the parent study, 88 (37%) consented to participate, and 59 (27 randomized to inosine and 32 to placebo) completed a baseline smartphone assessment. These 59 participants collectively completed 1,292 batteries of assessments. The proportion of participants who completed at least one smartphone assessment was 61% at 3, 54% at 6, and 35% at 12 months. Finger tapping speed correlated weakly with the part III motor portion (<i>r</i> = −0.16, left hand; <i>r</i> = −0.04, right hand) and total (<i>r</i> = −0.14) MDS-UPDRS. Gait speed correlated better with the same measures (<i>r</i> = −0.25, part III motor; <i>r</i> = −0.34, total). Over 6 months, finger tapping speed, gait speed, and memory scores did not differ between those randomized to active drug or placebo. <b><i>Conclusions:</i></b> Introducing a smartphone application midway into a phase 3 clinical trial was challenging. Measures of bradykinesia and gait speed correlated modestly with traditional outcomes and were consistent with the study’s overall findings, which found no benefit of the active drug.

Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption

The assessment of health and disease requires a set of criteria to define health status and progression. These health measures are referred to as “endpoints.” A “digital endpoint” is defined by its use of sensor-generated data often collected outside of a clinical setting such as in a patient’s free-living environment. Applicable sensors exist in an array of devices and can be applied in a diverse set of contexts. For example, a smartphone’s microphone might be used to diagnose or predict mild cognitive impairment due to Alzheimer’s disease or a wrist-worn activity monitor (such as those found in smartwatches) may be used to measure a drug’s effect on the nocturnal activity of patients with sickle cell disease. Digital endpoints are generating considerable excitement because they permit a more authentic assessment of the patient’s experience, reveal formerly untold realities of disease burden, and can cut drug discovery costs in half. However, before these benefits can be realized, effort must be applied not only to the technical creation of digital endpoints but also to the environment that allows for their development and application. The future of digital endpoints rests on meaningful interdisciplinary collaboration, sufficient evidence that digital endpoints can realize their promise, and the development of an ecosystem in which the vast quantities of data that digital endpoints generate can be analyzed. The fundamental nature of health care is changing. With coronavirus disease 2019 serving as a catalyst, there has been a rapid expansion of home care models, telehealth, and remote patient monitoring. The increasing adoption of these health-care innovations will expedite the requirement for a digital characterization of clinical status as current assessment tools often rely upon direct interaction with patients and thus are not fit for purpose to be administered remotely. With the ubiquity of relatively inexpensive sensors, digital endpoints are positioned to drive this consequential change. It is therefore not surprising that regulators, physicians, researchers, and consultants have each offered their assessment of these novel tools. However, as we further describe later, the broad adoption of digital endpoints will require a cooperative effort. In this article, we present an analysis of the current state of digital endpoints. We also attempt to unify the perspectives of the parties involved in the development and deployment of these tools. We conclude with an interdependent list of challenges that must be collaboratively addressed before these endpoints are widely adopted.

Real-World Evidence for a Smartwatch-Based Parkinson’s Motor Assessment App for Patients Undergoing Therapy Changes

<b><i>Introduction:</i></b> Parkinson’s disease (PD) is poorly quantified by patients outside the clinic, and paper diaries have problems with subjective descriptions and bias. Wearable sensor platforms; however, can accurately quantify symptoms such as tremor, dyskinesia, and bradykinesia. Commercially available smartwatches are equipped with accelerometers and gyroscopes that can measure motion for objective evaluation. We sought to evaluate the clinical utility of a prescription smartwatch-based monitoring system for PD utilizing periodic task-based motor assessment. <b><i>Methods:</i></b> Sixteen patients with PD used a smartphone- and smartwatch-based monitoring system to objectively assess motor symptoms for 1 week prior to instituting a doctor recommended change in therapy and for 4 weeks after the change. After 5 weeks the participants returned to the clinic to discuss their results with their doctor, who made therapy recommendations based on the reports and his clinical judgment. Symptom scores were synchronized with the medication diary and the temporal effects of therapy on weekly and hourly timescales were calculated. <b><i>Results:</i></b> Thirteen participants successfully completed the study and averaged 4.9 assessments per day for 3 days per week during the study. The doctor instructed 8 participants to continue their new regimens and 5 to revert to their previous regimens. The smartwatch-based assessments successfully captured intraday fluctuations and short- and long-term responses to therapies, including detecting significant improvements (<i>p</i> &#x3c; 0.05) in at least one symptom in 7 participants. <b><i>Conclusions:</i></b> The smartwatch-based app successfully captured temporal trends in symptom scores following application of new therapy on hourly, daily, and weekly timescales. These results suggest that validated smartwatch-based PD monitoring can provide clinically relevant information and may reduce the need for traditional office visits for therapy adjustment.

Developing a Novel Measurement of Sleep in Rheumatoid Arthritis: Study Proposal for Approach and Considerations

The development of novel digital endpoints (NDEs) using digital health technologies (DHTs) may provide opportunities to transform drug development. It requires a multidisciplinary, multi-study approach with strategic planning and a regulatory-guided pathway to achieve regulatory and clinical acceptance. Many NDEs have been explored; however, success has been limited. To advance industry use of NDEs to support drug development, we outline a theoretical, methodological study as a use-case proposal to describe the process and considerations when developing and obtaining regulatory acceptance for an NDE to assess sleep in patients with rheumatoid arthritis (RA). RA patients often suffer joint pain, fatigue, and sleep disturbances (SDs). Although many researchers have investigated the mobility of joint functions using wearable technologies, the research of SD in RA has been limited due to the availability of suitable technologies. We proposed measuring the improvement of sleep as the novel endpoint for an anti-TNF therapy and described the meaningfulness of the measure, considerations of tool selection, and the design of clinical validation. The recommendations from the FDA patient-focused drug development guidance, the Clinical Trials Transformation Initiative (CTTI) pathway for developing novel endpoints from DHTs, and the V3 framework developed by the Digital Medicine Society (DiMe) have been incorporated in the proposal. Regulatory strategy and engagement pathways are also discussed.

First Regulatory Qualification of a Novel Digital Endpoint in Duchenne Muscular Dystrophy: A Multi-Stakeholder Perspective on the Impact for Patients and for Drug Development in Neuromuscular Diseases

<b><i>Background:</i></b> Functional outcome measures used to assess efficacy in clinical trials of investigational treatments for rare neuromuscular diseases like Duchenne muscular dystrophy (DMD) are performance-based tasks completed by the patient during hospital visits. These are prone to bias and may not reflect motor abilities in real-world settings. Digital tools, such as wearable devices and other remote sensors, provide the opportunity for continuous, objective, and sensitive measurements of functional ability during daily life. Maintaining ambulation is of key importance to individuals with DMD. Stride velocity 95th centile (SV95C) is the first wearable acquired digital endpoint to receive qualification from the European Medicines Agency (EMA) to quantify the ambulation ability of ambulant DMD patients aged ≥5 years in drug therapeutic studies; it is also currently under review for the US Food and Drug Administration (FDA) qualification. <b><i>Summary:</i></b> Focusing on SV95C as a key example, we describe perspectives of multiple stakeholders on the promise of novel digital endpoints in neuromuscular disease drug development.

Beyond the Therapist’s Office: Merging Measurement-Based Care and Digital Medicine in the Real World

This viewpoint focuses on the ways in which digital medicine and measurement-based care can be utilized in tandem to promote better assessment, patient engagement, and an improved quality of psychiatric care. To date, there has been an underutilization of digital measurement in psychiatry, and there is little discussion of the feedback and patient engagement process in digital medicine. Measurement-based care is a recognized evidence-based strategy that engages patients in an understanding of their outcome data. When implemented as designed, providers review the scores and trends in outcome immediately and then provide feedback to their patients. However, the process is typically confined to office visits, which does not provide a complete picture of a patient’s progress and functioning. The process is labor intensive, even with digital feedback systems, but the integration of passive metrics obtained through wearables and apps can supplement office-based observations. This enhanced measurement-based care process can provide a picture of real-world patient functioning through passive metrics (activity, sleep, etc.). This can potentially engage patients more in their health data and involve a critically needed therapeutic alliance component in digital medicine.

Advanced Machine Learning Tools to Monitor Biomarkers of Dysphagia: A Wearable Sensor Proof-of-Concept Study

<b><i>Introduction:</i></b> Difficulty swallowing (dysphagia) occurs frequently in patients with neurological disorders and can lead to aspiration, choking, and malnutrition. Dysphagia is typically diagnosed using costly, invasive imaging procedures or subjective, qualitative bedside examinations. Wearable sensors are a promising alternative to noninvasively and objectively measure physiological signals relevant to swallowing. An ongoing challenge with this approach is consolidating these complex signals into sensitive, clinically meaningful metrics of swallowing performance. To address this gap, we propose 2 novel, digital monitoring tools to evaluate swallows using wearable sensor data and machine learning. <b><i>Methods:</i></b> Biometric swallowing and respiration signals from wearable, mechano-acoustic sensors were compared between patients with poststroke dysphagia and nondysphagic controls while swallowing foods and liquids of different consistencies, in accordance with the Mann Assessment of Swallowing Ability (MASA). Two machine learning approaches were developed to (1) classify the severity of impairment for each swallow, with model confidence ratings for transparent clinical decision support, and (2) compute a similarity measure of each swallow to nondysphagic performance. Task-specific models were trained using swallow kinematics and respiratory features from 505 swallows (321 from patients and 184 from controls). <b><i>Results:</i></b> These models provide sensitive metrics to gauge impairment on a per-swallow basis. Both approaches demonstrate intrasubject swallow variability and patient-specific changes which were not captured by the MASA alone. Sensor measures encoding respiratory-swallow coordination were important features relating to dysphagia presence and severity. Puree swallows exhibited greater differences from controls than saliva swallows or liquid sips (<i>p</i> &#x3c; 0.037). <b><i>Discussion:</i></b> Developing interpretable tools is critical to optimize the clinical utility of novel, sensor-based measurement techniques. The proof-of-concept models proposed here provide concrete, communicable evidence to track dysphagia recovery over time. With refined training schemes and real-world validation, these tools can be deployed to automatically measure and monitor swallowing in the clinic and community for patients across the impairment spectrum.