A post pandemic roadmap toward remote assessment for neuromuscular disorders: limitations and opportunities

Recent advances in technology and expanding therapeutic opportunities in neuromuscular disorders has resulted in greater interest in and development of remote assessments. Over the past year, the rapid and abrupt COVID-19 shutdowns and stay-at-home orders imposed challenges to routine clinical management and clinical trials. As in-person services were severely limited, clinicians turned to remote assessments through telehealth to allow for continued care. Typically, disease-specific clinical outcome assessments (COAs) for neuromuscular disorders (NMD) are developed over many years through rigorous and iterative processes to fully understand their psychometric properties. While efforts were underway towards developing remote assessments for NMD before the pandemic, few if any were fully developed or validated. These included assessments of strength, respiratory function and patient-reported outcomes, as well as wearable technology and other devices to quantify physical activity and function. Without many choices, clinicians modified COAs for a virtual environment recognizing it was not yet known how they compared to standard in-person administration. Despite being able to quickly adapt to the demands of the COVID-19 pandemic, these experiences with remote assessments uncovered limitations and opportunities. It became clear that existing COAs required modifications for use in a virtual environment limiting the interpretation of the information gathered. Still, the opportunity for real-world evaluation and reduced patient burden were clear benefits to remote assessment and may provide a more robust understanding and characterization of disease impact in NMD. Hence, we propose a roadmap navigating an informed post-pandemic path toward development and implementation of safe and successful use of remote assessments for patients with NMD.

An Aging Focused Unobtrusive and Privacy-Preserving Digital Behaviorome

Digital measures are increasingly used as objective health measures in remote-monitoring settings. In addition to their use in purely clinical research, such as in clinical trials, one promising application area for sensor-derived digital measures is in technology-assisted ageing and ageing-related research. In this context, digital measures may be used to measure the risk of certain adverse events such as falls, and also to provide novel research insights into ageing and ageing-related conditions, like cognitive impairment. While major emphasis has been placed on deriving one or more digital measures from wearable devices, a more holistic approach inspired by systems biology that leverages large, non-exhaustive sets of digital measures may prove highly beneficial. Such an approach would be useful if combined with modern big data approaches like machine learning. As such, extensive sets of digital measures, which may be referred to as digital behavioromes, could help characterise new phenotypes in deep phenotyping efforts. These measures could also assist in the discovery of novel digital biomarkers or in the creation of digital clinical outcome assessments. While clinical research into digital measures focuses primarily on measures derived from wearable devices, proven technology used for long-term remote monitoring of older adults is generally contactless, unobtrusive, and privacy-preserving. In this context, we introduce and describe a digital behaviorome: a large, non-exhaustive set of digital measures based entirely on contactless, unobtrusive, and privacy-preserving sensor technologies. We also demonstrate how such a behaviorome can be used to build digital clinical outcome assessments that are relevant to ageing and derived from machine learning. These outcomes included fall risk, frailty, mild cognitive impairment, and late-life depression. With the exception of late-life depression, all digital outcome assessments demonstrated a promising ability (ROC AUC ≥ 0.7) to discriminate between positive and negative health outcomes, often in the range of comparable work with wearable devices. Finally, we highlight the possibility of using these digital behaviorome-based outcome assessments to discover novel potential digital biomarkers for each outcome. Here, we found reasonable contributors but also some potentially interesting new candidates regarding fall risk and mild cognitive impairment.

Clinical Outcome Assessment in Cancer Rehabilitation and the Central Role of Patient-Reported Outcomes

The aim of cancer rehabilitation is to help patients regain functioning and social participation. In order to evaluate and optimize rehabilitation, it is important to measure its outcomes in a structured way. In this article, we review the different types of clinical outcome assessments (COAs), including Clinician-Reported Outcomes (ClinROs), Observer-Reported Outcomes (ObsROs), Performance Outcomes (PerfOs), and Patient-Reported Outcomes (PROs). A special focus is placed on PROs, which are commonly defined as any direct report from the patient about their health condition without any interpretation by a third party. We provide a narrative review of available PRO measures (PROMs) for relevant outcomes, discuss the current state of PRO implementation in cancer rehabilitation, and highlight trends that use PROs to benchmark value-based care. Furthermore, we provide examples of PRO usage, highlight the benefits of electronic PRO (ePRO) collection, and offer advice on how to select, implement, and integrate PROs into the cancer rehabilitation setting to maximize efficiency.

Consensus Guidelines for Improving Quality of Assessment and Training for Neuromuscular Diseases

Critical components of successful evaluation of clinical outcome assessments (COAs) in multisite clinical trials and clinical practice are standardized training, administration, and documented reliability of scoring. Experiences of evaluators, alongside patient differences from regional standards of care, may contribute to heterogeneity in clinical center’s expertise. Achieving low variability and high reliability of COA is fundamental to clinical research and to give confidence in our ability to draw rational, interpretable conclusions from the data collected. The objective of this manuscript is to provide a framework to guide the learning process for COAs for use in clinics and clinical trials to maximize reliability and validity of COAs in neuromuscular disease (NMD). This is a consensus-based guideline with contributions from fourteen leading experts in clinical outcomes and the field of clinical outcome training in NMD. This framework should guide reliable and valid assessments in NMD specialty clinics and clinical trials. This consensus aims to expedite study start up with a progressive training pathway ranging from research naïve to highly experienced clinical evaluators. This document includes recommendations for education guidelines and roles and responsibilities of key stakeholders in COA assessment and implementation to ensure quality and consistency of outcome administration across different settings.

Remote Delivery of Motor Function Assessment and Training for Clinical Trials in Neuromuscular Disease: A Response to the COVID-19 Global Pandemic

Clinical outcome assessments of function or strength, assessed by physical therapists, are commonly used as primary endpoints in clinical trials, natural history studies and within clinics for individuals with neuromuscular disorders. These evaluations not only inform the efficacy of investigational agents in clinical trials, but also importantly track disease trajectory to prospectively advise need for equipment, home and work modifications, and other assistive devices. The COVID-19 pandemic had a global impact on the safety and feasibility of in-person visits and assessments, necessitating rapid development of mitigation strategies to ensure ongoing collection of key clinical trial endpoints and access to expert clinical care despite travel restrictions. Physical therapists who are expert in neuromuscular disorders working across clinics, countries, and clinical trials developed initial guidelines and methods for the suitability and feasibility of performing remote evaluations. A number of Sponsors introduced amendments to their study protocols to enable remote evaluations, supported by live video streaming of the assessment to their local clinical evaluators. Similarly, application of these techniques to clinical telemedicine enabled objective evaluations for use in payer discussions, equipment procurement, and general access to expert physical therapy services. Here we report on our methodology for adapting current practices to remote testing and considerations for remote evaluations.

Key Measurement Concepts and Appropriate Clinical Outcome Assessments in Pediatric Achondroplasia Clinical Trials

Background: This study aimed to identify fit-for-purpose clinical outcome assessments (COAs) to evaluate physical function and social and emotional well-being in clinical trials enrolling a pediatric population with achondroplasia. Qualitative interviews lasting 90 minutes were conducted in the US with children/adolescents with achondroplasia and/or their caregivers. Interviews utilized concept elicitation methodology to explore experiences and priorities for treatment outcomes. Cognitive debriefing methodology explored relevance and understanding of selected COAs. Results: Interviews (N=36) were conducted with caregivers of children age 0–2 years (n=8) and 3–7 years (n=7) and child/caregiver dyads with children age 8–11 years (n=15) and 12–17 years (n=6). Children/caregivers identified pain, short stature, impacts on physical functioning (e.g. reach), and impacts on well-being (e.g. negative attention/comments) as key bothersome aspects of achondroplasia. Caregivers considered an increase in height (n=9/14, 64%) and an improvement in limb proportion (n=11/14, 71%) as successful treatment outcomes. The Childhood Health Assessment Questionnaire (CHAQ) and Quality of Life in Short Stature Youth (QoLISSY-Brief) were cognitively debriefed. CHAQ items evaluating activities, reaching, and hygiene were most relevant. QoLISSY-Brief items evaluating reaching, height bother, being treated differently, and height preventing doing things others could were most relevant. The CHAQ and QoLISSY-Brief instructions, item wording, response scales/options and recall period were well understood by caregivers and adolescents age 12–17. Some children aged 8–11 had difficulty reading, understanding, or required caregiver input. Feedback informed minor amendments to the CHAQ and the addition of a seven-day recall period to the QoLISSY-Brief. These amendments were subsequently reviewed and confirmed in N=12 interviews with caregivers of children age 0–11 (n=9) and adolescents age 12–17 (n=3).Conclusions: Achondroplasia impacts physical functioning and well-being. An increase in height and improvement in limb proportion are considered to be important treatment outcomes, but children/adolescents and their caregivers expect that a successful treatment should also improve important functional outcomes such as reach. The CHAQ (adapted for achondroplasia) and QoLISSY-Brief are relevant and appropriate measures of physical function and emotional/social well-being for pediatric achondroplasia trials; patient-report is recommended for age 12–17 years and caregiver-report is recommended for age 0–11 years.

Comparison of Anchor- and Distribution-Based Methods for Estimating Thresholds of Meaningful Within-Patient Change Using Simulated PROMIS PF 20a Data Under Various Joint Distribution Characteristic Conditions

PurposeTo compare the performance of anchor-based and distribution-based methods for estimating thresholds of meaningful within-patient change of clinical outcome assessments in conditions reflecting data characteristics of small- to medium-sized clinical trials.MethodsData sets were generated from the joint distributions of the PROMIS PF 20a T-score changes and a seven-point global change anchor measure. The 108 simulation conditions (1,000 replications per condition) included combinations of three marginal distributions of T-score changes, three improvement percentages in the anchor measure, four levels of responsiveness correlations, and three sample sizes. Threshold estimation methods included mean change, median change, ROC curve, predictive modeling, half SD, and SEM. Relative bias, precision, accuracy, and measurement significance of the estimates were evaluated based on comparison with true thresholds and IRT-based individual reliable changes of PROMIS scores. Quantile regression models were applied to select and interpret effects of simulation conditions on estimation bias.ResultsWhen PROMIS T-score changes were distributed normally, the predictive modeling method performed best with 50% or more responders identified by the anchor; the mean and median methods were preferred with 30% responders. For skewed distributions, the median method and ROC method gained more advantages. Among the evaluated study conditions, the improvement percentage condition had the most obvious effects on estimation bias.ConclusionTo establish accurate and precise thresholds, clinical researchers are recommended to prioritize study designs with at least 50% anchor-defined responders and strongly responsive target endpoints with highly reliable scoring calibration and to select optimal anchor-based methods given the data characteristics.