scholarly journals Decision thresholds and minimal important difference estimates for evidence-based practice and policy (Part 2)

2021 ◽  
Vol 47 (1) ◽  
Author(s):  
Beth McDougall ◽  
Mike Reid ◽  
Souvik Mitra ◽  
Bradley Johnston

Understanding core concepts in epidemiology and biostatistics is crucial for evidence-based clinical practice and policy. In this second installment of our two-part series on threshold concepts, we transition from understanding the ubiquitous p-value to tools and measures for decision making among clinicians-in-training, highlighting the growing importance of utilizing explicit and evidence-based approaches to make appropriate and efficient decisions. We review two related decision-making concepts: (1) Minimal Important Difference (MID) estimates and (2) Decision Thresholds, focusing specifically on patient-reported outcome measures (PROMs). These terms and many other related expressions are used regularly, and often interchangeably, but what are they? Why are they valuable? And how can they be used to support evidence-based decision-making in clinical contexts and develop strong clinical practice guidelines? We conclude our brief review on the utility of these measures with a spotlight on a local example of how the theory underlying MID estimates and decision thresholds is currently being embedded in electronic platforms in primary care contexts targeting depression in Nova Scotia.

2019 ◽  
Vol 7 (3_suppl) ◽  
pp. 2325967119S0003
Author(s):  
Jamila N. Aberdeen ◽  
Heather F. Stewart ◽  
Rebecca K. Frank Burnett ◽  
Elliot Greenberg

Purpose: Patient reported outcome measures (PROs) allow physical therapists (PTs) the ability to objectively understand a patients’ perception of their symptoms, functional status, and health related quality of life. Although professional organizations have issued recommendations for PRO use, many of these measures were developed and validated within the adult population which may limit their application to pediatric orthopedics. It is currently unknown which measures are being used within this population, and thus the purpose of this study is to evaluate the current use of PROs among pediatric sports PT and determine how PRO information is used in clinical care. Methods: An online survey, developed in REDCap™, was administered via email to members of the Sports Section Youth Athlete Special Interest Group (YASIG) and the Pediatric Research in Sports Medicine Society (PRiSM), over the course of 8 weeks. The survey was developed and pilot tested by 4 physical therapists and 3 orthopedic surgeons and consisted of 24 possible questions, taking 3-5 minutes to complete. Descriptive statistics and frequency tallies were utilized to analyze the data. Results: There was a total of 70 respondents (response rate YASIG 17%; response rate PRiSM 90%) who completed the questionnaire in its entirety. There was a wide range of clinical experience with 31% reporting 0-5 years, 23%, 6-10 years, 16%, 11-15 years, and 30% >16 years. The majority (54%) reported working in a hospital based outpatient setting or private practice (24%). Ninety four percent (n=66) of respondents reported using PROs, with 100% (n=66) of these subjects issuing them at the initial visit, 94% (n=62) at discharge, and 91% (n=60) monthly. The Neck Disability Index (76%, n=50), Oswestry (76%, n=50), and QuickDASH (68%, n=45) were most frequently used for neck, back and shoulder disorders, respectively. The Lower Extremity Functional Scale (LEFS) was the most widely utilized measure for multiple body regions including 74% (n=49) for either hip or knee dysfunction, and 26% (n=16) for ankle. In general, knee disorders demonstrated the highest degree of variability in scale selection with 52% (n=34) using the IKDC, 35% (n=23) using the Pedi-IKDC and 20% (n=13) (using the KOOS). The information obtained from PROs was used to demonstrate effectiveness of treatment (80%, n=53), inform clinical decisions (77%, n=51), satisfy insurance requirements (59%, n=39), used for goal writing (60%, n=40), and research (36%, n=24). Only 6% (n=4) of PTs indicated that PROs did not impact clinical reasoning within their plan of care. When asked how the information from PROs is used within clinical practice, it was noted that 71% (n=47) of PTs would ‘revise physical therapy goals’ if scores were either higher or lower than expected and 38% (n=25) would ‘refer patient back to the physician’ if PRO results showed lack of progress or regression. In addition, 20% (n=13) of PTs noted they utilize the results from PROs to help inform discharge decision making. When asked regarding barriers to using PROs, PTs reported inadequate time (42%, n=28), difficultly remembering to administer (38%, n=25) and uncertainty regarding which PRO to use (21%, n=14) as the main impediments in using PROs. Conclusions: Our results indicate that the majority of pediatric sports PTs are using PROs to establish efficacy of treatment, inform clinical decision making and set goals. Inadequate time and indecision with regards to which scale to use, were identified as barriers to use. Knee disorders demonstrated the largest variability in scale use. Interestingly, the LEFS was reported at high frequencies for the hip, knee and ankle joints despite the availability of other joint specific measures available. The Oswestry and NDI are also used by 76% of respondents despite neither tool being validated in pediatrics. This finding may be due to the absence of any pediatric specific alternative measures. Clinical Relevance: The Center for Medicare and Medicaid implemented value based purchasing program per the mandate of the Affordable Care Act. The Act established a performance based approach to payment with a goal of ensuring better clinical outcomes and improved patient experience. As a result, there is an increased emphasis on using PROs to demonstrate efficacy and functional improvement. It is reassuring that many PTs are using PROs and using the obtained data to drive clinical care. However, the high variability in scale choice makes comparative outcomes research difficult. In addition, the majority of the PROs used are not validated within the pediatric population and thus may not be appropriate tools for assessing these patients perception of care delivered or even represent their functional/athletic limitations. Given the high prevalence of use and importance to clinical practice, the need for psychometric testing and/or scale development specifically for pediatric sports population is imperative.


2019 ◽  
Author(s):  
Natasha Anne Roberts ◽  
Kimberly Alexander ◽  
David Wyld ◽  
Monika Janda

BACKGROUND Patient-reported outcome measures (PROMs) are tools that enable patients to directly report their own assessments of well-being, or symptoms, in a structured and consistent way. Despite the usefulness of PROMs in optimizing health outcomes, their use in clinical practice is not routine. PROMs are complex to integrate into the clinical setting, with many elements potentially impacting on the success of implementation. For this reason, a protocol has been developed to guide a systematic review to collate information on implementation as presented in the randomized controlled trials (RCTs) to date. OBJECTIVE The primary objective of this systematic review is to identify and synthesize factors available from RCT data about the fidelity of PROM interventions in clinical practice. The secondary objective will be an assessment of how implementation factors impact fidelity outcomes. METHODS Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting standards will be followed. MEDLINE, EMBASE, and the Cumulative Index to Nursing and Allied Health Literature via OvidSP will be accessed using a defined search strategy. Grey literature and ClinicalTrials.gov will be reviewed for unpublished studies. Data extraction will be done to identify fidelity and factors impacting implementation, summarized using a narrative synthesis. An evidence-based implementation science framework will assist in identifying potential elements of importance and their effect on the process and outcomes of implementation. A meta-analysis to assess the impact of implementation factors will be attempted. A Cochrane risk of bias tool will be used. RESULTS This protocol has received funding, and searches of databases will commence at the end of May 2019. It is planned that this systematic review will be finalized for publication in (December) 2019. CONCLUSIONS Applying an implementation science evidence-based framework to the published literature may identify factors present in the data that impact on the implementation of PROMs into routine clinical care. This systematic review aims to improve understanding of how these factors impact the fidelity of this intervention, so that PROMs can be more effectively used in the care of patients. This systematic review can also offer more detailed information about the process and outcomes of successful implementation of PROMs.


10.2196/14579 ◽  
2019 ◽  
Vol 8 (11) ◽  
pp. e14579
Author(s):  
Natasha Anne Roberts ◽  
Kimberly Alexander ◽  
David Wyld ◽  
Monika Janda

Background Patient-reported outcome measures (PROMs) are tools that enable patients to directly report their own assessments of well-being, or symptoms, in a structured and consistent way. Despite the usefulness of PROMs in optimizing health outcomes, their use in clinical practice is not routine. PROMs are complex to integrate into the clinical setting, with many elements potentially impacting on the success of implementation. For this reason, a protocol has been developed to guide a systematic review to collate information on implementation as presented in the randomized controlled trials (RCTs) to date. Objective The primary objective of this systematic review is to identify and synthesize factors available from RCT data about the fidelity of PROM interventions in clinical practice. The secondary objective will be an assessment of how implementation factors impact fidelity outcomes. Methods Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting standards will be followed. MEDLINE, EMBASE, and the Cumulative Index to Nursing and Allied Health Literature via OvidSP will be accessed using a defined search strategy. Grey literature and ClinicalTrials.gov will be reviewed for unpublished studies. Data extraction will be done to identify fidelity and factors impacting implementation, summarized using a narrative synthesis. An evidence-based implementation science framework will assist in identifying potential elements of importance and their effect on the process and outcomes of implementation. A meta-analysis to assess the impact of implementation factors will be attempted. A Cochrane risk of bias tool will be used. Results This protocol has received funding, and searches of databases will commence at the end of May 2019. It is planned that this systematic review will be finalized for publication in (December) 2019. Conclusions Applying an implementation science evidence-based framework to the published literature may identify factors present in the data that impact on the implementation of PROMs into routine clinical care. This systematic review aims to improve understanding of how these factors impact the fidelity of this intervention, so that PROMs can be more effectively used in the care of patients. This systematic review can also offer more detailed information about the process and outcomes of successful implementation of PROMs. International Registered Report Identifier (IRRID) PRR1-10.2196/14579


2017 ◽  
Vol 158 (1) ◽  
pp. 8-15 ◽  
Author(s):  
Thomas L. Carroll ◽  
Stella E. Lee ◽  
Robin Lindsay ◽  
Drew Locandro ◽  
Gregory W. Randolph ◽  
...  

The assessment of patient-reported outcome measures (PROMs) in the outpatient setting is gaining momentum in clinical and research venues. Implementing this data capture into one’s practice, however, is not a one-size-fits-all venture, and it is critical to determine when, how, and where to include these patient-centered assessments. This installment of the “Evidence-Based Medicine in Otolaryngology” series provides insight into the implementation process and experiences with successful incorporation of PROMs into clinical practice. Specifically, 4 differing clinical scenarios and collection techniques are described, including data acquisition protocols, formats for clinician data usage, and applications of PROM results in clinical and research scenarios.


Author(s):  
Titus A. A. Beentjes ◽  
Steven Teerenstra ◽  
Hester Vermeulen ◽  
Peter J. J. Goossens ◽  
Maria W. G. Nijhuis-van der Sanden ◽  
...  

Abstract Purpose Complementary interventions for persons with severe mental illness (SMI) focus on both personal recovery and illness self-management. This paper aimed to identify the patient-reported outcome measures (PROMs) associated with the most relevant and meaningful change in persons with SMI who attended the Illness Management and Recovery Programme (IMR). Methods The effect of the IMR was measured with PROMs concerning recovery, illness self-management, burden of symptoms and quality of life (QoL). From the QoL measures, an anchor was chosen based on the most statistically significant correlations with the PROMs. Then, we estimated the minimal important difference (MID) for all PROMs using an anchor-based method supported by distribution-based methods. The PROM with the highest outcome for effect score divided by MID (the effect/MID index) was considered to be a measure of the most relevant and meaningful change. Results All PROMs showed significant pre–post-effects. The QoL measure ‘General Health Perception (Rand-GHP)’ was identified as the anchor. Based on the anchor method, the Mental Health Recovery Measure (MHRM) showed the highest effect/MID index, which was supported by the distribution-based methods. Because of the modifying gender covariate, we stratified the MID calculations. In most MIDs, the MHRM showed the highest effect/MID indexes. Conclusion Taking into account the low sample size and the gender covariate, we conclude that the MHRM was capable of showing the most relevant and meaningful change as a result of the IMR in persons with SMI.


2020 ◽  
Author(s):  
Titus Beentjes ◽  
Steven Teerenstra ◽  
Hester Vermeulen ◽  
Maria W.G. Nijhuis-van der Sanden ◽  
Betsie G.I. van Gaal ◽  
...  

Abstract Background: Complementary interventions for persons with severe mental illness (SMI) provide broad strategies for recovery and illness self-management. It is not known which outcome measure can be considered to be relevant for persons with SMI. This knowledge can motivate a professional to offer and stimulate a person to participate in that intervention. This paper aimed to identify the outcome measures that determine the most relevant and meaningful change and capture the benefits of a complementary intervention. Methods: By using anchor-based and distribution-based methods, we estimated the minimal important difference (MID) to determine which outcome measure persons improved in beyond the MID to reflect a relevant change in pre-post effect of a complementary intervention, in casu the Illness Management and Recovery programme (IMR).Results: The anchor MID was based on the results of the measure Rand General Health Perception (Rand-GHP). On all MIDs, the Mental Health Recovery Measure (MHRM) had the highest score on the effect compared to its MIDs, and also on all MIDs the MHRM had the highest percentages of participants that scored above the MID. Conclusion: The Rand-GHP is considered to be an excellent measure for investigating the MID as a result of an intervention. The results of our study can be used in shared decision-making processes to determine which intervention is suitable for a person with SMI. A person who desires a recovery outcome, as measured by the MHRM, can be recommended to do the IMR programme.


Sign in / Sign up

Export Citation Format

Share Document