scholarly journals The PROMIS of Better Outcome Assessment: Responsiveness, Floor and Ceiling Effects, and Internet Administration

2011 ◽  
Vol 38 (8) ◽  
pp. 1759-1764 ◽  
Author(s):  
JAMES FRIES ◽  
MATTHIAS ROSE ◽  
ESWAR KRISHNAN

Objective.Use of item response theory (IRT) and, subsequently, computerized adaptive testing (CAT), under the umbrella of the NIH-PROMIS initiative (National Institutes of Health – Patient-Reported Outcomes Measurement Information System), to bring strong new assets to the development of more sensitive, more widely applicable, and more efficiently administered patient-reported outcome (PRO) instruments. We present data on current progress in 3 crucial areas: floor and ceiling effects, responsiveness to change, and interactive computer-based administration over the Internet.Methods.We examined nearly 1000 patients with rheumatoid arthritis and related diseases in a series of studies including a one-year longitudinal examination of detection of change; compared responsiveness of the Legacy SF-36 and HAQ-DI instruments with IRT-based instruments; performed a randomized head-to-head trial of 4 modes of item administration; and simulated the effect of lack of floor and ceiling items upon statistical power and sample sizes.Results.IRT-based PROMIS instruments are more sensitive to change, resulting in the potential to reduce sample size requirements substantially by up to a factor of 4. The modes of administration tested did not differ from each other in any instance by more than one-tenth of a standard deviation. Floor and ceiling effects greatly reduce the number of available subjects, particularly at the ceiling.Conclusion.Failure to adequately address floor and ceiling effects, which determine the range of an instrument, can result in suboptimal assessment of many patients. Improved items, improved instruments, and computer-based administration improve PRO assessment and represent a fundamental advance in clinical outcomes research.

2020 ◽  
Vol 8 (4) ◽  
pp. 232596712091554 ◽  
Author(s):  
Christina J. Hajewski ◽  
Jacqueline E. Baron ◽  
Natalie A. Glass ◽  
Kyle R. Duchman ◽  
Matthew Bollier ◽  
...  

Background: The Patient-Reported Outcome Measurement Information System (PROMIS) was developed to improve patient-reported outcome measures (PROMs) and administration through a computer adaptive test (CAT). The PROMIS physical function-CAT (PF-CAT) has not been investigated in patients with patellofemoral instability (PFI). Purpose/Hypothesis: The purpose of this study was to evaluate the construct validity of the PROMIS PF-CAT with previously validated tools for measuring PROMs in patients with a diagnosis of PFI. We hypothesized that the PF-CAT will have the strongest correlations with other PROMs that evaluated PF as well as moderate correlations with PROMs that measured other health domains. Study Design: Cohort study (diagnosis); Level of evidence, 2. Methods: Patients enrolled in this study who underwent operative intervention for PFI completed the following evaluations preoperatively: PROMIS PF-CAT, 36-Item Short Form Health Survey (SF-36), Knee injury and Osteoarthritis Outcome Score (KOOS), EuroQol-5 dimensions (EQ-5D), and Kujala Anterior Knee Pain Scale (AKPS). Correlation coefficients and the percentage of patients achieving the highest and lowest possible outcome score of each instrument were calculated to assess floor and ceiling effects. Statistical significance was defined as P < .05. Results: In total, 91 participants (63.7% females; mean age, 20.1 ± 7.2 years) completed the questionnaires. PF-CAT had the lowest number of questions (4.3 ± 1.1). The strongest correlations were between the PF-CAT and SF-36 PF subscale ( r = 0.78; P < .01), AKPS ( r = 0.68; P < .01), and KOOS Activities of Daily Living subscale ( r = 0.68; P < .01). Correlation was moderate between the PF-CAT and the KOOS subscales of Sports/Recreation ( r = 0.58; P < .01), Quality of Life ( r = 0.53; P < .01), and Symptoms ( r = 0.47; P < .01). The PROMIS PF-CAT demonstrated no floor or ceiling effects. Conclusion: In patients with PFI, construct validity of the PROMIS PF-CAT was supported by strong correlations demonstrated between the PF-CAT and PROMs evaluating PF and moderate correlations with those assessing other health domains. Our results demonstrated a low respondent burden and no floor or ceiling effects associated with the PROMIS PF-CAT. The PROMIS PF-CAT may be considered a beneficial alternative to previously established PF PROMs for preoperative evaluation of patients with PFI.


2018 ◽  
Vol 25 (13) ◽  
pp. 1791-1799 ◽  
Author(s):  
Brian C Healy ◽  
Jonathan Zurawski ◽  
Cindy T Gonzalez ◽  
Tanuja Chitnis ◽  
Howard L Weiner ◽  
...  

Background: To date, the computerized adaptive testing (CAT) version of the Neuro-quality of life (QOL) has not been assessed in a large sample of people with multiple sclerosis (MS). Objective: The aim of this study was to assess the associations between the CAT version of Neuro-QOL and other clinical and patient-reported outcome measures. Methods: Subjects ( n = 364) enrolled in SysteMS completed the CAT version of the Neuro-QOL and the 36-Item Short Form Survey (SF-36) within 4 weeks of a clinical exam that included the Multiple Sclerosis Functional Composite-4 (MSFC-4). The correlations between the Neuro-QOL domains and the MSFC-4 subscores and the SF-36 scores were calculated. The changes over time in the Neuro-QOL and other measures were also examined. Results: The lower extremity functioning score of the Neuro-QOL showed the highest correlations with MSFC-4 components including Timed 25-Foot Walk, 9-Hole Peg Test, and cognitive score. The expected domains of the Neuro-QOL showed high correlations with the SF-36 subscores, and some Neuro-QOL domains were associated with many SF-36 subscores. There was limited longitudinal change on the Neuro-QOL domains over 12 months, and the change was not associated with change on other measures. Conclusion: The CAT version of the Neuro-QOL shows many of the expected associations with clinical and patient-reported outcome measures.


2018 ◽  
Vol 26 (1) ◽  
pp. 129-140 ◽  
Author(s):  
Heather Taffet Gold ◽  
Raj J Karia ◽  
Alissa Link ◽  
Rachel Lebwohl ◽  
Joseph D Zuckerman ◽  
...  

We integrated and optimized patient-reported outcome measures into the electronic health record to provide quantitative, objective data regarding patients’ health status, which is important for patient care, payer contracts, and research. With a multidisciplinary team from information technology, clinical informatics, population health, and physician champions, we used formal human–computer interaction techniques and user-centered design to integrate several technology platforms and computerized adaptive testing for the National Institutes of Health Patient-Reported Outcomes Measurement Information System. The patient-reported outcome measure system leverages software frequently used by health systems and provides data for research and clinical care via a mobile-responsive web application using Symfony, with REDCap for configuring assessments and de-identified data storage. The system incorporates Oracle databases and Epic flowsheets. Patients complete patient-reported outcome measures, with data viewable in MyChart and Epic Synopsis Reports. Researchers can access data portals. The highly usable, successful patient-reported outcome measures platform is acceptable to patients and clinicians and achieved 73 percent overall completion rates.


2019 ◽  
Vol 101-B (7) ◽  
pp. 768-778 ◽  
Author(s):  
V. P. Galea ◽  
P. Rojanasopondist ◽  
L. H. Ingelsrud ◽  
H. E. Rubash ◽  
C. Bragdon ◽  
...  

AimsThe primary aim of this study was to quantify the improvement in patient-reported outcome measures (PROMs) following total hip arthroplasty (THA), as well as the extent of any deterioration through the seven-year follow-up. The secondary aim was to identify predictors of PROM improvement and deterioration.Patients and MethodsA total of 976 patients were enrolled into a prospective, international, multicentre study. Patients completed a battery of PROMs prior to THA, at three months post-THA, and at one, three, five, and seven-years post-THA. The Harris Hip Score (HHS), the 36-Item Short-Form Health Survey (SF-36) Physical Component Summary (PCS), the SF-36 Mental Component Summary (MCS), and the EuroQol five-dimension three-level (EQ-5D) index were the primary outcomes. Longitudinal changes in each PROM were investigated by piece-wise linear mixed effects models. Clinically significant deterioration was defined for each patient as a decrease of one half of a standard deviation (group baseline).ResultsImprovements were noted in each PROM between the preoperative and one-year visits, with one-year values exceeding age-matched population norms. Patients with difficulty in self-care experienced less improvement in HHS (odds ratio (OR) 2.2; p = 0.003). Those with anxiety/depression experienced less improvement in PCS (OR -3.3; p = 0.002) and EQ-5D (OR -0.07; p = 0.005). Between one and seven years, obesity was associated with deterioration in HHS (1.5 points/year; p = 0.006), PCS (0.8 points/year; p < 0.001), and EQ-5D (0.02 points/year; p < 0.001). Preoperative difficulty in self-care was associated with deterioration in HHS (2.2 points/year; p < 0.001). Preoperative pain from other joints was associated with deterioration in MCS (0.8 points/year; p < 0.001). All aforementioned factors were associated with clinically significant deterioration in PROMs (p < 0.035), except anxiety/depression with regard to PCS (p = 0.060).ConclusionThe present study finds that patient factors affect the improvement and deterioration in PROMs over the medium term following THA. Special attention should be given to patients with risk factors for decreased PROMs, both preoperatively and during follow-up. Cite this article: Bone Joint J 2019;101-B:768–778.


2018 ◽  
Vol 47 (1) ◽  
pp. 181-188 ◽  
Author(s):  
Allen D. Nicholson ◽  
Hafiz F. Kassam ◽  
Steven D. Pan ◽  
Jacob E. Berman ◽  
Theodore A. Blaine ◽  
...  

Background: The Patient-Reported Outcomes Measurement Information System (PROMIS) Global-10 was recently developed to assess physical and mental health and provide an estimated EuroQol-5 Dimension (EQ-5D) score. This instrument needs to be validated for specific patient cohorts such as those with rotator cuff pathology. Hypothesis: There is moderate to high correlation between the PROMIS Global-10 and legacy patient-reported outcome measures; PROMIS Global-10 will not show ceiling effects; and estimated EQ-5D scores will show good correlation and low variance with actual EQ-5D scores. Study Design: Cohort study (diagnosis); Level of evidence, 2. Methods: A total of 323 patients with rotator cuff disease were prospectively enrolled before treatment. Each patient completed the PROMIS Global-10, EQ-5D, American Shoulder and Elbow Surgeons (ASES) shoulder assessment form, and Single Assessment Numeric Evaluation (SANE), and those with known rotator cuff tears completed the Western Ontario Rotator Cuff Index (WORC). Spearman correlations were calculated. Bland-Altman agreement tests were conducted between estimated EQ-5D scores from the PROMIS and actual EQ-5D scores. Ceiling and floor effects were assessed, defined as ≥15% respondents with highest or lowest possible score. Results: Correlation between the PROMIS Global-10 and EQ-5D was excellent (0.70, P < .0001). Correlation of the PROMIS physical scores was excellent-good with the ASES (0.62, P < .0001), good with the WORC (0.47, P < .0001), and good with the SANE (0.41, P < .0005). Correlation between the PROMIS mental scores was poor with the ASES (0.34, P < .0001), the WORC (0.32, P = .0016), and the SANE (0.24, P < .0001). No floor or ceiling effects were found. Agreement analysis showed substantial variance in individual scores, despite the overall similarity in mean scores between the estimated and actual EQ-5D scores, indicating poor agreement. Bland-Altman 95% limits of agreement for estimated EQ-5D scores ranged from 34% below to 31% above actual EQ-5D scores. Conclusion: Physical function scores of the PROMIS Global-10 show high correlation with legacy patient-reported outcome instruments, suggesting that it is a reliable tool for outcome assessment in a population with rotator cuff pathology. The large variability in 95% limit of agreement suggested that the estimated EQ-5D scores from the PROMIS Global-10 cannot replace traditional EQ-5D scores.


2020 ◽  
Vol 5 (4) ◽  
pp. 2473011420S0006
Author(s):  
Devon C. Nixon ◽  
Chong Zhang ◽  
Maxwell Weinberg ◽  
Angela Presson ◽  
Florian Nickisch

Category: Other Introduction/Purpose: Patient satisfaction has garnered interest as a tool to measure healthcare quality. The Press Ganey (PG) instrument was designed to assess outpatient patient satisfaction by probing multiple domains of the patient experience including clinic wait-times and overall assessment of the provider practice. However, orthopaedic studies in total joint arthroplasty, spine, and hand patients have offered conflicting relationships between PG satisfaction metrics and patient-reported outcome (PRO) measures. No prior study has assessed the relationship between PG and PROs in foot and ankle patients. Here, we tested if there was an association between PROMIS outcomes and PG satisfaction scores. Methods: Between 2015 and 2017, new patient visits to orthopedic foot and ankle clinics at a single academic center were retrospectively analyzed. Patients that completed PG satisfaction surveys were included for study. All patients who completed PG surveys had completed either one or both of the PROMIS physical function (PF) or pain interference (PI) metrics administered by computerized adaptive testing. Negative binomial regressions were used to compare PRO scores to PG overall satisfaction and PG satisfaction with care provider, adjusting for patient characteristics. Results were reported as dissatisfaction score ratios along with 95% confidence intervals (CIs) and P-values. Results: Out of 3,984 new patient visits, only 441 completed the PG survey (11.3% response rate). Ceiling effects were seen with PG data as 64% of our patients reported perfect satisfaction with care provider and 27% had perfect overall satisfaction. Higher function on the PROMIS PF was weakly associated with increased overall satisfaction (Ratio=0.82, 95% CI: 0.68-0.99, P=0.039) and increased satisfaction with care provider (Ratio=0.60, 95% CI: 0.40-0.92, P=0.019). However, pain (PROMIS PI) was not associated with overall satisfaction or with satisfaction with care provider. Conclusion: Patient satisfaction was weakly related to patient-reported function but not pain among new foot and ankle clinic patients. It appears from our data that PG and PROMIS measure differing components of patient care. Further, our findings showcase the limitations of the PG tool including low response rates and notable ceiling effects. Until we achieve a broader understanding of the relationship between patient satisfaction and clinical and surgical outcomes, we would caution against the use of PG data in the assessment of quality of care and reimbursement for orthopaedic foot and ankle surgery. [Table: see text]


2019 ◽  
Vol 7 (7) ◽  
pp. 232596711985500 ◽  
Author(s):  
Christopher N. Carender ◽  
Matthew J. Bollier ◽  
Brian R. Wolf ◽  
Kyle R. Duchman ◽  
Qiang An ◽  
...  

Background: Use of the Patient-Reported Outcomes Measurement Information System (PROMIS) instrument has not yet been validated in patients undergoing operative treatment for patellofemoral malalignment and chondral disease. Purpose: To evaluate the PROMIS Physical Function Computer Adaptive Testing (PF CAT) instrument in a population of patients with patellofemoral malalignment and chondral disease relative to established patient-reported outcome (PRO) instruments. Study Design: Cohort study (diagnosis); Level of evidence, 2. Methods: Eligible patients were prospectively enrolled at the time of indication for surgery and completed 5 PRO instruments preoperatively: 36-Item Short Form Health Survey (SF-36); Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC); Marx activity rating scale (Marx); EuroQol 5-dimension, 5-level instrument (EQ-5D-5L); and PROMIS PF CAT. Convergent and discriminant validity was assessed by measuring correlations between PROMIS PF CAT and other PRO instruments, including the Knee injury and Osteoarthritis Outcome Score (KOOS). Strength of correlation was measured by use of Spearman correlation coefficients. Results: In total, 37 patients (40 knees) were enrolled in the study. All knees underwent Fulkerson osteotomy and concomitant cartilage procedure (29 chondroplasty, 6 allograft, 5 microfracture). Mean patient age was 33.4 years, and 73% of knees were in female patients. Correlations ( r) between PROMIS PF CAT and knee PRO instruments were as follows: SF-36 PF ( r = 0.80; P < .01); KOOS Pain ( r = 0.74; P < .01); KOOS Symptoms ( r = 0.47; P < .01); KOOS Quality of Life ( r = 0.68; P < .01); KOOS Sports and Recreation ( r = 0.72; P < .01); KOOS Activities of Daily Living (ADL) ( r = 0.80; P < .01); WOMAC Function ( r = 0.80; P < .01); WOMAC Pain ( r = 0.72; P < .01); WOMAC Stiffness ( r = 0.38; P = .02); Marx ( r = 0.22; P = .31); and EQ-5D-5L ( r = 0.72; P < .01). Neither floor nor ceiling effects were observed in PROMIS PF CAT or KOOS ADL. Mean (±SD) question burden with PROMIS PF CAT was 5.6 ± 0.6 questions. Conclusion: In patients with patellofemoral malalignment and chondral disease, PROMIS PF CAT is an efficient and reliable PRO instrument to preoperatively assess patients across a spectrum of knee function without floor or ceiling effects.


2016 ◽  
Vol 45 (2) ◽  
pp. 449-453 ◽  
Author(s):  
Chris A. Anthony ◽  
Natalie A. Glass ◽  
Kyle Hancock ◽  
Matt Bollier ◽  
Brian R. Wolf ◽  
...  

Background: Shoulder instability is a relatively common condition occurring in 2% of the population. PROMIS (Patient-Reported Outcome Measurement Information System) was developed by the National Institutes of Health in an effort to advance patient-reported outcome (PRO) instruments by developing question banks for major health domains. Purpose: To compare PROMIS instruments to current PRO instruments in patients who would be undergoing operative intervention for recurrent shoulder instability. Study Design: Cohort study (diagnosis); Level of evidence, 2. Methods: A total of 74 patients with a primary diagnosis of shoulder instability who would be undergoing surgery were asked to fill out the American Shoulder and Elbow Surgeons shoulder assessment form (ASES), Marx shoulder activity scale (Marx), Short Form–36 Health Survey Physical Function subscale (SF-36 PF), Western Ontario Shoulder Instability Index (WOSI), PROMIS physical function computer adaptive test (PF CAT), and PROMIS upper extremity item bank (UE). Correlation between PRO instruments was defined as excellent (>0.7), excellent-good (0.61-0.7), good (0.4-0.6), and poor (0.2-0.3). Results: Utilization of the PROMIS UE demonstrated excellent correlation with the SF-36 PF ( r = 0.78, P < .01) and ASES ( r = 0.71, P < .01); there was excellent-good correlation with the EQ-5D ( r = 0.66, P < .01), WOSI ( r = 0.63, P < .01), and PROMIS PF CAT ( r = 0.63, P < .01). Utilization of the PROMIS PF CAT demonstrated excellent correlation with the SF-36 PF ( r = 0.72, P < .01); there was excellent-good correlation with the ASES ( r = 0.67, P < .01) and PROMIS UE ( r = 0.63, P < .01). When utilizing the PROMIS UE, ceiling effects were present in 28.6% of patients aged 18 to 21 years. Patients, on average, answered 4.6 ± 1.8 questions utilizing the PROMIS PF CAT. Conclusion: The PROMIS UE and PROMIS PF CAT demonstrated good to excellent correlation with common shoulder and upper extremity PRO instruments as well as the SF-36 PF in patients with shoulder instability. In patients aged ≤21 years, there were significant ceiling effects utilizing the PROMIS UE. While the PROMIS PF CAT appears appropriate for use in adults of any age, our findings demonstrate that the PROMIS UE has significant ceiling effects in patients with shoulder instability who are ≤21 years old, and we do not recommend use of the PROMIS UE in this population.


2017 ◽  
Vol 2 (3) ◽  
pp. 2473011417S0000
Author(s):  
Adam Bennett ◽  
Malik Siddique ◽  
Jayasree Ramas Ramaskandhan

Category: Ankle Arthritis Introduction/Purpose: Pilon fractures of the distal tibia are high energy injuries often occurring in a young patient cohort. Despite optimal acute fracture management, post traumatic osteoarthritis of the tibiotalar joint is a common corollary. Standard treatment is by way of tibiotalar arthrodesis, however the lack of motion at the ankle afforded by this treatment is often poorly tolerated by patients and can lead to arthrosis of adjacent joints. Total ankle replacement (TAR) offers an attractive pain relieving and motion preserving option for this patient subgroup. Here we report the two-year patient reported outcome measures for a cohort of patients undergoing TAR for osteoarthritis secondary to tibial pilon fracture and compare them to the outcomes for other indications for total ankle replacement. Methods: The results of patient-reported outcome measures (PROMs) for TAR performed between March 2006 and November 2014 by a single surgeon at a single institution in the United Kingdom were reviewed. Data collected included Foot and Ankle Outcome Scores (FAOS) (WOMAC pain/stiffness/function); SF-36 General Health Questionnaire; number of comorbidities; self- reported BMI and patient satisfaction scores. Time points for data collection were pre-operatively and then at one and two years post-operatively. Clinical notes and radiographs were reviewed to highlight the indication for TAR and were categorised into the following sub-groups: osteoarthritis (OA); rheumatoid arthritis (RA); arthritis following pilon fracture; arthritis following ankle fracture; and post-traumatic arthritis without previous fracture (PTOA). FAOS and SF-36 were analysed using a general linear model to assess variance and by way of repeated measures ANOVA. Demographic data was assessed using student t-test and chi- squared analysis. Patient satisfaction scores were evaluated using cross-tabulation and chi-squared examination. Results: The pilon fracture group had the youngest average age and highest BMI (56.5 years; mean BMI 31.6). There was no statistically significant difference in number of reported comorbidities between subgroups (p>0.05). Significant improvement in FAOS pain, function and stiffness scores was seen in all subgroups from pre-operatively to one year post-operatively (p=0.01, 0.05 and 0.03 respectively). No further statistically significant improvement in the same scores was seen at two years. SF-36 analysis demonstrated similar results for all subgroups. There was general improvement in all parameters except for general health (p=0.890 at one year) and reduction in physical function in the RA group from first to second post-operative years (p=0.046). Patient satisfaction survey showed similar results for all subgroups. Conclusion: Our study has demonstrated that statistically significant improvement is seen in FAOS pain, function and stiffness scores in all subgroups of patients undergoing TAR by one year post surgery and that this improvement is maintained by two year follow up. There was no major difference in any of the reported outcomes between subgroups by two years post-operatively, suggesting that TAR performed for arthritis secondary to pilon fracture is a realistic alternative to tibio-talar arthrodesis.


Author(s):  
Trevor R. Gulbrandsen ◽  
Zain M. Khazi ◽  
Matthew Bollier ◽  
Brian Wolf ◽  
Christopher Larson ◽  
...  

AbstractThe purpose of this study was to establish preoperative validity of the Patient-Reported Outcomes Measurement Information System physical function computer adaptive test (PROMIS PF-CT) with legacy patient-reported outcome measures (PROMs) for meniscal root tears (MRTs). Our study included 51 patients (52 knees) with MRT. Patients completed PROMIS PF-CT, Short Form 36 (SF-36 physical function, pain, general health, vitality, social function, emotional well-being, role limitations due to physical health, and role limitations due to emotional problems), Knee Injury and Osteoarthritis Outcome Score (KOOS pain, symptoms, activities of daily living [ADLs], sports, and quality of life [QOL]), Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC pain, stiffness, and function), EuroQol-5 dimensions (EQ-5D), and Knee Activity Scale questionnaires at their preoperative visit. Correlations between the PROMs listed above were evaluated along with floor and ceiling effects. Correlations were defined as weak (0.2–0.39), moderate (0.4–0.59), strong (0.6–0.79), and very strong (0.8–1.0). Preoperative data showed that PROMIS PF-CT has a strong correlation with SF-36 PF, KOOS-ADL, WOMAC-function, and EQ-5D; and moderate correlation with KOOS-sport, KOOS-pain, KOOS-symptoms, KOOS-QOL, WOMAC-pain, and WOMAC-stiffness. The Knee Activity Scale did not show any significant correlation with PROMIS PF-CT (r = 0.12, p = 0.2080). Of all the PROMs administered, PROMIS PF-CT demonstrated no floor or ceiling effects compared with 11.54% ceiling effect in KOOS-sports, and 5.77% floor effect in KOOS-ADL. On average, patients answered fewer PROMIS PF-CT questions (4.15 ± 0.72). PROMIS PF-CT is a valuable tool to assess preoperative patient-reported physical function in patients that may undergo MRT repair. It correlates strongly with other well-established PROMs. It also demonstrated no floor or ceiling effects and demonstrated a low test burden in our sample of 52 knees. This is a level III, prognostic retrospective comparative study.


Sign in / Sign up

Export Citation Format

Share Document