scholarly journals Advancing the efficiency and efficacy of patient reported outcomes with multivariate computer adaptive testing

2017 ◽  
Vol 24 (5) ◽  
pp. 897-902 ◽  
Author(s):  
Scott Morris ◽  
Mike Bass ◽  
Mirinae Lee ◽  
Richard E Neapolitan
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Patient Reported Outcomes ◽  
Mean Squared Error ◽  
Adaptive Testing ◽  
Computer Adaptive Testing ◽  
Measurement Information ◽  
Response Theory ◽  
Patient Reported ◽  
Patient Burden

Abstract Objective: The Patient Reported Outcomes Measurement Information System (PROMIS) initiative developed an array of patient reported outcome (PRO) measures. To reduce the number of questions administered, PROMIS utilizes unidimensional item response theory and unidimensional computer adaptive testing (UCAT), which means a separate set of questions is administered for each measured trait. Multidimensional item response theory (MIRT) and multidimensional computer adaptive testing (MCAT) simultaneously assess correlated traits. The objective was to investigate the extent to which MCAT reduces patient burden relative to UCAT in the case of PROs. Methods: One MIRT and 3 unidimensional item response theory models were developed using the related traits anxiety, depression, and anger. Using these models, MCAT and UCAT performance was compared with simulated individuals. Results: Surprisingly, the root mean squared error for both methods increased with the number of items. These results were driven by large errors for individuals with low trait levels. A second analysis focused on individuals aligned with item content. For these individuals, both MCAT and UCAT accuracies improved with additional items. Furthermore, MCAT reduced the test length by 50%. Discussion: For the PROMIS Emotional Distress banks, neither UCAT nor MCAT provided accurate estimates for individuals at low trait levels. Because the items in these banks were designed to detect clinical levels of distress, there is little information for individuals with low trait values. However, trait estimates for individuals targeted by the banks were accurate and MCAT asked substantially fewer questions. Conclusion: By reducing the number of items administered, MCAT can allow clinicians and researchers to assess a wider range of PROs with less patient burden.

scholarly journals The Emergence of Item Response Theory Models and the Patient Reported Outcomes Measurement Information Systems

2016 ◽  
Vol 38 (4) ◽  
Author(s):  
Steven P. Reise
Keyword(s):  
Information Systems ◽  
Item Response Theory ◽  
Item Response ◽  
Patient Reported Outcomes ◽  
Cognitive Domain ◽  
Future Research ◽  
Measurement Information ◽  
Response Theory ◽  
Outcomes Measurement ◽  
Patient Reported

Item response theory (IRT) models emerged to solve practical testing problems in large-scale cognitive achievement and aptitude assessment. Within the last decade, an explosion of IRT applications have occurred in the non-cognitive domain. In this report, I highlight the development, implementation, and results of a single project: Patient Reported Outcomes Measurement Information Systems (PROMIS). The PROMIS projectreflects the state-of-the-art application of IRT in the non-cognitive domain, and has produced important advancements in patient reported outcomes measurement.However, the project also illustrates challenges that confront researchers wishing to apply IRT to non-cognitive constructs. These challenges are: a) selecting a population to set the metric for interpretation of item parameters, b) working with non-normal quasi-continuous latent traits, and c) working with narrow-bandwidth constructs that potentially have a limitedpool of potential indicators. Differences between cognitive and non-cognitive measurement contexts are discussed and directions for future research suggested.

scholarly journals Item Response Theory, Computerized Adaptive Testing, and PROMIS: Assessment of Physical Function

2013 ◽  
Vol 41 (1) ◽  
pp. 153-158 ◽  
Author(s):  
James F. Fries ◽  
James Witter ◽  
Matthias Rose ◽  
David Cella ◽  
Dinesh Khanna ◽  
...  
Keyword(s):  
Item Response Theory ◽  
Physical Function ◽  
Item Response ◽  
Computerized Adaptive Testing ◽  
Short Form ◽  
Adaptive Testing ◽  
Self Report ◽  
Measurement Information ◽  
Response Theory ◽  
Patient Reported

Objective.Patient-reported outcome (PRO) questionnaires record health information directly from research participants because observers may not accurately represent the patient perspective. Patient-reported Outcomes Measurement Information System (PROMIS) is a US National Institutes of Health cooperative group charged with bringing PRO to a new level of precision and standardization across diseases by item development and use of item response theory (IRT).Methods.With IRT methods, improved items are calibrated on an underlying concept to form an item bank for a “domain” such as physical function (PF). The most informative items can be combined to construct efficient “instruments” such as 10-item or 20-item PF static forms. Each item is calibrated on the basis of the probability that a given person will respond at a given level, and the ability of the item to discriminate people from one another. Tailored forms may cover any desired level of the domain being measured. Computerized adaptive testing (CAT) selects the best items to sharpen the estimate of a person’s functional ability, based on prior responses to earlier questions. PROMIS item banks have been improved with experience from several thousand items, and are calibrated on over 21,000 respondents.Results.In areas tested to date, PROMIS PF instruments are superior or equal to Health Assessment Questionnaire and Medical Outcome Study Short Form-36 Survey legacy instruments in clarity, translatability, patient importance, reliability, and sensitivity to change.Conclusion.Precise measures, such as PROMIS, efficiently incorporate patient self-report of health into research, potentially reducing research cost by lowering sample size requirements. The advent of routine IRT applications has the potential to transform PRO measurement.

scholarly journals Using Item Response Theory to Identify Responders to Treatment: Examples with the Patient-Reported Outcomes Measurement Information System (PROMIS®) Physical Function Scale and Emotional Distress Composite

Psychometrika ◽  
2021 ◽  
Author(s):  
Ron D. Hays ◽  
Karen L. Spritzer ◽  
Steven P. Reise
Keyword(s):  
Item Response Theory ◽  
Physical Function ◽  
Item Response ◽  
Emotional Distress ◽  
Classical Test Theory ◽  
Test Theory ◽  
Measurement Information ◽  
Response Theory ◽  
Reliable Change ◽  
Patient Reported

AbstractThe reliable change index has been used to evaluate the significance of individual change in health-related quality of life. We estimate reliable change for two measures (physical function and emotional distress) in the Patient-Reported Outcomes Measurement Information System (PROMIS®) 29-item health-related quality of life measure (PROMIS-29 v2.1). Using two waves of data collected 3 months apart in a longitudinal observational study of chronic low back pain and chronic neck pain patients receiving chiropractic care, and simulations, we compare estimates of reliable change from classical test theory fixed standard errors with item response theory standard errors from the graded response model. We find that unless true change in the PROMIS physical function and emotional distress scales is substantial, classical test theory estimates of significant individual change are much more optimistic than estimates of change based on item response theory.

scholarly journals Using item response theory to address vulnerabilities in FFQ

2017 ◽  
Vol 118 (5) ◽  
pp. 383-391 ◽  
Author(s):  
Josh B. Kazman ◽  
Jonathan M. Scott ◽  
Patricia A. Deuster
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Dietary Habits ◽  
Assessment Tool ◽  
Design Stage ◽  
Measurement Information ◽  
Response Theory ◽  
Us Army ◽  
Item Functioning ◽  
Patient Reported

AbstractThe limitations for self-reporting of dietary patterns are widely recognised as a major vulnerability of FFQ and the dietary screeners/scales derived from FFQ. Such instruments can yield inconsistent results to produce questionable interpretations. The present article discusses the value of psychometric approaches and standards in addressing these drawbacks for instruments used to estimate dietary habits and nutrient intake. We argue that a FFQ or screener that treats diet as a ‘latent construct’ can be optimised for both internal consistency and the value of the research results. Latent constructs, a foundation for item response theory (IRT)-based scales (e.g. Patient Reported Outcomes Measurement Information System) are typically introduced in the design stage of an instrument to elicit critical factors that cannot be observed or measured directly. We propose an iterative approach that uses such modelling to refine FFQ and similar instruments. To that end, we illustrate the benefits of psychometric modelling by using items and data from a sample of 12 370 Soldiers who completed the 2012 US Army Global Assessment Tool (GAT). We used factor analysis to build the scale incorporating five out of eleven survey items. An IRT-driven assessment of response category properties indicates likely problems in the ordering or wording of several response categories. Group comparisons, examined with differential item functioning (DIF), provided evidence of scale validity across each Army sub-population (sex, service component and officer status). Such an approach holds promise for future FFQ.

Sign in / Sign up

Export Citation Format

Share Document