scholarly journals ICF Linking and Cognitive Interviewing Are Complementary Methods for Optimizing Content Validity of Outcome Measures: An Integrated Methods Review

2021 ◽  
Vol 2 ◽  
Author(s):  
Joy C. MacDermid

Content validity is a fundamental requirement of outcome measures. After reviewing operational needs and existing definitions, content validity we as defined as: the extent to which a measure provides a comprehensive and true assessment of the key relevant elements of a specified construct or attribute across a defined range, clearly and equitably for a stated target audience and context. ICF linkage rules from 2002, 2005, and 2019 have provide increasingly clear processes for describing and evaluating content of outcome measures. ICF Core Sets provide international reference standards of the core constructs of importance for different health conditions. Both are important as reference standards during content validation. To summarize their use as reference standards, the following summary indicators were proposed: (1) Measure to ICF linkage, (2) Measure to (Brief or Comprehensive) Core Set Absolute Linkage, (3) Measure to (Brief or Comprehensive) Core Set Unique Linkage, (4) Core Set Representation, and (5) Core Set Unique Disability Representation. Methods to assess how respondents engage with content are needed to complement ICF-linking. Cognitive interviewing is an ideal method since it used to explore how respondents interpret and calibrate response to individual items on an outcome measure. We proposed a framework for classifying these responses: Clarity/Comprehension, Relevance, Inadequate response definition, Reference Point, Perspective modification, and Calibration Across Items. Our analysis of 24 manuscripts that used ICF linking for content validation since updated linking rules were published found that authors typically used linking to validate existing measures, involved multiple raters, used 2005 linking rules, summarized content at a concept level (e.g., impairment, activity, participation) and/or use core sets as a reference standard. Infrequently, ICF linking was used to create item pools/conceptual frameworks for new measures, applied the full scope of the 2019 linking rules, used summary indicators, or integrated ICF-linking with qualitative methods like cognitive interviews. We conclude that ICF linkage is a powerful tool for content validity during development or validation of PROM. Best practices include use of updated ICF linking rules, triangulation of ICF linking with participant assessments of clarity and relevance preferably obtained using cognitive interview methods, and application of defined summary indicators.

2016 ◽  
Vol 33 (S1) ◽  
pp. S365-S365
Author(s):  
O. Pino ◽  
G. Guilera ◽  
E. Rojo ◽  
J. Gómez-Benito

ObjectiveThe aim this presentation is present the results of the preparatory studies were presented at an international consensus conference, a multi-stage, iterative, decision-making and consensus process that took place 12–14 May 2015 in Barcelona, Spain. At this consensus conference, schizophrenia experts from different countries worldwide and working in a broad range of professions decided which ICF categories should be included in the first version of the ICF Core Sets for schizophrenia.MethodFour preliminary studies intend to capture the researcher's perspective, the patient's perspective, the expert's perspective and the clinician's perspective, respectively, on the most relevant aspects of functioning of persons living with schizophrenia. The final definition of ICF Core Sets for schizophrenia have been determined by integrating the results of preliminary studies in a consensus conference with international expert.ResultThe experts included 97 categories in the Comprehensive ICF Core Set and 25 categories in the Brief ICF-CS. The specific categories of each ICF-CS are shown in this presentation. The Comprehensive ICF-CS can guide multidisciplinary assessments of functioning in persons with schizophrenia, and the brief version is ideal for use in both clinical and epidemiological research, since it includes a small and practical number of categories, but sufficiently wide for finding utility in clinical assessments.ConclusionICF-CS are being designed with the goal of providing useful standards for research, clinical practice and teaching, and it will stimulate research and will improve understanding of functioning, health and environmental factors in schizophrenia.Disclosure of interestThe authors have not supplied their declaration of competing interest.


2021 ◽  
Vol 44 (2) ◽  
pp. 159-165
Author(s):  
Anupa Pathak ◽  
Haxby J Abbott ◽  
Saurab Sharma ◽  
Daniel Cury Ribeiro ◽  
Allen W. Heinemann

2009 ◽  
Vol 69 (01) ◽  
pp. 102-107 ◽  
Author(s):  
A Boonen ◽  
J Braun ◽  
I E van der Horst Bruinsma ◽  
F Huang ◽  
W Maksymowych ◽  
...  

Objective:To report on the results of a standardised consensus process agreeing on concepts typical and/or relevant when classifying functioning and health in patients with ankylosing spondylitis (AS) based on the International Classification of Functioning and Health (ICF).Methods:Experts in AS from different professional and geographical backgrounds attended a consensus conference and were divided into three working groups. Rheumatologists were selected from members of the Assessment of SpondyloArthritis international Society (ASAS). Other health professionals were recommended by ASAS members. The aim was to compose three working groups with five to seven participants to allow everybody’s contribution in the discussions. Experts selected ICF categories that were considered typical and/or relevant for AS during a standardised consensus process by integrating evidence from preceding studies in alternating working group and plenary discussions. A Comprehensive ICF Core Set was selected for the comprehensive classification of functioning and a Brief ICF Core Set for application in trials.Results:The conference was attended by 19 experts from 12 countries. Eighty categories were included in the Comprehensive Core Set, which included 23 Body functions, 19 Body structures, 24 Activities and participation and 14 Environmental factors. Nineteen categories were selected for the Brief Core Set, which included 6 Body functions, 4 Body structures, 7 Activities and participation and 2 Environmental factors.Conclusion:The Comprehensive and Brief ICF Core Sets for AS are now available and aim to represent the external reference to define consequences of AS on functioning.


2020 ◽  
Vol 18 (1) ◽  
Author(s):  
Armaghan Dabbagh ◽  
Joy C. MacDermid ◽  
Tara L. Packham ◽  
Luciana G. Macedo

Abstract Background Accurate diagnosis of carpal tunnel syndrome (CTS) is essential for directing appropriate treatment; and for making decisions about work injury claims. The Kamath and Stothard Questionnaire (KSQ) is a self-reported tool used for the diagnosis of CTS. Comprehensibility and comprehensiveness of this questionnaire are critical to diagnostic performance and need to be established. The purpose of the study was to describe how potential respondents, clinicians, and measurement researchers interpret KSQ questions in order to identify and resolve potential sources of misclassification. Methods Hand therapists, measurement researchers, participants with CTS, and a control group were interviewed using cognitive interviewing techniques (talk aloud, semi-structured interview probes) in Hamilton, Canada. All interviews were recorded and transcribed verbatim. A directed content analysis was done to analyze the interviews using a previously established framework. Findings Eighteen participants were interviewed. Areas, where questions were unclear to some participants, were recorded and categorized into five themes: Clarity and Comprehension (52%), Relativeness (38%), Inadequate Response Definition (4%), Perspective Modifiers (4%), and Reference Point (2%). Respondents also identified several symptoms of CTS that are not covered by the KSQ that might be of diagnostic value, e.g., weakness and dropping items. Conclusion The content validity of the current iteration of the KSQ was not established. The problematic questions identified in the study have been reported to have low specificity and negative predictive values in a previous quantitative study. The content validity issues identified may explain the poor performance. Recommendations were made to modify the wording of the KSQ and the potential addition of three new questions. Future studies should determine whether the modified questionnaire can provide better diagnostic accuracy and psychometric properties. The results of this study may assist in ruling in or out CTS diagnosis to a wide variety of target audience, such as hand specialists, physical and occupational therapists, as well as family doctors.


PLoS ONE ◽  
2021 ◽  
Vol 16 (12) ◽  
pp. e0261413
Author(s):  
Abinethaa Paramasivam ◽  
Atul Jaiswal ◽  
Renu Minhas ◽  
Peter Holzhey ◽  
Karen Keyes ◽  
...  

Background Individuals with deafblindness experience a combination of hearing and vision impairments. The World Health Organization has developed a global framework referred to as the International Classification of Functioning, Disability and Health (ICF) to describe health and functioning. From the full ICF classification, a selection of categories, referred to as ICF Core Sets, provide users with a tool to describe functioning and disability in specific health conditions. There has been no ICF Core Set created for deafblindness. Given that core sets are instrumental in improving clinical practice, research, and service delivery, the aim of this study is to develop an ICF Core Set for deafblindness. Methods As part of the preparatory phase in the ICF Core Set development, there are four studies that will be conducted. This includes the [1] systematic literature review that examines the researcher’s perspective, [2] qualitative study focusing on the individuals with deafblindness experience, [3] experts survey that looks at health professional’s perspective, and [4] empirical study that examines the clinical perspective. The studies will be conducted using the principles outlined by the ICF Research Branch for the development of ICF Core Sets. The systematic literature review protocol was submitted for registration on PROSPERO CRD42021247952. Discussion An ICF Core Set created for deafblindness will benefit individuals living with deafblindness who are often excluded from social participation, policies, and services. An ICF Core Set for deafblindness will have a significant impact on healthcare professionals, policymakers, researchers, service providers and individuals with deafblindness by facilitating communication among all stakeholder to support the functioning of those with deafblindness.


Author(s):  
Johan H. Sengers ◽  
Femke I. Abma ◽  
Loes Wilming ◽  
Pepijn D. D. M. Roelofs ◽  
Yvonne F. Heerkens ◽  
...  

2020 ◽  
Author(s):  
Rochelle Furtado ◽  
Joy C MacDermid ◽  
Dianne Bryant ◽  
Kenneth J Faber ◽  
George S Athwal

Abstract Background: The shortened version of the Western Ontario Rotator Cuff Index (Short-WORC) is a patient reported outcome measure that evaluates quality of life (QoL) of patients with rotator cuff pathology. However, formal content validation of the full or Short-WORC has not been reported. This study aims to understand how 1) people interpret and calibrate responses to items on the Short-WORC and 2) compensatory strategies that might enhance function and thereby affect responses. Methods: This study uses cognitive interviewing, a qualitative methodology that focuses on the interpretation of questionnaire items. Patients with rotator cuff disorders (n=10), clinicians (n=6) and measurement researchers (n=10) were interviewed using a talk aloud structured interview that evaluated each of the 7 items of the Short-WORC. All interviews were recorded and transcribed verbatim by one researcher (R.F). Analysis was done through an open coding scheme using a previously established framework. Results: Overall, the items on the Short-WORC were well received by participants. Through the interviews, the 6 themes of: Comprehension, Inadequate response definition, Reference Point, Relevance, Perspective Modifiers and Calibration Across Items emerged. The items of working above the shoulder (90%), compensating with the unaffected arm (88%) and lifting heavy objects (92%) were the most relevant to participants. Participants calibrated their scores on the items of sleeping and styling (19%) the most. Perspective modifiers of gender, influenced the calibrations of items of styling your hair (30%) and dressing or undressing (19%). Compensatory strategies of task-re allocation and using assistive devices/resources were frequently mentioned by participants. Overall, participants had minor comprehension issues, but found the 7- items of the Short-WORC to be relevant to QoL. Conclusions: Therefore, the findings demonstrate that the Short-WORC is not cognitively complex, but varies with patient perspectives. Overall, the Short-WORC provides evidence of demonstrating strong content validity when used for rotator cuff disorder patients.


Sign in / Sign up

Export Citation Format

Share Document