scholarly journals Questionnaire validation practice within a theoretical framework: a systematic descriptive literature review of health literacy assessments

BMJ Open ◽  
2020 ◽  
Vol 10 (6) ◽  
pp. e035974
Author(s):  
Melanie Hawkins ◽  
Gerald R Elsworth ◽  
Elizabeth Hoban ◽  
Richard H Osborne
Keyword(s):  
Health Literacy ◽  
Literature Review ◽  
Health Equity ◽  
Theoretical Framework ◽  
Evidence Based ◽  
Validity Evidence ◽  
Validity Testing ◽  
Testing Framework ◽  
Literacy Assessments ◽  
Health Assessments

ObjectiveValidity refers to the extent to which evidence and theory support the adequacy and appropriateness of inferences based on score interpretations. The health sector is lacking a theoretically-driven framework for the development, testing and use of health assessments. This study used the Standards for Educational and Psychological Testing framework of five sources of validity evidence to assess the types of evidence reported for health literacy assessments, and to identify studies that referred to a theoretical validity testing framework.MethodsA systematic descriptive literature review investigated methods and results in health literacy assessment development, application and validity testing studies. Electronic searches were conducted in EBSCOhost, Embase, Open Access Theses and Dissertations and ProQuest Dissertations. Data were coded to the Standards’ five sources of validity evidence, and for reference to a validity testing framework.ResultsCoding on 46 studies resulted in 195 instances of validity evidence across the five sources. Only nine studies directly or indirectly referenced a validity testing framework. Evidence based on relations to other variables is most frequently reported.ConclusionsThe health and health equity of individuals and populations are increasingly dependent on decisions based on data collected through health assessments. An evidence-based theoretical framework provides structure and coherence to existing evidence and stipulates where further evidence is required to evaluate the extent to which data are valid for an intended purpose. This review demonstrates the use of the Standards’ theoretical validity testing framework to evaluate sources of evidence reported for health literacy assessments. Findings indicate that theoretical validity testing frameworks are rarely used to collate and evaluate evidence in validation practice for health literacy assessments. Use of the Standards’ theoretical validity testing framework would improve evaluation of the evidence for inferences derived from health assessment data on which public health and health equity decisions are based.

scholarly journals Questionnaire validation practice: a protocol for a systematic descriptive literature review of health literacy assessments

BMJ Open ◽  
2019 ◽  
Vol 9 (10) ◽  
pp. e030753 ◽  
Author(s):  
Melanie Hawkins ◽  
Gerald R Elsworth ◽  
Richard H Osborne
Keyword(s):  
Health Literacy ◽  
Literature Review ◽  
Quantitative Research ◽  
Theoretical Framework ◽  
Limited Range ◽  
Research Area ◽  
Validity Evidence ◽  
Theoretical Frameworks ◽  
Validity Testing ◽  
Literacy Assessments

IntroductionContemporary validity testing theory holds that validity lies in the extent to which a proposed interpretation and use of test scores is justified, the evidence for which is dependent on both quantitative and qualitative research methods. Despite this, we hypothesise that development and validation studies for assessments in the field of health primarily report a limited range of statistical properties, and that a systematic theoretical framework for validity testing is rarely applied. Using health literacy assessments as an exemplar, this paper outlines a protocol for a systematic descriptive literature review about types of validity evidence being reported and if the evidence is reported within a theoretical framework.Methods and analysisA systematic descriptive literature review of qualitative and quantitative research will be used to investigate the scope of validation practice in the rapidly growing field of health literacy assessment. This review method employs a frequency analysis to reveal potentially interpretable patterns of phenomena in a research area; in this study, patterns in types of validity evidence reported, as assessed against the criteria of the 2014Standards for Educational and Psychological Testing, and in the number of studies using a theoretical validity testing framework. The search process will be consistent with the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement. Outcomes of the review will describe patterns in reported validity evidence, methods used to generate the evidence and theoretical frameworks underpinning validation practice and claims. This review will inform a theoretical basis for future development and validity testing of health assessments in general.Ethics and disseminationEthics approval is not required for this systematic review because only published research will be examined. Dissemination of the review findings will be through publication in a peer-reviewed journal, at conference presentations and in the lead author’s doctoral thesis.

scholarly journals Validity arguments for patient-reported outcomes: justifying the intended interpretation and use of data

2021 ◽  
Vol 5 (1) ◽  
Author(s):  
Melanie Hawkins ◽  
Gerald R. Elsworth ◽  
Sandra Nolte ◽  
Richard H. Osborne
Keyword(s):  
Literature Review ◽  
Patient Reported Outcomes ◽  
Measurement Instrument ◽  
Self Report ◽  
Validity Evidence ◽  
Measurement Instruments ◽  
Validity Testing ◽  
Patient Reported ◽  
Intended Interpretation ◽  
Interpretive Argument

Abstract Background Contrary to common usage in the health sciences, the term “valid” refers not to the properties of a measurement instrument but to the extent to which data-derived inferences are appropriate, meaningful, and useful for intended decision making. The aim of this study was to determine how validity testing theory (the Standards for Educational and Psychological Testing) and methodology (Kane’s argument-based approach to validation) from education and psychology can be applied to validation practices for patient-reported outcomes that are measured by instruments that assess theoretical constructs in health. Methods The Health Literacy Questionnaire (HLQ) was used as an example of a theory-based self-report assessment for the purposes of this study. Kane’s five inferences (scoring, generalisation, extrapolation, theory-based interpretation, and implications) for theoretical constructs were applied to the general interpretive argument for the HLQ. Existing validity evidence for the HLQ was identified and collated (as per the Standards recommendation) through a literature review and mapped to the five inferences. Evaluation of the evidence was not within the scope of this study. Results The general HLQ interpretive argument was built to demonstrate Kane’s five inferences (and associated warrants and assumptions) for theoretical constructs, and which connect raw data to the intended interpretation and use of the data. The literature review identified 11 HLQ articles from which 57 sources of validity evidence were extracted and mapped to the general interpretive argument. Conclusions Kane’s five inferences and associated warrants and assumptions were demonstrated in relation to the HLQ. However, the process developed in this study is likely to be suitable for validation planning for other measurement instruments. Systematic and transparent validation planning and the generation (or, as in this study, collation) of relevant validity evidence supports developers and users of PRO instruments to determine the extent to which inferences about data are appropriate, meaningful and useful (i.e., valid) for intended decisions about the health and care of individuals, groups and populations.

scholarly journals Using the Health Literacy Questionnaire (HLQ) with Providers in the Early Intervention Setting: A Qualitative Validity Testing Study

2020 ◽  
Vol 17 (7) ◽  
pp. 2603 ◽  
Author(s):  
Catherine J. Leslie ◽  
Melanie Hawkins ◽  
Diane L. Smith
Keyword(s):  
Early Intervention ◽  
Health Literacy ◽  
Health Equity ◽  
The United States ◽  
Validity Testing ◽  
Child Health Outcomes ◽  
Health Literacy Questionnaire ◽  
Low Health Literacy ◽  
Study Results ◽  
Provider Perspective

More than one in four parents in the United States of America (USA) have low health literacy, which is associated with reduced health equity and negatively impacts child health outcomes. Early intervention (EI) programs are optimally placed to build the health literacy capacity of caregivers, which could improve health equity. The health literacy of interdisciplinary EI providers has not previously been measured. This study used the Health Literacy Questionnaire (HLQ) with EI providers (n = 10) to investigate evidence based on response (cognitive) processes. Narratives from cognitive interviews gave reasons for HLQ score choices, and concordance and discordance between HLQ item intent descriptions and narrative data were assessed using thematic analysis. Results found scales with highest concordance for Scales 3, 6, and 9 (each 96%, n = 24). Concordance was lowest on Scale 5 (88%, n = 22), although still strong with only 12% discordance. Three themes reflecting discordance were identified: (1) Differences between Australian and USA culture/health systems; (2) Healthcare provider perspective; and (3) Participants with no health problems to manage. Results show strong concordance between EI providers’ narrative responses and item intents. Study results contribute validity evidence for the use of HLQ data to inform interventions that build health literacy capacity of EI providers to then empower and build the health literacy of EI parents.

An empirical review of gambling expansion and gambling-related harm

2018 ◽  
Vol 64 (5-6) ◽  
pp. 295-306 ◽  
Author(s):  
Debi A. LaPlante ◽  
Heather M. Gray ◽  
Pat M. Williams ◽  
Sarah E. Nelson
Keyword(s):  
Literature Review ◽  
Peer Review ◽  
Empirical Data ◽  
Empirical Studies ◽  
Expansion Method ◽  
Responsible Gambling ◽  
Review Literature ◽  
Evidence Based ◽  
Empirical Comparison ◽  
Related Harm

Abstract. Aims: To discuss and review the latest research related to gambling expansion. Method: We completed a literature review and empirical comparison of peer reviewed findings related to gambling expansion and subsequent gambling-related changes among the population. Results: Although gambling expansion is associated with changes in gambling and gambling-related problems, empirical studies suggest that these effects are mixed and the available literature is limited. For example, the peer review literature suggests that most post-expansion gambling outcomes (i. e., 22 of 34 possible expansion outcomes; 64.7 %) indicate no observable change or a decrease in gambling outcomes, and a minority (i. e., 12 of 34 possible expansion outcomes; 35.3 %) indicate an increase in gambling outcomes. Conclusions: Empirical data related to gambling expansion suggests that its effects are more complex than frequently considered; however, evidence-based intervention might help prepare jurisdictions to deal with potential consequences. Jurisdictions can develop and evaluate responsible gambling programs to try to mitigate the impacts of expanded gambling.

Health Literacy: A Necessary Element for Achieving Health Equity

2015 ◽  
Vol 5 (7) ◽  
Author(s):  
Robert A. Logan ◽  
◽  
Winston F. Wong ◽  
Michael Villaire ◽  
Gem Daus ◽  
...  
Keyword(s):  
Health Literacy ◽  
Health Equity

Best Practices for Data Visualization: Creating and Evaluating a Report for an Evidence-Based Fall Prevention Program (Preprint)

2018 ◽  
Author(s):  
Srijesa Khasnabish ◽  
Zoe Burns ◽  
Madeline Couch ◽  
Mary Mullin ◽  
Randall Newmark ◽  
...  
Keyword(s):  
Literature Review ◽  
Best Practices ◽  
Implementation Science ◽  
Data Visualization ◽  
Systematic Literature Review ◽  
Fall Prevention ◽  
Prevention Program ◽  
Evidence Based ◽  
Data Display ◽  
Fall Prevention Program

BACKGROUND Data visualization experts have identified core principles to follow when creating visual displays of data that facilitate comprehension. Such principles can be applied to creating effective reports for clinicians that display compliance with quality improvement protocols. A basic tenet of implementation science is continuous monitoring and feedback. Applying best practices for data visualization to reports for clinicians can catalyze implementation and sustainment of new protocols. OBJECTIVE To apply best practices for data visualization to create reports that clinicians find clear and useful. METHODS First, we conducted a systematic literature review to identify best practices for data visualization. We applied these findings and feedback collected via a questionnaire to improve the Fall TIPS Monthly Report (FTMR), which shows compliance with an evidence-based fall prevention program, Fall TIPS (Tailoring Interventions for Patient Safety). This questionnaire was based on the requirements for effective data display suggested by expert Stephen Few. We then evaluated usability of the FTMR using a 15-item Health Information Technology Usability Evaluation Scale (Health-ITUES). Items were rated on a 5-point Likert scale from strongly disagree (1) to strongly agree (5). RESULTS The results of the systematic literature review emphasized that the ideal data display maximizes the information communicated while minimizing the cognitive efforts involved with data interpretation. Factors to consider include selecting the correct type of display (e.g. line vs bar graph) and creating simplistic reports. The qualitative and quantitative evaluations of the original and final FTMR revealed improved perceptions of the visual display of the reports and their usability. Themes that emerged from the staff interviews emphasized the value of simplified reports, meaningful data, and usefulness to clinicians. The mean (SD) rating on the Health-ITUES scale when evaluating the original FTMR was 3.86 (0.19) and increased to 4.29 (0.11) when evaluating the revised FTMR (Mann Whitney U Test, z=-12.25, P<0.001). CONCLUSIONS Best practices identified through a systematic review can be applied to create effective reports for clinician use. The lessons learned from evaluating FTMR perceptions and measuring usability can be applied to creating effective reports for clinician use in the context of other implementation science projects.

scholarly journals Development and validity testing of the Adolescent Health Literacy Questionnaire (AHLQ): Protocol for a mixed methods study within the Irish school setting

BMJ Open ◽  
2020 ◽  
Vol 10 (11) ◽  
pp. e039920
Author(s):  
Ailbhe Spillane ◽  
Sarahjane Belton ◽  
Clare McDermott ◽  
Johann Issartel ◽  
Richard H Osborne ◽  
...  
Keyword(s):  
Mixed Methods ◽  
Health Literacy ◽  
Young People ◽  
Adolescent Health ◽  
Pearson Correlation ◽  
Adult Population ◽  
School Setting ◽  
Mixed Methods Study ◽  
Advisory Panel ◽  
Validity Testing

IntroductionHealth literacy research has focused predominantly on the adult population, and much less is understood about this concept from an adolescent perspective. The tools currently available to measure adolescent health literacy have been adapted from adult versions. This limits their applicability to young people because of the developmental characteristics that impact on adolescents’ behaviour, including impulse control and judgement skills. This protocol describes the intended development and validity testing of a questionnaire to measure health literacy in adolescents.Methods and analysisThis protocol describes this mixed methods study that has three phases: the first phase will involve grounded research with adolescents using qualitative group interviews, co-design and concept mapping workshops to understand what health and healthy behaviours mean to adolescents and to explore their health literacy needs and the potential domains for the questionnaire. The draft health literacy domains identified will be presented to the youth advisory panel, and the questionnaire will be altered based on their feedback. Cognitive pretesting of the questionnaire items will also be conducted. Phase 2 will involve piloting the questionnaire to a two-stage random sample of young people in five urban and rural schools in Ireland. Test–retest reliability will be conducted using Pearson correlation coefficient. Confirmatory factor analysis will also be conducted to analyse the psychometric properties of the questionnaire. Phase 3 will involve the questionnaire being rolled out to a nationally representative sample of adolescents (n=6052) in Ireland to assess their levels of health literacy.Ethics and disseminationEthical approval to conduct this study has been granted from the University College Dublin Human Research Ethics Committee – Sciences (LS-20–08). Informed assent from adolescents and informed consent from parents/guardians will be sought. The findings of this research will be disseminated at national and international conferences, as well as through publication in peer-reviewed journals.

scholarly journals An Exit Strategy for the Definitional Elusiveness: A Three-Dimensional Framework for Social Entrepreneurship

2021 ◽  
Vol 13 (2) ◽  
pp. 563
Author(s):  
Bing Ran ◽  
Scott Weller
Keyword(s):  
Social Change ◽  
Literature Review ◽  
Systematic Literature Review ◽  
Social Entrepreneurship ◽  
Common Ground ◽  
Three Dimensional ◽  
Theoretical Framework ◽  
Future Research ◽  
Exit Strategy ◽  
Conceptual Paper

Despite the growing utility and prevalence of social entrepreneurship, an accepted definition remains elusive and infeasible. Yet, it is imperative that the principles guiding social entrepreneurship are identified so that common ground is established to facilitate future research. On the basis of a systematic literature review, this conceptual paper proposes a theoretical framework outlining social entrepreneurship as a three-dimensional framework as a function of continua of “social” and “business” logics, “beneficial” and “detrimental” social change logics, and “innovation” and “mundane” logics. The framework accommodates the fuzziness and ambiguity associated with social entrepreneurship whilst remaining a workable, identifiable construct. By accounting for the shifting logics practiced by social entrepreneurship that both influence and are influenced by the organizational environment, this framework provides an exit strategy for the definitional elusiveness of social entrepreneurship. The resultant structures and functions of social entrepreneurship are shaped by these constraints as reflected by the fluidity and flexibility endorsed by the framework. Four avenues for future research regarding social entrepreneurship are recommended on the basis of the framework proposed in this article.

Sign in / Sign up

Export Citation Format

Share Document