scholarly journals A Scale to Assess the Methodological Quality of Studies Assessing Usability of Electronic Health Products and Services: Delphi Study Followed by Validity and Reliability Testing

10.2196/14829 ◽  
2019 ◽  
Vol 21 (11) ◽  
pp. e14829 ◽  
Author(s):  
Anabela G Silva ◽  
Patrícia Simões ◽  
Rita Santos ◽  
Alexandra Queirós ◽  
Nelson P Rocha ◽  
...  
Keyword(s):  
Physical Activity ◽  
Construct Validity ◽  
Delphi Study ◽  
Methodological Quality ◽  
Mobile Apps ◽  
Intraclass Correlation ◽  
Delphi Panel ◽  
High Quality ◽  
Electronic Health

Background The usability of electronic health (eHealth) and mobile health apps is of paramount importance as it impacts the quality of care. Methodological quality assessment is a common practice in the field of health for different designs and types of studies. However, we were unable to find a scale to assess the methodological quality of studies on the usability of eHealth products or services. Objective This study aimed to develop a scale to assess the methodological quality of studies assessing usability of mobile apps and to perform a preliminary analysis of of the scale’s feasibility, reliability, and construct validity on studies assessing usability of mobile apps, measuring aspects of physical activity. Methods A 3-round Delphi panel was used to generate a pool of items considered important when assessing the quality of studies on the usability of mobile apps. These items were used to write the scale and the guide to assist its use. The scale was then used to assess the quality of studies on usability of mobile apps for physical activity, and it assessed in terms of feasibility, interrater reliability, and construct validity. Results A total of 25 experts participated in the Delphi panel, and a 15-item scale was developed. This scale was shown to be feasible (time of application mean 13.10 [SD 2.59] min), reliable (intraclass correlation coefficient=0.81; 95% CI 0.55-0.93), and able to discriminate between low- and high-quality studies (high quality: mean 9.22 [SD 0.36]; low quality: mean 6.86 [SD 0.80]; P=.01). Conclusions The scale that was developed can be used both to assess the methodological quality of usability studies and to inform its planning.

scholarly journals A Scale to Assess the Methodological Quality of Studies Assessing Usability of Electronic Health Products and Services: Delphi Study Followed by Validity and Reliability Testing (Preprint)

2019 ◽  
Author(s):  
Anabela G Silva ◽  
Patrícia Simões ◽  
Rita Santos ◽  
Alexandra Queirós ◽  
Nelson P Rocha ◽  
...  
Keyword(s):  
Physical Activity ◽  
Construct Validity ◽  
Delphi Study ◽  
Methodological Quality ◽  
Mobile Apps ◽  
Intraclass Correlation ◽  
Delphi Panel ◽  
High Quality ◽  
Electronic Health

BACKGROUND The usability of electronic health (eHealth) and mobile health apps is of paramount importance as it impacts the quality of care. Methodological quality assessment is a common practice in the field of health for different designs and types of studies. However, we were unable to find a scale to assess the methodological quality of studies on the usability of eHealth products or services. OBJECTIVE This study aimed to develop a scale to assess the methodological quality of studies assessing usability of mobile apps and to perform a preliminary analysis of of the scale’s feasibility, reliability, and construct validity on studies assessing usability of mobile apps, measuring aspects of physical activity. METHODS A 3-round Delphi panel was used to generate a pool of items considered important when assessing the quality of studies on the usability of mobile apps. These items were used to write the scale and the guide to assist its use. The scale was then used to assess the quality of studies on usability of mobile apps for physical activity, and it assessed in terms of feasibility, interrater reliability, and construct validity. RESULTS A total of 25 experts participated in the Delphi panel, and a 15-item scale was developed. This scale was shown to be feasible (time of application mean 13.10 [SD 2.59] min), reliable (intraclass correlation coefficient=0.81; 95% CI 0.55-0.93), and able to discriminate between low- and high-quality studies (high quality: mean 9.22 [SD 0.36]; low quality: mean 6.86 [SD 0.80]; <italic>P</italic>=.01). CONCLUSIONS The scale that was developed can be used both to assess the methodological quality of usability studies and to inform its planning.

scholarly journals Determining the psychometric properties of a novel questionnaire to measure “preparedness for the future” (Prep FQ)

2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Daren K. Heyland ◽  
J. Paige Pope ◽  
Xuran Jiang ◽  
Andrew G. Day
Keyword(s):  
Quality Of Life ◽  
Construct Validity ◽  
Health Behaviors ◽  
Psychometric Properties ◽  
Well Being ◽  
High Quality ◽  
The Future ◽  
Quality Of Death ◽  
Test Retest Reliability

Abstract Background People are living longer than ever before. However, with living longer comes increased problems that negatively impact on quality of life and the quality of death. Tools are needed to help individuals assess whether they are practicing the best attitudes and behaviors that are associated with a future long life, high quality of life, high quality of death and a satisfying post-death legacy. The purpose of paper is to describe the process we used to develop a novel questionnaire (“Preparedness for the Future Questionnaire™ or Prep FQ”) and to define its psychometric properties. Methods Using a multi-step development procedure, items were generated, for the new questionnaire after which the psychometric properties were tested with a heterogeneous sample of 502 Canadians. Using an online polling panel, respondents were asked to complete demographic questions as well as the Prep-FQ, Global Rating of Life Satisfaction, the Keyes Psychological Well-Being scale and the Short-Form 12. Results The final version of the questionnaire contains 34 items in 8 distinct domains (“Medico-legal”, “Social”, “Psychological Well-being”, “Planning”, “Enrichment”, “Positive Health Behaviors”, “Negative Health Behaviors”, and “Late-life Planning”). We observed minimum missing data and good usage of all response options. The average overall Prep FQ score is 51.2 (SD = 13.3). The Cronbach alphas assessing internal reliability for the Prep FQ domains ranged from 0.33 to 0.88. The intra-class correlation coefficient (ICC) used to assess the test–retest reliability had an overall score of 0.87. For the purposes of establishing construct validity, all the pre-specified relationships between Prep FQ and the other questionnaires were met. Conclusion Analyses of this novel measure offered support for its face validity, construct validity, test–retest reliability, and internal consistency. With the development of this useful and valid scale, future research can utilize this measure to engage people in the process of comprehensively assessing and improving their state of preparedness for the future, tracking their progress along the way. Ultimately, this program of research aims to improve the quality and quantity of peoples live by helping them ‘think ahead’ and ‘plan ahead’ on the aspects of their daily life that matter to their future.

scholarly journals The quality of guidelines in non-pharmacological prevention and management of chemotherapy-induced neuropathy: A review

2018 ◽  
Author(s):  
Shahin Salarvand ◽  
Simin Hemati ◽  
Payman Adibi ◽  
Fariba Taleghani
Keyword(s):  
Methodological Quality ◽  
Stakeholder Involvement ◽  
Common Adverse Effect ◽  
Adaptation Process ◽  
Medical Sciences ◽  
High Quality ◽  
Agree Ii ◽  
Pharmacological Prevention ◽  
Agree Ii Instrument

Chemotherapy-induced peripheral neuropathy is a common adverse effect occurring in patients undergoing neurotoxic chemotherapy. However, there is no FDA-approved treatment option for it. Given the importance of clinical practice guidelines in this area, this study aimed to determine the methodological quality of extant CIPN guidelines. The study was done as part of the adaptation process of CIPN related CPGs at Isfahan University of Medical Sciences, Iran. A systematic search of published CPGs about chemotherapy-induced CIPN in which the AGREE II instrument was applied for appraising CPGs of CIPN was performed. In general, amongst all of the AGREE II Instrument’s domains in the evaluated CPGs, the clarity of presentation and stakeholder involvement domains took favorable scores; and other domains obtained unfavorable and relatively favorable scores. The quality of cancer therapy-induced neuropathy CPGs needs to be improved and designing high-quality CPGs must be considered.

scholarly journals Impact of Adjustment for Quality on Results of Metaanalyses of Diagnostic Accuracy

2007 ◽  
Vol 53 (2) ◽  
pp. 164-172 ◽  
Author(s):  
Mariska Leeflang ◽  
Johannes Reitsma ◽  
Rob Scholten ◽  
Anne Rutjes ◽  
Marcello Di Nisio ◽  
...  
Keyword(s):  
Diagnostic Accuracy ◽  
Quality Assessment ◽  
Confidence Intervals ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Patient Characteristics ◽  
Clinical Decision ◽  
High Quality ◽  
Poor Reporting

Abstract Background: We examined whether and to what extent different strategies of defining and incorporating quality of included studies affect the results of metaanalyses of diagnostic accuracy. Methods: We evaluated the methodological quality of 487 diagnostic-accuracy studies in 30 systematic reviews with the QUADAS (Quality Assessment of Diagnostic-Accuracy Studies) checklist. We applied 3 strategies that varied both in the definition of quality and in the statistical approach to incorporate the quality-assessment results into metaanalyses. We compared magnitudes of diagnostic odds ratios, widths of their confidence intervals, and changes in a hypothetical clinical decision between strategies. Results: Following 2 definitions of quality, we concluded that only 70 or 72 of 487 studies were of “high quality”. This small number was partly due to poor reporting of quality items. None of the strategies for accounting for differences in quality led systematically to accuracy estimates that were less optimistic than ignoring quality in metaanalyses. Limiting the review to high-quality studies considerably reduced the number of studies in all reviews, with wider confidence intervals as a result. In 18 reviews, the quality adjustment would have resulted in a different decision about the usefulness of the test. Conclusions: Although reporting the results of quality assessment of individual studies is necessary in systematic reviews, reader wariness is warranted regarding claims that differences in methodological quality have been accounted for. Obstacles for adjusting for quality in metaanalyses are poor reporting of design features and patient characteristics and the relatively low number of studies in most diagnostic reviews.

scholarly journals Quality of the Evidence Supporting the Role of Oral Nutritional Supplements in the Management of Malnutrition: An Overview of Systematic Reviews and Meta-Analyses

2020 ◽  
Author(s):  
Christine Baldwin ◽  
Rosemary Smith ◽  
Michelle Gibbs ◽  
C Elizabeth Weekes ◽  
Peter W Emery
Keyword(s):  
At Risk ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Hospital Readmissions ◽  
Nutritional Supplement ◽  
Measurement Tool ◽  
High Quality ◽  
Oral Nutritional Supplements ◽  
Meta Analyses

ABSTRACT There is considerable heterogeneity across the findings of systematic reviews of oral nutritional supplement (ONS) interventions, presenting difficulties for healthcare decision-makers and patients alike. It is not known whether heterogeneity arises from differences in patient populations or relates to methodological rigor. This overview aimed to collate and compare findings from systematic reviews of ONSs compared with routine care in adult patients who were malnourished or at risk of malnutrition with any clinical condition and to examine their methodological quality. Three electronic databases were searched to July 2019, supplemented with hand-searching. Data on all outcomes were extracted and review methodological quality assessed using A MeaSurement Tool for Assessment of systematic Reviews (AMSTAR). Twenty-two reviews were included, 11 in groups from mixed clinical backgrounds and 11 in specific clinical conditions. Ninety-one meta-analyses were identified for 12 different outcomes but there was discordance between results. Significant benefits of ONSs were reported in 4 of 4 analyses of energy intake, 7 of 11 analyses of body weight, 7 of 22 analyses of mortality, 10 of 17 analyses of complications (total and infectious), 1 of 3 analyses of muscle strength, 4 of 9 analyses of body composition/nutritional status, 2 of 14 analyses of length of stay, and 2 of 5 analyses of hospital readmissions. Ten reviews were high quality (AMSTAR scores 8–11), 9 moderate (AMSTAR scores 3–8), and 3 poor (AMSTAR scores 0–3). Methodological deficiencies were limitations to searches, poor reporting of heterogeneity, and failure to incorporate quality of evidence into any recommendations. Discordance between reviews was not markedly reduced when only high-quality reviews were considered. Evidence for the effects of ONS in malnourished patients or those who are at risk of malnutrition is uncertain, and discordance in results can arise from differences in clinical background of patients or the etiological basis of malnutrition.

scholarly journals Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria

2005 ◽  
Vol 21 (2) ◽  
pp. 240-245 ◽  
Author(s):  
Silvia Evers ◽  
Mariëlle Goossens ◽  
Henrica de Vet ◽  
Maurits van Tulder ◽  
André Ament
Keyword(s):  
Health Economic ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Economic Evaluations ◽  
Delphi Panel ◽  
Policy Makers ◽  
Core Set ◽  
International Experts ◽  
Economic Criteria

Objectives:The aim of the Consensus on Health Economic Criteria (CHEC) project is to develop a criteria list for assessment of the methodological quality of economic evaluations in systematic reviews. The criteria list resulting from this CHEC project should be regarded as a minimum standard.Methods:The criteria list has been developed using a Delphi method. Three Delphi rounds were needed to reach consensus. Twenty-three international experts participated in the Delphi panel.Results:The Delphi panel achieved consensus over a generic core set of items for the quality assessment of economic evaluations. Each item of the CHEC-list was formulated as a question that can be answered by yes or no. To standardize the interpretation of the list and facilitate its use, the project team also provided an operationalization of the criteria list items.Conclusions:There was consensus among a group of international experts regarding a core set of items that can be used to assess the quality of economic evaluations in systematic reviews. Using this checklist will make future systematic reviews of economic evaluations more transparent, informative, and comparable. Consequently, researchers and policy-makers might use these systematic reviews more easily. The CHEC-list can be downloaded freely fromhttp://www.beoz.unimaas.nl/chec/.

scholarly journals Public open spaces and physical activity: disparities of resources in Florianópolis

2019 ◽  
Vol 53 ◽  
pp. 112
Author(s):  
Sofia Wolker Manta ◽  
Rodrigo Siqueira Reis ◽  
Tânia Rosane Bertoldo Benedetti ◽  
Cassiano Ricardo Rech
Keyword(s):  
Physical Activity ◽  
Low Income ◽  
High Income ◽  
Sociodemographic Characteristics ◽  
Open Spaces ◽  
High Quality ◽  
Census Tracts ◽  
Physical Activity Facilities ◽  
Public Open Spaces

ABSTRACT OBJECTIVE: To analyze the association between sociodemographic characteristics of census tracts and the presence/quality of public open spaces and physical activity facilities. METHODS: A cross-sectional study was conducted in 643 census tracts in Florianópolis, Brazil, the presence and quality of public open spaces and physical activity facilities were objectively analyzed and the data by census tracts using Geographic Information Systems was treated. Outcomes were analyzed considering the census tracts as having: ≥ 1 public open spaces; ≥ 1 public open spaces with high quality; ≥ 2 physical activity facilities and high-quality physical activity facilities. Sociodemographic characteristics were the independent variables. Logistic regression analysis was performed. RESULTS: Census tracts with a medium-income (OR = 1.8; 95%CI 1.1–3.0) and high-income (OR = 2.4; 95%CI 1.4–4.0), in those with medium (OR = 1.7; 95%CI 1.0–2.7) and high residential density (OR = 2.0; 95%CI 1,2–3.3), and with higher proportions of older adults (OR = 3.3; 95%CI 1.9–5.7) had a higher proportion of public open spaces. Census tracts with higher proportions of children/adolescents (OR = 0.3; 95%CI 0.2–0.6) and non-white residents (OR= 0.6; 95%CI 0.3–0.9) were less likely to contain public open spaces. The tracts with medium (OR = 4.0; 95%CI 1.4–11.3) and high-income (OR = 3.6; 95%CI 1.2–10.2) were more likely to contain public open spaces with ≥ 2 structures for physical activity, compared with those with low-income. We observed the inverse in sectors with a high proportion of non-white residents (OR = 0.3; 95%CI 0.1–0.9). CONCLUSIONS: Census tracts with higher proportions of children or adolescents, non-white individuals and those in the low-income strata had lower odds of containing public open spaces and physical activity facilities.

scholarly journals Development and evaluation of the nurse quality of communication with patient questionnaire

2010 ◽  
Vol 138 (1-2) ◽  
pp. 79-84 ◽  
Author(s):  
Mira Vukovic ◽  
Branislav Gvozdenovic ◽  
Branka Stamatovic-Gajic ◽  
Miodrag Ilic ◽  
Tomislav Gajic
Keyword(s):  
Construct Validity ◽  
Correlation Coefficient ◽  
Pearson Correlation ◽  
Intraclass Correlation ◽  
Repeated Measurements ◽  
Patient Communication ◽  
Patient Questionnaire ◽  
Pearson Correlation Coefficient ◽  
Patient Relationship

Introduction. Nurse/patient relationship as a complex interrelation or as an interaction of the factor patient and factor nurse has been a subject of a number of studies during the past ten years. Nurse/patient communication is a special entity, usually observed within a framework of the wider nurse/ patient relationship. In that regard, we wanted to develop a standardized questionnaire that could reliably measure the quality of communication between nurse and patient, and be used by nurses. Objectives. The main goal of this study was to develop and evaluate construct validity of the Nurse Quality of Communication with Patient Questionnaire (NQCPQ), as well as to evaluate its reliability. The goal was also to establish a measure of inter-raters reliability, using two repeated measurements of results by items and scores of the NQCPQ, on the same observed units by two assessors. Methods. The starting NQCPQ that consists of 25 items, was filled in by two groups of nurses. Each nurse was questioned during morning and afternoon shifts, in order to evaluate their communication with hospitalized patients, using marks from 1 to 6. To evaluate construct validity, we used the analysis of main components, while reliability was assessed using intraclass correlation coefficient and Cronbach-alpha coefficient. To evaluate interraters reliability, we used Pearson correlation coefficient. Results. Using a group of 118 patients, we explained 86% of the unknown, regarding the investigated phenomenon (communication nurse/patient), using one component by which we separated 6 items of the questionnaire. Inter-item correlation (?) in this component was 0.96. Pearson correlation coefficient was highly significant, value 0.7 by item, and correlation coefficient for scores at repeated measurements was 0.84. Conclusion. NQCPQ is 6-item instrument with high construct validity. It can be used to measure quality of nurse/patient communication in a simple, fast and reliable way. It could contribute to more adequate research and defining of this problem, and as such could be used in studies of interaction of psychometric, clinical, biochemical, socio-cultural, demographic and other parameters as well.

Adaptation, Test-Retest Reliability, and Construct Validity of the Physical Activity Neighborhood Environment Scale in Nigeria (PANES-N)

2013 ◽  
Vol 10 (8) ◽  
pp. 1079-1090 ◽  
Author(s):  
Adewale L. Oyeyemi ◽  
James F. Sallis ◽  
Adetoyeje Y. Oyeyemi ◽  
Mariam M. Amin ◽  
Ilse De Bourdeaudhuij ◽  
...  
Keyword(s):  
Physical Activity ◽  
Construct Validity ◽  
Intraclass Correlation ◽  
Cognitive Testing ◽  
Neighborhood Environment ◽  
Analysis Of Covariance ◽  
Environmental Perceptions ◽  
Retest Reliability ◽  
Environment Scale ◽  
Test Retest Reliability

Background:This study adapted the Physical Activity Neighborhood Environment Scale (PANES) to the Nigerian context and assessed the test-retest reliability and construct validity of the Nigerian version (PANES-N).Methods:A multidisciplinary panel of experts adapted the original PANES to reflect the built and social environment of Nigeria. The adapted PANES was subjected to cognitive testing and test retest reliability in a diverse sample of Nigerian adults (N = 132) from different neighborhood types. Intraclass Correlation Coefficients (ICC) was used to assess test-retest reliability, and construct validity was investigated with Analysis of Covariance for differences in environmental attributes between neighborhoods.Results:Four of the 17 items on the original PANES were significantly modified, 3 were removed and 2 new items were incorporated into the final version of adapted PANES-N. Test-retest reliability was substantial to almost perfect (ICC = 0.62–1.00) for all items on the PANES-N, and residents of neighborhoods in the inner city reported higher residential density, land use mix and safety, but lower pedestrian facilities and aesthetics than did residents of government reserved area/new layout neighborhoods.Conclusion:The PANES-N appears promising for assessing environmental perceptions related to physical activity in Nigeria, but further testing is required to assess its applicability across Africa.

scholarly journals Measuring the Quality of Clinical Skills Mobile Apps for Student Learning: Systematic Search, Analysis, and Comparison of Two Measurement Scales (Preprint)

2020 ◽  
Author(s):  
Tehmina Gladman ◽  
Grace Tylee ◽  
Steve Gallagher ◽  
Jonathan Mair ◽  
Rebecca Grainger
Keyword(s):  
Student Learning ◽  
Teaching And Learning ◽  
Rating Scale ◽  
Clinical Skills ◽  
Mobile Apps ◽  
Mobile App ◽  
Systematic Search ◽  
Measurement Scales ◽  
High Quality

BACKGROUND Mobile apps are widely used in health professions, which increases the need for simple methods to determine the quality of apps. In particular, teachers need the ability to curate high-quality mobile apps for student learning. OBJECTIVE This study aims to systematically search for and evaluate the quality of clinical skills mobile apps as learning tools. The quality of apps meeting the specified criteria was evaluated using two measures—the widely used Mobile App Rating Scale (MARS), which measures general app quality, and the Mobile App Rubric for Learning (MARuL), a recently developed instrument that measures the value of apps for student learning—to assess whether MARuL is more effective than MARS in identifying high-quality apps for learning. METHODS Two mobile app stores were systematically searched using clinical skills terms commonly found in medical education and apps meeting the criteria identified using an approach based on PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A total of 9 apps were identified during the screening process. The apps were rated independently by 2 reviewers using MARS and MARuL. RESULTS The intraclass correlation coefficients (ICCs) for the 2 raters using MARS and MARuL were the same (MARS ICC [two-way]=0.68; <i>P</i>&lt;.001 and MARuL ICC [two-way]=0.68; <i>P</i>&lt;.001). Of the 9 apps, Geeky Medics-OSCE revision (MARS Android=3.74; MARS iOS=3.68; MARuL Android=75; and MARuL iOS=73) and OSCE PASS: Medical Revision (MARS Android=3.79; MARS iOS=3.71; MARuL Android=69; and MARuL iOS=73) scored highly on both measures of app quality and for both Android and iOS. Both measures also showed agreement for the lowest rated app, Patient Education Institute (MARS Android=2.21; MARS iOS=2.11; MARuL Android=18; and MARuL iOS=21.5), which had the lowest scores in all categories except information (MARS) and professional (MARuL) in both operating systems. MARS and MARuL were both able to differentiate between the highest and lowest quality apps; however, MARuL was better able to differentiate apps based on teaching and learning quality. CONCLUSIONS This systematic search and rating of clinical skills apps for learning found that the quality of apps was highly variable. However, 2 apps—Geeky Medics-OSCE revision and OSCE PASS: Medical Revision—rated highly for both versions and with both quality measures. MARS and MARuL showed similar abilities to differentiate the quality of the 9 apps. However, MARuL’s incorporation of teaching and learning elements as part of a multidimensional measure of quality may make it more appropriate for use with apps focused on teaching and learning, whereas MARS’s more general rating of quality may be more appropriate for health apps targeting a general health audience. Ratings of the 9 apps by both measures also highlighted the variable quality of clinical skills mobile apps for learning. CLINICALTRIAL

Sign in / Sign up

Export Citation Format

Share Document