Rater reliability and construct validity of a mobile application for posture analysis

BACKGROUND Mobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Consequently, instruments of high psychometric quality for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA in various health domains. Only few validation studies investigating its psychometric quality exist with selected samples of MHAs. No study has evaluated the construct validity of the MARS and concurrent validity to other instruments. OBJECTIVE This study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS. METHODS MARS scoring data was pooled from 15 international app quality reviews to evaluate the psychometric properties of the MARS. The MARS measures app quality across four dimensions: engagement, functionality, aesthetics and information quality. App quality is determined for each dimension and overall. Construct validity was evaluated by assessing related competing confirmatory models that were explored by confirmatory factor analysis (CFA). A combination of non-centrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices was used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations between the MARS and 1) another quality assessment tool called ENLIGHT, and 2) user star-rating extracted from app stores were investigated. Reliability was determined using Omega. Objectivity was assessed in terms of intra-class correlation. RESULTS In total, MARS ratings from 1,299 MHA covering 15 different health domains were pooled for the analysis. Confirmatory factor analysis confirmed a bifactor model with a general quality factor and an additional factor for each subdimension (RMSEA=0.074, TLI=0.922, CFI=0.940, SRMR=0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC=0.82). The overall MARS rating was positively associated with ENLIGHT (r=0.91, P<0.01) and user-ratings (r=0.14, P<0.01). CONCLUSIONS he psychometric evaluation of the MARS demonstrated its suitability for the quality assessment of MHAs. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.

Download Full-text

Adaptation and Validation of a Nutrition Environment Measures Survey for University Grab-and-Go Establishments

Canadian Journal of Dietetic Practice and Research ◽

10.3148/cjdpr-2015-036 ◽

2016 ◽

Vol 77 (1) ◽

pp. 17-24 ◽

Cited By ~ 6

Author(s):

Brian K.C. Lo ◽

Leia Minaker ◽

Alicia N.T. Chan ◽

Jessica Hrgetic ◽

Catherine L. Mah

Keyword(s):

Construct Validity ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Comparison Method ◽

Nutrition Information ◽

Perfect Agreement ◽

Cross Sectional ◽

Rater Reliability ◽

University Of Toronto ◽

Nutrition Environment

Purpose: To adapt and validate a survey instrument to assess the nutrition environment of grab-and-go establishments at a university campus. Methods: A version of the Nutrition Environment Measures Survey for grab-and-go establishments (NEMS-GG) was adapted from existing NEMS instruments and tested for reliability and validity through a cross-sectional assessment of the grab-and-go establishments at the University of Toronto. Product availability, price, and presence of nutrition information were evaluated. Cohen’s kappa coefficient and intra-class correlation coefficients (ICC) were assessed for inter-rater reliability, and construct validity was assessed using the known-groups comparison method (via store scores). Results: Fifteen grab-and-go establishments were assessed. Inter-rater reliability was high with an almost perfect agreement for availability (mean κ = 0.995) and store scores (ICC = 0.999). The tool demonstrated good face and construct validity. About half of the venues carried fruit and vegetables (46.7% and 53.3%, respectively). Regular and healthier entrée items were generally the same price. Healthier grains were cheaper than regular options. Six establishments displayed nutrition information. Establishments operated by the university’s Food Services consistently scored the highest across all food premise types for nutrition signage, availability, and cost of healthier options. Conclusions: Health promotion strategies are needed to address availability and variety of healthier grab-and-go options in university settings.

Download Full-text

The Schedule for the Assessment of Drug-Induced Movement Disorders (SADIMoD): inter-rater reliability and construct validity

The International Journal of Neuropsychopharmacology ◽

10.1017/s1461145701002589 ◽

2001 ◽

Vol 4 (04) ◽

Cited By ~ 15

Author(s):

Anton J. M. Loonen ◽

Cees H. Doorschot ◽

Dianne A. van Hemert ◽

Marc C. J. M. Oostelbos ◽

Arnold E. S. Sijben ◽

...

Keyword(s):

Construct Validity ◽

Movement Disorders ◽

Drug Induced ◽

Rater Reliability

Download Full-text

16937 Evaluation of construct validity and inter-rater reliability of the Objective Structured Assessment of Technical Skills (OSATS) tool for assessment of surgical skills of dermatology trainees

Journal of the American Academy of Dermatology ◽

10.1016/j.jaad.2020.06.389 ◽

2020 ◽

Vol 83 (6) ◽

pp. AB75

Author(s):

Kristina Navrazhina ◽

Murad Alam ◽

Rebecca Tung ◽

David Surprenant ◽

Brienne D. Cressey ◽

...

Keyword(s):

Construct Validity ◽

Technical Skills ◽

Surgical Skills ◽

Rater Reliability

Download Full-text

Construct validity and test-retest reliability of a free mobile application for spatio-temporal gait analysis in Parkinson’s disease patients

Gait & Posture ◽

10.1016/j.gaitpost.2020.04.004 ◽

2020 ◽

Vol 79 ◽

pp. 86-91

Author(s):

Sergio Clavijo-Buendía ◽

Francisco Molina-Rueda ◽

Patricia Martín-Casas ◽

Paulina Ortega-Bastidas ◽

Esther Monge-Pereira ◽

...

Keyword(s):

Parkinson’S Disease ◽

Parkinson's Disease ◽

Construct Validity ◽

Gait Analysis ◽

Mobile Application ◽

Retest Reliability ◽

Spatio Temporal ◽

Test Retest Reliability

Download Full-text

The Reliability and Validity of Wearable Inertial Sensors Coupled with the Microsoft Kinect to Measure Shoulder Range-of-Motion

Sensors ◽

10.3390/s20247238 ◽

2020 ◽

Vol 20 (24) ◽

pp. 7238

Author(s):

Peter Beshara ◽

Judy F. Chen ◽

Andrew C. Read ◽

Pierre Lagadec ◽

Tian Wang ◽

...

Keyword(s):

Construct Validity ◽

Range Of Motion ◽

Inertial Sensors ◽

Objective Assessment ◽

Reliability And Validity ◽

Human Movement ◽

Microsoft Kinect ◽

Rater Reliability ◽

Shoulder Range Of Motion ◽

Shoulder Flexion

Background: Objective assessment of shoulder joint active range of motion (AROM) is critical to monitor patient progress after conservative or surgical intervention. Advancements in miniature devices have led researchers to validate inertial sensors to capture human movement. This study investigated the construct validity as well as intra- and inter-rater reliability of active shoulder mobility measurements using a coupled system of inertial sensors and the Microsoft Kinect (HumanTrak). Methods: 50 healthy participants with no history of shoulder pathology were tested bilaterally for fixed and free ROM: (1) shoulder flexion, and (2) abduction using HumanTrak and goniometry. The repeat testing of the standardised protocol was completed after seven days by two physiotherapists. Results: All HumanTrak shoulder movements demonstrated adequate reliability (intra-class correlation (ICC) ≥ 0.70). HumanTrak demonstrated higher intra-rater reliability (ICCs: 0.93 and 0.85) than goniometry (ICCs: 0.75 and 0.53) for measuring free shoulder flexion and abduction AROM, respectively. Similarly, HumanTrak demonstrated higher intra-rater reliability (ICCs: 0.81 and 0.94) than goniometry (ICCs: 0.70 and 0.93) for fixed flexion and abduction AROM, respectively. Construct validity between HumanTrak and goniometry was adequate except for free abduction. The differences between raters were predominately acceptable and below ±10°. Conclusions: These results indicated that the HumanTrak system is an objective, valid and reliable way to assess and track shoulder ROM.

Download Full-text