Assessing the Quality of Mobile Apps Used by Occupational Therapists: Evaluation Using the User Version of the Mobile Application Rating Scale

BACKGROUND Patients suffering from inflammatory bowel diseases (IBD) frequently need long-term medical treatment. Mobile apps promise to complement and improve IBD management, however, so far there is no scientific analysis of their quality. OBJECTIVE To evaluate the quality of German IBD mobile apps targeting inflammatory bowel disease patients and physicians treating IBD patients using the Mobile Application Rating Scale (MARS). METHODS The German Apple App Store and Google Play store were systematically searched to identify German IBD mobile apps for patients and physician use. MARS was used to independently assess app quality by 6 physicians, 3 using Android and 3 using iOS smartphones. Apps were randomly assigned so that the 4 apps with the most downloads were rated by all raters and the remaining apps were rated by 1 Android and 1 iOS user. RESULTS In total, we identified 1764 apps in the Apple App Store and Google Play Store. After removing apps that were not related to IBD (1386) and not available in German (317), 61 apps remained. After removing duplicates (3), congress (7), journal (4), and clinical study apps (6), as well as excluding apps that were available in only one of the two app stores (20) and apps which can only be used with an additional device (7) , 14 apps fulfilled the inclusion criteria. The app “CED Dokumentation und Tipps” had the highest overall MARS score with 4.11/5. Median MARS scores of the 14 apps were ranging from 2.38 and 4.11. All of the analyzed apps addressed patients. Three apps are classified as medical products. There was no significant difference between iOS and Android raters. CONCLUSIONS The quality of German IBD apps is largely heterogeneous. Furthermore, despite promising international study results, little evidence exists proving a clinical benefit for German IBD apps. MARS seems to be a suitable method to identify relevant apps. Accordingly, clinical studies and also patient inclusion in the app development process are needed to effectively implement mobile apps in routine care.

Download Full-text

Analysis of Publicly Available German Mobile Apps for Psoriasis Patients using the Mobile Application Rating Scale (MARS). (Preprint)

10.2196/preprints.34017 ◽

2021 ◽

Author(s):

Christian Lull ◽

Jan Alwin von Ahnen ◽

Georg Groß ◽

Victor Olsavszky ◽

Johannes Knitza ◽

...

Keyword(s):

Mobile Application ◽

Rating Scale ◽

Mobile Apps ◽

Management Strategies ◽

Skin Lesions ◽

Main Function ◽

Health Apps ◽

Erythematous Plaques ◽

Google Play

BACKGROUND Psoriasis is a chronic inflammatory skin disease. The visibility of the erythematous plaques on the skin as well as the pain and itch caused by the skin lesions frequently leads to psychological distress in patients. Smartphone apps are widespread, easily accessible and could effectively complement current management strategies of psoriasis patients. OBJECTIVE To systematically identify and objectively assess the quality of currently publicly available German apps for psoriasis patients and compile brief, ready-to-use app descriptions. METHODS We conducted a systematic search and assessment of German apps for psoriasis patients available in the Google Play and Apple App Store. The identified apps were randomly assigned to and independently rated using the Mobile Application Rating Scale (MARS) by three reviewers. Apps were ranked accordingly to their mean MARS rating and the highest ranked app was evaluated independently by two psoriasis patients using the User Version of the Mobile Application Rating Scale (uMARS). App information, including origin, main function and technical aspects and was congregated. RESULTS In total, we were able to identify 95 different apps for psoriasis, of which 15 were available in both app stores, 5 were not specifically intended for psoriasis patients and 1 was designed for clinical trials only. Consequently, the remaining nine apps were included in the final evaluation. Mean MARS score varied between 3.51 and 4.18. The app with the highest mean MARS score was “Psoriasis Helferin” (4.18/5) . When rated by patients, the app was rated lower in all MARS subcategories, resulting in a mean uMARS score of 3.48. CONCLUSIONS App quality was heterogenous and only a minority of identified apps was available in both app stores. The discrepant patient results highlight the importance of involving patients when developing and evaluating health apps. CLINICALTRIAL Deutsches Register klinische Studien (DRKS): DRKS00020963

Download Full-text

Psychometric evaluation of the Mobile Application Rating Scale (MARS) (Preprint)

10.2196/preprints.17689 ◽

2020 ◽

Cited By ~ 2

Author(s):

Yannik Terhorst ◽

Paula Philippi ◽

Lasse Sander ◽

Dana Schultchen ◽

Sarah Paganini ◽

...

Keyword(s):

Health Care ◽

Construct Validity ◽

Mobile Application ◽

Concurrent Validity ◽

Rating Scale ◽

Psychometric Evaluation ◽

Confirmatory Factor ◽

Health Domains ◽

Psychometric Quality

BACKGROUND Mobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Consequently, instruments of high psychometric quality for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA in various health domains. Only few validation studies investigating its psychometric quality exist with selected samples of MHAs. No study has evaluated the construct validity of the MARS and concurrent validity to other instruments. OBJECTIVE This study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS. METHODS MARS scoring data was pooled from 15 international app quality reviews to evaluate the psychometric properties of the MARS. The MARS measures app quality across four dimensions: engagement, functionality, aesthetics and information quality. App quality is determined for each dimension and overall. Construct validity was evaluated by assessing related competing confirmatory models that were explored by confirmatory factor analysis (CFA). A combination of non-centrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices was used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations between the MARS and 1) another quality assessment tool called ENLIGHT, and 2) user star-rating extracted from app stores were investigated. Reliability was determined using Omega. Objectivity was assessed in terms of intra-class correlation. RESULTS In total, MARS ratings from 1,299 MHA covering 15 different health domains were pooled for the analysis. Confirmatory factor analysis confirmed a bifactor model with a general quality factor and an additional factor for each subdimension (RMSEA=0.074, TLI=0.922, CFI=0.940, SRMR=0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC=0.82). The overall MARS rating was positively associated with ENLIGHT (r=0.91, P<0.01) and user-ratings (r=0.14, P<0.01). CONCLUSIONS he psychometric evaluation of the MARS demonstrated its suitability for the quality assessment of MHAs. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.

Download Full-text

An Analysis of Apps with a Medication List Functionality for Older People with Heart Failure: Evaluation using the Mobile Application Rating Scale (MARS) and the IMS Institute for Healthcare Informatics Functionality Score (Preprint)

10.2196/preprints.30674 ◽

2021 ◽

Author(s):

Yohanca Maria Diaz-Skeete ◽

David McQuaid ◽

Adewale Samuel Akinosun ◽

Idongesit Ekerete ◽

Natacha Carragher ◽

...

Keyword(s):

Heart Failure ◽

Mobile Application ◽

Rating Scale ◽

Scientific Evidence ◽

Medication List ◽

Healthcare Informatics ◽

Medication History ◽

Clinic Efficiency ◽

Google Play

BACKGROUND Managing the care of older patients with heart failure (HF) largely centres on medication management. Due to their frequent medication or dosing changes, an app supporting them to keep an up-to-date list of medication could be advantageous. During COVID-19 times, HF outpatients’ consultations are taking place virtually or by phone. An app with the capability to share the medication list with healthcare professionals before consultation could support the clinic efficiency, for example, reducing consultation time. However, the influence of apps on maintaining an up to date medication history for older adults with HF in Ireland remains largely unexplored. OBJECTIVE The objectives of this review are twofold: to review apps with a medication list functionality and to evaluate the quality of the apps included in the review using the Mobile Application Rating Scale (MARS) and the IMS Institute for Healthcare Informatics functionality scale. METHODS A systematic search of apps was conducted in June 2019 using the Google Play StoreTM and iTunes App StoreTM. The MARS was used independently by four researchers to assess the quality of the apps using an Android phone and an iPad. Apps were also evaluated using the IMS Institute for Healthcare Informatics functionality score. RESULTS Google play and iOS app stores searches identified 483 potential apps (292 from Android stores and 191 from Apple stores). Six apps met the inclusion criteria. Medisafe app had the highest overall MARS score (4/5) and the medication list & medical records app had the lowest overall score (2.5/5). Five out of the six apps achieved an acceptable quality MARS score (>3.0). Two apps scored the maximum number of features (n=11) according to the IMS Institute for Healthcare Informatics functionality score and two scored the lowest (n=5). The apps had on average 8 functions based on the IMS functionality criteria (range 5 to 11). CONCLUSIONS The quality of current apps with a medication list functionality varies regarding their technical aspects. Most of the apps reviewed have an acceptable MARS objective quality. However, the subjective quality or satisfaction with the apps was poor. Only three apps are based on scientific evidence and have been previously tested. Two apps featured all the IMS Institute for Healthcare Informatics functionalities and half do not provide clear instructions on how to enter medication data, do not display vital parameters data in an easy to understand format and do not guide users on how or when to take their medication. CLINICALTRIAL N/A

Download Full-text

Assessing Apps for Patients with Genitourinary Tumors Using the Mobile Application Rating Scale (MARS): Systematic Search in App Stores and Content Analysis (Preprint)

10.2196/preprints.17609 ◽

2019 ◽

Author(s):

Miguel Ángel Amor-García ◽

Roberto Collado-Borrell ◽

Vicente Escudero-Vilaplana ◽

Alejandra Melgarejo-Ortuño ◽

Ana Herranz-Alonso ◽

...

Keyword(s):

Prostate Cancer ◽

Health Care ◽

Health Professionals ◽

Health Care Professionals ◽

Mobile Application ◽

Rating Scale ◽

Cross Sectional ◽

Cancer Symptoms ◽

Genitourinary Cancers

BACKGROUND The large number of available cancer apps and their impact on the population necessitates a transparent, objective, and comprehensive evaluation by app experts, health care professionals, and users. To date, there have been no analyses or classifications of apps for patients with genitourinary cancers, which are among the most prevalent types of cancer. OBJECTIVE The objective of our study was to analyze the quality of apps for patients diagnosed with genitourinary cancers using the Mobile Application Rating Scale (MARS) and identify high-quality apps. METHODS We performed an observational cross-sectional descriptive study of all smartphone apps for patients diagnosed with genitourinary cancers available on iOS and Android platforms. In July 2019, we searched for all available apps for patients with genitourinary cancers (bladder, prostate, cervical, uterine, endometrial, kidney, testicular, and vulvar) or their caregivers. Apps were downloaded and evaluated, and the general characteristics were entered into a database. The evaluation was performed by 2 independent researchers using the MARS questionnaire, which rates 23 evaluation criteria clustered in 5 domains (Engagement, Functionality, Esthetics, Information, and Subjective Quality) on a scale from 1 to 5. RESULTS In total, 46 apps were analyzed. Of these, 31 (67%) were available on Android, 6 (13%) on iOS, and 9 (20%) on both platforms. The apps were free in 89% of cases (41/46), and 61% (28/46) had been updated in the previous year. The apps were intended for prostate cancer in 30% of cases (14/46) and cervical cancer in 17% (8/46). The apps were mainly informative (63%, 29/46), preventive (24%, 11/46), and diagnostic (13%, 6/46). Only 7/46 apps (15%) were developed by health care organizations. The mean MARS score for the overall quality of the 46 apps was 2.98 (SD 0.77), with a maximum of 4.63 and a minimum of 1.95. Functionality scores were quite similar for most of the apps, with the greatest differences in Engagement and Esthetics, which showed acceptable scores in one-third of the apps. The 5 apps with the highest MARS score were the following: “Bladder cancer manager,” “Kidney cancer manager,” “My prostate cancer manager,” “Target Ovarian Cancer Symptoms Diary,” and “My Cancer Coach.” We observed statistically significant differences in the MARS score between the operating systems and the developer types (P<.001 and P=.01, respectively), but not for cost (P=.62). CONCLUSIONS MARS is a helpful methodology to decide which apps can be prescribed to patients and to identify which features should be addressed to improve these tools. Most of the apps designed for patients with genitourinary cancers only try to provide data about the disease, without coherent interactivity. The participation of health professionals in the development of these apps is low; nevertheless, we observed that both the participation of health professionals and regular updates were correlated with quality.

Download Full-text

Quality of Mobile Apps for Care Partners of People with Alzheimer’s Disease and Related Dementias: Mobile App Rating Scale Evaluation (Preprint)

10.2196/preprints.33863 ◽

2021 ◽

Author(s):

Nicole E Werner ◽

Janetta C Brown ◽

Priya Loganathar ◽

Richard J Holden

Keyword(s):

Rating Scale ◽

Mobile Apps ◽

Mobile App ◽

Future Research ◽

App Development ◽

Care Partners ◽

Acceptable Quality ◽

The Us ◽

Google Play

BACKGROUND The over 11 million care partners in the US who provide care to people living with Alzheimer’s disease and related dementias (ADRD) cite persistent and pervasive unmet needs related to all aspects of their caregiving role. The proliferation of mobile applications (apps) for care partners has potential to meet the care partners’ needs, but the quality of apps is unknown. OBJECTIVE The present study aimed to 1) evaluate the quality of publicly available apps for care partners of people living with ADRD and 2) identify design features of low- and high-quality apps to guide future research and app development. METHODS We searched the US Apple and Google Play app stores with the criteria that the app needed to be 1) available in US Google play or Apple app stores, 2) directly accessible to users “out of the box”, 3) primarily intended for use by an informal (family, friend) caregiver or caregivers of a person with dementia. The included apps were then evaluated using the Mobile App Rating Scale (MARS), which includes descriptive app classification and rating using 23 items across five dimensions: engagement, functionality, aesthetics, information, and subjective quality. Next, we computed descriptive statistics for each rating. To identify recommendations for future research and app development, we categorized rater comments on the score driving factors for each item and what the app could have done to improve the score for that item. RESULTS We evaluated 17 apps (41% iOS only, 12% Android only, 47% both iOS and Android). We found that on average, the apps are of minimally acceptable quality. Although we identified apps above and below minimally acceptable quality, many apps had broken features and were rated as below acceptable for engagement and information. CONCLUSIONS Minimally acceptable quality is likely insufficient to meet care partner needs. Future research should establish minimum quality standards across dimensions for mobile apps for care partners. The design features of high-quality apps we identified in this research can provide the foundation for benchmarking those standards.

Download Full-text

Toward Gamified Pain Management Apps: Mobile Application Rating Scale–Based Quality Assessment of Pain-Mentor’s First Prototype Through an Expert Study

JMIR Formative Research ◽

10.2196/13170 ◽

2020 ◽

Vol 4 (5) ◽

pp. e13170

Author(s):

Alexandra Hoffmann ◽

Corinna A Faust-Christmann ◽

Gregor Zolynski ◽

Gabriele Bleser

Keyword(s):

Chronic Pain ◽

Pain Management ◽

Mobile Application ◽

Rating Scale ◽

Pain Treatment ◽

Semistructured Interview ◽

Content Quality ◽

Management Techniques ◽

Health Experts

Background The use of health apps to support the treatment of chronic pain is gaining importance. Most available pain management apps are still lacking in content quality and quantity as their developers neither involve health experts to ensure target group suitability nor use gamification to engage and motivate the user. To close this gap, we aimed to develop a gamified pain management app, Pain-Mentor. Objective To determine whether medical professionals would approve of Pain-Mentor’s concept and content, this study aimed to evaluate the quality of the app’s first prototype with experts from the field of chronic pain management and to discover necessary improvements. Methods A total of 11 health professionals with a background in chronic pain treatment and 2 mobile health experts participated in this study. Each expert first received a detailed presentation of the app. Afterward, they tested Pain-Mentor and then rated its quality using the mobile application rating scale (MARS) in a semistructured interview. Results The experts found the app to be of excellent general (mean 4.54, SD 0.55) and subjective quality (mean 4.57, SD 0.43). The app-specific section was rated as good (mean 4.38, SD 0.75). Overall, the experts approved of the app’s content, namely, pain and stress management techniques, behavior change techniques, and gamification. They believed that the use of gamification in Pain-Mentor positively influences the patients’ motivation and engagement and thus has the potential to promote the learning of pain management techniques. Moreover, applying the MARS in a semistructured interview provided in-depth insight into the ratings and concrete suggestions for improvement. Conclusions The experts rated Pain-Mentor to be of excellent quality. It can be concluded that experts perceived the use of gamification in this pain management app in a positive manner. This showed that combining pain management with gamification did not negatively affect the app’s integrity. This study was therefore a promising first step in the development of Pain-Mentor.

Download Full-text

Measuring the Quality of Clinical Skills Mobile Apps for Student Learning: Systematic Search, Analysis, and Comparison of Two Measurement Scales (Preprint)

10.2196/preprints.25377 ◽

2020 ◽

Author(s):

Tehmina Gladman ◽

Grace Tylee ◽

Steve Gallagher ◽

Jonathan Mair ◽

Rebecca Grainger

Keyword(s):

Student Learning ◽

Teaching And Learning ◽

Rating Scale ◽

Clinical Skills ◽

Mobile Apps ◽

Mobile App ◽

Systematic Search ◽

Measurement Scales ◽

High Quality

BACKGROUND Mobile apps are widely used in health professions, which increases the need for simple methods to determine the quality of apps. In particular, teachers need the ability to curate high-quality mobile apps for student learning. OBJECTIVE This study aims to systematically search for and evaluate the quality of clinical skills mobile apps as learning tools. The quality of apps meeting the specified criteria was evaluated using two measures—the widely used Mobile App Rating Scale (MARS), which measures general app quality, and the Mobile App Rubric for Learning (MARuL), a recently developed instrument that measures the value of apps for student learning—to assess whether MARuL is more effective than MARS in identifying high-quality apps for learning. METHODS Two mobile app stores were systematically searched using clinical skills terms commonly found in medical education and apps meeting the criteria identified using an approach based on PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A total of 9 apps were identified during the screening process. The apps were rated independently by 2 reviewers using MARS and MARuL. RESULTS The intraclass correlation coefficients (ICCs) for the 2 raters using MARS and MARuL were the same (MARS ICC [two-way]=0.68; P<.001 and MARuL ICC [two-way]=0.68; P<.001). Of the 9 apps, Geeky Medics-OSCE revision (MARS Android=3.74; MARS iOS=3.68; MARuL Android=75; and MARuL iOS=73) and OSCE PASS: Medical Revision (MARS Android=3.79; MARS iOS=3.71; MARuL Android=69; and MARuL iOS=73) scored highly on both measures of app quality and for both Android and iOS. Both measures also showed agreement for the lowest rated app, Patient Education Institute (MARS Android=2.21; MARS iOS=2.11; MARuL Android=18; and MARuL iOS=21.5), which had the lowest scores in all categories except information (MARS) and professional (MARuL) in both operating systems. MARS and MARuL were both able to differentiate between the highest and lowest quality apps; however, MARuL was better able to differentiate apps based on teaching and learning quality. CONCLUSIONS This systematic search and rating of clinical skills apps for learning found that the quality of apps was highly variable. However, 2 apps—Geeky Medics-OSCE revision and OSCE PASS: Medical Revision—rated highly for both versions and with both quality measures. MARS and MARuL showed similar abilities to differentiate the quality of the 9 apps. However, MARuL’s incorporation of teaching and learning elements as part of a multidimensional measure of quality may make it more appropriate for use with apps focused on teaching and learning, whereas MARS’s more general rating of quality may be more appropriate for health apps targeting a general health audience. Ratings of the 9 apps by both measures also highlighted the variable quality of clinical skills mobile apps for learning. CLINICALTRIAL

Download Full-text

Mobile apps for detecting falsified and substandard drugs: A systematic review

PLoS ONE ◽

10.1371/journal.pone.0246061 ◽

2021 ◽

Vol 16 (2) ◽

pp. e0246061

Author(s):

Agustín Ciapponi ◽

Manuel Donato ◽

A. Metin Gülmezoglu ◽

Tomás Alconada ◽

Ariel Bardach

Keyword(s):

Systematic Review ◽

Rating Scale ◽

Mobile Apps ◽

Mobile App ◽

Economic Consequences ◽

Google Scholar ◽

Medical Database ◽

Bar Code ◽

Medical Databases

The use of substandard and counterfeit medicines (SCM) leads to significant health and economic consequences, like treatment failure, rise of antimicrobial resistance, extra expenditures of individuals or households and serious adverse drug reactions including death. Our objective was to systematically search, identify and compare relevant available mobile applications (apps) for smartphones and tablets, which use could potentially affect clinical and public health outcomes. We carried out a systematic review of the literature in January 2020, including major medical databases, and app stores. We used the validated Mobile App Rating Scale (MARS) to assess the quality of apps, (1 worst score, 3 acceptable score, and 5 best score). We planned to evaluate the accuracy of the mobile apps to detect SCM. We retrieved 335 references through medical databases and 42 from Apple, Google stores and Google Scholar. We finally included two studies of the medical database, 25 apps (eight from the App Store, eight from Google Play, eight from both stores, and one from Google Scholar), and 16 websites. We only found one report on the accuracy of a mobile apps detecting SCMs. Most apps use the imprint, color or shape for pill identification, and only a few offer pill detection through photographs or bar code. The MARS mean score for the apps was 3.17 (acceptable), with a maximum of 4.9 and a minimum of 1.1. The ‘functionality’ dimension resulted in the highest mean score (3.4), while the ‘engagement’ and ‘information’ dimensions showed the lowest one (3.0). In conclusion, we found a remarkable evidence gap about the accuracy of mobile apps in detecting SCMs. However, mobile apps could potentially be useful to screen for SCM by assessing the physical characteristics of pills, although this should still be assessed in properly designed research studies.

Download Full-text

Evaluation of Android and Apple Store Depression Applications Based on Mobile Application Rating Scale

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph182312505 ◽

2021 ◽

Vol 18 (23) ◽

pp. 12505

Author(s):

Jaime Martín-Martín ◽

Antonio Muro-Culebras ◽

Cristina Roldán-Jiménez ◽

Adrian Escriche-Escuder ◽

Irene De-Torres ◽

...

Keyword(s):

Health Status ◽

Mobile Application ◽

Rating Scale ◽

Behavioral Changes ◽

Average Score ◽

App Store ◽

The United Kingdom ◽

Treatment Of Depression ◽

Mobile Health Applications

There are a large number of mobile applications that allow the monitoring of health status. The quality of the applications is only evaluated by users and not by standard criteria. This study aimed to examine depression-related applications in major mobile application stores and to analyze them using the rating scale tool Mobile Application Rating Scale (MARS). A search of digital applications for the control of symptoms and behavioral changes in depression was carried out in the two reference mobile operating systems, Apple (App Store) and Android (Play Store), by means of two reviewers with a blind methodology between September and October 2019 in stores from Spain and the United Kingdom. Eighteen applications from the Android Play Store and twelve from the App Store were included in this study. The quality of the applications was evaluated using the MARS scale from 1 (inadequate) to 5 (excellent). The average score of the applications based on the MARS was 3.67 ± 0.53. The sections with the highest scores were “Functionality” (4.51) and “Esthetics” (3.98) and the lowest “Application Subjective quality” (2.86) and “Information” (3.08). Mobile Health applications for the treatment of depression have great potential to influence the health status of users; however, applications come to the digital market without health control.

Download Full-text