Content and Quality of Infant Feeding Smartphone Apps: Five-Year Update on a Systematic Search and Evaluation (Preprint)

BACKGROUND Parents use apps to access information on child health, but there are no standards for providing evidence-based advice, support, and information. Well-developed apps that promote appropriate infant feeding and play can support healthy growth and development. A 2015 systematic assessment of smartphone apps in Australia about infant feeding and play found that most apps had minimal information, with poor readability and app quality. OBJECTIVE This study aimed to systematically evaluate the information and quality of smartphone apps providing information on breastfeeding, formula feeding, introducing solids, or infant play for consumers. METHODS The Google Play store and Apple App Store were searched for free and paid Android and iPhone Operating System (iOS) apps using keywords for infant feeding, breastfeeding, formula feeding, and tummy time. The apps were evaluated between September 2018 and January 2019 for information content based on Australian guidelines, app quality using the 5-point Mobile App Rating Scale, readability, and suitability of health information. RESULTS A total of 2196 unique apps were found and screened. Overall, 47 apps were evaluated, totaling 59 evaluations for apps across both the Android and iOS platforms. In all, 11 apps had affiliations to universities and health services as app developers, writers, or editors. Furthermore, 33 apps were commercially developed. The information contained within the apps was poor: 64% (38/59) of the evaluations found no or low coverage of information found in the Australian guidelines on infant feeding and activity, and 53% (31/59) of the evaluations found incomplete or incorrect information with regard to the depth of information provided. Subjective app assessment by health care practitioners on whether they would use, purchase, or recommend the app ranged from poor to acceptable (median 2.50). Objective assessment of the apps’ engagement, functionality, aesthetics, and information was scored as acceptable (median 3.63). The median readability score for the apps was at the American Grade 8 reading level. The suitability of health information was rated superior or adequate for content, reading demand, layout, and interaction with the readers. CONCLUSIONS The quality of smartphone apps on infant feeding and activity was moderate based on the objective measurements of engagement, functionality, aesthetics, and information from a reliable source. The overall quality of information on infant feeding and activity was poor, indicated by low coverage of topics and incomplete or partially complete information. The key areas for improvement involved providing evidence-based information consistent with the Australian National Health and Medical Research Council’s Infant Feeding Guidelines. Apps supported and developed by health care professionals with adequate health service funding can ensure that parents are provided with credible and reliable resources.

Download Full-text

Content and Quality of Infant Feeding Smartphone Apps: Five-Year Update on a Systematic Search and Evaluation

JMIR mhealth and uhealth ◽

10.2196/17300 ◽

2020 ◽

Vol 8 (5) ◽

pp. e17300 ◽

Cited By ~ 2

Author(s):

Heilok Cheng ◽

Alison Tutt ◽

Catherine Llewellyn ◽

Donna Size ◽

Jennifer Jones ◽

...

Keyword(s):

Health Care ◽

Health Information ◽

Infant Feeding ◽

Rating Scale ◽

Mobile App ◽

Formula Feeding ◽

Evidence Based ◽

Smartphone Apps ◽

Low Coverage

Background Parents use apps to access information on child health, but there are no standards for providing evidence-based advice, support, and information. Well-developed apps that promote appropriate infant feeding and play can support healthy growth and development. A 2015 systematic assessment of smartphone apps in Australia about infant feeding and play found that most apps had minimal information, with poor readability and app quality. Objective This study aimed to systematically evaluate the information and quality of smartphone apps providing information on breastfeeding, formula feeding, introducing solids, or infant play for consumers. Methods The Google Play store and Apple App Store were searched for free and paid Android and iPhone Operating System (iOS) apps using keywords for infant feeding, breastfeeding, formula feeding, and tummy time. The apps were evaluated between September 2018 and January 2019 for information content based on Australian guidelines, app quality using the 5-point Mobile App Rating Scale, readability, and suitability of health information. Results A total of 2196 unique apps were found and screened. Overall, 47 apps were evaluated, totaling 59 evaluations for apps across both the Android and iOS platforms. In all, 11 apps had affiliations to universities and health services as app developers, writers, or editors. Furthermore, 33 apps were commercially developed. The information contained within the apps was poor: 64% (38/59) of the evaluations found no or low coverage of information found in the Australian guidelines on infant feeding and activity, and 53% (31/59) of the evaluations found incomplete or incorrect information with regard to the depth of information provided. Subjective app assessment by health care practitioners on whether they would use, purchase, or recommend the app ranged from poor to acceptable (median 2.50). Objective assessment of the apps’ engagement, functionality, aesthetics, and information was scored as acceptable (median 3.63). The median readability score for the apps was at the American Grade 8 reading level. The suitability of health information was rated superior or adequate for content, reading demand, layout, and interaction with the readers. Conclusions The quality of smartphone apps on infant feeding and activity was moderate based on the objective measurements of engagement, functionality, aesthetics, and information from a reliable source. The overall quality of information on infant feeding and activity was poor, indicated by low coverage of topics and incomplete or partially complete information. The key areas for improvement involved providing evidence-based information consistent with the Australian National Health and Medical Research Council’s Infant Feeding Guidelines. Apps supported and developed by health care professionals with adequate health service funding can ensure that parents are provided with credible and reliable resources.

Download Full-text

Comprehensively addressing postpartum maternal health: a content and image review of commercially available mobile health apps

BMC Pregnancy and Childbirth ◽

10.1186/s12884-021-03785-7 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Laura Tucker ◽

Alan Cuevas Villagomez ◽

Tamar Krishnamurti

Keyword(s):

Maternal Health ◽

Health Information ◽

Mobile Health ◽

Rating Scale ◽

The United States ◽

People Of Color ◽

Evidence Based ◽

Or Education ◽

Google Play

Abstract Background The United States is currently facing a maternal morbidity and mortality crisis, with the highest rates of any resource-rich nation. In efforts to address this, new guidelines for postpartum care suggest that mobile health (mHealth) apps can help provide complementary clinical support for new mothers during the postpartum period. However, to date no study has evaluated the quality of existing mHealth tools targeted to this time period in terms of sufficiency of maternal health information, inclusivity of people of color, and app usability. Methods Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards were used to review the peripartum apps from the Apple and Google Play stores in either the Health/Fitness, Medical, or Education categories. Apps were evaluated for extent and quality of maternal health information and inclusivity of people of color using an a priori coding scheme. App usability was evaluated using the Mobile Application Rating Scale (MARS) score. Results Of the 301 apps from the Apple and Google Play stores, 25 met criteria for final evaluation. Of the 30 maternal health topics coded for, the median number addressed by apps was 19.5 (65%). Peripartum behaviors were more frequently addressed than peripartum outpatient care topics and peripartum acute health risks. The coverage of maternal health information and inclusivity of people of color in app imagery both correlated positively with the MARS usability score of the app. Only 8 apps (32%) portrayed greater than 24% images of people of color- the percent of non-white Americans according to 2019 census estimates. There was no correlation between MARS usability score and number of app users, as estimated by number of ratings for the app available on the app store. In addition, apps with evidence-based maternal health information had greater MARS engagement, information, and aesthetics scores. However, presence of evidence-based information did not correlate with greater numbers of app users. Conclusions Current commercially available peripartum apps range widely in quality. Overall current app offerings generally do not provide adequate maternal health information and are not optimally accessible to the target users in terms of inclusivity of women of color or app usability. Apps delivering evidence-based information and more usable design are more likely to meet these standards but are not more likely to be downloaded by users.

Download Full-text

A content analysis of the quality and behaviour change techniques of smartphone apps promoting the Mediterranean diet

Proceedings of The Nutrition Society ◽

10.1017/s0029665120002190 ◽

2020 ◽

Vol 79 (OCE2) ◽

Author(s):

Daniel McAleese ◽

Angeliki Papadaki

Keyword(s):

Content Analysis ◽

Mediterranean Diet ◽

Behaviour Change ◽

Rating Scale ◽

Mobile App ◽

Subjective Quality ◽

Behaviour Change Techniques ◽

Smartphone Apps ◽

The Mediterranean

AbstractThe popularity of smartphone apps is rising globally. However, the quality of widely available health apps and their effectiveness to promote behaviour change, by incorporating behavioural change techniques (BCTs) that have been suggested to result in behaviour change, is questionable. The aim of this study was to conduct a content analysis of smartphone apps that utilise the Mediterranean diet (MD), a dietary pattern linked to numerous health benefits, to promote healthy dietary behaviours. The specific objectives were to evaluate the quality of these apps and their use of BCTs. The iTunes Store and Google Play were searched for all available apps (free and requiring payment to download) promoting the MD. Apps were excluded if they were not available in English, did not focus only on the MD or the download process was corrupt. The Mobile App Rating Scale (MARS) was used to assess app engagement, functionality, aesthetics and information quality (the average of which represents a quality mean score (QMS)) and subjective quality. The Abraham & Michie behaviour change taxonomy was adapted to create a 20-item checklist to assess the presence of BCTs in the eligible apps. Thirty nine apps (64.1% of which were free to download) were analysed. The overall QMS was 2.96 ± 0.46. The lowest and highest scored QMS constructs were engagement (2.42 ± 0.62) and functionality (3.95 ± 0.45), respectively, whilst the overall mean subjective quality score was 1.71 ± 0.51 (scores’ range 1–5). Apps were scored higher for aesthetics if they required payment (3.14 vs. 2.78, p = 0.047). Apps incorporated an average of 3.10 ± 1.87 BCTs, with no difference in the number of BCTs present according to platform or cost. From the five BCTs that have been suggested to promote behaviour change, ‘self-monitoring’ was present in six apps, ‘intention formation’ in 15 and ‘goal setting’ in one. None of the apps incorporated the ‘review of behavioural goals’ and ‘feedback on performance’ BCTs. The total QMS score was positively associated with the presence of BCTs (r = 0.409, p = 0.008). Based on the MARS assessment, apps promoting the MD that are available in the public domain were of moderate quality. In contrast, the low presence, particularly of efficacious, BCTs and low subjective quality scores suggest that apps are actually of poor quality and potentially not effective in promoting behaviour change towards an MD diet. These findings provide important insights for the future development of high-quality apps to promote the MD at population level.

Download Full-text

Mobile Apps for Speech-Language Therapy in Adults With Communication Disorders: Review of Content and Quality

JMIR mhealth and uhealth ◽

10.2196/18858 ◽

2020 ◽

Vol 8 (10) ◽

pp. e18858

Author(s):

Atiyeh Vaezipour ◽

Jessica Campbell ◽

Deborah Theodoros ◽

Trevor Russell

Keyword(s):

Rating Scale ◽

Mobile Apps ◽

Communication Disorders ◽

Mobile App ◽

Evidence Based ◽

Language Therapy ◽

Eligibility Criteria ◽

App Store ◽

Google Play

Background Worldwide, more than 75% of people with acquired brain injury (ABI) experience communication disorders. Communication disorders are impairments in the ability to communicate effectively, that is, sending, receiving, processing, and comprehending verbal and nonverbal concepts and symbols. Such disorders may have enduring impacts on employment, social participation, and quality of life. Technology-enabled interventions such as mobile apps have the potential to increase the reach of speech-language therapy to treat communication disorders. However, ensuring that apps are evidence-based and of high quality is critical for facilitating safe and effective treatment for adults with communication disorders. Objective The aim of this review is to identify mobile apps that are currently widely available to adults with communication disorders for speech-language therapy and to assess their content and quality using the validated Mobile App Rating Scale (MARS). Methods Google Play Store, Apple App Store, and webpages were searched to identify mobile apps for speech-language therapy. Apps were included in the review if they were designed for the treatment of adult communication disorders after ABI, were in English, and were either free or for purchase. Certified speech-language pathologists used the MARS to assess the quality of the apps. Results From a total of 2680 apps identified from Google Play Store, Apple App Store, and web searches, 2.61% (70/2680) apps met the eligibility criteria for inclusion. Overall, 61% (43/70) were available for download on the iPhone Operating System (iOS) platform, 20% (14/70) on the Android platform, and 19% (13/70) on both iOS and Android platforms. A content analysis of the apps revealed 43 apps for language, 17 apps for speech, 8 apps for cognitive communication, 6 apps for voice, and 5 apps for oromotor function or numeracy. The overall MARS mean score was 3.7 out of 5, SD 0.6, ranging between 2.1 and 4.5, with functionality being the highest-scored subscale (4.3, SD 0.6), followed by aesthetics (3.8, SD 0.8), information (3.4, SD 0.6), and engagement (3.3, SD 0.6). The top 5 apps were Naming Therapy (4.6/5), Speech Flipbook Standard (4.6/5), Number Therapy (4.5/5), Answering Therapy, and Constant Therapy (4.4/5). Conclusions To our knowledge, this is the first study to systematically identify and evaluate a broad range of mobile apps for speech-language therapy for adults with communication disorders after sustaining ABI. We found a lack of interactive and engaging elements in the apps, a critical factor in sustaining self-managed speech-language therapy. More evidence-based apps with a focus on human factors, user experience, and a patient-led design approach are required to enhance effectiveness and long-term use.

Download Full-text

Quality of Psychoeducational Apps for Military Members With Mild Traumatic Brain Injury: An Evaluation Utilizing the Mobile Application Rating Scale (Preprint)

10.2196/preprints.19807 ◽

2020 ◽

Author(s):

Chelsea Jones ◽

Kaitlin O'Toole ◽

Kevin Jones ◽

Suzette Brémault-Phillips

Keyword(s):

Health Care ◽

Military Personnel ◽

Health Care Professionals ◽

Mobile Application ◽

Best Practice ◽

Rating Scale ◽

Evidence Based ◽

Smartphone Apps ◽

Military Members ◽

The Us

BACKGROUND Military personnel have an elevated risk of sustaining mild traumatic brain injuries (mTBI) and postconcussion symptoms (PCS). Smartphone apps that provide psychoeducation may assist those with mTBI or PCS to overcome unique barriers that military personnel experience with stigma and access to health care resources. OBJECTIVE This study aims to (1) use the Mobile Application Rating Scale (MARS) to evaluate smartphone apps purporting to provide psychoeducation for those who have sustained an mTBI or a PCS; (2) explore the relevance, utility, and effectiveness of these apps in facilitating symptom management and overall recovery from mTBI and PCS among military personnel; and (3) discuss considerations pertinent to health care professionals and patients with mTBI when considering the use of mobile health (mHealth), including apps for mTBI psychoeducation. METHODS A five-step systematic search for smartphone apps for military members with mTBI or PCS was conducted on January 31, 2020. Cost-free apps meeting the inclusion criteria were evaluated using the MARS and compared with evidence-based best practice management protocols for mTBI and PCS. RESULTS The search yielded a total of 347 smartphone apps. After applying the inclusion and exclusion criteria, 13 apps were subjected to evaluation. Two apps were endorsed by the US Department of Veterans Affairs and the US Department of Defense; all the others (n=11) were developed for civilians. When compared with evidence-based best practice resources, the apps provided various levels of psychoeducational content. There are multiple considerations that health care professionals and those who sustain an mTBI or a PCS have to consider when choosing to use mHealth and selecting a specific app for mTBI psychoeducation. These may include factors such as the app platform, developer, internet requirement, cost, frequency of updates, language, additional features, acknowledgment of mental health, accessibility, military specificity, and privacy and security of data. CONCLUSIONS Psychoeducational interventions have a good evidence base as a treatment for mTBI and PCS. The use of apps for this purpose may be clinically effective, cost-effective, confidential, user friendly, and accessible. However, more research is needed to explore the effectiveness, usability, safety, security, and accessibility of apps designed for mTBI management.

Download Full-text

Breast Cancer-Related Apps in Google Play and App Store: Evaluate of Their Functionality and Quality

10.21203/rs.3.rs-448075/v1 ◽

2021 ◽

Author(s):

Aydanur AYDIN ◽

Ayla GÜRSOY

Keyword(s):

Breast Cancer ◽

Mobile Applications ◽

Rating Scale ◽

Mobile App ◽

Innovative Technology ◽

Average Score ◽

Evidence Based ◽

Health Technologies ◽

App Store

Abstract Aim: This study aimed to evaluate the functionality and quality of breast cancer-related apps which can be accessed from Turkey. Methods: The research covers 707 mobile applications, which were searched using the keywords "cancer", "oncology" and "breast cancer" in Google and App store stores between January and June 2019. The quality of apps was assessed with the user version of the Mobile App Rating Scale (MARS). Results: Mobile applications focus on communication, education and treatment order / plan categories. The average MARS quality score for applications in all of the two stores was determined as 3.42. According to the average score of the scale sub-dimensions; It was seen that they were ranked as functionality, aesthetics, information quality and participation. MARS scale sub-groups were listed as functionality, aesthetics, knowledge quality, and participation. Conclusion: e-Health is a subject that has just started to be used actively in health. Although basic information about breast cancer was used, information on prevention, early diagnosis and precautions was relatively limited and was not personalized. Support from healthcare professionals and further efforts should be made to develop evidence-based breast cancer knowledge and innovative technology and applications regarding cancer. Implications for practice: e-Health technologies are a good tool in breast cancer, but appear to lack evidence-based content and individual specific areas such as age and education level.

Download Full-text

The Impact of Health Information Technology on the Quality of Medical and Health Care: A Systematic Review

Health Information Management Journal ◽

10.1177/183335830903800305 ◽

2009 ◽

Vol 38 (3) ◽

pp. 26-37 ◽

Cited By ~ 90

Author(s):

Aziz Jamal ◽

Kirsten McKenzie ◽

Michele Clark

Keyword(s):

Health Care ◽

Information Technology ◽

Health Information Technology ◽

Health Information ◽

Patient Outcomes ◽

Evidence Based ◽

Specific Patient ◽

Evidence Based Guidelines ◽

The Impact

The aim of this study was to systematically review the published evidence of the impact of health information technology (HIT) or health information systems (HIS) on the quality of healthcare, focusing on clinicians' adherence to evidence-based guidelines and the corresponding impact this had on patient clinical outcomes. The review covered the use of health information technologies and systems in both medical care (i.e. clinical and surgical) and other areas such as allied health and preventive services. Studies were included in the review if they examined the impact of Electronic Health Record (EHR), Computerised Provider Order-Entry (CPOE), or Decision Support System (DS); and if the primary outcomes of the studies were focused on the level of compliance with evidence-based guidelines among clinicians. Measurements considered relevant to the review were either of changes in clinical processes resulting from a change of the providers' behaviour, or of specific patient outcomes that demonstrated the effectiveness of a particular treatment given by providers. Of 23 studies included in the current review, 17 assessed the impact of HIT/HIS on health care practitioners' performance. A positive improvement, in relation to their compliance with evidence-based guidelines, was seen in 14 studies. Studies that included an assessment of patient outcomes, however, showed insufficient evidence of either clinically or statistically important improvements. Although the number of studies reviewed was relatively small, the findings demonstrated consistency with similar previous reviews of this nature in that wide scale use of HIT has been shown to increase clinician's adherence to guidelines.

Download Full-text

Weight Management Apps in Saudi Arabia: Evaluation of Features and Quality (Preprint)

10.2196/preprints.19844 ◽

2020 ◽

Author(s):

Dalal M Alshathri ◽

Abeer S Alhumaimeedy ◽

Ghada Al-Hudhud ◽

Aseel Alsaleh ◽

Sara Al-Musharaf ◽

...

Keyword(s):

Weight Management ◽

Six Sigma ◽

Rating Scale ◽

Mobile App ◽

Overweight And Obesity ◽

Evidence Based ◽

User Perceptions ◽

App Store ◽

The Mean

BACKGROUND Weight management apps may provide support and management options for individuals with overweight and obesity. Research on the quality of weight management mHealth apps among the Saudi population is insufficient despite frequent use. OBJECTIVE The aims of this study were to explore user perceptions of weight management apps, explore reasons for starting and stopping app use, appraise the quality of weight management apps available in the App Store, and compare the features currently available within the app market and those that are most desirable to weight management app users. METHODS A web-based survey consisted of 31 open and closed questions about sociodemographic information, general health questions, app use, app user perceptions, and discontinuation of app use. The quality of the weight management apps available on the App Store was assessed using the Mobile App Rating Scale and evidence-based strategies. We also used six sigma evaluations to ensure that the quality measured by the tools consistently meets customer expectations. RESULTS Data from the survey were analyzed. Of the respondents, 30.17% (324/1074) had used a weight management app, 18.16% (195/1074) used the apps and stopped, and 51.68% (555/1074) had never used a weight management app. Of apps mentioned, 23 met the inclusion criteria. The overall average Mobile App Rating Scale quality of apps was acceptable; 30% (7/23) received a quality mean score of 4 or higher (out of 5), and 30% (7/23) did not meet the acceptability score of 3 or higher. Evidence-based strategy results showed that feedback was not observed in any of the apps, and motivation strategy was observed in only 1 app. The sigma results of evidence-based strategies reflect that most of the apps fail to pass the mean. CONCLUSIONS App users desired a feature that allows them to communicate with a specialist, which is a missing in the available free apps. Despite the large number and accessibility of weight management apps, the quality and features of most are variable. It can be concluded from six sigma results that passing the mean does not ensure that the quality is consistently distributed through all app quality properties and Mobile App Rating Scale and evidence-based strategies do not give developers an indication of the acceptance of their apps by mobile users. This finding stresses the importance of reevaluating the passing criterion, which is ≥50% for designing an effective app.

Download Full-text

Assessing the Quality of Mobile Health-Related Apps: Interrater Reliability Study of Two Guides (Preprint)

10.2196/preprints.26471 ◽

2020 ◽

Author(s):

Jordi Miró ◽

Pere Llorens-Vernet

Keyword(s):

Health Care ◽

Health Care Professionals ◽

Interrater Reliability ◽

Rating Scale ◽

Mobile App ◽

App Development ◽

Different Types ◽

Health Related ◽

Medical Category

BACKGROUND There is a huge number of health-related apps available, and the numbers are growing fast. However, many of them have been developed without any kind of quality control. In an attempt to contribute to the development of high-quality apps and enable existing apps to be assessed, several guides have been developed. OBJECTIVE The main aim of this study was to study the interrater reliability of a new guide — the Mobile App Development and Assessment Guide (MAG) — and compare it with one of the most used guides in the field, the Mobile App Rating Scale (MARS). Moreover, we also focused on whether the interrater reliability of the measures is consistent across multiple types of apps and stakeholders. METHODS In order to study the interrater reliability of the MAG and MARS, we evaluated the 4 most downloaded health apps for chronic health conditions in the medical category of IOS and Android devices (ie, App Store and Google Play). A group of 8 reviewers, representative of individuals that would be most knowledgeable and interested in the use and development of health-related apps and including different types of stakeholders such as clinical researchers, engineers, health care professionals, and end users as potential patients, independently evaluated the quality of the apps using the MAG and MARS. We calculated the Krippendorff alpha for every category in the 2 guides, for each type of reviewer and every app, separately and combined, to study the interrater reliability. RESULTS Only a few categories of the MAG and MARS demonstrated a high interrater reliability. Although the MAG was found to be superior, there was considerable variation in the scores between the different types of reviewers. The categories with the highest interrater reliability in MAG were “Security” (α=0.78) and “Privacy” (α=0.73). In addition, 2 other categories, “Usability” and “Safety,” were very close to compliance (health care professionals: α=0.62 and 0.61, respectively). The total interrater reliability of the MAG (ie, for all categories) was 0.45, whereas the total interrater reliability of the MARS was 0.29. CONCLUSIONS This study shows that some categories of MAG have significant interrater reliability. Importantly, the data show that the MAG scores are better than the ones provided by the MARS, which is the most commonly used guide in the area. However, there is great variability in the responses, which seems to be associated with subjective interpretation by the reviewers.

Download Full-text

Assessing the Quality of Mobile Health-Related Apps: Interrater Reliability Study of Two Guides

JMIR mhealth and uhealth ◽

10.2196/26471 ◽

2021 ◽

Vol 9 (4) ◽

pp. e26471

Author(s):

Jordi Miró ◽

Pere Llorens-Vernet

Keyword(s):

Health Care ◽

Health Care Professionals ◽

Interrater Reliability ◽

Rating Scale ◽

Mobile App ◽

App Development ◽

Different Types ◽

Health Related ◽

Medical Category

Background There is a huge number of health-related apps available, and the numbers are growing fast. However, many of them have been developed without any kind of quality control. In an attempt to contribute to the development of high-quality apps and enable existing apps to be assessed, several guides have been developed. Objective The main aim of this study was to study the interrater reliability of a new guide — the Mobile App Development and Assessment Guide (MAG) — and compare it with one of the most used guides in the field, the Mobile App Rating Scale (MARS). Moreover, we also focused on whether the interrater reliability of the measures is consistent across multiple types of apps and stakeholders. Methods In order to study the interrater reliability of the MAG and MARS, we evaluated the 4 most downloaded health apps for chronic health conditions in the medical category of IOS and Android devices (ie, App Store and Google Play). A group of 8 reviewers, representative of individuals that would be most knowledgeable and interested in the use and development of health-related apps and including different types of stakeholders such as clinical researchers, engineers, health care professionals, and end users as potential patients, independently evaluated the quality of the apps using the MAG and MARS. We calculated the Krippendorff alpha for every category in the 2 guides, for each type of reviewer and every app, separately and combined, to study the interrater reliability. Results Only a few categories of the MAG and MARS demonstrated a high interrater reliability. Although the MAG was found to be superior, there was considerable variation in the scores between the different types of reviewers. The categories with the highest interrater reliability in MAG were “Security” (α=0.78) and “Privacy” (α=0.73). In addition, 2 other categories, “Usability” and “Safety,” were very close to compliance (health care professionals: α=0.62 and 0.61, respectively). The total interrater reliability of the MAG (ie, for all categories) was 0.45, whereas the total interrater reliability of the MARS was 0.29. Conclusions This study shows that some categories of MAG have significant interrater reliability. Importantly, the data show that the MAG scores are better than the ones provided by the MARS, which is the most commonly used guide in the area. However, there is great variability in the responses, which seems to be associated with subjective interpretation by the reviewers.

Download Full-text