Prioritization of Quality Principles for Health Apps Using the Kano Model: Survey Study

Background Health apps are often used without adequately taking aspects related to their quality under consideration. This may partially be due to inadequate awareness about necessary criteria and how to prioritize them when evaluating an app. Objective The aim of this study was to introduce a method for prioritizing quality attributes in the mobile health context. To this end, physicians were asked about their assessment of nine app quality principles relevant in health contexts and their responses were used as a basis for designing a method for app prioritization. Ultimately, the goal was to aid in making better use of limited resources (eg, time) by assisting with the decision as to the specific quality principles that deserve priority in everyday medical practice and those that can be given lower priority, even in cases where the overall principles are rated similarly. Methods A total of 9503 members of two German professional societies in the field of orthopedics were invited by email to participate in an anonymous online survey over a 1-month period. Participants were asked to rate a set of nine app quality principles using a Kano survey with functional and dysfunctional (ie, positively and negatively worded) questions. The evaluation was based on the work of Kano (baseline), supplemented by a self-designed approach. Results Among the 9503 invited members, 382 completed relevant parts of the survey (return rate of 4.02%). These participants were equally and randomly assigned to two groups (test group and validation group, n=191 each). Demographic characteristics did not significantly differ between groups (all P>.05). Participants were predominantly male (328/382, 85.9%) and older than 40 years (290/382, 75.9%). Given similar ratings, common evaluation strategies for Kano surveys did not allow for conclusive prioritization of the principles, and the same was true when using the more elaborate approach of satisfaction and dissatisfaction indices following the work of Timko. Therefore, an extended, so-called “in-line-of-sight” method was developed and applied for this evaluation. Modified from the Timko method, this approach is based on a “point of view” (POV) metric, which generates a ranking coefficient. Although the principles were previously almost exclusively rated as must-be (with the exception of resource efficiency), which was not conducive to their prioritization, the new method applied from the must-be POV resulted in identical rankings for the test and validation groups: (1) legal conformity, (2) content validity, (3) risk adequacy, (4) practicality, (5) ethical soundness, (6) usability, (7) transparency, (8) technical adequacy, and (9) resource efficiency. Conclusions Established survey methodologies based on the work of Kano predominantly seek to categorize the attributes to be evaluated. The methodology presented here is an interesting option for prioritization, and enables focusing on the most important criteria, thus saving valuable time when reviewing apps for use in the medical field, even with otherwise largely similar categorization results. The extent to which this approach is applicable beyond the scenario presented herein requires further investigation.

Designing Effective eHealth Interventions for Underserved Groups: Five Lessons From a Decade of eHealth Intervention Design and Deployment

Despite the proliferation of eHealth interventions, such as web portals, for health information dissemination or the use of mobile apps and wearables for health monitoring, research has shown that underserved groups do not benefit proportionately from these eHealth interventions. This is largely because of usability issues and the lack of attention to the broader structural, physical, and psychosocial barriers to technology adoption and use. The objective of this paper is to draw lessons from a decade of experience in designing different user-centered eHealth interventions (eg, web portals and health apps) to inform future work in leveraging technology to address health disparities. We draw these lessons from a series of interventions from the work we have done over 15 years in the Viswanath laboratory at the Dana-Farber Cancer Institute and Harvard TH Chan School of Public Health, focusing on three projects that used web portals and health apps targeted toward underserved groups. The projects were the following: Click to Connect, which was a community-based eHealth intervention that aimed to improve internet skills and health literacy among underserved groups by providing home access to high-speed internet, computer, and internet training classes, as well as a dedicated health web portal with ongoing technical support; PLANET MassCONECT, which was a knowledge translation project that built capacity among community-based organizations in Boston, Lawrence, and Worcester in Massachusetts to adopt evidence-based health promotion programs; and Smartphone App for Public Health, which was a mobile health research that facilitated both participatory (eg, surveys) and passive data (eg, geolocations and web-browsing behaviors) collection for the purpose of understanding tobacco message exposure in individuals’ built environment. Through our work, we distilled five key principles for researchers aiming to design eHealth interventions for underserved groups. They are as follows: develop a strategic road map to address communication inequalities (ie, a concrete action plan to identify the barriers faced by underserved groups and customize specific solutions to each of them), engage multiple stakeholders from the beginning for the long haul, design with usability—readability and navigability—in mind, build privacy safeguards into eHealth interventions and communicate privacy–utility tradeoffs in simplicity, and strive for an optimal balance between open science aspirations and protection of underserved groups.

A Review of the Quality and Impact of Mobile Health Apps

Mobile health applications (apps) have transformed the possibilities for health promotion and disease self-management; however, their promise is not fully realized owing to their reliance on commercial ecosystems for development and distribution. This review provides an overview of the types of mobile health apps and describes key stakeholders in terms of how apps are used, developed, and regulated. I outline key challenges facing consumers, public health professionals, and policy makers in evaluating the quality of health apps and summarize what is known about the impact of apps on health outcomes and health equity. I suggest that factors within the wider mobile ecosystem largely define the impact of health apps and, most notably, practices around the collection and commercialization of user data. Finally, I suggest that upstream public health strategies, grounded in an understanding of corporate influences on health, are necessary to promote healthy digital environments in which mobile health app innovation can flourish. Expected final online publication date for the Annual Review of Public Health, Volume 43 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Gamifying App-Based Low-Intensity Psychological Interventions to Prevent Sports Injuries in Young Athletes: A Review and Some Guidelines

Sports injuries have become a real health concern. Particularly noticeable is the increasing number of severe sports injuries among young people. Sports injury (SI) is a multifactorial event where many internal and external, proximal and remote factors play a role in a recursive way, including physical and psychological variables. Accordingly, many voices expressing the need of tackling that and several prevention programs have arisen. Nevertheless, different barriers and limitations prevent a wide extension of well-controlled programs, closely monitored by highly specialized professionals in ordinary sports grass-root organizations. These have helped flourishing different low intensity (LI)-interventions and e-Health apps focusing on both physical warmup, training and fitness, and mental skills aimed at reducing athlete’s vulnerability to SIs. This kind of intervention usually uses self-administered techniques and/or non-specialized staff that can effectively monitoring the program. In fact, LI-interventions have shown to be effective coping with different health and psychological issues. However, these interventions face an important challenge: the lack of engagement people usually show. The current paper proposes how gamification can contribute to the engagement to such interventions. Based on the mechanics–dynamics–aesthetics framework to analyze game design, the paper suggests a set of guidelines app- and web-LI interventions aimed at preventing SIs should include to foster motivation and reduce attrition.

OP128 Improving Literature Searching For Evidence On Health Apps: The National Institute For Health And Care Excellence (NICE) MEDLINE And Embase (Ovid) Health Apps Search Filters

IntroductionLiterature searching for evidence on apps in bibliographic databases is challenging because they are often described with inconsistent terminology. Information Specialists from the United Kingdom's National Institute for Health and Care Excellence (NICE) have developed validated search filters for retrieving evidence about apps from MEDLINE and Embase (Ovid) reliably.MethodsMedical informatics journals were hand-searched to create a ‘gold standard’ set of app references. The gold standard set was divided into two sets. The development set provided the search terms for the filters. The filters were validated by calculating their recall against the validation set. Target recall was >90%.A case study was then conducted to compare the number-needed-to-read (NNR) of the filters with previous non-validated MEDLINE and Embase app search strategies used for the ‘MIB214 myCOPD app’ NICE topic. NNR is the number of references screened to find each relevant reference.ResultsThe MEDLINE and Embase filters achieved 98.6 percent and 98.5 percent recall against the validation set, respectively. In the case study they achieved 100 percent recall, reducing NNR from 348 to 147 in MEDLINE and from 456 to 271 in Embase.ConclusionsThe novel NICE health apps search filters retrieve evidence on apps from MEDLINE and Embase effectively and more efficiently than previous non-validated search strategies used at NICE.

Connecting Older Adults With Mental Health Apps: A Survey of Provider App and Education Material Use

Many older veterans have access to mobile devices and are interested in using apps for mental health self-management, but few have ever downloaded health apps. To address the need for awareness of and access to VA mental health apps, we developed patient educational materials aimed towards older (or novice) users of mobile devices. The present study explored health care providers’ and staff’s perceptions about use of mental health mobile applications (apps) with older veterans and examined potential utility of these patient educational materials. Requestors of mobile device education materials (N = 90) were surveyed when ordering materials and again 4 months later. Baseline and follow-up surveys assessed frequency of app recommendation, and comfort recommending apps. Baseline surveys examined perceived advantages of apps; follow-up surveys examined perceived utility of the educational materials. Descriptive statistics and qualitative analysis were conducted. Most requesters (68.5%) initially were not comfortable using apps, yet perceived many advantages to using apps and hoped materials could facilitate app use. At follow-up, requestors felt more comfortable recommending apps alongside our materials. Qualitative analysis revealed perceived advantages to using the education materials. The benefits of developing and disseminating educational materials for providers to share with older veterans helped support older veterans’ app use, and potentially increased providers’ comfort with and frequency of recommending apps to their older patients. Access to educational materials can mitigate discomfort among providers in recommending apps to older users and may bring about valuable discussions about apps which support mental health.