Medical Mobile App Classification Using the National Institute for Health and Care Excellence Evidence Standards Framework for Digital Health Technologies: Interrater Reliability Study

Background Clinical governance of medical mobile apps is challenging, and there is currently no standard method for assessing the quality of such apps. In 2018, the National Institute for Health and Care Excellence (NICE) developed a framework for assessing the required level of evidence for digital health technologies (DHTs), as determined by their clinical function. The framework can potentially be used to assess mobile apps, which are a subset of DHTs. To be used reliably in this context, the framework must allow unambiguous classification of an app’s clinical function. Objective The objective of this study was to determine whether mobile health apps could be reliably classified using the NICE evidence standards framework for DHTs. Methods We manually extracted app titles, screenshots, and content descriptions for all apps listed on the National Health Service (NHS) Apps Library website on July 12, 2019; none of the apps were downloaded. Using this information, 2 mobile health (mHealth) researchers independently classified each app to one of the 4 functional tiers (ie, 1, 2, 3a, and 3b) described in the NICE digital technologies evaluation framework. Coders also answered contextual questions from the framework to identify whether apps were deemed to be higher risk. Agreement between coders was assessed using Cohen κ statistic. Results In total, we assessed 76 apps from the NHS Apps Library. There was classification agreement for 42 apps. Of these, 0 apps were unanimously classified into Tier 1; 24, into Tier 2; 15, into Tier 3a; and 3, into Tier 3b. There was disagreement between coders in 34/76 cases (45%); interrater agreement was poor (Cohen κ=0.32, 95% CI 0.16-0.47). Further investigation of disagreements highlighted 5 main explanatory themes: apps that did not correspond to any tier, apps that corresponded to multiple tiers, ambiguous tier descriptions, ambiguous app descriptions, and coder error. Conclusions The current iteration of the NICE evidence standards framework for DHTs did not allow mHealth researchers to consistently and unambiguously classify digital health mobile apps listed on the NHS app library according to their functional tier.

Download Full-text

Medical Mobile App Classification Using the National Institute for Health and Care Excellence Evidence Standards Framework for Digital Health Technologies: Interrater Reliability Study (Preprint)

10.2196/preprints.17457 ◽

2019 ◽

Author(s):

Khine Nwe ◽

Mark Erik Larsen ◽

Natalie Nelissen ◽

David Chi-Wai Wong

Keyword(s):

Mobile Health ◽

Mobile Apps ◽

Digital Health ◽

Mobile App ◽

Evaluation Framework ◽

Tier 2 ◽

Level Of Evidence ◽

Health Technologies ◽

Mobile Health Apps ◽

Classification Agreement

BACKGROUND Clinical governance of medical mobile apps is challenging, and there is currently no standard method for assessing the quality of such apps. In 2018, the National Institute for Health and Care Excellence (NICE) developed a framework for assessing the required level of evidence for digital health technologies (DHTs), as determined by their clinical function. The framework can potentially be used to assess mobile apps, which are a subset of DHTs. To be used reliably in this context, the framework must allow unambiguous classification of an app’s clinical function. OBJECTIVE The objective of this study was to determine whether mobile health apps could be reliably classified using the NICE evidence standards framework for DHTs. METHODS We manually extracted app titles, screenshots, and content descriptions for all apps listed on the National Health Service (NHS) Apps Library website on July 12, 2019; none of the apps were downloaded. Using this information, 2 mobile health (mHealth) researchers independently classified each app to one of the 4 functional tiers (ie, 1, 2, 3a, and 3b) described in the NICE digital technologies evaluation framework. Coders also answered contextual questions from the framework to identify whether apps were deemed to be higher risk. Agreement between coders was assessed using Cohen κ statistic. RESULTS In total, we assessed 76 apps from the NHS Apps Library. There was classification agreement for 42 apps. Of these, 0 apps were unanimously classified into Tier 1; 24, into Tier 2; 15, into Tier 3a; and 3, into Tier 3b. There was disagreement between coders in 34/76 cases (45%); interrater agreement was poor (Cohen κ=0.32, 95% CI 0.16-0.47). Further investigation of disagreements highlighted 5 main explanatory themes: apps that did not correspond to any tier, apps that corresponded to multiple tiers, ambiguous tier descriptions, ambiguous app descriptions, and coder error. CONCLUSIONS The current iteration of the NICE evidence standards framework for DHTs did not allow mHealth researchers to consistently and unambiguously classify digital health mobile apps listed on the NHS app library according to their functional tier.

Download Full-text

Exclusion by Design: Lack of Information about Community Participation in COVID-19-related Mobile Apps and its Relationship to Digital Health Disparities (Preprint)

10.2196/preprints.30872 ◽

2021 ◽

Author(s):

Muhammed Yassin Idris ◽

Maya Korin ◽

Faven Araya ◽

Sayeeda Chowdhury ◽

Humberto Brown ◽

...

Keyword(s):

Community Participation ◽

Mobile Health ◽

Medical Information ◽

Mobile Apps ◽

Digital Health ◽

Community Based ◽

Health Technologies ◽

Community Needs ◽

App Development ◽

Communities Of Color

UNSTRUCTURED The rate and scale of transmission of COVID-19 overwhelmed healthcare systems worldwide, particularly in under-resourced communities of color that already faced a high prevalence of pre-existing health conditions. One way the health ecosystem has tried to address the pandemic is by creating mobile apps for telemedicine, dissemination of medical information, and disease tracking. As these new mobile health tools continue to be a primary format for healthcare, more attention needs to be given to their equitable distribution, usage, and accessibility. In this viewpoint collaboratively written by a community-based organization and a health app development research team, we present results of our systematic search and analysis of community engagement in mobile apps released between February and December 2020 to address the COVID-19 pandemic. We provide an overview of apps’ features and functionalities but could not find any publicly available information regarding whether these apps incorporated participation from communities of color disproportionately impacted by the pandemic. We argue that while mobile health technologies are a form of intellectual property, app developers should make public the steps taken to include community participation in app development. These steps could include community needs assessment, community feedback solicited and incorporated, and community participation in evaluation. These are factors that community-based organizations look for when assessing whether to promote digital health tools among the communities they serve. Transparency about the participation of community organizations in the process of app development would increase buy-in, trust, and usage of mobile health apps in communities where they are needed most.

Download Full-text

Mobile Health Apps for Medical Emergencies: Systematic Review

JMIR mhealth and uhealth ◽

10.2196/18513 ◽

2020 ◽

Vol 8 (12) ◽

pp. e18513

Author(s):

Alejandro Plaza Roncero ◽

Gonçalo Marques ◽

Beatriz Sainz-De-Abajo ◽

Francisco Martín-Rodríguez ◽

Carlos del Pozo Vegas ◽

...

Keyword(s):

Health Care ◽

Mobile Health ◽

Mobile Apps ◽

Current Status ◽

Health Technologies ◽

Health Apps ◽

App Store ◽

Mobile Health Apps ◽

Medical Emergencies ◽

Google Play

Background Mobile health apps are used to improve the quality of health care. These apps are changing the current scenario in health care, and their numbers are increasing. Objective We wanted to perform an analysis of the current status of mobile health technologies and apps for medical emergencies. We aimed to synthesize the existing body of knowledge to provide relevant insights for this topic. Moreover, we wanted to identify common threads and gaps to support new challenging, interesting, and relevant research directions. Methods We reviewed the main relevant papers and apps available in the literature. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology was used in this review. The search criteria were adopted using systematic methods to select papers and apps. On one hand, a bibliographic review was carried out in different search databases to collect papers related to each application in the health emergency field using defined criteria. On the other hand, a review of mobile apps in two virtual storage platforms (Google Play Store and Apple App Store) was carried out. The Google Play Store and Apple App Store are related to the Android and iOS operating systems, respectively. Results In the literature review, 28 papers in the field of medical emergency were included. These studies were collected and selected according to established criteria. Moreover, we proposed a taxonomy using six groups of applications. In total, 324 mobile apps were found, with 192 identified in the Google Play Store and 132 identified in the Apple App Store. Conclusions We found that all apps in the Google Play Store were free, and 73 apps in the Apple App Store were paid, with the price ranging from US $0.89 to US $5.99. Moreover, 39% (11/28) of the included studies were related to warning systems for emergency services and 21% (6/28) were associated with disaster management apps.

Download Full-text

Nick Noguez and Michael Gonzalez Discuss their Research Project: Are Social Features in Mobile Health Apps Effective in Engaging Users? A Content Analysis of Smoking Cessation Mobile Apps

The Boller Review ◽

10.18776/tcu/br/4/104 ◽

2019 ◽

Vol 4 ◽

Author(s):

Nick Noguez And Michael Gonzalez

Keyword(s):

Smoking Cessation ◽

Content Analysis ◽

Mobile Health ◽

Social Engagement ◽

Rating Scale ◽

Mobile Apps ◽

Mobile App ◽

Health Apps ◽

Social Features ◽

Mobile Health Apps

Despite the ubiquity of smartphone ownership and the increasing integration of social engagement features in smoking cessation apps to engage users, thesocial engagement features that exist in current smoking cessation apps and how effective these social features are in engaging users remain unclear. To fill the gap in the literature, a content analysis of free and paid smoking cessation mobile apps isconducted to examine a) the presence of socialengagement features(e.g., social support, social announcement, social referencing) and non-social engagement features (e.g., personal environmental changes, goal setting), and b) their relationship with user ratingsand engagement scores (e.g., Mobile App rating scale [MARS]). The findings will not only extend the mobile health apps engagement typology,but also inform smoking cessation mobile apps design.

Download Full-text

Mobile Health Apps for Medical Emergencies: Systematic Review (Preprint)

10.2196/preprints.18513 ◽

2020 ◽

Author(s):

Alejandro Plaza Roncero ◽

Gonçalo Marques ◽

Beatriz Sainz-De-Abajo ◽

Francisco Martín-Rodríguez ◽

Carlos del Pozo Vegas ◽

...

Keyword(s):

Health Care ◽

Mobile Health ◽

Mobile Apps ◽

Current Status ◽

Health Technologies ◽

Health Apps ◽

App Store ◽

Mobile Health Apps ◽

Medical Emergencies ◽

Google Play

BACKGROUND Mobile health apps are used to improve the quality of health care. These apps are changing the current scenario in health care, and their numbers are increasing. OBJECTIVE We wanted to perform an analysis of the current status of mobile health technologies and apps for medical emergencies. We aimed to synthesize the existing body of knowledge to provide relevant insights for this topic. Moreover, we wanted to identify common threads and gaps to support new challenging, interesting, and relevant research directions. METHODS We reviewed the main relevant papers and apps available in the literature. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology was used in this review. The search criteria were adopted using systematic methods to select papers and apps. On one hand, a bibliographic review was carried out in different search databases to collect papers related to each application in the health emergency field using defined criteria. On the other hand, a review of mobile apps in two virtual storage platforms (Google Play Store and Apple App Store) was carried out. The Google Play Store and Apple App Store are related to the Android and iOS operating systems, respectively. RESULTS In the literature review, 28 papers in the field of medical emergency were included. These studies were collected and selected according to established criteria. Moreover, we proposed a taxonomy using six groups of applications. In total, 324 mobile apps were found, with 192 identified in the Google Play Store and 132 identified in the Apple App Store. CONCLUSIONS We found that all apps in the Google Play Store were free, and 73 apps in the Apple App Store were paid, with the price ranging from US $0.89 to US $5.99. Moreover, 39% (11/28) of the included studies were related to warning systems for emergency services and 21% (6/28) were associated with disaster management apps.

Download Full-text

Mobile Apps for Health Behavior Change: Protocol for a Systematic Review (Preprint)

10.2196/preprints.16931 ◽

2019 ◽

Author(s):

Madison Milne-Ives ◽

Ching Lam ◽

Michelle Helena Van Velthoven ◽

Edward Meinert

Keyword(s):

Systematic Review ◽

Health Behavior ◽

Behavior Change ◽

Mobile Health ◽

Mobile Apps ◽

Health Behavior Change ◽

Mobile App ◽

Risk Of Bias ◽

Health Apps ◽

Mobile Health Apps

BACKGROUND The popularity and ubiquity of mobile apps have rapidly expanded in the past decade. With a growing focus on patient interaction with health management, mobile apps are increasingly used to monitor health and deliver behavioral interventions. The considerable variation in these mobile health apps, from their target patient group to their health behavior, and their behavioral change strategy, has resulted in a large but incohesive body of literature. OBJECTIVE The purpose of this protocol is to provide an overview of the current landscape, theories behind, and effectiveness of mobile apps for health behavior change. METHODS The Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols will be used to structure this protocol. The focus of the systematic review is guided by a population, intervention, comparator, and outcome framework. A systematic search of Medline, EMBASE, CINAHL, and Web of Science will be conducted. Two authors will independently screen the titles and abstracts of identified references and select studies according to the eligibility criteria. Any discrepancies will then be discussed and resolved. One reviewer will extract data into a standardized form, which will be validated by a second reviewer. Risk of bias was assessed using the Cochrane Collaboration Risk of Bias tool, and a descriptive analysis will summarize the effectiveness of all the apps. RESULTS As of November 2019, the systematic review has been completed and is in peer review for publication. CONCLUSIONS This systematic review will summarize the current mobile app technologies and their effectiveness, usability, and coherence with behavior change theory. It will identify areas of improvement (where there is no evidence of efficacy) and help inform the development of more useful and engaging mobile health apps. INTERNATIONAL REGISTERED REPORT PRR1-10.2196/16931

Download Full-text

Mobile Apps for Health Behavior Change: Protocol for a Systematic Review

JMIR Research Protocols ◽

10.2196/16931 ◽

2020 ◽

Vol 9 (1) ◽

pp. e16931

Author(s):

Madison Milne-Ives ◽

Ching Lam ◽

Michelle Helena Van Velthoven ◽

Edward Meinert

Keyword(s):

Systematic Review ◽

Health Behavior ◽

Behavior Change ◽

Mobile Health ◽

Mobile Apps ◽

Health Behavior Change ◽

Mobile App ◽

Risk Of Bias ◽

Health Apps ◽

Mobile Health Apps

Background The popularity and ubiquity of mobile apps have rapidly expanded in the past decade. With a growing focus on patient interaction with health management, mobile apps are increasingly used to monitor health and deliver behavioral interventions. The considerable variation in these mobile health apps, from their target patient group to their health behavior, and their behavioral change strategy, has resulted in a large but incohesive body of literature. Objective The purpose of this protocol is to provide an overview of the current landscape, theories behind, and effectiveness of mobile apps for health behavior change. Methods The Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols will be used to structure this protocol. The focus of the systematic review is guided by a population, intervention, comparator, and outcome framework. A systematic search of Medline, EMBASE, CINAHL, and Web of Science will be conducted. Two authors will independently screen the titles and abstracts of identified references and select studies according to the eligibility criteria. Any discrepancies will then be discussed and resolved. One reviewer will extract data into a standardized form, which will be validated by a second reviewer. Risk of bias was assessed using the Cochrane Collaboration Risk of Bias tool, and a descriptive analysis will summarize the effectiveness of all the apps. Results As of November 2019, the systematic review has been completed and is in peer review for publication. Conclusions This systematic review will summarize the current mobile app technologies and their effectiveness, usability, and coherence with behavior change theory. It will identify areas of improvement (where there is no evidence of efficacy) and help inform the development of more useful and engaging mobile health apps. Trial Registration PROSPERO CRD42019155604; https://tinyurl.com/sno4lcu International Registered Report Identifier (IRRID) PRR1-10.2196/16931

Download Full-text

Establishing a digital health platform in an academic medical center supporting rural communities

Journal of Clinical and Translational Science ◽

10.1017/cts.2020.11 ◽

2020 ◽

Vol 4 (5) ◽

pp. 384-388

Author(s):

Anita Walden ◽

Aaron S. Kemp ◽

Linda J. Larson-Prior ◽

Thomas Kim ◽

Jennifer Gan ◽

...

Keyword(s):

Rural Communities ◽

Mobile Health ◽

Medical Center ◽

Digital Health ◽

Academic Medical Center ◽

Healthcare Providers ◽

Training Models ◽

Health Technologies ◽

Regional Conference ◽

Patient Advocates

AbstractThe University of Arkansas for Medical Sciences (UAMS), like many rural states, faces clinical and research obstacles to which digital innovation is seen as a promising solution. To implement digital technology, a mobile health interest group was established to lay the foundation for an enterprise-wide digital health innovation platform. To create a foundation, an interprofessional team was established, and a series of formal networking events was conducted. Three online digital health training models were developed, and a full-day regional conference was held featuring nationally recognized speakers and panel discussions with clinicians, researchers, and patient advocates involved in digital health programs at UAMS. Finally, an institution-wide survey exploring the interest in and knowledge of digital health technologies was distributed. The networking events averaged 35–45 attendees. About 100 individuals attended the regional conference with positive feedback from participants. To evaluate mHealth knowledge at the institution, a survey was completed by 257 UAMS clinicians, researchers, and staff. It revealed that there are opportunities to increase training, communication, and collaboration for digital health implementation. The inclusion of the mobile health working group in the newly formed Institute for Digital Health and Innovation provides a nexus for healthcare providers and researches to facilitate translational research.

Download Full-text

The Digital Health Revolution and People with Disabilities: Perspective from the United States

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph17020381 ◽

2020 ◽

Vol 17 (2) ◽

pp. 381 ◽

Cited By ~ 4

Author(s):

Mike Jones ◽

Frank DeRuyter ◽

John Morris

Keyword(s):

United States ◽

Mobile Health ◽

Chronic Conditions ◽

Digital Health ◽

Wearable Sensors ◽

Healthcare Delivery ◽

People With Disabilities ◽

The United States ◽

Models Of Care ◽

Health Technologies

This article serves as the introduction to this special issue on Mobile Health and Mobile Rehabilitation for People with Disabilities. Social, technological and policy trends are reviewed. Needs, opportunities and challenges for the emerging fields of mobile health (mHealth, aka eHealth) and mobile rehabilitation (mRehab) are discussed. Healthcare in the United States (U.S.) is at a critical juncture characterized by: (1) a growing need for healthcare and rehabilitation services; (2) maturing technological capabilities to support more effective and efficient health services; (3) evolving public policies designed, by turns, to contain cost and support new models of care; and (4) a growing need to ensure acceptance and usability of new health technologies by people with disabilities and chronic conditions, clinicians and health delivery systems. Discussion of demographic and population health data, healthcare service delivery and a public policy primarily focuses on the U.S. However, trends identified (aging populations, growing prevalence of chronic conditions and disability, labor shortages in healthcare) apply to most countries with advanced economies and others. Furthermore, technologies that enable mRehab (wearable sensors, in-home environmental monitors, cloud computing, artificial intelligence) transcend national boundaries. Remote and mobile healthcare delivery is needed and inevitable. Proactive engagement is critical to ensure acceptance and effectiveness for all stakeholders.

Download Full-text

Adoption of enterprise architecture for healthcare in AeHIN member countries

BMJ Health & Care Informatics ◽

10.1136/bmjhci-2020-100136 ◽

2020 ◽

Vol 27 (1) ◽

pp. e100136

Author(s):

Jitendra Jonnagaddala ◽

Guan N Guo ◽

Sean Batongbacal ◽

Alvin Marcelo ◽

Siaw-Teng Liaw

Keyword(s):

Low Income ◽

Enterprise Architecture ◽

Digital Health ◽

Management Training ◽

Senior Management ◽

Evaluation Framework ◽

Asia Pacific ◽

Expert Advice ◽

Middle Income ◽

Health Technologies

BackgroundHealthcare organisations are undergoing a major transformational shift in the use of information and digital health technologies. Enterprise architecture (EA) has been incrementally adopted in many healthcare organisations globally to facilitate this change. EA can increase the effectiveness of an organisation’s digital health capabilities and resources. However, little is known about the status of EA adoption in low-income and middle-income countries. This study aimed to evaluate the challenges, goals and benefits associated with adoption of EA for healthcare in the Asia eHealth Information Network (AeHIN) member countries .MethodsWe developed an EA Adoption Evaluation framework with four principal layers: governance, strategy, EA and performance. The framework guided the development of a questionnaire to investigate the goals, challenges and benefits faced before and during EA adoption by healthcare organisations.Sample26 participants from 18 healthcare organisations in the Asia-Pacific region representing 11 countries. Organisations included Ministries of Health, Universities, Non-Governmental Organisations and Technical Advisory Groups.FindingsOnly 5 of the 18 organisations had begun adopting EA. The goals expressed for EA adoption were to address issues such as interoperability, lack of technical infrastructure and poor alignment of business and information technology strategies. Cost reduction was less emphasised. The main challenges to adopting EA was the lack of EA knowledge, leadership and involvement of senior management.ConclusionThe adoption of EA is incipient in AeHIN member healthcare organisations. To encourage EA adoption, these organisations need to invest in internal capacity building, senior management training and seek independent EA expert advice to systematically identify and address the barriers to adopting EA.

Download Full-text