The App Rating Inventory: A Novel Method for Evaluating Mobile Apps (Preprint)
BACKGROUND A prodigious number of mobile health apps have flooded the market. The lack of guidelines for identifying high-quality apps from the overwhelming number of available apps creates confusion forestalling clinical adoption. OBJECTIVE The Defense Health Agency’s (DHA) Connected Health Branch developed the app rating inventory (ARI), an objective rating system with capability for broad application across condition areas. METHODS During the development of the ARI, three rounds of testing were conducted to enhance the tool’s performance, reduce redundancy, validate the ARI’s broad application, and assess potential subjectivity. RESULTS The ARI is a 28-item, three-criterion tool. The evidence criterion contains six items, and the content and customizability criterion each contain eleven items. Scoring is based on a simple binary system: either the app contains the feature or it does not. The 28 items are weighted equally; no one item is considered more (or less) important than any other. Each rated app receives four scores: a score for evidence, content, and customizability, and a total score (the sum of the three categories.). Higher scores indicate that the app obtained a positive score on more items than a similar app with a lower score. The evidence, content and customizability scores allow a clinician to make focused decisions when selecting an app for clinical use. CONCLUSIONS Using a two-phased process (market research followed by ratings), the ARI is able to evaluate apps for evidence, content and customizability. Scoring systems provide guidance; they filter down hundreds of apps in a disease category to a handful for consideration. Indeed, apps are not new medicine; in many cases, they are a novel delivery system for proven interventions.