Development and Validation of an Interpretable Conditional RNN for Weight Change Prediction Using an Obesity Management Mobile App (Preprint)

BACKGROUND In obesity management, whether patients lose 5% or more of their initial weight is a critical factor in their clinical outcome. However, evaluations that only take this approach cannot identify and distinguish between individuals whose weight change varies and those who steadily lose weight. Evaluation of weight loss considering the volatility of weight change through a mobile-based intervention for obesity can facilitate the understanding of individuals’ behavior and weight changes from a longitudinal perspective. OBJECTIVE With machine learning approach, we examined weight loss trajectories and explored the factors related to behavioral and app usage characteristics that induce weight loss. METHODS We used the lifelog data of 19,784 individuals who enrolled in a 16-week obesity management program on the healthcare app Noom in the US during August 8, 2013 to August 8, 2019. We performed K-means clustering with dynamic time warping to cluster the weight loss time series and inspected the quality of clusters with the total sum of distance within the clusters. To identify the usage factors to determine clustering assignment, we longitudinally compared weekly usage statistics with effect size on a weekly basis. RESULTS Initial Body Mass Index (BMI) of participants was 33.9±5.9 kg/m2, and ultimately reached an average BMI of 32.0±5.7 kg/m2. In their weight log, 5 Clusters were identified: Cluster 1 (sharp decrease) showed a high proportion of weight reduction class between 10% and 15%—the highest among the five clusters (n=2,364/12,796, 18.9%)—followed by Cluster 2 (moderate decrease), Cluster 3 (increase), Cluster 4 (yoyo), Cluster 5 (other). In comparison between cluster 2 and cluster 4, although the effect size of difference in the average meal input adherence and average weight input adherence did not show a significant difference in the first week, it increased continuously for 7 weeks (Cohen’s d=0.408; 0.38). CONCLUSIONS With machine learning approach clustering shape-based timeseries similarity, this study identified 5 weight loss trajectories in mobile weight management app. Overall adherence and early adherence related to self-monitoring emerged as a potential predictor of these trajectories.

Download Full-text

Personalized weight change prediction in the first week of life

Clinical Nutrition ◽

10.1016/j.clnu.2018.04.001 ◽

2019 ◽

Vol 38 (2) ◽

pp. 689-696 ◽

Cited By ~ 4

Author(s):

Mélanie Wilbaux ◽

Severin Kasser ◽

Julia Gromann ◽

Isabella Mancino ◽

Tania Coscia ◽

...

Keyword(s):

Weight Change ◽

Change Prediction

Download Full-text

The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study (Preprint)

10.2196/preprints.11500 ◽

2018 ◽

Author(s):

Leming Zhou ◽

Jie Bao ◽

I Made Agus Setiawan ◽

Andi Saptono ◽

Bambang Parmanto

Keyword(s):

Reliability And Validity ◽

Correlation Coefficients ◽

Internal Consistency Reliability ◽

Mobile App ◽

Target User ◽

System Usability Scale ◽

System Usability ◽

Study Participants ◽

Development And Validation ◽

Mhealth Apps

BACKGROUND After a mobile health (mHealth) app is created, an important step is to evaluate the usability of the app before it is released to the public. There are multiple ways of conducting a usability study, one of which is collecting target users’ feedback with a usability questionnaire. Different groups have used different questionnaires for mHealth app usability evaluation: The commonly used questionnaires are the System Usability Scale (SUS) and Post-Study System Usability Questionnaire (PSSUQ). However, the SUS and PSSUQ were not designed to evaluate the usability of mHealth apps. Self-written questionnaires are also commonly used for evaluation of mHealth app usability but they have not been validated. OBJECTIVE The goal of this project was to develop and validate a new mHealth app usability questionnaire. METHODS An mHealth app usability questionnaire (MAUQ) was designed by the research team based on a number of existing questionnaires used in previous mobile app usability studies, especially the well-validated questionnaires. MAUQ, SUS, and PSSUQ were then used to evaluate the usability of two mHealth apps: an interactive mHealth app and a standalone mHealth app. The reliability and validity of the new questionnaire were evaluated. The correlation coefficients among MAUQ, SUS, and PSSUQ were calculated. RESULTS In this study, 128 study participants provided responses to the questionnaire statements. Psychometric analysis indicated that the MAUQ has three subscales and their internal consistency reliability is high. The relevant subscales correlated well with the subscales of the PSSUQ. The overall scale also strongly correlated with the PSSUQ and SUS. Four versions of the MAUQ were created in relation to the type of app (interactive or standalone) and target user of the app (patient or provider). A website has been created to make it convenient for mHealth app developers to use this new questionnaire in order to assess the usability of their mHealth apps. CONCLUSIONS The newly created mHealth app usability questionnaire—MAUQ—has the reliability and validity required to assess mHealth app usability.

Download Full-text

Spanish Version of the System Usability Scale for the Assessment of Electronic Tools: Development and Validation

JMIR Human Factors ◽

10.2196/21161 ◽

2020 ◽

Vol 7 (4) ◽

pp. e21161

Author(s):

Magdalena Del Rocio Sevilla-Gonzalez ◽

Lizbeth Moreno Loaeza ◽

Laura Sofia Lazaro-Carrera ◽

Brigette Bourguet Ramirez ◽

Anabel Vázquez Rodríguez ◽

...

Keyword(s):

Mobile App ◽

Spanish Version ◽

Face Validity ◽

Electronic Systems ◽

Validity Index ◽

Clinical Counseling ◽

System Usability Scale ◽

System Usability ◽

The Face ◽

Development And Validation

Background The System Usability Scale (SUS) is a common metric used to assess the usability of a system, and it was initially developed in English. The implementation of electronic systems for clinical counseling (eHealth and mobile health) is increasing worldwide. Therefore, tools are needed to evaluate these applications in the languages and regional contexts in which the electronic tools are developed. Objective This study aims to translate, culturally adapt, and validate the original English version of the SUS into a Spanish version. Methods The translation process included forward and backward translation. Forward translations were made by 2 native Spanish speakers who spoke English as their second language, and a backward translation was made by a native English speaker. The Spanish SUS questionnaire was validated by 10 experts in mobile app development. The face validity of the questionnaire was tested with 10 mobile phone users, and the reliability testing was conducted among 88 electronic application users. Results The content validity index of the new Spanish SUS was good, as indicated by a rating of 0.92 for the relevance of the items. The questionnaire was easy to understand, based on a face validity index of 0.94. The Cronbach α was .812 (95% CI 0.748-0.866; P<.001). Conclusions The new Spanish SUS questionnaire is a valid and reliable tool to assess the usability of electronic tools among Spanish-speaking users.

Download Full-text

A Facial Recognition Mobile App for Patient Safety and Biometric Identification: Design, Development, and Validation

JMIR mhealth and uhealth ◽

10.2196/11472 ◽

2019 ◽

Vol 7 (4) ◽

pp. e11472 ◽

Cited By ~ 4

Author(s):

Byoungjun Jeon ◽

Boseong Jeong ◽

Seunghoon Jee ◽

Yan Huang ◽

Youngmin Kim ◽

...

Keyword(s):

Patient Safety ◽

Facial Recognition ◽

Mobile App ◽

Biometric Identification ◽

Design Development ◽

Development And Validation

Download Full-text

Development and validation of an interpretable Conditional RNN for weight change prediction an obesity management mobile app (Preprint)

10.2196/preprints.22183 ◽

2020 ◽

Author(s):

Ho Heon Kim ◽

Young In Kim ◽

Yu Rang Park

Keyword(s):

Weight Loss ◽

Predictive Model ◽

Mobile App ◽

Percentage Error ◽

Obesity Management ◽

Contributing Factors ◽

Negative Contribution ◽

Actual Weight ◽

Average Percentage ◽

Time Invariant

BACKGROUND As an alternative to on-site obesity management, a mobile-based intervention has been given more attention. Despite the rise of mobile interventions for obesity, there are lost opportunities to achieve better outcomes due to the lack of a predictive model using currently existing health data collected longitudinally and cross-sectionally. OBJECTIVE This study aimed to develop a predictive model for weight to be used in mobile-based interventions using interpretable AI, and to explore the contributing factors to weight loss. METHODS Using lifelong of mobile application users (Noom) who used a weight loss program for 16 weeks in the U.S., an interpretable recurrent neural network for the prediction of weight after intervention considering both time-variant variables and time-invariant variables was developed. This interpretable model was trained and validated with fivefold cross-validation testing (training set: 70%; testing: 30%) using lifelog data of app users for weight loss. Mean average percent error (MAPE) between actual weight loss and predicted weight, and contribution coefficients for model interpretation. To better understand the behavior factors to weight loss or gain, the contributing factors were calculated by the contribution coefficients in test sets to interpret the effects of contributing factors to weight loss. RESULTS A total of 17,867 eligible users were included in the analysis. The overall mean average percentage error of the model was 3.50% and the errors of the model declined from 3.78% to 3.45% by observing the data at the end of the program. The time level contribution was shown to be equally distributed at 0.0625 in each week, but this gradually decreased as it approached 16 weeks. Factors such as usage pattern, weight input frequency and meal input adherence, exercise, and sharp decreases in weight trajectories had negative contribution coefficients of -0.021, -0.032, -0.015, and -0.066, respectively. As for time-invariant variables, males had a -0.091 contribution coefficient. CONCLUSIONS An interpretable artificial intelligence to utilize both data and time fixed data can forecast weight loss precisely after obesity management application while preserving model transparency. This week to week prediction model is expected to improve weight loss and provide a global explanation of contributing factors, leading to better outcomes.

Download Full-text

Spanish Version of the System Usability Scale for the Assessment of Electronic Tools: Development and Validation (Preprint)

10.2196/preprints.21161 ◽

2020 ◽

Author(s):

Magdalena Del Rocio Sevilla-Gonzalez ◽

Lizbeth Moreno Loaeza ◽

Laura Sofia Lazaro-Carrera ◽

Brigette Bourguet Ramirez ◽

Anabel Vázquez Rodríguez ◽

...

Keyword(s):

Mobile App ◽

Spanish Version ◽

Face Validity ◽

Electronic Systems ◽

Validity Index ◽

Clinical Counseling ◽

System Usability Scale ◽

System Usability ◽

The Face ◽

Development And Validation

BACKGROUND The System Usability Scale (SUS) is a common metric used to assess the usability of a system, and it was initially developed in English. The implementation of electronic systems for clinical counseling (eHealth and mobile health) is increasing worldwide. Therefore, tools are needed to evaluate these applications in the languages and regional contexts in which the electronic tools are developed. OBJECTIVE This study aims to translate, culturally adapt, and validate the original English version of the SUS into a Spanish version. METHODS The translation process included forward and backward translation. Forward translations were made by 2 native Spanish speakers who spoke English as their second language, and a backward translation was made by a native English speaker. The Spanish SUS questionnaire was validated by 10 experts in mobile app development. The face validity of the questionnaire was tested with 10 mobile phone users, and the reliability testing was conducted among 88 electronic application users. RESULTS The content validity index of the new Spanish SUS was good, as indicated by a rating of 0.92 for the relevance of the items. The questionnaire was easy to understand, based on a face validity index of 0.94. The Cronbach α was .812 (95% CI 0.748-0.866; <i>P</i><.001). CONCLUSIONS The new Spanish SUS questionnaire is a valid and reliable tool to assess the usability of electronic tools among Spanish-speaking users.

Download Full-text