A process for reviewing mental health apps: Using the One Mind PsyberGuide Credibility Rating System

Objective Given the increasing number of publicly available mental health apps, we need independent advice to guide adoption. This paper discusses the challenges and opportunities of current mental health app rating systems and describes the refinement process of one prominent system, the One Mind PsyberGuide Credibility Rating Scale (PGCRS). Methods PGCRS Version 1 was developed in 2013 and deployed for 7 years, during which time a number of limitations were identified. Version 2 was created through multiple stages, including a review of evaluation guidelines and consumer research, input from scientific experts, testing, and evaluation of face validity. We then re-reviewed 161 mental health apps using the updated rating scale, investigated the reliability and discrepancy of initial scores, and updated ratings on the One Mind PsyberGuide public app guide. Results Reliabilities across the scale's 9 items ranged from −0.10 to 1.00, demonstrating that some characteristics of apps are more difficult to rate consistently. The average overall score of the 161 reviewed mental health apps was 2.51/5.00 (range 0.33–5.00). Ratings were not strongly correlated with app store star ratings, suggesting that credibility scores provide different information to what is contained in star ratings. Conclusion PGCRS summarizes and weights available information in 4 domains: intervention specificity, consumer ratings, research, and development. Final scores are created through an iterative process of initial rating and consensus review. The process of updating this rating scale and integrating it into a procedure for evaluating apps demonstrates one method for determining app quality.

Download Full-text

Multipurpose Mobile Apps for Mental Health in Chinese App Stores: Content Analysis and Quality Evaluation (Preprint)

10.2196/preprints.34054 ◽

2021 ◽

Author(s):

Xiaoqian Wu ◽

Lin Xu ◽

PengFei Li ◽

TingTing Tang ◽

Cheng Huang

Keyword(s):

Mental Health ◽

Rating Scale ◽

Mobile Apps ◽

Scientific Basis ◽

Mobile App ◽

Security Measures ◽

Privacy And Security ◽

Health Apps ◽

Mental Health Awareness ◽

Privacy Measures

BACKGROUND Mental disorders impose varying degrees of burden on patients and their surroundings. However, people are reluctant to take the initiative to seek mental health services because of the uneven distribution of resources and stigmatization. Thus, mobile apps are considered an effective way to eliminate these obstacles and improve mental health awareness. OBJECTIVE This study aimed to evaluate the quality, function, privacy measures, and evidence-based and professional background of multipurpose mental health apps in Chinese commercial app stores. METHODS A systematic search was conducted on iOS and Android platforms in China to identify multipurpose mental health apps. Two independent reviewers evaluated the identified mobile apps using Mobile App Rating Scale (MARS). Each app was downloaded, and the general characteristics, privacy and security measures, development background, and functional characteristics of each app were evaluated. RESULTS A total of 40 apps were analyzed, of which 35 apps (87.5%) were developed by companies and 33 apps (82.5%) provided links to access the privacy policy; 52.5% did not mention the involvement of relevant professionals or the guidance of scientific basis in the app development process. The main built-in functions of these apps include psychological education (38/40, 95%), self-assessment (34/40, 85%), and counseling (33/40, 83%). The overall quality average MARS score of the 40 apps was 3.53 (standard deviation 0.39), and the total score was between 2.96 and 4.30. The total score of MARS was significantly positively correlated with the scores of each subscale (r = 0.62–0.88; P <.001). However, the user score of the app market was not significantly correlated with the total score of MARS (r = 0.23; P =.19). CONCLUSIONS The quality of multipurpose mental health apps in China’s main app market is generally good and provides various functional combinations. However, health professionals are less involved in the development of these apps, and the privacy protection policy of the apps also needs to be described in more detail. This study provides a reference for the development of multipurpose mental health apps.

Download Full-text

Assessing risk of self-harm in acute paediatric settings: a multicentre exploratory evaluation of the CYP-MH SAPhE instrument

BMJ Open ◽

10.1136/bmjopen-2020-043762 ◽

2021 ◽

Vol 11 (5) ◽

pp. e043762

Author(s):

Joseph C Manning ◽

Tim Carter ◽

Gemma Walker ◽

Jane Coad ◽

Aimee Aubeeluck

Keyword(s):

Mental Health ◽

Psychometric Properties ◽

Internal Consistency ◽

Rating Scale ◽

Assessment Instrument ◽

Face Validity ◽

Secondary Outcome ◽

Delphi Consensus ◽

Healthcare Environments ◽

Self Harm

ObjectiveTo psychometrically assess the Children and Young People-Mental Health Self-harm Assessment in Paediatric healthcare Environments (CYP-MH SAPhE) instrument for the identification of immediate risk of self-harm in CYP, aged 10–19 years, in acute paediatric wards or emergency departments.DesignThe CYP-MH SAPhE Instrument was developed through a robust scoping review and Delphi consensus with 30 clinicians/topic experts. To evaluate the psychometric properties, a multicentre exploratory study was conducted.SettingThree acute hospitals in the UK.Participants163 CYP presenting at acute hospital settings with primary mental health (cases) or physical health (non-cases) conditions.Primary and secondary outcome measuresPsychometric properties of the CYP-MH SAPhE instrument were evaluated through Principle Axis Factoring (PAF) with Oblimin (Kaiser normalisation) alongside measures of internal consistency (Cronbach’s α), convergent, discriminant and face validity.ResultsPAF of the dichotomous items (n=9) loaded onto three factors (1) behaviours and intentions; (2) suicidality and (3) self-harm. Factors 1 (Cronbach’s α=0.960) and 3 (Cronbach’s α=1) had high internal consistency. There was: good level of agreement between raters (kappa=0.65); a moderately positive correlation between the CYP-MH SAPhE instrument and the Columbia-Suicide Severity Rating Scale; and discrimination between cases and non-cases across the three factors (factor 1: m=88 vs 70; factor 2: m=102 vs 70; factor 3: m=104 vs 68). Assessment of face validity resulted in six items being removed, culminating in an eight question, rapid assessment instrument.ConclusionsThe results support the CYP-MH SAPhE Tool as a potentially reliable and valid instrument to identify immediate risk of self-harm in CYP presenting to acute paediatric healthcare environments, which is a burgeoning and significant global health issue.

Download Full-text

Mobile-based interventions for common mental disorders in youth: a systematic evaluation of pediatric health apps

Child and Adolescent Psychiatry and Mental Health ◽

10.1186/s13034-021-00401-6 ◽

2021 ◽

Vol 15 (1) ◽

Author(s):

Matthias Domhardt ◽

Eva-Maria Messner ◽

Anna-Sophia Eder ◽

Sophie Engler ◽

Lasse B. Sander ◽

...

Keyword(s):

Mental Health ◽

Mental Disorders ◽

Mental Health Professionals ◽

Rating Scales ◽

Rating Scale ◽

Common Mental Disorders ◽

Mobile App ◽

Health Apps ◽

Anxiety Depression

Abstract Background The access to empirically-supported treatments for common mental disorders in children and adolescents is often limited. Mental health apps might extend service supplies, as they are deemed to be cost-efficient, scalable and appealing for youth. However, little is known about the quality of available apps. Therefore, we aimed to systematically evaluate current mobile-based interventions for pediatric anxiety, depression and posttraumatic stress disorder (PTSD). Methods Systematic searches were conducted in Google Play Store and Apple App Store to identify relevant apps. To be eligible for inclusion, apps needed to be: (1) designed to target either anxiety, depression or PTSD in youth (0–18 years); (2) developed for children, adolescents or caregivers; (3) provided in English or German; (4) operative after download. The quality of eligible apps was assessed with two standardized rating systems (i.e., Mobile App Rating Scale (MARS) and ENLIGHT) independently by two reviewers. Results Overall, the searches revealed 3806 apps, with 15 mental health apps (0.39%) fulfilling our inclusion criteria. The mean overall scores suggested a moderate app quality (MARS: M = 3.59, SD = 0.50; ENLIGHT: M = 3.22, SD = 0.73). Moreover, only one app was evaluated in an RCT. The correlation of both rating scales was high (r = .936; p < .001), whereas no significant correlations were found between rating scales and user ratings (p > .05). Conclusions Our results point to a rather poor overall app quality, and indicate an absence of scientific-driven development and lack of methodologically sound evaluation of apps. Thus, future high-quality research is required, both in terms of theoretically informed intervention development and assessment of mental health apps in RCTs. Furthermore, institutionalized best-practices that provide central information on different aspects of apps (e.g., effectiveness, safety, and data security) for patients, caregivers, stakeholders and mental health professionals are urgently needed.

Download Full-text

User Experience, Engagement, and Popularity in Mental Health Apps: Secondary Analysis of App Analytics and Expert App Reviews (Preprint)

10.2196/preprints.30766 ◽

2021 ◽

Author(s):

Benjamin T Kaveladze ◽

Akash R Wasil ◽

John B Bunyi ◽

Veronica Ramirez ◽

Stephen M Schueller

Keyword(s):

Mental Health ◽

User Experience ◽

Rating Scale ◽

Secondary Analysis ◽

Secondary Data ◽

Mobile App ◽

Health Apps ◽

App Store ◽

Expert Ratings ◽

The Relationship

BACKGROUND User experience and engagement are critical to mental health apps’ abilities to support users. However, limited work has examined the relationship between user experience, engagement, and app popularity. Given that apps vary immensely in their popularity, understanding why some mental health apps are more appealing or engaging to users can inform efforts to develop better apps. OBJECTIVE We aimed to examine relationships between user experience, engagement, and popularity. To do so, we examined links between subjective measures of user experience and objective measures of app popularity and engagement. METHODS We conducted a pre-registered secondary data analysis in a sample of 56 mental health apps. To measure user experience, we used expert ratings on the Mobile App Rating Scale (MARS) and consumer ratings from the Apple app store and Google Play app store. To measure engagement, we acquired estimates of measures of monthly active users (MAU) and user retention. To measure app popularity, we used download count, total app revenue, and MAU again. RESULTS MARS total score was significantly and positively correlated with app-level revenue (T=0.30, P=.002), MAU (T= 0.39, P<.001), and downloads (T=0.41, P<.001). However, neither the MARS total score nor any of its subscales (Engagement, Functionality, Aesthetics, nor Information) were significantly correlated with user retention 1, 7, or 30 days after downloading. Also, MARS total score was not significantly correlated with app store rating. CONCLUSIONS Popular mental health apps receive better ratings of user experience than less popular ones. However, user experience (as operationalized by the MARS) does not predict sustained engagement with mental health apps. Collaboration between industry and academic teams may better advance a science of engagement and help to make mental health apps more effective and appealing.

Download Full-text

The Carroll Rating Scale for Depression I. Development, Reliability and Validation

The British Journal of Psychiatry ◽

10.1192/bjp.138.3.194 ◽

1981 ◽

Vol 138 (3) ◽

pp. 194-200 ◽

Cited By ~ 263

Author(s):

Bernard J. Carroll ◽

Michael Feinberg ◽

Peter E. Smouse ◽

Sarah G. Rawson ◽

John F. Greden

Keyword(s):

Concurrent Validity ◽

Rating Scale ◽

Face Validity ◽

Strongly Correlated ◽

Validity And Reliability ◽

Additional Advantage ◽

Self Rating ◽

Rating Instrument ◽

Observer Ratings ◽

Hamilton Rating Scale

SummaryThe Carroll rating scale (CRS) was developed as a self rating instrument for depression, closely matching the information content and specific items of the Hamilton rating scale (HRS). The CRS was found to have acceptable face validity and reliability. The concurrent validity of the CRS was acceptable, based on comparisons with the HRS and the Beck Depression Inventory (BDI). The internal consistency of the CRS was very similar to that of the HRS. The CRS contained information about HRS scores beyond what could be predicted from BDI scores, but the BDI did not predict HRS scores beyond what could be predicted from CRS scores. The CRS and BDI scores were strongly correlated and both had access to a subjective dimension of depression that could not be predicted from HRS scores. The complementary uses of self ratings and observer ratings are evident from these results. The CRS may be a useful alternative to the BDI as a self rating scale, with the additional advantage of closer correspondence to the HRS.

Download Full-text

Veterans Affairs and the Department of Defense mental health apps: A systematic literature review.

Psychological Services ◽

10.1037/ser0000289 ◽

2019 ◽

Vol 16 (2) ◽

pp. 196-207 ◽

Cited By ~ 10

Author(s):

Christine E. Gould ◽

Brian C. Kok ◽

Vanessa K. Ma ◽

Aimee Marie L. Zapata ◽

Jason E. Owen ◽

...

Keyword(s):

Mental Health ◽

Literature Review ◽

Systematic Literature Review ◽

Department Of Defense ◽

Veterans Affairs ◽

Health Apps

Download Full-text

PHUBBING BEHAVIOR AND ROMANTIC RELATIONSHIP: MECHANISM OF MENTAL HEALTH AMONG MARRIED COUPLES

Foundation University Journal of Psychology ◽

10.33897/fujp.v4i1.70 ◽

2020 ◽

Vol 4 (1) ◽

pp. 103-137

Author(s):

Rabia Zonash Mir

Keyword(s):

Mental Health ◽

Romantic Relationship ◽

Rating Scale ◽

Short Form ◽

Married Couple ◽

Married Couples ◽

Mental Health Issue ◽

Health Issues ◽

Arranged Marriage ◽

Love Marriage

Does Phubbing Behavior and Romantic Relation leads to Mental Health issues among married couple? The present study intended to explore the effect of how phubbing behavior and romantic relationships are affecting mental health of married couples. For the study purpose a sample of 120 Married couples were taken between the age ranges 20-60 years of age. Partner Phubbing Rating Scale developed by Roberts and David (2015) was used to measure phubbing behavior among married couples. Romantic Partner Scale (RPS) developed by Zacchilli, Hendricks, and Hendricks, (2012) was applied to assess the romantic relationship between both partners and the third scale used was short form of Mental Health Continuum developed by Keyes (2005) in order to assess the mental health issue among married couples. Phubbing behavior positively predicted interactional activity and negatively predicts compromise, avoidance, separation, dominance and submission. Phubbing behavior negatively predicts mental health among married couples. Gender difference indicates that males are higher on romantic relationship as well as mental health as compared to females. As far as demographic variables are concerned, based on the findings of current research, it was concluded that there was no gender differences found in phubbing behavior, romantic relationship, and mental health of married couples. Phubbing behavior is significantly higher in love marriage couples in comparison with arranged marriage couples.

Download Full-text

An intervention for depressive symptoms using a commercial game in a mental health impatient setting. (Preprint)

10.2196/preprints.13890 ◽

2019 ◽

Author(s):

Pablo Rodrigo Guzman Cortez ◽

Matias Marzocchi ◽

Neus Freixa Fontanals ◽

Mercedes Balcells-Olivero

Keyword(s):

Mental Health ◽

Depressive Symptoms ◽

Health Outcomes ◽

Serious Games ◽

Rating Scale ◽

Depressive Symptomatology ◽

Pilot Test ◽

Potential Health ◽

Depressive Episodes ◽

Hamilton Rating Scale

BACKGROUND Computerized mental health interventions have shown evidence of their potential benefit for mental health outcomes in young users. All of the studied interventions available in the review and scientific literature can be classified as "serious games". Serious games are computerized interventions designed from the start with the objective of improving specific desired health outcomes. Moreover, there are reports of users experiencing subjective benefits in mental health after playing specific commercial games. These were games not intentionally made with a therapeutic objective in the design process. An example is the videogame "Journey", first released for the Playstation 3 console in 2012 which won "Game of the Year" in the 2013 D.I.C.E awards. The creator of the game describes the game as a short, 2-3-hour narrative experience in which the player goes through the "Hero's Journey" following a classic 3-part structure. There were more than 100 testimonials from players describing how the game helped them cope with psychological or personal issues. Some of them explicitly described recovering from depressive episodes through playing the game. OBJECTIVE To conduct a pilot test of the efficacy of the videogame Journey in reducing depressive symptoms in an acute impatient setting METHODS Depressive symptomatology was measured before and after the intervention using the Hamilton Rating Scale for Depression (HRSD) The intervention was conducted in an isolated room using a Playstation 3 console with the videogame "Journey" developed by Thatgamecompany. No internet access was allowed. The game was played over the course of 4 30-45 min sessions in a two week period. RESULTS The initial score in the Hamilton Rating Scale for Depression (HRSD) was 30, indicating a very severe depression. After the intervention the HRSD score was 10, showing a mild depression. CONCLUSIONS The Videogame Journey, a commercial game first available for the Playstation 3 console in 2012, was not created as a serious game with potential health benefits. Our pilot test is the first case report of a commercial game showing a potential effect in reducing depressive symptoms, which is consistent with the previous informal reports of users online.

Download Full-text

Social support and mental health status of internal migrants in Beijing

Social Behavior and Personality An International Journal ◽

10.2224/sbp.8193 ◽

2020 ◽

Vol 48 (4) ◽

pp. 1-16

Author(s):

Ying Zhou ◽

Jianhua Wang

Keyword(s):

Mental Health ◽

Social Support ◽

Health Status ◽

Marital Status ◽

Rating Scale ◽

Mental Health Disorder ◽

Mental Health Status ◽

Self Report ◽

Monthly Income ◽

Internal Migrants

We investigated the mental health status of 320 internal migrants in Beijing according to gender, age, marital status, and monthly income, and examined the relationship between their mental health status and social support mechanisms. Participants completed the self-report Symptom Checklist-90-R and Social Support Rating Scale. Results showed that their mental health was significantly worse than the Chinese adult norm as assessed in 2017. Participants' social support varied according to age, marital status, and monthly income. Female participants younger than 30 years old with a monthly income lower than 3,000 yuan comprised the group with the most mental health disorder symptoms. They thus required greater personal attention to their health. The results suggested that social support can predict mental health among internal migrants. Directions for further research are discussed.

Download Full-text

Finding Talent Among Elementary English Learners: A Validity Study of the HOPE Teacher Rating Scale

Gifted Child Quarterly ◽

10.1177/0016986220985942 ◽

2021 ◽

pp. 001698622098594

Author(s):

Nielsen Pereira

Keyword(s):

English Language Learners ◽

English Learners ◽

English Language ◽

Rating Scales ◽

Rating Scale ◽

Teacher Rating ◽

Esl Teachers ◽

Invariance Testing ◽

Scale Scores ◽

The One

The purpose of this study was to investigate the validity of the HOPE Scale for identifying gifted English language learners (ELs) and how classroom and English as a second language (ESL) teacher HOPE Scale scores differ. Seventy teachers completed the HOPE Scale on 1,467 students in grades K-5 and four ESL teachers completed the scale on 131 ELs. Measurement invariance tests indicated that the HOPE Scale yields noninvariant latent means across EL and English proficient (EP) samples. However, confirmatory factor analysis results support the use of the scale with ELs or EP students separately. Results also indicate that the rating patterns of classroom and ESL teachers were different and that the HOPE Scale does not yield valid data when used by ESL teachers. Caution is recommended when using the HOPE Scale and other teacher rating scales to compare ELs to EP students. The importance of invariance testing before using an instrument with a population that is different from the one(s) for which the instrument was developed is discussed.

Download Full-text