scholarly journals Prioritization of Quality Principles for Health Apps Using the Kano Model: Survey Study

10.2196/26563 ◽  
2022 ◽  
Vol 10 (1) ◽  
pp. e26563
Author(s):  
Christin Malinka ◽  
Ute von Jan ◽  
Urs-Vito Albrecht

Background Health apps are often used without adequately taking aspects related to their quality under consideration. This may partially be due to inadequate awareness about necessary criteria and how to prioritize them when evaluating an app. Objective The aim of this study was to introduce a method for prioritizing quality attributes in the mobile health context. To this end, physicians were asked about their assessment of nine app quality principles relevant in health contexts and their responses were used as a basis for designing a method for app prioritization. Ultimately, the goal was to aid in making better use of limited resources (eg, time) by assisting with the decision as to the specific quality principles that deserve priority in everyday medical practice and those that can be given lower priority, even in cases where the overall principles are rated similarly. Methods A total of 9503 members of two German professional societies in the field of orthopedics were invited by email to participate in an anonymous online survey over a 1-month period. Participants were asked to rate a set of nine app quality principles using a Kano survey with functional and dysfunctional (ie, positively and negatively worded) questions. The evaluation was based on the work of Kano (baseline), supplemented by a self-designed approach. Results Among the 9503 invited members, 382 completed relevant parts of the survey (return rate of 4.02%). These participants were equally and randomly assigned to two groups (test group and validation group, n=191 each). Demographic characteristics did not significantly differ between groups (all P>.05). Participants were predominantly male (328/382, 85.9%) and older than 40 years (290/382, 75.9%). Given similar ratings, common evaluation strategies for Kano surveys did not allow for conclusive prioritization of the principles, and the same was true when using the more elaborate approach of satisfaction and dissatisfaction indices following the work of Timko. Therefore, an extended, so-called “in-line-of-sight” method was developed and applied for this evaluation. Modified from the Timko method, this approach is based on a “point of view” (POV) metric, which generates a ranking coefficient. Although the principles were previously almost exclusively rated as must-be (with the exception of resource efficiency), which was not conducive to their prioritization, the new method applied from the must-be POV resulted in identical rankings for the test and validation groups: (1) legal conformity, (2) content validity, (3) risk adequacy, (4) practicality, (5) ethical soundness, (6) usability, (7) transparency, (8) technical adequacy, and (9) resource efficiency. Conclusions Established survey methodologies based on the work of Kano predominantly seek to categorize the attributes to be evaluated. The methodology presented here is an interesting option for prioritization, and enables focusing on the most important criteria, thus saving valuable time when reviewing apps for use in the medical field, even with otherwise largely similar categorization results. The extent to which this approach is applicable beyond the scenario presented herein requires further investigation.

2020 ◽  
Author(s):  
Christin Malinka ◽  
Ute von Jan ◽  
Urs-Vito Albrecht

BACKGROUND Health apps are often used without adequately taking aspects related to their quality under consideration. This may partially be due to inadequate awareness about necessary criteria and how to prioritize them when evaluating an app. OBJECTIVE The aim of this study was to introduce a method for prioritizing quality attributes in the mobile health context. To this end, physicians were asked about their assessment of nine app quality principles relevant in health contexts and their responses were used as a basis for designing a method for app prioritization. Ultimately, the goal was to aid in making better use of limited resources (eg, time) by assisting with the decision as to the specific quality principles that deserve priority in everyday medical practice and those that can be given lower priority, even in cases where the overall principles are rated similarly. METHODS A total of 9503 members of two German professional societies in the field of orthopedics were invited by email to participate in an anonymous online survey over a 1-month period. Participants were asked to rate a set of nine app quality principles using a Kano survey with functional and dysfunctional (ie, positively and negatively worded) questions. The evaluation was based on the work of Kano (baseline), supplemented by a self-designed approach. RESULTS Among the 9503 invited members, 382 completed relevant parts of the survey (return rate of 4.02%). These participants were equally and randomly assigned to two groups (test group and validation group, n=191 each). Demographic characteristics did not significantly differ between groups (all <i>P</i>&gt;.05). Participants were predominantly male (328/382, 85.9%) and older than 40 years (290/382, 75.9%). Given similar ratings, common evaluation strategies for Kano surveys did not allow for conclusive prioritization of the principles, and the same was true when using the more elaborate approach of satisfaction and dissatisfaction indices following the work of Timko. Therefore, an extended, so-called “in-line-of-sight” method was developed and applied for this evaluation. Modified from the Timko method, this approach is based on a “point of view” (POV) metric, which generates a ranking coefficient. Although the principles were previously almost exclusively rated as <i>must-be</i> (with the exception of resource efficiency), which was not conducive to their prioritization, the new method applied from the <i>must-be</i> POV resulted in identical rankings for the test and validation groups: (1) legal conformity, (2) content validity, (3) risk adequacy, (4) practicality, (5) ethical soundness, (6) usability, (7) transparency, (8) technical adequacy, and (9) resource efficiency. CONCLUSIONS Established survey methodologies based on the work of Kano predominantly seek to categorize the attributes to be evaluated. The methodology presented here is an interesting option for prioritization, and enables focusing on the most important criteria, thus saving valuable time when reviewing apps for use in the medical field, even with otherwise largely similar categorization results. The extent to which this approach is applicable beyond the scenario presented herein requires further investigation.


10.2196/16442 ◽  
2019 ◽  
Vol 7 (11) ◽  
pp. e16442 ◽  
Author(s):  
Urs-Vito Albrecht ◽  
Theodor Framke ◽  
Ute von Jan

Background Meta-information provided about health apps on app stores is often the only readily available source of quality-related information before installation. Objective The purpose of this study was to assess whether physicians deem a predefined set of quality principles as relevant for health apps; whether they are able to identify corresponding information in a given sample of app descriptions; and whether, and how, this facilitates their informed usage decisions. Methods All members of the German Society for Internal Medicine were invited by email to participate in an anonymous online survey over a 6-week period. Participants were randomly assigned one app description focusing on cardiology or pulmonology. In the survey, participants were asked three times about whether the assigned description sufficed for a usage decision: they were asked (1) after giving an appraisal of the relevance of nine predefined app quality principles, (2) after determining whether the descriptions covered the quality principles, and (3) after they assessed the availability of detailed quality information by means of 25 additional key questions. Tests for significance of changes in their decisions between assessments 1 and 2, and between assessments 2 and 3, were conducted with the McNemar-Bowker test of symmetry. The effect size represents the discordant proportion ratio sum as a quotient of the test statistics of the Bowker test and the number of observation units. The significance level was set to alpha=.05 with a power of 1-beta=.95. Results A total of 441 of 724 participants (60.9%) who started the survey fully completed the questionnaires and were included in the evaluation. The participants predominantly rated the specified nine quality principles as important for their decision (approximately 80%-99% of ratings). However, apart from the practicality criterion, information provided in the app descriptions was lacking for both groups (approximately 51%-92%). Reassessment of the apps led to more critical assessments among both groups. After having familiarized themselves with the nine quality principles, approximately one-third of the participants (group A: 63/220, 28.6%; group B: 62/221, 28.1%) came to more critical usage decisions in a statistically significant manner (McNemar-Bowker test, groups A and B: P<.001). After a subsequent reassessment with 25 key questions, critical appraisals further increased, although not in a statistically significant manner (McNemar-Bowker, group A: P=.13; group B: P=.05). Conclusions Sensitizing physicians to the topic of quality principles via questions about attitudes toward established quality principles, and letting them apply these principles to app descriptions, lead to more critical appraisals of the sufficiency of the information they provided. Even working with only nine generic criteria was sufficient to bring about the majority of decision changes. This may lay the foundation for aiding physicians in their app-related decision processes, without unduly taking up their valuable time.


2019 ◽  
Author(s):  
Miguel Angel Mayer ◽  
Octavi Rodríguez Blanco ◽  
Antonio Torrejon

BACKGROUND In the last few years, the number of mobile apps for health professionals has increased exponentially. Nevertheless, there is a lack of knowledge about the professional use, training requirements, and quality perception of these apps among health care professionals such as nurses. Considering that the nursing profession is the largest segment of health care workforce in many countries such as Spain, the impact of the use of health apps by these professionals can be critical to the future of modern health care. OBJECTIVE The main objective of this study was to determine if nurses were using health apps professionally and what types of apps they were using. The secondary objectives were (1) to find out if, among nurses, there is a need for training in the use of health apps and (2) to explore nurses’ perceptions of health professional apps, determining whether there is a need for a certification process for health apps and the type of institution or organization that should review and validate these apps for professional use. METHODS After an initial piloting survey, all registered nurses at the Nursing Association of Barcelona were invited to participate in a 34-item online survey. Eventually, 1293 nurses participated in the survey; however, 52 did not complete the survey properly, omitting both age or gender information, and they were excluded from the analysis. RESULTS About half of the respondents (600/1241, 48.35%) had health professional apps installed on their devices and were included for analysis. Most participants in the survey were women (474/600, 79.0%) and the remaining were men (126/600, 21.0%). The most popular types of apps used and installed among nurses were related to drug information, health calculators, and health guidelines. Overall, 97.0% (582/600) of nurses thought that the health apps should be certified, and 80.0% (480/600) agreed that the certification process should be carried out by professional or health institutions. Furthermore, 14.5% (87/600) of participants mentioned that they were asked by their patients to prescribe a health app and only 6.5% (28/430) recommended them often. Most nurses (354/433, 81.8%) who answered the question about the importance of receiving specific training on using and prescribing health apps considered this point a very relevant issue. CONCLUSIONS About half of the nurses in Catalonia use health apps for professional purposes, and they believe that these types of tools should be validated and certified by health or professional institutions before using them in clinical environments. Although the prescription of health apps in clinical environments is infrequent among nurses, they would be willing to prescribe apps if they were certified by a health organization. Finally, among nurses, there is a need for training in using and prescribing health apps for health care purposes.


2019 ◽  
Author(s):  
Urs-Vito Albrecht ◽  
Christin Malinka ◽  
Sarah Long ◽  
Tobias Raupach ◽  
Gerd Hasenfuß ◽  
...  

BACKGROUND Currently, there are no binding requirements for manufacturers prescribing which information must be included in the app descriptions of health apps. OBJECTIVE The aim of this study was to investigate how medical students perceive a selection of quality principles, intended for usage decisions in the app context, and establish whether the information presented in a sample of app descriptions is perceived as sufficient for facilitating an informed usage decision. METHODS A total of 123 students (mean age 24.2 years, SD 3.4) participating in a 6-week teaching module covering cardiology and pulmonology at the University of Göttingen (original enrollment 152 students, response rate 80.9%) were included. Students were asked to read 3 store description texts of cardiological or pneumological apps and initially assess whether the descriptions sufficed for a usage decision. Subsequently, they were queried on their perception of the relevance of 9 predefined quality principles, formulated for usage decisions. An appraisal of whether the app description texts contained sufficient information to satisfy these quality principles followed. By means of 20 guiding questions, participants were then asked to identify relevant information (or a lack thereof) within the descriptions. A reassessment of whether the description texts sufficed for making a usage decision ensued. A total of 343 complete datasets were obtained. RESULTS A majority of the quality principles were described as “very important” and “important” for making a usage decision. When accessed via the predefined principles, students felt unable to identify sufficient information within the app descriptions in 68.81% (2124/3087) of cases. Notably, information regarding undesired effects (91.8%, 315/343), ethical soundness (90.1%, 309/343), measures taken to avert risks (89.2%, 306/343), conflicts of interest (88.3%, 303/343), and the location of data storage (87.8%, 301/343) was lacking. Following participants’ engagement with the quality principles, statistically significant changes in their assessment of whether the app descriptions sufficed for a usage decision can be seen—McNemar-Bowker test (3)=45.803919, P<.001, Cohen g=.295. In 34.1% (117/343) cases, the assessment was revised. About 3 quarters of changed assessments were seen more critically (76.9%, 90/117). Although, initially, 70% (240/343) had been considered “sufficient,” this rate was reduced to 54.2% (186/343) in the second assessment. CONCLUSIONS In a considerable number of app descriptions, participants were unable to locate the information necessary for making an informed usage decision. Participants’ sensitization to the quality principles led to changes in their assessment of app descriptions as a tool for usage decisions. Better transparency in app descriptions released by manufacturers and the exposure of users to quality principles could collectively form the basis for well-founded usage decisions.


10.2196/13375 ◽  
2019 ◽  
Vol 7 (2) ◽  
pp. e13375 ◽  
Author(s):  
Urs-Vito Albrecht ◽  
Christin Malinka ◽  
Sarah Long ◽  
Tobias Raupach ◽  
Gerd Hasenfuß ◽  
...  

10.2196/15195 ◽  
2019 ◽  
Vol 7 (11) ◽  
pp. e15195 ◽  
Author(s):  
Miguel Angel Mayer ◽  
Octavi Rodríguez Blanco ◽  
Antonio Torrejon

Background In the last few years, the number of mobile apps for health professionals has increased exponentially. Nevertheless, there is a lack of knowledge about the professional use, training requirements, and quality perception of these apps among health care professionals such as nurses. Considering that the nursing profession is the largest segment of health care workforce in many countries such as Spain, the impact of the use of health apps by these professionals can be critical to the future of modern health care. Objective The main objective of this study was to determine if nurses were using health apps professionally and what types of apps they were using. The secondary objectives were (1) to find out if, among nurses, there is a need for training in the use of health apps and (2) to explore nurses’ perceptions of health professional apps, determining whether there is a need for a certification process for health apps and the type of institution or organization that should review and validate these apps for professional use. Methods After an initial piloting survey, all registered nurses at the Nursing Association of Barcelona were invited to participate in a 34-item online survey. Eventually, 1293 nurses participated in the survey; however, 52 did not complete the survey properly, omitting both age or gender information, and they were excluded from the analysis. Results About half of the respondents (600/1241, 48.35%) had health professional apps installed on their devices and were included for analysis. Most participants in the survey were women (474/600, 79.0%) and the remaining were men (126/600, 21.0%). The most popular types of apps used and installed among nurses were related to drug information, health calculators, and health guidelines. Overall, 97.0% (582/600) of nurses thought that the health apps should be certified, and 80.0% (480/600) agreed that the certification process should be carried out by professional or health institutions. Furthermore, 14.5% (87/600) of participants mentioned that they were asked by their patients to prescribe a health app and only 6.5% (28/430) recommended them often. Most nurses (354/433, 81.8%) who answered the question about the importance of receiving specific training on using and prescribing health apps considered this point a very relevant issue. Conclusions About half of the nurses in Catalonia use health apps for professional purposes, and they believe that these types of tools should be validated and certified by health or professional institutions before using them in clinical environments. Although the prescription of health apps in clinical environments is infrequent among nurses, they would be willing to prescribe apps if they were certified by a health organization. Finally, among nurses, there is a need for training in using and prescribing health apps for health care purposes.


2020 ◽  
Author(s):  
Michael Bollwerk ◽  
Bernd Schlipphak ◽  
Joscha Stecker ◽  
Jens Hellmann ◽  
Gerald Echterhoff ◽  
...  

Threat perceptions towards immigrants continue to gain importance in the context of growing international migration. To reduce associated intergroup conflicts, it is crucial to understand the personal and contextual determinants of perceived threat. In a large online survey study (N = 1,184), we investigated the effects of ideology (i.e., Right-Wing Authoritarianism and Social Dominance Orientation), subjective societal status (SSS) and their interaction effects in predicting symbolic and realistic threat perceptions towards Middle Eastern immigrants. Results showed that ideology (higher RWA and SDO) and lower SSS significantly predicted both symbolic and realistic threat, even after controlling for income, education, age, and gender. Furthermore, ideology and SSS interacted significantly in predicting realistic threat, with higher levels of SDO and RWA enhancing the effect of SSS. In the discussion, we focus on the implications of our findings with respect to understanding societal conflicts, discuss methodological limitations, and provide directions for future research.


2020 ◽  
Author(s):  
Khanh Ngoc Cong Duong ◽  
Tien Nguyen Le Bao ◽  
Phuong Thi Lan Nguyen ◽  
Thanh Vo Van ◽  
Toi Phung Lam ◽  
...  

BACKGROUND The first nationwide lockdown due to the COVID-19 pandemic was implemented in Vietnam from April 1 to 15, 2020. Nevertheless, there has been limited information on the impact of COVID-19 on the psychological health of the public. OBJECTIVE This study aimed to estimate the prevalence of psychological issues and identify the factors associated with the psychological impact of COVID-19 during the first nationwide lockdown among the general population in Vietnam. METHODS We employed a cross-sectional study design with convenience sampling. A self-administered, online survey was used to collect data and assess psychological distress, depression, anxiety, and stress of participants from April 10 to 15, 2020. The Impact of Event Scale-Revised (IES-R) and the Depression, Anxiety, and Stress Scale-21 (DASS-21) were utilized to assess psychological distress, depression, anxiety, and stress of participants during social distancing due to COVID-19. Associations across factors were explored using regression analysis. RESULTS A total of 1385 respondents completed the survey. Of this, 35.9% (n=497) experienced psychological distress, as well as depression (n=325, 23.5%), anxiety (n=195, 14.1%), and stress (n=309, 22.3%). Respondents who evaluated their physical health as average had a higher IES-R score (beta coefficient [B]=9.16, 95% CI 6.43 to 11.89), as well as higher depression (B=5.85, 95% CI 4.49 to 7.21), anxiety (B=3.64, 95% CI 2.64 to 4.63), and stress (B=5.19, 95% CI 3.83 to 6.56) scores for DASS-21 than those who rated their health as good or very good. Those who self-reported their health as bad or very bad experienced more severe depression (B=9.57, 95% CI 4.54 to 14.59), anxiety (B=7.24, 95% CI 3.55 to 10.9), and stress (B=10.60, 95% CI 5.56 to 15.65). Unemployment was more likely to be associated with depression (B=3.34, 95% CI 1.68 to 5.01) and stress (B=2.34, 95% CI 0.84 to 3.85). Regarding worries about COVID-19, more than half (n=755, 54.5%) expressed concern for their children aged &lt;18 years, which increased their IES-R score (B=7.81, 95% CI 4.98 to 10.64) and DASS-21 stress score (B=1.75, 95% CI 0.27 to 3.24). The majority of respondents (n=1335, 96.4%) were confident about their doctor’s expertise in terms of COVID-19 diagnosis and treatment, which was positively associated with less distress caused by the outbreak (B=–7.84, 95% CI –14.58 to –1.11). CONCLUSIONS The findings highlight the effect of COVID-19 on mental health during the nationwide lockdown among the general population in Vietnam. The study provides useful evidence for policy decision makers to develop and implement interventions to mitigate these impacts. CLINICALTRIAL


2020 ◽  
Author(s):  
Muhammad Jee ◽  
Alsa Khan

BACKGROUND Understanding teens' relationship with technology is a pressing topic in this digital era. There seem to be both benefit and harmful implications that originate from use of technology by teens. Approximately 95% of teenagers have access to a smartphone, and several studies show a positive correlation between screen addiction and trends of Anxiety and Depression. While, at the same time, research shows that two-thirds of teenagers believe that technology is a necessity for connecting and making new friends. OBJECTIVE The aim of this research is to understand teens’ perceptions of their own and others’ relationship with personal technology and what future relationships do teens aspire to have with their technology. This will be fundamental in helping designers, healthcare practitioners and youth counselors inform design in technology accordingly. METHODS This research paper explores teens and technology relationship via three lenses. Lens-1 is teens' perception of their current relationship with their personal technology as users. Lens-2 is teens' perception of others' (parents, siblings or friends) relationship with personal technology as bystanders. Lens-3 is teens’ aspiration of how they wish to relate to their personal technology. We conducted an online survey with 619 teens in the 13-19 years age range. RESULTS We found that teens selected 'Essential,' 'Distractive,' and 'Addictive' most commonly to describe their own and others' relationship with personal technology. Whereas teens selected 'Provides an escape' more to describe their own relationship with technology. In contrast, they selected 'It's just a tool' and 'Creates Barrier' more to describe others' relationship with technology. These trends are consistent across ages and genders. Additionally, we found that 13-15 year-olds, particularly the majority of females in this age range, wish to see their future technology as their 'best friend.' However, 16-17 year-olds’ top choice was 'I don't believe in personal connection with mobile technology,’ and 18-19 year-olds’ top choice was 'My personal assistant.’ CONCLUSIONS Our three lenses approach is holistic as it allowed comparison of how teens perceive their relationship with personal technology as users vs. bystanders and from current vs. aspirational perspectives. This research not just confirms what is already known, but uncovers critical new associations more exclusive to teens' own relationship with technology. This paper presents related design implications to inform personal tech design thinking, including our guideline to shift 'user experience design' process to 'user relationship design.'


2021 ◽  
Vol 8 ◽  
pp. 237437352110076
Author(s):  
Hyllore Imeri ◽  
Erin Holmes ◽  
Shane Desselle ◽  
Meagen Rosenthal ◽  
Marie Barnard

Chronic conditions (CCs) management during the COVID-19 pandemic and the impact of the pandemic on patient activation (PA) and health locus of control (HLOC) remain unknown. This cross-sectional online survey study examined the role of COVID-19 pandemic-related worry or fear in PA and HLOC among patients with CCs. Individuals with CCs (n = 300) were recruited through MTurk Amazon. The questionnaire included sociodemographic questions, the Patient Activation Measure, and the Multidimensional Health Locus of Control–Form B. Out of the 300 participants, 9.7% were diagnosed with COVID-19, and 7.3% were hospitalized. Patients with cancer, chronic kidney disease, chronic obstructive pulmonary disease, drug abuse/substance abuse, and stroke reported significant difficulties in managing their CCs due to worry or fear because of COVID-19. More than half of the sample (45.7%) reported COVID-19-related worry or fear about managing their CCs, and these patients had lower PA and lower external HLOC compared to patients not affected by COVID-19-related worry or fear. Health professionals should provide more support for patients facing difficulties in managing their CCs during the COVID-19 pandemic.


Sign in / Sign up

Export Citation Format

Share Document