scholarly journals Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems

2016 ◽  
Vol 24 (e1) ◽  
pp. e55-e60 ◽  
Author(s):  
Reza Khajouei ◽  
Misagh Zahiri Esfahani ◽  
Yunes Jahani
Keyword(s):  
Information Systems ◽  
Evaluation Methods ◽  
Usability Evaluation ◽  
Health Care Information ◽  
Office Management ◽  
Usability Problems ◽  
Significant Difference ◽  
Health Care Information Systems ◽  
Cognitive Walkthrough ◽  
Usability Evaluation Methods

Objective: There are several user-based and expert-based usability evaluation methods that may perform differently according to the context in which they are used. The objective of this study was to compare 2 expert-based methods, heuristic evaluation (HE) and cognitive walkthrough (CW), for evaluating usability of health care information systems. Materials and methods: Five evaluators independently evaluated a medical office management system using HE and CW. We compared the 2 methods in terms of the number of identified usability problems, their severity, and the coverage of each method. Results: In total, 156 problems were identified using the 2 methods. HE identified a significantly higher number of problems related to the “satisfaction” attribute (P = .002). The number of problems identified using CW concerning the “learnability” attribute was significantly higher than those identified using HE (P = .005). There was no significant difference between the number of problems identified by HE, based on different usability attributes (P = .232). Results of CW showed a significant difference between the number of problems related to usability attributes (P < .0001). The average severity of problems identified using CW was significantly higher than that of HE (P < .0001). Conclusion: This study showed that HE and CW do not differ significantly in terms of the number of usability problems identified, but they differ based on the severity of problems and the coverage of some usability attributes. The results suggest that CW would be the preferred method for evaluating systems intended for novice users and HE for users who have experience with similar systems. However, more studies are needed to support this finding.

scholarly journals Comparison Usability Evaluation Methods in a Health Information System: Heuristic Evaluation versus Cognitive Walkthrough Method

2021 ◽  
Author(s):  
Mehrdad Farzandipour ◽  
Ehsan Nabovati ◽  
Hamidreza Tadayon ◽  
Monireh Sadeqi Jabali
Keyword(s):  
Information System ◽  
Evaluation Method ◽  
Health Information System ◽  
Evaluation Methods ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Usability Problems ◽  
Significant Difference ◽  
Cognitive Walkthrough ◽  
Usability Evaluation Methods

Abstract Background There are some inconsistencies regarding the selection of the most appropriate usability evaluation method. The present study aimed to compare two expert-based evaluation methods in a nursing module, as the most widely used module of a Hospital Information System (HIS). Methods The Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods were used by five independent evaluators to evaluate the nursing module of Shafa HIS. In this regard, the number, severity and ratio of the recognized problems according to the usability attributes were compared using two evaluation methods. Results The use of the HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively. The average severity of the recognized problems was 2.32 in the HE method and 2.77 in the CW evaluation method; however, there was a significant difference between the number and severity of recognized usability problems by these methods (P < 0.001). Some problems, which were associated with effectiveness, satisfaction and error, were better recognized by the HE method; however, CW evaluation method was more successful in recognizing problems of learnability, efficiency and memorability. Conclusion The HE method recognized more problems with a lower average severity. On the other hand, CW could recognize fewer problems with a higher average severity. Regarding the evaluation goal, HE method would be used to improve effectiveness, increase satisfaction and decrease the number of errors. Furthermore, CW evaluation method is recommended to be used to improve the learnability, efficiency and memorability of the system.

scholarly journals COMPARISON OF HEURISTIC EVALUATION AND COGNITIVE WALKTHROUGH METHODS IN DOING USABILITY EVALUATION OF MOBILE-BASED DEL EGOV CENTRE HOSPITAL INFORMATION SYSTEM

SEMINASTIKA ◽  
2021 ◽  
Vol 3 (1) ◽  
pp. 99-106
Author(s):  
Gracella Tambunan ◽  
Lit Malem Ginting
Keyword(s):  
Mobile Application ◽  
Evaluation Method ◽  
Likert Scale ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Usability Problems ◽  
Interactive Product ◽  
Cognitive Walkthrough ◽  
Centre Hospital ◽  
Usability Evaluation Methods

Usability is a factor that indicates the success of an interactive product or system, such as a mobile application. The increasing use of smartphones demands a more accurate and effective usability evaluation method to find usability problems, so that they can be used for product improvement in the development process. This study compares the Cognitive Walkthrough method with Heuristic Evaluation in evaluating the usability of the SIRS Del eGov Center mobile application. Evaluation with these two methods will be carried out by three evaluators who act as experts. Finding problems and recommending improvements from each method will produce an improvement prototype made in the form of a high-fidelity prototype. Each prototype will be tested against ten participants using the Usability Testing method, which will generate scores through the SUS table. From the test scores, the percentage of Likert scale and the success rate of each prototype will be found. The results show that between the two usability evaluation methods, the Heuristic Evaluation method is the more effective method, finds more usability problems, and has a higher Likert scale percentage, which is 66.5%, while Cognitive Walkthrough is 64.75%.

scholarly journals Usability as a challenge in agricultural engineering

2006 ◽  
Author(s):  
Hannu Haapala ◽  
Piia Nurkka ◽  
Kim Kaustell ◽  
Tiina Mattila ◽  
Juha Suutarinen
Keyword(s):  
Product Development ◽  
Initial Phase ◽  
Evaluation Methods ◽  
Usability Evaluation ◽  
Electronic Control ◽  
Engineering Research ◽  
Agricultural Engineering ◽  
Mobile Work ◽  
Usability Problems ◽  
Usability Evaluation Methods

One of the primary goals of research activities has always been to apply the newest potential results into practise. This is the case especially in engineering research. Recently, as productivity has gained importance as a quality measure for research, rapid application of the results has become even more important. Consequently, applicability has been lifted as a major criterion in the application for public funding thus promoting product development as an integrated part of research projects. No product (or research result) does have any impact if it is not taken into practical use. The end customers of research are supposed to take the developed products into active practical use. Without that phase all our efforts are put in vain. Usability is an important part of user acceptance. According to Nielsen (1993) system acceptability includes a social and a practical part. To be practically acceptable the product should be e.g. economical, compatible, reliable and useful. To be useful the product should be functionally suitable for the user’s tasks and usable. Usability includes easy learn ability, efficiency in use, remember ability, lack of errors in operation and subjective pleasure. In agricultural research there are distinct problems in usability when considering the phase oftaking the results into practical use. Of course, there are challenges also in the initial phase of research such as choosing research topics and later in the initial phase of product development. Usability, however, by far dictates user experience and thus decides if the product is taken into wide use or not. Consequently, MTT Agricultural Engineering Research has set usability and acceptability research as an important research topic. Usability in agricultural engineering is a complex issue since the context of use is variable. Mobile work is typical to agricultural producers.In this study, an example of usability evaluation is presented. Mobile work in the context of electronic control of precision combined drilling is evaluated. The research themes were:1. How great a challenge is usability in Precision Agriculture (PA)?- is it the cause for poor market penetration?2. Which usability evaluation methods are applicable to PA?- are there special issues in PA, or agricultural engineering generally, which limit the feasibilityof some methods?3. Which kind of usability problems can be detected with one selected method (heuristicevaluation)?- a demonstration of usability evaluation: the Human-Machine Interface (HMI) of a precisioncombined drillAccording to literature usability issues have not been a central issue in electronics development in agriculture. Poor experiences of unacceptable operation could be one reason for the customers not relying on new electronic control systems such as those of PA. There are multiple potential usability evaluation methods for agricultural engineering. The results from the case study performed show that heuristic evaluation is a suitable method for detecting design deficiencies in electronic control of mobile PA. To get a wider picture, further studies with other methods and applications should be done.

The Evaluator Effect in Usability Studies: Problem Detection and Severity Judgments

1998 ◽  
Vol 42 (19) ◽  
pp. 1336-1340 ◽  
Author(s):  
Niels Ebbe Jacobsen ◽  
Morten Hertzum ◽  
Bonnie E. John
Keyword(s):  
Evaluation Methods ◽  
Usability Evaluation ◽  
Individual Variability ◽  
Potential Threat ◽  
Usability Test ◽  
Usability Problems ◽  
Problem Detection ◽  
Usability Evaluation Methods ◽  
Selection Of

Usability studies are commonly used in industry and applied in research as a yardstick for other usability evaluation methods. Though usability studies have been studied extensively, one potential threat to their reliability has been left virtually untouched: the evaluator effect. In this study, four evaluators individually analyzed four videotaped usability test sessions. Only 20% of the 93 detected problems were detected by all evaluators, and 46% were detected by only a single evaluator. From the total set of 93 problems the evaluators individually selected the ten problems they considered most severe. None of the selected severe problems appeared on all four evaluators' top-10 lists, and 4 of the 11 problems that were considered severe by more than one evaluator were only detected by one or two evaluators. Thus, both detection of usability problems and selection of the most severe problems are subject to considerable individual variability.

The State of Art in the Science of Usability Evaluation Methods: A Kuhnian Perspective

2000 ◽  
Vol 44 (37) ◽  
pp. 577-580
Author(s):  
Niels Ebbe Jacobsen ◽  
Anker Helms Jørgensen
Keyword(s):  
Evaluation Methods ◽  
Usability Evaluation ◽  
Scientific Revolutions ◽  
Usability Problems ◽  
Research Activities ◽  
Current State ◽  
Technological Artifacts ◽  
Three Stages ◽  
Usability Evaluation Methods ◽  
State Of Art

In the 1980's researchers and practitioners developed usability evaluation methods (UEMs) that aimed at identifying usability problems in technological artifacts. Among the most known UEMs are Usability Test, Heuristic Evaluation and Cognitive Walkthrough. The development of UEMs was followed by research activities aiming at evaluating and further developing these UEMs. As the methods in addition have gained wide acceptance in design practice, the field of UEMs seems to have matured considerably. However, closer inspection reveals that there is still considerable lack of coherence and agreement. The publication of a controversial paper by Gray & Salzman (1998) underscored this point in that they questioned the methodological validity of five previously published experimental UEM studies. In addition, ten distinguished UEM researchers' very different opinions based on the Gray & Salzman paper left the impression that research in the field of UEM is far from being coherent. In order to throw light upon the current state of art of science in the field of UEMs, this paper analyzes the maturity of the field based on Thomas Kuhn's theory of scientific revolutions. We find that the field is currently in the first of three stages, the pre-paradigmatic stage, as it is lacking a general conceptual framework, as basic terms are ill-defined, and as researchers “facts-gather” almost randomly in the absence of a reason for seeking some particular form of more recondite information.

Sign in / Sign up

Export Citation Format

Share Document