Determining the Effectiveness of the Usability Problem Inspector: A Theory-Based Model and Tool for Finding Usability Problems

2003 ◽  
Vol 45 (3) ◽  
pp. 455-482 ◽  
Author(s):  
Terence S. Andre ◽  
H. Rex Hartson ◽  
Robert C. Williges
Keyword(s):  
Interface Design ◽  
Cost Effective ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Evaluation Tool ◽  
Stage Of Development ◽  
Usability Problems ◽  
Usability Problem ◽  
Potential Applications ◽  
Cognitive Walkthrough

Despite the increased focus on usability and on the processes and methods used to increase usability, a substantial amount of software is unusable and poorly designed. Much of this is attributable to the lack of cost-effective usability evaluation tools that provide an interaction-based framework for identifying problems. We developed the user action framework and a corresponding evaluation tool, the usability problem inspector (UPI), to help organize usability concepts and issues into a knowledge base. We conducted a comprehensive comparison study to determine if our theory-based framework and tool could be effectively used to find important usability problems in an interface design, relative to two other established inspection methods (heuristic evaluation and cognitive walkthrough). Results showed that the UPI scored higher than heuristic evaluation in terms of thoroughness, validity, and effectiveness and was consistent with cognitive walkthrough for these same measures. We also discuss other potential advantages of the UPI over heuristic evaluation and cognitive walkthrough when applied in practice. Potential applications of this work include a cost-effective alternative or supplement to lab-based formative usability evaluation during any stage of development.

Examination of the Use of Nielsen’s 10 Usability Heuristics & Outlooks for the Future

2017 ◽  
Vol 61 (1) ◽  
pp. 1472-1475 ◽  
Author(s):  
Emily Gonzalez-Holland ◽  
Daphne Whitmer ◽  
Larry Moralez ◽  
Mustapha Mouloua
Keyword(s):  
Design Process ◽  
Interface Design ◽  
Cost Effective ◽  
Rapid Expansion ◽  
Heuristic Methods ◽  
Heuristic Evaluation ◽  
Usability Problems ◽  
The Future ◽  
Usability Evaluation Methods ◽  
Practical Implications

Heuristics are commonly employed throughout various stages of the design process to evaluate the usability of interfaces. Heuristic Evaluation (HE) provides researchers with a cost effective and practical means to effectively assess designs. In this article, we aim to outline the development and application of one of the most frequently cited set of heuristic evaluation tools, Nielsen’s (1994) 10 usability heuristics. Nielsen’s heuristics have not only been applied to various modalities of interface design, but have also been compared to other usability evaluation methods. Moreover, in many cases they have been modified so that they can be applied in an ever-changing socio-technical environment. In reviewing these developments, we propose theoretical and practical implications of these heuristic methods and present an outlook for the future. We argue that with the rapid expansion and growth of technology in the last 20 years, Nielsen’s 10 usability heuristics may need an update to remain consistent with modern usability problems.

scholarly journals COMPARISON OF HEURISTIC EVALUATION AND COGNITIVE WALKTHROUGH METHODS IN DOING USABILITY EVALUATION OF MOBILE-BASED DEL EGOV CENTRE HOSPITAL INFORMATION SYSTEM

SEMINASTIKA ◽  
2021 ◽  
Vol 3 (1) ◽  
pp. 99-106
Author(s):  
Gracella Tambunan ◽  
Lit Malem Ginting
Keyword(s):  
Mobile Application ◽  
Evaluation Method ◽  
Likert Scale ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Usability Problems ◽  
Interactive Product ◽  
Cognitive Walkthrough ◽  
Centre Hospital ◽  
Usability Evaluation Methods

Usability is a factor that indicates the success of an interactive product or system, such as a mobile application. The increasing use of smartphones demands a more accurate and effective usability evaluation method to find usability problems, so that they can be used for product improvement in the development process. This study compares the Cognitive Walkthrough method with Heuristic Evaluation in evaluating the usability of the SIRS Del eGov Center mobile application. Evaluation with these two methods will be carried out by three evaluators who act as experts. Finding problems and recommending improvements from each method will produce an improvement prototype made in the form of a high-fidelity prototype. Each prototype will be tested against ten participants using the Usability Testing method, which will generate scores through the SUS table. From the test scores, the percentage of Likert scale and the success rate of each prototype will be found. The results show that between the two usability evaluation methods, the Heuristic Evaluation method is the more effective method, finds more usability problems, and has a higher Likert scale percentage, which is 66.5%, while Cognitive Walkthrough is 64.75%.

Usability Evaluation of E-Learning Systems

2011 ◽  
pp. 3897-3903 ◽  
Author(s):  
Shirish C. Srivastava ◽  
Shalini Chandra ◽  
Hwee Ming Lam
Keyword(s):  
Interface Design ◽  
Usability Evaluation ◽  
Learning System ◽  
Heuristic Evaluation ◽  
Usability Problems ◽  
E Learning ◽  
Small Set ◽  
User Groups ◽  
Definition Of ◽  
Usability Evaluation Methods

Usability evaluation which refers to a series of activities that are designed to measure the effectiveness of a system as a whole, is an important step for determining the acceptance of system by the users. Usability evaluation is becoming important since both user groups, as well as tasks, are increasing in size and diversity. Users are increasingly becoming more informed and, consequently, have higher expectations from the systems. Moreover “system interface” has become a commodity and, hence, user acceptance plays a major role in the success of the system. Currently, there are various usability evaluation methods in vogue, like cognitive walkthrough, think aloud, claims analysis, heuristic evaluation, and so forth. However, for this study we have chosen heuristic evaluation because it is relatively inexpensive, logistically uncomplicated, and is often used as a discount usability-engineering tool (Nielsen, 1994). Heuristic evaluation is a method for finding usability problems in a user interface design by having a small set of evaluators examine an interface and judge its compliance with recognized usability principles. The rest of the chapter is organized as follows: we first look at the definition of e-learning, followed by concepts of usability, LCD, and heuristics. Subsequently, we introduce a methodology for heuristic usability evaluation (Reeves, Benson, Elliot, Grant, Holschuh, Kim, Kim, Lauber, & Loh, 2002), and then use these heuristics for evaluating an existing e-learning system, GETn2. We offer our recommendations for the system and end with a discussion on the contributions of our chapter.

scholarly journals Comparison Usability Evaluation Methods in a Health Information System: Heuristic Evaluation versus Cognitive Walkthrough Method

2021 ◽  
Author(s):  
Mehrdad Farzandipour ◽  
Ehsan Nabovati ◽  
Hamidreza Tadayon ◽  
Monireh Sadeqi Jabali
Keyword(s):  
Information System ◽  
Evaluation Method ◽  
Health Information System ◽  
Evaluation Methods ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Usability Problems ◽  
Significant Difference ◽  
Cognitive Walkthrough ◽  
Usability Evaluation Methods

Abstract Background There are some inconsistencies regarding the selection of the most appropriate usability evaluation method. The present study aimed to compare two expert-based evaluation methods in a nursing module, as the most widely used module of a Hospital Information System (HIS). Methods The Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods were used by five independent evaluators to evaluate the nursing module of Shafa HIS. In this regard, the number, severity and ratio of the recognized problems according to the usability attributes were compared using two evaluation methods. Results The use of the HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively. The average severity of the recognized problems was 2.32 in the HE method and 2.77 in the CW evaluation method; however, there was a significant difference between the number and severity of recognized usability problems by these methods (P < 0.001). Some problems, which were associated with effectiveness, satisfaction and error, were better recognized by the HE method; however, CW evaluation method was more successful in recognizing problems of learnability, efficiency and memorability. Conclusion The HE method recognized more problems with a lower average severity. On the other hand, CW could recognize fewer problems with a higher average severity. Regarding the evaluation goal, HE method would be used to improve effectiveness, increase satisfaction and decrease the number of errors. Furthermore, CW evaluation method is recommended to be used to improve the learnability, efficiency and memorability of the system.

Cognitive Graphical Walkthrough Interface Evaluation

2006 ◽  
pp. 73-78 ◽  
Author(s):  
Athanasis Karoulis ◽  
Stavros Demetriadis ◽  
Andreas Pombortsis
Keyword(s):  
Interface Design ◽  
Formative Evaluation ◽  
Evaluation Method ◽  
Main Idea ◽  
Point Of View ◽  
Cognitive Domain ◽  
Usability Problems ◽  
Usability Problem ◽  
Basic Characteristics ◽  
Under Construction

Interface evaluation of a software system is a procedure intended to identify and propose solutions for usability problems caused by the specific software design. The term evaluation generally refers to the process of “gathering data about the usability of a design or product by a specified group of users for a particular activity within a specified environment or work context” (Preece et al., 1994, p. 602). As already stated, the main goal of an interface evaluation is to discover usability problems. A usability problem may be defined as anything that interferes with a user’s ability to efficiently and effectively complete tasks (Karat et al., 1992). The most applied interface evaluation methodologies are the expert-based and the empirical (user-based) evaluations. Expert evaluation is a relatively cheap and efficient formative evaluation method applied even on system prototypes or design specifications up to the almost-ready-to-ship product. The main idea is to present the tasks supported by the interface to an interdisciplinary group of experts, who will take the part of would-be users and try to identify possible deficiencies in the interface design. According to Reeves (1993), expert-based evaluations are perhaps the most applied evaluation strategy. They provide a crucial advantage that makes them more affordable compared to the empirical ones; in general, it is easier and cheaper to find experts rather than users who are eager to perform the evaluation. The main idea is that experts from different cognitive domains (at least one from the domain of HCI and one from the cognitive domain under evaluation) are asked to judge the interface, everyone from his or her own point of view. It is important that they all are experienced, so they can see the interface through the eyes of the user and reveal problems and deficiencies of the interface. One strong advantage of the methods is that they can be applied very early in the design cycle, even on paper mock-ups. The expert’s expertise allows the expert to understand the functionality of the system under construction, even if the expert lacks the whole picture of the product. A first look at the basic characteristics would be sufficient for an expert. On the other hand, user-based evaluations can be applied only after the product has reached a certain level of completion.

Heuristic Evaluation of eGLU-Box: A Semi-automatic Usability Evaluation Tool for Public Administrations

2019 ◽  
pp. 75-86
Author(s):  
Stefano Federici ◽  
Maria Laura Mele ◽  
Rosa Lanzilotti ◽  
Giuseppe Desolda ◽  
Marco Bracalenti ◽  
...  
Keyword(s):  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Evaluation Tool

scholarly journals Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems

2016 ◽  
Vol 24 (e1) ◽  
pp. e55-e60 ◽  
Author(s):  
Reza Khajouei ◽  
Misagh Zahiri Esfahani ◽  
Yunes Jahani
Keyword(s):  
Information Systems ◽  
Evaluation Methods ◽  
Usability Evaluation ◽  
Health Care Information ◽  
Office Management ◽  
Usability Problems ◽  
Significant Difference ◽  
Health Care Information Systems ◽  
Cognitive Walkthrough ◽  
Usability Evaluation Methods

Objective: There are several user-based and expert-based usability evaluation methods that may perform differently according to the context in which they are used. The objective of this study was to compare 2 expert-based methods, heuristic evaluation (HE) and cognitive walkthrough (CW), for evaluating usability of health care information systems. Materials and methods: Five evaluators independently evaluated a medical office management system using HE and CW. We compared the 2 methods in terms of the number of identified usability problems, their severity, and the coverage of each method. Results: In total, 156 problems were identified using the 2 methods. HE identified a significantly higher number of problems related to the “satisfaction” attribute (P = .002). The number of problems identified using CW concerning the “learnability” attribute was significantly higher than those identified using HE (P = .005). There was no significant difference between the number of problems identified by HE, based on different usability attributes (P = .232). Results of CW showed a significant difference between the number of problems related to usability attributes (P &lt; .0001). The average severity of problems identified using CW was significantly higher than that of HE (P &lt; .0001). Conclusion: This study showed that HE and CW do not differ significantly in terms of the number of usability problems identified, but they differ based on the severity of problems and the coverage of some usability attributes. The results suggest that CW would be the preferred method for evaluating systems intended for novice users and HE for users who have experience with similar systems. However, more studies are needed to support this finding.

scholarly journals EVALUASI DESAIN ANTARMUKA APLIKASI BPJSTKU MENGGUNAKAN HEURISTIC EVALUATION

BUANA ILMU ◽  
2021 ◽  
Vol 6 (1) ◽  
pp. 82-97
Author(s):  
Rini Malfiany ◽  
Donny Apdian ◽  
Shofa Shofiah Hilabi
Keyword(s):  
Old Age ◽  
User Satisfaction ◽  
Interface Design ◽  
Mobile Application ◽  
Quantitative Methods ◽  
Evaluation Method ◽  
Heuristic Evaluation ◽  
Average Value ◽  
Old Age Security ◽  
Usability Problems

BPJS Ketenagakerjaan adalah program jaminan sosial yang memberikan fasilitas layanan kepesertaan bagi tenaga kerja ataupun perusahaan (pemberi kerja). Salah satu program BPJS Ketenagakerjaan yang wajib diikuti ialah program Jaminan Hari Tua (JHT). Untuk mengetahui berapa saldo yang kita dapatkan setiap tahunnya, kita bisa mengetahui nya melalui sebuah aplikasi mobile untuk jaminan Hari Tua (JHT) yaitu aplikasi BPJSTKU. Aplikasi BPJSTKu merupakan aplikasi mobile untuk memberikan pelayanan bagi peserta yang dikhususkan pada tenaga kerja untuk mendapatkan informasi mengenai Saldo Jaminan Hari Tua (JHT) beserta rincian dari Saldo Jaminan Hari Tua (JHT) tahunan. Selain itu aplikasi tersebut menyediakan informasi mengenai data perserta BPJSTKU, simulasi Saldo Jaminan Hari Tua (JHT) juga menyediakan Formulir pengajuan Klaim pengambilan JHT secara online. Tujuan dibuatnya penelitian ini untuk mengetahui kepuasan atau kekecewaan dari pengguna aplikasi BPJSTKU ini, karena kepuasan pengguna itu merupakan salah satu faktor untuk menentukan tingkat usability sebuah aplikasi. Untuk menemukan masalah usability pada desain aplikasi BPJSTKU, penelitian ini dibuat dengan menggunakan metode evaluasi Heuristic. Jenis penelitian menggunakan metode kuantitatif karena mengukur kepuasan pengguna dengan Tingkat severity ratings pada masalah usability yang ditentukan dengan skala 0 sampai 4. Lokasi penelitian dilakukan di Karawang dengan sampel staf STMIK Rosma yang menggunakan Aplikasi BPJSTKU. Dari hasil pengukuran evaluasi heuristic menunjukkan kategori minor usability problem, perbaikan masalah ini diberikan prioritas yang rendah dengan nilai rata-rata 1.88 atau skala 2. Kata Kunci : Evaluasi Heuristic, Desain Antarmuka, BPJSTKU BPJS Employment is a social security program that provides membership service facilities for workers or companies (employers). One of the BPJS Employment programs that must be followed is the Old Age Security (JHT) program. To find out how much balance we get each year, we can find out through a mobile application for Old Age Security (JHT), namely the BPJSTKU application. The BPJSTKu application is a mobile application to provide services for participants devoted to the workforce to obtain information about the Old Age Security Balance (JHT) along with details of the annual Old Age Security Balance (JHT). In addition, the application provides information on BPJSTKU participant data, Simulation of Old Age Security Balance (JHT) also provides an online claim form for taking JHT. The purpose of this research is to find out the satisfaction or disappointment of the users of this BPJSTKU application, because user satisfaction is one of the factors to determine the usability level of an application. To find usability problems in the BPJSTKU application design, this study was made using the Heuristic evaluation method. This type of research uses quantitative methods because it measures user satisfaction with severity ratings on usability problems which are determined on a scale of 0 to 4. The research location was conducted in Karawang with a sample of STMIK Rosma staff who used the BPJSTKU application. From the results of the heuristic evaluation measurement, it shows the category of minor usability problems, repairing this problem is given a low priority with an average value of 1.88 or a scale of 2. Keywords: Heuristic Evaluation, Interface Design, BPJSTKU

scholarly journals ANALISIS USER INTERFACE TERHADAP WEBSITE AKTA ONLINE BANYUWANGI MENGGUNAKAN METODE HEURISTIC EVALUATION

2020 ◽  
Vol 4 (3) ◽  
pp. 103
Author(s):  
Siti Vika Ngainul Fitri ◽  
Oktalia Juwita ◽  
Tio Dharmawan
Keyword(s):  
Information Technology ◽  
User Interface ◽  
Interface Design ◽  
User Interface Design ◽  
Evaluation Methods ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Interface Formation ◽  
Usability Evaluation Methods ◽  
Interface Designs

Banyuwangi Regency has a new innovation called " Lahir Procot Pulang Bawa Akta " which in this innovation is realized in the form of an Online Deed Website. Every information technology has an interface that can be a link between the user and the technology itself. Interface formation is influenced by needs, and information technology has different interface designs according to the needs of its users. The User Interface has the aim of making it easier for users to operate information technology that can make users feel comfortable using the application or technology. Heuristic Evaluation is one of the Usability evaluation methods that can be used to determine the extent to which a system is used by users to achieve certain goals with effectiveness, efficiency and satisfaction. This research is a research that is focused on the use of Heuristic Evaluation based on user interface design aspects of application usability through observation, interviews and questionnaires to users.

scholarly journals A fuzzy method for the usability evaluation of nuclear medical equipment

2021 ◽  
Vol 9 (2A) ◽  
Author(s):  
Cláudio Henrique dos Santos Grecco ◽  
Isaac José Antonio Luquetti Dos Santos ◽  
Marcos Santana Farias ◽  
Larissa Pereira De Farias ◽  
Alfredo Marques Vianna Filho
Keyword(s):  
Interface Design ◽  
Medical Equipment ◽  
Evaluation Method ◽  
Usability Evaluation ◽  
International Standards ◽  
Product Development Process ◽  
Usability Problems ◽  
Practical Evaluation ◽  
Wide Range ◽  
High Degree

To avoid errors when handling nuclear medical equipment, it is important to develop products with a high degree of usability. This can be achieved by performing usability evaluations in the product development process to detect and mitigate potential usability problems. Usability evaluation focuses on how well users can learn and use a product to achieve their goals. To gather information about usability, practitioners use a variety of methods that gather feedback from users about an existing interface or plans related to a new interface. A wide range of usability evaluation methods have been proposed, but few methods focus on developing an objective and practical evaluation method for usability. Moreover, the usability evaluations are based on human judgments and most methods cannot fully solve the subjectivity of these evaluations. In order to remedy this deficiency, the purpose of this work is to adopt a Fuzzy Set Theory (FST) approach to establish a method for the usability evaluation of nuclear medical equipment based on usability heuristics for user interface design and international standards for ergonomics of human-system interaction. To exemplify the method we performed a usability evaluating of the Digital Spectrometer ESP 13004 by testing it with representative users. The results showed that the method is a proactive tool to provide a basis for checking usability of medical device interfaces.

Sign in / Sign up

Export Citation Format

Share Document