Which is a Better Method of Web Evaluation? a Comparison of User Testing and Heuristic Evaluation

2002 ◽  
Vol 46 (14) ◽  
pp. 1256-1260 ◽  
Author(s):  
Wei-Siong Tan ◽  
R. R. Bishu
Keyword(s):  
Usability Evaluation ◽  
Security And Privacy ◽  
Development Phase ◽  
Heuristic Evaluation ◽  
User Testing ◽  
Usability Problems ◽  
Efficiency And Effectiveness ◽  
Evaluation Techniques ◽  
Heuristic Analysis

Besides recognizing the importance of incorporating usability evaluation techniques in the design and development phase of any user interface (UI), it is also very important that designers recognize the benefits and limitations of the different usability inspection methods. This is because the quality of the usability evaluation is dependent on the method used. Two of the more popular usability evaluation techniques are user testing and heuristic analysis. The main objective of this study was to compare the efficiency and effectiveness between user testing and heuristic analysis in evaluating four different commercial websites. Comparing the proportion of usability problems and the type of problems addressed by these two methods both in the early and later stage of the design process does this. The results showed that both user testing and heuristic analysis addressed very different usability problems and with the exception compatibility and security and privacy problems, where heuristic analysis outperforms user testing, both methods are equally efficient and effective in addressing different categories of usability problems.

scholarly journals Perancangan Usability Website Interface Sistem Informasi Kerusakan Laboratorium Universitas AMIKOM Yogyakarta

Respati ◽  
2017 ◽  
Vol 12 (3) ◽  
Author(s):  
Mulia Sulistiyono
Keyword(s):  
Usability Testing ◽  
Ease Of Use ◽  
Mobile Technologies ◽  
Heuristic Evaluation ◽  
User Testing ◽  
First Semester ◽  
Software Applications ◽  
Usability Problems ◽  
Usability Analysis

INTISASIUsability sebagai ukuran kualitas pengalaman pengguna seringkali dikatakan sebagai suatu nilai penerimaan (acceptance) seseorang terhadap suatu produk ketika berinteraksi dengan sebuah antarmuka. Termasuk situs web, aplikasi perangkat lunak, teknologi bergerak, maupun peralatan-peralatan lain yang dioperasikan oleh pengguna. Kata usability mengacu pada sebuah metode yang digunakan untuk meningkatkan kemudahan penggunaan selama proses desain. Sistem Informasi Pelaporan Kerusakan Laboratorium (SIPKL) berfungsi memberikan segala bentuk informasi kerusakan yang terjadi didalam laboratorium. SIPKL merupakan sistem yang baru digunakan mulai semester genap tahun 2014, banyak sekali tanggapan dari para penggunanya yang terdiri dari : asisten praktikum, dosen praktikum, student staff/teknisi lab, serta laboran sehingga perlu ada evaluasi secara detail untuk mengetahui seberapa mudah website SIPKL digunakan salah satunya dengan analisa usability. Banyak metode yang dapat digunakan untuk mengidentifikasi masalah usability terutama dari sisi interface web. Dalam penelitian ini untuk mengukur aspek usability pada SIPKL dengan metode User Testing by Expert dan Heuristic Evaluation. Semua Atribut yang diteliti dengan evaluasi heuristic menunjukkan bahwa kendali dan kebebasan pengguna (Terdapat tombol bantu saat error, nilai : 1,70)  Fitur Bantuan dan Dokumentasi (terdapat menu help, nilai : 1,50), serta  Fleksibilitas dan Efisiensi (tata letak menu yang familiar, nilai : 2,10) memiliki masalah usability dengan tingkat kepuasan yang rendah dibanding atribut yang lain. sedangkan hasil user testing by expert memberikan rekomendasi pengembangan fungsional konten, stimulus bagi pengguna agar lebih mudah menggunakan website SIPKL..Kata kunci: Usability, Website Interface, Sistem Informasi, Usability Testing, Evaluasi Heuristik ABSTRACTUsability as a measure of the quality of the user experience is often said to be a person's acceptance of a product when interacting with an interface. Includes websites, software applications, mobile technologies, and other equipment operated by the user. The usability word refers to a method used to improve ease of use during the design process. . Sistem Informasi Pelaporan Kerusakan Laboratorium (SIPKL) serves to provide all forms of damage information that occurs in the laboratory. SIPKL is a newly used system from the first semester of 2014, a lot of responses from its users consisting of: lab assistant, practicum lecturer, student staff / lab technician, and laboran so there needs to be detailed evaluation to find out how easy SIPKL website used wrong only with usability analysis. Many methods can be used to identify usability problems especially from the web interface side. In this study to measure aspects of usability on SIPKL with User Testing by Expert method and Heuristic Evaluation. All Attributes studied with a heuristic evaluation show that control and freedom of the user (There are auxiliary buttons when error, value: 1.70) Help and Documentation Feature (there is a help menu, value: 1.50), and Flexibility and Efficiency (menu layout which is familiar, value: 2.10) has usability problems with a lower satisfaction level than other attributes., whereas the result of user testing by expert gives recommendations for the development of functional content, the stimulus for the user to make it easier to use the SIPKL website.Keywords :Usability, Website Interface, Information System, Usability Testing, Heuristik Evaluation 

scholarly journals Key Design Characteristics for Developing Usable E-Commerce Websites in the Arab World

10.28945/3569 ◽  
2016 ◽  
Vol 19 ◽  
pp. 253-275 ◽  
Author(s):  
Layla Hasan
Keyword(s):  
Developing Countries ◽  
Evaluation Method ◽  
Arab World ◽  
Usability Evaluation ◽  
The Internet ◽  
Heuristic Evaluation ◽  
Priority Level ◽  
Good Design ◽  
Usability Problems ◽  
Design Characteristics

This research aims to suggest key design characteristics that are necessary for developing usable e-commerce websites in the Arab world. A comprehensive usability evaluation of four leading Arab e-commerce websites was conducted using the heuristic evaluation method. The results identified major and minor usability problems and major and minor good design characteristics on the selected websites. Based on the results, 51 key design characteristics were suggested. The recommended key design characteristics comprised two levels according to their priority: level one which includes mandatory key design characteristics and level two which includes supplementary design characteristics. The key design characteristics in each level were categorized under specific pages and areas that can be found on any e-commerce website. Such categorizations could direct website evaluators and designers to important pages and areas that should be considered to improve the overall usability of e-commerce websites. The results of this research are particularly important to developing countries which are still facing challenges that may affect the design and accessibility of usable and useful websites. These relate to low speed of accessing the Internet and a lack of website designers who have experience in customers’ needs and websites’ usable design characteristics.

scholarly journals COMPARISON OF HEURISTIC EVALUATION AND COGNITIVE WALKTHROUGH METHODS IN DOING USABILITY EVALUATION OF MOBILE-BASED DEL EGOV CENTRE HOSPITAL INFORMATION SYSTEM

SEMINASTIKA ◽  
2021 ◽  
Vol 3 (1) ◽  
pp. 99-106
Author(s):  
Gracella Tambunan ◽  
Lit Malem Ginting
Keyword(s):  
Mobile Application ◽  
Evaluation Method ◽  
Likert Scale ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Usability Problems ◽  
Interactive Product ◽  
Cognitive Walkthrough ◽  
Centre Hospital ◽  
Usability Evaluation Methods

Usability is a factor that indicates the success of an interactive product or system, such as a mobile application. The increasing use of smartphones demands a more accurate and effective usability evaluation method to find usability problems, so that they can be used for product improvement in the development process. This study compares the Cognitive Walkthrough method with Heuristic Evaluation in evaluating the usability of the SIRS Del eGov Center mobile application. Evaluation with these two methods will be carried out by three evaluators who act as experts. Finding problems and recommending improvements from each method will produce an improvement prototype made in the form of a high-fidelity prototype. Each prototype will be tested against ten participants using the Usability Testing method, which will generate scores through the SUS table. From the test scores, the percentage of Likert scale and the success rate of each prototype will be found. The results show that between the two usability evaluation methods, the Heuristic Evaluation method is the more effective method, finds more usability problems, and has a higher Likert scale percentage, which is 66.5%, while Cognitive Walkthrough is 64.75%.

scholarly journals Usability Evaluation of An Electronic Medication Administration Record (eMAR) Application

2011 ◽  
Vol 02 (02) ◽  
pp. 202-224 ◽  
Author(s):  
S. Iribarren ◽  
S. Kapsandoy ◽  
S. Perri ◽  
N. Staggers ◽  
J. Guo
Keyword(s):  
Patient Safety ◽  
Medication Administration ◽  
Usability Evaluation ◽  
Staff Nurses ◽  
Call To Action ◽  
Clinical Nurses ◽  
Usability Problems ◽  
Evaluation Techniques ◽  
Promote Patient Safety ◽  
Electronic Medication Administration

SummaryBackground: Electronic medication administration records (eMARs) have been widely used in recent years. However, formal usability evaluations are not yet available for these vendor applications, especially from the perspective of nurses, the largest group of eMAR users.Objective: To conduct a formal usability evaluation of an implemented eMAR.Methods: Four evaluators examined a commercial vendor eMAR using heuristic evaluation techniques. The evaluators defined seven tasks typical of eMAR use and independently evaluated the application. Consensus techniques were used to obtain 100% agreement of identified usability problems and severity ratings. Findings were reviewed with 5 clinical staff nurses and the Director of Clinical Informatics who verified findings with a small group of clinical nurses.Results: Evaluators found 60 usability problems categorized into 233 heuristic violations. Match, Error, and Visibility heuristics were the most frequently violated. Administer Medication and Order and Modify Medications tasks had the highest number of heuristic violations and usability problems rated as major or catastrophic.Conclusion: The high number of usability problems could impact the effectiveness, efficiency and satisfaction of nurses’ medication administration activities and may include concerns about patient safety. Usability is a joint responsibility between sites and vendors. We offer a call to action for usability evaluations at all sites and eMAR application redesign as necessary to improve the user experience and promote patient safety.

Usability Evaluation of E-Learning Systems

2011 ◽  
pp. 3897-3903 ◽  
Author(s):  
Shirish C. Srivastava ◽  
Shalini Chandra ◽  
Hwee Ming Lam
Keyword(s):  
Interface Design ◽  
Usability Evaluation ◽  
Learning System ◽  
Heuristic Evaluation ◽  
Usability Problems ◽  
E Learning ◽  
Small Set ◽  
User Groups ◽  
Definition Of ◽  
Usability Evaluation Methods

Usability evaluation which refers to a series of activities that are designed to measure the effectiveness of a system as a whole, is an important step for determining the acceptance of system by the users. Usability evaluation is becoming important since both user groups, as well as tasks, are increasing in size and diversity. Users are increasingly becoming more informed and, consequently, have higher expectations from the systems. Moreover “system interface” has become a commodity and, hence, user acceptance plays a major role in the success of the system. Currently, there are various usability evaluation methods in vogue, like cognitive walkthrough, think aloud, claims analysis, heuristic evaluation, and so forth. However, for this study we have chosen heuristic evaluation because it is relatively inexpensive, logistically uncomplicated, and is often used as a discount usability-engineering tool (Nielsen, 1994). Heuristic evaluation is a method for finding usability problems in a user interface design by having a small set of evaluators examine an interface and judge its compliance with recognized usability principles. The rest of the chapter is organized as follows: we first look at the definition of e-learning, followed by concepts of usability, LCD, and heuristics. Subsequently, we introduce a methodology for heuristic usability evaluation (Reeves, Benson, Elliot, Grant, Holschuh, Kim, Kim, Lauber, & Loh, 2002), and then use these heuristics for evaluating an existing e-learning system, GETn2. We offer our recommendations for the system and end with a discussion on the contributions of our chapter.

Heuristic Evaluation on M-Learning Applications

2017 ◽  
pp. 38-55
Author(s):  
Christofer Ramos ◽  
Flávio Anthero Nunes Vianna dos Santos ◽  
Monique Vandresen
Keyword(s):  
Comparative Study ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Considerable Evidence ◽  
Usability Problems ◽  
Systematic Methodology ◽  
E Learning ◽  
The Comparative Study ◽  
Learning Issues ◽  
Usability Evaluation Methods

Heuristic evaluation stands out among the usability evaluation methods regarding its benefits related to time and costs. Nevertheless, generic heuristic sets require improvements when it comes to specific interfaces as seen on m-learning applications that have acquired considerable evidence within the current technologic context. Regarding the lack of studies aimed at interfaces of this sort, the authors propose, through a systematic methodology, the comparative study between a heuristic set specific to the assessment on e-learning interfaces and other, on mobile. The identified usability problems were matched with the aspects of coverage, distribution, redundancy, context and severity, in a way that it was possible to understand the efficiency of each set in covering m-learning issues. Among the findings, e-learning's heuristic set could detect a larger number of usability problems not found by mobile's.

A novel approach for usability evaluation of mobile applications

2021 ◽  
Vol 65 (1) ◽  
pp. 437-441
Author(s):  
Junho Park ◽  
Maryam Zahabi
Keyword(s):  
Graphical User Interface ◽  
Mobile Applications ◽  
Human Subject ◽  
Mobile Apps ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
User Testing ◽  
Novel Approach ◽  
Design Cycle ◽  
Level Model

Mobile applications (apps) have been massively developed and released in recent years. In this competitive market, the users’ adoption of the app or satisfaction greatly depends on its usability. Therefore, tech companies and app developers are looking for quick and easy approaches to evaluate the usability of their products early in the design cycle. The objective of this study was to introduce a novel approach combining the capabilities of the gesture-level model (GLM) and heuristic evaluation to assess the usability of mobile apps in early stages of the design cycle and without the need for costly and time-consuming user testing. The approach can be run on a graphical user interface and includes uncertainties regarding user skill level and distraction. Preliminary results suggested that this approach can be used for usability evaluation of mobile apps; however, its validity and generalizability to other domains need to be validated in the future with human-subject experiments.

scholarly journals Development and Usability Evaluation of a Facebook-Based Intervention Program for Childhood Cancer Patients: Mixed Methods Study

10.2196/18779 ◽  
2020 ◽  
Vol 22 (7) ◽  
pp. e18779
Author(s):  
Bu Kyung Park ◽  
Ji Yoon Kim ◽  
Valerie E Rogers
Keyword(s):  
Focus Group ◽  
Childhood Cancer ◽  
Intervention Program ◽  
Management Strategies ◽  
Usability Evaluation ◽  
Self Management ◽  
Heuristic Evaluation ◽  
Childhood Cancers ◽  
User Testing ◽  
Final Version

Background Childhood cancers previously considered to be incurable now have 5-year survival rates up to 84%. Nevertheless, these patients remain at risk of morbidity and mortality from therapy-related complications. Thus, patient education and self-management strategies for promoting a healthy lifestyle are of tantamount importance for improving short- and long-term health outcomes. A Facebook-based “Healthy Teens for Soaam” (a Korean term for childhood cancers) program was developed to help improve knowledge and self-management practices of teens with cancer related to their disease and treatment. Objective The two-fold purpose of this usability study was (1) to describe the process of developing an 8-week Facebook-based intervention program for teens with cancer, and (2) to evaluate its usability to refine the program. Methods Multiple phases and methods were employed to develop and evaluate the usability of the program. Study phases included: (1) needs assessment through focus group interviews and qualitative content analysis, (2) development of module content, (3) expert review and feedback on module content, (4) Facebook-based program development, (5) usability evaluation by heuristic evaluation, (6) usability evaluation by targeted end-user testing, and (7) modification and final version of the program. Usability of the final version was confirmed through feedback loops of these phases. Results Based on 6 focus group discussion sessions, it was determined that teens with cancer were interested in seeing stories of successful childhood cancer cases and self-management after discharge, and preferred multimedia content over text. Therefore, each Facebook module was redesigned to include multimedia materials such as relevant video clips tailored for teens. Usability assessed by heuristic evaluation and user testing revealed several critical usability issues, which were then revised. Potential end users tested the final program and perceived it to be usable and useful for teens with cancer. Conclusions To our knowledge, “Healthy Teens for Soaam” is the first Facebook-based intervention program for teens with cancer. We actively worked with current childhood cancer patients and survivors to develop and improve this program, achieved good usability, and met the expressed needs and preferences of target end users. This 8-week Facebook-based educational program for teens with cancer, developed as the first step of an upcoming intervention study, will be useful for improving knowledge and self-management strategies of teens.

scholarly journals E-Learning Usability Evaluation Menggunakan Fuzzy Logic dan Usulan Alternatif Desain Interaktif Learning Management System (LMS) Chamilo

2019 ◽  
Vol 18 (1) ◽  
pp. 65-70
Author(s):  
Arif Rinaldi Dikananda ◽  
Harry Budi Santoso ◽  
Raditya Danar Dana ◽  
Dadang Sudrajat
Keyword(s):  
Fuzzy Logic ◽  
User Satisfaction ◽  
Fuzzy Inference ◽  
Usability Evaluation ◽  
Natural Languages ◽  
Fuzzy Variables ◽  
Usability Problems ◽  
System Usability ◽  
E Learning

E-Learning as well as learning media in general needs to be evaluated to find out and measure how much effectiveness, efficiency and user satisfaction is for the quality of the overall learning process. One effort that can be done to find out and evaluate the quality of learning is to use usability evaluation. Usability measurements require data derived from questionnaires presented using a Likert scale. The data illustrates the perceptions of users who have uncertainties because they are very subjective so they have the potential to cause misinterpretations. Fuzzy logic can be used to evaluate e-Learning reusability because fuzzy logic has the advantage of resolving a problem that contains uncertainty / ambiguity, which in this case is in accordance with the context of usability problems that are often presented in natural languages that have uncertainties, such as "how effective? "," How efficient? "And" how much user satisfaction. By using the Mamdani model Fuzzy Inference an increase in system usability with a score of 3.06 with a membership level of 0.9961 in the Moderate Usability stack. With the application of fuzzy variables and fuzzy rules, the process of evaluating system usability can be done with natural language that is easier to understand.

scholarly journals Enhancing Usability of E-Learning Platform: A Case Study of Khan Academy

sjesr ◽  
2021 ◽  
Vol 4 (2) ◽  
pp. 40-50
Author(s):  
Maham Sheikh ◽  
Abdul Hafeez Muhammad ◽  
Quadri Noorul hasan Naveed
Keyword(s):  
Online Education ◽  
User Satisfaction ◽  
Heuristic Evaluation ◽  
Learning Platform ◽  
Usability Problems ◽  
E Learning ◽  
Learning Platforms ◽  
Khan Academy ◽  
Evaluation Techniques

Human-Computer Interaction (HCI) is a subject that measures usability and evaluates the interface of a system, software, or product to see its efficiency, effectiveness, and user satisfaction. On the other hand, at the time, the pandemic spread of Coronavirus (Covid-19) having an immense impact on schools, universities, and other educational sectors. In this current unpredictable situation, the importance of online education is visible, therefore, the usability of e-learning platforms truly plays an essential role in our lives. This study is about to evaluate the usability of a very common e-learning portal i.e., Khan Academy by user testing and heuristic evaluation techniques. Effectiveness, efficiency, and satisfaction of Khan Academy were measured quantitatively, while usability issues were identified qualitatively through heuristic evaluation. The study result shows that Khan Academy suffers from certain usability problems which need to be addressed. The study also presents relevant recommendations which are scrutinized and approved by IT specialists and UI/UX designers. The study concluded that suggested amendments are important to consider enhancing the usability of different e-Learning portals generally and Khan Academy particularly.

Sign in / Sign up

Export Citation Format

Share Document