scholarly journals Effect of improving the usability of an e-learning resource: a randomized trial

2014 ◽  
Vol 38 (2) ◽  
pp. 155-160 ◽  
Author(s):  
Mogamat Razeen Davids ◽  
Usuf M. E. Chikte ◽  
Mitchell L. Halperin
Keyword(s):  
Randomized Trial ◽  
User Satisfaction ◽  
Rating Scale ◽  
Usability Evaluation ◽  
Learning Material ◽  
Transfer Of Knowledge ◽  
Learning Resource ◽  
Usability Problems ◽  
System Usability Scale ◽  
E Learning

Optimizing the usability of e-learning materials is necessary to reduce extraneous cognitive load and maximize their potential educational impact. However, this is often neglected, especially when time and other resources are limited. We conducted a randomized trial to investigate whether a usability evaluation of our multimedia e-learning resource, followed by fixing of all problems identified, would translate into improvements in usability parameters and learning by medical residents. Two iterations of our e-learning resource [ version 1 (V1) and version 2 (V2)] were compared. V1 was the first fully functional version and V2 was the revised version after all identified usability problems were addressed. Residents in internal medicine and anesthesiology were randomly assigned to one of the versions. Usability was evaluated by having participants complete a user satisfaction questionnaire and by recording and analyzing their interactions with the application. The effect on learning was assessed by questions designed to test the retention and transfer of knowledge. Participants reported high levels of satisfaction with both versions, with good ratings on the System Usability Scale and adjective rating scale. In contrast, analysis of video recordings revealed significant differences in the occurrence of serious usability problems between the two versions, in particular in the interactive HandsOn case with its treatment simulation, where there was a median of five serious problem instances (range: 0–50) recorded per participant for V1 and zero instances (range: 0–1) for V2 ( P < 0.001). There were no differences in tests of retention or transfer of knowledge between the two versions. In conclusion, usability evaluation followed by a redesign of our e-learning resource resulted in significant improvements in usability. This is likely to translate into improved motivation and willingness to engage with the learning material. In this population of relatively high-knowledge participants, learning scores were similar across the two versions.

scholarly journals USABILITY TESTING PADA OFFICE ADD-INS DALAM MENGINTEGRASIKAN SISTEM REKONSTRUKSI MATERI DAN SISTEM MANAJEMEN PEMBELAJARAN

2018 ◽  
Vol 4 (2) ◽  
Author(s):  
A.A. Gede Yudhi Paramartha ◽  
I Gede Partha Sindu ◽  
Agus Aan Jiwa Permana
Keyword(s):  
User Satisfaction ◽  
Management System ◽  
Usability Testing ◽  
Learning Material ◽  
Electronic Learning ◽  
System Usability Scale ◽  
System Usability ◽  
E Learning ◽  
Learning Management ◽  
Microsoft Office

ABSTRACT<br />The development of electronic learning materials requires various applications that are used to support electronic learning. The lecturer makes the material by utilizing a document processing application to create learning material, using a search engine to obtain teaching materials, and a Learning Management System to publish learning material. This causes the lecturer to have to move to one application to another application. In previous research, to overcome these problems, an add-ins was developed in Microsoft Office PowerPoint that integrates Learning Management Systems as a media for publishing learning materials, and Material Reconstruction System as a source of reusable teaching materials by utilizing the search engines provided. These add-ins function as an intermediary between the two systems with Microsoft PowerPoint, so users only need to use one application to develop learning material and publish it to e-learning. In this study, the add-ins that have been developed are tested using the usability testing method. Usability testing is done to determine the level of user satisfaction in using the add-ins and obtain feedback for system improvements. The method used is Retrospective Think Aloud method to obtain feedback from users, and the System Usability Scale to determine the level of user satisfaction in using the add-ins. The results showed that users were satisfied with the add-ins that had been developed. in addition, feedback from users was also successfully obtained and recommendations for add-ins improvements were made based on these recommendations.<br />Keywords : Learning Management System, Office Add-ins, Usability Testing<br />ABSTRAK<br />Pengembangan materi pembelajaran elektronik membutuhkan berbagai aplikasi yang digunakan untuk mendukung pembelajaran elektronik. Dosen membuat materi dengan memanfaatkan aplikasi pengolah dokumen untuk membuat materi, menggunakan mesin pencari untuk memperoleh bahan ajar, dan Sistem Manajemen Pembelajaran untuk mempublikasikan materi pembelajaran. Hal ini menyebabkan dosen harus berpindah ke satu aplikasi ke aplikasi yang lain. Pada penelitian sebelumnya, untuk mengatasi permasalahan tersebut, dikembangkan sebuah add-ins di Microsoft Office PowerPoint yang mengintegrasikan Sistem Manajemen Pembelajaran sebagai media publikasi materi pembelajaran, dan Sistem Rekonstruksi Materi sebagai sumber bahan ajar dengan memanfaatkan mesin pencari yang disediakan. Add-ins tersebut berfungsi sebagai perantara antar kedua sistem tersebut dengan Microsoft PowerPoint, sehingga pengguna hanya perlu menggunakan satu aplikasi untuk mengembangkan materi pembelajaran hingga mempublikasikan materi tersebut ke e-learning. Pada penelitian ini, add-ins yang telah dikembangkan diuji dengan menggunakan metode usability testing. Usability testing dilakukan untuk mengetahui tingkat kepuasan pengguna dalam menggunakan add-ins dan memperoleh umpan balik untuk perbaikan sistem. Metode yang digunakan adalah metode Retrospective Think Aloud (RTA) untuk memperoleh dari pengguna, dan System Usability Scale (SUS) untuk mengetahui tingkat kepuasan pengguna dalam menggunakan add-ins. Hasil penelitian menunjukkan bahwa pengguna merasa puas dengan add-ins yang telah dikembangkan. selain itu, umpan balik dari pengguna juga berhasil diperoleh dan dibuatkan rekomendasi perbaikan berdasarkan rekomendasi tersebut.<br />Kata kunci: Sistem Manajemen Pembelajaran, Office Add-ins, Usability Testing

scholarly journals E-Learning Usability Evaluation Menggunakan Fuzzy Logic dan Usulan Alternatif Desain Interaktif Learning Management System (LMS) Chamilo

2019 ◽  
Vol 18 (1) ◽  
pp. 65-70
Author(s):  
Arif Rinaldi Dikananda ◽  
Harry Budi Santoso ◽  
Raditya Danar Dana ◽  
Dadang Sudrajat
Keyword(s):  
Fuzzy Logic ◽  
User Satisfaction ◽  
Fuzzy Inference ◽  
Usability Evaluation ◽  
Natural Languages ◽  
Fuzzy Variables ◽  
Usability Problems ◽  
System Usability ◽  
E Learning

E-Learning as well as learning media in general needs to be evaluated to find out and measure how much effectiveness, efficiency and user satisfaction is for the quality of the overall learning process. One effort that can be done to find out and evaluate the quality of learning is to use usability evaluation. Usability measurements require data derived from questionnaires presented using a Likert scale. The data illustrates the perceptions of users who have uncertainties because they are very subjective so they have the potential to cause misinterpretations. Fuzzy logic can be used to evaluate e-Learning reusability because fuzzy logic has the advantage of resolving a problem that contains uncertainty / ambiguity, which in this case is in accordance with the context of usability problems that are often presented in natural languages that have uncertainties, such as "how effective? "," How efficient? "And" how much user satisfaction. By using the Mamdani model Fuzzy Inference an increase in system usability with a score of 3.06 with a membership level of 0.9961 in the Moderate Usability stack. With the application of fuzzy variables and fuzzy rules, the process of evaluating system usability can be done with natural language that is easier to understand.

scholarly journals EVALUATING THE QUALITY OF E-LEARNING MATERIAL

2017 ◽  
Vol 2 ◽  
pp. 74
Author(s):  
Ilze Kazaine
Keyword(s):  
Quality Evaluation ◽  
Rating Scale ◽  
Quality Criteria ◽  
Evaluation Process ◽  
Learning Material ◽  
Material Quality ◽  
Electronic Learning ◽  
E Learning ◽  
Study Process

An increasing number of educational institutions in the study process uses one of the e-learning systems. Consequently, more and more students are offered learning materials in electronic format. E-materials in distance learning and e-learning is one of the most important elements, therefore much attention and enough time should be paid for their development. There are a number of studies on e-learning quality, where criteria of quality are discussed in the context of chosen e-learning environment and the process of implementation. This article examines only the quality of e-material. The aim was to find a way to reduce the effort and time of electronic learning material quality evaluation. The study used content analysis by summarizing the most important factors influencing the quality for teaching materials. Based on the quality criteria mentioned in the literature and personal experience, a criterion, which affects quality of e-learning material, were summarized and grouped. The criteria were grouped into four groups resulting from didactic, media, usability and formal quality. Quality evaluation is performed by using one of the methods used in software engineering - checklist. Based on the identified quality criteria a checklist was established. In order to facilitate the evaluation process a web-based tool is offered. The tool includes a defined checklist with assessment rating scale and three levels of impact. Evaluation of material quality is shown in the terms of percentage. After testing the tool, it could be used for course developers, program managers or other persons involved in evaluation process of e-learning resources.

Usability Evaluation of E-Learning Systems

2011 ◽  
pp. 3897-3903 ◽  
Author(s):  
Shirish C. Srivastava ◽  
Shalini Chandra ◽  
Hwee Ming Lam
Keyword(s):  
Interface Design ◽  
Usability Evaluation ◽  
Learning System ◽  
Heuristic Evaluation ◽  
Usability Problems ◽  
E Learning ◽  
Small Set ◽  
User Groups ◽  
Definition Of ◽  
Usability Evaluation Methods

Usability evaluation which refers to a series of activities that are designed to measure the effectiveness of a system as a whole, is an important step for determining the acceptance of system by the users. Usability evaluation is becoming important since both user groups, as well as tasks, are increasing in size and diversity. Users are increasingly becoming more informed and, consequently, have higher expectations from the systems. Moreover “system interface” has become a commodity and, hence, user acceptance plays a major role in the success of the system. Currently, there are various usability evaluation methods in vogue, like cognitive walkthrough, think aloud, claims analysis, heuristic evaluation, and so forth. However, for this study we have chosen heuristic evaluation because it is relatively inexpensive, logistically uncomplicated, and is often used as a discount usability-engineering tool (Nielsen, 1994). Heuristic evaluation is a method for finding usability problems in a user interface design by having a small set of evaluators examine an interface and judge its compliance with recognized usability principles. The rest of the chapter is organized as follows: we first look at the definition of e-learning, followed by concepts of usability, LCD, and heuristics. Subsequently, we introduce a methodology for heuristic usability evaluation (Reeves, Benson, Elliot, Grant, Holschuh, Kim, Kim, Lauber, & Loh, 2002), and then use these heuristics for evaluating an existing e-learning system, GETn2. We offer our recommendations for the system and end with a discussion on the contributions of our chapter.

Heuristic Evaluation on M-Learning Applications

2017 ◽  
pp. 38-55
Author(s):  
Christofer Ramos ◽  
Flávio Anthero Nunes Vianna dos Santos ◽  
Monique Vandresen
Keyword(s):  
Comparative Study ◽  
Usability Evaluation ◽  
Heuristic Evaluation ◽  
Considerable Evidence ◽  
Usability Problems ◽  
Systematic Methodology ◽  
E Learning ◽  
The Comparative Study ◽  
Learning Issues ◽  
Usability Evaluation Methods

Heuristic evaluation stands out among the usability evaluation methods regarding its benefits related to time and costs. Nevertheless, generic heuristic sets require improvements when it comes to specific interfaces as seen on m-learning applications that have acquired considerable evidence within the current technologic context. Regarding the lack of studies aimed at interfaces of this sort, the authors propose, through a systematic methodology, the comparative study between a heuristic set specific to the assessment on e-learning interfaces and other, on mobile. The identified usability problems were matched with the aspects of coverage, distribution, redundancy, context and severity, in a way that it was possible to understand the efficiency of each set in covering m-learning issues. Among the findings, e-learning's heuristic set could detect a larger number of usability problems not found by mobile's.

University students’ heuristic usability ınspection of the national library of Turkey website

2018 ◽  
Vol 70 (1) ◽  
pp. 66-77 ◽  
Author(s):  
Yavuz Inal
Keyword(s):  
University Students ◽  
Digital Libraries ◽  
Usability Evaluation ◽  
Assessment Process ◽  
National Library ◽  
Content Type ◽  
Usability Problems ◽  
System Usability Scale ◽  
Wide Range ◽  
Library Websites

Purpose The purpose of this paper is to evaluate the usability of the National Library of Turkey website developed for all citizens in the country to retrieve information available in a wide range of areas. Design/methodology/approach The evaluation performed by 57 undergraduate university students involved the identification and analysis of usability problems in the selected website. The assessment process was mainly based on Nielsen’s Heuristics. The data obtained in the evaluation process were enriched using the System Usability Scale (SUS) and the Net Promoter Score (NPS) to conduct a more detailed analysis. Findings The participants identified many usability problems concerning the evaluated website and considered it as having low usability characteristics. The most violated heuristic item was found to be “consistency and standards” whereas the least violated heuristic item was “match between system and the real world.” The total number of the usability problems in heuristic evaluation correlated significantly and negatively with the SUS and the NPS results indicating that the more participants identified usability problems, the less they considered the website as usable and recommendable. Research limitations/implications The participants were not usability experts, however they were selected from among the students who received the Human Computer Interaction course to ensure that they had sufficient information and experience concerning the evaluation of a website with heuristics. Besides, the study was limited to a small number of university students. The implication is that results of this study have potential to guide libraries, which plan to adapt themselves to the digital world by delivering web services, by addressing critical points influencing users’ points of views toward library websites and their usability evaluation result. Practical implications Depending on the changes in user habits regarding the retrieval of information, libraries have made considerable investments in web technologies to meet their users’ demands, and recently, digital libraries have begun to take over physical libraries. Users always need to perform tasks efficiently, effectively and satisfactorily when using websites. As one of the most crucial sources of digital materials, library websites are expected to have usable characteristics that satisfactorily meet user requirements. Therefore, the practical implication is that the findings of the study will guide designers, developers and practitioners in the development of library websites. Originality/value In the context of usability evaluation of digital libraries, this is the first study to analyze effects of usability problems identified by users during the heuristic inspection on their overall evaluation score and willingness to recommend the website to their friends or colleagues.

scholarly journals Development and evaluation of a multimedia e-learning resource for electrolyte and acid-base disorders

2011 ◽  
Vol 35 (3) ◽  
pp. 295-306 ◽  
Author(s):  
Mogamat Razeen Davids ◽  
Usuf M. E. Chikte ◽  
Mitchell L. Halperin
Keyword(s):  
Brain Size ◽  
Text Messages ◽  
Teaching Approach ◽  
Plasma Sodium ◽  
Acid Base ◽  
Learning Resource ◽  
System Usability Scale ◽  
System Usability ◽  
E Learning ◽  
The Impact

This article reports on the development and evaluation of a Web-based application that provides instruction and hands-on practice in managing electrolyte and acid-base disorders. Our teaching approach, which focuses on concepts rather than details, encourages quantitative analysis and a logical problem-solving approach. Identifying any dangers to the patient is a vital first step. Concepts such as an “appropriate response” to a given perturbation and the need for electroneutrality in body fluids are used repeatedly. Our Electrolyte Workshop was developed using Flash and followed an iterative design process. Two case-based tutorials were built in this first phase, with one tutorial including an interactive treatment simulation. Users select from a menu of therapies and see the impact of their choices on the patient. Appropriate text messages are displayed, and changes in body compartment sizes, brain size, and plasma sodium concentrations are illustrated via Flash animation. Challenges encountered included a shortage of skilled Flash developers, budgetary constraints, and challenges in communication between the authors and the developers. The application was evaluated via user testing by residents and specialists in internal medicine. Satisfaction was measured with a questionnaire based on the System Usability Scale. The mean System Usability Scale score was 78.4 ± 13.8, indicating a good level of usability. Participants rated the content as being scientifically sound; they liked the teaching approach and felt that concepts were conveyed clearly. They indicated that the application held their interest, that it increased their understanding of hyponatremia, and that they would recommend this learning resource to others.

scholarly journals Enhancing Usability of E-Learning Platform: A Case Study of Khan Academy

sjesr ◽  
2021 ◽  
Vol 4 (2) ◽  
pp. 40-50
Author(s):  
Maham Sheikh ◽  
Abdul Hafeez Muhammad ◽  
Quadri Noorul hasan Naveed
Keyword(s):  
Online Education ◽  
User Satisfaction ◽  
Heuristic Evaluation ◽  
Learning Platform ◽  
Usability Problems ◽  
E Learning ◽  
Learning Platforms ◽  
Khan Academy ◽  
Evaluation Techniques

Human-Computer Interaction (HCI) is a subject that measures usability and evaluates the interface of a system, software, or product to see its efficiency, effectiveness, and user satisfaction. On the other hand, at the time, the pandemic spread of Coronavirus (Covid-19) having an immense impact on schools, universities, and other educational sectors. In this current unpredictable situation, the importance of online education is visible, therefore, the usability of e-learning platforms truly plays an essential role in our lives. This study is about to evaluate the usability of a very common e-learning portal i.e., Khan Academy by user testing and heuristic evaluation techniques. Effectiveness, efficiency, and satisfaction of Khan Academy were measured quantitatively, while usability issues were identified qualitatively through heuristic evaluation. The study result shows that Khan Academy suffers from certain usability problems which need to be addressed. The study also presents relevant recommendations which are scrutinized and approved by IT specialists and UI/UX designers. The study concluded that suggested amendments are important to consider enhancing the usability of different e-Learning portals generally and Khan Academy particularly.

Barriers and Enablers which Affect the Implementation of Intelligent Guidance Systems for Patients in Chinese Tertiary Transfer Hospitals: Usability Evaluation (Preprint)

2020 ◽  
Author(s):  
Hengkui Cao ◽  
Qiqing Bi ◽  
Richard Evans ◽  
Lining Shen
Keyword(s):  
Linear Regression ◽  
Multiple Linear Regression ◽  
User Satisfaction ◽  
Linear Regression Analysis ◽  
User Experiences ◽  
Usability Problems ◽  
System Usability Scale ◽  
System Usability ◽  
Hospital Visits ◽  
Guidance Systems

BACKGROUND Hospital information systems have been widely adopted across China, in recent years, significantly changing the way in which hospitals manage their processes and patient visits. Intelligent Guidance Systems for Patients (IGSP), which resemble humanoid characteristics using Artificial Intelligence (AI), assist patients in navigating hospital visits, obtaining medical guidance, consultation, and other medical services. Despite their widespread adoption in tertiary transfer hospitals in China, usability studies on such systems are scarce. To date, there is no practical or standardized measurement of system usability, leading to difficult inspection, maintenance and servicing processes. OBJECTIVE IGSP can improve user experiences pre, during and post hospital visits. This study aims to determine the usability deficiency of IGSP and understand how various factors influence users’ satisfaction during use. METHODS Employing the requirements set out in the ISO9214-11:2018 standard, two inspection methods were used with 3 experts and 360 recruited end-users. First, the Heuristic evaluation method was employed to detect usability problems and to demonstrate any violations of Nielsen’s 10 heuristic principles. Second, the System Usability Scale (SUS) was applied to evaluate participants’ satisfaction towards IGSP. Finally, analysis of variance tests and a multiple linear regression analysis was performed to establish correlations between user satisfaction and various characteristics. RESULTS A total of 78 problems violated the heuristic principles for 169 times. These problems were divided into five categories: voice interaction, in-hospital navigation, medical consultation, interactive interface design, and miscellaneous. The average SUS score for IGSP was 72.8 (SD 14.69), indicating a good usability level. User satisfaction scores differed among hospitals (F=3.513, P=.031), age of user (F=6.010, P=.000), and smartphone usage time (F=4.781, P=.000). The multiple linear regression model reflected participants’ education level (t=-2.41, p= .016) and gender (t=-2.080, p=.038), while medical industry (t=-2.074, p=0.039) had a negative effect on user satisfaction scores. Conversely, daily smartphone usage times (t= 2.470, p=.014) helped to increase user satisfaction scores. CONCLUSIONS This study demonstrates that IGSP can improve user experiences during interaction between patient and hospital. However, system usability deficiencies did exist; some of which must be addressed. Our findings are considered conducive to tertiary transfer hospitals in developing user-friendly IGSP for improving patient experiences. Tertiary hospitals should take full advantage of IGSP to help reduce waiting times and meet the various needs of patients pre, during and post hospital visit. User-centered conception should be integrated into the process of design and modification of IGSP.

Sign in / Sign up

Export Citation Format

Share Document