Impact of Pre-Study Exploration on System Usability Scale and Task Success Rates for Automotive Interfaces

2017 ◽  
Author(s):  
Satheesh Kumar Chandran ◽  
James Forbes ◽  
Carrie Bittick ◽  
Kathleen Allanson ◽  
Santosh Erupaka ◽  
...  
Keyword(s):  
Success Rates ◽  
System Usability Scale ◽  
System Usability ◽  
Task Success ◽  
And Task

scholarly journals Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics

2015 ◽  
Vol 23 (1) ◽  
pp. 5-11 ◽  
Author(s):  
Mattias Georgsson ◽  
Nancy Staggers
Keyword(s):  
Task Performance ◽  
Quantitative Methods ◽  
Task Completion ◽  
Time On Task ◽  
Diabetes Diagnosis ◽  
Success Rates ◽  
User Characteristics ◽  
System Usability Scale ◽  
System Usability ◽  
Task Success

Abstract Objective Mobile health (mHealth) systems are becoming more common for chronic disease management, but usability studies are still needed on patients’ perspectives and mHealth interaction performance. This deficiency is addressed by our quantitative usability study of a mHealth diabetes system evaluating patients’ task performance, satisfaction, and the relationship of these measures to user characteristics. Materials and Methods We used metrics in the International Organization for Standardization (ISO) 9241-11 standard. After standardized training, 10 patients performed representative tasks and were assessed on individual task success, errors, efficiency (time on task), satisfaction (System Usability Scale [SUS]) and user characteristics. Results Tasks of exporting and correcting values proved the most difficult, had the most errors, the lowest task success rates, and consumed the longest times on task. The average SUS satisfaction score was 80.5, indicating good but not excellent system usability. Data trends showed males were more successful in task completion, and younger participants had higher performance scores. Educational level did not influence performance, but a more recent diabetes diagnosis did. Patients with more experience in information technology (IT) also had higher performance rates. Discussion Difficult task performance indicated areas for redesign. Our methods can assist others in identifying areas in need of improvement. Data about user background and IT skills also showed how user characteristics influence performance and can provide future considerations for targeted mHealth designs. Conclusion Using the ISO 9241-11 usability standard, the SUS instrument for satisfaction and measuring user characteristics provided objective measures of patients’ experienced usability. These could serve as an exemplar for standardized, quantitative methods for usability studies on mHealth systems.

scholarly journals User Satisfaction Reward Estimation Across Domains: Domain-independent Dialogue Policy Learning

2021 ◽  
Vol 12 (2) ◽  
pp. 81-114
Author(s):  
Stefan Ultes ◽  
Wolfgang Maier
Keyword(s):  
User Satisfaction ◽  
Objective Measure ◽  
Dialogue Systems ◽  
Success Rates ◽  
Estimation Model ◽  
Spoken Dialogue Systems ◽  
Task Success ◽  
Domain Independent ◽  
Similar Task ◽  
And Task

Learning suitable and well-performing dialogue behaviour in statistical spoken dialogue systems has been in the focus of research for many years. While most work that is based on reinforcement learning employs an objective measure like task success for modelling the reward signal, we propose to use a reward signal based on user satisfaction. We propose a novel estimator and show that it outperforms all previous estimators while learning temporal dependencies implicitly. We show in simulated experiments that a live user satisfaction estimation model may be applied resulting in higher estimated satisfaction whilst achieving similar success rates. Moreover, we show that a satisfaction estimation model trained on one domain may be applied in many other domains that cover a similar task. We verify our findings by employing the model to one of the domains for learning a policy from real users and compare its performance to policies using user satisfaction and task success acquired directly from the users as reward.

The Relationship Between User Mouse-based Performance And Subjective Usability Assessments

2016 ◽  
Vol 60 (1) ◽  
pp. 1174-1178
Author(s):  
Philip Kortum ◽  
Claudia Ziegler Acemyan
Keyword(s):  
Performance Metrics ◽  
User Performance ◽  
Success Rates ◽  
Website Usability ◽  
Objective Metrics ◽  
System Usability Scale ◽  
System Usability ◽  
Effectiveness And Efficiency ◽  
Subjective Usability ◽  
The Relationship

Objective metrics, such as effectiveness and efficiency, are often considered to be the best website usability measurements. User performance metrics that can be collected remotely, such as mouse clicks and the distance the mouse has traveled show particular promise. However, no studies have demonstrated a direct relationship between subjective usability measures and these two objective user performance metrics. In this paper, thirty participants completed five different tasks of varying difficulty on a commercial website. Mouse clicks, the distance the mouse moved, success rates, and System Usability Scale (SUS) scores were collected for each task. Results showed that participants made fewer mouse clicks on tasks at which they were successful than on tasks they failed. Participants moved the mouse over twice as far on failed tasks as compared to successful tasks. The correlations between SUS scores and the two mouse-based measurements were remarkably strong.

Dual-goal facilitation in Wason's 2–4–6 task: What mediates successful rule discovery?

2006 ◽  
Vol 59 (5) ◽  
pp. 873-885 ◽  
Author(s):  
Maggie Gale ◽  
Linden J. Ball
Keyword(s):  
Mixed Valence ◽  
Hypotheses Testing ◽  
Rule Discovery ◽  
The Novel ◽  
Success Rates ◽  
Positivity Bias ◽  
Task Success ◽  
Counterfactual Model ◽  
Complementarity Theory ◽  
And Task

The standard 2–4–6 task requires discovery of a single rule and produces success rates of about 20%, whereas the dual-goal (DG) version requests discovery of two complementary rules and elevates success to over 60%. The experiment examined two explanations of DG superiority: Evans’ (1989) positivity-bias account, and Wharton, Cheng, and Wickens’ (1993) goal-complementarity theory. Two DG conditions were employed that varied the linguistic labelling of rules (either positively labelled Dax vs. Med, or mixed-valence “fits” vs. “does not fit”). Solution-success results supported the goal-complementarity theory since facilitation arose in both DG conditions relative to single-goal tasks, irrespective of the linguistic labelling of hypotheses. DG instructions also altered quantitative and qualitative aspects of hypothesis-testing behaviour, and analyses revealed the novel result that the production of at least a single descending triple mediates between DG instructions and task success. We propose that the identification of an appropriate contrast class that delimits the scope of complementary rules may be facilitated through the generation of a descending instance. Overall, our findings can best be accommodated by Oaksford and Chater's (1994) iterative counterfactual model of hypotheses testing, which can readily subsume key elements of the goal-complementarity theory.

scholarly journals PEMBANGUNAN DASHBOARD BEASISWA DAN PINJAMAN PROGRAM STUDI INFORMATIKA UKDW

2020 ◽  
Vol 12 (1) ◽  
pp. 1-13
Author(s):  
Agustinus Rendi Walewowan ◽  
Willy Sudiarto Raharjo ◽  
Gloria Virginia
Keyword(s):  
System Usability Scale ◽  
System Usability ◽  
Grade Scale ◽  
Task Success

Ketua program studi (kaprodi) memiliki banyak tugas dan tanggung jawab yang harus dilaksanakan dalam kegiatan akademik. Salah satu tugas dari seorang kaprodi adalah melaporkan seluruh pelaksanaan kegiatan di suatu prodi. Untuk itu, diperlukan suatu sistem yang dapatdigunakan untuk memantau kegiatan operasional sehari-hari dan memberikan laporan. Dashboard adalah sebuah tampilan panel informasi yang digunakan dalam suatu organisasi untuk mengevaluasi suatu masalah sehingga memudahkan seseorang untuk mengambil keputusan. Penelitian ini bertujuan melakukan perancangan sebuah dashboard beasiswa dan pinjaman dengan menggunakan metode prototyping. Prototyping adalah metode pengembangan perangkat lunak, yang berupa model fisik kerja sistem dan berfungsi sebagai versi awal dari sistem. [1]. Hasil rata-rata pengujian task success pada kedua iterasi, yaitu 96,66 sehingga sistem yang dibangun dapat dikatakan cukup efektif dalam menampilkan informasi beasiswa dan pinjaman serta mudah untuk dipelajari. Evaluasi desain antarmuka yang dilakukan menggunakan System Usability Scale (SUS) kepada 5 orang responden pada masing-masing iterasi dan menghasilkan skor SUS 75,2 pada iterasi I dan 76,6 pada iterasi II. Berdasarkan hasil tersebut diperoleh rata-rata skor SUS untuk kedua iterasi, yaitu 75,9. Dengan demikian, antarmuka sistem dinyatakan baik dengan grade scale bernilai C, adjective rating bernilai Good, dan acceptability ranges dapat diterima (acceptable).

Pengujian Usability Website Pondok Pesantren Qodratullah Menggunakan System Usability Scale

2020 ◽  
Vol 6 (1) ◽  
pp. 34-43
Author(s):  
Fitri Purwaningtias ◽  
Usman Ependi
Keyword(s):  
Evaluation Process ◽  
Boarding School ◽  
Educational Institutions ◽  
Prospective Students ◽  
A Value ◽  
System Usability Scale ◽  
System Usability ◽  
Grade Scale ◽  
Final Score ◽  
Group B

Website saat ini telah digunakan diberbagai jenis instansi termasuk instansi pendidikan seperti Pondok Pesantren Qodratullah. Saat ini website Pondok Pesantren Qodratullah menjadi tulang punggung dalam penyebaran informasi terkain pondok pesantren kepada wali santri, alumni, calon santri dan masyarakat luas. Mengingat pentingnya website bagi Pondok Pesantren Qodratullah maka perlu untuk dilakukan evaluasi apakah informasi yang diberikan dan website yang ada telah memiliki nilai kebergunaan bagi pengguna atau tidak. Untuk itu di dalam penelitian ini dilakukan evaluasi untuk melihat perspektif pengguna terhadap website. Prose evaluasi dilakukan dengan system usability scale dengan sepuluh instrumen sebagai pernyataan evaluasi. Hasil evaluasi menunjukkan bahwa website Pondok Pesantren Qodratullah mendapatkan nilai akhir 88. Nilai 88 berarti website Pondok Pesantren Qodratullah mendapatkan adjective rating yang excellence, grade scale tergolong kelompok B dan tingkat acceptability termasuk acceptable. The website is currently used in various types of institutions including educational institutions such as Qodratullah Islamic Boarding School. Currently the Qodratullah Islamic Boarding School website is the backbone in the dissemination of information about Islamic boarding schools to the guardians of students, alumni, prospective students and the wider community. Considering the importance of the website for the Qodratullah Islamic Boarding School, it is necessary to evaluate whether the information provided, and the existing website have a useful value for the user or not. For this reason, in this study an evaluation was conducted to see the user's perspective on the website. The evaluation process is carried out with a system usability scale with ten instruments as evaluation statements. Evaluation results show that the Qodratullah Islamic Boarding School website gets a final score of 88. A value of 88 means that the Qodratullah Islamic Boarding School website gets an adjective rating that excellence, grade scale belongs to group B and the level of acceptability is acceptable

scholarly journals Comparison of the validity, perceived usefulness and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability: a study protocol

BMJ Open ◽  
2021 ◽  
Vol 11 (8) ◽  
pp. e050448
Author(s):  
Romaric Marcilly ◽  
Wu Yi Zheng ◽  
Regis Beuscart ◽  
Melissa T Baysari
Keyword(s):  
Human Factors ◽  
Convergent Validity ◽  
Perceived Usefulness ◽  
Decision Makers ◽  
Free Text ◽  
Alert System ◽  
System Usability Scale ◽  
System Usability ◽  
Ethical Approval ◽  
Alerting Systems

IntroductionResearch has shown that improvements to the usability of medication alert systems are needed. For designers and decisions-makers to assess usability of their alert systems, two paper-based tools are currently available: the instrument for evaluating human-factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aims to compare the validity, usability and usefulness of both tools to identify their strengths and limitations and assist designers and decision-makers in making an informed decision about which tool is most suitable for assessing their current or prospective system.Methods and analysisFirst, TEMAS and I-MeDeSA will be translated into French. This translation will be validated by three experts in human factors. Then, in 12 French hospitals with a medication alert system in place, staff with expertise in the system will evaluate their alert system using the two tools successively. After the use of each tool, participants will be asked to fill in the System Usability Scale (SUS) and complete a survey on the understandability and perceived usefulness of each tool. Following the completion of both assessments, participants will be asked to nominate their preferred tool and relay their opinions on the tools. The design philosophy of TEMAS and I-MeDeSA differs on the calculation of a score, impacting the way the comparison between the tools can be performed. Convergent validity will be evaluated by matching the items of the two tools with respect to the usability dimensions they assess. SUS scores and answers to the survey will be statistically compared for I-MeDeSA and TEMAS to identify differences. Free-text responses in surveys will be analysed using an inductive approach.Ethics and disseminationEthical approval is not required in France for a study of this nature. The results will be published in a peer-reviewed journal.

The Relation of the Perceptions of Aesthetics and Usability

2020 ◽  
Vol 64 (1) ◽  
pp. 1876-1880
Author(s):  
Dahlia Alharoon ◽  
Douglas J. Gillan
Keyword(s):  
Product Design ◽  
Research Study ◽  
System Usability Scale ◽  
System Usability ◽  
Perceived Usability

Aesthetics and usability both play critical roles in product design. But how might measurement of these two conceptually-different features of products interfere with one another? The current research study examines the effect of differences in aesthetics on perceived usability. Participants completed three tasks on a simulated website with a low usability interface. One group of participants used an interface with high aesthetics, whereas a second group interacted with an interface with poor aesthetics. Both groups rated the usability and aesthetics of the interface after completing the tasks. The aesthetics manipulation was effective in that the high aesthetics group provided higher ratings on two aesthetics scales than did the low aesthetics group; however, differences in aesthetics had no significant effect on usability as measured by the System Usability Scale (SUS). These findings support the idea that users make independent judgments of usability and aesthetics.

scholarly journals Increasing Usability of Homecare Applications for Older Adults: A Case Study

Designs ◽  
2019 ◽  
Vol 3 (2) ◽  
pp. 23 ◽  
Author(s):  
Christos Panagopoulos ◽  
Andreas Menychtas ◽  
Panayiotis Tsanakas ◽  
Ilias Maglogiannis
Keyword(s):  
Emergency Response ◽  
Independent Living ◽  
The Elderly ◽  
Usability Assessment ◽  
Technological Advances ◽  
System Usability Scale ◽  
System Usability ◽  
Practical Guidelines ◽  
Depth Interviews

As the world’s population is ageing, the field dealing with technology adoption by seniors has made headway in the scientific community. Recent technological advances have enabled the development of intelligent homecare systems that support seniors’ independent living and allow monitoring of their health status. However, despite the amount of research to understand the requirements of systems designed for the elderly, there are still unresolved usability issues that often prevent seniors from enjoying the benefits that modern ICT technologies may offer. This work presents a usability assessment of “HeartAround”, an integrated homecare solution incorporating communication functionalities, as well as health monitoring and emergency response features. An assessment with the system usability scale (SUS) method, along with in-depth interviews and qualitative analysis, has provided valuable insights for designing homecare systems for seniors, and validated some effective practical guidelines.

MOTION PLANNING USING PREDICTED PERCEPTIVE CAPABILITY

2009 ◽  
Vol 06 (03) ◽  
pp. 435-457 ◽  
Author(s):  
PHILIPP MICHEL ◽  
JOEL CHESTNUTT ◽  
SATOSHI KAGAMI ◽  
KOICHI NISHIWAKI ◽  
JAMES J. KUFFNER ◽  
...  
Keyword(s):  
Motion Planning ◽  
Planning Process ◽  
Humanoid Robots ◽  
Experimental Results ◽  
Success Rates ◽  
Perception System ◽  
Task Success

We present an approach to motion planning for humanoid robots that aims to ensure reliable execution by augmenting the planning process to reason about the robot's ability to successfully perceive its environment during operation. By efficiently simulating the robot's perception system during search, our planner utilizes a perceptive capability metric that quantifies the 'sensability' of the environment in each state given the task to be accomplished. We have applied our method to the problem of planning robust autonomous grasping motions and walking sequences as performed by an HRP-2 humanoid. A fast GPU-accelerated 3D tracker is used for perception, with a grasp planner and footstep planner incorporating reasoning about the robot's perceptive capability. Experimental results show that considering information about the predicted perceptive capability ensures that sensing remains operational throughout the grasping or walking sequence and yields higher task success rates than perception-unaware planning.

Sign in / Sign up

Export Citation Format

Share Document