scholarly journals Cognitive Testing of an Electronic Consent Platform: Researcher Perspectives

2021 ◽  
Author(s):  
Daniel Robins ◽  
Rachel Brody ◽  
Irena Parvanova ◽  
Joseph Finkelstein
Keyword(s):  
Qualitative Interview ◽  
Cognitive Testing ◽  
Valuable Insight ◽  
Patient Understanding ◽  
Domain Experts ◽  
Expert Review ◽  
System Usability Scale ◽  
Job Titles ◽  
System Usability ◽  
The Mean

This study focuses on feedback from domain experts to assess usability and acceptance of the E-Consent electronic consent platform. Quantitative and qualitative data were captured throughout the usability inspection, which was structured around a cognitive walkthrough with heuristics evaluation. Additional surveys measured biobanking knowledge and attitudes and familiarity with informed consent. A semi-structured qualitative interview captured open-ended feedback. 23 researchers of various ages and job titles were included for analysis. The System Usability Scale (SUS) provided a standardized reference for usability and satisfaction, and the mean result of 86.7 corresponds with an ‘above average’ usability rating in the >90th percentile. Overall, participants believe that electronic consenting using this platform will be faster than previous workflows while enhancing patient understanding, and human rapport is still a key component of the consent process. Expert review has provided valuable insight and actionable information that will be used to further enhance this maturing platform.

scholarly journals NLPReViz: an interactive tool for natural language processing on clinical text

2017 ◽  
Vol 25 (1) ◽  
pp. 81-87 ◽  
Author(s):  
Gaurav Trivedi ◽  
Phuong Pham ◽  
Wendy W Chapman ◽  
Rebecca Hwa ◽  
Janyce Wiebe ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
User Study ◽  
Clinical Text ◽  
Domain Experts ◽  
Expert Review ◽  
System Usability Scale ◽  
Average System ◽  
Colonoscopy Quality

Abstract The gap between domain experts and natural language processing expertise is a barrier to extracting understanding from clinical text. We describe a prototype tool for interactive review and revision of natural language processing models of binary concepts extracted from clinical notes. We evaluated our prototype in a user study involving 9 physicians, who used our tool to build and revise models for 2 colonoscopy quality variables. We report changes in performance relative to the quantity of feedback. Using initial training sets as small as 10 documents, expert review led to final F1scores for the “appendiceal-orifice” variable between 0.78 and 0.91 (with improvements ranging from 13.26% to 29.90%). F1for “biopsy” ranged between 0.88 and 0.94 (−1.52% to 11.74% improvements). The average System Usability Scale score was 70.56. Subjective feedback also suggests possible design improvements.

scholarly journals Usability evaluation of a virtual reality smartphone app for a living museum

2021 ◽  
Author(s):  
Mohd Kamal Othman ◽  
Altynai Nogoibaeva ◽  
Lai San Leong ◽  
Mohamad Hardyman Barawi
Keyword(s):  
Prior Experience ◽  
Usability Evaluation ◽  
Test Results ◽  
System Usability Scale ◽  
Significant Difference ◽  
System Usability ◽  
Usability Issue ◽  
The Mean ◽  
Group 2 ◽  
Group 1

AbstractThis paper elaborates the empirical evidence of a usability evaluation of a VR and non-VR virtual tour application for a living museum. The System Usability Scale (SUS) was used in between participants experiments (Group 1: non-VR version and Group 2: VR version) with 40 participants. The results show that the mean scores of all components for the VR version are higher compared to the non-VR version, overall SUS score (72.10 vs 68.10), usability score (75.50 vs 71.70), and learnability (58.40 vs 57.00). Further analysis using a two-tailed independent t test showed no difference between the non-VR and VR versions. Additionally, no significant difference was observed between the groups in the context of gender, nationality, and prior experience (other VR tour applications) for overall SUS score, usability score, and learnability score. Α two-tailed independent t test indicated no significant difference in the usability score between participants with VR experience and no VR experience. However, a significant difference was found between participants with VR experience and no VR experience for both SUS score (t(38) = 2.17, p = 0.037) and learnability score (t(38) = 2.40, p = 0.021). The independent t test results indicated a significant difference between participant with and without previous visits to SCV for the usability score (t(38) = −2.31, p = 0.027), while there was no significant differences observed in other components. It can be concluded that both versions passed based on the SUS score. However, the sub-scale usability and learnability scores indicated some usability issue.

scholarly journals Evaluasi tingkat kebergunaan aplikasi Administrasi Penduduk menggunakan teknik System Usability Scale

2019 ◽  
Vol 5 (1) ◽  
pp. 63 ◽  
Author(s):  
Usman Ependi ◽  
Ade Putra ◽  
Febriyanti Panjaitan
Keyword(s):  
Population Data ◽  
Evaluation Instrument ◽  
System Usability Scale ◽  
System Technique ◽  
System Usability ◽  
Grade Scale ◽  
Final Score ◽  
The Mean ◽  
The Village

Kebergunaan (usability) adalah salah satu faktor penting dalam sebuah aplikasi. Saat ini, Dinas Pemberdayaan Masyarakat Desa (PMD) Kabupaten Ogan Ilir telah menyediakan aplikasi administrasi penduduk desa sebagai usaha untuk mencapai tertib administrasi, terutama berkaitan dengan data penduduk. Untuk itu, penelitian ini mengusulkan evaluasi tingkat kebergunaan aplikasi dengan teknik System Usability Scale untuk melihat kelayakan aplikasi yang disediakan untuk digunakan. System usability scale memiliki sepuluh pernyataan sebagai instrumen evaluasi dan tiga indikator penilaian, yaitu acceptability, grade scale, dan adjective rating. Masing-masing pernyataan menjadi indikator untuk melihat tingkat kebergunaan aplikasi. Hasil evaluasi menunjukkan bahwa mayoritas responden dapat memanfaatkan aplikasi dengan baik yang dibuktikan dari nilai rerata setiap pernyataan > 3. Selain itu, aplikasi administrasi penduduk desa juga dapat dikategorikan sebagai excellent serta berada pada grade B dengan nilai akhir 88,5.  Usability is one of the most important factors of an application. Currently, the PMD Office of Ogan Ilir District has provided villages with administrative applications as an effort to achieve administration order and boost its effectiveness while analyzing population data. This research evaluates the usability and feasibility levels of applications using the scale system technique. This consists of ten statements as an evaluation instrument with acceptability, grade scale, and adjective rating used as test indicators. The evaluation results shows, that majority of respondents argue that the application is easily proven from the mean score of each statement > 3. In addition, the village administration application is in very good category and it is in grade B with a final score of 88.5.

scholarly journals Characteristics, Usability, and Users Experience of a System Combining Cognitive and Physical Therapy in a Virtual Environment: Positive Bike

Sensors ◽  
2018 ◽  
Vol 18 (7) ◽  
pp. 2343 ◽  
Author(s):  
Elisa Pedroli ◽  
Luca Greci ◽  
Desirèe Colombo ◽  
Silvia Serino ◽  
Pietro Cipresso ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Elderly Patients ◽  
Evaluation Study ◽  
Structured Interview ◽  
Flow State ◽  
Virtual Reality System ◽  
Domain Experts ◽  
System Usability Scale ◽  
System Usability ◽  
State Scale

We present the architecture and usability evaluation of virtual reality system—“Positive Bike”—designed for improving cognitive and motor conditions in frail elderly patients. The system consists of a cycle-ergometer integrated in an immersive virtual reality system (CAVE) which allows combining motor and cognitive exercises according to a “dual-task” paradigm. We tested the usability and user’s experience of the prototype in a pilot evaluation study that involved five elderly patients. The prototype was tested in one-session training to understand the limitations and areas for improvement of our system. The evaluation consisted in (i) usability assessment using the system usability scale; (ii) evaluation of user’s engagement using the flow state scale; and (iii) expert evaluation involving interviews with domain experts. Results showed a good usability, both for system usability scale and the semi-structured interview. The level of flow (i.e., enjoyment with the task at hand) measured using the short flow state scale, was also high. Analysis of semi-structured interview carried out with domain experts provided further indications to improve the system. Overall, these findings show that, despite some limitations, the system is usable and provides an enjoyable user’s experience.

scholarly journals Validation of a Novel Electronic Device for Medication Adherence Monitoring of Ambulatory Patients

Pharmacy ◽  
2019 ◽  
Vol 7 (4) ◽  
pp. 155 ◽  
Author(s):  
Isabelle Arnet ◽  
Jean-Pierre Rothen ◽  
Valerie Albert ◽  
Kurt Hersberger
Keyword(s):  
Medication Adherence ◽  
Electronic Device ◽  
Daily Intake ◽  
Elderly Outpatients ◽  
System Usability Scale ◽  
System Usability ◽  
Ambulatory Patients ◽  
The Mean ◽  
Adherence Monitoring ◽  
Very High

Several methods exist for measuring medication adherence. The Time4MedTM device (Adherence Innovations, Hong Kong) is a small, electronic card to affix on medication packaging that records date and time of intakes when a button is pushed. We aimed to validate the device with an emphasis on polypharmacy. Twenty volunteers used Time4MedTM devices with a virtual thrice daily intake over 14 days. Diary-recorded date and time were compared to electronically-stored events. Functionality, reliability and recovery for different stress conditions were calculated. User‘s acceptability was assessed with the System Usability Scale (SUS). Eleven elderly outpatients (mean age 80.2 ± 8.1 years) taking >3 medications daily used the device over 4 weeks. Volunteers logged 847 events. Functionality (100%), sensitivity (94.9%), specificity (99.4%) and recovery (100%) were high. Dropping the smart card and storing it in a refrigerator caused either the recording of false events or no recording at all. The mean SUS score was 82.6 (SD 14.8), demonstrating excellent acceptability. Satisfaction was very high for volunteers and patients, except for pushing the button. Time4MedTM devices are highly accurate in recording, retaining and delivering electronic data of multiple medication intake. They are well accepted by elderly patients. They can be recommended in clinical studies and for practitioners who desire to elucidate adherence patterns of ambulatory patients.

Pengujian Usability Website Pondok Pesantren Qodratullah Menggunakan System Usability Scale

2020 ◽  
Vol 6 (1) ◽  
pp. 34-43
Author(s):  
Fitri Purwaningtias ◽  
Usman Ependi
Keyword(s):  
Evaluation Process ◽  
Boarding School ◽  
Educational Institutions ◽  
Prospective Students ◽  
A Value ◽  
System Usability Scale ◽  
System Usability ◽  
Grade Scale ◽  
Final Score ◽  
Group B

Website saat ini telah digunakan diberbagai jenis instansi termasuk instansi pendidikan seperti Pondok Pesantren Qodratullah. Saat ini website Pondok Pesantren Qodratullah menjadi tulang punggung dalam penyebaran informasi terkain pondok pesantren kepada wali santri, alumni, calon santri dan masyarakat luas. Mengingat pentingnya website bagi Pondok Pesantren Qodratullah maka perlu untuk dilakukan evaluasi apakah informasi yang diberikan dan website yang ada telah memiliki nilai kebergunaan bagi pengguna atau tidak. Untuk itu di dalam penelitian ini dilakukan evaluasi untuk melihat perspektif pengguna terhadap website. Prose evaluasi dilakukan dengan system usability scale dengan sepuluh instrumen sebagai pernyataan evaluasi. Hasil evaluasi menunjukkan bahwa website Pondok Pesantren Qodratullah mendapatkan nilai akhir 88. Nilai 88 berarti website Pondok Pesantren Qodratullah mendapatkan adjective rating yang excellence, grade scale tergolong kelompok B dan tingkat acceptability termasuk acceptable. The website is currently used in various types of institutions including educational institutions such as Qodratullah Islamic Boarding School. Currently the Qodratullah Islamic Boarding School website is the backbone in the dissemination of information about Islamic boarding schools to the guardians of students, alumni, prospective students and the wider community. Considering the importance of the website for the Qodratullah Islamic Boarding School, it is necessary to evaluate whether the information provided, and the existing website have a useful value for the user or not. For this reason, in this study an evaluation was conducted to see the user's perspective on the website. The evaluation process is carried out with a system usability scale with ten instruments as evaluation statements. Evaluation results show that the Qodratullah Islamic Boarding School website gets a final score of 88. A value of 88 means that the Qodratullah Islamic Boarding School website gets an adjective rating that excellence, grade scale belongs to group B and the level of acceptability is acceptable

scholarly journals Comparison of the validity, perceived usefulness and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability: a study protocol

BMJ Open ◽  
2021 ◽  
Vol 11 (8) ◽  
pp. e050448
Author(s):  
Romaric Marcilly ◽  
Wu Yi Zheng ◽  
Regis Beuscart ◽  
Melissa T Baysari
Keyword(s):  
Human Factors ◽  
Convergent Validity ◽  
Perceived Usefulness ◽  
Decision Makers ◽  
Free Text ◽  
Alert System ◽  
System Usability Scale ◽  
System Usability ◽  
Ethical Approval ◽  
Alerting Systems

IntroductionResearch has shown that improvements to the usability of medication alert systems are needed. For designers and decisions-makers to assess usability of their alert systems, two paper-based tools are currently available: the instrument for evaluating human-factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aims to compare the validity, usability and usefulness of both tools to identify their strengths and limitations and assist designers and decision-makers in making an informed decision about which tool is most suitable for assessing their current or prospective system.Methods and analysisFirst, TEMAS and I-MeDeSA will be translated into French. This translation will be validated by three experts in human factors. Then, in 12 French hospitals with a medication alert system in place, staff with expertise in the system will evaluate their alert system using the two tools successively. After the use of each tool, participants will be asked to fill in the System Usability Scale (SUS) and complete a survey on the understandability and perceived usefulness of each tool. Following the completion of both assessments, participants will be asked to nominate their preferred tool and relay their opinions on the tools. The design philosophy of TEMAS and I-MeDeSA differs on the calculation of a score, impacting the way the comparison between the tools can be performed. Convergent validity will be evaluated by matching the items of the two tools with respect to the usability dimensions they assess. SUS scores and answers to the survey will be statistically compared for I-MeDeSA and TEMAS to identify differences. Free-text responses in surveys will be analysed using an inductive approach.Ethics and disseminationEthical approval is not required in France for a study of this nature. The results will be published in a peer-reviewed journal.

The Relation of the Perceptions of Aesthetics and Usability

2020 ◽  
Vol 64 (1) ◽  
pp. 1876-1880
Author(s):  
Dahlia Alharoon ◽  
Douglas J. Gillan
Keyword(s):  
Product Design ◽  
Research Study ◽  
System Usability Scale ◽  
System Usability ◽  
Perceived Usability

Aesthetics and usability both play critical roles in product design. But how might measurement of these two conceptually-different features of products interfere with one another? The current research study examines the effect of differences in aesthetics on perceived usability. Participants completed three tasks on a simulated website with a low usability interface. One group of participants used an interface with high aesthetics, whereas a second group interacted with an interface with poor aesthetics. Both groups rated the usability and aesthetics of the interface after completing the tasks. The aesthetics manipulation was effective in that the high aesthetics group provided higher ratings on two aesthetics scales than did the low aesthetics group; however, differences in aesthetics had no significant effect on usability as measured by the System Usability Scale (SUS). These findings support the idea that users make independent judgments of usability and aesthetics.

scholarly journals Impact of a Smart-Ring-Based Feedback System on the Quality of Chest Compressions in Adult Cardiac Arrest: A Randomized Preliminary Study

2021 ◽  
Vol 18 (10) ◽  
pp. 5408
Author(s):  
Chiwon Ahn ◽  
Seungjae Lee ◽  
Jongshill Lee ◽  
Jaehoon Oh ◽  
Yeongtak Song ◽  
...  
Keyword(s):  
Primary Outcome ◽  
Feedback System ◽  
Controlled Study ◽  
Feedback Group ◽  
Compression Depth ◽  
System Usability Scale ◽  
The Mean ◽  
Preliminary Study ◽  
The Ideal

This study aimed to assess the effectiveness of a novel chest compression (CC) smart-ring-based feedback system in a manikin simulation. In this randomized, crossover, controlled study, we evaluated the effect of smart-ring CC feedback on cardiopulmonary resuscitation (CPR). The learnability and usability of the tool were evaluated with the System Usability Scale (SUS). Participants were divided into two groups and each performed CCs with and without feedback 2 weeks apart, using different orders. The primary outcome was compression depth; the proportion of accurate-depth (5–6 cm) CCs, CC rate, and the proportion of complete CCs (≤1 cm of residual leaning) were assessed additionally. The feedback group and the non-feedback group showed significant differences in compression depth (52.1 (46.3–54.8) vs. 47.1 (40.5–49.9) mm, p = 0.021). The proportion of accurate-depth CCs was significantly higher in the interventional than in the control condition (88.7 (30.0–99.1) vs. 22.6 (0.0–58.5%), p = 0.033). The mean SUS score was 83.9 ± 8.7 points. The acceptability ranges were ‘acceptable’, and the adjective rating was ‘excellent’. CCs with smart-ring feedback could help achieve the ideal range of depth during CPR. The smart-ring may be a valuable source of CPR feedback.

scholarly journals Increasing Usability of Homecare Applications for Older Adults: A Case Study

Designs ◽  
2019 ◽  
Vol 3 (2) ◽  
pp. 23 ◽  
Author(s):  
Christos Panagopoulos ◽  
Andreas Menychtas ◽  
Panayiotis Tsanakas ◽  
Ilias Maglogiannis
Keyword(s):  
Emergency Response ◽  
Independent Living ◽  
The Elderly ◽  
Usability Assessment ◽  
Technological Advances ◽  
System Usability Scale ◽  
System Usability ◽  
Practical Guidelines ◽  
Depth Interviews

As the world’s population is ageing, the field dealing with technology adoption by seniors has made headway in the scientific community. Recent technological advances have enabled the development of intelligent homecare systems that support seniors’ independent living and allow monitoring of their health status. However, despite the amount of research to understand the requirements of systems designed for the elderly, there are still unresolved usability issues that often prevent seniors from enjoying the benefits that modern ICT technologies may offer. This work presents a usability assessment of “HeartAround”, an integrated homecare solution incorporating communication functionalities, as well as health monitoring and emergency response features. An assessment with the system usability scale (SUS) method, along with in-depth interviews and qualitative analysis, has provided valuable insights for designing homecare systems for seniors, and validated some effective practical guidelines.

Sign in / Sign up

Export Citation Format

Share Document