User testing when creating a technological project

Abstract Background Vaginal mesh implants are medical devices used in a number of operations to treat stress urinary incontinence and pelvic organ prolapse. Although many of these operations have delivered good outcomes, some women have experienced serious complications that have profoundly affected their quality of life. To ensure that evolving patient information is up-to-date, accurate and appropriate, the Transvaginal Mesh Oversight Group ‘user-tested’ a newly developed Scottish patient resource, the first to focus exclusively on the issue of complications. The aim of this research was to gather feedback on usability, content, language and presentation to inform the development of the resource from a user perspective. Methods The experience of using the patient resource was captured through semi-structured interviews that followed a ‘think-aloud’ protocol. The interviewer observed each participant as they went through the resource, asking questions and making field notes. Participants’ comments were then categorised using a validated model of user experience and subsequently analysed thematically. Results Thirteen people participated in the user testing interviews, including women with lived experience of mesh implants (n = 7), a convenience sample of staff working for Healthcare Improvement Scotland (n = 5) and a patient’s carer (n = 1). The majority of participants considered the resource as clear and helpful. Respondents reported that some presentational aspects promoted usability and understandability, including the use of a font that is easy to read, bullet lists, coloured headings and simple language. Barriers included the reliance on some technical language and an explicit anatomical diagram. Participants endorsed the valuable role of health professionals as co-mediators of patient information. Conclusions The findings illustrate the value of undertaking in-depth user-testing for patient information resources before their dissemination. The study highlighted how the direct guidance or navigation of a patient information resource by a health professional could increase its salience and accuracy of interpretation by patients, their families and carers. These insights may also be useful to other developers in improving patient information.

Download Full-text

Costs and Cost-Effectiveness of User-Testing of Health Professionals’ Guidelines to Reduce the Frequency of Intravenous Medicines Administration Errors by Nurses in the United Kingdom: A Probabilistic Model Based on Voriconazole Administration

Applied Health Economics and Health Policy ◽

10.1007/s40258-021-00675-z ◽

2021 ◽

Author(s):

Matthew D. Jones ◽

Bryony Dean Franklin ◽

D. K. Raynor ◽

Howard Thom ◽

Margaret C. Watson ◽

...

Keyword(s):

Cost Effectiveness ◽

Medication Error ◽

Low Cost ◽

Cost Savings ◽

Credible Interval ◽

Probability Of Detection ◽

Sensitivity Analyses ◽

Decision Analytic Model ◽

Published Data ◽

User Testing

Abstract Aim In the UK, injectable medicines are often prepared and administered by nurses following the Injectable Medicines Guide (IMG). Our earlier study confirmed a higher frequency of correct administration with user-tested versus standard IMG guidelines. This current study aimed to model the cost-effectiveness of user-testing. Methods The costs and cost-effectiveness of user-testing were explored by modifying an existing probabilistic decision-analytic model. The adapted model considered administration of intravenous voriconazole to hospital inpatients by nurses. It included 11 error types, their probability of detection and level of harm. Model inputs (including costs) were derived from our previous study and other published data. Monte Carlo simulation using 20,000 samples (sufficient for convergence) was performed with a 5-year time horizon from the perspective of the 121 NHS trusts and health boards that use the IMG. Sensitivity analyses were undertaken for the risk of a medication error and other sources of uncertainty. Results The net monetary benefit at £20,000/quality-adjusted life year was £3,190,064 (95% credible interval (CrI): −346,709 to 8,480,665), favouring user-testing with a 96% chance of cost-effectiveness. Incremental cost-savings were £240,943 (95% CrI 43,527–491,576), also favouring user-tested guidelines with a 99% chance of cost-saving. The total user testing cost was £6317 (95% CrI 6012–6627). These findings were robust to assumptions about a range of input parameters, but greater uncertainty was seen with a lower medication error risk. Conclusions User-testing of injectable medicines guidelines is a low-cost intervention that is highly likely to be cost-effective, especially for high-risk medicines.

Download Full-text

A DATA-DRIVEN APPROACH TO USER-EXPERIENCE-FOCUSED MODEL-BASED ROADMAPPING FOR NEW PRODUCT PLANNING

Proceedings of the Design Society ◽

10.1017/pds.2021.7 ◽

2021 ◽

Vol 1 ◽

pp. 61-70

Author(s):

Ilia Iuskevich ◽

Andreas-Makoto Hein ◽

Kahina Amokrane-Ferka ◽

Abdelkrim Doufene ◽

Marija Jankovic

Keyword(s):

New Product Development ◽

User Experience ◽

Planning Process ◽

New Product ◽

Data Driven ◽

User Needs ◽

User Testing ◽

Model Based ◽

Data Driven Approach ◽

To Receive

AbstractUser experience (UX) focused business needs to survive and plan its new product development (NPD) activities in a highly turbulent environment. The latter is a function of volatile UX and technology trends, competition, unpredictable events, and user needs uncertainty. To address this problem, the concept of design roadmapping has been proposed in the literature. It was argued that tools built on the idea of design roadmapping have to be very flexible and data-driven (i.e., be able to receive feedback from users in an iterative manner). At the same time, a model-based approach to roadmapping has emerged, promising to achieve such flexibility. In this work, we propose to incorporate design roadmapping to model-based roadmapping and integrate it with various user testing approaches into a single tool to support a flexible data-driven NPD planning process.

Download Full-text

Things Change: Comparing Results Using Historical Data and User Testing for Evaluating a Recommendation Task

Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems ◽

10.1145/3334480.3382945 ◽

2020 ◽

Author(s):

Soon-Gyo Jung ◽

Joni Salminen ◽

Shammur A. Chowdhury ◽

Dianne Ramirez Robillos ◽

Bernard J. Jansen

Keyword(s):

Historical Data ◽

User Testing

Download Full-text

Evaluating the Usability of University Waste-Sorting Systems

Proceedings of the Human Factors and Ergonomics Society Annual Meeting ◽

10.1177/1071181320641494 ◽

2020 ◽

Vol 64 (1) ◽

pp. 2046-2050

Author(s):

Nathan Dumessa ◽

Katie Lucaites ◽

Hannah Solini ◽

Jeremy Lopez ◽

Paige Lawton ◽

...

Keyword(s):

User Testing ◽

Experimental Protocol ◽

Clemson University ◽

Human Centered Design ◽

Testing Methodology ◽

Design Practices ◽

Competitive Price ◽

Large Populations ◽

New System

Recycling is a widely adopted practice that can reduce waste in landfills and increase profits for companies, corporations, and universities, such as Clemson, that sell recyclable materials. For these materials to be sold at a competitive price, however, they must be uncontaminated. This is a considerable barrier for Clemson University because users’ knowledge and experience with recycling do not align with Clemson University’s expected recycling practices. In an effort to bridge this gap, we followed human-centered design practices to generate and test prototypes of new system signage. We found that users were more accurate and confident in their sorting behavior when interacting with the new signage. Users also perceived the new signage as more usable than the old signage. In addition to these findings, the results of this research include a user testing methodology and experimental protocol that can be implemented in other contexts where large populations sort waste.

Download Full-text