scholarly journals Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems

2019 ◽  
Vol 29 (2) ◽  
pp. 325-333 ◽  
Author(s):  
Olalekan Lee Aiyegbusi
Keyword(s):  
User Interfaces ◽  
Digital Health ◽  
Usability Testing ◽  
Target Population ◽  
Patient Reported Outcome ◽  
End Users ◽  
Methodological Issues ◽  
Stage Of Development ◽  
Usability Tests ◽  
Patient Reported

Abstract Introduction Recent advances in information technology and improved access to the internet have led to a rapid increase in the adoption and ownership of electronic devices such as touch screen smartphones and tablet computers. This has also led to a renewed interest in the field of digital health also referred to as telehealth or electronic health (eHealth). There is now a drive to collect these PROs electronically using ePRO systems. Method However, the user interfaces of ePRO systems need to be adequately assessed to ensure they are not only fit for purpose but also acceptable to patients who are the end users. Usability testing is a technique that involves the testing of systems, products or websites with participants drawn from the target population. Usability testing can assist ePRO developers in the evaluation of ePRO user interface. The complexity of ePRO systems; stage of development; metrics to measure; and the use of scenarios, moderators and appropriate sample sizes are key methodological issues to consider when planning usability tests. Conclusion The findings from usability testing may facilitate the improvement of ePRO systems making them more usable and acceptable to end users. This may in turn improve the adoption of ePRO systems post-implementation. This article highlights the key methodological issues to consider and address when planning usability testing of ePRO systems.

Integrating Semiotics Perception in Usability Testing to Improve Usability Evaluation

2013 ◽  
pp. 145-169 ◽  
Author(s):  
Muhammad Nazrul Islam ◽  
Franck Tétard
Keyword(s):  
User Interfaces ◽  
Usability Testing ◽  
End Users ◽  
Usability Evaluation ◽  
Computer Applications ◽  
Computer Application ◽  
Semiotic Analysis ◽  
Usability Problems ◽  
Intuitive Interface ◽  
Methodological Guidelines

User interfaces of computer applications encompass a number of objects such as navigation links, buttons, icons, and thumbnails. In this chapter, these are called interface signs. The content and functions of a computer application are generally directed by interface signs to provide the system’s logic to the end users. The interface signs of a usable application need to be intuitive to end users and therefore a necessary part of usability evaluation. Assessing sign intuitiveness can be achieved through a semiotic analysis. This study demonstrates how a semiotic assessment of interface signs’ intuitiveness yielded a number of benefits. For instance, (i) it provides an overall idea of interface signs’ intuitiveness to the end users to interpret the meaning of interface signs, (ii) it assists in finding usability problems and also in (iii) recommending possible solutions, (iv) provides background for introducing guidelines to design user-intuitive interface signs, (v) helps in constructing heuristic checklist from semiotics perspective to evaluate an application, (vi) no additional resource and extra budget are needed. This study also presents a list of methodological guidelines to obtain the perceived benefits of integrating semiotic perception in usability testing for practitioners.

An Interactive Tool for the Participatory Design of Product Interface

2012 ◽  
Author(s):  
Loris Barbieri ◽  
Agostino Angilica ◽  
Fabio Bruno ◽  
Maurizio Muzzupappa
Keyword(s):  
Design Process ◽  
User Interfaces ◽  
Participatory Design ◽  
Mixed Reality ◽  
End Users ◽  
User Centered Design ◽  
Accurate Analysis ◽  
Usability Tests ◽  
Interactive Tool ◽  
Novel Approach

The importance of participatory design (PD) is progressively increasing thanks to its capacity to explore a wide variety of concepts, thus increasing the opportunity to create a successful product. In fact the design process should not be a solo activity, as designers often need inputs and other points of view, especially from end-users. According to the ultimate idea of PD, end-users are actively involved in the various activities of the product development to ensure that their needs and desires are satisfied. This paper presents a novel approach to the participatory design of product interfaces in a user-centered design (UCD) process. The approach is based on an interactive tool that allows end-users to design custom user interfaces of household appliances taking advantage of their own needs and experiences. The tool incorporates the analytical and more abstract knowledge of the designers codified in the form of aesthetical, technological and manufacturing constraints (i.e., limitations in the number and geometry of interface components, a limited number of colors, a discretization of the area where interface widgets are placed). This solution allows the end-users to directly design their favorite interface without the interference of any other subject. Through an accurate analysis of the choices done by the users, the designers are able to access to the deepest level of the users’ expression in order to catch their latent needs and tacit knowledge. The tool has been designed in order to make possible to immediately perform usability tests on the designed interface by using a Mixed Reality prototype. The paper describes the development of the tool and proposes a methodology that has been specifically addressed to include this tool in a design process based on UCD principles. Both the tool and the methodology are presented through the description of a case-study related to the redesign of a washing machine dashboard. Experimental results show that the proposed tool can be an effective support to design product interfaces during PD sessions.

VP75 Improving Access To Ultra-Orphan Medicines In NHS Scotland

2019 ◽  
Vol 35 (S1) ◽  
pp. 93-93
Author(s):  
Noreen Downes ◽  
Jan Jones ◽  
Anne Lee ◽  
Ailsa Brown ◽  
Pauline McGuire ◽  
...  
Keyword(s):  
Unmet Need ◽  
Clinical Effectiveness ◽  
Patient Reported Outcome Measures ◽  
Patient Reported Outcome ◽  
Quality Adjusted Life Year ◽  
Final Decision ◽  
Stage Of Development ◽  
Patient Reported ◽  
Evidence Collection ◽  
Clinical Benefits

IntroductionMedicines for very rare conditions present challenges for healthcare globally due to uncertain evidence and often extremely high costs. In 2014, SMC introduced an ultra-orphan framework placing less emphasis on the cost per quality adjusted life year (QALY). Despite this, many medicines continued to be not recommended. A new pathway aimed at improved patient access based on further evidence collection is now being implemented.MethodsThe development of the new pathway has involved collaboration with key stakeholders including patient groups, the pharmaceutical industry, and clinicians. Medicines that meet a new definition (based on four criteria including the prevalence of the condition treated) will be appraised by the SMC committee and a data collection plan will then be agreed with the pharmaceutical company.ResultsFrom April 2019, medicines validated as ultra-orphans will initially be appraised using the broader decision-making framework and the SMC committee will outline key uncertainties in the clinical effectiveness. The medicine will then be available for a period of at least three years while further data are gathered, potentially comprising ongoing clinical trials, registry data, and patient reported outcome measures. SMC will then re-assess the clinical and economic evidence to inform a final decision on routine use of the medicine in NHS Scotland.ConclusionsThe new pathway for ultra-orphan medicines will allow further evidence on their longer-term clinical benefits to be collected before a final decision on routine use. This approach reflects the current direction of travel in medicines regulation, by making medicines that address an unmet need available to patients at an earlier stage of development.

Usability Testing of an Electronic Patient-Reported Outcome System for Survivors of Critical Illness

2016 ◽  
Vol 25 (4) ◽  
pp. 340-349 ◽  
Author(s):  
C. E. Cox ◽  
N. G. Wysham ◽  
A. H. Kamal ◽  
D. M. Jones ◽  
B. Cass ◽  
...  
Keyword(s):  
Critical Illness ◽  
Usability Testing ◽  
Patient Reported Outcome ◽  
Patient Reported

scholarly journals The use of patient-reported outcome measures in primary care: applications, benefits and challenges

2021 ◽  
Vol 5 (S2) ◽  
Author(s):  
Krista Brower ◽  
Margo Schmitt-Boshnick ◽  
Michel Haener ◽  
Shea Wilks ◽  
Allison Soprovich
Keyword(s):  
Primary Care ◽  
Outcome Measurement ◽  
Local Population ◽  
Target Population ◽  
Patient Reported Outcome Measures ◽  
Patient Reported Outcome ◽  
Full Potential ◽  
System Quality ◽  
Allied Health Professionals ◽  
Patient Reported

AbstractPROMs use in primary care has expanded from simply describing patient populations to contributing to decision-making, in response to the increasingly complex, ever-changing healthcare environment. In Alberta, primary care is organized into primary care networks (PCNs), where family physicians are grouped geographically and supported by allied health professionals. PCNs implement programs and services in response to local population health needs with frequent evaluation, often incorporating PROMs for this purpose. As PCN programs and services vary greatly across Alberta, so do their use of PROMs. An area of commonality is the use of the EQ-5D-5L instrument; 29 out of 41 PCNs are registered and licensed to use the instrument. It is often administrated by paper, pre- and post-program, and in combination with other specific measures, depending on the program or target population. Some PCNs share programming and therefore outcome measurement, but often the selection, implementation (including training and administration procedures) and evaluation/reporting of PROMs are unique to the PCN. As well, data analysis is largely dependent on the size and capacity of the PCN. Using PROMs for PCN program evaluation supports clinical understanding and complements clinical outcomes. PROMs describe the population attending a program, as well as provide an element of consistency when examining trends across multiple programs or timepoints. This contributes to inquiries and decisions around program development, components, administrative features, resource allocation and delivery. Challenges of PROMs use in primary care include the absence of cohesive data capture technology. This limits data capabilities and presents difficulties with data fidelity, storage, export, and analysis. Additionally, this real-world application lacks a control arm and presents methodological challenges for comparative research purposes. Furthermore, capturing long term patient outcomes poses administrative challenges of multiple follow ups. More research is required into best reporting mechanisms to ensure the data is used to its full potential. To overcome these challenges, leadership and clinician engagement are key. As well, determining consistent PCN PROM reporting requirements will ensure data are comparable across PCNs and contribute to provincial level evaluations, further supporting the movement towards overall health system quality improvement.

scholarly journals Combining end-user recruitment methods for usability testing of eTriage Service

2019 ◽  
Vol 11 (4) ◽  
Author(s):  
Marja Helmi Ylilehto ◽  
Eini Saarivesi ◽  
Jarmo Reponen ◽  
Pia Liljamo
Keyword(s):  
Healthcare Professionals ◽  
Digital Health ◽  
Participation Rate ◽  
Usability Testing ◽  
Well Being ◽  
Active Role ◽  
End Users ◽  
Site Testing ◽  
Recruitment Methods ◽  
Online Recruiting

The Finnish national eHealth and eSocial strategy emphasizes citizens’ active role in promoting their own well-being by improving information management and implementing self-management and online services. In the Emergency Hub, part of the Finnish online health portal (Health Village), an eTriage Service is being developed by ICT experts and healthcare professionals. To make the eTriage Service available to the public, the user interface must comply with relevant quality and safety regulations. The aim of this paper is to describe the recruitment methods used for eTriage Service usability testing and the feasibility of those methods. The results of the actual usability testing are not discussed in this article. Two different recruiting methods were combined: online recruiting with remote testing and organized on-site testing occasions. A total of 219 volunteer end-users were recruited and 115 (52.5%) of them performed the usability testing. A better participation rate was achieved with organized on-site testing occasions, but the method consumed significantly more time and effort on the part of developers. A sufficient number and variety of end-users were recruited by combining different recruiting methods. Online recruiting with remote usability testing helps reduce the costs and effort of developers but may require a longer period of time to achieve a sufficient number of testers. A complex or highly novel, self-performing test process without any support might affect negatively the number of testers available by the online recruitment. It also seems that usability testing for digital health services can be more attractive to healthcare professionals than to persons with no healthcare education background.

Development of PRO-PMs assessing symptoms following completion of curative-intent chemotherapy.

2021 ◽  
Vol 39 (28_suppl) ◽  
pp. 171-171
Author(s):  
Katherine Eresian Chenok ◽  
Feifei Ye ◽  
Kristen K. McNiff Landrum ◽  
Emma Hoo ◽  
Valerie Kong ◽  
...  
Keyword(s):  
Demographic Data ◽  
Reliability And Validity ◽  
Psychometric Testing ◽  
Target Population ◽  
Patient Reported Outcome ◽  
Quality Data ◽  
Patient Centered ◽  
Curative Intent ◽  
Lung Cancers ◽  
Patient Reported

171 Background: Few patient-reported outcome performance measures (PRO-PMs) have been validated for the cancer population. The testing that has occurred mostly focuses on advanced cancers despite the fact that the majority of people with cancer are diagnosed with earlier stage disease. We developed and tested PRO-PMs to assess quality of life, pain and fatigue in adult patients completing curative-intent chemotherapy for breast, colon and non-small cell lung cancers. Our goal is to develop measures that target symptoms that impact entry into the survivorship phase. Methods: We recruited 20 diverse test sites from the Michigan Oncology Quality Collaborative (MOQC) and the Alliance of Dedicated Cancer Centers (ADCC). Test sites enrolled patients, administered surveys, and collected clinical and demographic data. A Technical Expert Panel and the Patient and Caregiver Council selected PROMs and provided testing guidance. We assessed data collection feasibility and clinician/staff/patient burden throughout the testing process. Results: PROMIS instruments were selected due to psychometric testing in the target population, public availability and acceptability to patients and test sites. 1,753 patients were enrolled between 10/1/19 – 3/31/21. The COVID public health emergency disrupted testing and resulted in lower than expected enrolled patients/completed surveys; however, adaptations led to expansions in survey administration methods. Preliminary practice-level performance results from 10 sites show variation across sites for pain interference (mean = 50.5, SD = 2.8, with a range of 44.6—54.6) and fatigue (mean 49.2, SD = 2.8, with a range of 44.6--54.3). Some test sites reported PRO implementation to be burdensome; however, most patients evaluated did not find survey completion to be burdensome. Conclusions: Next steps include testing risk adjustment variables/model, creating adjusted performance scores, reliability and validity testing. Despite the consensus goal of PRO-PMs in oncology, barriers to implementation persist and important methodologic barriers exist (e.g., ability to achieve sufficient sample size in an oncology practice; defining the most appropriate numerator calculation that reports the desired quality concept and is appropriate for the PROM being used; analytic best practices for PRO-PM adjustment/testing). This project is contributing to the knowledge base as we seek to ensure that PRO-PMs provide meaningful, actionable, patient-centered quality data with benefits that outweigh the burden of implementation.

Usability Testing: Making it Work for the Army

2009 ◽  
Vol 53 (25) ◽  
pp. 1868-1872
Author(s):  
Pamela A. Savage-Knepshield
Keyword(s):  
Resource Constraints ◽  
Usability Testing ◽  
End Users ◽  
Lessons Learned ◽  
Test Procedures ◽  
Thinking Aloud ◽  
Usability Test ◽  
Usability Tests ◽  
Hybrid Solutions ◽  
Early Formative

During an early formative usability test, representative end-users perform a set of tasks using a prototype or simulation of a system. While performing tasks and thinking aloud, a skilled specialist observes and takes notes recording how easily a participant performs the tasks and logging issues encountered. Sessions are conducted in a usability test lab, while interested parties watch from a separate observation room through a one-way mirror. Due to cost, schedule, and resource constraints many elements of conventional usability testing are not viable for use when testing equipment with Warfighters. To overcome these challenges and meet the needs of system developers, usability test procedures were modified and hybrid solutions developed. Descriptions and examples of these modifications are provided as are lessons learned and insights gained through their use during the course of seven usability tests.

scholarly journals Questionnaires in otology: a systematic mapping review

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Koen Viergever ◽  
Jeroen T. Kraak ◽  
Els. M. Bruinewoud ◽  
Johannes C. F. Ket ◽  
Sophia E. Kramer ◽  
...  
Keyword(s):  
Hearing Loss ◽  
English Language ◽  
English Literature ◽  
Adult Population ◽  
Target Population ◽  
Patient Reported Outcome Measures ◽  
Patient Reported Outcome ◽  
Comprehensive Overview ◽  
Ear Diseases ◽  
Patient Reported

Abstract Background Patient-reported outcome measures (PROMs) are valuable tools in assessing the quality of health care from a patient perspective and are increasingly used by otologists. However, selecting the right questionnaire has proven to be a difficult and time-consuming task. To facilitate this process, we will provide a comprehensive overview of existing questionnaires. Methods A systematic literature search has been conducted on August 26, 2019, using the EMBASE and PubMed medical databases. 13,345 unique records were extracted. Questionnaires addressing any otologic complaint (tinnitus, hearing loss, earache, otorrhoea, and ear-related pressure sensation, vertigo, itch, or dysgeusia) were identified. All questionnaires were evaluated for eligibility by two independent researchers. Inclusion criteria were adult population, closed-ended questions, English language of the questionnaire, and the availability of the original article describing the development of the instrument or a validation paper describing the validation process written in English. Objective Create a comprehensive overview of all validated closed-ended otology questionnaires for adults and demonstrate their basic characteristics. Main outcome measure The number of questionnaires in English literature for the adult population, subdivided per symptom and target population. Results A total of 155 unique questionnaires were selected: 33 tinnitus questionnaires, 23 vertigo questionnaires, 84 hearing loss questionnaires, and 15 multiple complaint questionnaires. A protocol for further questionnaire comparison is presented. Discussion Two separate sequential searches were needed to identify unique questionnaires and to identify their development/validation paper. Although many ear diseases create multiple symptoms, the majority of the questionnaires were symptom specific. Conclusion Many questionnaires concerning ear-related symptoms exist and predominantly concern hearing loss, vertigo, or tinnitus. Only a few questionnaires cover the multiple complaints that ear diseases can create. The presented overview is the most comprehensive overview of otology questionnaires in literature to date. It will serve as a basis for questionnaire selection by professionals and could serve as a protocol for questionnaire selection in other fields. Systematic review registration PROSPERO CRD42017058155

scholarly journals Moving from development to implementation of digital innovations within the NHS: myHealthE, a remote monitoring system for tracking patient outcomes in child and adolescent mental health services.

2021 ◽  
Author(s):  
Anna Charlotte Morris ◽  
Zina Ibrahim ◽  
Omer S Moghraby ◽  
Argyris Stringaris ◽  
Ian M Grant ◽  
...  
Keyword(s):  
Mental Health ◽  
Mental Health Services ◽  
Health Services ◽  
Digital Health ◽  
Patient Reported Outcome Measures ◽  
Patient Reported Outcome ◽  
Remote Monitoring System ◽  
Iterative Development ◽  
Patient Reported ◽  
Health Care Records

This paper reports our experience of developing, implementing, and evaluating myHealthE (MHE); a digital innovation for Child and Adolescents Mental Health Services (CAMHS) which automates the remote collection and reporting of Patient Reported Outcome Measures (PROMs) into National Health Services (NHS) electronic health care records. We describe the logistical and governance issues encountered in developing the MHE interface with patient identifiable information, and the steps taken to overcome these development barriers. We describe the applications architecture and hosting environment to enable it to be operable within the NHS, as well the as the capabilities needed within the technical team to bridge the gap between academic developers and NHS operational teams. We present evidence on the feasibility and acceptability for this system within clinical services and describe the process of iterative development, highlighting additional functions which were incorporated to increase system utility. This article provides a framework with which to plan, develop and implement automated PROM collection from remote devices back to NHS infrastructure. The challenges and solutions described in this paper will be pertinent to other digital health innovation researchers aspiring to deploy interoperable systems within NHS clinical systems.

Sign in / Sign up

Export Citation Format

Share Document