Usability testing of a Web-based solution for the collection of electronic patient-reported outcomes in people living with HIV in Nouvelle Aquitaine, France (Preprint)

BACKGROUND Collecting patient-reported outcomes can be of great value for both research and chronic diseases management. We endeavoured to develop a new facet of the ANRS CO3 Aquitaine cohort study’s web-based data capture and visualization system (APPEGE ® 2.0) for the collection of electronic patient-reported outcomes in people living with HIV care for in Aquitaine, France. OBJECTIVE Given the novelty of the proposed data collection method for our setting and specific characteristics of the target population, we sought to evaluate the initial usability of a prototype of an electronic patient-reported outcomes (ePRO) information system (ARPEGE® 2.0). METHODS Two successive rounds of empirical, task-based, usability testing were conducted, involving eight “experts” and then six people living with HIV. Evaluators provided written feedback during round 1 and oral feedback during round 2. Evaluators who completed the full set of tasks responded to the Systems Usability Scale. We assessed changes in SUS scores between rounds and concluded usability testing when SUS scores reached a ceiling effect, defining “good” usability a priori as a usability score of 70. RESULTS Insights were generated regarding the visibility of system status and the match between the system and the real world that improved the solution’s usability markedly. Experts reported mean SUS scores of 65 +- 18.87 and patients reported mean SUS scores of 85 +- 5.4 (p=0.032). CONCLUSIONS Software modifications, informed by successive rounds of usability testing, resulted in sufficient gains in usability to undertake piloting. Usability testing also prompted us to find the appropriate balance between optimal security and ease of use. CLINICALTRIAL https://clinicaltrials.gov/ct2/show/NCT03296202 (Archived by WebCite at http://www.webcitation.org/6zgOBArps)

Download Full-text

Web-Based Module for the Collection of Electronic Patient-Reported Outcomes in People Living With HIV in Nouvelle Aquitaine, France: Usability Evaluation

JMIR Formative Research ◽

10.2196/15013 ◽

2019 ◽

Vol 3 (4) ◽

pp. e15013 ◽

Cited By ~ 1

Author(s):

Diana Barger ◽

Olivier Leleux ◽

Valérie Conte ◽

Vincent Sapparrart ◽

Marie Gapillout ◽

...

Keyword(s):

Patient Reported Outcomes ◽

Usability Testing ◽

Target Population ◽

Ease Of Use ◽

People Living With Hiv ◽

Research Staff ◽

Web Based ◽

System Usability Scale ◽

Patient Reported ◽

Living With Hiv

Background Patient-reported outcomes (PROs) can be of great value for both research and chronic disease management. We developed a new module of the ANRS CO3 Aquitaine cohort study’s Web-based data capture and visualization solution (APPEGE 2.0) for the collection of electronic PROs among people living with HIV cared for in Nouvelle Aquitaine, France. Objective This study aimed to evaluate the usability of 2 successively developed prototypes of ARPEGE 2.0’s electronic PROs module before launching a pilot study, owing to the novelty of the proposed data collection method for our setting and specific characteristics of the target population. Methods A total of 2 sequential rounds of empirical, task-based usability evaluations were conducted, involving 8 research staff and then 7 people living with HIV. Evaluators provided written feedback during round 1 and oral feedback during round 2. Evaluators who completed the full set of tasks responded to the System Usability Scale (SUS). We assessed changes in SUS scores between rounds and concluded usability testing when SUS scores reached a ceiling effect, defining good usability a priori as a usability score of 70. Results Insights were generated regarding the visibility of system status and the match between the system and the real world that improved the module’s usability. Research staff evaluators reported mean SUS scores of 65 (SD 18.87) and patient evaluators reported mean SUS scores of 85 (SD 5.4; P=.032). Conclusions Software modifications, informed by successive rounds of usability testing, resulted in sufficient gains in usability to undertake piloting. Insights generated during evaluations prompted us to find the appropriate balance between optimal security and ease of use. Trial Registration ClinicalTrials.gov NCT03296202; https://clinicaltrials.gov/ct2/show/NCT03296202 International Registered Report Identifier (IRRID) RR2-10.2196/10.2196/resprot.9439

Download Full-text

Integrating Electronic Patient-Reported Outcome Measures into Routine HIV Care and the ANRS CO3 Aquitaine Cohort�s Data Capture and Visualization System (QuAliV): Protocol for a Formative Research Study (Preprint)

10.2196/preprints.9439 ◽

2017 ◽

Author(s):

Diana Barger ◽

Olivier Leleux ◽

Valérie Conte ◽

Vincent Sapparrart ◽

Marie Gapillout ◽

...

Keyword(s):

Quality Of Life ◽

Patient Reported Outcomes ◽

Patient Reported Outcome ◽

Hiv Care ◽

People Living With Hiv ◽

Web Based ◽

Visualization System ◽

Patient Reported ◽

Living With Hiv

BACKGROUND Effective antiretroviral therapy has greatly reduced HIV-related morbidity and mortality, dramatically changing the demographics of the population of people living with HIV. The majority of people living with HIV in France are well cared for insofar as their HIV infection is concerned but remain at risk for age-associated comorbidities. Their long-term, potentially complex, and growing care needs make the routine, longitudinal assessment of health-related quality of life and other patient-reported outcomes of relevance in the current treatment era. OBJECTIVE We aim to describe the development of a Web-based electronic patient-reported outcomes system for people living with HIV linked to the ANRS CO3 Aquitaine cohort’s data capture and visualization system (ARPEGE) and designed to facilitate the electronic collection of patient-reported data and ultimately promote better patient-physician communication and quality of care (both patient satisfaction and health outcomes). METHODS Participants who meet the eligibility criteria will be invited to engage with the Web-based electronic patient-reported outcomes system and provided with the information necessary to create a personal patient account. They will then be able to access the electronic patient-reported outcomes system and complete a set of standardized validated questionnaires covering health-related quality of life (World Health Organization's Quality of Life Instrument in HIV infection, named WHOQOL-HIV BREF) and other patient-reported outcomes. The information provided via questionnaires will ultimately be presented in a summary format for clinicians, together with the patient’s HIV care history. RESULTS The prototype of the Web-based electronic patient-reported outcome system will be finalized and the first 2 formative research phases of the study (prototyping and usability testing) will be conducted from December 2017 to May 2018. We describe the sequential processes planned to ensure that the proposed electronic patient-reported outcome system is ready for formal pilot testing, referred to herein as phases 1a and 1b. We also describe the planned pilot-testing designed to evaluate the acceptability and use of the system from the patient’s perspective (phase 2). CONCLUSIONS As the underlying information technology solution, ARPEGE, has being developed in-house, should the feasibility study presented here yield promising results, the panel of services provided via the proposed portal could ultimately be expanded and used to experiment with health-promoting interventions in aging people living with HIV in hospital-based care or adapted for use in other patient populations. CLINICALTRIAL ClinicalTrials.gov NCT03296202; https://clinicaltrials.gov/ct2/show/NCT03296202 (Archived by WebCite at http://www.webcitation.org/6zgOBArps) REGISTERED REPORT IDENTIFIER RR1-10.2196/9439

Download Full-text

Usability testing of the PRO-CTCAE measurement system in patients with cancer.

Journal of Clinical Oncology ◽

10.1200/jco.2013.31.15_suppl.e17560 ◽

2013 ◽

Vol 31 (15_suppl) ◽

pp. e17560-e17560 ◽

Cited By ~ 2

Author(s):

Maria R. Fawzy ◽

Amy Pickar Abernethy ◽

Martin W. Schoen ◽

Lauren J. Rogak ◽

Tito R. Mendoza ◽

...

Keyword(s):

Adverse Events ◽

Measurement System ◽

Patient Reported Outcomes ◽

Interactive Voice Response ◽

Usability Testing ◽

Ease Of Use ◽

Cooperative Groups ◽

Web Based ◽

Oncology Research ◽

Patient Reported

e17560 Background: Use of electronic Patient-Reported Outcomes (ePROs) are increasingly employed in patient care and oncology research. However, these systems can be difficult to use, preventing adoption and deployment. The objective is to evaluate the patient usability of the National Cancer Institute’s (NCI) Patient Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) Measurement System. Methods: PRO-CTCAE is an item bank of 124 questions evaluating 78 symptomatic adverse events, as well as a software system that includes a web-based and interactive voice response system [IVRS] to administer these items to patients, and electronic features to customize survey creation, schedule survey administration, export data, and generate alerts and reports. A protocol was developed for two rounds of usability testing to evaluate understanding and ease of navigation reported by patients undergoing cancer treatment. Software was refined between rounds and retested. Patients were asked to login and complete PRO-CTCAE items either in clinic or at home, then were immediately debriefed (semi-scripted) by a trained interviewer. Results: 173 patients at 3 cancer centers and affiliated community network sites were enrolled in the study between 1/11-5/12 (45% male; 32% non-white; 26% high school or less). Usability issues identified in Round 1 (N=37) included difficulty in using radio buttons, absence of survey progress indicators, and problems with system login. Modifications included increasing size of buttons, improving the progress bar feature, and simplifying the interface to focus users on survey questions. In Round 2 (N=136), remaining issues were related to general difficulty with Internet use, such as navigating to the URL. IVRS testing indicated high scores of ease of use and completion of items, but users had difficulty adding unique symptoms to the system. Conclusions: Iterative usability testing identified improvements to web-based and IVRS for patient self-reporting and achieved a more favorable user experience. The feasibility of integrating PRO-CTCAE into trials to gather symptomatic adverse events directly from patients is currently being evaluated in the cancer cooperative groups.

Download Full-text

Assessing the Content Validity of a New Patient-Reported Measure of Barriers to Antiretroviral Therapy Adherence for Electronic Administration in Routine HIV Care: Proposal for a Web-Based Delphi Study (Preprint)

10.2196/preprints.12836 ◽

2018 ◽

Author(s):

Kim Engler ◽

Sara Ahmed ◽

David Lessard ◽

Serge Vicente ◽

Bertrand Lebouché

Keyword(s):

Antiretroviral Therapy ◽

Hiv Care ◽

People Living With Hiv ◽

Free Text ◽

Art Adherence ◽

Web Based ◽

Specific Patient ◽

Patient Reported ◽

Electronic Administration ◽

Oriented Research

BACKGROUND Adherence to lifesaving antiretroviral therapy (ART) for HIV infection remains a challenge for many patients. Routine screening for barriers to ART adherence could help make HIV care more patient-centered and prevent virologic rebound or failure. Our team is currently developing a new HIV-specific patient-reported outcome measure (PROM) of these barriers for use in Canada and France along with a digital app for its electronic administration. In our previous work, we developed the PROM’s multidimensional conceptual framework and generated 100 English items, which have been translated to French. OBJECTIVE This study aims to use a Web-based Delphi to help validate and select the content of this new HIV-specific PROM, based on the perspective of anglophone and francophone patients and providers in Canada and France. Here, we present the proposal for this Delphi. METHODS This modified Delphi will involve a diverse panel of patients (n=32) and providers (n=52) recruited especially from the 9 sites of the PROM development study (site locations in Canada: Montreal, Toronto, Vancouver; in France: Paris, Nantes, Clermont-Ferrand, Saint-Martin, Cayenne). Overall, 2 rounds of Web-based questionnaires will be conducted. The threshold for consensus is set at 60% and will determine which items are carried forward to the second round. Per item, 3 aspects will be rated: importance as a barrier to ART adherence, relevance for HIV care, and clarity. In both rounds, space will be available for free text comments. Overall comprehensiveness will be assessed in the second round. RESULTS This study has undergone a methodological review by experts in patient-oriented research. It has received approval from a research ethics board of the McGill University Health Centre. It is financially supported, in part, by the Canadian Institutes of Health Research’s Strategy for Patient-Oriented Research-Quebec Support Unit (M006). As of May 21, 2019, 15 people living with HIV and 25 providers completed the first round of the Delphi (24 from Canada and 16 from France). CONCLUSIONS To our knowledge, this is the first Delphi to seek consensus on the most relevant and clinically actionable barriers to ART adherence, collecting opinions on an extensive list of barriers. Drawing on a relatively large and diverse panel of HIV patients and providers, it essentially engages key stakeholders in decision making about the PROM’s final content, helping to ensure its utility and adoption. INTERNATIONAL REGISTERED REPOR PRR1-10.2196/12836

Download Full-text

1259. Household Income and Its Relationship with Patient-Reported Outcomes Among Older People Living with HIV

Open Forum Infectious Diseases ◽

10.1093/ofid/ofz360.1122 ◽

2019 ◽

Vol 6 (Supplement_2) ◽

pp. S453-S454

Author(s):

Peter Mazonson ◽

Jeff Berko ◽

Theoren Loo ◽

Lynsay MacLaren ◽

Erik S Lowman

Keyword(s):

Health Outcomes ◽

Household Income ◽

Patient Reported Outcomes ◽

Income Group ◽

People Living With Hiv ◽

Annual Household Income ◽

Patient Reported ◽

Part Time ◽

Living With Hiv ◽

The Relationship

Abstract Background Socioeconomic factors have been identified as a root cause of a wide range of health outcomes. However, there are no studies that describe the impact of these factors on patient-reported outcomes (PROs) among older (age 50+) people living with HIV (PLWH). This study examines the relationship between annual household income, sociodemographic factors, and several PROs among older PLWH. Methods A cross-sectional analysis examined the relationships between self-reported annual household income, sociodemographic information, and validated PROs. Statistical differences within sociodemographic groups were determined using chi-squared tests, and within PROs using bivariate risk ratios. Results Of 922 participants, the median age was 58 years (range: 50–88). The majority of participants were male (89%), gay (79%), and white (70%). Fifty-five percent reported an annual household income of less than $50,000 per year and 45% reported a household income of $50,000 or greater. Among people in the lower-income group, 33% were working full or part time, 24% were retired, and 43% were disabled, whereas among people in the higher income group, 76% were working full or part time, 19% were retired, and 5% were disabled (Table 1). Bivariate analysis showed that while there was not a significant relationship between age and income, income was significantly associated with work status, race, gender, education, relationship status, sexual orientation, and having enough money to meet basic needs. People with lower household income were significantly more likely to be depressed, anxious, and lonely, and to have 4 or more comorbid conditions (Table 2). They were also less likely to have high resilience, high social well-being, and high quality of life. Conclusion To the best of our knowledge, this is the first examination of the relationship between self-reported annual household income and PROs among older PLWH. In these bivariate analyses, income was positively associated with desirable PROs, and negatively associated with undesirable PROs. To be successful, programs designed to improve health outcomes for older PLWH must take into account the economic challenges faced by many in this group. Disclosures All authors: No reported disclosures.

Download Full-text

AO Patient Outcomes Center: Design, Implementation, and Evaluation of a Software Application for the Collection of Patient-Reported Outcome Measures in Orthopedic Outpatient Clinics (Preprint)

10.2196/preprints.10880 ◽

2018 ◽

Author(s):

Nan E Rothrock ◽

Michael Bass ◽

Andrea Blumenthal ◽

Richard C Gershon ◽

Beate Hanson ◽

...

Keyword(s):

Patient Outcomes ◽

User Satisfaction ◽

Patient Reported Outcomes ◽

Clinical Care ◽

Patient Reported Outcome ◽

Ease Of Use ◽

Measurement Information ◽

Web Based ◽

Software Application ◽

Patient Reported

BACKGROUND Patient-reported outcomes are increasingly utilized in routine orthopedic clinical care. Computer adaptive tests (CATs) from the Patient-Reported Outcomes Measurement Information System (PROMIS) offer a brief and precise assessment that is well suited for collection within busy clinical environments. However, software apps that support the administration and scoring of CATs, provide immediate access to patient-reported outcome (PRO) scores, and minimize clinician burden are not widely available. OBJECTIVE Our objective was to design, implement, and test the feasibility and usability of a Web-based system for collecting CATs in orthopedic clinics. METHODS AO Patient Outcomes Center (AOPOC) was subjected to 2 rounds of testing. Alpha testing was conducted in 3 orthopedic clinics to evaluate ease of use and feasibility of integration in clinics. Patients completed an assessment of PROMIS CATs and a usability survey. Clinicians participated in a brief semistructured interview. Beta-phase testing evaluated system performance through load testing and usability of the updated version of AOPOC. In both rounds of testing, user satisfaction, bugs, change requests, and performance of PROMIS CATs were captured. RESULTS Patient feedback supported the ease of use in completing an assessment in AOPOC. Across both phases of testing, clinicians rated AOPOC as easy to use but noted difficulties in integrating a Web-based software application within their clinics. PROMIS CATs performed well; the default assessment of 2 CATs was completed quickly (mean 9.5 items) with a satisfactory range of measurement. CONCLUSION AOPOC was demonstrated to be an easy-to-learn and easy-to-use software application for patients and clinicians that can be integrated into orthopedic clinical care. The workflow disruption in integrating any type of PRO collection must be addressed if patients’ voices are to be better integrated in clinical care.

Download Full-text

Assessing the Content Validity of a New Patient-Reported Measure of Barriers to Antiretroviral Therapy Adherence for Electronic Administration in Routine HIV Care: Proposal for a Web-Based Delphi Study

JMIR Research Protocols ◽

10.2196/12836 ◽

2019 ◽

Vol 8 (8) ◽

pp. e12836 ◽

Cited By ~ 1

Author(s):

Kim Engler ◽

Sara Ahmed ◽

David Lessard ◽

Serge Vicente ◽

Bertrand Lebouché

Keyword(s):

Antiretroviral Therapy ◽

Hiv Care ◽

People Living With Hiv ◽

Free Text ◽

Art Adherence ◽

Web Based ◽

Specific Patient ◽

Patient Reported ◽

Electronic Administration ◽

Oriented Research

Background Adherence to lifesaving antiretroviral therapy (ART) for HIV infection remains a challenge for many patients. Routine screening for barriers to ART adherence could help make HIV care more patient-centered and prevent virologic rebound or failure. Our team is currently developing a new HIV-specific patient-reported outcome measure (PROM) of these barriers for use in Canada and France along with a digital app for its electronic administration. In our previous work, we developed the PROM’s multidimensional conceptual framework and generated 100 English items, which have been translated to French. Objective This study aims to use a Web-based Delphi to help validate and select the content of this new HIV-specific PROM, based on the perspective of anglophone and francophone patients and providers in Canada and France. Here, we present the proposal for this Delphi. Methods This modified Delphi will involve a diverse panel of patients (n=32) and providers (n=52) recruited especially from the 9 sites of the PROM development study (site locations in Canada: Montreal, Toronto, Vancouver; in France: Paris, Nantes, Clermont-Ferrand, Saint-Martin, Cayenne). Overall, 2 rounds of Web-based questionnaires will be conducted. The threshold for consensus is set at 60% and will determine which items are carried forward to the second round. Per item, 3 aspects will be rated: importance as a barrier to ART adherence, relevance for HIV care, and clarity. In both rounds, space will be available for free text comments. Overall comprehensiveness will be assessed in the second round. Results This study has undergone a methodological review by experts in patient-oriented research. It has received approval from a research ethics board of the McGill University Health Centre. It is financially supported, in part, by the Canadian Institutes of Health Research’s Strategy for Patient-Oriented Research-Quebec Support Unit (M006). As of May 21, 2019, 15 people living with HIV and 25 providers completed the first round of the Delphi (24 from Canada and 16 from France). Conclusions To our knowledge, this is the first Delphi to seek consensus on the most relevant and clinically actionable barriers to ART adherence, collecting opinions on an extensive list of barriers. Drawing on a relatively large and diverse panel of HIV patients and providers, it essentially engages key stakeholders in decision making about the PROM’s final content, helping to ensure its utility and adoption. International Registered Report Identifier (IRRID) PRR1-10.2196/12836

Download Full-text

Comparison of Clinical Outcomes of Persons Living With HIV by Enrollment Status in Washington, DC: Evaluation of a Large Longitudinal HIV Cohort Study (Preprint)

10.2196/preprints.16061 ◽

2019 ◽

Author(s):

Jenevieve Opoku ◽

Rupali K Doshi ◽

Amanda D Castel ◽

Ian Sorensen ◽

Michael Horberg ◽

...

Keyword(s):

Cohort Study ◽

Health Outcomes ◽

Odds Ratio ◽

Adjusted Odds Ratio ◽

Hiv Care ◽

Disease Stage ◽

People Living With Hiv ◽

Hiv Disease ◽

Persons Living With Hiv ◽

Living With Hiv

BACKGROUND HIV cohort studies have been used to assess health outcomes and inform the care and treatment of people living with HIV disease. However, there may be similarities and differences between cohort participants and the general population from which they are drawn. OBJECTIVE The objective of this analysis was to compare people living with HIV who have and have not been enrolled in the DC Cohort study and assess whether participants are a representative citywide sample of people living with HIV in the District of Columbia (DC). METHODS Data from the DC Health (DCDOH) HIV surveillance system and the DC Cohort study were matched to identify people living with HIV who were DC residents and had consented for the study by the end of 2016. Analysis was performed to identify differences between DC Cohort and noncohort participants by demographics and comorbid conditions. HIV disease stage, receipt of care, and viral suppression were evaluated. Adjusted logistic regression assessed correlates of health outcomes between the two groups. RESULTS There were 12,964 known people living with HIV in DC at the end of 2016, of which 40.1% were DC Cohort participants. Compared with nonparticipants, participants were less likely to be male (68.0% vs 74.9%, P<.001) but more likely to be black (82.3% vs 69.5%, P<.001) and have a heterosexual contact HIV transmission risk (30.3% vs 25.9%, P<.001). DC Cohort participants were also more likely to have ever been diagnosed with stage 3 HIV disease (59.6% vs 47.0%, P<.001), have a CD4 <200 cells/µL in 2017 (6.2% vs 4.6%, P<.001), be retained in any HIV care in 2017 (72.9% vs 59.4%, P<.001), and be virally suppressed in 2017. After adjusting for demographics, DC Cohort participants were significantly more likely to have received care in 2017 (adjusted odds ratio 1.8, 95% CI 1.70-2.00) and to have ever been virally suppressed (adjusted odds ratio 1.3, 95% CI 1.20-1.40). CONCLUSIONS These data have important implications when assessing the representativeness of patients enrolled in clinic-based cohorts compared with the DC-area general HIV population. As participants continue to enroll in the DC Cohort study, ongoing assessment of representativeness will be required.

Download Full-text

Depressive Symptoms, Gender, Disclosure, and HIV Care Stage Among People Living with HIV in Cameroon

AIDS and Behavior ◽

10.1007/s10461-021-03425-3 ◽

2021 ◽

Author(s):

Angela M. Parcesepe ◽

Molly Remch ◽

Anastase Dzudie ◽

Rogers Ajeh ◽

Denis Nash ◽

...

Keyword(s):

Depressive Symptoms ◽

Hiv Care ◽

People Living With Hiv ◽

Living With Hiv

Download Full-text

The Utilisation of Payment Models Across the HIV Continuum of Care: Systematic Review of Evidence

AIDS and Behavior ◽

10.1007/s10461-021-03329-2 ◽

2021 ◽

Author(s):

Tiago Rua ◽

Daniela Brandão ◽

Vanessa Nicolau ◽

Ana Escoval

Keyword(s):

Positive Impact ◽

Cost Effective ◽

Relative Magnitude ◽

Hiv Care ◽

People Living With Hiv ◽

Hiv Continuum Of Care ◽

Clinical Conditions ◽

Study Designs ◽

Living With Hiv ◽

Payment Models

AbstractThe increasing chronicity and multimorbidities associated with people living with HIV have posed important challenges to health systems across the world. In this context, payment models hold the potential to improve care across a spectrum of clinical conditions. This study aims to systematically review the evidence of HIV performance-based payments models. Literature searches were conducted in March 2020 using multiple databases and manual searches of relevant papers. Papers were limited to any study design that considers the real-world utilisation of performance-based payment models applied to the HIV domain. A total of 23 full-text papers were included. Due to the heterogeneity of study designs, the multiple types of interventions and its implementation across distinct areas of HIV care, direct comparisons between studies were deemed unsuitable. Most evidence focused on healthcare users (83%), seeking to directly affect patients' behaviour based on principles of behavioural economics. Despite the variability between interventions, the implementation of performance-based payment models led to either a neutral or positive impact throughout the HIV care continuum. Moreover, this improvement was likely to be cost-effective or, at least, did not compromise the healthcare system’s financial sustainability. However, more research is needed to assess the durability of incentives and its appropriate relative magnitude.

Download Full-text