Validation of a real-time patient-reported experience measurement tool for cancer patients in Ontario.

2016 ◽  
Vol 34 (7_suppl) ◽  
pp. 174-174
Author(s):  
Lesley Moody ◽  
Sarah Benn ◽  
Luciano Ieraci ◽  
Saurabh Ingale ◽  
Simron Singh

174 Background: Cancer Care Ontario (CCO) organizes and ensures quality cancer care for 16 million residents in Ontario. CCO's goal is to create a tool to enable reporting of the patient experience in real time. The study objective was to develop a validated real-time (close to or at point of care) patient reported experience measurement (PREM) tool to drive quality improvement, advance system planning and ensure cost effective resource allocation based on patients’ needs, wants and preferences in ambulatory oncology treatment phase. Methods: Consisting of 28 questions – 18 core questions; 2 open-ended responses; and 8 demographic questions the tool underwent cognitive testing with patient family advisors and was validated using a pilot at 6 cancer centres from April to August, 2015. It was administered using several delivery modes: tablets, patient portals, post cards and paper. The patient journey in visiting centres was conceptualized as containing the steps or domains of interest: arrival, appointment/consultation and overall experience. To validate the patient experience conceptual model, an exploratory factor analysis was performed and compared to CCO’s 8 domain patient experience model. Reliability and validity of the tool was tested using Cronbach’s alpha and multiple variable regression analysis in partnership with Ipsos Reid. Results: 557 responses (496 electronic and 61 paper) formed the basis of the analysis. The tool was shown to be both valid and reliable - an overall adjusted R2 value of 64% from multiple variable regression on overall patient experience; and an overall Cronbach alpha value of 0.83, respectively. Individual Cronbach alpha values ranged from: 0.69 for “appointment/consultation”, 0.55 for “overall experience” and 0.24 for patient “arrival.” Conclusions: Statistical analysis results were used to produce a final validated tool for deployment to systematically capture PREMs through an electronic system in Ontario, Canada in 2016.To our knowledge this is the first of its kind. Collection of real time PREMS will ensure local and system level planning and quality improvement in the cancer system are rooted in what matters most to patients and their families.

2014 ◽  
Vol 32 (30_suppl) ◽  
pp. 240-240
Author(s):  
Nancy Kraetschmer ◽  
Alysha Glazer ◽  
Esther Green ◽  
Laura Macdougall ◽  
Simron Singh

240 Background: Cancer Care Ontario (CCO) is an Ontario government agency that drives system wide quality improvement in disease prevention and screening, the delivery of care and the patient experience (PE), for cancer, chronic kidney disease and access to care for key health services. CCO partners with 14 Regional Cancer Programs across the province. CCO measures the cancer PE retrospectively through a paper-based Ambulatory Oncology Patient Satisfaction Survey but there are limitations such as the delay in reporting of results. CCO is developing and implementing a province-wide electronic real-time PE survey to measure PE. An electronic survey tool will enable the collection, analysis and reporting of PE data more quickly to drive quality improvement. Methods: A current state assessment was undertaken in fall 2013 to understand support for and readiness of CCO’s Regional Cancer Centres (RCCs) to move forward with measuring PE in real-time, at point of care. 14 semi-structured interviews were conducted with RCCs Regional Vice Presidents (RVP); over 800 patients participated in focus groups in 11 of 14 regions; a targeted online survey was completed by 16 administrators in 14 regions. Results: Interest in a real-time measurement (RTM) approach was overwhelmingly positive – 94% of patients and 100% of RVPs thought that measuring PE in real-time is important. Patients want the real-time survey to be short - 61% of patients reported that the number of questions they are willing to answer is 5-10, with 18% indicating 15 questions. 62% of patients reported that they were willing to spend 2-6 minutes answering a survey, with 24% reporting 10 minutes. 36% of the RCCs have ad hoc RTM strategies and 21% have utilized an electronic tool. There was no co-ordination or data sharing between RCPs. Conclusions: CCO is implementing a province-wide RTM strategy which includes the validation of a RTM PE survey and procurement of an electronic solution to capture, analyze and report PE data. CCO will initially pilot the survey and electronic tool with the intention of a provincial rollout to cancer centres upon a post-pilot evaluation.


2017 ◽  
Vol 35 (8_suppl) ◽  
pp. 111-111
Author(s):  
Lesley Moody ◽  
Suman Dhanju ◽  
Sarah Benn ◽  
Isabela Olaru ◽  
Nadia Sliz ◽  
...  

111 Background: Cancer Care Ontario organizes and ensures quality cancer care for 13.5 million residents in Ontario, Canada and is systematically deploying a web-based tool, called Electronic Patient Reported Experience Measures (ePREM), through a touch-screen platform previously deployed within Regional Cancer Centres (RCCs). ePREM has been used to disseminate the first patient reported experience measure (PREM), called Your Voice Matters (YVM). YVM will be collected on all patients in Ontario undergoing cancer treatment in real time, creating the largest known, linkable patient experience dataset. Methods: Implementation feasibility and readiness was assessed across all 14 Regional Cancer Programs through a provincial assessment which informed the phased implementation. Implementation was rooted in a change management framework: clinical and administrative champions, available resources, existing technical environment and competing program priorities. In March 2016 a four-wave implementation was initiated controlling for readiness and centre volumes, with a focus on extensive stakeholder engagement, tool training, launch and operational support. To date, ePREM is fully operational in 10 of 15 RCCs. Results: Provincial deployment was 66% complete in October 2016. By March 2017 full provincial implementation will be complete. Successful implementation has been directly linked to an adaptive design within the implementation and change management framework as a four-wave roll out. Key factors included: multi-faceted communications with centre leadership, and implementation teams. During this initial data stabilization phase, there were 9,266 completed surveys submitted and 10,932 partial survey responses submitted. Conclusions: PREMs are appropriate quality improvement indicators for cancer patients within the treatment phase. ePREMs enables systematic, linkable collection of real time PREMs to improve the patient experience in Ontario and contribute to the planning of new initiatives. Upon full provincial implementation ePREMs will allow for the largest known dataset of PREMs. Successful implementation has been directly linked to an adaptive, four-wave approach.


2013 ◽  
Vol 31 (31_suppl) ◽  
pp. 60-60
Author(s):  
Donna Chung ◽  
Aniruddha Dwarakanath ◽  
Charlotte Williams ◽  
Kathy Pritchard-Jones ◽  
James Mountford ◽  
...  

60 Background: London Cancer aims to use transparency of service and quality measures to drive improvement in cancer care in North and East London and surrounding areas, serving our population of 3.5m people. Whilst the on-going implementation of the National Outcome and Service Dataset for UK is expected to take 18 months we have chosen to develop quality measures with the teams accountable to deliver the service by using available data from a variety of existent sources and illustrate this in value scorecards tracking the patient pathway. Methods: Building on an engagement exercise with patients, clinicians and charities in 2011 to identify which outcomes mattered most to patients, a small set of key pathways metrics was identified for each site specific cancer pathway board to monitor their progress in implementing integrated cancer care. Metrics were selected only if (a) clinically useful, in line with the current work plan and improvement effort; (b) accessible on a recurrent basis and requiring minimal manual effort; (c) facilitate the understanding of the patient pathway; (d) align with London Cancer objectives in improving survival, patient experience, and access to innovation and clinical trials. Pathway boards were invited to contribute with the intention to provide a pathway metrics value scorecards on a quarterly basis. Results: The first set pathway metrics scorecards, with 31 metrics, were published by June 2013. Key items include adherence to established best practice (16), data completeness (6), survival (3), pathway efficiency (3), and patient experience (3). Conclusions: Pathway metrics are reported at a system level, reflecting the care for our local population, against measures that are important to them and will allow visibility of success. Whilst current pathway metric development is limited by the availability of meaningful data we aim to build on the existing metrics in an iterative fashion. For this we are working with stakeholders to improve data quality.


2020 ◽  
Vol 38 (4_suppl) ◽  
pp. 33-33 ◽  
Author(s):  
Rebecca A Snyder ◽  
Rebecca Wardrop ◽  
Alexander Mclain ◽  
Alexander A. Parikh ◽  
Anna Cass

33 Background: Although studies have identified demographic and clinical factors associated with quality colorectal cancer care, the association between patient-reported experience of care and quality of care is unknown. Our primary aim was to assess the relationship between patient-reported experience of care and receipt of guideline-concordant colon cancer (CC) treatment. Methods: Fee-For-Service Medicare beneficiaries with resected stage I-III CC (2003-2013) were identified in the linked SEER registry and Consumer Assessment of Healthcare Providers and Systems patient experience survey (SEER-CAHPS) dataset. Patient-reported ratings were compared based on receipt of care consistent with recommended treatment guidelines [resection of ≥ 12 lymph nodes (LN) (stage I-III) and receipt of adjuvant chemotherapy (stage III)]. Linear regression was performed to compare mean patient experience scores by receipt of guideline concordant care, adjusting for patient and hospital factors. Results: 1010 patients with stage I-III CC were identified (mean age 76.7, SE 6.9). Of these, 58.4% of stage I (n = 192/329) and 73.4% of stage II (n = 298/406) patients underwent resection of ≥ 12 LN. Among stage III patients, 76.0% (n = 209/275) underwent resection of ≥ 12 LN and 52.4% (n = 144/275) received adjuvant chemotherapy. By multivariable analysis, patient-reported ratings of health care quality, personal and specialty physicians, customer service, physician communication, getting needed care, and getting care quickly were similar among patients who received guideline-concordant treatment compared to those who did not. However, mean ratings of overall health care quality [91.3 (SE 2.0) vs. 82.4 (SE 1.7), p = 0.0004] and getting needed care [92.8 (SE 2.4) vs. 86.8 (SE 2.0), p = 0.047] were higher among stage III patients who received guideline concordant care compared to those who did not. Conclusions: Patient-reported ratings of health care quality and ability to get needed care are associated with guideline concordant cancer care among elderly patients with stage III CC. Further investigation is needed to determine if patient-reported experience correlates with other clinical measures of quality of colorectal cancer care.


Author(s):  
Kathi Mooney ◽  
Donna L. Berry ◽  
Meagan Whisenant ◽  
Daniel Sjoberg

Poorly controlled symptoms are common and debilitating during cancer treatment and can affect functional status and quality of life, health care resource utilization, treatment adherence, and cancer survivorship. Historically, the patient experience, including symptoms during treatment, has not been tracked or documented in the patient health record. Measurement of patient-reported outcomes (PROs), including symptoms, is an essential component to cancer care focused on the illness impact to the patient and family. PROs can be useful at the individual level for monitoring and promoting symptom care both in the clinic and remotely and at the population level for aggregating population data for use in research and quality improvement initiatives. Implementation of PROs in cancer clinical care requires a carefully thought out process to overcome challenges related to integrating PROs into existing electronic health records and clinical work flow. Issues with implementing PRO collection may include making decisions about measurement tools, modes of delivery, frequency of measurement, and interpretation that are guided by a clarification of the purpose for collecting PROs. We focus on three aspects of PRO use: (1) improving care for individual patients, (2) analyzing aggregated data to improve care and outcomes overall, and (3) considerations in implementing PRO collection.


2014 ◽  
Vol 32 (30_suppl) ◽  
pp. 260-260
Author(s):  
Izumi Kamiya ◽  
Ayako Okuyama ◽  
Kayoko Katayama ◽  
Natsumi Yamashita ◽  
Keizo Akuta ◽  
...  

260 Background: Patient-reported experiences of cancer care are an important outcome of cancer control programs. To establish a nation-wide system to monitor progress in cancer control policies, we piloted a patient experience survey to six hospitals in Japan. Methods: We conducted a self-administered questionnaire survey to a total of 1,804 adult cancer patients receiving cancer treatment in six hospitals (three cancer centers, two general hospitals, and one academic institution) from July 2013 to Mar 2014. Patients were asked to answer 94 questions covering eight dimensions of cancer experience: 1) decision-making, 2) care coordination, 3) patient education, 4) pain control, 5) emotional support, 6) family support, 7) access to care, and 8) care continuity. Results: Eighty percent of the patients reported that their treatment preferences were respected in the decision-making process, but a large proportion of patients (60%) also noted that they preferred to have their treatment decisions made for them by their physicians. Many (32%) expressed difficulty in communicating their questions and concerns to their physicians at the time of diagnosis. Only one fifth of patients were informed at the time of diagnosis that they can seek for a second opinion from other providers. Average patient-reported wait time to surgery was 30 days, which was considered to be long by a third of the patients. Eighty percent of patients felt that their care was well-coordinated by a multidisciplinary team, while % also felt that they received adequate emotional support from their medical staff. Relatively small proportion of outpatients (77%) felt that they had access to medical staff when they had medical questions, compared to nearly all patients in an inpatient setting. Only 65% of inpatients and 40% of outpatients felt that they had received best available pain control during their care. Less than half of the patients were able to communicate their preferred place of care after discharge with their healthcare provider. Conclusions: Patient-reported experiences of cancer care are an important outcome measure of cancer policy performance. This pilot study served to reveal some of the important on in future nationwide surveys.


2017 ◽  
Vol 35 (8_suppl) ◽  
pp. 182-182 ◽  
Author(s):  
Doris Howell ◽  
Madeline Li ◽  
Zeev Rosberger ◽  
Nicole Montgomery ◽  
Carole Mayer ◽  
...  

182 Background: The Improving Patient Experience and Health Outcomes Collaborative (iPEHOC) aims to improve health outcomes through uptake of electronic patient reported outcome measures (e-PROMs) in oncology practices in Ontario and Quebec. Building on screening with the Edmonton Symptom Assessment System (ESAS), e-PROMs were triggered based on cut scores to focus multidimensional assessment and management of pain (BPI), fatigue (CFS), anxiety (GAD-7) and depression (PHQ-9). Methods: Multifaceted implementation strategies and practice change coaching facilitated the use of e-PROMs to improve symptom outcomes. A mixed-method, pre-post quasi-experimental design assessed process and impact of the intervention on symptom screening rates, symptom burden, patient experience and activation, clinician satisfaction, team collaboration and health care use. Mann-Whitney U statistics examined significance of change from baseline to the 8-month post comparison. Qualitative data explored uptake of e-PROMs in practice. Results: Over the implementation period 10,248 ESAS screens were completed in iPEHOC clinics; 17.5% triggered an additional e-PROM. A slight improvement was noted in person-centeredness of communication (mean change of 1.43 to 1.37; four-point scale of 1 = very satisfied, 4 = very dissatisfied) and in team collaboration. A significant increase in patient activation levels (p = 0.045) was related to decreased emergency department visits (2% pre/post change, p = 0.81) and hospitalization within 30 days of an e-PROMs completion (2.2% change, p = 0.034) in Ontario. Patients and clinicians perceived e-PROMs as valuable to focus communication in the clinic visit and for shared treatment planning. Focus group data suggests that patients use e-PROMs as a ‘self-check-in’, to communicate their symptoms and normalize disclosure of depression in clinical care. Conclusions: Uptake of e-PROMs in diverse settings is complex and demanding. Improving symptom management quality requires PROM data to be fed-back for ‘real-time’ use in the clinical encounter and practice change facilitation for meaningful use in routine care.


Blood ◽  
2020 ◽  
Vol 136 (Supplement 1) ◽  
pp. 35-36
Author(s):  
Amit Sanyal ◽  
James M. Heun ◽  
Jessica Sweeney ◽  
Clemens Janssen

INTRODUCTION Adverse effects are common during treatment of hematological malignancies. Treatment toxicities can impact quality of life [1], impose financial hardship and cause cancer related distress[2]. Symptom monitoring using electronic technology can facilitate early detection of complications[3], reduce symptom burden[4], cost of care[5] and improve survival[6]. Cancer treatment also increases risk of mortality from infections such as coronavirus disease 2019 (COVID-19) and routine screening has been recommended[7]. METHODS We developed an application that periodically delivers toxicity questionnaires to patients during treatment . Based on NCI- PRO-CTCAE™, the questions are delivered through SMS or e-mail. Patient responses crossing prespecified thresholds trigger automated alerts on a dashboard, resulting in additional interventions as needed. Nature and time to intervention is tracked. Patient experience is measured using a Likert-scale and free-text box. Centers for Disease Control recommended COVID-19 screening questions were incorporated. Finally, a distress thermometer for cancer distress screening has been recently added. The app was offered to patients with hematological cancers in a community-based cancer center. RESULTS Since introduction in April 2020, we have enrolled 37 patients. 9 patients had chronic lymphocytic leukemia, 6 diffuse large B cell, 5 mantle cell, 4 Hodgkin's and 3 follicular lymphoma. 2 each had chronic myelogenous, multiple myeloma and Richter's syndrome. 1 each had hairy cell leukemia, acute myelogenous leukemia and T Cell lymphoma. Median age was 64 years (range 24-85). Patient experience has been favorable. On a scale of 1-5, 85.5% rated the experience as 3 or higher. Median patient engagement, calculated by dividing the number of forms completions by number of days enrolled was 34.2% (0.9-66.2 %). Symptom tracker captured 536 responses. Fatigue (153), no symptoms (152), shortness of breath (57), nausea/vomiting, diarrhea (46) and numbness/tingling (28) were the most common response categories. Of 1107 completed check ins, 75 triggered flags. There were 2 hospitalizations for neutropenic fever with the remainder managed as outpatients. Average time between patient generated response and provider intervention was 90.9 minutes. 88% follow-ups were completed within 1 business day. COVID-19 screening module captured 1096 responses. 988 were no symptoms. All positive responses (44 diarrhea, 39 cough, 23 shortness of breath and 2 fever) were false positives. Distress thermometer implemented a week before data cut-off captured 2 responses, 1 in the physical and 1 in the psychological domain. CONCLUSION We demonstrate feasibility of electronic capture of treatment toxicities and offer proof of concept that a mobile app can be used for infection screening. Additionally, the quick response time by care team indicated a high adoption rate. REFERENCES Doorduijn J, B.I., Holt B, Steijaert M, Uyl-de Groot C, Sonneveld P., Self-reported quality of life in elderly patients with aggressive non-Hodgkin's lymphoma treated with CHOP chemotherapy. . European Journal of Hemtology 2005. 75(2): p. 116-123.Troy JD, L.S., Samsa GP, Feliciano J, Richhariya A, LeBlanc TW., Patient-reported distress in Hodgkin lymphoma across the survivorship continuum. Supportive Care Cancer, 2019. 27(7): p. 2453-2462.Stover A M, H.S., Deal A M, Stricker C T, Bennett A V, Carr P M, Jansen J, Kottschade L A, Dueck A C, Basch E M, Methods for alerting clinicians to concerning symptom questionnaire responses during cancer care: Approaches from two randomized trials (STAR, AFT-39 PRO-TECT). Journal of Clinical Oncology 2018. 36(30 supplement): p. 158.Mooney KH, B.S., Wong B, Whisenant M, Donaldson G, Automated home monitoring and management of patient-reported symptoms during chemotherapy: results of the symptom care at home RCT. Cancer Medicine, 2017. 6(3): p. 537-546.Barkley R, S.M.-J., Wang J, Blau S, Page RD, Reducing Cancer Costs Through Symptom Management and Triage Pathways. Journal of Oncology Practice, 2019. 15(2): p. e91-e97.Denis F, B.E., Septans AL, Urban T, Dueck AC, Letellier C., Two-Year Survival Comparing Web-Based Symptom Monitoring vs Routine Surveillance Following Treatment for Lung Cancer. JAMA, 2019. 321(3): p. 306-307.ASCO Special Report: A guide to cancer care delivery during COVID-19 pandemic. 2020, ASCO: Alexandria, VA. Disclosures Janssen: wellbe Inc.: Current Employment.


2017 ◽  
Vol 35 (8_suppl) ◽  
pp. 225-225
Author(s):  
Erica Bridge ◽  
Lesley Gotlib Conn ◽  
Suman Dhanju ◽  
Simron Singh ◽  
Lesley Moody

225 Background: In cancer care, patient experience is most often measured quantitatively. However, this approach has limitations in understanding the entire patient experience. Qualitative methods provide an opportunity to understand self reported and more complex issues most important to patients and their family members. This qualitative study explores the patient experience of outpatient cancer treatment throughout Ontario, Canada. Methods: Qualitative content analysis was done by two researchers using data from the annual Ambulatory Oncology Patient Satisfaction Survey (AOPSS) from 2013-15. The AOPSS is a retrospective, paper-based, mailed survey, designed to capture the experiences of patients who are currently receiving cancer treatment or who have received cancer treatment within the previous six months. Patients surveyed were asked “Is there anything else you would like to tell us about your cancer care services?” The National Research Corporation of Canada’s eight dimensions of patient-centred care was used to guide the analysis. Results: 5,391 patients responded to the open-ended question. Demographic information includes: gender, age, education level and disease site. 7,328 coded responses were generated, of which 3,658 (49.9%) were related to eight dimensions of patient-centred care with accompanying sub-dimensions: patients’ preferences (23.5%): dignity and respect; preferences and decision-making; care provider behaviours; emotional support (21.5%): caring, compassion and comfort; anxiety and stress; resources; and sensitive diagnosis; coordination of care (15.8%): care provider and treatment coordination; wait times; information and education (15.7%): patient-care provider communication; information needs; care provider knowledge and skills; access (9.4%): centre location; transportation and accommodation; system; parking; continuity and transition (8.1%): centre personnel; continuity between departments; continuity in the community; discharge; physical comfort (5.1%): environment; pain management and family and friends (0.8%). Conclusions: Qualitative analysis is an effective tool to capture patient experience on a population level.


2020 ◽  
Vol 2 (1) ◽  
pp. e000033
Author(s):  
Hyun Jung Song ◽  
Sarah Dennis ◽  
Jean-Frédéric Levesque ◽  
Mark Harris

ObjectiveTo identify barriers (patient, provider, practice and system levels) to consider when implementing patient experience surveys in Australian general practice and enablers of their systematic use to inform service improvement in clinical practice as well as the broader health system.Methods and analysisAn expert consultation and qualitative content analysis of cross-sectional, open-text survey data. Data were collected from key international and Australian experts in the areas of measurement and quality improvement in general practice.ResultsResponses from 20 participants from six countries were included in the study. Participants discussed the importance of ensuring value and relevance of surveys to stakeholders. Lack of resources, IT infrastructure, capacity building and sustained funding were identified as barriers to implementing surveys. Participants discussed the importance of clearly defining and communicating the purpose of surveys and agreed on the value of using patient experience to inform reflective, team-based learning at the practice level. Opinions differed on the use of patient experience data at the system level, with some questioning its utility or fairness for external performance reporting. Others recommended the aggregation and reporting of these data under certain conditions, including for the purpose of triangulation with other quality and outcome data. The study identified an evidence gap in the assessment and interpretation of patient experience data at the practice and system levels, including the analysis and contextualisation of survey findings at the system level.ConclusionPatient experience surveys have potential for guiding practice level quality improvement, but many barriers to their implementation remain. There is need for greater research and policy efforts to understand how this information can be used at the system level for improving Australian general practice.


Sign in / Sign up

Export Citation Format

Share Document