scholarly journals Developing and Applying a Formative Evaluation Framework for Health Information Technology Implementations: Qualitative Investigation

10.2196/15068 ◽  
2020 ◽  
Vol 22 (6) ◽  
pp. e15068 ◽  
Author(s):  
Kathrin Cresswell ◽  
Robin Williams ◽  
Aziz Sheikh
Keyword(s):  
Information Technology ◽  
Health Information Technology ◽  
Health Information ◽  
Structural Changes ◽  
Formative Evaluation ◽  
Clinical Decision ◽  
Evaluation Framework ◽  
Electronic Prescribing ◽  
Data Set ◽  
Formative Evaluations

Background There is currently a lack of comprehensive, intuitive, and usable formative evaluation frameworks for health information technology (HIT) implementations. We therefore sought to develop and apply such a framework. This study describes the Technology, People, Organizations, and Macroenvironmental factors (TPOM) framework we developed. Objective The aim was to develop and apply a formative evaluation framework for HIT implementations, highlighting interrelationships between identified dimensions and offering guidance for implementers. Methods We drew on an initial prototype framework developed as part of a literature review exploring factors for the effective implementation of HIT. In addition, we used qualitative data from three national formative evaluations of different HIT interventions (electronic health record, electronic prescribing, and clinical decision support functionality). The combined data set comprised 19 case studies of primarily hospital settings, and included 703 semistructured interviews, 663 hours of observations, and 864 documents gathered from a range of care settings across National Health Service (NHS) England and NHS Scotland. Data analysis took place over a period of 10 years and was guided by a framework informed by the existing evidence base. Results TPOM dimensions are intimately related and each include a number of subthemes that evaluators need to consider. Although technological functionalities are crucial in getting an initiative off the ground, system design needs to be cognizant of the accompanying social and organizational transformations required to ensure that technologies deliver the desired value for a variety of stakeholders. Wider structural changes, characterized by shifting policy landscapes and markets, influence technologies and the ways they are used by organizations and staff. Conclusions The TPOM framework supports formative evaluations of HIT implementation and digitally enabled transformation efforts. There is now a need for prospective application of the TPOM framework to determine its value.

scholarly journals Developing and Applying a Formative Evaluation Framework for Health Information Technology Implementations: Qualitative Investigation (Preprint)

2019 ◽  
Author(s):  
Kathrin Cresswell ◽  
Robin Williams ◽  
Aziz Sheikh
Keyword(s):  
Information Technology ◽  
Health Information Technology ◽  
Health Information ◽  
Structural Changes ◽  
Formative Evaluation ◽  
Clinical Decision ◽  
Evaluation Framework ◽  
Electronic Prescribing ◽  
Data Set ◽  
Formative Evaluations

BACKGROUND There is currently a lack of comprehensive, intuitive, and usable formative evaluation frameworks for health information technology (HIT) implementations. We therefore sought to develop and apply such a framework. This study describes the Technology, People, Organizations, and Macroenvironmental factors (TPOM) framework we developed. OBJECTIVE The aim was to develop and apply a formative evaluation framework for HIT implementations, highlighting interrelationships between identified dimensions and offering guidance for implementers. METHODS We drew on an initial prototype framework developed as part of a literature review exploring factors for the effective implementation of HIT. In addition, we used qualitative data from three national formative evaluations of different HIT interventions (electronic health record, electronic prescribing, and clinical decision support functionality). The combined data set comprised 19 case studies of primarily hospital settings, and included 703 semistructured interviews, 663 hours of observations, and 864 documents gathered from a range of care settings across National Health Service (NHS) England and NHS Scotland. Data analysis took place over a period of 10 years and was guided by a framework informed by the existing evidence base. RESULTS TPOM dimensions are intimately related and each include a number of subthemes that evaluators need to consider. Although technological functionalities are crucial in getting an initiative off the ground, system design needs to be cognizant of the accompanying social and organizational transformations required to ensure that technologies deliver the desired value for a variety of stakeholders. Wider structural changes, characterized by shifting policy landscapes and markets, influence technologies and the ways they are used by organizations and staff. CONCLUSIONS The TPOM framework supports formative evaluations of HIT implementation and digitally enabled transformation efforts. There is now a need for prospective application of the TPOM framework to determine its value.

scholarly journals Informatics and interaction: Applying human factors principles to optimize the design of clinical decision support for sepsis

2019 ◽  
Vol 26 (1) ◽  
pp. 642-651
Author(s):  
Laura Schubel ◽  
Danielle L Mosby ◽  
Joseph Blumenthal ◽  
Muge Capan ◽  
Ryan Arnold ◽  
...  
Keyword(s):  
Information Technology ◽  
Decision Support ◽  
Human Factors ◽  
Organ Failure ◽  
Health Information Technology ◽  
Health Information ◽  
Clinical Decision Support ◽  
Clinical Care ◽  
Clinical Decision ◽  
Wide Range

In caring for patients with sepsis, the current structure of electronic health record systems allows clinical providers access to raw patient data without imputation of its significance. There are a wide range of sepsis alerts in clinical care that act as clinical decision support tools to assist in early recognition of sepsis; however, there are serious shortcomings in existing health information technology for alerting providers in a meaningful way. Little work has been done to evaluate and assess existing alerts using implementation and process outcomes associated with health information technology displays, specifically evaluating clinician preference and performance. We developed graphical model displays of two popular sepsis scoring systems, quick Sepsis Related Organ Failure Assessment and Predisposition, Infection, Response, Organ Failure, using human factors principles grounded in user-centered and interaction design. Models will be evaluated in a larger research effort to optimize alert design to improve the collective awareness of high-risk populations and develop a relevant point-of-care clinical decision support system for sepsis.

scholarly journals Effects of health information technology on patient outcomes: a systematic review

2015 ◽  
Vol 23 (5) ◽  
pp. 1016-1036 ◽  
Author(s):  
Samantha K Brenner ◽  
Rainu Kaushal ◽  
Zachary Grinspan ◽  
Christine Joyce ◽  
Inho Kim ◽  
...  
Keyword(s):  
Information Technology ◽  
Patient Safety ◽  
Health Information Technology ◽  
Health Information ◽  
Clinical Decision ◽  
Cochrane Library ◽  
Health It ◽  
Term Care ◽  
Safety Outcomes

Abstract Objective To systematically review studies assessing the effects of health information technology (health IT) on patient safety outcomes. Materials and Methods The authors employed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement methods. MEDLINE, Cumulative Index to Nursing Allied Health (CINAHL), EMBASE, and Cochrane Library databases, from 2001 to June 2012, were searched. Descriptive and comparative studies were included that involved use of health IT in a clinical setting and measured effects on patient safety outcomes. Results Data on setting, subjects, information technology implemented, and type of patient safety outcomes were all abstracted. The quality of the studies was evaluated by 2 independent reviewers (scored from 0 to 10). A total of 69 studies met inclusion criteria. Quality scores ranged from 1 to 9. There were 25 (36%) studies that found benefit of health IT on direct patient safety outcomes for the primary outcome measured, 43 (62%) studies that either had non-significant or mixed findings, and 1 (1%) study for which health IT had a detrimental effect. Neither the quality of the studies nor the rate of randomized control trials performed changed over time. Most studies that demonstrated a positive benefit of health IT on direct patient safety outcomes were inpatient, single-center, and either cohort or observational trials studying clinical decision support or computerized provider order entry. Discussion and Conclusion Many areas of health IT application remain understudied and the majority of studies have non-significant or mixed findings. Our study suggests that larger, higher quality studies need to be conducted, particularly in the long-term care and ambulatory care settings.

From Bench to Bed: Bridging from Informatics Theory to Practice

2014 ◽  
Vol 53 (06) ◽  
pp. 511-515 ◽  
Author(s):  
R. Haux ◽  
C. U. Lehmann
Keyword(s):  
Information Technology ◽  
Decision Support ◽  
Health Information Technology ◽  
Health Information ◽  
Hospital Admissions ◽  
Clinical Decision ◽  
Clinical Informatics ◽  
Documentation Quality ◽  
Theory To Practice ◽  
Complex Patient

SummaryBackground: In 2009, the journal Applied Clinical Informatics (ACI) commenced publication. Focused on applications in clinical informatics, ACI was intended to be a companion journal to Methods of Information in Medicine (MIM). Both journals are official journals of IMIA, the International Medical Informatics Association.Objectives: To explore, after five years, which congruencies and interdependencies exist in publications of these journals and to determine if gaps exist. To achieve this goal, major topics discussed in ACI and in MIM had to be analysed. Finally, we wanted to explore, whether the intention of publishing these companion journals to provide an information bridge from informatics theory to informatics practice and from practice to theory could be supported by this model. In this manuscript we will report on congruencies and interdependencies from practise to theory and on major topis in ACI. Further results will be reported in a second paper.Methods: Retrospective, prolective observational study on recent publications of ACI and MIM. All publications of the years 2012 and 2013 from these journals were indexed and analysed.Results: Hundred and ninety-six publications have been analysed (87 ACI, 109 MIM). In ACI publications addressed care coordination, shared decision support, and provider communication in its importance for complex patient care and safety and quality. Other major themes included improving clinical documentation quality and efficiency, effectiveness of clinical decision support and alerts, implementation of health information technology systems including discussion of failures and succeses. An emerging topic in the years analyzed was a focus on health information technology to predict and prevent hospital admissions and managing population health including the application of mobile health technology. Congruencies between journals could be found in themes, but with different focus in its contents. Interdependencies from practise to theory found in these publications, were only limited.Conclusions: Bridging from informatics theory to practise and vice versa remains a major component of successful research and practise as well as a major challenge.

Health information technology foresight for Iran: A Delphi study of experts’ views to inform future policymaking

2019 ◽  
Vol 50 (1-2) ◽  
pp. 76-87
Author(s):  
Morteza Hemmat ◽  
Haleh Ayatollahi ◽  
Mohammadreza Maleki ◽  
Fatemeh Saghafi
Keyword(s):  
Information Technology ◽  
Health Information Technology ◽  
Health Information ◽  
Delphi Study ◽  
Clinical Decision Making ◽  
Clinical Decision ◽  
Faculty Members ◽  
Future Health ◽  
Third Phase ◽  
Patient Provider Communication

Background: Policymakers require a systematic approach when planning for information technology needs in healthcare. Objective: The aim of this study was to obtain experts’ predictions of future health information technology (HIT) needs until 2025 for Iran in relation to the relative importance of key technologies, expected timeframe of realisation, areas that may be impacted upon and obstacles to achieving these goals. Method: This article presents results from the third phase (a Delphi study) of a larger mixed-method study. Policymakers from the Iranian Ministry of Health and faculty members from different medical universities across the country who were expert in the field of HIT were invited to participate ( n = 61). Results: Participants (39) completed the first-round questionnaire and 24 completed the second. The development of personal health records ( n = 32, 82.0%), the development of clinical decision-making systems ( n = 30, 76.9%) and the use of business intelligence for collecting and analysing clinical and financial data ( n = 32, 82.0%) were predicted to occur after 2025. The healthcare areas predicted to experience the greatest impact from most HITs were facilitating patient–provider communication and improving healthcare quality. Key barriers to achieving HITs were related to weaknesses in planning and limited financial resources for most technologies. Conclusion: By identifying the areas of impact and the barriers to achieving the HIT goals, more accurate planning is possible and resources can be allocated according to priorities.

scholarly journals Assessment of Health Information Technology Interventions in Evidence-Based Medicine: A Systematic Review by Adopting a Methodological Evaluation Framework

Healthcare ◽  
2018 ◽  
Vol 6 (3) ◽  
pp. 109 ◽  
Author(s):  
Stella Christopoulou ◽  
Theodore Kotsilieris ◽  
Ioannis Anagnostopoulos
Keyword(s):  
Systematic Review ◽  
Information Technology ◽  
Health Information Technology ◽  
Health Information ◽  
Evidence Based Medicine ◽  
Information Technologies ◽  
Health Service Delivery ◽  
Evaluation Framework ◽  
Evidence Based ◽  
Based Medicine

Background: The application of Health Information Technologies (HITs) can be an effective way to advance medical research and health services provision. The two-fold objective of this work is to: (i) identify and review state-of-the-art HITs that facilitate the aims of evidence-based medicine and (ii) propose a methodology for HIT assessment. Methods: The systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Furthermore, we consolidated existing knowledge in the field and proposed a Synthesis Framework for the Assessment of Health Information Technology (SF/HIT) in order to evaluate the joint use of Randomized Controlled Trials (RCTs) along with HITs in the field of evidence-based medicine. Results: 55 articles met the inclusion criteria and refer to 51 (RCTs) published between 2008 and 2016. Significant improvements in healthcare through the use of HITs were observed in the findings of 31 out of 51 trials—60.8%. We also confirmed that RCTs are valuable tools for assessing the effectiveness, acceptability, safety, privacy, appropriateness, satisfaction, performance, usefulness and adherence. Conclusions: To improve health service delivery, RCTs apply and exhibit formalization by providing measurable outputs. Towards this direction, we propose the SF/HIT as a framework which may help researchers to carry out appropriate evaluations and extend their studies.

scholarly journals Everything but the Kitchen Sink: A Mixed Methods Approach to Health IT Usability Testing

2019 ◽  
Vol 63 (1) ◽  
pp. 718-722
Author(s):  
Laura Schubel ◽  
Robin Littlejohn ◽  
Ronald Romero Barrientos ◽  
Ryan Arnold ◽  
Muge Capan ◽  
...  
Keyword(s):  
Information Technology ◽  
Health Information Technology ◽  
Health Information ◽  
Usability Testing ◽  
Clinical Decision ◽  
User Needs ◽  
Health It ◽  
Mixed Methods Approach ◽  
Healthcare Environment ◽  
Think Aloud Protocols

Usability testing has become a necessary step to successfully implement health information technology, but despite the expanded variety of usability methods, systems still are not guaranteed to meet user needs in a hectic and dynamic healthcare environment. The present work discusses some of the metrics employed with the intent of usability testing clinical decision support alerts for the early identification of sepsis patients. By utilizing methods such as eye tracking, think aloud protocols, human computer interaction, and more, health information technology can be adequately developed to ensure effective, efficacious, and optimal integration.

scholarly journals Transforming AADE7 for Use in an Evaluation Framework for Health Information Technology in Diabetes Mellitus

2021 ◽  
pp. 193229682098584
Author(s):  
Min Soon Kim ◽  
Uzma Khan ◽  
Suzanne A. Boren ◽  
Ploypun Narindrarangkura ◽  
Qing Ye ◽  
...  
Keyword(s):  
Information Technology ◽  
Health Information Technology ◽  
Health Information ◽  
Real Life ◽  
Self Care ◽  
Evaluation Framework ◽  
Self Management ◽  
Diabetes Research ◽  
Patient Centered ◽  
Diabetes Educators

There is no validated framework to evaluate health information technology (HIT) for diabetes self-management education and support (DSMES). AADE7 Self-Care Behaviors is a patient-centered DSMES designed by the American Association of Diabetes Educators (AADE). We developed a codebook based on the AADE7 Self-Care Behaviors principles as an evaluation framework. In this commentary, we demonstrate the real-life applications of this codebook through three diabetes research studies. The first study analyzed features of mobile diabetes applications. The second study evaluated provider documentation patterns in electronic health records (EHRs) to deliver ongoing patient-centered DSMES. The third study analyzed feedback messages from diabetes apps. We found that this codebook, based on AADE7, can be instrumental as a framework for research, as well as real-life use in HIT for DSMES principles.

scholarly journals Compatibility of Family History Cancer Guidelines With Meaningful Use Standards

2017 ◽  
pp. 1-9
Author(s):  
Ayesha Aziz ◽  
Lance Pflieger ◽  
Nathaniel O’Connell ◽  
Joshua Schiffman ◽  
Brandon M. Welch
Keyword(s):  
Information Technology ◽  
Decision Support ◽  
Health Information Technology ◽  
Health Information ◽  
Clinical Decision Support ◽  
Meaningful Use ◽  
Clinical Decision ◽  
Cancer Type ◽  
Cancer Guidelines ◽  
Technology Systems

Purpose To evaluate the potential of implementing established family cancer guidelines as clinical decision support within meaningful use (MU)–compliant health information technology systems. Methods We conducted a systematic analysis of cancer guidelines involving family health history (FHx) published before 2015. By comparing existing cancer guideline statements to current MU FHx standard requirements, we determined whether the cancer guideline statements could be implemented as clinical decision support. For guidelines that could not implemented, we determined the primary reasons for incompatibility. Results A total of 531 statements from 55 guidelines published by 11 different organizations were reviewed and analyzed. Overall, 18% to 66% of guideline statements could or could not be implemented in MU-compliant health information technology systems, depending on which MU standard was used. Health Level Seven (HL7) models performed better than SNOMED models. Implementability of guideline statements varied by cancer type and guideline organizations. The greatest deficiencies in implementability of statements were largely a result of the fact that MU standards required only first-degree relatives and that FHx terms used in guidelines statements were ambiguous. Conclusion FHx cancer guidelines and MU-based systems vary widely and are mostly incompatible. We identified sources of incompatibility and made recommendations that could improve the implementability of FHx cancer guidelines. Our findings and recommendations can enhance the use of established FHx cancer risk guidelines in routine clinical workflows.

Sign in / Sign up

Export Citation Format

Share Document