Interfaces for Searching and Triaging Large Document Sets: An Ontology-Supported Visual Analytics Approach

We investigate the design of ontology-supported, progressively disclosed visual analytics interfaces for searching and triaging large document sets. The goal is to distill a set of criteria that can help guide the design of such systems. We begin with a background of information search, triage, machine learning, and ontologies. We review research on the multi-stage information-seeking process to distill the criteria. To demonstrate their utility, we apply the criteria to the design of a prototype visual analytics interface: VisualQUEST (Visual interface for QUEry, Search, and Triage). VisualQUEST allows users to plug-and-play document sets and expert-defined ontology files within a domain-independent environment for multi-stage information search and triage tasks. We describe VisualQUEST through a functional workflow and culminate with a discussion of ongoing formative evaluations, limitations, future work, and summary.

Promoting De-Implementation of Inappropriate Antimicrobial Use in Cardiac Device Procedures By Expanding Audit and Feedback: Protocol for Hybrid III Type Effectiveness/ Implementation Quasi-Experimental Study

Background: Despite a strong evidence base and clinical guidelines specifically recommending against prolonged post-procedural antimicrobial use, studies indicate that the practice is common following cardiac device procedures. Formative evaluations conducted by the study team suggest that inappropriate antimicrobial use may be driven by information silos that drive provider belief that antimicrobials are not harmful, in part due to lack of complete feedback about all types of clinical outcomes. De-implementation is recognized as an important area of research that can lead to reductions in unnecessary, wasteful, or harmful practices, such as excess antimicrobial use following cardiac device procedures; however, investigations into strategies that lead to successful de-implementation are limited. The overarching hypothesis to be tested in this trial is that a bundle of implementation strategies that includes audit and feedback about direct patient harms caused by inappropriate prescribing can lead to successful de-implementation of guideline-discordant care.Methods: We propose a Hybrid Type III effectiveness-implementation stepped-wedge intervention trial at three high-volume, high-complexity VA medical centers. The main study intervention (an informatics-based, real-time audit-and-feedback tool) was developed based on learning/unlearning theory and formative evaluations and guided by the integrated-Promoting Action on Research Implementation in Health Services (i-PARIHS) Framework. Elements of the bundled and multifaceted implementation strategy to promote appropriate prescribing will include audit-and-feedback reports that include information about antibiotic harms, stakeholder engagement, patient and provider education, identification of local champions, and blended facilitation. The primary study outcome is adoption of evidence-based practice (de-implementation of inappropriate antimicrobial use). Clinical outcomes (cardiac device infections, acute kidney injuries and Clostridioides difficile infections) are secondary. Qualitative interviews will assess relevant implementation outcomes (acceptability, adoption, fidelity, feasibility).Discussion: De-implementation theory suggests that factors that may have a particularly strong influence on de-implementation include strength of the underlying evidence, the complexity of the intervention, and patient and provider anxiety and fear about changing an established practice. This study will assess whether a multifaceted intervention mapped to identified de-implementation barriers leads to measurable improvements in provision of guideline-concordant antimicrobial use. Findings will improve understanding about factors that impact successful or unsuccessful de-implementation of harmful or wasteful healthcare practices.Trial Registration: Clinicaltrials.gov (NCT05020418)

Evolución del comportamiento de los estudiantes de Biomecánica en el campus virtual

Este estudio compara el uso realizado del campus virtual por los estudiantes de la asignatura de Biomecánica del Grado en Podología de la Universidad Complutense de Madrid, durante los cursos académicos, 2017-18 y 2019-20, y su relación con el rendimiento académico. En el segundo curso la metodología docente se implementó introduciendo evaluaciones formativas en el aula y la clase invertida en las prácticas clínicas de sala. Se extrajeron de las asignaturas virtualizadas un total de 52.779 registros almacenados en la plataforma Moodle entre ambos cursos académicos. Los logs fueron depurados, anonimizados y analizados con los programas R y SPSS v.25. Frente a un comportamiento errático y malos resultados del primer curso, en el segundo los resultados académicos mejoraron y el uso del campus correlacionó significativamente con las calificaciones finales de la asignatura y con las notas de prácticas. Los resultados sugieren que las evaluaciones formativas desarrolladas en el aula y el grado de motivación y compromiso conseguido con la metodología invertida para las prácticas de sala han sido clave para lograr un uso útil del CV y mejorar el rendimiento académico.This study compares the use of the virtual campus performed by the students of Biomechanics, a subject of the Podiatry Degree from the Complutense University of Madrid during 2017-18 and 2019-20 academic courses, and its relationship with the academic marks. In the second academic course, teaching methodology was implemented applying formative evaluations in the classroom and flipped classroom in their clinical practices. A total of 52,779 records from the virtualized subjects stored in Moodle platform in both academic courses were extracted. The logs were debugged, anonymized and analysed with the R and SPSS v.25 programs. In contrast from the erratic behaviour and the poor results in the first course, in the second one, the academic performance improved, and the use of virtual campus significantly correlated with both the final and the practice grades. Results suggest that the formative evaluations carried out in the classroom and the attained motivation and commitment levels with the flipped classroom practices were key to achieving a better CV performance and improving the academic grades.

The success and failure of trainers and trainees: a configurational analysis of training programs

Purpose The purpose of this study is to examine formative evaluations of the training programs in order to uncover the configurations of designs which lead to the success of failure of trainers and trainees and to identify any common designs that generate this success or failure. Design/methodology/approach Data is taken from an entire calendar year using a 2017 database involving 429 training events by 264 trainers and 2,264 trainees and configurations are identified using a fuzzy set qualitative comparative analysis (fsQCA). Findings The results show that for trainers more configurations (five) are shown to lead to success than failure while for trainees fewer (three) lead to success than failure(six). One configuration leading to success and two leading to failure are common to both. Practical implications Therefore, this suggests that consideration of configurations which lead to success should be put in place when designing training programs and damaging pathways avoided. Originality/value This paper has an original approach as it analyses the success and failure of trainers and trainees concurrently through use of the fsQCA method.

Development of Language Learning Based on Multimedia Devices for Autistic Student

The purpose of this study to develop a learning device of Indonesian language based on multimedia for autistic students. This research was conducted at SD Islam Alam dan Sains Al- Jannah. The students involved in the research were 20 students. The developed learning device uses the instructional model approach by Dick Carey and Carey (2009). Subjects of the research were 20 students. The developed learning device consists on applications, videos and manual book for both student and teachers. The developed products have been reviewed by a subject matter expert, an instructional design expert, a language expert, a media expert and an information technology expert. A series of formative evaluations through try-out examination has been conducted. The improvement of the learning device has been developed based on the previous formative evaluations. The results of this study shows developed learning device could improve the student learning outcomes effectively. Based on this result, the learning device could be recommended to be used as an alternative learning resource. Keywords: development of learning devices, autistic student, Indonesian language, multimedia

New Trends in Formative-Summative Evaluations for Adult Education

The aim of this article was to review the different evaluation approaches for adult learners and the effect on promoting the quality of teaching and learning. This study aimed to identify new trends in adult education formative-summative evaluations. Data were collected from multiple peer-reviewed sources in a comprehensive literature review covering the period from January 2014 to March 2019. A total of 22 peer-reviewed studies were included in this study. Results were systematically analyzed to answer three questions as follows: what are the new trends in the summative and formative evaluations of adult learners? What are the new trends in the summative and formative evaluations of adult learners engaged in distance learning? And what are the outcomes/drawbacks in the summative and formative evaluations of adult learners? An analysis of the existing literature indicated that those who instruct adults must use a wide variety of pre- and post-assessment tools to match students’ differences with their needs. It also highlighted the importance of “assessment for learning” rather than “assessment of learning” and “learning-oriented assessment” (LOA) for lifelong learning, thus preparing adult learners for future responsibilities and decision making. It also indicated the importance of reflection and immediate feedback for the adult learner. Assessment of mental phenomena such as creativity should have defined terms. The findings of this article supported the argument for more attention to be paid to new trends in evaluations used in adult education. One important result of this kind of evaluation is its facilitation of self-confidence within the adult learning setting.

Developing and Applying a Formative Evaluation Framework for Health Information Technology Implementations: Qualitative Investigation

Background There is currently a lack of comprehensive, intuitive, and usable formative evaluation frameworks for health information technology (HIT) implementations. We therefore sought to develop and apply such a framework. This study describes the Technology, People, Organizations, and Macroenvironmental factors (TPOM) framework we developed. Objective The aim was to develop and apply a formative evaluation framework for HIT implementations, highlighting interrelationships between identified dimensions and offering guidance for implementers. Methods We drew on an initial prototype framework developed as part of a literature review exploring factors for the effective implementation of HIT. In addition, we used qualitative data from three national formative evaluations of different HIT interventions (electronic health record, electronic prescribing, and clinical decision support functionality). The combined data set comprised 19 case studies of primarily hospital settings, and included 703 semistructured interviews, 663 hours of observations, and 864 documents gathered from a range of care settings across National Health Service (NHS) England and NHS Scotland. Data analysis took place over a period of 10 years and was guided by a framework informed by the existing evidence base. Results TPOM dimensions are intimately related and each include a number of subthemes that evaluators need to consider. Although technological functionalities are crucial in getting an initiative off the ground, system design needs to be cognizant of the accompanying social and organizational transformations required to ensure that technologies deliver the desired value for a variety of stakeholders. Wider structural changes, characterized by shifting policy landscapes and markets, influence technologies and the ways they are used by organizations and staff. Conclusions The TPOM framework supports formative evaluations of HIT implementation and digitally enabled transformation efforts. There is now a need for prospective application of the TPOM framework to determine its value.

1101. What Do Electrophysiologists Think about Peri-Procedural Antibiotics? A Qualitative Assessment of Factors Driving Use and Facilitators for Implementing Change

Background Prolonged courses of antimicrobials are common following cardiac device procedures, but there are little data to explain drivers of this practice and factors that may facilitate change. Methods We conducted formative evaluations consisting of semi-structured, qualitative interviews with electrophysiologists (EP) to identify perceived barriers to discontinuing post-procedure antimicrobial prophylaxis and factors that may facilitate improvements. A directed content analysis approach was used to map qualitative responses to key factors in the Proctor Implementation Outcomes Framework, with flexibility to allow for new themes to emerge. Interviews ceased after data saturation was reached. Results 13 interviews were conducted with EPs representing diverse US regions (Northeast, Midwest, South, West) and diverse settings of care (academic, community, VA). Responses to questions about antimicrobial use and willingness (or lack thereof) to stop post-procedural antimicrobials most commonly mapped to the acceptability domain; feasibility, fidelity, cost and appropriateness were also frequently identified factors (see figure for exemplary quotes). Themes that emerged during the interview process associated with prolonged antimicrobial prescribing included beliefs and knowledge, local culture and normative behaviors, and organizational structure. There was a strong “cultural inertia” to conform to normative practices within an institution. Reasons for this ranged from reports of streamlining processes for clinical staff to ensure standardized care across all patients and concerns about being perceived as an “outlier.” Infectious diseases staff were important influencers of practice and potential facilitators of improvement. Conclusion Formative evaluations of stakeholders are essential for designing successful implementation interventions to facilitate behavioral change. Local culture appeared to be a major driver of antimicrobial use. The desire to conform to normative behaviors and to promote institutional standardization suggests that strategies to facilitate implementation of antimicrobial stewardship guidelines must include facility-level changes, rather than individual-provider-level interventions. Disclosures All authors: No reported disclosures.