scholarly journals Using Evaluation to Inform the BioSense Platform: Results from a 2018 Survey

2019 ◽  
Vol 11 (1) ◽  
Author(s):  
Cassandra N. Davis
Keyword(s):  
User Satisfaction ◽  
Present Status ◽  
Syndromic Surveillance ◽  
Data Flow ◽  
Data Access ◽  
Comfort Level ◽  
Acceptance Testing ◽  
Free Text ◽  
Data Access Control ◽  
System Usability Scale

ObjectiveTo assess the present status of utility, functionality, usability and user satisfaction of the BioSense Platform.IntroductionSince 2015, CDC’s Division of Health Informatics and Surveillance staff have conducted evaluations to provide information on the utility, functionality, usability and user satisfaction associated with the National Syndromic Surveillance Program’s BioSense Platform tools. The BioSense Platform tools include: 1) Access and Management Center (AMC), a tool that enables site administrators to manage users and data permissions; 2) Electronic Surveillance System for Early Notification of Community-based Epidemics (ESSENCE), a software application that enables syndromic surveillance related data visualization and analysis; 3) Adminer, a tool that allows users to access site data on the datamart; and 4) Rstudio, an application that can be used for data analysis and visualization. The evaluation findings have informed activities that led to improvements in functionality, development or procurement of platform associated tools, and development of resource materials. In May 2018, NSSP conducted an evaluation with eight jurisdictions that participated in the first user acceptance testing (UAT) evaluations in 2015. The purpose of the evaluation was to assess the present status of utility, functionality, usability and user satisfaction of the tools on the BioSense Platform, and delineate progress since 2015.MethodsCDC’s evaluation framework and utilization-focused evaluation were used to inform and engage stakeholders, develop the evaluation questions, metrics, and methodology. Eight selected jurisdictions participated in an online, Epi-Info survey that captured quantitative and qualitative information. Prior to the survey, participants received a presentation about the evolution of the BioSense Platform since 2015, and were provided an overview of components to evaluate. The participants were asked to assess the following key areas based on use of the BioSense Platform within the past 30 days: 1) the utility, functionality and usability of the AMC, ESSENCE, Adminer and Rstudio; 2) how well the enhanced data flow has enabled them to conduct syndromic surveillance activities; 3) usefulness of the quick start guides. Additionally, participants were asked to provide suggestions for other improvements to the BioSense Platform, and to indicate their overall satisfaction. Descriptive statistics were generated and thematic analysis was conducted to identify themes from qualitative responses.ResultsOverall, participant’s responses remained positive about the utility, functionality, usability and overall satisfaction of the BioSense Platform. Participants indicated using the BioSense Platform regularly (e.g. daily, weekly and/or monthly) within those 30 days. Certain functions have been used more than others across the various tools to conduct syndromic surveillance, with at least 50% of participants reporting use. These included creating data access rules, viewing and verifying raw and processed data, running time series, conducting free-text queries, and assessing data details and total ER visit counts by hospital, county/region, or state. The challenges ranged from tool performance to user interpretation of the function. Participants reported that the enhanced data flow improved their data quality and helped identify issues. Although participants scored ESSENCE to have average usability per the system usability scale (SUS score=63.5 in 2018), the BioSense Platform and its tools were reported as useful by 88% of participants. Further, participants continue to be comfortable using the AMC, however creating data access rules that are outside of simple use cases continue to be a challenge. Participants comfort level with Adminer improved from 2016 to 2018 with all participants reporting comfortable in using the tool. The use of each tool’s quick start guides varied. Of those who used the guides, all of the participants agreed that the Adminer and Data Dictionary guides were useful. There was a smaller number of participants agreeing that the other guides were useful. Lastly, participants provided recommendations to improving the BioSense Platform. The most frequent recommendations were improving the data access control architecture, and sharing aggregate data with hospitals in their state.ConclusionsThe development and operationalization of the BioSense Platform and associated tools has been in an environment of continuing advancements in technology and changing public health needs and priorities. Up-to-date evaluation activities have helped to ensure that BioSense is best suited to address these challenges and meet the syndromic surveillance needs of users. Overall, the findings outlined above indicate that the functionality and utility of BioSense are well suited to meet user needs.ReferencesBangor A, Kortm P, Miller J. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. Journal of Usability Studies. 2009; 4:114-123. 

scholarly journals Impact of the NSSP’s transition to ESSENCE on chief complaint field-based syndromes

2018 ◽  
Vol 10 (1) ◽  
Author(s):  
Rasneet S Kumar ◽  
Jessica R White
Keyword(s):  
Time Series ◽  
Syndromic Surveillance ◽  
Data Access ◽  
Case Finding ◽  
Chief Complaint ◽  
Analytical Tool ◽  
Free Text ◽  
Guidance Document ◽  
Maricopa County ◽  
Analytical Tools

Objective: To evaluate the effect and implications of changing the chief complaint field during the National Syndromic Surveillance Program (NSSP) transition from BioSense 2.0 analytical tools to BioSense Platform – ESSENCEIntroduction: In January 2017, the NSSP transitioned their BioSense analytical tools to Electronic Surveillance System for Early Notification of Community-Based Epidemics (ESSENCE). The chief complaint field in BioSense 2.0 was a concatenation of the record’s chief complaint, admission reason, triage notes, and diagnostic impression. Following the transition to ESSENCE, the chief complaint field was comprised of the first chief complaint entered or the first admission reason, if the chief complaint was blank. Furthermore, the ESSENCE chief complaint field was electronically parsed (i.e., the original chief complaint text was altered to translate abbreviations and remove punctuation). This abstract highlights key findings from Maricopa County Department of Public Health’s evaluation of the new chief complaint field, its impact on heat-related illness syndromic surveillance, and implications for ongoing surveillance efforts.Methods: For this evaluation, we used the heat-related illness query recommended in Council of State and Territorial Epidemiologists’ (CSTE)2016 Guidance Document for Implementing Heat-Related Illness Syndromic Surveillance. Before the transition, we used BioSense 2.0’s, phpMyAdmin analytical tool to generate a list of patients who visited Maricopa County emergency departments or inpatient hospitals between 5/1/2016 – 9/30/2016 due to heat-related illness. After the transition, we used the CC and DD Category “Heat-related Illness, v1” in ESSENCE, which was based on the CSTE heat-related illness query, to generate a list of patients for the same time period. We compared the line-lists and time-series trends from phpMyAdmin and ESSENCE.Results: The phpMyAdmin analytical tool identified 785 heat-related illness records with the query (Figure). 642 (82%) of these heat-related illness records were also captured by ESSENCE. Reasons for 143 (18%) records not being identified by ESSENCE included: the patient’s admission reason field contained keywords that were not available in the ESSENCE chief complaint field (n=94, 66%); data access changed, which disabled access to patients who resided in zip codes that crossed a county border (30, 21%); discrepancies between ESSENCE parsing and text in the original chief complaint (11, 8%); heat-related illness discharge diagnoses were removed by the facility after the phpMyAdmin line-list for heat-related illness was extracted (7, 5%); and one record was undetermined. Conversely, ESSENCE captured 36 additional heat-related illness records, not previously captured by phpMyAdmin. Reasons included: a query exclusion term was located in the patient’s admission reason but not the ESSENCE chief complaint field (16, 44%); a heat-related illness discharge diagnosis code was added by the facility after the data were extracted by phpMyAdmin (4, 11%); and 16 (44%) were undetermined. Time-series trend evaluation revealed a significant correlation between the two surveillance tools (Pearson coefficient = 0.97, p < 0.01).Conclusions: Though the data trends over time were not significantly affected by changes in the chief complaint field, differences in the field’s composition have important implications for syndromic surveillance practitioners. Free-text queries designed to search the chief complaint field in ESSENCE may not retrieve records previously identified with BioSense 2.0 analytical tools, which may limit individual case-finding capacity. The elimination of admission reason from the chief complaint field in ESSENCE has the greatest effect on case-finding capacity. Furthermore, surveillance reports produced by ESSENCE cannot be directly compared to reports that were previously published with data from BioSense 2.0. These limitations may be addressed if ESSENCE creates a feature that allows users to easily query fields (e.g., admission reason) in addition to the chief compliant field.

scholarly journals Controlled Sharing Mechanism of Data Based on the Consortium Blockchain

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Jin Li ◽  
Songqi Wu ◽  
Yundan Yang ◽  
Fenghui Duan ◽  
Hui Lu ◽  
...  
Keyword(s):  
Data Storage ◽  
Data Flow ◽  
Electric Energy ◽  
Electrical Energy ◽  
Data Access ◽  
Original Data ◽  
Third Party ◽  
Data Accessibility ◽  
Energy Data ◽  
Data Access Control

In the process of sharing data, the costless replication of electric energy data leads to the problem of uncontrolled data and the difficulty of third-party access verification. This paper proposes a controlled sharing mechanism of data based on the consortium blockchain. The data flow range is controlled by the data isolation mechanism between channels provided by the consortium blockchain by constructing a data storage consortium chain to achieve trusted data storage, combining attribute-based encryption to complete data access control and meet the demands for granular data accessibility control and secure sharing; the data flow transfer ledger is built to record the original data life cycle management and effectively record the data transfer process of each data controller. Taking the application scenario of electric energy data sharing as an example, the scheme is designed and simulated on the Linux system and Hyperledger Fabric. Experimental results have verified that the mechanism can effectively control the scope of access to electrical energy data and realize the control of the data by the data owner.

scholarly journals A Lightweight Blockchain-Based IoT Identity Management Approach

2021 ◽  
Vol 13 (2) ◽  
pp. 24
Author(s):  
Mohammed Amine Bouras ◽  
Qinghua Lu ◽  
Sahraoui Dhelim ◽  
Huansheng Ning
Keyword(s):  
Identity Management ◽  
Single Point ◽  
Data Access ◽  
Emerging Technology ◽  
Management Approach ◽  
Proof Of Concept ◽  
Data Access Control ◽  
And Storage ◽  
Privacy Issues ◽  
Centralized System

Identity management is a fundamental feature of Internet of Things (IoT) ecosystem, particularly for IoT data access control. However, most of the actual works adopt centralized approaches, which could lead to a single point of failure and privacy issues that are tied to the use of a trusted third parties. A consortium blockchain is an emerging technology that provides a neutral and trustable computation and storage platform that is suitable for building identity management solutions for IoT. This paper proposes a lightweight architecture and the associated protocols for consortium blockchain-based identity management to address privacy, security, and scalability issues in a centralized system for IoT. Besides, we implement a proof-of-concept prototype and evaluate our approach. We evaluate our work by measuring the latency and throughput of the transactions while using different query actions and payload sizes, and we compared it to other similar works. The results show that the approach is suitable for business adoption.

scholarly journals Comparison of the validity, perceived usefulness and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability: a study protocol

BMJ Open ◽  
2021 ◽  
Vol 11 (8) ◽  
pp. e050448
Author(s):  
Romaric Marcilly ◽  
Wu Yi Zheng ◽  
Regis Beuscart ◽  
Melissa T Baysari
Keyword(s):  
Human Factors ◽  
Convergent Validity ◽  
Perceived Usefulness ◽  
Decision Makers ◽  
Free Text ◽  
Alert System ◽  
System Usability Scale ◽  
System Usability ◽  
Ethical Approval ◽  
Alerting Systems

IntroductionResearch has shown that improvements to the usability of medication alert systems are needed. For designers and decisions-makers to assess usability of their alert systems, two paper-based tools are currently available: the instrument for evaluating human-factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aims to compare the validity, usability and usefulness of both tools to identify their strengths and limitations and assist designers and decision-makers in making an informed decision about which tool is most suitable for assessing their current or prospective system.Methods and analysisFirst, TEMAS and I-MeDeSA will be translated into French. This translation will be validated by three experts in human factors. Then, in 12 French hospitals with a medication alert system in place, staff with expertise in the system will evaluate their alert system using the two tools successively. After the use of each tool, participants will be asked to fill in the System Usability Scale (SUS) and complete a survey on the understandability and perceived usefulness of each tool. Following the completion of both assessments, participants will be asked to nominate their preferred tool and relay their opinions on the tools. The design philosophy of TEMAS and I-MeDeSA differs on the calculation of a score, impacting the way the comparison between the tools can be performed. Convergent validity will be evaluated by matching the items of the two tools with respect to the usability dimensions they assess. SUS scores and answers to the survey will be statistically compared for I-MeDeSA and TEMAS to identify differences. Free-text responses in surveys will be analysed using an inductive approach.Ethics and disseminationEthical approval is not required in France for a study of this nature. The results will be published in a peer-reviewed journal.

scholarly journals Usability of an Affective Emotional Learning Tutoring System for Mobile Devices

2021 ◽  
Vol 13 (14) ◽  
pp. 7890
Author(s):  
Tao-Hua Wang ◽  
Hao-Chiang Koong Lin ◽  
Hong-Ren Chen ◽  
Yueh-Min Huang ◽  
Wei-Ting Yeh ◽  
...  
Keyword(s):  
Sustainable Development ◽  
Mobile Devices ◽  
User Satisfaction ◽  
Quantitative Research ◽  
Negative Emotions ◽  
User Interaction ◽  
Feedback Mechanism ◽  
Quality Education ◽  
Tutoring System ◽  
System Usability Scale

To echo the United Nations formulated Sustainable Development Goals (SDGs), SDG 4 is to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. Furthermore, high-quality education is the base on which human lives can be improved and sustainable development can be accomplished. Therefore, the affective emotional tutoring system established in this study enables learning via mobile devices, which are indispensable in daily life. The real-time interactive agent in the system guides learners to turn negative emotions into positive ones. We explored the usability of and user satisfaction with the affective emotional tutoring system. Sixty-two students participated in the study which used a quantitative research design to explore a learning situation. The overall usability of the system was evaluated with the System Usability Scale (SUS), and the Questionnaire for User Interaction Satisfaction (QUIS) was used to evaluate user satisfaction with the different elements of the system. The results showed that both the usability of and satisfaction with the affective emotional tutoring system were high. The emotional feedback mechanism of the system can help learners turn negative emotions into positive ones.

Sign in / Sign up

Export Citation Format

Share Document