Evaluation Metrics for Health Chatbots: A Delphi Study

2021 ◽  
Author(s):  
Kerstin Denecke ◽  
Alaa Abd-Alrazaq ◽  
Mowafa Househ ◽  
Jim Warren
Keyword(s):  
Delphi Study ◽  
Research Literature ◽  
Ease Of Use ◽  
Evaluation Framework ◽  
Evaluation Metrics ◽  
Sensitive Data ◽  
The Third ◽  
The Core ◽  
Core Set ◽  
Font Type

Abstract Background In recent years, an increasing number of health chatbots has been published in app stores and described in research literature. Given the sensitive data they are processing and the care settings for which they are developed, evaluation is essential to avoid harm to users. However, evaluations of those systems are reported inconsistently and without using a standardized set of evaluation metrics. Missing standards in health chatbot evaluation prevent comparisons of systems, and this may hamper acceptability since their reliability is unclear. Objectives The objective of this paper is to make an important step toward developing a health-specific chatbot evaluation framework by finding consensus on relevant metrics. Methods We used an adapted Delphi study design to verify and select potential metrics that we retrieved initially from a scoping review. We invited researchers, health professionals, and health informaticians to score each metric for inclusion in the final evaluation framework, over three survey rounds. We distinguished metrics scored relevant with high, moderate, and low consensus. The initial set of metrics comprised 26 metrics (categorized as global metrics, metrics related to response generation, response understanding and aesthetics). Results Twenty-eight experts joined the first round and 22 (75%) persisted to the third round. Twenty-four metrics achieved high consensus and three metrics achieved moderate consensus. The core set for our framework comprises mainly global metrics (e.g., ease of use, security content accuracy), metrics related to response generation (e.g., appropriateness of responses), and related to response understanding. Metrics on aesthetics (font type and size, color) are less well agreed upon—only moderate or low consensus was achieved for those metrics. Conclusion The results indicate that experts largely agree on metrics and that the consensus set is broad. This implies that health chatbot evaluation must be multifaceted to ensure acceptability.

Consent

2018 ◽  
Author(s):  
Martin Krzywdzinski
Keyword(s):  
Production Systems ◽  
Employee Behavior ◽  
Labor Disputes ◽  
Performance Expectations ◽  
The Third ◽  
The Core ◽  
Core Requirements ◽  
High Level

This chapter deals with the dependent variable of the study: consent. It analyses workplace consent in Russia and China using three indicators that refer to the core requirements of the production systems in automotive companies regarding employee behavior: first, standardized work; and second, compliance with expectations in terms of flexibility, cooperation, and a commitment to improving processes. The third indicator of consent (or the lack of it) is the absence or presence of open criticism, resistance, and labor disputes. The chapter reveals significant and unexpected differences between the Chinese and Russian sites on all three indicators. While the Chinese factories exhibit (with some variance between the companies), a relatively high level of consent, the Russian plants have problems with standardized work, the acceptance of performance expectations, and to some extent with labor disputes.

Assessment in the Applied Studio in Higher Education

2019 ◽  
pp. 765-791
Author(s):  
Kelly A. Parkes
Keyword(s):  
Higher Education ◽  
Research Literature ◽  
The Third ◽  
Studio Teachers ◽  
Applied Studio

This chapter outlines the various lines of inquiry and research about assessment in the applied studio and makes recommendations for further research. It comprises three sections: the first focused on briefly defining the applied setting, the second on assessment of musical learning in the applied studio, and the third section on the assessment of teaching in the applied studio. The chapter explores previous research literature and illustrates the need for further work. It will be relevant for musicians preparing to become applied studio teachers, current applied studio teachers, and administrators seeking to evaluate the work undertaken in applied studios.

Duoethnography: A Polytheoretical Approach to (Re)Storing, (Re)Storying the Meanings That One Gives

2020 ◽  
pp. 396-423
Author(s):  
John Joseph Norris ◽  
Richard D. Sawyer
Keyword(s):  
Social Justice ◽  
Critical Theory ◽  
Third Space ◽  
The Third ◽  
The Core ◽  
Wide Range ◽  
Core Elements ◽  
Feminist Inquiry

This chapter summarizes the advancement of duoethnography throughout its fifteen-year history, employing examples from a variety of topics in education and social justice to provide a wide range of approaches that one may take when conducting a duoethnography. A checklist articulates what its cofounders consider the core elements of duoethnographies, additional features that may or may not be employed and how some studies purporting to be duoethnographies may not be so. The chapter indicates connections between duoethnography and a number of methodological concepts including the third space, the problematics of representation, feminist inquiry, and critical theory using published examples by several duoethnographers.

scholarly journals Costa, cancer and coronavirus: contractualism as a guide to the ethics of lockdown

2021 ◽  
pp. medethics-2020-107103
Author(s):  
Stephen David John ◽  
Emma J Curran
Keyword(s):  
Moral Theory ◽  
Cost Benefit Analysis ◽  
Cost Benefit ◽  
Benefit Analysis ◽  
Life And Death ◽  
The Third ◽  
The Core ◽  
Key Issues ◽  
Moral Distinction ◽  
Self Defence

Lockdown measures in response to the COVID-19 pandemic involve placing huge burdens on some members of society for the sake of benefiting other members of society. How should we decide when these policies are permissible? Many writers propose we should address this question using cost-benefit analysis (CBA), a broadly consequentialist approach. We argue for an alternative non-consequentialist approach, grounded in contractualist moral theorising. The first section sets up key issues in the ethics of lockdown, and sketches the apparent appeal of addressing these problems in a CBA frame. The second section argues that CBA fundamentally distorts the normative landscape in two ways: first, in principle, it allows very many morally trivial preferences—say, for a coffee—might outweigh morally weighty life-and-death concerns; second, it is insensitive to the core moral distinction between victims and vectors of disease. The third section sketches our non-consequentialist alternative, grounded in Thomas Scanlon’s contractualist moral theory. On this account, the ethics of self-defence implies a strong default presumption in favour of a highly restrictive, universal lockdown policy: we then ask whether there are alternatives to such a policy which are justifiable to all affected parties, paying particular attention to the complaints of those most burdened by policy. In the fourth section, we defend our contractualist approach against the charge that it is impractical or counterintuitive, noting that actual CBAs face similar, or worse, challenges.

scholarly journals A Fast and Exact Greedy Algorithm for the Core–Periphery Problem

Symmetry ◽  
2020 ◽  
Vol 12 (1) ◽  
pp. 94 ◽  
Author(s):  
Dario Fasino ◽  
Franca Rinaldi
Keyword(s):  
Structural Analysis ◽  
Optimization Problem ◽  
Polynomial Time Algorithm ◽  
Time Algorithm ◽  
Combinatorial Optimization Problem ◽  
Connected Subgraph ◽  
Optimal Solutions ◽  
The Core ◽  
Core Set ◽  
Key Concepts

The core–periphery structure is one of the key concepts in the structural analysis of complex networks. It consists of a partitioning of the node set of a given graph or network into two groups, called core and periphery, where the core nodes induce a well-connected subgraph and share connections with peripheral nodes, while the peripheral nodes are loosely connected to the core nodes and other peripheral nodes. We propose a polynomial-time algorithm to detect core–periphery structures in networks having a symmetric adjacency matrix. The core set is defined as the solution of a combinatorial optimization problem, which has a pleasant symmetry with respect to graph complementation. We provide a complete description of the optimal solutions to that problem and an exact and efficient algorithm to compute them. The proposed approach is extended to networks with loops and oriented edges. Numerical simulations are carried out on both synthetic and real-world networks to demonstrate the effectiveness and practicability of the proposed algorithm.

Desire for Self-healing

2017 ◽  
Vol 12 (1-2) ◽  
pp. 317-335
Author(s):  
Ngar-sze Lau
Keyword(s):  
Theravada Buddhism ◽  
Self Healing ◽  
Core Knowledge ◽  
The Third ◽  
The Core ◽  
Physical Suffering ◽  
Mind And Body ◽  
Core Practice ◽  
The Relationship ◽  
Contemporary Chinese

Abstract This practice report describes how Chinese meditators understand the “four foundations of mindfulness” (satipaṭṭhāna, sinianzhu 四念住) as a remedy for both mental and physical suffering. In the tradition of Theravāda Buddhism, satipaṭṭhāna is particularly recognized as the core knowledge for understanding the relationship between mind and body, and the core practice leading to liberation from suffering. Based on interviews with Chinese meditation practitioners, this study develops three main themes concerning how they have alleviated afflictions through the practice of satipaṭṭhāna. The first theme highlights how practitioners learn to overcome meditation difficulties with “right attitude.” The second theme is about practicing awareness with “six sense doors” open in order to facilitate the balance of the “five faculties.” The third theme explores how practitioners cultivate daily life practice through an understanding of the nature of mind and body as impermanent and as not-self. This paper details how these themes and embodied practices of satipaṭṭhāna constitute ways of self-healing for urban educated Buddhists in the contemporary Chinese context.

scholarly journals Developing a Delphi-Based Comprehensive Core Set from the International Classification of Functioning, Disability, and Health Framework for the Rehabilitation of Patients with Burn Injuries

2021 ◽  
Vol 18 (8) ◽  
pp. 3970
Author(s):  
Yu-Ru Lin ◽  
Jr-Yi Wang ◽  
Shun-Cheng Chang ◽  
Kwang-Hwa Chang ◽  
Hung-Chou Chen ◽  
...  
Keyword(s):  
Burn Injury ◽  
Functional Limitations ◽  
Rehabilitation Program ◽  
International Classification ◽  
Burn Injuries ◽  
International Classification Of Functioning ◽  
Disability And Health ◽  
The Core ◽  
Core Set

Burn injuries cause disability and functional limitations in daily living. In a 2015 fire explosion in Taiwan, 499 young people sustained burn injuries. The construction of an effective and comprehensive rehabilitation program that enables patients to regain their previous function is imperative. The International Classification of Functioning, Disability, and Health (ICF) includes multiple dimensions that can contribute to meeting this goal. An ICF core set was developed in this study for Taiwanese patients with burns. A consensus process using three rounds of the Delphi technique was employed. A multidisciplinary team of 30 experts from various institutions was formed. The questionnaire used in this study comprised 162 ICF second-level categories relevant to burn injuries. A 5-point Likert scale was used, and participants assigned a weight to the effect of each category on daily activities after burns. The consensus among ratings was assessed using Spearman’s ρ and semi-interquartile range indices. The core set for post-acute SCI was developed from categories that attained a mean score of ≥4.0 in the third round of the Delphi exercise. The core ICF set contained 68 categories. Of these, 19 comprised the component of body functions, 5 comprised body structures, 37 comprised activities and participation, and 7 comprised environmental factors. This preliminary core set offers a comprehensive system for disability assessment and verification following burn injury. The core set provides information for effective rehabilitation strategy setting for patients with burns. Further feasibility and validation studies are required in the future.

scholarly journals An Analysis of Current Trends in CBR Research Using Multi-View Clustering

AI Magazine ◽  
2010 ◽  
Vol 31 (2) ◽  
pp. 45 ◽  
Author(s):  
Derek Greene ◽  
Jill Freyne ◽  
Barry Smyth ◽  
Pádraig Cunningham
Keyword(s):  
Case Based Reasoning ◽  
Research Papers ◽  
The Core ◽  
Current Trends ◽  
Research Challenges ◽  
Core Set ◽  
Research Themes ◽  
Promising Application ◽  
The Moment ◽  
Case Based

The European Conference on Case-Based Reasoning (CBR) in 2008 marked 15 years of international and European CBR conferences where almost seven hundred research papers were published. In this report we review the research themes covered in these papers and identify the topics that are active at the moment. The main mechanism for this analysis is a clustering of the research papers based on both co-citation links and text similarity. It is interesting to note that the core set of papers has attracted citations from almost three thousand papers outside the conference collection so it is clear that the CBR conferences are a sub-part of a much larger whole. It is remarkable that the research themes revealed by this analysis do not map directly to the sub-topics of CBR that might appear in a textbook. Instead they reflect the applications-oriented focus of CBR research, and cover the promising application areas and research challenges that are faced.

scholarly journals THE IMPLEMENTATION OF INDONESIA’S PUBLIC POLICY TO IMMIGRATION SELECTIVE POLICY BASED ON LAW NO. 6 OF 2011

2018 ◽  
Vol 1 (2) ◽  
pp. 169-178
Author(s):  
Muhammad Azzam Alfarizi
Keyword(s):  
Public Policy ◽  
Human Rights ◽  
Service Needs ◽  
Human Beings ◽  
The Third ◽  
The Core ◽  
The Government ◽  
The Individual ◽  
The Right ◽  
Service Standards

The inherent right of the individual is an affirmation that human beings must be treated properly and civilized and must be respected, as the sounding of the second precept is: "Just and Civilized Humanity". Human rights are manifestations of the third principle, namely: "Indonesian Unity". If all rights are fulfilled, reciprocally the unity and integrity will be created. Rights are also protected and upheld as is the agreement of the fourth precepts that reads: "Democracy Led by Wisdom in Consultation / Representation". Human Rights also recognizes the right of every person for the honor and protection of human dignity and dignity, which is in accordance with the fifth precepts which read: "Social Justice for All Indonesian People" PASTI Values ​​which are the core values ​​of the Ministry of Law and Human Rights which is an acronym of Professional, Accountable, Synergistic, Transparent and Innovative is an expression of the performance of the immigration apparatus in providing human rights based services. If these values ​​are in line with the values ​​contained in Pancasila, the criteria for evaluating human rights-based public services are based on the accessibility and availability of facilities; the availability of alert officers and compliance of officials, employees, and implementers of Service Standards for each service area will be easily achieved. It is fitting that immigration personnel in providing services must be in accordance with the principles of human rights-based services and in harmony with the Pancasila philosophy. This is as an endeavor in fulfilling service needs in accordance with the mandate of the 1945 Constitution, provisions of applicable laws and human rights principles for every citizen and population for services provided by the government in this case Immigration.  

Sign in / Sign up

Export Citation Format

Share Document