elicitation studies
Recently Published Documents


TOTAL DOCUMENTS

34
(FIVE YEARS 16)

H-INDEX

8
(FIVE YEARS 3)

2022 ◽  
Vol 29 (1) ◽  
pp. 1-70
Author(s):  
Radu-Daniel Vatavu ◽  
Jacob O. Wobbrock

We clarify fundamental aspects of end-user elicitation, enabling such studies to be run and analyzed with confidence, correctness, and scientific rigor. To this end, our contributions are multifold. We introduce a formal model of end-user elicitation in HCI and identify three types of agreement analysis: expert , codebook , and computer . We show that agreement is a mathematical tolerance relation generating a tolerance space over the set of elicited proposals. We review current measures of agreement and show that all can be computed from an agreement graph . In response to recent criticisms, we show that chance agreement represents an issue solely for inter-rater reliability studies and not for end-user elicitation, where it is opposed by chance disagreement . We conduct extensive simulations of 16 statistical tests for agreement rates, and report Type I errors and power. Based on our findings, we provide recommendations for practitioners and introduce a five-level hierarchy for elicitation studies.


2022 ◽  
Vol 8 ◽  
Author(s):  
Anastasia K. Ostrowski ◽  
Jenny Fu ◽  
Vasiliki Zygouras ◽  
Hae Won Park ◽  
Cynthia Breazeal

As voice-user interfaces (VUIs), such as smart speakers like Amazon Alexa or social robots like Jibo, enter multi-user environments like our homes, it is critical to understand how group members perceive and interact with these devices. VUIs engage socially with users, leveraging multi-modal cues including speech, graphics, expressive sounds, and movement. The combination of these cues can affect how users perceive and interact with these devices. Through a set of three elicitation studies, we explore family interactions (N = 34 families, 92 participants, ages 4–69) with three commercially available VUIs with varying levels of social embodiment. The motivation for these three studies began when researchers noticed that families interacted differently with three agents when familiarizing themselves with the agents and, therefore, we sought to further investigate this trend in three subsequent studies designed as a conceptional replication study. Each study included three activities to examine participants’ interactions with and perceptions of the three VUIS in each study, including an agent exploration activity, perceived personality activity, and user experience ranking activity. Consistent for each study, participants interacted significantly more with an agent with a higher degree of social embodiment, i.e., a social robot such as Jibo, and perceived the agent as more trustworthy, having higher emotional engagement, and having higher companionship. There were some nuances in interaction and perception with different brands and types of smart speakers, i.e., Google Home versus Amazon Echo, or Amazon Show versus Amazon Echo Spot between the studies. In the last study, a behavioral analysis was conducted to investigate interactions between family members and with the VUIs, revealing that participants interacted more with the social robot and interacted more with their family members around the interactions with the social robot. This paper explores these findings and elaborates upon how these findings can direct future VUI development for group settings, especially in familial settings.


2021 ◽  
Vol 5 (ISS) ◽  
pp. 1-23
Author(s):  
Marco Moran-Ledesma ◽  
Oliver Schneider ◽  
Mark Hancock

When interacting with virtual reality (VR) applications like CAD and open-world games, people may want to use gestures as a means of leveraging their knowledge from the physical world. However, people may prefer physical props over handheld controllers to input gestures in VR. We present an elicitation study where 21 participants chose from 95 props to perform manipulative gestures for 20 CAD-like and open-world game-like referents. When analyzing this data, we found existing methods for elicitation studies were insufficient to describe gestures with props, or to measure agreement with prop selection (i.e., agreement between sets of items). We proceeded by describing gestures as context-free grammars, capturing how different props were used in similar roles in a given gesture. We present gesture and prop agreement scores using a generalized agreement score that we developed to compare multiple selections rather than a single selection. We found that props were selected based on their resemblance to virtual objects and the actions they afforded; that gesture and prop agreement depended on the referent, with some referents leading to similar gesture choices, while others led to similar prop choices; and that a small set of carefully chosen props can support multiple gestures.


2021 ◽  
pp. 1-27
Author(s):  
PETER COLLINS

This article aims to provide a fresh approach to the study of hypercorrection, the misguided application of a real or imagined rule – typically in response to prescriptive pressure – in which the speaker's attempt to be ‘correct’ leads to an ‘incorrect’ result. Instead of more familiar sources of information on hypercorrection such as attitude elicitation studies and prescriptive commentary, insights are sought from quantitative and qualitative data extracted from the 2-billion-word Global Web-based English corpus (GloWbE; Davies 2013). Five categories are investigated: case-marked pronouns, -ly and non-ly adverbs, agreement with number-transparent nouns, (extended uses of) irrealis were, and ‘hyperforeign’ noun suffixation. The nature and extent of hypercorrection in these categories, across the twenty English varieties represented in GloWbE, are investigated and discussed. Findings include a tendency for hypercorrection to be more common in American than in British English, and more prevalent in the ‘Inner Circle’ (IC) than in the ‘Outer Circle’ (OC) varieties (particularly with established constructions which have been the target of institutionalised prescriptive commentary over a long period of time).


2021 ◽  
pp. 82-91
Author(s):  
Andrea Bellucci ◽  
Telmo Zarraonandia ◽  
Paloma Díaz ◽  
Ignacio Aedo

2020 ◽  
Vol 4 (4) ◽  
pp. 88
Author(s):  
Adam S. Williams ◽  
Jason Garcia ◽  
Fernando De Zayas ◽  
Fidel Hernandez ◽  
Julia Sharp ◽  
...  

Gesture elicitation studies are a popular means of gaining valuable insights into how users interact with novel input devices. One of the problems elicitation faces is that of legacy bias, when elicited interactions are biased by prior technologies use. In response, methodologies have been introduced to reduce legacy bias. This is the first study that formally examines the production method of reducing legacy bias (i.e., repeated proposals for a single referent). This is done through a between-subject study that had 27 participants per group (control and production) with 17 referents placed in a virtual environment using a head-mounted display. This study found that over a range of referents, legacy bias was not significantly reduced over production trials. Instead, production reduced participant consensus on proposals. However, in the set of referents that elicited the most legacy biased proposals, production was an effective means of reducing legacy bias, with an overall reduction of 11.93% for the chance of eliciting a legacy bias proposal.


Author(s):  
Santiago Villarreal-Narvaez ◽  
Jean Vanderdonckt ◽  
Radu-Daniel Vatavu ◽  
Jacob O. Wobbrock

2020 ◽  
Author(s):  
Ru Han ◽  
Clément François ◽  
Mondher Toumi

AbstractBackgroundHealth state utility values (HSUVs) identified from utility elicitation studies are widely used in pharmacoeconomic evaluations for chronic hepatitis C (CHC) and are particularly instrumental in health technology assessment (HTA) evaluation like the National Institute for Health and Clinical Excellence (NICE).ObjectiveThe objective of this study is to identify HSUVs used in cost-utility analyses (CUAs) for CHC in Europe and evaluate the impact of HSUVs selection on cost-effectiveness results in terms of incremental cost per quality-adjusted life-year (QALY) gained (ICER).MethodsA systematic search of pharmacoeconomic evaluations for CHC was updated in Medline and Embase from the period of 2012-2017 to the period of 2017-2020. Data on health states, HSUVs and utility elicitation studies were extracted. The difference in HSUVs of the same health state in different CUAs and the difference between HSUVs of one health state and of the interlink health state in the same CUAs were calculated. A quality assessment was performed to evaluate the selection of HSUVs in CUAs. Sets of HSUVs identified were used in a re-constructed CUA model to assess the impact on ICER.ResultsTwenty-six CUAs conducted in European countries and referring to 17 utility elicitation studies were included. The difference in HSUVs of the same health states in different CUAs ranged from 0.021 (liver transplant) to 0.468 (decompensated cirrhosis). The difference between HSUVs of one health state and of the interlink health state of next disease severity level was calculated between health state of F0-F1/mild and F2-F3/moderate (n=11, 0.040 to 0.110), F2-F3/moderate and F4/compensated cirrhosis (n=18, 0.027 to 0.130), compensated cirrhosis and decompensated cirrhosis (n=22, 0.020 to 0.100), decompensated cirrhosis and hepatocellular carcinoma (n=24, 0.000 to 0.200), hepatocellular carcinoma and liver transplant in the first year (n=17, −0.329 to 0.170) and liver transplant in the first year and in subsequent years (n=17, −0.340 to 0.000). The utility elicitation study selected by most CUAs (n=11)was recommended as the source of HSUVs, as least for the CUAs conducted in the UK, based on the results of quality assessment. Seven sets of HSUVs were generated to fit the re-constructed model and changed the results of incremental analysis from being cost-effective to not cost-effective (ICER raging from £2,460 to £24,954 per QALY gained), and to dominated in the UK setting.ConclusionsThe CUAs for CHC were found to apply various HSUVs from different utility elicitation studies in the same health state. This variability of HSUVs has the potential to significantly affect ICER and ICER-based reimbursement decision. A rigorous selection of HSUVs in CUAs to inform healthcare resource allocation is suggested for future studies of CUAs and guideline development.


2020 ◽  
pp. 1-27
Author(s):  
Sara MORADLOU ◽  
Xiaobei ZHENG ◽  
Ye TIAN ◽  
Jonathan GINZBURG

Abstract In this paper we consider the order of emergence of comprehension of wh-questions and polar-questions. We argue that considerations of complexity and input favour the earlier emergence of polar questions; on the other hand, if one assumes that question understanding emerges as a consequence of interactive learning this favours (certain) wh-questions, as well as a small subclass of polar questions. We offer corpus evidence from the Providence corpus that (a certain class of) wh-questions are in fact understood earlier than the polar-questions. We test this observation using elicitation studies on German and Chinese speaking children. Our results confirm the finding from the corpus study and are in line with an interactive learning perspective for the emergence of understanding of questions.


Sign in / Sign up

Export Citation Format

Share Document