User interactions with health insurance decision aids: A user study with retrospective think-aloud (Preprint)

BACKGROUND Two barriers to effective enrollment decisions are low health insurance literacy and lack of knowledge about how to choose a plan. To remedy these issues, digital decision aids have been used to increase the knowledge of plan options and to guide the decision process. Previous research has shown that the way information is presented in a decision aid can impact consumer choice, and existing health insurance decision aids vary in their design, content, and layout. Commercial virtual benefits counselors (VBCs) are digital decision aids that provide decision support by mimicking the guidance provided by an in-person human resources (HR) counselor, whereas more traditional HR websites provide information that requires self-directed navigation through the system. However, few studies have compared how decision processes are impacted by these different methods of providing information. OBJECTIVE This study aims to examine how individuals interact with two different types of health insurance decision aids (<i>guided</i> VBCs that mimic conversations with a real HR counselor and <i>self-directed</i> HR websites that provide a broad range of detailed information) to make employer-provided health insurance decisions. METHODS In total, 16 employees from a local state university completed a user study in which they made mock employer-provided health insurance decisions using 1 of 2 systems (VBC vs HR website). Participants took part in a retrospective think-aloud interview, cued using eye-tracking data to understand decision aid interactions. In addition, pre- and postexperiment measures of literacy and knowledge and decision conflict and usability of the system were also examined. RESULTS Both the VBC and HR website had positive benefits for health insurance knowledge and literacy. Previous health insurance knowledge also impacted how individuals used decision aids. Individuals who scored lower on the pre-experiment knowledge test focused on different decision factors and were more conflicted about their final enrollment decisions than those with higher knowledge test scores. Although both decision aids resulted in similar changes in the Health Insurance Literacy Measure and knowledge test scores, perceived usability differed. Website navigation was not intuitive, and it took longer to locate information, although users appreciated that it had more details; the VBC website was easier to use but had limited information. Lower knowledge participants, in particular, found the website to be less useful and harder to use than those with higher health insurance knowledge. Finally, out-of-pocket cost estimation tools can lead to confusion when they do not highlight the factors that contribute to the cost estimate. CONCLUSIONS This study showed that health insurance decision aids help individuals improve their confidence in selecting and using health insurance plans. However, previous health insurance knowledge plays a significant role in how users interact with and benefit from decision aids, even when information is presented in different formats.

Download Full-text

Improving the Usability of Organizational Data Systems

International Journal of Digital Curation ◽

10.2218/ijdc.v16i1.592 ◽

2021 ◽

Vol 16 (1) ◽

pp. 21

Author(s):

Chung-Yi Hou ◽

Matthew S. Mayernik

Keyword(s):

User Interfaces ◽

User Study ◽

Data Repository ◽

Data Systems ◽

User Interactions ◽

Data Repositories ◽

Web Interfaces ◽

Design And Implementation ◽

Usability Assessment ◽

Assessment Techniques

For research data repositories, web interfaces are usually the primary, if not the only, method that data users have to interact with repository systems. Data users often search, discover, understand, access, and sometimes use data directly through repository web interfaces. Given that sub-par user interfaces can reduce the ability of users to locate, obtain, and use data, it is important to consider how repositories’ web interfaces can be evaluated and improved in order to ensure useful and successful user interactions. This paper discusses how usability assessment techniques are being applied to improve the functioning of data repository interfaces at the National Center for Atmospheric Research (NCAR). At NCAR, a new suite of data system tools is being developed and collectively called the NCAR Digital Asset Services Hub (DASH). Usability evaluation techniques have been used throughout the NCAR DASH design and implementation cycles in order to ensure that the systems work well together for the intended user base. By applying user study, paper prototype, competitive analysis, journey mapping, and heuristic evaluation, the NCAR DASH Search and Repository experiences provide examples for how data systems can benefit from usability principles and techniques. Integrating usability principles and techniques into repository system design and implementation workflows helps to optimize the systems’ overall user experience.

Download Full-text

An Investigation of User Behaviour Consistency for Context-Aware Information Retrieval Systems

International Journal of Advanced Pervasive and Ubiquitous Computing ◽

10.4018/japuc.2009100105 ◽

2009 ◽

Vol 1 (4) ◽

pp. 69-90 ◽

Cited By ~ 1

Author(s):

Adam Grzywaczewski ◽

Rahat Iqbal ◽

Anne James ◽

John Halloran

Keyword(s):

Information Retrieval ◽

User Study ◽

User Interaction ◽

Context Aware ◽

Interaction Patterns ◽

Frequent User ◽

User Interactions ◽

Information Retrieval Systems ◽

Finite Set ◽

Domain Expertise

Users interact with the Internet in dynamic environments that require the IR system to be context aware. Modern IR systems take advantage of user location, browsing history or previous interaction patterns, but a significant number of contextual factors that impact the user information retrieval process are not yet available. Parameters like the emotional state of the user and user domain expertise affect the user experience significantly but are not understood by IR systems. This article presents results of a user study that simplifies the way context in IR and its role in the systems’ efficiency is perceived. The study supports the hypothesis that the number of user interaction contexts and the problems that a particular user is trying to solve is related to lifestyle. Therefore, the IR system’s perception of the interaction context can be reduced to a finite set of frequent user interactions.

Download Full-text

View and Share

International Journal of Mobile Human Computer Interaction ◽

10.4018/jmhci.2010070102 ◽

2010 ◽

Vol 2 (3) ◽

pp. 15-30 ◽

Cited By ~ 2

Author(s):

Andrew Greaves ◽

Enrico Rukzio

Keyword(s):

Mobile Phone ◽

Mobile Devices ◽

User Study ◽

User Feedback ◽

User Interactions ◽

Multiple Users ◽

Single User

Co-present viewing and sharing of images on mobile devices is a popular but very cumbersome activity. Firstly, it is difficult to show a picture to a group of friends due to the small mobile phone screen and secondly it is difficult to share media between multiple friends, e.g., when considering Bluetooth usage and technical limitations, limited input and repeated user interactions. This paper introduces the View & Share system which allows mobile phone users to spontaneously form a group and engage in the viewing and sharing of images. A member of the group has a personal projector (e.g., projector phone) which is used to view pictures collaboratively. View & Share supports sharing with a single user, multiple users or all users, allows members to borrow the projected display and provides a private viewing mode between co-located users. This paper reports on the View & Share system, its implementation and an explorative user study with 12 participants showing the advantages of our system and user feedback.

Download Full-text

Leveraging depth data in remote robot teleoperation interfaces for general object manipulation

The International Journal of Robotics Research ◽

10.1177/0278364919888565 ◽

2019 ◽

Vol 39 (1) ◽

pp. 39-53

Author(s):

David Kent ◽

Carl Saldanha ◽

Sonia Chernova

Keyword(s):

User Study ◽

3D Visualization ◽

Object Manipulation ◽

Degree Of Freedom ◽

Task Completion ◽

User Interactions ◽

Depth Data ◽

Wide Range ◽

Novel Approaches ◽

General Object

Robust remote teleoperation of high-degree-of-freedom manipulators is of critical importance across a wide range of robotics applications. Contemporary robot manipulation interfaces primarily utilize a free positioning pose specification approach to independently control each degree of freedom in free space. In this work, we present two novel interfaces, constrained positioning and point-and-click. Both novel approaches incorporate scene information from depth data into the grasp pose specification process, effectively reducing the number of 3D transformations the user must input. The novel interactions are designed for 2D image streams, rather than traditional 3D virtual scenes, further reducing mental transformations by eliminating the controllable camera viewpoint in favor of fixed physical camera viewpoints. We present interface implementations of our novel approaches, as well as free positioning, in both 2D and 3D visualization modes. In addition, we present results of a 90-participant user study evaluation comparing the effectiveness of each approach for a set of general object manipulation tasks, and the effects of implementing each approach in 2D image views versus 3D depth views. The results of our study show that point-and-click outperforms both free positioning and constrained positioning by significantly increasing the number of tasks completed and significantly reducing task failures and grasping errors, while significantly reducing the number of user interactions required to specify poses. In addition, we found that regardless of the interaction approach, the 2D visualization mode resulted in significantly better performance than the 3D visualization mode, with statistically significant reductions in task failures, grasping errors, task completion time, number of interactions, and user workload, all while reducing bandwidth requirements imposed by streaming depth data.

Download Full-text

arriba-lib: Analyses of user interactions with an electronic library of decision aids on the basis of log data

Informatics for Health and Social Care ◽

10.3109/17538157.2012.654841 ◽

2012 ◽

Vol 37 (4) ◽

pp. 264-276 ◽

Cited By ~ 1

Author(s):

Oliver Hirsch ◽

Elisabeth Szabo ◽

Heidemarie Keller ◽

Lena Kramer ◽

Tanja Krones ◽

...

Keyword(s):

Decision Aids ◽

Electronic Library ◽

User Interactions ◽

Log Data

Download Full-text

View and Share

Social and Organizational Impacts of Emerging Mobile Devices ◽

10.4018/978-1-4666-0194-9.ch009 ◽

2012 ◽

pp. 156-171

Author(s):

Andrew Greaves ◽

Enrico Rukzio

Keyword(s):

Mobile Phone ◽

Mobile Devices ◽

User Study ◽

User Feedback ◽

User Interactions ◽

Multiple Users ◽

Single User

Co-present viewing and sharing of images on mobile devices is a popular but very cumbersome activity. Firstly, it is difficult to show a picture to a group of friends due to the small mobile phone screen and secondly it is difficult to share media between multiple friends, e.g., when considering Bluetooth usage and technical limitations, limited input and repeated user interactions. This paper introduces the View & Share system which allows mobile phone users to spontaneously form a group and engage in the viewing and sharing of images. A member of the group has a personal projector (e.g., projector phone) which is used to view pictures collaboratively. View & Share supports sharing with a single user, multiple users or all users, allows members to borrow the projected display and provides a private viewing mode between co-located users. This paper reports on the View & Share system, its implementation and an explorative user study with 12 participants showing the advantages of our system and user feedback.

Download Full-text

The Impacts of Mutual Collaboration Experience and Domain Knowledge Levels on CAIS Behavior: An Experimental Study

Libri ◽

10.1515/libri-2016-0061 ◽

2017 ◽

Vol 67 (1) ◽

Cited By ~ 1

Author(s):

Dan Wu ◽

Shaobo Liang ◽

Xue Xiang

Keyword(s):

Information Seeking ◽

Domain Knowledge ◽

User Study ◽

Knowledge Level ◽

User Interactions ◽

User Data ◽

Group 3 ◽

Collaborative Information Seeking ◽

Academic Information ◽

Group 2

AbstractSome studies have investigated participants’ information-seeking practices from the perspective of mutual collaboration experience and level of domain knowledge. This study provides insights into collaborative academic information-seeking (CAIS) behaviour. The article explores whether groups with different levels of domain knowledge and mutual collaboration experiences had different CAIS behaviour. It also asked whether domain knowledge level or mutual collaboration experience had an impact on CAIS behaviour. We describe a user study with 18 participants in nine pairs with an experimental collaborative information-seeking tool, the participants categorized into three types of groups: high domain knowledge level and few mutual collaboration experiences (Group 1), low domain knowledge level and many mutual collaboration experiences (Group 2), and no domain knowledge level and no mutual collaboration experience (Group 3). Quantitative and qualitative data analysis were used to analyse the user data collected. The results showed that compared with members of Group 3, participants in Groups 1 and 2 had a better understanding of search tasks and were aware of the ways of completing the tasks successfully. They did not depend on the information-retrieval system when constructing search queries, and adopted diverse cooperation strategies. They were more likely to recommend information to their partners. Domain knowledge had greater impact on CAIS behaviour than collaboration experience. The findings help us to understand social interactions among community members and help CAIS researchers to understand user interactions and inform information system designers as they design collaborative systems to facilitate social communication in the information-seeking process. Our work was limited by the group types we chose and the small group size, which could affect the generalizability of our findings and should be addressed in future studies.

Download Full-text

Understanding credibility judgements for web search snippets

Aslib Journal of Information Management ◽

10.1108/ajim-07-2018-0181 ◽

2019 ◽

Vol 71 (3) ◽

pp. 368-391 ◽

Cited By ~ 3

Author(s):

Markus Kattenbeck ◽

David Elsweiler

Keyword(s):

Search Engine ◽

Web Search ◽

User Study ◽

Think Aloud ◽

Prior Beliefs ◽

Web Based ◽

Content Type ◽

Think Aloud Protocols ◽

Search Engine Result ◽

Informational Cues

Purpose It is well known that information behaviour can be biased in countless ways and that users of web search engines have difficulty in assessing the credibility of results. Yet, little is known about how search engine result page (SERP) listings are used to judge credibility and in which if any way such judgements are biased. The paper aims to discuss these issues. Design/methodology/approach Two studies are presented. The first collects data by means of a controlled, web-based user study (N=105). Studying judgements for three controversial topics, the paper examines the extent to which users agree on credibility, the extent to which judgements relate to those applied by objective assessors and to what extent judgements can be predicted by the users’ position on and prior knowledge of the topic. A second, qualitative study (N=9) utilises the same setup; however, transcribed think-aloud protocols provide an understanding of the cues participants use to estimate credibility. Findings The first study reveals that users are very uncertain when assessing credibility and their impressions often diverge from objective judges who have fact checked the sources. Little evidence is found indicating that judgements are biased by prior beliefs or knowledge, but differences are observed in the accuracy of judgements across topics. Qualitatively analysing think-aloud transcripts from participants think-aloud reveals ten categories of cues, which participants used to determine the credibility of results. Despite short listings, participants utilised diverse cues for the same listings. Even when the same cues were identified and utilised, different participants often interpreted these differently. Example transcripts show how participants reach varying conclusions, illustrate common mistakes made and highlight problems with existing SERP listings. Originality/value This study offers a novel perspective on how the credibility of SERP listings is interpreted when assessing search results. Especially striking is how the same short snippets provide diverse informational cues and how these cues can be interpreted differently depending on the user and his or her background. This finding is significant in terms of how search engine results should be presented and opens up the new challenge of discovering technological solutions, which allow users to better judge the credibility of information sources on the web.

Download Full-text