Measuring Effectiveness of Graph Visualizations: A Cognitive Load Perspective

2009 ◽  
Vol 8 (3) ◽  
pp. 139-152 ◽  
Author(s):  
Weidong Huang ◽  
Peter Eades ◽  
Seok-Hee Hong
Keyword(s):  
Cognitive Load ◽  
Task Performance ◽  
Performance Measures ◽  
User Study ◽  
Processing System ◽  
Mental Effort ◽  
Human Memory ◽  
Individual Performance ◽  
Cognitive Effort ◽  
Graph Visualization

Graph visualizations are typically evaluated by comparing their differences in effectiveness, measured by task performance such as response time and accuracy. Such performance-based measures have proved to be useful in their own right. There are some situations, however, where the performance measures alone may not be sensitive enough to detect differences. This limitation can be seen from the fact that the graph viewer may achieve the same level of performance by devoting different amounts of cognitive effort. In addition, it is not often that individual performance measures are consistently in favor of a particular visualization. This makes design and evaluation difficult in choosing one visualization over another. In an attempt to overcome the above-mentioned limitations, we measure the effectiveness of graph visualizations from a cognitive load perspective. Human memory as an information processing system and recent results from cognitive load research are reviewed first. The construct of cognitive load in the context of graph visualization is proposed and discussed. A model of user task performance, mental effort and cognitive load is proposed thereafter to further reveal the interacting relations between these three concepts. A cognitive load measure called mental effort is introduced and this measure is further combined with traditional performance measures into a single multi-dimensional measure called visualization efficiency. The proposed model and measurements are tested in a user study for validity. Implications of the cognitive load considerations in graph visualization are discussed.

scholarly journals Analyzing Relationships Between Causal and Assessment Factors of Cognitive Load: Associations Between Objective and Subjective Measures of Cognitive Load, Stress, Interest, and Self-Concept

2021 ◽  
Vol 6 ◽  
Author(s):  
Nina Minkley ◽  
Kate M. Xu ◽  
Moritz Krell
Keyword(s):  
Heart Rate ◽  
Molecular Biology ◽  
Cognitive Load ◽  
Task Performance ◽  
Perceived Stress ◽  
Mental Effort ◽  
Theoretical Framework ◽  
Self Concept ◽  
Mental Load ◽  
Causal Factors

The present study is based on a theoretical framework of cognitive load that distinguishes causal factors (learner characteristics affecting cognitive load e.g., self-concept; interest; perceived stress) and assessment factors (indicators of cognitive load e.g., mental load; mental effort; task performance) of cognitive load. Various assessment approaches have been used in empirical research to measure cognitive load during task performance. The most common methods are subjective self-reported questionnaires; only occasionally objective physiological measures such as heart rates are used. However, the convergence of subjective and objective approaches has not been extensively investigated yet, leaving unclear the meaning of each kind of measure and its validity. This study adds to this body of research by analyzing the relationship between these causal and assessment (subjective and objective) factors of cognitive load. The data come from three comparable studies in which high school students (N = 309) participated in a one-day out of school molecular biology project and completed different tasks about molecular biology structures and procedures. Heart rate variability (objective cognitive load) was measured via a chest belt. Subjective cognitive load (i.e., mental load and mental effort) and causal factors including self-concept, interest, and perceived stress were self-reported by participants on questionnaires. The findings show that a) objective heart rate measures of cognitive load are related to subjective measures of self-reported mental effort but not of mental load; b) self-reported mental effort and mental load are better predictors of task performance than objective heart rate measures of cognitive load; c) self-concept, interest and perceived stress are associated with self-reported measures of mental load and mental effort, and self-concept is associated with one of the objective heart rate measures. The findings are discussed based on the theoretical framework of cognitive load and implications for the validity of each measure are proposed.

Effects of Aging on Subjective Workload and Performance

1992 ◽  
Vol 36 (2) ◽  
pp. 156-160 ◽  
Author(s):  
Douglas L. Boyer ◽  
Jay G. Pollack ◽  
F. Thomas Eggemeier
Keyword(s):  
Task Performance ◽  
Performance Measures ◽  
Older Workers ◽  
Short Term Memory ◽  
Age Groups ◽  
Memory Task ◽  
Mental Effort ◽  
The United States ◽  
Central Processing ◽  
Subjective Workload

Demographics indicate that the population in the United States and other industrialized nations is growing older, and that the number of older workers and systems users can be expected to increase substantially over the next several decades. In order to assess possible differences between age groups the mental workload experienced by older adults as compared to that experienced by younger adults was investigated. Two tasks were utilized to assess short term memory (continuous recognition) and psychomotor (first-order unstable tracking) performance. The workload of each task was assessed with the Subjective Workload Assessment Technique (SWAT). Memory task performance measures and subjective workload ratings indicated a decrement in performance and an increase in workload for the older group relative to the younger group. Psychomotor task performance measures and subjective workload ratings indicated no difference between the age groups. It is hypothesized that the memory task makes greater demands on central processing resources than the psychomotor task used in this study. In support of this hypothesis, an analysis of the changes in ratings on the individual SWAT dimensions of time, mental effort and psychological stress revealed that an increase occurred only on the mental effort dimension for the memory task. This study implies that designers should 1) reduce or provide design features that lessen memory laden task performance for older workers, and 2) give more weight to the reduction of central processing resource requirements in trade-off studies.

Use of the Secondary Task Technique for Tracking User Attention

2006 ◽  
pp. 673-679
Author(s):  
Robert S. Owen
Keyword(s):  
Task Performance ◽  
Dual Task ◽  
Secondary Task ◽  
Processing System ◽  
Mental Effort ◽  
Primary Task ◽  
Finger Tapping ◽  
Mobile Telephones ◽  
Problem Solving Process ◽  
Attention Demands

The notion that the human information processing system has a limit in resource capacity has been used for over 100 years as the basis for the investigation of a variety of constructs and processes, such as mental workload, mental effort, attention, elaboration, information overload, and such. The dual task or secondary task technique presumes that the consumption of processing capacity by one task will leave less capacity available for the processing of a second concurrent task. When both tasks attempt to consume more capacity than is available, the performance of one or both tasks must suffer, and this will presumably result in the observation of degraded task performance. Consider, for example, the amount of mental effort devoted to solving a difficult arithmetic problem. If a person is asked to tap a pattern with a finger while solving the problem, we might be able to discover the more difficult parts of the problem solving process by observing changes in the performance of the secondary task of finger tapping. While a participant is reading a chapter of text in a book or on a Web browser, we might be able to use this same technique to find the more interesting, involving, or confusing passages of the text. Many implementations of the secondary task technique have been used for more than a century, such as the maintenance of hand pressure (Lechner, Bradbury, & Bradley, 1998; Welch, 1898), the maintenance of finger tapping patterns (Friedman, Polson, & Dafoe, 1988; Jastrow, 1892; Kantowitz & Knight, 1976), the performance of mental arithmetic (Bahrick, Noble, & Fitts, 1954; Wogalter & Usher, 1999), and the speed of reaction time to an occasional flash of light, a beep, or a clicking sound (e.g., Bourdin, Teasdale, & Nourgier, 1998; Owen, Lord, & Cooper, 1995; Posener & Bois, 1971). In using the secondary task technique, the participant is asked to perform a secondary task, such as tapping a finger in a pattern, while performing the primary task of interest. By tracking changes in secondary task performance (e.g., observing erratic finger tapping), we can track changes in processing resources being consumed by the primary task. This technique has been used in a wide variety of disciplines and situations. It has been used in advertising to study the effects of more or less suspenseful parts of a TV program on commercials (Owen et al., 1995) and in studying the effects of time-compressed audio commercials (Moore, Hausknecht, & Thamodaran, 1986). It has been used in sports to detect attention demands during horseshoe pitching (Prezuhy & Etnier, 2001) and rock climbing (Bourdin et al., 1998), while others have used it to study attention associated with posture control in patients who are older or suffering from brain disease (e.g., Maylor & Wing, 1996; Muller, Redfern, Furman, & Jennings, 2004). Murray, Holland, and Beason (1998) used a dual task study to detect the attention demands of speaking in people who suffer from aphasia after a stroke. Others have used the secondary task technique to study the attention demands of automobile driving (e.g., Baron & Kalsher, 1998), including the effects of distractions such as mobile telephones (Patten, Kircher, Ostlund, & Nilsson, 2004) and the potential of a fragrance to improve alertness (Schieber, Werner, & Larsen, 2000). Koukounas and McCabe (2001) and Koukounas and Over (1999) have used it to study the allocation of attention resources during sexual arousal. The notion of decreased secondary task performance due to a limited-capacity processing system is not simply a laboratory curiosity. Consider, for example, the crash of a Jetstream 3101 airplane as it was approaching for landing, killing all on board.

scholarly journals Can EEG Be Adopted as a Neuroscience Reference for Assessing Software Programmers’ Cognitive Load?

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2338
Author(s):  
Júlio Medeiros ◽  
Ricardo Couceiro ◽  
Gonçalo Duarte ◽  
João Durães ◽  
João Castelhano ◽  
...  
Keyword(s):  
Software Development ◽  
Cognitive Load ◽  
Mental Effort ◽  
Research Area ◽  
Cognitive State ◽  
Accurate Identification ◽  
Cognitive States ◽  
Code Complexity ◽  
Time Accuracy ◽  
Future Evaluation

An emergent research area in software engineering and software reliability is the use of wearable biosensors to monitor the cognitive state of software developers during software development tasks. The goal is to gather physiologic manifestations that can be linked to error-prone scenarios related to programmers’ cognitive states. In this paper we investigate whether electroencephalography (EEG) can be applied to accurately identify programmers’ cognitive load associated with the comprehension of code with different complexity levels. Therefore, a controlled experiment involving 26 programmers was carried. We found that features related to Theta, Alpha, and Beta brain waves have the highest discriminative power, allowing the identification of code lines and demanding higher mental effort. The EEG results reveal evidence of mental effort saturation as code complexity increases. Conversely, the classic software complexity metrics do not accurately represent the mental effort involved in code comprehension. Finally, EEG is proposed as a reference, in particular, the combination of EEG with eye tracking information allows for an accurate identification of code lines that correspond to peaks of cognitive load, providing a reference to help in the future evaluation of the space and time accuracy of programmers’ cognitive state monitored using wearable devices compatible with software development activities.

On the readability of leaders in boundary labeling

2018 ◽  
Vol 18 (1) ◽  
pp. 110-132
Author(s):  
Lukas Barth ◽  
Andreas Gemsa ◽  
Benjamin Niedermann ◽  
Martin Nöllenburg
Keyword(s):  
Detailed Analysis ◽  
Task Performance ◽  
User Study ◽  
Point Of View ◽  
Practical Point ◽  
Subjective Preference ◽  
Two Measures ◽  
Preference Ratings ◽  
Boundary Labeling

External labeling deals with annotating features in images with labels that are placed outside of the image and are connected by curves (so-called leaders) to the corresponding features. While external labeling has been extensively investigated from a perspective of automatization, the research on its readability has been neglected. In this article, we present the first formal user study on the readability of leader types in boundary labeling, a special variant of external labeling that considers rectangular image contours. We consider the four most studied leader types (straight, L-shaped, diagonal, and S-shaped) with respect to their performance, that is, whether and how fast a viewer can assign a feature to its label and vice versa. We give a detailed analysis of the results regarding the readability of the four models and discuss their aesthetic qualities based on the users’ preference judgments and interviews. As a consequence of our experiment, we can generally recommend L-shaped leaders as the best compromise between measured task performance and subjective preference ratings, while straight and diagonal leaders received mixed ratings in the two measures. S-shaped leaders are generally not recommended from a practical point of view.

scholarly journals Notetaking and Writing from Hypertexts in L1 and L2

2008 ◽  
Vol 156 ◽  
pp. 31-50
Author(s):  
Marie-Laure Barbier ◽  
Annie Piolat ◽  
Jean-Yves Roussey ◽  
Françoise Raby
Keyword(s):  
Cognitive Load ◽  
Word Processor ◽  
Cognitive Effort ◽  
Cognitive Resources ◽  
Main Task ◽  
Writing Processes ◽  
Random Intervals ◽  
Paper Condition ◽  
Copy And Paste ◽  
L1 And L2

This study analyzes the cognitive effort and linguistic procedures of sixty students using information taken from an experimental website in L1 (French) and in L2 (English). The students navigated on the website and took notes on paper or with a word processor. A triple-task paradigm was used to estimate the cognitive load of reading, notetaking, and writing processes in L2. The students had to perform two additional tasks while a main task (notetaking, for example) was being carried out. They had to react as fast as possible to sound signals sent out at random intervals. They also had to identify what they were doing at the time the sound signal was heard (reading, notetaking, or writing). The study focuses on the way the students managed their cognitive resources while exploring the website, selecting and writing down the ideas they considered useful, and reconstructing them later when producing their own text. Surprisingly, no difference in cognitive load was observed between L1 and L2. By relying almost exclusively on the copy and paste functions to retrieve information from the website, the participants using a word processor in L2 succeeded in making reading a less costly activity, and they performed similarly to the notetakers in L1. The students’ difficulties in L2 became apparent only in the paper condition. The strategies and linguistic procedures of the students are described and related to the ways teachers can approach the new dimensions of notetaking and writing with a computer.

scholarly journals Taxing the brain to uncover lying? Meta-analyzing the effect of imposing cognitive load on the reaction-time costs of lying

2018 ◽  
Author(s):  
Bruno Verschuere ◽  
Nils Köbis ◽  
yoella meyer ◽  
David Gertler Rand ◽  
Shaul Shalvi
Keyword(s):  
Reaction Time ◽  
Response Time ◽  
Cognitive Load ◽  
Lie Detection ◽  
Mental Effort ◽  
Cognitive Resources ◽  
Truth Telling ◽  
Total N ◽  
The Difference ◽  
The Brain

Lying typically requires greater mental effort than telling the truth. Imposing cognitive load may improve lie detection by limiting the cognitive resources needed to lie effectively, thereby increasing the difference in speed between truths and lies. We test this hypothesis meta-analytically. Across 21 studies using response-time (RT) paradigms (11 unpublished; total N = 792), we consistently found that truth telling was faster than lying, but found no evidence that imposing cognitive load increased that difference (Control, d = 1.45; Load, d = 1.28). Instead, load significantly decreased the lie-truth RT difference by increasing the RT of truths, g = -.18, p = .027. Our findings therefore suggest that imposing cognitive load does not necessarily improve RT-based lie detection, and may actually worsen it by taxing the mental system and thus impeding people’s ability to easily—and thus quickly—tell the truth

Sign in / Sign up

Export Citation Format

Share Document