Evaluation of Cognitive Load

2009 ◽  
pp. 101-119
Author(s):  
Slava Kalyuga
Keyword(s):  
Cognitive Load ◽  
Learning Environments ◽  
Rating Scales ◽  
Instructional Materials ◽  
Subjective Rating ◽  
Instructional Efficiency ◽  
General Evaluation ◽  
Learning Conditions ◽  
Schematic Knowledge ◽  
Evaluation Of Learning

Availability of valid and usable measures of cognitive load involved in learning is essential for providing support for cognitive load-based explanations of the effects predicted and described in cognitive load theory as well as for general evaluation of learning conditions. Besides, the evaluation of cognitive load may provide another indicator of levels of learner expertise in addition to performance scores. As mentioned before, due to the available schematic knowledge base, more knowledgeable learners are expected to perform their tasks with lower mental effort than novices. Even though simple subjective rating scales remain the most often used measures of cognitive load imposed by instructional materials, new more sophisticated techniques are being developed, especially in multimodal environments associated with performance of complex cognitive tasks. This chapter provides a brief overview of traditional, as well as some novel methods for measuring and evaluating cognitive load. Some recently developed approaches to using these measures in estimating instructional efficiency of learning environments are also discussed.

Evaluating and Managing Cognitive Load in Games

2009 ◽  
pp. 719-737 ◽  
Author(s):  
Slava Kalyuga ◽  
Jan L. Plass
Keyword(s):  
Cognitive Load ◽  
Rating Scales ◽  
Educational Games ◽  
Cognitive Architecture ◽  
Human Mind ◽  
Subjective Rating ◽  
Cognitive Resources ◽  
Verbal Protocols ◽  
Game Based Learning ◽  
Cognitive Limitations

This chapter provides an overview of our cognitive architecture and its implications for the design of game-based learning environments. Design of educational technologies should take into account how the human mind works and what its cognitive limitations are. Processing limitations of working memory, which becomes overloaded if more than a few chunks of information are processed simultaneously, represent a major factor influencing the effectiveness of learning in educational games. The chapter describes different types and sources of cognitive load and the specific demands of games on cognitive resources. It outlines information presentation design methods for dealing with potential cognitive overload, and presents some techniques (subjective rating scales, dual-task techniques, and concurrent verbal protocols) that could be used for evaluating cognitive load in electronic gaming in education.

scholarly journals Comparing Two Subjective Rating Scales Assessing Cognitive Load During Technology-Enhanced STEM Laboratory Courses

2021 ◽  
Vol 6 ◽  
Author(s):  
Michael Thees ◽  
Sebastian Kapp ◽  
Kristin Altmeyer ◽  
Sarah Malone ◽  
Roland Brünken ◽  
...  
Keyword(s):  
Cognitive Load ◽  
Rating Scales ◽  
Measurement Data ◽  
Subjective Rating ◽  
Universal Method ◽  
Factorial Structure ◽  
Electric Circuits ◽  
Laboratory Courses ◽  
Extraneous Load ◽  
The Given

Cognitive load theory is considered universally applicable to all kinds of learning scenarios. However, instead of a universal method for measuring cognitive load that suits different learning contexts or target groups, there is a great variety of assessment approaches. Particularly common are subjective rating scales, which even allow for measuring the three assumed types of cognitive load in a differentiated way. Although these scales have been proven to be effective for various learning tasks, they might not be an optimal fit for the learning demands of specific complex environments such as technology-enhanced STEM laboratory courses. The aim of this research was therefore to examine and compare the existing rating scales in terms of validity for this learning context and to identify options for adaptation, if necessary. For the present study, the two most common subjective rating scales that are known to differentiate between load types (the cognitive load scale by Leppink et al. and the naïve rating scale by Klepsch et al.) were slightly adapted to the context of learning through structured hands-on experimentation where elements such as measurement data, experimental setups, and experimental tasks affect knowledge acquisition. N = 95 engineering students performed six experiments examining basic electric circuits where they had to explore fundamental relationships between physical quantities based on the observed data. Immediately after the experimentation, the students answered both adapted scales. Various indicators of validity, which considered the scales’ internal structure and their relation to variables such as group allocation as participants were randomly assigned to two conditions with a contrasting spatial arrangement of the measurement data, were analyzed. For the given dataset, the intended three-factorial structure could not be confirmed, and most of the a priori-defined subscales showed insufficient internal consistency. A multitrait–multimethod analysis suggests convergent and discriminant evidence between the scales which could not be confirmed sufficiently. The two contrasted experimental conditions were expected to result in different ratings for the extraneous load, which was solely detected by one adapted scale. As a further step, two new scales were assembled based on the overall item pool and the given dataset. They revealed a three-factorial structure in accordance with the three types of load and seemed to be promising new tools, although their subscales for extraneous load still suffer from low reliability scores.

scholarly journals Measuring Cognitive Load: Are There More Valid Alternatives to Likert Rating Scales?

2021 ◽  
Vol 6 ◽  
Author(s):  
Kim Ouwehand ◽  
Avalon van der Kroef ◽  
Jacqueline Wong ◽  
Fred Paas
Keyword(s):  
Problem Solving ◽  
Cognitive Load ◽  
Cognitive Processes ◽  
Rating Scales ◽  
Complex Problem ◽  
Simple Problem ◽  
Subjective Rating ◽  
Visual Appearance ◽  
Research Findings ◽  
Theory Research

Cognitive load researchers have used varying subjective techniques based on rating scales to quantify experienced cognitive load. Although it is generally assumed that subjects can introspect on their cognitive processes and have no difficulty in assigning numerical values to the imposed cognitive load, little is known about how visual characteristics of the rating scales influence the validity of the cognitive load measure. In this study we look at validity of four subjective rating scales (within groups) differing in visual appearance by participants rating perceived difficulty and invested mental effort in response to working on simple and complex weekday problems. We used two numerical scales (the nine-point Likert scale most often used in Cognitive load theory research and a Visual Analogue Scale ranging between 0–100%) and two pictorial scales (a scale consisting of emoticons ranging from a relaxed blue-colored face to a stressed red-colored face and an “embodied” scale picturing nine depicted weights from 1–9 kg). Results suggest that numerical scales better reflect cognitive processes underlying complex problem solving while pictorial scales Underlying simple problem solving. This study adds to the discussion on the challenges to quantify cognitive load through various measurement methods and whether subtleties in measurements could influence research findings.

Measuring cognitive load with subjective rating scales during problem solving: differences between immediate and delayed ratings

2014 ◽  
Vol 43 (1) ◽  
pp. 93-114 ◽  
Author(s):  
Annett Schmeck ◽  
Maria Opfermann ◽  
Tamara van Gog ◽  
Fred Paas ◽  
Detlev Leutner
Keyword(s):  
Problem Solving ◽  
Cognitive Load ◽  
Rating Scales ◽  
Subjective Rating

Attention Deficit and Disruptive Behavior Disorder Symptoms

2001 ◽  
Vol 17 (1) ◽  
pp. 25-35 ◽  
Author(s):  
G. Leonard Burns ◽  
James A. Walsh ◽  
David R. Patterson ◽  
Carol S. Holte ◽  
Rita Sommers-Flanagan ◽  
...  
Keyword(s):  
Attention Deficit ◽  
Rating Scales ◽  
Behavior Disorder ◽  
Disruptive Behavior Disorder ◽  
Subjective Rating ◽  
Time Interval ◽  
Assessment Procedure ◽  
Frequency Count ◽  
Hyperactivity Disorder ◽  
Oppositional Defiant

Summary: Rating scales are commonly used to measure the symptoms of attention deficit/hyperactivity disorder (ADHD), oppositional defiant disorder (ODD), and conduct disorder (CD). While these scales have positive psychometric properties, the scales share a potential weakness - the use of vague or subjective rating procedures to measure symptom occurrence (e. g., never, occasionally, often, and very often). Rating procedures based on frequency counts for a specific time interval (e. g., never, once, twice, once per month, once per week, once per day, more than once per day) are less subjective and provide a conceptually better assessment procedure for these symptoms. Such a frequency count procedure was used to obtain parent ratings on the ADHD, ODD, and CD symptoms in a normative (nonclinical) sample of 3,500 children and adolescents. Although the current study does not provide a direct comparison of the two types of rating procedures, the results suggest that the frequency count procedure provides a potentially more useful way to measure these symptoms. The implications of the results are noted for the construction of rating scales to measure the ADHD, ODD, and CD symptoms.

DESIGNING FOR AUTISM: AN ASPECTSS™ POST-OCCUPANCY EVALUATION OF LEARNING ENVIRONMENTS

2018 ◽  
Vol 12 (3) ◽  
pp. 308 ◽  
Author(s):  
Magda Mostafa
Keyword(s):  
Learning Environments ◽  
Children With Autism ◽  
Autism Spectrum ◽  
Built Environments ◽  
Outdoor Learning ◽  
Learning Spaces ◽  
Post Occupancy Evaluation ◽  
8Th Grade ◽  
Evaluation Of Learning

The objective of this paper is to demonstrate the application of the Autism ASPECTSS™ Design Index in the Post-Occupancy Evaluation of existing learning environments for children along the autism spectrum. First published in 2014 this index outlines 7 design criteria that have been hypothesized to support environments conducive of learning for children with autism spectrum disorder (ASD). Using the index as a framework, this paper outlines a case study of a Post-Occupancy Evaluation (POE) of an existing pre-K-8th grade public charter purpose-built school for children on the autism spectrum. The tools used for the evaluation were: the ASPECTSS scoring of the school through a survey of teachers and administrators; on-site behavioral in-class observation; and focus groups of parents, teachers, staff and administrators. The results informed a design retro-fit proposal that strived to assess any ASPECTSS compliance issues and implement the index across the learning spaces, therapy spaces, support services and outdoor learning environments of the school. This paper will outline the application of the index and the resultant design from this process. The results will strive to present a scalable and replicable methodology and prototype for improving existing built environments for learners with ASD.

scholarly journals Remote monitoring technologies in Alzheimer’s disease: design of the RADAR-AD study

2021 ◽  
Vol 13 (1) ◽  
Author(s):  
Marijn Muurling ◽  
◽  
Casper de Boer ◽  
Rouba Kozak ◽  
Dorota Religa ◽  
...  
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Remote Monitoring ◽  
Rating Scales ◽  
Functional Decline ◽  
Cognitive Domain ◽  
Subjective Rating ◽  
Functional Domain ◽  
Smartphone Applications ◽  
Monitoring Technologies

Abstract Background Functional decline in Alzheimer’s disease (AD) is typically measured using single-time point subjective rating scales, which rely on direct observation or (caregiver) recall. Remote monitoring technologies (RMTs), such as smartphone applications, wearables, and home-based sensors, can change these periodic subjective assessments to more frequent, or even continuous, objective monitoring. The aim of the RADAR-AD study is to assess the accuracy and validity of RMTs in measuring functional decline in a real-world environment across preclinical-to-moderate stages of AD compared to standard clinical rating scales. Methods This study includes three tiers. For the main study, we will include participants (n = 220) with preclinical AD, prodromal AD, mild-to-moderate AD, and healthy controls, classified by MMSE and CDR score, from clinical sites equally distributed over 13 European countries. Participants will undergo extensive neuropsychological testing and physical examination. The RMT assessments, performed over an 8-week period, include walk tests, financial management tasks, an augmented reality game, two activity trackers, and two smartphone applications installed on the participants’ phone. In the first sub-study, fixed sensors will be installed in the homes of a representative sub-sample of 40 participants. In the second sub-study, 10 participants will stay in a smart home for 1 week. The primary outcome of this study is the difference in functional domain profiles assessed using RMTs between the four study groups. The four participant groups will be compared for each RMT outcome measure separately. Each RMT outcome will be compared to a standard clinical test which measures the same functional or cognitive domain. Finally, multivariate prediction models will be developed. Data collection and privacy are important aspects of the project, which will be managed using the RADAR-base data platform running on specifically designed biomedical research computing infrastructure. Results First results are expected to be disseminated in 2022. Conclusion Our study is well placed to evaluate the clinical utility of RMT assessments. Leveraging modern-day technology may deliver new and improved methods for accurately monitoring functional decline in all stages of AD. It is greatly anticipated that these methods could lead to objective and real-life functional endpoints with increased sensitivity to pharmacological agent signal detection.

Sign in / Sign up

Export Citation Format

Share Document