Assessment of Task-Specific Expertise

Author(s):  
Slava Kalyuga

Main implication of the expertise reversal effect is the need to tailor instructional techniques and procedures to changing levels of learner expertise in a specific task domain. In order to design adaptive procedures capable of tailoring instruction in real-time, it is necessary to have online measures of learner expertise. Such measures should be rapid enough to be used in real time. At the same time, they need to have sufficient diagnostic power to detect different levels of task-specific expertise. One of the previously mentioned reasons for low practical applicability of the results of studies in Aptitude-Treatment Interactions were inadequate aptitude measures. Most of the assessment methods used in those studies were psychometric instruments designed for selection purposes (e.g., large batteries of aptitude tests based on artificially simplified tasks administered mostly in laboratory conditions). Another suggested reason was unsuitability of those methods for dynamic, real-time applications while learners proceeded through a single learning session. This chapter describes a rapid diagnostic approach to the assessment of learner task-specific expertise that has been intentionally designed for rapid online application in adaptive learning environments. The method was developed using an analogy to experimental procedures applied in classical studies of chess expertise mentioned in Chapter I. In those studies, realistic board configurations were briefly presented for subsequent replications. With the described diagnostic approach, learners are briefly presented with a problem situation and required to indicate their first solution step in this problem situation or to rapidly verify suggested steps at various stages of a problem solution procedure.

Author(s):  
Slava Kalyuga

The rapid diagnostic approach to evaluating levels of learner task-specific expertise was introduced in Chapter IV and used in several studies that were subsequently described throughout this book. The rapid diagnostic techniques (first-step method and rapid verification technique) were instrumental in investigating some instances of the expertise reversal effect and in optimizing levels of cognitive load in faded worked example procedures (Section II and Chapter XI). This chapter describes some specific adaptive procedures based on rapid diagnostic methods for evaluating ongoing levels of learner task specific expertise. Two specific approaches to the design of adaptive instruction are considered, adaptive procedures based on rapid measures of performance and adaptive procedures based on combined measures of performance and cognitive load (efficiency measures). The expertise reversal effect established interactions between learner levels of task-specific expertise and effectiveness of different instructional methods. The major instructional implication of this effect is the need to tailor instructional methods and procedures to dynamically changing levels of learner expertise in a specific class of tasks within a domain. The rapid diagnostic approach was successfully used for real-time evaluation of levels of learner task-specific expertise in adaptive online tutorials in the domains of linear algebra equations (Kalyuga & Sweller, 2004; 2005) and vector addition motion problems in kinematics (Kalyuga, 2006) for high school students. Both first step diagnostic method and rapid verification technique were applied in adaptive procedures. According to the rapid assessment-based tailoring approach, these tutorials provided dynamic selection of levels of instructional guidance that were optimal for learners with different levels of expertise based on real-time online measures of these levels. The general designs of those studies were similar. In learner-adapted groups, at the beginning of training sessions, each student was provided with an appropriate level of instructional guidance according to the outcome of the initial rapid pretest. Then during the session, depending on the outcomes of the ongoing rapid tests, the student was allowed to proceed to the next learning stage or was required to repeat the same stage and then take the rapid test again. At each subsequent stage, a lower level of guidance was provided to learners (e.g., worked-out components of solution procedures were gradually omitted and progressively replaced with problem solving steps), and a higher level of the rapid diagnostic tasks was used at the end of the stage. In control non-adapted groups, learners either studied all tasks that were included in the corresponding stages of the training session of their yoked participants, or were required to study the whole set of tasks available in the tutorial.


Author(s):  
Slava Kalyuga

This chapter describes some specific adaptive procedures for tailoring levels of instructional guidance to individual levels of learner task-specific expertise to optimize cognitive resources available to learning. Recent studies in expertise reversal effect that were reviewed in previous chapters indicate that instructional design principles that benefit low-knowledge users may disadvantage more experienced ones. This reversal in the relative effectiveness of different instructional methods is due to the increase in cognitive load that is required for integration of presented supporting information with learners’ available knowledge structures. The major implication of these findings for multimedia design is the need to tailor levels of instructional support to individual levels of learner task-specific expertise. The procedures for adapting levels of instructional guidance suggested in this chapter have been developed in conjunction with empirically established interactions between levels of learner expertise and optimal instructional techniques and procedures. The chapter starts with the description of the processes and approaches to learning complex cognitive skills. The appropriate design models for learning complex skills are reviewed and different ways of varying levels of learner control in such models are described. The relations between levels of learner task-specific expertise and optimal levels of instructional guidance are then discussed. Also, empirical studies of the expertise reversal for instructional guidance and sequencing of learning tasks are reviewed. The completion tasks and faded worked examples are specific instructional methods used in the described studies for managing levels of instructional guidance in adaptive learning environments. Real-time monitoring of levels of learner task-specific expertise using rapid cognitive diagnostic methods was used in some of these studies.


2021 ◽  
Vol 4 ◽  
Author(s):  
Thomas Wilschut ◽  
Florian Sense ◽  
Maarten van der Velde ◽  
Zafeirios Fountas ◽  
Sarah C. Maaß ◽  
...  

Memorising vocabulary is an important aspect of formal foreign-language learning. Advances in cognitive psychology have led to the development of adaptive learning systems that make vocabulary learning more efficient. One way these computer-based systems optimize learning is by measuring learning performance in real time to create optimal repetition schedules for individual learners. While such adaptive learning systems have been successfully applied to word learning using keyboard-based input, they have thus far seen little application in word learning where spoken instead of typed input is used. Here we present a framework for speech-based word learning using an adaptive model that was developed for and tested with typing-based word learning. We show that typing- and speech-based learning result in similar behavioral patterns that can be used to reliably estimate individual memory processes. We extend earlier findings demonstrating that a response-time based adaptive learning approach outperforms an accuracy-based, Leitner flashcard approach in learning efficiency (demonstrated by higher average accuracy and lower response times after a learning session). In short, we show that adaptive learning benefits transfer from typing-based learning, to speech based learning. Our work provides a basis for the development of language learning applications that use real-time pronunciation assessment software to score the accuracy of the learner’s pronunciations. We discuss the implications for our approach for the development of educationally relevant, adaptive speech-based learning applications.


2021 ◽  
Author(s):  
Jiao Yin ◽  
Ming Jian Tang ◽  
Jinli Cao ◽  
Hua Wang ◽  
Mingshan You

2020 ◽  
Vol 10 (2) ◽  
pp. 42
Author(s):  
Othmar Othmar Mwambe ◽  
Phan Xuan Tan ◽  
Eiji Kamioka

Adaptive Educational Hypermedia Systems (AEHS) play a crucial role in supporting adaptive learning and immensely outperform learner-control based systems. AEHS’ page indexing and hyperspace rely mostly on navigation supports which provide the learners with a user-friendly interactive learning environment. Such AEHS features provide the systems with a unique ability to adapt learners’ preferences. However, obtaining timely and accurate information for their adaptive decision-making process is still a challenge due to the dynamic understanding of individual learner. This causes a spontaneous changing of learners’ learning styles that makes hard for system developers to integrate learning objects with learning styles on real-time basis. Thus, in previous research studies, multiple levels navigation supports have been applied to solve this problem. However, this approach destroys their learning motivation because of imposing time and work overload on learners. To address such a challenge, this study proposes a bioinformatics-based adaptive navigation support that was initiated by the alternation of learners’ motivation states on a real-time basis. EyeTracking sensor and adaptive time-locked Learning Objects (LOs) were used. Hence, learners’ pupil size dilation and reading and reaction time were used for the adaption process and evaluation. The results show that the proposed approach improved the AEHS adaptive process and increased learners’ performance up to 78%.


Author(s):  
Muhammad Febrian Rachmadhan Amri ◽  
I Made Sukarsa ◽  
I Ketut Adi Purnawan

The online business era causes the form of transactions to occur so quickly that the information stored in the data warehouse becomes invalid. Companies are required to have a strong system, which is a system that is real time in order to be able to perform data loading into the media repository that resides on different hosts in the near-real time. Data Warehouse is used as a media repository of data that has the nature of subject-oriented, integrated, time-variant, and is fixed. Data Warehouse can be built into real time management with the advantages possessed and utilize Change Data Capture. Change Data Capture (CDC) is a technique that can be used as problem solution to build real time data warehousing (RTDW). The binary log approach in change data capture is made to record any data manipulation activity that occurs at the OLTP level and is managed back before being stored into the Data Warehouse (loading process). This can improve the quality of data management so that the creation of the right information, because the information available is always updated. Testing shows that Binary Log approach in Change Data Capture (BinlogCDC) is able to generate real time data management, valid current information, dynamic communication between systems, and data management without losing any information from data manipulation.


Sign in / Sign up

Export Citation Format

Share Document