Evaluation of a Scenario-Based Socratic Style of Teaching and Learning Practice

This chapter presents the central features of a knowledge-based system, evaluation method, which is deeply rooted to the principle of the Socratic style learning in law school. Software system evaluation is placed in the context of a hybrid legal intelligent tutoring system, Guidance for Business Merger and Acquisition (GBMA) process. The legal knowledge for GBMA is presented in two forms, as rules and previously decided cases. Besides distinguishing the two different forms of knowledge representation, the chapter outlines the actual use of these forms in a computational framework designed to generate a plausible solution for a given case by using rule-based reasoning (RBR) and case-based reasoning (CBR) in an integrated environment. The nature of the suitability assessment of a solution has been considered as a multiple-criteria decision-making process in GBMA evaluation. The evaluation was performed by a combination of discussions and questionnaires with different user groups in a scenario-based teaching and learning practice.

Effectiveness of a numeracy intelligent tutoring system in kindergarten: A conceptual replication

Intelligent Tutoring Systems are a genre of highly adaptive software providing individualized instruction. The current study was a conceptual replication of a previous randomized control trial that incorporated the intelligent tutoring system Native Numbers, a program designed for early numeracy instruction. As a conceptual replication, we kept the method of instruction, the demographics, the number of kindergarten classrooms (n = 3), and the same numeracy and intrinsic motivation screeners as the original study. We changed the time of year of instruction, changed the control group to a wait-control group, added a maintenance assessment for the first group of participants, and included a mathematical language assessment. Analysis of within- and between-group differences using repeated measures ANOVA indicated gains of numeracy were significant only after using Native Numbers (Partial Eta Square = 0.147). Results of intrinsic motivation and mathematical language were not significant. The effect size of numeracy achievement did not reach that of the original study (Partial Eta Square = 0.622). Here, we compared the two studies, discussed plausible reasons for differences in the magnitude of effect sizes, and provided suggestions for future research.

Listening practice for learners of English: towards an intelligent tutoring system

We develop a web-application for practice of listening skills by learners of English, which allows users to listen to pre-recorded sound-files and respond to multiple-choice questions. They receive feedback as to the accuracy of their responses, and they are navigated through the set of items in one of two ways according to the group they are randomly assigned to. Members of a control group are guided from one item to the next depending on success or failure with each new item, and the difficulty ratings of the remaining items. For the experiment group, item selection is made in an adaptive fashion: selecting items through automatic predictions based on individual performance and observations of other students’ interactions with the platform, as well as known item attributes obtained through tagging. Based on the cognitive literature, we also provide listeners with the option of controlling the speed of presentation of the listening items.

Improving Comprehension: Intelligent Tutoring System Explaining the Domain Rules When Students Break Them

Intelligent tutoring systems have become increasingly common in assisting students but are often aimed at isolated subject-domain tasks without creating a scaffolding system from lower- to higher-level cognitive skills, with low-level skills often neglected. We designed and developed an intelligent tutoring system, CompPrehension, which aims to improve the comprehension level of Bloom’s taxonomy. The system features plug-in-based architecture, easily adding new subject domains and learning strategies. It uses formal models and software reasoners to solve the problems and judge the answers, and generates explanatory feedback about the broken domain rules and follow-up questions to stimulate the students’ thinking. We developed two subject domain models: an Expressions domain for teaching the expression order of evaluation, and a Control Flow Statements domain for code-tracing tasks. The chief novelty of our research is that the developed models are capable of automatic problem classification, determining the knowledge required to solve them and so the pedagogical conditions to use the problem without human participation. More than 100 undergraduate first-year Computer Science students took part in evaluating the system. The results in both subject domains show medium but statistically significant learning gains after using the system for a few days; students with worse previous knowledge gained more. In the Control Flow Statements domain, the number of completed questions correlates positively with the post-test grades and learning gains. The students’ survey showed a slightly positive perception of the system.

Refactoring the Whitby Intelligent Tutoring System for Clean Architecture

Whitby is the server-side of an Intelligent Tutoring System application for learning System-Theoretic Process Analysis (STPA), a methodology used to ensure the safety of anything that can be represented with a systems model. The underlying logic driving the reasoning behind Whitby is Situation Calculus, which is a many-sorted logic with situation, action, and object sorts. The Situation Calculus is applied to Ontology Authoring and Contingent Scaffolding: the primary activities within Whitby. Thus many fluents and actions are aggregated in Whitby from these two sub-applications and from Whitby itself, but all are available through a common situation query interface that does not depend upon any of the fluents or actions. Each STPA project in Whitby is a single situation term, which is queried for fluents that include the ontology, and to determine what pedagogical interventions to offer. Initially Whitby was written in Prolog using a module system. In the interest of a cleaner architecture and implementation with improved code reuse and extensibility, the initial application was refactored into Logtalk. This refactoring includes decoupling the Situation Calculus reasoner, Ontology Authoring framework, and Contingent Scaffolding framework into third-party libraries that can be reused in other applications. This extraction was achieved by inverting dependencies via Logtalk protocols and categories, which are reusable interfaces and components that provide functionally cohesive sets of predicate declarations and predicate definitions. In this paper the architectures of two iterations of Whitby are evaluated with respect to the motivations behind the refactor: clean architecture enabling code reuse and extensibility.