Learning Processes in Interactive CALL Systems: Linking Automatic Feedback, System Logs, and Learning Outcomes

Interactive digital tools increasingly used for language learning can provide detailed system logs (e.g., number of attempts, responses submitted), and thereby a window into the user’s learning processes. To date, SLA researchers have made little use of such data to understand the relationships between learning conditions, processes, and outcomes. To fill this gap, we analyzed and interpreted detailed logs from an ICALL system used in a randomized controlled field study where 205 German learners of English in secondary school received either general or specific corrective feedback on grammar exercises. In addition to explicit pre-/post-test results, we derived 19 learning process variables from the system log. Exploratory factor analysis revealed three latent factors underlying these process variables: effort,accuracy focus, and time on task. Accuracy focus and finish time (a process variable that did not load well on any factors) significantly predicted pre-/post-test gain scores with a medium effect size. We then clustered learners based on their process patterns and found that the specific feedback group tended to demonstrate particular learning processes and that these patterns moderate the advantage of specific feedback. We discuss the implications of analyzing system logs for SLA, CALL, and education researchers and call for more collaboration.

Anomaly Detection using system logs

Anomaly detection is a very important step in building a secure and trustworthy system. Manually it is daunting to analyze and detect failures and anomalies. In this paper, we proposed an approach that leverages the pattern matching capabilities of Convolution Neural Network (CNN) for anomaly detection in system logs. Features from log files are extracted using a windowing technique. Based on this feature, a one-dimensional image (1×n dimension) is generated where the pixel values of an image correlate with the features of the logs. On these images, the 1D Convolution operation is applied followed by max pooling. Followed by Convolution layers, a multi-layer feed-forward neural network is used as a classifier that learns to classify the logs as normal or abnormal from the representation created by the convolution layers. The model learns the variation in log pattern for normal and abnormal behavior. The proposed approach achieved improved accuracy compared to existing approaches for anomaly detection in Hadoop Distributed File System (HDFS) logs.

Improving prediction of students’ performance in intelligent tutoring systems using attribute selection and ensembles of different multimodal data sources

The aim of this study was to predict university students’ learning performance using different sources of performance and multimodal data from an Intelligent Tutoring System. We collected and preprocessed data from 40 students from different multimodal sources: learning strategies from system logs, emotions from videos of facial expressions, allocation and fixations of attention from eye tracking, and performance on posttests of domain knowledge. Our objective was to test whether the prediction could be improved by using attribute selection and classification ensembles. We carried out three experiments by applying six classification algorithms to numerical and discretized preprocessed multimodal data. The results show that the best predictions were produced using ensembles and selecting the best attributes approach with numerical data.

Can System Logs Enhance the Performance of Credit Scoring? – Evidence from an Internet Bank in Korea

This study is the first to examine whether the performance of credit rating, one of the most important data-based decision-making of banks, can be improved by using banking system log data that is extensively accumulated inside the bank for system operation. This study uses the log data recorded for the mobile app system of Kakaobank, a leading internet bank used by more than 14 million people in Korea. After generating candidate variables from Kakaobank's vast log data, we develop a credit scoring model by utilizing variables with high information values. Consequently, the discrimination power of the new model compared to the credit bureau grades was significantly improved by 1.84% points based on the Kolmogorov–Smirnov statistics. Therefore, the results of this study imply that if a bank utilizes its log data that have already been extensively accumulated inside the bank, decision-making systems, including credit scoring, can be efficiently improved at a low cost.

ConAnomaly: Content-Based Anomaly Detection for System Logs

Enterprise systems typically produce a large number of logs to record runtime states and important events. Log anomaly detection is efficient for business management and system maintenance. Most existing log-based anomaly detection methods use log parser to get log event indexes or event templates and then utilize machine learning methods to detect anomalies. However, these methods cannot handle unknown log types and do not take advantage of the log semantic information. In this article, we propose ConAnomaly, a log-based anomaly detection model composed of a log sequence encoder (log2vec) and multi-layer Long Short Term Memory Network (LSTM). We designed log2vec based on the Word2vec model, which first vectorized the words in the log content, then deleted the invalid words through part of speech tagging, and finally obtained the sequence vector by the weighted average method. In this way, ConAnomaly not only captures semantic information in the log but also leverages log sequential relationships. We evaluate our proposed approach on two log datasets. Our experimental results show that ConAnomaly has good stability and can deal with unseen log types to a certain extent, and it provides better performance than most log-based anomaly detection methods.

Exploring the interplay between question-answering systems and communication with instructors in facilitating learning

PurposeQuestion-answering (QA) systems are being increasingly applied in learning contexts. However, the authors’ understanding of the relationship between such tools and traditional QA channels remains limited. Focusing on question-answering learning activities, the current research investigates the effect of QA systems on students' learning processes and outcomes, as well as the interplay between two QA channels, that is, QA systems and communication with instructors.Design/methodology/approachThe authors designed and implemented a QA system for two university courses, and collected data from questionnaires and system logs that recorded the interaction between students and the system throughout a semester.FindingsThe results show that using a QA system alone does not improve students' learning processes or outcomes. However, the use of a QA system significantly improves the positive effect of instructor communication.Originality/valueThis study contributes to the literature on learning and education technology, and provides practical guidance on how to incorporate QA tools in learning.