Item Pool Quality Control in Educational Testing: Change Point Model, Compound Risk, and Sequential Detection

In standardized educational testing, test items are reused in multiple test administrations. To ensure the validity of test scores, the psychometric properties of items should remain unchanged over time. In this article, we consider the sequential monitoring of test items, in particular, the detection of abrupt changes to their psychometric properties, where a change can be caused by, for example, leakage of the item or change of the corresponding curriculum. We propose a statistical framework for the detection of abrupt changes in individual items. This framework consists of (1) a multistream Bayesian change point model describing sequential changes in items, (2) a compound risk function quantifying the risk in sequential decisions, and (3) sequential decision rules that control the compound risk. Throughout the sequential decision process, the proposed decision rule balances the trade-off between two sources of errors, the false detection of prechange items, and the nondetection of postchange items. An item-specific monitoring statistic is proposed based on an item response theory model that eliminates the confounding from the examinee population which changes over time. Sequential decision rules and their theoretical properties are developed under two settings: the oracle setting where the Bayesian change point model is completely known and a more realistic setting where some parameters of the model are unknown. Simulation studies are conducted under settings that mimic real operational tests.

Synthesis of Plans to Increase the Results of Ordinary National Education Test by All-Round Tutoring Process for Primary School Grade 6

The objective of this research is to study the results of activities to increase the scores of Ordinary National Education Test. Cluster; teachers of Phetchaburi Rajabhat University comparing the results of Ordinary National Education Test in 2017-2018 and studying the satisfaction of the activities. The target group is 49 schools in Phetchaburi and Prachuap Khiri Khan Provinces, data were analyzed using mean and standard deviation. The study found that the difference of the scores of the Ordinary National Education Test was higher in 32 schools and there is a difference in scores of Ordinary National Education Test tests lower by 2 schools, representing 94.12, with the satisfaction of the participation in the activity of increasing the basic educational testing at the basic level is at a high level with an average of 4.22, standard deviations 0.73, which the participants are satisfied with the process. The process of organizing activities was at the highest with an average of 4.28, standard deviations 0.76 and continues organizing activities to increase the scores of Ordinary National Education Test.

Multiracial Individuals and Educational Testing

A literature review focused on quantitative measures and methods regarding multiracial individuals and educational testing revealed that multiracial individuals are uniquely different than monoracial individuals in terms of their racial identity and these unique identities interact with test scores. Until recently, this uniqueness has been ignored by institutions and within the field of educational testing. The uniqueness of multiracial identity should be taken into consideration when using test measures to make decisions for selection and when comparing group outcomes. The review provides a brief picture regarding the history of categorization of multiracial individuals and current research which connects the multiracial experience to test score performance, followed by information on data collection, data coding, data analysis, implications, and recommendations. Suggested methods to address the methodological and analytical challenges of how to categorize multiracial individuals for purposes of group comparisons are challenging and frankly, unsatisfying. Yet, there are some clear recommendations such as allowing individuals to check as many racial/ethnic categories that apply to their identity versus forcing a choice of one race or using “Other” as an option. The limited research regarding multiracial individuals and educational tests supports the need for further research in this field.

Automated Essay Scoring Using Deep Learning Algorithms

The recent transition from paper to digitally based assessment has brought many positive changes in educational testing. For example, many high-stakes exams have started implementing essay-type questions because they allow students to creatively express their understanding with their own words. To reduce the burden of scoring these items, the implementation of automated essay scoring (AES) systems have gained more attention. However, despite some of the successful demonstrations, AES still encountered many criticisms from practitioners. Such concerns often include prediction accuracy and interpretability of the scoring algorithms. Hence, overcoming these challenges is critical for AES to be widely adopted in the field. The purpose of this chapter is to introduce deep learning AES models and to describe how certain aspects of the models can be used to overcome the challenges of prediction accuracy and interpretability of the scoring algorithms.