scholarly journals Item Pool Quality Control in Educational Testing: Change Point Model, Compound Risk, and Sequential Detection

2021 ◽  
pp. 107699862110590
Author(s):  
Yunxiao Chen ◽  
Yi-Hsuan Lee ◽  
Xiaoou Li
Keyword(s):  
Psychometric Properties ◽  
Change Point ◽  
Decision Rules ◽  
Educational Testing ◽  
Sequential Decision ◽  
Point Model ◽  
Test Items ◽  
Abrupt Changes ◽  
Change Point Model ◽  
Over Time

In standardized educational testing, test items are reused in multiple test administrations. To ensure the validity of test scores, the psychometric properties of items should remain unchanged over time. In this article, we consider the sequential monitoring of test items, in particular, the detection of abrupt changes to their psychometric properties, where a change can be caused by, for example, leakage of the item or change of the corresponding curriculum. We propose a statistical framework for the detection of abrupt changes in individual items. This framework consists of (1) a multistream Bayesian change point model describing sequential changes in items, (2) a compound risk function quantifying the risk in sequential decisions, and (3) sequential decision rules that control the compound risk. Throughout the sequential decision process, the proposed decision rule balances the trade-off between two sources of errors, the false detection of prechange items, and the nondetection of postchange items. An item-specific monitoring statistic is proposed based on an item response theory model that eliminates the confounding from the examinee population which changes over time. Sequential decision rules and their theoretical properties are developed under two settings: the oracle setting where the Bayesian change point model is completely known and a more realistic setting where some parameters of the model are unknown. Simulation studies are conducted under settings that mimic real operational tests.

A Bayesian Change Point Model for Historical Time Series Analysis

2004 ◽  
Vol 12 (4) ◽  
pp. 354-374 ◽  
Author(s):  
Bruce Western ◽  
Meredith Kleykamp
Keyword(s):  
Time Series ◽  
Change Point ◽  
Regression Coefficients ◽  
Simulation Methods ◽  
Historical Time ◽  
Point Location ◽  
Point Model ◽  
Political Relationships ◽  
Standard Models ◽  
Change Point Model

Political relationships often vary over time, but standard models ignore temporal variation in regression relationships. We describe a Bayesian model that treats the change point in a time series as a parameter to be estimated. In this model, inference for the regression coefficients reflects prior uncertainty about the location of the change point. Inferences about regression coefficients, unconditional on the change-point location, can be obtained by simulation methods. The model is illustrated in an analysis of real wage growth in 18 OECD countries from 1965–1992.

scholarly journals Discriminating between Long Memory and Volatility Shifts

2016 ◽  
Vol 36 (4) ◽  
Author(s):  
Aonan Zhang ◽  
Robertas Gabrys ◽  
Piotr Kokoszka
Keyword(s):  
Long Memory ◽  
Change Point ◽  
Kernel Estimator ◽  
Practical Implementation ◽  
Point Model ◽  
Variance Estimators ◽  
Long Run ◽  
The Us ◽  
Change Point Model ◽  
Stock Indexes

We develop a practical implementation of the test proposed in Berkes, Horv´ath, Kokoszka, and Shao (2006) designed to distinguish between a change-point model and a long memory model. Our implementation is calibrated to distinguish between a shift in volatility of returns and long memory in squared returns. It uses a kernel estimator of the long-run variance of squared returns with the maximal lag selected by a data driven procedure which depends on the sample size, the location of the estimated change point and the direction of the apparent volatility shift (increase versus decrease). In a simulations study, we also consider other long-run variance estimators, including the VARHAC estimator, but we find that they lead to tests with inferior performance. Applied to returns on indexes and individual stocks, our test indicates that even for the same asset, a change-point model may be preferable for a certain period of time, whereas there is evidence of long memory in another period of time. Generally there is stronger evidence for long memory in the eight years ending June 2006 than in the eight years starting January 1992. This pattern is most pronounced for US stock indexes and shares in the US financial sector.

A Bayesian hierarchical change point model with parameter constraints

2020 ◽  
pp. 096228022094809
Author(s):  
Hong Li ◽  
Andreana Benitez ◽  
Brian Neelon
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Cognitive Decline ◽  
Change Point ◽  
Normal Aging ◽  
Point Model ◽  
Bayesian Hierarchical ◽  
Random Slope ◽  
Change Point Model ◽  
Parameter Constraints

Alzheimer’s disease is the leading cause of dementia among adults aged 65 or above. Alzheimer’s disease is characterized by a change point signaling a sudden and prolonged acceleration in cognitive decline. The timing of this change point is of clinical interest because it can be used to establish optimal treatment regimens and schedules. Here, we present a Bayesian hierarchical change point model with a parameter constraint to characterize the rate and timing of cognitive decline among Alzheimer’s disease patients. We allow each patient to have a unique random intercept, random slope before the change point, random change point time, and random slope after the change point. The difference in slope before and after a change point is constrained to be nonpositive, and its parameter space is partitioned into a null region (representing normal aging) and a rejection region (representing accelerated decline). Using the change point time, the estimated slope difference, and the threshold of the null region, we are able to (1) distinguish normal aging patients from those with accelerated cognitive decline, (2) characterize the rate and timing for patients experiencing cognitive decline, and (3) predict personalized risk of progression to dementia due to Alzheimer’s disease. We apply the approach to data from the Religious Orders Study, a national cohort study of aging Catholic nuns, priests, and lay brothers.

Sign in / Sign up

Export Citation Format

Share Document