Controlling Decremental and Inflationary Effects in Reliability Estimation Resulting from Violations of Assumptions

2001 ◽  
Vol 89 (2) ◽  
pp. 403-424 ◽  
Author(s):  
Gilbert Becker
Keyword(s):  
Opposite Direction ◽  
Measurement Errors ◽  
Classical Test Theory ◽  
Reliability Estimation ◽  
Test Theory ◽  
Measuring Instruments ◽  
Measuring Instrument ◽  
Correlated Errors ◽  
Classical Test ◽  
Practical Level

Two assumptions in classical test theory, essential tau-equivalence and independence of measurement errors, when violated may produce attenuated or inflated estimates of reliability, respectively. Inflation stemming from correlated errors can be controlled by a procedure in which systematically created equivalent halves of a given measuring instrument are administered across two occasions. When poor approximations to equivalent halves are constructed for this purpose, however, distortion in the opposite direction may result, being sometimes quite large when measuring instruments are not essentially tau-equivalent (or, at the practical level, unidimensional). The nature of these decrements are discussed and illustrated, and a number of procedures for eliminating them introduced.

scholarly journals Recent proposals to estimate transient error within the classical test theory tradition

2006 ◽  
Vol 32 (4) ◽  
Author(s):  
G K Huysamen
Keyword(s):  
Classical Test Theory ◽  
Estimation Method ◽  
Reliability Estimation ◽  
Test Theory ◽  
Coefficient Alpha ◽  
Classical Test ◽  
Sources Of Error ◽  
Transient Error ◽  
Different Sources ◽  
Generalisability Theory

Reliability is conceptually defined in terms of consistency across test occasions but coefficient alpha, the most popular reliability estimation method, precludes the examination of such consistency. Three recent proposals to estimate transient error separately within a classical test theory tradition, and the results that they have yielded are reviewed. The merits of these proposals are compared with those of generalisability theory which differentiates between different sources of error variation. Although the procedures reviewed cannot match the advantages of generalisability theory, they may be sufficient in many applications.

On the Far Reaching Effects of Using Biased Estimates of Score Reliability: An Examination of the Problem in 20 Data Analyses

2001 ◽  
Vol 89 (2) ◽  
pp. 291-307 ◽  
Author(s):  
Gilbert Becker
Keyword(s):  
Classical Test Theory ◽  
Test Theory ◽  
Correlated Errors ◽  
Basic Assumptions ◽  
Classical Test ◽  
Data Analyses ◽  
Reliability Estimates ◽  
Score Reliability ◽  
Measurement Units ◽  
Biased Estimates

Violation of either of two basic assumptions in classical test theory may lead to biased estimates of reliability. Violation of the assumption of essential tau-equivalence may produce underestimates, and the presence of correlated errors among measurement units may result in overestimates. The ubiquity of circumstances in which this problem may occur is not fully comprehended by many workers. This article surveys a variety of settings in which biased reliability estimates may be found in an effort to increase awareness of the prevalence of the problem.

On the Integrity of Reliability Estimation in Classical Test Theory: The Case for an Additive Coefficient of Stability

2008 ◽  
Vol 103 (2) ◽  
pp. 545-565
Author(s):  
Gilbert Becker
Keyword(s):  
Classical Test Theory ◽  
Theory Development ◽  
Error Component ◽  
Reliability Estimation ◽  
Test Theory ◽  
Extended Model ◽  
Classical Test ◽  
Scale Reliability ◽  
Recent Developments ◽  
Alternate Forms

This article addresses deficiencies in the most widely used estimators of reliability and draws attention to the reason that this issue is important. Accurate calibration of relationships between constructs is critical to theory development. Unless workers have accurate estimates of scale reliability, accurate estimates of those relationships will not be forthcoming because the classical disattenuation formula requires them. This article shows that classical test theory can easily accommodate the delineation of its error component E in test scores into two sources, inconsistency across content ( E1) and inconsistency across time ( E2). Viewed from this extended model, the alternate forms approach to reliability estimation is complete in that it gauges simultaneously both sources of error. Because that approach is rarely used today for that purpose, the integrity of estimation has been lost. In its place arose estimators of partial reliability—those for estimating generalizability over one medium or the other, but not both, thereby precluding the additivity of error components. Recent developments promise to restore the integrity of the alternate forms approach without the need for alternate forms and suggest an additive alternative to the current nonadditive coefficient of stability.

scholarly journals A Simple Model to Determine the Efficient Duration of Exams

2020 ◽  
pp. 001316442096316
Author(s):  
Jules L. Ellis
Keyword(s):  
Theoretical Model ◽  
Simple Model ◽  
Measurement Errors ◽  
Classical Test Theory ◽  
Theory Model ◽  
Test Theory ◽  
Classical Test ◽  
The Way

This study develops a theoretical model for the costs of an exam as a function of its duration. Two kind of costs are distinguished: (1) the costs of measurement errors and (2) the costs of the measurement. Both costs are expressed in time of the student. Based on a classical test theory model, enriched with assumptions on the context, the costs of the exam can be expressed as a function of various parameters, including the duration of the exam. It is shown that these costs can be minimized in time. Applied in a real example with reliability .80, the outcome is that the optimal exam time would be much shorter and would have reliability .675. The consequences of the model are investigated and discussed. One of the consequences is that optimal exam duration depends on the study load of the course, all other things being equal. It is argued that it is worthwhile to investigate empirically how much time students spend on preparing for resits. Six variants of the model are distinguished, which differ in their weights of the errors and in the way grades affect how much time students study for the resit.

Sign in / Sign up

Export Citation Format

Share Document