The New Test Theory

It is shown that a minimal assumption should be added to the assumptions of Classical Test Theory (CTT) in order to have positive inter-item correlations, which are regarded as a basis for the aggregation of items. Moreover, it is shown that the assumption of zero correlations between the error score estimates is substantially violated in the population of individuals when the number of items is small. Instead, a negative correlation between error score estimates occurs. The reason for the negative correlation is that the error score estimates for different items of a scale are based on insufficient true score estimates when the number of items is small. A test of the assumption of uncorrelated error score estimates by means of structural equation modeling (SEM) is proposed that takes this effect into account. The SEM-based procedure is demonstrated by means of empirical examples based on the Edinburgh Handedness Inventory and the Eysenck Personality Questionnaire-Revised.

Download Full-text

The Effect of Alternative Scoring Procedures on the Measurement Properties of a Self-Administered Depression Scale

European Journal of Psychological Assessment ◽

10.1027/1015-5759/a000371 ◽

2019 ◽

Vol 35 (1) ◽

pp. 55-62 ◽

Cited By ~ 1

Author(s):

Noboru Iwata ◽

Akizumi Tsutsumi ◽

Takafumi Wakita ◽

Ryuichi Kumagai ◽

Hiroyuki Noguchi ◽

...

Keyword(s):

Classical Test Theory ◽

Depression Scale ◽

Psychometric Testing ◽

Epidemiologic Studies ◽

Test Theory ◽

Measurement Properties ◽

Irt Model ◽

Response Alternatives ◽

Polytomous Item Response ◽

Θ Point

Abstract. To investigate the effect of response alternatives/scoring procedures on the measurement properties of the Center for Epidemiologic Studies Depression Scale (CES-D) which has the four response alternatives, a polytomous item response theory (IRT) model was applied to the responses of 2,061 workers and university students (1,640 males, 421 females). Test information functions derived from the polytomous IRT analyses on the CES-D data with various scoring procedures indicated that: (1) the CES-D with its standard (0-1-2-3) scoring procedure should be useful for screening to detect subjects with “at high-risk” of depression if the θ point showing the highest information corresponds to the cut-off point, because of its extremely higher information; (2) the CES-D with the 0-1-1-2 scoring procedure could cover wider range of depressive severity, suggesting that this scoring procedure might be useful in cases where more exhaustive discrimination in symptomatology is of interest; and (3) the revised version of CES-D with replacing original positive items into negatively revised items outperformed the original version. These findings have never been demonstrated by the classical test theory analyses, and thus the utility of this kind of psychometric testing should be warranted to further investigation for the standard measures of psychological assessment.

Download Full-text

Using the Rating Scale Model to Examine the Psychometric Properties of the Self-Efficacy Scale for Driver Competence

European Journal of Psychological Assessment ◽

10.1027/1015-5759/a000063 ◽

2011 ◽

Vol 27 (3) ◽

pp. 164-170 ◽

Cited By ~ 1

Author(s):

Anna Sundström

Keyword(s):

Psychometric Properties ◽

Self Efficacy ◽

Rating Scale ◽

Measurement Precision ◽

The Self ◽

Test Theory ◽

Self Report ◽

Scale Model ◽

Validity And Reliability ◽

Two Samples

This study evaluated the psychometric properties of a self-report scale for assessing perceived driver competence, labeled the Self-Efficacy Scale for Driver Competence (SSDC), using item response theory analyses. Two samples of Swedish driving-license examinees (n = 795; n = 714) completed two versions of the SSDC that were parallel in content. Prior work, using classical test theory analyses, has provided support for the validity and reliability of scores from the SSDC. This study investigated the measurement precision, item hierarchy, and differential functioning for males and females of the items in the SSDC as well as how the rating scale functions. The results confirmed the previous findings; that the SSDC demonstrates sound psychometric properties. In addition, the findings showed that measurement precision could be increased by adding items that tap higher self-efficacy levels. Moreover, the rating scale can be improved by reducing the number of categories or by providing each category with a label.

Download Full-text

Is Test Theory Ready for a New Generation of Tests?

Contemporary Psychology ◽

10.1037/034116 ◽

1994 ◽

Vol 39 (4) ◽

pp. 425-426

Author(s):

Elizabeth Bull Danielson

Keyword(s):

Test Theory ◽

New Generation

Download Full-text

Construction and psychometric properties of a computer memory battery using classical test theory and item response theory

PsycEXTRA Dataset ◽

10.1037/e578442014-080 ◽

2011 ◽

Author(s):

Aristides Ferreira

Keyword(s):

Item Response Theory ◽

Psychometric Properties ◽

Item Response ◽

Classical Test Theory ◽

Test Theory ◽

Computer Memory ◽

Response Theory ◽

Classical Test

Download Full-text

Applications of generalizability theory and their relations to classical test theory and structural equation modeling.

Psychological Methods ◽

10.1037/met0000107 ◽

2018 ◽

Vol 23 (1) ◽

pp. 1-26 ◽

Cited By ~ 14

Author(s):

Walter P. Vispoel ◽

Carrie A. Morris ◽

Murat Kilinc

Keyword(s):

Structural Equation Modeling ◽

Structural Equation ◽

Generalizability Theory ◽

Classical Test Theory ◽

Test Theory ◽

Equation Modeling ◽

Classical Test

Download Full-text

Cross-cultural comparability of DSM-IV-criteria for alcohol dependence – Findings from classical and modern test theory

Suchttherapie ◽

10.1055/s-0031-1284664 ◽

2011 ◽

Vol 12 (S 01) ◽

Author(s):

M Steppan ◽

L Kraus ◽

D Piontek ◽

A Pabst

Keyword(s):

Alcohol Dependence ◽

Cross Cultural ◽

Test Theory ◽

Dsm Iv

Download Full-text

Revision of a Criterion-Referenced Vocabulary Test Using Generalizability Theory

JALT Journal - JALT Journal 24.1 ◽

10.37546/jaltjj31.1-4 ◽

2009 ◽

Vol 31 (1) ◽

pp. 81

Author(s):

Takeaki Kumazawa

Keyword(s):

Generalizability Theory ◽

Classical Test Theory ◽

Performance Testing ◽

Error Variance ◽

Test Theory ◽

Vocabulary Test ◽

Multiple Sources ◽

Generalizability Coefficient ◽

Criterion Referenced ◽

Generalizability Study

Classical test theory (CTT) has been widely used to estimate the reliability of measurements. Generalizability theory (G theory), an extension of CTT, is a powerful statistical procedure, particularly useful for performance testing, because it enables estimating the percentages of persons variance and multiple sources of error variance. This study focuses on a generalizability study (G study) conducted to investigate such variance components for a paper-pencil multiple-choice vocabulary test used as a diagnostic pretest. Further, a decision study (D study) was conducted to compute the generalizability coefficient (G coefficient) for absolute decisions. The results of the G and D studies indicated that 46% of the total variance was due to the items effect; further, the G coefficient for absolute decisions was low. 古典的テスト理論は尺度の信頼性を測定するため広く用いられている。古典的テスト理論の応用である一般化可能性理論（G理論）は特にパフォーマンステストにおいて有効な分析手法であり、受験者と誤差の要因となる分散成分の割合を測定することができる。本研究では診断テストとして用いられた多岐選択式語彙テストの分散成分を測定するため一般化可能性研究（G研究）を行った。さらに、決定研究（D研究）では絶対評価に用いる一般化可能性係数を算出した。G研究とD研究の結果、項目の分散成分が全体の分散の46%を占め、また信頼度指数は高くなかった。

Download Full-text