For a number of years Dutch foreign language examinations have comprised tests of reading comprehension in only one format: the m.c. question with four options. Now research is being carried out into the feasibility of adding another format: a modified cloze in which deletions are not random but deliberate and in which options are offered. Through these modifications a number of problems adhering to the regular cloze are avoided, and at the same time it is possible to maintain the concept of reading comprehension which has thus far governed the construction of FL examinations, the idea, that is, that reading comprehension is a unitary skill to be measured by questions requiring testees to integrate related information derived from text units larger than the sentence and often from diverse elements in the text.
As it was precisely this that experts doubted cloze could measure, the central question to be answered was: can an 'm.c./ fill-in tesť be constructed which appeals to the same skill as the traditional m.c. format, but which may offer certain advantages (such as greater efficiency).
First of all, the new test format was applied to two texts taken from a regular exam of reading comprehension in EFL, resulting in two tests of 26 items each - in the exam the texts had been accompanied by 11 and 12 questions respectively. Next, in a linguistic analysis, the relations between the items and the text were specified. It was found that each item in the two tests required more understanding of the text than just of the immediate environment of the deletion. The specifications were then submitted to the judgement of language testing experts at Cito, who confirmed the findings in the vast majority of cases. Subsequently, the new tests and the original ones were administered to a population of over 500 Dutch students in order to tests these hypotheses:
1. The psychometric qualities of the m.c./fill in tests are just as good as those of the conventional tests, if not better.
2. The items in the m.c/fill in tests appeal to the same skill as the conventional m.citems.
3. The m.c./fill in tests are more difficult than the corresponding conventional tests.
Hypotheses 1 and 3 were confirmed, allowing these tentative conclusions: the new test format is more efficient than the traditional one and offers testees less undue support in understanding texts. The second hypothesis was not confirmed, as the Rasch model for the three combinations of tests had to be rejected on formal grounds. However, as the model did not constitute an altogether inadequate description of the data, and as combining the various test formats did not affect the measure of fit negatively, it is still hoped that further analysis may show that the two formats do not appeal to fundamentally different dimensions.
APPLYING RASCH MODEL TO MEASURE STUDENTS` READING COMPREHENSION
