The Role of Order and Sequence of Options in Multiple-choice Questions for High-stakes Tests of English Language Proficiency

2020 ◽  
Vol 9 (6) ◽  
pp. 110
Author(s):  
Talip Karanfil ◽  
Steve Neufeld
Keyword(s):  
Language Proficiency ◽  
English Language ◽  
Multiple Choice ◽  
High Volume ◽  
English Language Proficiency ◽  
Proficiency Tests ◽  
Multiple Choice Questions ◽  
High Stakes ◽  
Test Items ◽  
Two Factors

High-stakes and high-volume English language proficiency tests typically rely on multiple-choice questions (MCQs) to assess reading and listening skills. Due to the Covid-19 pandemic, more institutions are using MCQs via online assessment platforms, which facilitate shuffling the order of options within test items to minimize cheating. There is scant research on the role that order and sequence of options plays in MCQs, so this study examined the results of a paper-based, high-stakes English proficiency test administered in two versions. Each version had identical three-option MCQs but with different ordering of options. The test-takers were chosen to ensure a very similar profile of language ability and level for the groups who took the two versions. The findings indicate that one in four questions exhibited significantly different levels of difficulty and discrimination between the two versions. The study identifies order dominance and sequence priming as two factors that influence the outcomes of MCQs, both of which can accentuate or diminish the power of attraction of the correct and incorrect options. These factors should be carefully considered when designing MCQs in high-stakes language proficiency tests and shuffling of options in either paper-based or computer-based testing.

A comparison of reliability and precision of subscore reporting methods for a state English language proficiency assessment

2017 ◽  
Vol 35 (2) ◽  
pp. 297-317 ◽  
Author(s):  
Tanya Longabach ◽  
Vicki Peyton
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Language Proficiency ◽  
English Language ◽  
English Language Proficiency ◽  
Test Theory ◽  
Policy Implications ◽  
Proficiency Tests ◽  
Response Theory ◽  
K 12

K–12 English language proficiency tests that assess multiple content domains (e.g., listening, speaking, reading, writing) often have subsections based on these content domains; scores assigned to these subsections are commonly known as subscores. Testing programs face increasing customer demands for the reporting of subscores in addition to the total test scores in today’s accountability-oriented educational environment. Although reporting subscores can provide much-needed information for teachers, administrators, and students about proficiency in the test domains, one of the major drawbacks of subscore reporting includes their lower reliability as compared to the test as a whole. In addition, viewing language domains as if they were not interrelated, and reporting subscores without considering this relationship between domains, may be contradictory to the theory of language acquisition. This study explored several methods of assigning subscores to the four domains of a state English language proficiency test, including classical test theory (CTT)-based number correct, unidimensional item response theory (UIRT), augmented item response theory (A-IRT), and multidimensional item response theory (MIRT), and compared the reliability and precision of these different methods across language domains and grade bands. The first two methods assessed proficiency in the domains separately, without considering the relationship between domains; the last two methods took into consideration relationships between domains. The reliability and precision of the CTT and UIRT methods were similar and lower than those of A-IRT and MIRT for most domains and grade bands; MIRT was found to be the most reliable method. Policy implications and limitations of this study, as well as directions for further research, are discussed.

High Stakes Testing: Issues, Implications, and Remedies for English Language Learners

2008 ◽  
Vol 78 (2) ◽  
pp. 260-329 ◽  
Author(s):  
Ronald W. Solórzano
Keyword(s):  
English Language Learners ◽  
No Child Left Behind ◽  
Language Learners ◽  
Language Proficiency ◽  
English Language ◽  
High Stakes Testing ◽  
Proficiency Tests ◽  
High Stakes ◽  
Left Behind ◽  
High Stakes Tests

This article discusses the issues and implications of high stakes tests on English language learners (ELLs). As ELLs are being included in all high stakes assessments tied to accountability efforts (e.g., No Child Left Behind), it is crucial that issues related to the tests be critically evaluated relative to their use. In this case, academic achievement tests are analyzed relative to their norming samples and validity to determine their usefulness to ELLs. Also, commonly used language proficiency tests are examined relative to definitions of proficiency, technical quality, alignment with criteria for language classification and reclassification, and their academic predictive validity. Based on the synthesis of the literature, the author concludes that high stakes tests as currently constructed are inappropriate for ELLs, and most disturbing is their continued use for high stakes decisions that have adverse consequences. The author provides recommendations for addressing the issues related to high stakes tests and ELLs.

scholarly journals Introducing and Testing a Measurement Tool for English Language Proficiency: Aisha’s Tool

2019 ◽  
Vol 9 (3) ◽  
pp. 319
Author(s):  
Aisha M. Alhussain
Keyword(s):  
Language Proficiency ◽  
English Language ◽  
English Proficiency ◽  
English Language Proficiency ◽  
Measurement Tool ◽  
Proficiency Tests ◽  
Test Results ◽  
High Performing ◽  
Highly Effective ◽  
Study Participants

The IELTS English proficiency tests are discussed as being highly effective in determining students’ level of proficiency in the language. However, the study points out that the processes involved in the administration of the tests along with the associated cost make affect the effectiveness of its use in the assessment of learners. A Sentence Pattern test is offered as an alternative with 97 participants taking part in the assessment to test its effectiveness. Each of the non-native study participants is subjected to both the SP test and the IELTS test for the establishment of the correlation in the results posted for the two tests. The findings demonstrate that the students’ performance in the IELTS test correlated with their corresponding SP test results. High performing students in the IELTS test also posted high scores in their SP test. As demonstrated in the study, the correlation in the results illustrates the effectiveness of the SP test as an alternative for the IELTS tests in proving English language proficiency.

Assessing the use of multiple-choice translation items in English proficiency tests: The case of the national English proficiency test in Turkey

2019 ◽  
Vol 0 (0) ◽  
Author(s):  
Betül Hazal Dinçer ◽  
Elena Antonova-Unlu ◽  
Alper Kumcu
Keyword(s):  
Language Proficiency ◽  
Proficiency Test ◽  
English Proficiency ◽  
Language Teaching ◽  
Multiple Choice ◽  
National Language ◽  
Proficiency Tests ◽  
Test Items ◽  
Significant Difference ◽  
Sample Test

AbstractThe use of translation for language teaching and assessment, by and large, has been abandoned with the adoption of audio-lingual and communicative approaches in language teaching. As a result, nowadays translation items are not commonly used for measuring language proficiency in international language proficiency tests (e. g. TOEFL, IELTS).However, there are several countries that still use translation items in their national language proficiency tests (e. g. Turkey, Japan, China, Romania among others). The present study aims to examine whether or not multiple-choice translation items are an appropriate tool for measuring proficiency in English. To this end, the perceived level of difficulty and validity of multiple-choice translation items in the National English Proficiency Test (YDS) in Turkey were examined. The findings revealed that the participants did significantly better on the translation items than on the rest of the test items. They also perceived the translation items as the easiest among all the rest items in YDS. Moreover, while YDS as a whole indicated a strong validity based on correlation with TOEFL PBT Reading Sample Test, the translation items indicated moderate validity. Importantly, there was a significant difference between the two correlations. These findings suggest that multiple-choice translation items are likely to lower the overall validity of YDS tests, inflate the scores of test-takers and, thus, might be considered as problematic for the quality of the tests.

scholarly journals The Relationship between English Language Proficiency, Academic Achievement and Self-Esteem of Non-Native-English-Speaking Students

2016 ◽  
Vol 9 (5) ◽  
pp. 147 ◽  
Author(s):  
Smitha Dev ◽  
Sura Qiqieh
Keyword(s):  
Academic Achievement ◽  
Language Proficiency ◽  
English Language ◽  
Rating Scale ◽  
Self Esteem ◽  
English Language Proficiency ◽  
Abu Dhabi ◽  
Proficiency Tests ◽  
Language Fluency ◽  
The Relationship

<p class="apa">The present study aims to find out the relationship between English Language proficiency, self-esteem, and academic achievement of the students in Abu Dhabi University (ADU). The variables were analyzed using ‘t’ test, chi-squire and Pearson’s product moment correlation. In addition, Self-rating scale, Self-esteem inventory and Language proficiency tests were used to measure the variables. The data were collected from 200 male and female students from Abu Dhabi University. The study could not find out any positive relationship among the variables. It is also revealed that language fluency (IELTS) has no direct impact on the ADU students’ self-esteem scores and academic achievement (GPA).</p>

Reviews of English Language Proficiency Tests

1990 ◽  
Vol 74 (2) ◽  
pp. 224
Author(s):  
Mary Emily Call ◽  
J. Charles Alderson ◽  
Karl J. Krahnke ◽  
Charles Stansfield
Keyword(s):  
Language Proficiency ◽  
English Language ◽  
English Language Proficiency ◽  
Proficiency Tests
Sign in / Sign up

Export Citation Format

Share Document