Developing CEFR-related language proficiency tests: A focus on the role of piloting

2018 ◽  
Vol 8 (2) ◽  
pp. 333-352
Author(s):  
Caroline Shackleton
Keyword(s):  
Language Proficiency ◽  
Development Process ◽  
Test Development ◽  
Difficulty Level ◽  
Proficiency Tests ◽  
Cut Scores ◽  
Test Items ◽  
Test Specifications ◽  
Listening Proficiency ◽  
The University

Abstract Most language proficiency exams in Europe are presently developed so that reported scores can be related to the Common European Framework of Reference for Languages (CEFR; (Council of Europe. 2001. Common European framework of reference for languages: learning, teaching, assessment. Cambridge: Cambridge University Press.). Before any CEFR linking process can take place, such tests should be shown to be both valid and reliable, as “if an exam is not valid or reliable, it is meaningless to link it to the CEFR [and] a test that is not reliable cannot, by definition, be valid” (Alderson, Charles J. 2012. Principles and practice in language testing: compliance or conflict? Presentation at TEA SIG Conference: Innsbruck. http://tea.iatefl.org/inns.html (accessed May 2017).). In the test development process, tasks developed based on test specifications must therefore be piloted in order to check that test items perform as predicted. The present article focuses on the statistical analysis of test trial data provided by the piloting of three B1 listening tasks carried out at the University of Granada’s Modern Language Center (CLM). Here, results from a detailed Rasch analysis of the data showed the test to be consistently measuring a unidimensional construct of listening ability. In order to confirm that the test contains items at the correct difficulty level, teacher judgements of candidates’ listening proficiency were also collected. The test was found to separate A2 and B1 candidates well; used in conjunction with the establishment of appropriate cut scores, the reported score can be considered an accurate representation of CEFR B1 listening proficiency. The study demonstrates how Rasch measurement can be used as part of the test development process in order to make improvements to test tasks and hence create more reliable tests

The Role of Order and Sequence of Options in Multiple-choice Questions for High-stakes Tests of English Language Proficiency

2020 ◽  
Vol 9 (6) ◽  
pp. 110
Author(s):  
Talip Karanfil ◽  
Steve Neufeld
Keyword(s):  
Language Proficiency ◽  
English Language ◽  
Multiple Choice ◽  
High Volume ◽  
English Language Proficiency ◽  
Proficiency Tests ◽  
Multiple Choice Questions ◽  
High Stakes ◽  
Test Items ◽  
Two Factors

High-stakes and high-volume English language proficiency tests typically rely on multiple-choice questions (MCQs) to assess reading and listening skills. Due to the Covid-19 pandemic, more institutions are using MCQs via online assessment platforms, which facilitate shuffling the order of options within test items to minimize cheating. There is scant research on the role that order and sequence of options plays in MCQs, so this study examined the results of a paper-based, high-stakes English proficiency test administered in two versions. Each version had identical three-option MCQs but with different ordering of options. The test-takers were chosen to ensure a very similar profile of language ability and level for the groups who took the two versions. The findings indicate that one in four questions exhibited significantly different levels of difficulty and discrimination between the two versions. The study identifies order dominance and sequence priming as two factors that influence the outcomes of MCQs, both of which can accentuate or diminish the power of attraction of the correct and incorrect options. These factors should be carefully considered when designing MCQs in high-stakes language proficiency tests and shuffling of options in either paper-based or computer-based testing.

Online Assessment of Foreign Language Proficiency

2011 ◽  
pp. 82-97 ◽  
Author(s):  
Paula M. Winke
Keyword(s):  
Language Proficiency ◽  
Test Development ◽  
The United States ◽  
Development Project ◽  
The Internet ◽  
Online Assessment ◽  
Proficiency Tests ◽  
Commercial Test ◽  
College And University ◽  
Test Product

In this chapter, the challenges involved in creating online Arabic and Russian proficiency tests by using a commercial test product are discussed. Guidelines used for item and test development are presented, along with specific challenges test developers faced in designing computerized, semiadaptable tests. Also discussed are the issues involved in delivering the tests securely over the Internet to examinees, who took them on computers in college and university language labs across the United States and abroad. The chapter concludes with a list of five important lessons that could help others who are contemplating a similar test development project.

Online Assessment of Foreign Language Proficiency

2011 ◽  
pp. 2224-2233
Author(s):  
Paula M. Winke
Keyword(s):  
Language Proficiency ◽  
Test Development ◽  
The United States ◽  
Development Project ◽  
The Internet ◽  
Online Assessment ◽  
Proficiency Tests ◽  
Commercial Test ◽  
College And University ◽  
Test Product

In this chapter, the challenges involved in creating online Arabic and Russian proficiency tests by using a commercial test product are discussed. Guidelines used for item and test development are presented, along with specific challenges test developers faced in designing computerized, semiadaptable tests. Also discussed are the issues involved in delivering the tests securely over the Internet to examinees, who took them on computers in college and university language labs across the United States and abroad. The chapter concludes with a list of five important lessons that could help others who are contemplating a similar test development project.

Assessing the use of multiple-choice translation items in English proficiency tests: The case of the national English proficiency test in Turkey

2019 ◽  
Vol 0 (0) ◽  
Author(s):  
Betül Hazal Dinçer ◽  
Elena Antonova-Unlu ◽  
Alper Kumcu
Keyword(s):  
Language Proficiency ◽  
Proficiency Test ◽  
English Proficiency ◽  
Language Teaching ◽  
Multiple Choice ◽  
National Language ◽  
Proficiency Tests ◽  
Test Items ◽  
Significant Difference ◽  
Sample Test

AbstractThe use of translation for language teaching and assessment, by and large, has been abandoned with the adoption of audio-lingual and communicative approaches in language teaching. As a result, nowadays translation items are not commonly used for measuring language proficiency in international language proficiency tests (e. g. TOEFL, IELTS).However, there are several countries that still use translation items in their national language proficiency tests (e. g. Turkey, Japan, China, Romania among others). The present study aims to examine whether or not multiple-choice translation items are an appropriate tool for measuring proficiency in English. To this end, the perceived level of difficulty and validity of multiple-choice translation items in the National English Proficiency Test (YDS) in Turkey were examined. The findings revealed that the participants did significantly better on the translation items than on the rest of the test items. They also perceived the translation items as the easiest among all the rest items in YDS. Moreover, while YDS as a whole indicated a strong validity based on correlation with TOEFL PBT Reading Sample Test, the translation items indicated moderate validity. Importantly, there was a significant difference between the two correlations. These findings suggest that multiple-choice translation items are likely to lower the overall validity of YDS tests, inflate the scores of test-takers and, thus, might be considered as problematic for the quality of the tests.

The English Placement Test at the University of Illinois at Urbana-Champaign

2015 ◽  
Vol 48 (2) ◽  
pp. 284-287
Author(s):  
Sun Joo Chung ◽  
Iftikhar Haider ◽  
Ryan Boyd
Keyword(s):  
English Language ◽  
Relevant Information ◽  
Placement Test ◽  
Proficiency Tests ◽  
Esl Writing ◽  
University Of Illinois ◽  
Test Specifications ◽  
Institutional Placement ◽  
Admission Status ◽  
The University

At the University of Illinois at Urbana-Champaign (UIUC), the English Placement Test (EPT) is the institutional placement test that is used to place students into appropriate English as a second language (ESL) writing and/or pronunciation service courses. The EPT is used to assess the English ability of newly admitted international undergraduate and graduate students as validated against the English language demands of our campus (UIUC English Placement Test 2013). According to Davidson & Cho (2001), the current format of the EPT has maintained its quality and displayed evidence of validity through the use of detailed test specifications that align with the goals of the UIUC ESL writing and pronunciation service courses. UIUC offers these courses to international students who are accepted on a limited status based on their scores on standardized English proficiency tests (TOEFL or IELTS) and/or other relevant information in each student's admission dossier. Students accepted on limited admission status are required to take the EPT before start of instruction.

scholarly journals How Proficient is Proficient Enough?: A Look at Proficiency in ESL for Academic Success

2017 ◽  
Vol 2 ◽  
pp. 4
Author(s):  
Marcellino Berardo
Keyword(s):  
Academic Success ◽  
International Students ◽  
Language Proficiency ◽  
Academic Program ◽  
Class Discussion ◽  
Proficiency Tests ◽  
Academic English ◽  
University Of Kansas ◽  
Literature Reviews ◽  
The University

We teach English to international students for the purpose of academic achievement at the University of Kansas. The assumption is that students cannot be successful in university classes without the ability to use academic English well. This is most obvious at the lower levels. If the student cannot understand any spoken or written English, then s/he cannot take notes, read textbook chapters, write papers, take exams, or participate in class discussion. This assumption also implies that as the student’s English improves, proficiency will become less of a factor in academic success.This essay is intended to help us discuss assumptions associated with language proficiency and academic success and to identify how well students need to be able to use English in order to fulfill the university’s ESL requirement and be successful in their chosen academic program, at least as far as language is concerned. To this end, I begin by characterizing different kinds of proficiency. Then I highlight the literature reviews in this issue, one by Jennifer Hornbaker and one by Parul Sood, that bring up issues related to using proficiency tests as the sole indicators of linguistic readiness for academic success at the university.

scholarly journals How Proficient is Proficient Enough?: A Look at Proficiency in ESL for Academic Success

2017 ◽  
Vol 2 (3) ◽  
pp. 4-8
Author(s):  
Marcellino Berardo
Keyword(s):  
Academic Success ◽  
International Students ◽  
Language Proficiency ◽  
Academic Program ◽  
Class Discussion ◽  
Proficiency Tests ◽  
Academic English ◽  
University Of Kansas ◽  
Literature Reviews ◽  
The University

We teach English to international students for the purpose of academic achievement at the University of Kansas. The assumption is that students cannot be successful in university classes without the ability to use academic English well. This is most obvious at the lower levels. If the student cannot understand any spoken or written English, then s/he cannot take notes, read textbook chapters, write papers, take exams, or participate in class discussion. This assumption also implies that as the student’s English improves, proficiency will become less of a factor in academic success.This essay is intended to help us discuss assumptions associated with language proficiency and academic success and to identify how well students need to be able to use English in order to fulfill the university’s ESL requirement and be successful in their chosen academic program, at least as far as language is concerned. To this end, I begin by characterizing different kinds of proficiency. Then I highlight the literature reviews in this issue, one by Jennifer Hornbaker and one by Parul Sood, that bring up issues related to using proficiency tests as the sole indicators of linguistic readiness for academic success at the university.

Can-Do descriptors – Realigning English Language Curriculum at Higher Education Institution to CEFR

2020 ◽  
Vol 4 (2) ◽  
pp. 84
Author(s):  
Erda Wati Bakar
Keyword(s):  
Higher Education ◽  
Foreign Language ◽  
Language Proficiency ◽  
English Language ◽  
Education Institution ◽  
Ministry Of Education ◽  
Language Curriculum ◽  
English Curriculum ◽  
The Common ◽  
The University

The Common European Framework of Reference for Language (CEFR) has become the standard used to describe and evaluate students’ command of a second or foreign language. It is an internationally acknowledged standard language proficiency framework which many countries have adopted such as China, Thailand, Japan and Taiwan. Malaysia Ministry of Education is aware and realise the need for the current English language curriculum to be validated as to reach the international standard as prescribed by the CEFR. The implementation of CEFR has begun at primary and secondary level since 2017 and now higher education institutions are urged to align their English Language Curriculum to CEFR as part of preparation in receiving students who have been taught using CEFR-aligned curriculum at schools by year 2022. This critical reflection article elucidates the meticulous processes that we have embarked on in re-aligning our English Language Curriculum to the standard and requirements of CEFR. The paper concludes with a remark that the alignment of the English curriculum at the university needs full support from the management in ensuring that all the stakeholders are fully prepared, informed and familiar with the framework.

Sign in / Sign up

Export Citation Format

Share Document