Large-Scale Language Assessment

2016 ◽  
pp. 476-489
Author(s):  
Antony John Kunnan
Keyword(s):  
Large Scale ◽  
Language Assessment

THE WORKPLACE, THE SOCIETY AND THE WIDER WORLD: THE OFFSHORING AND OUTSOURCING INDUSTRY

2009 ◽  
Vol 29 ◽  
pp. 145-167 ◽  
Author(s):  
Liz Hamp-Lyons ◽  
Jane Lockwood
Keyword(s):  
Customer Service ◽  
Language Proficiency ◽  
Call Center ◽  
Large Scale ◽  
Call Centers ◽  
Emerging Market ◽  
Language Assessment ◽  
Poor Quality ◽  
The Philippines ◽  
Business Success

Workplace language assessment poses special issues for language testers, but also, when it becomes very large scale, it poses issues for language policy. This article looks at these issues, focusing on the offshore and outsourcing (O&O) industry as it is transitioning from native-speaking (NS) countries into nonnative-speaking (NNS) destinations such as India and the Philippines. This is obviously most impacted in call centers, where the ability of customer service representatives (CSRs) to communicate with ease with their native-English speaking customers is central to business success and can be key to a nation's economy. Having reviewed the (limited) research in this area, we take the Philippines as our example to explore how government, academe, and the business sector are dealing with the language proficiency and personnel-training issues caused by the exponential growth in this industry. Appropriate language assessments that are practical, while also being valid and reliable, are critical if the Philippines is to retain its position in this emerging market. Currently, call centers in Philippines complain of very poor recruitment rates due to poor language ability, and of poor quality communication outcomes measures: But how do they assess these key areas? We describe and evaluate the current situation in call center language assessment in the Philippines and discuss possible ways forward, for the Philippines and for the O&O industry more broadly.

Oral Language Assessment from an Ecolinguistic Perspective. An Empirical Investigation of Symbolic Competence

2016 ◽  
Vol 2 (2) ◽  
pp. 12-43
Author(s):  
Evangelia (Evelyn) Vovou
Keyword(s):  
Foreign Language ◽  
Construct Validity ◽  
Oral Language ◽  
Intercultural Competence ◽  
Large Scale ◽  
Empirical Investigation ◽  
Language Assessment ◽  
Social Nature ◽  
Educational Environments ◽  
The Common

Abstract Although today's educational environments are to a great extend multilingual, large-scale foreign language examinations test heterogeneous groups with homogeneous examination practices, without taking all ecolinguistic parameters into consideration. Trying to minimize this limitation by calibrating examinations to the sociolinguistic and intercultural competence definitions of the Common European Framework of Reference for Languages (CEFR), secures to an extend construct validity. However, the question still arises, if “one test fits all”. This paper focuses on oral foreign language assessment discourses, where discursive coconstruction and social nature of performance prevail. Adopting the ecolinguistic approach (Fill, 1996) the paper investigates the notion of symbolic competence (Kramsch & Whiteside, 2008) in the context of oral language examinations. By analyzing oral data the paper seeks to address, how ecolinguistic parameters concur in examination discourses and to what extend this effects the validity of measurement.

TRENDS IN COMPUTER-BASED SECOND LANGUAGE ASSESSMENT

2005 ◽  
Vol 25 ◽  
pp. 228-242 ◽  
Author(s):  
Joan Jamieson
Keyword(s):  
Second Language ◽  
Language Processing ◽  
Corpus Linguistics ◽  
Large Scale ◽  
Language Testing ◽  
Language Assessment ◽  
Adaptive Tests ◽  
Technological Advances ◽  
Computer Based ◽  
Computer Adaptive Tests

In the last 20 years, several authors have described the possible changes that computers may effect in language testing. Since ARAL's last review of general language testing trends (Clapham, 2000), authors in the Cambridge Language Assessment Series have offered various visions of how computer technology could alter the testing of second language skills. This chapter reflects these perspectives as it charts the paths recently taken in the field. Initial steps were made in the conversion of existing item types and constructs already known from paper-and- pencil testing into formats suitable for computer delivery. This conversion was closely followed by the introduction of computer-adaptive tests, which aim to make more, and perhaps, better, use of computer capabilities to tailor tests more closely to individual abilities and interests. Movement toward greater use of computers in assessment has been coupled with an assumption that computer-based tests should be better than their traditional predecessors, and some related steps have been taken. Corpus linguistics has provided tools to create more authentic assessments; the quest for authenticity has also motivated inclusion of more complex tasks and constructs. Both these innovations have begun to be incorporated into computer-based language tests. Natural language processing has also provided some tools for computerized scoring of essays, particularly relevant in large-scale language testing programs. Although computer use has not revolutionized all aspects of language testing, recent efforts have produced some of the research, technological advances, and improved pedagogical understanding needed to support progress.

scholarly journals The Evolving Concept of (Language) Assessment Literacy. Implications for Teacher Education

2021 ◽  
Vol 3 (1) ◽  
pp. 120-130
Author(s):  
Ildikó Csépes
Keyword(s):  
Teacher Education ◽  
Large Scale ◽  
Classroom Assessment ◽  
Language Assessment ◽  
Language Teachers ◽  
Assessment Literacy ◽  
Assessment Practices ◽  
Research Findings ◽  
Assessment Decisions ◽  
Context Specific

Language teachers’ assessment knowledge and skills have received considerable attention from language assessment researchers over the past few decades (Davison & Leung, 2009; Hill & McNamara, 2012; Rea-Dickins, 2001; Taylor, 2013). This seems to be linked to the increased professionalism expected of them in classroom-based assessments. However, teachers seem to face a number of challenges, including how large-scale standardized language exams influence their classroom assessment practices. Teachers’ assessment literacy, therefore, needs to be examined in order to explain their assessment decisions. In this paper, we review the concept of (language) assessment literacy, how it has evolved and how it is conceptualized currently. Recent interpretations seem to reflect a multidimensional, dynamic and situated view of (language) assessment literacy. Implications for teacher education are also highlighted by presenting research findings from studies that explored teachers’ and teacher candidates’ assessment literacy in various educational contexts. As a result, we can identify some common patterns in classroom assessment practices as well as context-specific training needs. Finally, we make a recommendation for tackling some of the challenges language teachers are facing in relation to classroom-based assessment in the Hungarian context.

scholarly journals Test usefulness of IELTS writing test tasks

2017 ◽  
Vol 11 (4) ◽  
pp. 1-9
Author(s):  
Neng Priyanti
Keyword(s):  
Construct Validity ◽  
Predictive Value ◽  
Large Scale ◽  
Language Assessment ◽  
The Other ◽  
Writing Test ◽  
Scale Test ◽  
Time Allotment ◽  
Comprehensive Framework ◽  
Small Parts

This analysis was conducted to find out the extent to which IELTS writing test tasks are useful for its intended purpose. The indicators of test usefulness used in this study are those of Bachman and Palmer (1996): construct validity, reliability, authenticity, interactiveness, impact and practicality. This framework is taken as a viewpoint as it offers a comprehensive framework of test usefulness that is specific to that of language assessment. The result of the analysis shows that the IELTS writing test tasks place more concern on reliability and practicality. While not necessarily neglecting the other indicators, the nature of the test as a large-scale test reduces its overall usefulness. The predictive value remains unclear as its construct validity is still under-represented. The authenticity is problematic as the tasks correspond to only small parts of real tasks in universities in terms of their length, types of tasks, time allotment, and range of topics offered.

scholarly journals Statistics for Classroom Language Assessment: Using Numbers Meaningfully

HOW ◽  
2020 ◽  
Vol 27 (2) ◽  
pp. 135-155
Author(s):  
Frank Giraldo
Keyword(s):  
Test Scores ◽  
Large Scale ◽  
Language Assessment ◽  
Statistical Information ◽  
Assessment System ◽  
Language Classroom ◽  
Classroom Language ◽  
Criterion Referenced ◽  
Assessment Decisions

Large-scale language testing uses statistical information to account for the quality of an assessment system. In this reflection article, I explain how basic statistics can be used meaningfully in the context of classroom language assessment. The paper explores a series of statistical calculations that can be used to examine test scores and assessment decisions in the language classroom. Therefore, interpretations for criterion-referenced assessment underlie the paper. Finally, I discuss limitations and include recommendations for teachers to use statistics.

scholarly journals ENHANCING TEACHERS’ LANGUAGE ASSESSMENT LITERACY VIA ACTIVITIES OF A PROFESSIONAL ORGANIZATION

2021 ◽  
pp. 4-9
Author(s):  
Olga Kvasova
Keyword(s):  
Large Scale ◽  
Language Assessment ◽  
Professional Organization ◽  
Assessment Literacy ◽  
University Teachers ◽  
For Profit ◽  
The Past ◽  
Future Events ◽  
Language Testing And Assessment ◽  
The University

The paper discusses the issues related to the language assessment literacy (LAL) of Ukrainian university teachers of English. The author overviews the development of LAL beginning in 2010 when the first survey of LAL state was conducted by her. The surveys conducted later enabled the comparison of teacher respondents’ perceptions of the advances and/or issues in LAL. Although the surveys were not large-scale, the samples were quite fully representative of the university teacher community which allowed to capture the major trends of LAL development. The author further describes the activities of a professional non-for-profit organization – Ukrainian Association for Language Testing and Assessment (UALTA) who made a major contribution to LAL development in the past 6 years by regularly holding training events. The latest event was an in-service teacher training (INSETT) course conducted online. The course has laid theoretical and organizational bases for the future events of such kind.

scholarly journals Language Assessment for Immigration: A Review of Validation Research Over the Last Two Decades

2021 ◽  
Vol 12 ◽  
Author(s):  
Don Yao ◽  
Matthew P. Wallace
Keyword(s):  
Decision Making ◽  
Test Scores ◽  
Large Scale ◽  
Test Validity ◽  
Language Assessment ◽  
Future Research ◽  
Negative Consequences ◽  
High Stakes ◽  
Language Tests ◽  
Assessment Use Argument

It is not uncommon for immigration-seekers to be actively involved in taking various language tests for immigration purposes. Given the large-scale and high-stakes nature those language tests possess, the validity issues (e.g., appropriate score-based interpretations and decisions) associated with them are of great importance as test scores may play a gate-keeping role in immigration. Though interest in investigating the validity of language tests for immigration purposes is becoming prevalent, there has to be a systematic review of the research foci and results of this body of research. To address this need, the current paper critically reviewed 11 validation studies on language assessment for immigration over the last two decades to identify what has been focused on and what has been overlooked in the empirical research and to discuss current research interests and future research trends. Assessment Use Argument (AUA) framework of Bachman and Palmer (2010), comprising four inferences (i.e., assessment records, interpretations, decisions, and consequences), was adopted to collect and examine evidence of test validity. Results showed the consequences inference received the most investigations focusing on immigration-seekers’ and policymakers’ perceptions on test consequences, while the decisions inference was the least probed stressing immigration-seekers’ attitude towards the impartiality of decision-making. It is recommended that further studies could explore more kinds of stakeholders (e.g., test developers) in terms of their perceptions on the test and investigate more about the fairness of decision-making based on test scores. Additionally, the current AUA framework includes only positive and negative consequences that an assessment may engender but does not take compounded consequences into account. It is suggested that further research could enrich the framework. The paper sheds some light on the field of language assessment for immigration and brings about theoretical, practical, and political implications for different kinds of stakeholders (e.g., researchers, test developers, and policymakers).

Recent developments in language assessment and the case of four large-scale tests of ESOL ability

2009 ◽  
Vol 42 (1) ◽  
pp. 1-40 ◽  
Author(s):  
Stephen Stoynoff
Keyword(s):  
Large Scale ◽  
Language Assessment ◽  
Review Article ◽  
Test Method ◽  
Recent Developments ◽  
Large Scale Tests

This review article surveys recent developments and validation activities related to four large-scale tests of L2 English ability: the iBT TOEFL, the IELTS, the FCE, and the TOEIC. In addition to describing recent changes to these tests, the paper reports on validation activities that were conducted on the measures. The results of this research constitute some of the evidence available to support claims that these tests are suitable for their intended purposes. The discussion is organized on the basis of a framework that considers test purpose, selected test method characteristics, and important aspects of test usefulness.

Sign in / Sign up

Export Citation Format

Share Document