Discourse Analysis in Second Language Speaking Assessment

2021 ◽  
pp. 335-346
Author(s):  
Kellie Frost
Keyword(s):  
Second Language ◽  
Discourse Analysis ◽  
Rating Scale ◽  
Test Taker ◽  
High Stakes ◽  
Spoken Discourse ◽  
Speaking Assessment ◽  
Testing Practices ◽  
Task Conditions ◽  
Task Types

Discourse analysis has been widely used in the field of language testing. This chapter provides an overview of research examining features of test-taker discourse across different task types and under different task conditions and the extent to which these features align with rating scale criteria. Attention is also drawn to discourse analytic studies of the language demands of study and work domains and the extent to which test tasks can elicit relevant features. The chapter concludes by reflecting on the challenges posed to existing high-stakes test constructs by increasing diversity in universities and workplaces and the potential for discourse analytic approaches to establish stronger alignments between testing practices and the aspects of spoken discourse relevant and valued in communication.

scholarly journals Topic and background knowledge effects on performance in speaking assessment

2016 ◽  
Vol 34 (1) ◽  
pp. 23-48 ◽  
Author(s):  
Nahal Khabbazbashi
Keyword(s):  
Language Proficiency ◽  
Background Knowledge ◽  
Self Report ◽  
Practical Significance ◽  
Systematic Effect ◽  
High Stakes ◽  
Speaking Assessment ◽  
Native Speakers Of English ◽  
Practical Effect ◽  
Task Types

This study explores the extent to which topic and background knowledge of topic affect spoken performance in a high-stakes speaking test. It is argued that evidence of a substantial influence may introduce construct-irrelevant variance and undermine test fairness. Data were collected from 81 non-native speakers of English who performed on 10 topics across three task types. Background knowledge and general language proficiency were measured using self-report questionnaires and C-tests respectively. Score data were analysed using many-facet Rasch measurement and multiple regression. Findings showed that for two of the three task types, the topics used in the study generally exhibited difficulty measures which were statistically distinct. However, the size of the differences in topic difficulties was too small to have a large practical effect on scores. Participants’ different levels of background knowledge were shown to have a systematic effect on performance. However, these statistically significant differences also failed to translate into practical significance. Findings hold implications for speaking performance assessment.

scholarly journals Metacognitive Instruction for Sustainable Learning: Learners’ Perceptions of Task Difficulty and Use of Metacognitive Strategies in Completing Integrated Speaking Tasks

2021 ◽  
Vol 13 (11) ◽  
pp. 6275
Author(s):  
Weiwei Zhang ◽  
Donglan Zhang ◽  
Lawrence Jun Zhang
Keyword(s):  
Foreign Language ◽  
Repeated Measures ◽  
Task Difficulty ◽  
Rating Scale ◽  
Empirical Support ◽  
Metacognitive Strategies ◽  
Production Model ◽  
High Stakes ◽  
Efl Learners ◽  
Metacognitive Instruction

This mixed-methods study investigated English-as-a-foreign-language (EFL) learners’ perceptions of task difficulty and their use of metacognitive strategies in completing integrated speaking tasks as empirical evidence for the effects of metacognitive instruction. A total of 130 university students were invited to complete four integrated speaking tasks and answer a metacognitive strategy inventory and a self-rating scale. A sub-sample of eight students participated in the subsequent interviews. One-way repeated measures MANOVA and structure coding with content analysis led to two main findings: (a) EFL learners’ use of metacognitive strategies, in particular, problem-solving, was considerably affected by their perceptions of task difficulty in completing the integrated speaking tasks; (b) EFL learners were not active users of metacognitive strategies in performing these tasks. These findings not only support the necessity of taking into account learners’ perceptions of task difficulty in designing lesson plans for metacognitive instruction, but also support a metacognitive instruction model. In addition, the findings provide empirical support for the utility of Kormos’ Bilingual Speech Production Model. As the integrated speaking tasks came from a high-stakes test, these findings also offer validity evidence for test development in language assessment to ascertain sustainable EFL learning for nurturing learner autonomy as an ultimate goal.

Assessing pragmatic competence in oral proficiency interviews at the C1 level with the new CEFR descriptors

2020 ◽  
Vol 16 (1) ◽  
pp. 87-121
Author(s):  
Bárbara Eizaga-Rebollar ◽  
Cristina Heras-Ramírez
Keyword(s):  
Rating Scales ◽  
Rating Scale ◽  
Oral Proficiency ◽  
Pragmatic Competence ◽  
Task Characteristics ◽  
The Common ◽  
Task Descriptions ◽  
Oral Proficiency Interviews ◽  
Task Types ◽  
And Task

AbstractThe study of pragmatic competence has gained increasing importance within second language assessment over the last three decades. However, its study in L2 language testing is still scarce. The aim of this paper is to research the extent to which pragmatic competence as defined by the Common European Framework of Reference for Languages (CEFR) has been accommodated in the task descriptions and rating scales of two of the most popular Oral Proficiency Interviews (OPIs) at a C1 level: Cambridge’s Certificate in Advanced English (CAE) and Trinity’s Integrated Skills in English (ISE) III. To carry out this research, OPI tests are first defined, highlighting their differences from L2 pragmatic tests. After pragmatic competence in the CEFR is examined, focusing on the updates in the new descriptors, CAE and ISE III formats, structure and task characteristics are compared, showing that, while the formats and some characteristics are found to differ, the structures and task types are comparable. Finally, we systematically analyse CEFR pragmatic competence in the task skills and rating scale descriptors of both OPIs. The findings show that the task descriptions incorporate mostly aspects of discourse and design competence. Additionally, we find that each OPI is seen to prioritise different aspects of pragmatic competence within their rating scale, with CAE focusing mostly on discourse competence and fluency, and ISE III on functional competence. Our study shows that the tests fail to fully accommodate all aspects of pragmatic competence in the task skills and rating scales, although the aspects they do incorporate follow the CEFR descriptors on pragmatic competence. It also reveals a mismatch between the task competences being tested and the rating scale. To conclude, some research lines are proposed.

Assessing the Role of Selective Fossilization Hypothesis in Determining Fossilizable Phonetic Errors in Tunisian EFL Learners’ Oral Output

2020 ◽  
Vol 2 ◽  
pp. 1-15
Author(s):  
Aicha Rahal ◽  
Chokri Smaoui
Keyword(s):  
Second Language ◽  
Longitudinal Study ◽  
Predictive Power ◽  
Rating Scale ◽  
First Language ◽  
Linguistic Features ◽  
Efl Learners ◽  
Phonetic Errors ◽  
Linguistic Phenomenon

Fossilization is said to be a distinctive characteristic of second language (L2) learning (Selinker, 1972, 1996; Han, 2004). It is the most pervasive among adult L2 learners (Han and Odlin, 2006). This linguistic phenomenon has been characterized by cessation of learning, even though the learner is exposed to frequent input. Based on the findings of the MA dissertation of the first researcher which is about ‘phonetic fossilization’ and where she conducted a longitudinal study, Han’s Selective Fossilization Hypothesis (SFL) is used to analyze the obtained fossilized phonetic errors in relation to L1 markedness and L2 robustness with a particular focus on fossilized vowel sounds. This is an analytical model for identifying both acquisitional and fossilizable linguistic features based on learners’ first language (L1) markedness and second language (L2) robustness. The article first gives an overview of the theory of Interlanguage and the phenomenon of fossilization. Then, it introduces SFL. This is an attempt to study fossilization scientifically. In other words, it tests the predictive power of a developed L1 Markedness and L2 Robustness rating scale based on Han’s (2009) model. The present study has pedagogic implications; it is an opportunity to raise teachers’ awareness on this common linguistic phenomenon.

scholarly journals A Rasch-Based Validation of ELT Certificate-LORT

2020 ◽  
Vol 13 (9) ◽  
pp. 94
Author(s):  
Xin Qu
Keyword(s):  
Teacher Candidates ◽  
Rating Scales ◽  
Rating Scale ◽  
Qualitative Interviews ◽  
Structured Interviews ◽  
High Stakes ◽  
Efl Teacher ◽  
The Difference ◽  
Group Interviews

The present study was executed with the purpose of validating ELT Certificate Lesson Observation and Report Task (ELTC-LORT), which was developed by China Language Assessment to certify China’s EFL teachers by performance-based testing. The ELT Certificate has high-stakes considering its impacts on candidates’ recruitment, ELT in China and quality of education, so it is crucially important for its validation so as to guarantee fairness and justice. The validity of task construct and rating rubric went through a process suited for many-facet Rasch measurement supplemented with qualitative interviews. Participants (N = 40) were provided with a video excerpt from a real EFL lesson, and required to deliver a report on the teacher’s performance. Two raters graded the records of the candidates’ reports using rating scales developed to measure EFL teacher candidates’ oral English proficiency and ability to analyze and evaluate teaching. Many-facet Rasch analysis demonstrated a successful estimation, with a noticeable spread among the participants and their traits, proving the task functioned well in measuring candidates’ performance and reflecting the difference of their ability. The raters were found to have good internal self-consistency, but not the same leniency. The rating scales worked well, with the average measures advancing largely in line with Rasch expectations. Semi-structured interviews as well as focus group interviews were executed to provide knowledge regarding the raters’ performance levels and the functionalities of the rating scale items. The findings provide implications for further research and practice of the Certificate.

scholarly journals The impact of pre-task planning on speaking test performance for English-medium university admission

2019 ◽  
Vol 36 (4) ◽  
pp. 505-526 ◽  
Author(s):  
Stefan O’Grady
Keyword(s):  
Second Language ◽  
Test Performance ◽  
English Language ◽  
Rating Scale ◽  
Statistical Significance ◽  
Task Type ◽  
Task Planning ◽  
Planning Time ◽  
University Admission ◽  
The Impact

This study investigated the impact of different lengths of pre-task planning time on performance in a test of second language speaking ability for university admission. In the study, 47 Turkish-speaking learners of English took a test of English language speaking ability. The participants were divided into two groups according to their language proficiency, which was estimated through a paper-based English placement test. They each completed four monologue tasks: two picture-based narrative tasks and two description tasks. In a balanced design, each test taker was allowed a different length of planning time before responding to each of the four tasks. The four planning conditions were 30 seconds, 1 minute, 5 minutes, and 10 minutes. Trained raters awarded scores to the test takers using an analytic rating scale and a context-specific, binary-choice rating scale, designed specifically for the study. The results of the rater scores were analysed by using a multifaceted Rasch measurement. The impact of pre-task planning on test scores was found to be influenced by four variables: the rating scale; the task type that test takers completed; the length of planning time provided; and the test takers’ levels of proficiency in the second language. Increases in scores were larger on the picture-based narrative tasks than on the two description tasks. The results also revealed a relationship between proficiency and pre-task planning, whereby statistical significance was only reached for the increases in the scores of the lowest-level test takers. Regarding the amount of planning time, the 5-minute planning condition led to the largest overall increases in scores. The research findings offer contributions to the study of pre-task planning and will be of particular interest to institutions seeking to assess the speaking ability of prospective students in English-medium educational environments.

Developing rating scales for the assessment of second language performance

1996 ◽  
Vol 13 ◽  
pp. 55-79 ◽  
Author(s):  
Carolyn E. Turner ◽  
John A. Upshur
Keyword(s):  
Second Language ◽  
Rating Scales ◽  
Rating Scale ◽  
Reliability And Validity ◽  
Sixth Grade ◽  
Language Performance ◽  
Specific Test ◽  
Binary Choices ◽  
Time Required ◽  
Second Language Performance

Abstract The two most common approaches to rating second language performance pose problems of reliability and validity. An alternative method utilizes rating scales that are empirically derived from samples of learner performance; these scales define boundaries between adjacent score levels rather than provide normative descriptions of ideal performances; the rating process requires making two or three binary choices about a language performance being rated. A procedure, that consists of a series of five explicit tasks, is used to construct a rating scale. The scale is designed for use with a specific population and a specific test task. A group of primary school ESL teachers used this procedure to make two speaking tests, including elicitation tasks and rating scales, for use in their school district. The tests were administered to 255 sixth grade learners. The scales were found to be highly accurate for scoring short speech samples, and were quite efficient in time required for scale development and rater training. Scales exhibit content relevance in the instructional setting. Development of this type of scale is recommended for use in high-stakes assessment.

Sign in / Sign up

Export Citation Format

Share Document