scholarly journals Diverging task orientations in L2 oral proficiency tests – a conversation analytic approach to participant understandings of pre-set discussion tasks

2013 ◽  
Vol 2 (1) ◽  
Author(s):  
Erica Sandlund ◽  
Pia Sundqvist
Keyword(s):  
Conversation Analysis ◽  
Standardized Tests ◽  
Oral Proficiency ◽  
Analytic Approach ◽  
Proficiency Tests ◽  
9Th Grade ◽  
Valid Assessment ◽  
National Tests ◽  
National Test

Abstract Presumably most students strive to do well in school and on national tests. However, even in standardized tests, students’ and examiners’ expectations on what it means to ‘do well’ may diverge in ways that are consequential to performance and assessment. In this paper, we examine how students and teachers in an L2 English peer–peer speaking national test (9th grade) display their understandings of appropriate ways of dealing with pre-set discussion tasks. Using conversation analysis and 38 recorded national tests in English in Sweden, we demonstrate, e.g., how teachers’ displayed understandings of how tasks should be appropriately handled steer the interactional trajectory between students in particular directions. The analysis shows that participants spend much time on negotiating understandings of the task-at-hand. We argue that in terms of valid assessment of oral proficiency, task understandings merit more attention, as task negotiations inevitably generate different conditions for different dyads and teachers.

Assessor accommodation in the V.C.E. Italian oral test

1997 ◽  
Vol 20 (1) ◽  
pp. 21-41 ◽  
Author(s):  
Caterina Cafarella
Keyword(s):  
Conversation Analysis ◽  
Proficiency Test ◽  
Oral Proficiency ◽  
Student Response ◽  
Proficiency Tests ◽  
Spoken Discourse ◽  
Assessment Task ◽  
Analysis Methodology ◽  
Oral Test ◽  
Interactive Nature

Abstract In oral proficiency tests there are occurrences of trouble in interaction such as misunderstanding, non hearing or lack of understanding which may cause breakdown in communication. Within the context of the question-answer framework of an oral proficiency test this study investigates the interactive nature of spoken discourse between students and assessors when there is trouble in talk as perceived by the assessors, with a focus on how they accommodate to the students. A sample of twenty oral transcripts and tapes of the 1992 Victorian Certificate of Education (V.C.E.) Italian Common Assessment Task (C.A.T. 2) were randomly selected and examined. By using Conversation Analysis methodology the purpose of the study was to investigate in repair sequences types of assessor accommodation – how the assessors modified their utterances – the kinds of trouble perceived by assessors, what triggered assessor accommodation and whether the accommodations facilitated student response and participation. This study has implications for assessor training since it highlights which strategies are most successful for ensuring student understanding, participation and appropriate responses as well as demonstrating why and in which environments assessors accommodate.

scholarly journals Det första nationella provet i samhällskunskap - en studie i bedömarsamstämmighet

2018 ◽  
Vol 12 (4) ◽  
pp. 13
Author(s):  
Arne Löfstedt
Keyword(s):  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Ninth Graders ◽  
National Agency ◽  
Final Grade ◽  
The Subject ◽  
And Mathematics ◽  
The Stability ◽  
National Tests ◽  
National Test

Skolämnet samhällskunskap som eget ämne existerar i princip enbart i de nordiska länderna. I många andra länder delar flera skolämnen på ämnesinnehållet, till exempel geografi och civics. Ämnesinnehållet är stort och genomgår ständig förändring. År 2013 genomfördes de första nationella proven i samhällskunskap i Sverige för årskurs 9. Med tanke på ämnets karaktär kan det vara speciellt viktigt att undersöka om dessa prov är ”rättvisa.” Avsikten med denna studie är att undersöka en aspekt av denna ”rättvisa”, nämligen interbedömarstabilitet, dvs om samma elevsvar ger upphov till samma bedömning oavsett bedömare. Skolverket i Sverige genomförde 2009 en större studie av de ämnen som då genomförde nationella prov och föreliggande studie försöker dels efterlikna och dels bygga ut upplägget från Skolverket. Studien genomfördes på de första nationella proven i samhällskunskap 2013. Genom att pröva olika reliabilitetsmått inom kategorierna ”consensus estimates”, och ”consistency estimates” analyseras resultaten, bland annat diskuteras måttet intraclass correlation. Syftet är också, då detta var de första proven, att skapa en ram för återkommande studier av Interbedömarreliabilitet. Upplägget med en större mängd lärare som genomför totalt tre bedömningar av de utvalda hela proven försöker också efterlikna bedömningssituationen ute på skolorna såtillvida att det var relativt många lärare med i studien, och de kom från olika skolor spridda över Sverige. Genom detta testas också bedömningsanvisningarnas stabilitet. Själva genomförandet var omfattande och tog två hela dagar. Resultaten pekar på en god överensstämmelse för provbetyget, det sammanfattande omdöme eleverna får. Studien avses att återupprepas under kommande år.Nyckelord: Samhällskunskap, nationella prov, interbedömarreliabilitet, intraclass correlationThe first national test in samhällskunskap – a study of interrater reliabilityAbstractThe Swedish school subject Samhällskunskap (Societal knowledge) exists basically only in the Nordic countries. In other countries a number of different subjects, such as geography and civics, share the content. The content of the subject is constantly changing, depending on how society is changing. The first national tests in Samhällskunskap for all Swedish ninth graders took place in 2013. A large part of the test contains constructed responses.  Given the characteristics of the subject we consider it especially important to investigate whether these tests are “fair” or not. The intent of this study is to investigate one aspect of “fairness”, interrater reliability, meaning the degree to which the same student responses are scored equally by different raters.  In 2009, the National Agency of Education in Sweden conducted a large study of the subjects Swedish, English and Mathematics. Our study aims to mimic and further develop the design of the study from 2009. Our study was carried out on the first national tests in 2013. The results were analyzed by exploring different reliability measures within the categories consensus estimates, and consistency estimates. As the 2013 tests were the first tests of its kind in Sweden the purpose was also to create a framework for regular studies of interrater reliability. The rater design with a relatively large number of teachers from all over the country, each assessing a total of three complete student test responses aimed at mimicking the way the tests are assessed in schools. This also allowed us to study the stability of our assessment rubrics. The study itself was extensive and took two days to perform. The results indicate a large compliance when it comes to the final grade of the test. The study is meant to be repeated in the coming years.Keywords: Social science, civics, national testing, interrater reliability, intraclass correlation

Interaction in Assessment-Oriented Role Play: A Conversation Analytic Approach

2015 ◽  
Vol 38 (4) ◽  
Author(s):  
Jian Hu
Keyword(s):  
Conversation Analysis ◽  
Quantitative Methods ◽  
Role Play ◽  
Analytic Approach ◽  
Interactional Competence ◽  
Strategic Competence ◽  
Collaborative Task ◽  
Video Recordings ◽  
Chinese University ◽  
Pair Work

AbstractTo date, only a handful of studies have investigated the interlocutor effects on peer-peer test discourse, and they focus almost exclusively on the paired format in the Cambridge speaking tests, which is mostly a discussion type collaborative task. In the oral English test administered by a Chinese university under the present study, role-play is a major test type. This study chose for analysis three out of over 100 video recordings of test takers participating in roleplay- based interaction. The author adopted conversation analysis (CA) and Young’s (2000) constructivist, practice-oriented view of interaction and competence to assist the interpretation of speech exchange throughout the interaction. It is evident from the data that learners make use of various interactional resources and employ different strategies in the assessmentbased role play. It could be tentatively concluded from the conversation analysis of the paired interaction that the interaction framework together with the participants’ strategic competence in negotiating their own interactional resources, to a great extent, determines their joint performance of the collaborative task. The configuration of pairing in terms of proficiency is found to have an impact on joint interaction performance and strategic use. The implications of the current study include: interactional competence could be more readily accessed via role play than discussion type of pair work; qualitative conversation analysis of test takers’ actual practices can reveal what quantitative methods are unable to detect, and therefore is an indispensable complement.

‘Positioning’ in the conversation analytic approach

2013 ◽  
Vol 23 (1) ◽  
pp. 16-39 ◽  
Author(s):  
Dennis Day ◽  
Susanne Kjaerbeck
Keyword(s):  
Conversation Analysis ◽  
Positioning Theory ◽  
Analytic Approach ◽  
Micro Level ◽  
Do So

From the perspective of ethnomethodology and conversation analysis (EM/CA), the concept of positioning may offer a compellingly rich metaphor for understanding identity and relations. There appears, however, to be no such analytical concept in EM/CA. Instead, the EM/CA approach offers concepts such as alignment-affiliation, identities and membership categories — all of them based on actional resources on the micro-level of talk. The aim of this article is to inquire if EM/CA tools for the analysis of identities and relations in talk might be considered interesting from the perspective of positioning theory. To do so, we offer EM/CA analyses of narrative and non-narrative data in which the in situ negotiation of identities and relations plays a major role.

scholarly journals Nasjonale prøver – hemmende styringsverktøy eller lokale redskap for praksisutvikling?

2018 ◽  
Vol 12 (4) ◽  
pp. 3
Author(s):  
Ole Petter Vestheim
Keyword(s):  
Standardized Tests ◽  
Theoretical Perspective ◽  
Management Tools ◽  
Practice Architectures ◽  
Group Interviews ◽  
National Tests ◽  
Practice Development ◽  
Shed Light ◽  
Development Practice ◽  
Over Time

Artikkelen bygger på empiri fra en studie som hadde til hensikt å undersøke hva som kjennetegner praksis(er) på skoler som over tid har oppnådd gode resul¬tater på nasjonale prøver. Studiens hensikt var å utvikle forskningsbasert kunnskap om forhold som er av betydning for å lykkes på de nasjonale prøvene og avdekke hvilken plass prøvene hadde i skolenes praksis. Data ble innhentet med bruk av kvalitative singel- og gruppeintervjuer med rektorer og lærere fra sju skoler. Det teoretiske perspektivet og funnene i studien bidrar til å kaste lys over hvordan nasjonale prøver, som inngår i skolenes praksisarkitekturer, bidrar til praksisutvikling.Nøkkelord: standardiserte prøver, nasjonale prøver, skoleutvikling, praksis, praksisarkitekturerNational tests – inhibitory management tools or local tools for practice development?AbstractThe article is based on empirical evidence from a study that intended to investi¬gate what characterizes practice(s) in schools that have achieved good results in national tests over time. The purpose of the study was to develop research-based knowledge about conditions that are important for the success in the national tests and to reveal the place that the tests had in the practice of the schools. Data was obtained using qualitative single and group interviews with principals and teachers from seven schools. The theoretical perspective and the findings in the study help to shed light on how national tests, which are part of the schools’ practice architectures, contribute to practice development.Keywords: standardized tests, national tests, school development, practice, practice architectures

Spreektoetsen Moderne Vreemde Talen

1996 ◽  
Vol 54 ◽  
pp. 145-152
Author(s):  
Peter Paffen
Keyword(s):  
Secondary Education ◽  
Reliability And Validity ◽  
Oral Proficiency ◽  
Original Model ◽  
Proficiency Tests ◽  
Grammar School ◽  
Spoken English ◽  
School Based ◽  
Speech Test ◽  
New Research

In 1988 CITO started research into the feasibility of valid and reliable oral proficiency tests based on communicative principles. This was to meet the demand for a communicative speech test to be used in school based examinations in secondary education. Using the Test of Spoken English as a guideline, tests for French, German and English were developed. Simultaneous research into the reliability and validity of the tests led to various adaptations of the original model. From 1992 onwards oral proficiency tests for each of the three languages in question have been published at levels VBO/MAVO, HAVO and VWO (approxi-mately: vocational, secondary modern and grammar school). The results of a user inquiry held in 1994 led to a number of further changes to improve the user-friend-liness of the tests. Early in 1996 a new research project concerning the reliability and validity of the tests was started. The results will be published in the autumn of 1996.

scholarly journals Moving forward with membership categorization analysis: Methods for systematic analysis

2012 ◽  
Vol 14 (3) ◽  
pp. 277-303 ◽  
Author(s):  
Elizabeth Stokoe
Keyword(s):  
Conversation Analysis ◽  
Analytic Approach ◽  
Membership Categorization Analysis ◽  
Systematic Analysis ◽  
The Road ◽  
Hierarchical Relationship ◽  
Analysis Methods ◽  
Membership Categorization ◽  
Run Off ◽  
Separate Discipline

This article has four aims. First, it will consider explicitly, and polemically, the hierarchical relationship between conversation analysis (CA) and membership categorization analysis (MCA). Whilst the CA ‘juggernaut’ flourishes, the MCA ‘milk float’ is in danger of being run off the road. For MCA to survive either as a separate discipline, or within CA as a focus equivalent to other ‘generic orders of conversation’, I suggest it must generate new types of systematic studies and reveal fundamental categorial practices. With such a goal in mind, the second aim of the article is to provide a set of clear analytic steps and procedures for conducting MCA, which are grounded in basic categorial and sequential concerns. Third, the article aims to demonstrate how order can be found in the intuitively ‘messy’ discourse phenomenon of membership categories, and how to approach their analysis systematically as a robust feature of particular action-oriented environments. Through the exemplar analyses, the final aim of the article is to promote MCA as a method for interrogating culture, reality and society, without recourse to its reputed ‘wild and promiscuous’ analytic approach.

Sign in / Sign up

Export Citation Format

Share Document