Measuring English language proficiency across subgroups: Using score equity assessment to evaluate test fairness

2018 ◽  
Vol 36 (2) ◽  
pp. 289-309 ◽  
Author(s):  
Hanwook Yoo ◽  
Venessa F. Manna ◽  
Lora F. Monfils ◽  
Hyeon-Joo Oh
Keyword(s):  
Language Proficiency ◽  
Test Scores ◽  
English Language ◽  
Educational Background ◽  
Test Taker ◽  
Background Characteristics ◽  
Test Fairness ◽  
Educational Backgrounds ◽  
Language Tests ◽  
Language Exposure

This study illustrates the use of score equity assessment (SEA) for evaluating the fairness of reported test scores from assessments intended for test takers from diverse cultural, linguistic, and educational backgrounds, using a workplace English proficiency test. Subgroups were defined by test-taker background characteristics that research has shown to be associated with performance on language tests. The characteristics studied included gender, age, educational background, language exposure, and previous experience with the assessment. Overall, the empirical results indicated that the statistical and psychometric methods used in producing test scores were not strongly influenced by the subgroups of test takers from which the scores were derived. This result provides evidence in support of the comparability and meaning of test scores across the various test-taker groups studied. This example may encourage language testing programs to incorporate SEA analyses to provide evidence to inform the validity and fairness of reported scores for all groups of test takers.

scholarly journals Examining sources of variability in repeaters’ L2 writing scores: The case of the PTE Academic writing section

2018 ◽  
Vol 36 (1) ◽  
pp. 3-25 ◽  
Author(s):  
Khaled Barkaoui
Keyword(s):  
Language Proficiency ◽  
Academic Writing ◽  
Test Scores ◽  
English Language ◽  
English Language Proficiency ◽  
Rate Of Change ◽  
Test Taker ◽  
Test Occasion ◽  
Writing Scores ◽  
Over Time

This study aimed to examine the sources of variability in the second-language (L2) writing scores of test-takers who repeated an English language proficiency test, the Pearson Test of English (PTE) Academic, multiple times. Examining repeaters’ test scores can provide important information concerning factors contributing to changes in test scores across test occasions. Data consisted of the scores and background data (e.g., gender, age) and other covariates (e.g., context, interval between tests, number of tests attempted) for a sample of 1,000 test-takers who each took PTE Academic three times or more. Multilevel modeling was used to estimate the contribution of various factors to variability in repeaters’ PTE Academic writing scores across test-takers and test occasions. The findings indicated that changes in PTE Academic writing scores followed a quadratic trajectory (i.e., initial score increases followed by a decline) and that, as expected, test-taker initial overall English language proficiency (as measured on other sections of the test) was the strongest predictor of differences in PTE Academic writing scores at test occasion one as well as variance (across test-takers) in the rate of change in writing scores over time. Measures of retesting effects were not significantly associated with changes in writing scores, while test-taker factors (e.g., age, gender, and purpose for taking the test) were significantly associated with writing scores at test occasion one, but not with the rate of change in writing scores over time. The study highlights the value of examining repeater’ L2 test scores and concludes with a call for more research on the sensitivity of L2 proficiency tests to changes in L2 proficiency over time and in relation to L2 instruction.

scholarly journals MALAYSIAN STUDENTS’ PERCEPTIONS OF THEIR INTERACTIONS IN UK SEMINARS

2020 ◽  
Vol 5 (2) ◽  
pp. 384-405
Author(s):  
Zurina Khairuddin ◽  
Zulaikha Khairuddin ◽  
Nadia Ibrahim
Keyword(s):  
Language Proficiency ◽  
International Student ◽  
English Language ◽  
Group Discussion ◽  
Educational Background ◽  
First Year ◽  
Verbal Interactions ◽  
First Year Students ◽  
Academic Cultures ◽  
Malaysian Students

Background and Purpose: Coming from different social and academic cultures, students may exhibit perceptions which are in contrast to the convention of the targeted culture. Hence, this study aims to explore how first-year Malaysian students perceive their interactions in UK seminars.   Methodology: Employing qualitative research method, nine Malaysian students were chosen based on two criteria: they have to be first-year students and registered in content modules in UK university. The data were collected from interview and focus group discussion, and the audio-recordings were transcribed and analysed utilising thematic analysis.   Findings: The participants of this study perceive their interactions differently. Semek, Enot, Fatin, Ammar, Ming and Izlin mentioned that they responded during seminar discussion if they were nominated by the tutor and most agreed that they did not ask questions because information was provided to them or they could ask the tutor or their friends after the seminar. Fatin however disagreed and asserted that students should ask questions if they did not understand the academic content well. Qaisara, Puspa and Semek also shared that their lack of English language proficiency influenced their lack of contribution which was not the case for Fatin and Ming. Being marginalised as an international student was one of the reasons Semek did not contribute to the seminar discussions in contrast to Qaisara who felt the need to contribute.   Contributions: The findings of this study suggest that academic institutions should be flexible in encouraging students to be engaged in seminars as coming from different educational background, these students may require additional help to socialise in academic setting and consequently become expert members.   Keywords: Malaysian students, students’ perceptions, students’ interactions, UK seminars, verbal interactions.   Cite as: Khairuddin, Z., Khairuddin, Z., & Ibrahim, N. (2020). Malaysian students’ perceptions of their interactions in UK seminars.  Journal of Nusantara Studies, 5(2), 384-405. http://dx.doi.org/10.24200/jonus.vol5iss2pp384-405

scholarly journals Chinese University Students’ Experience of WeChat-Based English-Language Vocabulary Learning

2021 ◽  
Vol 11 (9) ◽  
pp. 554
Author(s):  
Fan Li ◽  
Si Fan ◽  
Yanjun Wang ◽  
Jinjin Lu
Keyword(s):  
University Students ◽  
Language Proficiency ◽  
Test Scores ◽  
Online Teaching ◽  
English Language ◽  
Northern China ◽  
Chinese University ◽  
Chinese University Students ◽  
The Difference ◽  
The Many

The outbreak of COVID-19 worldwide in 2020 has posed tremendous challenges to higher education globally. Teaching English as a foreign language (TEFL) is among the many areas affected by the pandemic. The unexpected transition to online teaching has increased challenges for improving and/or retaining students’ language proficiency. WeChat, a popular social application in China, was widely used for TEFL at Chinese universities before COVID-19. However, it remains unclear whether the use of WeChat can facilitate Chinese university students’ English-language lexical proficiency during the pandemic. To fill this gap, the aim of the present study was two-fold: (1) it initially explored the relationship between the variables including students’ academic years, genders, and academic faculties/disciplines, and their lexical proficiency; and (2) it evaluated the effectiveness of a WeChat-assisted lexical learning (WALL) program in facilitating learning outcomes of English-language vocabulary. One hundred and thirty-three students at a university in Northern China participated in the WALL program for three weeks. As the results indicated, the independent variables had no correlation with the students’ lexical proficiency. More importantly, the students had a decline in the test scores after using the program, compared to their initial test scores. Moreover, the difference was reported to be medium. The findings further proposed questions on applying WeChat to vocabulary teaching in a large-scaled transition. The study is expected to provide insights for tertiary institutions, language practitioners, and student stakeholders to troubleshoot the potential problems regarding implementing WeChat-based TEFL pedagogies.

scholarly journals Enriching Cognizance at the Tertiary Level Students During Transition

2018 ◽  
Vol 7 (4.36) ◽  
pp. 665
Author(s):  
K. Jeyagowri ◽  
R. Abilasha ◽  
M. Ilankumaran
Keyword(s):  
Language Learning ◽  
Language Proficiency ◽  
English Language ◽  
Language Skills ◽  
Educational Background ◽  
Cognitive Skill ◽  
English Language Proficiency ◽  
Tertiary Level ◽  
College Level ◽  
Learning Skills

English language, as a global lingua franca, has its own respect and importance throughout the world. It is considered to be the shrine of knowledge. It is significant to acquire the English language proficiency, to be placed in good jobs. Though every college gives prime importance to English language, unfortunately most of the students fail to acquire good command over the language. In certain cases, students have the poor ability to comprehend learning at the college level because of their different educational background or capability. There are certain hurdles that affect the students from learning the language. Cognizance is an important component of the tertiary level students. Enriching cognizance of students refers to how the learners develop their knowledge in order to get a better idea of learning, awareness of the language skills they notice, and the observation of language learning. The objective of cognizance is to provide more knowledge to students. It promotes self-motivation of the students and enhances their basic skills of learning. Cognitive skill develops the student’s learning skills, notice, recollection and thoughts. This article deals with how cognizance in language learning is developed and thereby to sharpen their skills of learning. Furthermore, this paper analyses the causes and effects of the students’ transitional dilemma at the tertiary level in acquiring language efficacy. It offers some ideas and results based on the problems among the students in attaining the fluency in communication through developing cognition skills. Moreover, the study proposes to help the students to achieve the objective of education.   

Vocabulary learning

2011 ◽  
Vol 34 (2) ◽  
pp. 193-215 ◽  
Author(s):  
Farzad Dibaj
Keyword(s):  
Second Language ◽  
Language Learners ◽  
Language Proficiency ◽  
English Language ◽  
Vocabulary Learning ◽  
Educational Background ◽  
Vocabulary Knowledge ◽  
Third Language ◽  
Knowledge Scale ◽  
Word Difficulty

The purpose of this study is to compare the vocabulary learning of monolingual learners of English as a second language with bilingual learners of English as a third language. The study is based on data from 52 monolingual Persian-speaking learners of English and 45 bilingual Azeri-Persian-speaking learners of English. All the participants were females studying English as a foreign language at two universities in Iran. The informants were exposed to two incidental and four intentional vocabulary learning exercises. They were then measured at four difficulty levels using the Vocabulary Knowledge Scale (Paribakht & Wesche, 1997). Variables such as English language proficiency, intelligence, family educational background, gender, age and type of university were controlled. The third language learners outperformed their second language counterparts at all word difficulty levels. The findings are discussed in relation to bilinguals’ higher level of executive and inhibitory control.

scholarly journals Communicative Competence of College English Language Teachers: Using Results as Basis For An Action Plan

2021 ◽  
Vol 04 (06) ◽  
Author(s):  
Ma . Fe B. Belasoto ◽  
Keyword(s):  
Language Proficiency ◽  
Significant Relationship ◽  
English Language ◽  
Communicative Competence ◽  
Action Plan ◽  
Language Teachers ◽  
College English ◽  
Significant Difference ◽  
English Language Teachers ◽  
Language Exposure

This study determined the communicative competence of the college English language teachers of Northern Iloilo Polytechnic State College (NIPSC) in the 5th District of Iloilo through a researcher-developed instrument which had undergone validity and reliability testing. Utilizing mixed method approach and with forty-five (45) English language teachers who were chosen through complete enumeration, result showed that the respondents’ level of communicative competence in the areas of grammatical, sociolinguistic, strategic and discourse was “very good”, while the extent of their English language exposure was “sometimes” for the 95.56% and “always” for only 4. 44%. There was no significant relationship between the respondents’ number of years in teaching, written language proficiency, relevant seminars and trainings attended and all areas of communicative competence. Among the four areas of communicative competence, grammatical and sociolinguistic competence had significant relationship with their oral language proficiency and discourse competence had significant relationship with highest educational attainment. There was no significant relationship between their communicative competence and extent of English language exposure. There was no significant difference between the level of communicative competence of the respondents when they were grouped according to campus. The following factors were perceived to contribute to their communicative competence: the practice of the English language inside and outside the classroom, exposure to mainstream media, experience as English teachers, inherent intelligence, seminars or trainings attended, while the extent of their language exposure was perceived to be influenced by time, attitude or preference of the teacher, environment, teaching load, co-workers and students. Based on the result of the study an action plan was proposed to improve the efficiency, competence, and performance of the English language teachers at the College.

scholarly journals The Relationship Between Lexical Frequency Profiling Measures and Rater Judgements of Spoken and Written General English Language Proficiency on the CELPIP-General Test

2016 ◽  
Vol 32 ◽  
pp. 43 ◽  
Author(s):  
Scott Roy Douglas
Keyword(s):  
Language Proficiency ◽  
English Language ◽  
English Language Proficiency ◽  
Lexical Frequency ◽  
Frequency Bands ◽  
Language Tests ◽  
General Test ◽  
Quota Sampling ◽  
The Relationship ◽  
Independent Confirmation

Independent confirmation that vocabulary in use unfolds across levels of performance as expected can contribute to a more complete understanding of validity in standardized English language tests. This study examined the relationship between Lexical Frequency Profiling (LFP) measures and rater judgements of test-takers’ overall levels of performance in the Speaking and Writing modules of the CELPIP-General test. In particular, the potential of measures such as lexical stretch and number of frequency bands accessed was examined. Randomized quota sampling from previously rated test-taker responses resulted in 200 speaking samples and 200 writing samples being compiled to create corpora of 211,602 running words and 70,745 running words respectively. Pearson r was used to examine the relationships between the LFP measures and rater judgements of CELPIP levels. Results point to significant correlations, with increasing CELPIP levels of performance generally accompanied by test-takers’ increasing ability to produce greater numbers of words, deploy a greater variety of words, rely less on high-frequency vocabulary, tap into mid-frequency vocabulary, and access a greater number of frequency bands. These results underline the contribution of independently obtained lexical measures toward a fuller understanding of concurrent validity in standardized English language proficiency testing. La confirmation indépendante que le vocabulaire d’usage se répand sur plusieurs niveaux de performance tel que prévu peut contribuer à une meilleure interpréta- tion de la validité des tests standardisés de langue anglaise. Ce e étude a examiné le rapport entre les mesures de profilage de la fréquence lexicale et les évalua- tions de la performance globale des élèves aux modules de parole et de rédaction du Programme canadien d’évaluation du niveau de compétence linguistique en anglais (CELPIP). Plus précisément, on a examiné le potentiel des mesures telles l’étendue lexicale et le nombre de bandes de fréquences a eintes. L’échantillon- nage par quota aléatoire de réponses d’élèves déjà évaluées a entrainé la formation de 200 échantillons de parole et 200 échantillons de rédaction représentant deux corpora, un de 211 602 mots liés et l’autre de 70 745 mots liés. On a employé le coe cient de corrélation de Pearson pour examiner les rapports entre les mesures de la fréquence lexicale et les évaluations en fonction des niveaux du CELPIP. Les résultats dévoilent des corrélations signi catives entre, d’une part, les meilleures performances au CELPIP et, d’autre part, une capacité à produire une quantité et une variété plus importantes de mots; à moins recourir aux mots les plus fréquents; à puiser dans du vocabulaire à fréquence moyenne; et à accéder à un plus grand nombre de bandes de fréquence. Ces résultats soulignent la contribution des mesures lexicales obtenues indépendamment à la compréhension de la validité concourante des évaluations standardisées des compétences linguistiques en anglais. 

Measuring English language workplace proficiency across subgroups: Using CFA models to validate test score interpretation

2016 ◽  
Vol 34 (1) ◽  
pp. 101-126 ◽  
Author(s):  
Hanwook Yoo ◽  
Venessa F. Manna
Keyword(s):  
Factor Structure ◽  
English Language ◽  
Factor Model ◽  
Empirical Support ◽  
International Communication ◽  
Test Taker ◽  
Background Characteristics ◽  
Language Abilities ◽  
Score Interpretation ◽  
Study Results

This study assessed the factor structure of the Test of English for International Communication (TOEIC®) Listening and Reading test, and its invariance across subgroups of test-takers. The subgroups were defined by (a) gender, (b) age, (c) employment status, (d) time spent studying English, and (e) having lived in a country where English is the main language. The study results indicated that a correlated two-factor model corresponding to the two language abilities of listening and reading best accounted for the factor structure of the test. In addition, the underlying construct had the same structure across the test-taker subgroups studied. There were, however, significant differences in the means of the latent construct across the subgroups. This study provides empirical support for the current score reporting practice for the TOEIC test, suggests that the test scores have the same meaning across studied test-taker subgroups, and identifies possible test-taker background characteristics that affect English language abilities as measured by the TOEIC test.

Sign in / Sign up

Export Citation Format

Share Document