Predictability of IELTS in a high-stakes context: a mixed methods study of Chinese students’ perspectives on test preparation

High-stakes language tests are used around the world as a gatekeeping tool under the internationalization of higher education. However, the predictable aspect of the high-stakes language tests is seldom discussed, especially from students’ perspectives. This study aims to address this gap by aiming to better understand how certain factors and conditions contribute to the predictability issue of IELTS from students’ perspectives within a high-stakes context. This study used a mixed method approach to investigate the views and experiences of students within a Sino-UK joint college. The data collection was in two concurrent strands: online survey and group interviews. Findings suggested that IELTS can impact students negatively by narrowing their English learning scope, driving them into self-isolated way of study, doing repeated test-taking and buying predicted answers. Implications related to language test preparation are discussed in light of the findings.

Relating a concordance-based cloze test to the model of communicative language ability: a verbal protocol study

The literature of empirical studies on the concordance-based cloze test (ConCloze) is far and few between. This is despite the fact that it has a potential for item writing without the aid of native speakers and for making use of corpus-based technology in task design. This article explores the test-taking processes and strategies of a ConCloze item variant. The aim is to investigate the substantive aspect of the construct validity for the item type and increase the generalizability of the findings in the universe of admissible observations. The sample consists of 14 non-native English users who are in higher education and engage with 3 test tasks each, totaling 42 verbal reports. The sampling method is purposive sampling, in which their first language profiles are maximized in terms of heterogeneity for increased power of generalization. It is found that Reading concordance lines and recognizing clue words inside is a major process, and Assessing item components and testing a meaningful compatibility of a word in context a secondary one. A model of communicative language ability is used to provide a context for task use in this study, where strategic competence is represented in ConCloze substantive validity.

The practice of online assessment in an EFL context amidst COVID-19 pandemic: views from teachers

For many years, technology has been applied to improve the quality of language learning and teaching. However, the outbreak of COVID-19 pandemic accelerated the integration of technology in different language learning contexts. The sudden shift to online teaching faced educators with an array of challenges they had not experienced before. In particular, the teachers encountered many barriers with the online assessment of the students. To shed more light on what passed, following a qualitative mode of inquiry, the present study set out to find out how a group of 20 Iranian English language teachers at Persian Gulf University faced with the online assessment challenges posed by COVID-19. For this aim, the researchers conducted in-depth semi-structured retrospective interviews with the teachers at different times throughout the course. In addition, the teachers were asked to provide a narrative account of how they responded to the crisis. The analysis of the findings showed that after the shift to online assessment, the teachers initially faced with serious pedagogical, technical, administrative, and affective barriers, but as the course proceeded, they could adjust their practice with the new situation. However, the teachers recounted problems that still remained and negatively affected their practice. Overall, the study discusses that to improve the online assessment in the post-COVID-19 era several technological, pedagogical, and administrative measures should be taken into account. These would further improve the integration of the technology in the pedagogical context in the long run.

A phenomenographic study on language assessment literacy: hearing from Iranian university teachers

This study aims to disclose the Iranian university teachers’ perceptions of the fundamentals of language assessment literacy (LAL). To this aim, using purposive sampling, eighteen university teachers from two Iranian universities were invited to participate in semi-structured interviews. Their viewpoints were audio-recorded, transcribed, and subjected to a phenomenographic analysis. Findings yielded two overarching LAL domains: knowledge (e.g., having an acceptable level of digital LAL, satisfying ethical requirements, benefiting more from performance assessment, considering students’ individual differences, making assessment valid, assuring that tests are reliable, and having an acceptable level of pedagogical content knowledge) and skills (e.g., involving students in assessment, using alternative assessment methods, employing logically traditional assessment methods, informing students about test results, administering tests in standardized ways, using valid grading procedures, and bringing positive wash-back effects). After discussing the results, the study concludes with proposing a range of implications for different testing stakeholders and highlighting some avenues for further research.

The rise of contract cheating during the COVID-19 pandemic: a qualitative study through the eyes of academics in Kuwait

Contract cheating has gone rampant in higher education recently. When institutions switched to online learning during the COVID-19 pandemic, the percentage of contract cheating students climbed to unprecedented levels. Essay mills saw the lack of face-to-face interaction and proctoring on campus as an opportunity and used aggressive marketing methods to attract students. This study asked the opinions of 20 faculty members working in the English departments of private higher education institutions in Kuwait regarding contract cheating through interviews. The data was analyzed with MAXQDA 2020. The findings show that all faculty members can recognize contract cheating easily. Most of them see contract cheating as a serious problem in the higher education system, a threat to the reliability of language assessment, triggered by laziness, the social pressure to graduate with a high GPA, and exacerbated by the cheating opportunities in online education. Academics have developed certain individual strategies in their courses to curb the number of contract cheating students; however, institutional measures differ, and in some, there are no measures or sanctions on contract cheating students.

Comparison of test performance on paper-based testing (PBT) and computer-based testing (CBT) by English-majored undergraduate students in China

Computer-based testing (CBT), which refers to delivering assessments with computers, has been widely used in large English proficiency tests worldwide. Despite an increasing CBT in China, limited research is available concerning whether CBT can be used for the Test for English Majors-Band 4 (TEM 4). The current study investigated whether testing mode impacted TEM 4 score and factors (i.e., computer familiarity level and attitude towards CBT) that might correlate with performance on CBT of TEM 4. Overall 92 Chinese undergraduate students were randomly assigned to one of the groups, i.e., CBT or paper-based testing (PBT), and took the test. A mixed method was employed, including (1) quantitative and qualitative analysis of test performance in two modes, as well as CBT group participants’ computer familiarity and attitudes towards the mode; and (2) thematic analysis of semi-structured interviews. The results revealed that (1) test scores in CBT and PBT were comparable; (2) two items in the computer familiarity questionnaire, i.e., comfort level of reading articles on the computer and forgetting time when using computers, positively correlated with CBT scores; and (3) participants’ attitude towards CBT did not impact test performance.

Test-level and Item-level Model Fit Comparison of General vs. Specific Diagnostic Classification Models: A Case of True DCM

The current study compared the model fit indices, skill mastery probabilities, and classification accuracy of six Diagnostic Classification Models (DCMs): a general model (G-DINA) against five specific models (LLM, RRUM, ACDM, DINA, and DINO). To do so, the response data to the grammar and vocabulary sections of a General English Achievement Test, designed specifically for cognitive diagnostic purposes from scratch, was analyzed. The results of the test-level-model fit values obtained strong evidence in supporting the G-DINA and LLM models possessing the best model fit. In addition, the ACDM and RRUM were almost very identical to that of the G-DINA. The value indices of the DINO and DINA models were very close to each other but larger than those of the G-DINA and LLM. The model fit was also investigated at the item level, and the results revealed that model selection should be performed at the item level rather than the test level, and most of the specific models might perform well for the test. The findings of this study suggested that the relationships among the attributes of grammar and vocabulary are not ‘either-or’ compensatory or non-compensatory but a combination of both.

The comparative effect of group dynamic assessment (GDA) and computerized dynamic assessment (C-DA) on Iranian upper-intermediate EFL learners’ speaking complexity, accuracy, and fluency (CAF)

This study attempted to check the impact of two dynamic assessment (DA) models on speaking CAF. DA, as opposed to static assessment, is conceived as an interactive approach to assessment that integrates teaching and testing into a unified instructional engagement. To achieve the goals of this research, a convenience sample of 90 upper-intermediate male EFL learners that were randomly assigned into GDA, a C-DA, and a non-DA control group participated in the study. Before carrying out the treatment, a speaking pretest was administered to all three groups and their CAF scores were collected. Following that, the treatment using the aforementioned DA and non-DA conventional models was completed in 16 sessions. To check the impact of the treatment, a speaking post-test was given to the groups at the end of the study. Data analysis using ANOVA showed that C-DA and G-DA could significantly increase speaking CAF than the conventional non-DA instruction with C-DA being significantly better than G-DA. The results of this research propose that implementing DA, especially C-DA by the teachers, can enhance the speaking CAF of the L2 learners.

Assessing Iranian EFL teachers’ educational performance based on gender and years of teaching experience

In recent years, there has been an increasing interest in the assessment of Iranian English teachers’ performance. Besides, it was aimed to examine and compare the performance of teachers based on their gender differences and teaching experience. To the first aim, the Delphi technique was used to develop a questionnaire and the reliability of the Delphi questionnaire based on Cronbach’s alpha was .982. In the first round, 25 experts including university lecturers and experienced instructors in the field of English teaching were asked to answer open-ended questions regarding important issues in the evaluation of an English teacher. Then, the related themes emerged. Using emerged themes, a questionnaire including 100 questions was designed and measured on a linear scale (1 = not important to 5 = absolutely essential). After calculating the frequency of each item, the results were resent to the panel to rate the questions. In the last phase, three criteria including 1—the mean 4 and more, 2—standard deviation less than 1, 3—less than 10% of the participants do not answer to the item were considered to decide on the final questions and components of the questionnaire. The questionnaire was distributed using Google forms. One hundred and fifty questionnaires were filled correctly and analyzed using SPSS 22. Then, the validity of questionnaire was checked. Overall, it was seen that there was not any significant difference between teachers’ performance based on gender difference and teaching experience. The findings of the present study might have some implications for researchers, instructors, language teachers, school administrators, and the ministry of education.