Discourse Analysis in Second Language Speaking Assessment

Discourse analysis has been widely used in the field of language testing. This chapter provides an overview of research examining features of test-taker discourse across different task types and under different task conditions and the extent to which these features align with rating scale criteria. Attention is also drawn to discourse analytic studies of the language demands of study and work domains and the extent to which test tasks can elicit relevant features. The chapter concludes by reflecting on the challenges posed to existing high-stakes test constructs by increasing diversity in universities and workplaces and the potential for discourse analytic approaches to establish stronger alignments between testing practices and the aspects of spoken discourse relevant and valued in communication.

Dynamic Assessment of Learners of a Spoken Language

Dynamic assessment (DA), or the assessment of learning potential, is becoming recognized as an alternative method that has wide application within the assessment of language. In moving away from comparison to normative data, the assessment enables a wider range of children to be assessed, including all of those for whom the norms do not apply, such as children with autistic spectrum conditions, attention deficit hyperactivity disorder (ADHD), hearing loss, and bi- or multilingual children. In addition to differentiating language difficulties due to lack of experience with the target language from developmental language disorders (DLD), the DA procedure contributes a considerable amount of qualitative information about the learning skills of the test-taker. This chapter reviews the multiple models and methods of DA and the work that has been done to develop tools to assess language skills in first language learners.

Theoretical evaluation of partial credit scoring of the multiple-response test item

We compute and compare statistics of five different scoring rules for the selected-response type of test items where the number of keys is an arbitrary integer and the test-takers are perfectly rational agents. We consider a hypothetical test of factual recognition, in which the underlying ability that we seek to measure is the fraction of the item options that the test-taker truly recognizes (and not only guesses correctly), assumed directly proportional the test-taker’s domain knowledge. From these comparisons, two of these scoring rules are singled out as superior to the others.

Considerations for Addressing the Needs of Culturally and Linguistically Diverse Speech-Language Pathology Praxis Test Takers

Purpose The primary purpose of this article is to explore the speech-language pathology (SLP) Praxis test, a barrier to culturally and linguistically diverse (CLD) individuals entering the profession, by investigating first-time pass rates and mean scores by test taker race/ethnicity. Other potential barriers to licensure and certification, as well as solutions for mitigating these barriers, will also be addressed. Method SLP Praxis test data from two windows of time, 2008–2011 and 2014–2020, were compared for the following: (a) proportions of test taker race/ethnicity relative to U.S. demographic estimates of racial/ethnic group proportions overall, (b) proportions of racial/ethnic groups, and (c) trends in test-taker mean scores by race/ethnicity. First-attempt pass rates by racial/ethnic groups were also calculated for the 2014–2020 testing window. Results The percentage of some CLD SLP Praxis test-taker groups increased since the 2008–2011 testing window but is still not representative of U.S. racial/ethnic demographics. The first-attempt pass rates and overall mean scores of CLD test-taker groups remained substantially lower than White non–Hispanic/Latinx test takers. Conclusions Despite the encouraging trends in SLP Praxis test-taker racial/ethnic diversity, disparities persist between the racial/ethnic makeup of SLP Praxis test takers and the demographic makeup of the United States. Consequently, these disparities have implications for the continued lack of cultural representativeness seen in our workforce.

Penerapan Algoritma Fisher Yates untuk Pengacakan Soal Pada Sistem Ujian Kompetisi Wartawan

Procedures from the company to provide competency certification for and as official evidence that the news from the journalist is credible. In the practice of implementing the Competency Test, cheating is still found during the exam, various ways / forms of cheating that occur include cooperation between journalists (test participants). These problems are overcome by integrating the competency test system, using mobile technology and web applications that have been integrated so that they are more accessible. The Fisher Yates shuffle algorithm functions as a randomization of exam questions. Each test taker always has different exam questions, both from the numbering of questions and the order of answer choices. With the implementation of the Fisher Yates Suffle Algorithm, it can reduce acts of fraud or dishonesty by journalists. In addition, with the application of the Fisher Yates shuffle algorithm, it can make it easier for agencies to provide competent employee competencies. The results of the randomization test on the first 3 questions that appeared to the respondents resulted in the conclusion that the Fisher-Yates shuffle algorithm had a 100% success rate in randomizing the order of questions that were displayed to respondents

Answer Similarity Analysis at the Group Level

Answer similarity indices were developed to detect pairs of test takers who may have worked together on an exam or instances in which one test taker copied from another. For any pair of test takers, an answer similarity index can be used to estimate the probability that the pair would exhibit the observed response similarity or a greater degree of similarity under the assumption that the test takers worked independently. To identify groups of test takers with unusually similar response patterns, Wollack and Maynes suggested conducting cluster analysis using probabilities obtained from an answer similarity index as measures of distance. However, interpretation of results at the cluster level can be challenging because the method is sensitive to the choice of clustering procedure and only enables probabilistic statements about pairwise relationships. This article addresses these challenges by presenting a statistical test that can be applied to clusters of examinees rather than pairs. The method is illustrated with both simulated and real data.

Identifying Guessing in English Language Tests via Rasch Fit Statistics: An Exploratory Study

This article explores the issue of identifying guessers – with a specific focus on multiple-choice tests. Guessing has long been considered a problem due to the fact that it compromises validity. A test taker scoring higher than they should through guessing does not provide a picture of their actual ability. After an initial description of issues associated with guessing, the article then outlines approaches which have been taken to either discourage test takers from guessing or which attempt statistically to handle the problem. From this, the article moves to a novel way of identifying potential guessers: from the post hoc use of Rasch fit statistics. Two datasets, each consisting of approximately 200 beginner level English language test takers were split into two. In each dataset, half the test takers’ answers were randomised – to approximate guessing. Results obtained via a Rasch analysis of the data was then passed to an analyst who used the Rasch fit statistics to identify possible guessers. On each dataset, 80% of guessers were identified.

The impact of COVID-19 on psychometric assessment across industry and academia in South Africa

The coronavirus disease 2019 (COVID-19) pandemic had changed the world in unexpected ways and psychometric assessment was no exception. Despite the advancements made in online psychometric assessment implementation, the authors of this commentary reflected on their own experiences in the context of the psychology profession in South Africa, where psychology professionals had been faced with the dilemma of halting, postponing or adapting assessments for remote implementation. Remote implementation had many challenges, notably shifting the logistics, financial burden and accountability onto the test-taker. In addition, when implementing remote testing, considerations of supervised and unsupervised testing need to be considered in terms of flexibility, control, test-taker comfort, standardisation, costs, ethical concerns and crisis management. Whilst in the private sector, remote psychometric assessment had been met with resilience and innovation, in academia, remote psychometric research was faced with unique challenges which affect all aspects of the research process and access to participation. Across both industry and academia where psychometric assessments were conducted, the scores and results need to be interpreted with reflection and caution as the pandemic had led to an increase in psychological distress in addition to the unique contextual challenges that South Africa already faced.

FAKTOR PREDIKTOR KELULUSAN UJIAN KOMPETENSI NERS INDONESIA: TINJAUAN LITERATUR

Kelulusan mahasiswa keperawatan dalam mengikuti UKNI sejak periode ke-IV terus menerus menurun dan sangat rendah jika dibandingkan dengan beberapa negara yang tingkat kelulusannya diatas 80%. Tujuan Tujuan studi ini yaitu untuk melihat prediktor terhadap tingkat kelulusan uji kompetensi nasional yang dapat diajukan sebagai salah satu upaya perbaikan institusi Pendidikan. Metode Studi ini menggunakan desain literature review. Database yang digunakan yaitu PubMed, ClinicalKey Nursing, Google Scholar bahasa Inggris. Kata kunci sebagai berikut: nursing program “OR” nursing student “OR” undergraduate nursing program “AND” predictor NCLEX-RN “AND” NCLEX-RN Success. Pada database Google Scholar bahasa Indonesia, Portal Garuda dan pencarian sekunder menggunakan bahasa Indonesia dengan kata kunci program keperawatan “OR” mahasiswa perawat “OR” ners prekdiksi ujian kompetensi “OR” faktor yang mempengaruhi “DAN” Sukses Ukom. Kriteria inklusi yaitu (1) fokus pada prediktor ujian kompetensi, (2) diterapkan pada mahasiswa keperawatan, dan (3) dipublikasikan 10 tahun terakhir yaitu rentang tahun 2011-2020. Adapun kriteria eksklusi yaitu artikel yang tidak tersedia dalam teks lengkap, artikel berganda dan artikel yang tidak cocok dengan pertanyaan penelitian Hasil Hasil tinjauan literatur ini didapatkan 9 artikel yang sesuai dengan kriteria inklusi. Ada beberapa prediktor yang mempengaruhi keberhasilan dalam menghadapi ujian kompetensi diantaranya: prestasi akademik, hasil try out,bimbingan insentif, status akreditasi, Indepent test taker berbasis web e-learning dan faktor demografi.

Fostering Student Engagement in a Mobile Formative Assessment System for High-School Economics

In a mobile learning environment, students can learn via mobile devices without being limited by time and space. Therefore, it is vital to develop tools to assist students to learn and assess their knowledge in such environments. This paper presents a tool/application for formative self-assessment. The tool supports the selection of questions based on user-defined criteria concerning (1) the difficulty level; (2) the associated concepts; and (3) the purposes of the test taker. The main purpose of the presented tool is to better support the learning aims of the participants and to increase their engagement in the learning process. The focus of this study is to evaluate the tool using quizzes in Microeconomics to realize its potential in this specific domain. Teachers and students were involved in the experiments conducted. The experiments demonstrated that the presented tool is usable; it motivates the students and improves their understanding.