Palestinian EFL University Students’ Problems with the Reading Sections of the TOEFL Internet-based Test and the Revised TOEFL paper-delivered Test

Despite Palestinian university students’ problems with the Test of English as a Foreign Language Internet-based Test, no researchers approached this research area in the Palestinian English as a Foreign Language context. The present study attempted to answer a question focusing on Palestinian university students’ problems with the reading sections of the Test of English as a Foreign Language Internet-based Test and the revised Test of English as a Foreign Language paper-delivered test. The participants were 65 fourth-year students studying English at Al-Aqsa University. The researcher employed four instruments: a test, a group semi-structured interview, and two individual semi-structured interviews. Results showed the students’ problems with the reading sections of the two types of tests, such as the irrelevance of the topics of the tests to the students’ content background knowledge, students’ lack of exposure to lengthy passages, students’ slow reading speed, and the students’ lack of linguistic proficiency. Additionally, the study offered many recommendations for Educational Testing Service experts and Al-Aqsa university students and instructors.

Palestinian EFL University Students’ Problems with the Reading Sections of the TOEFL Internet-based Test and the Revised TOEFL paper-delivered Test

Despite Palestinian university students’ problems with the Test of English as a Foreign Language Internet-based Test, no researchers approached this research area in the Palestinian English as a Foreign Language context. The present study attempted to answer a question focusing on Palestinian university students’ problems with the reading sections of the Test of English as a Foreign Language Internet-based Test and the revised Test of English as a Foreign Language paper-delivered test. The participants were 65 fourth-year students studying English at Al-Aqsa University. The researcher employed four instruments: a test, a group semi-structured interview, and two individual semi-structured interviews. Results showed the students’ problems with the reading sections of the two types of tests, such as the irrelevance of the topics of the tests to the students’ content background knowledge, students’ lack of exposure to lengthy passages, students’ slow reading speed, and the students’ lack of linguistic proficiency. Additionally, the study offered many recommendations for Educational Testing Service experts and Al-Aqsa university students and instructors.

Identifying Mixture Components From Large-Scale Keystroke Log Data

In a computer-based writing assessment, massive keystroke log data can provide real-time information on students’ writing behaviors during text production. This research aims to quantify the writing process from a cognitive standpoint. The hope is that the quantification may contribute to establish a writing profile for each student to represent a student’s learning status. Such profiles may contain richer information to influence the ongoing and future writing instruction. Educational Testing Service (ETS) administered the assessment and collected a large sample of student essays. The sample used in this study contains nearly 1,000 essays collected across 24 schools in 18 U.S. states. Using a mixture of lognormal models, the main findings show that the estimated parameters on pause data are meaningful and interpretable with low-to-high cognitive processes. These findings are also consistent across two writing genres. Moreover, the mixture model captures aspects of the writing process not examined otherwise: (1) for some students, the model comparison criterion favored the three-component model, whereas for other students, the criterion favored the four-component model; and (2) students with low human scores have a wide range of values on the mixing proportion parameter, whereas students with higher scores do not possess this pattern.

The Net Generation: Tech-savvy or lost in virtual space?

Recent findings from PEW Internet and American Life studies in the US, the JISC Information Behaviour of the Researcher of the Future report and the Educational Testing Service 2006 ICT literacy assessment, all indicate that the students from the Y or Net Generation are not as tech-savvy as portrayed by the world’s media and large Internet software providers. If this is the case, then assumptions currently being made about the information-seeking behaviour of today’s students need to be rectified at the school level to ensure that tomorrow’s citizens are not disenfranchised or disempowered as users in a world where Governments are increasingly committed to the provision of essential services and information wholly online. This paper discusses the secondary results of a much larger PhD study on the information seeking behaviour of the Net Generation and the need for schools and particularly teacher librarians, to become more involved in teaching students how to use the electronic environment effectively.

Strategies Proposed for Listening Comprehension in PBT TOEFL: Tricks that Do Not Work

The Longman Preparation Course for the TOEFL, written by Phillips in 2003, is the most widely used coursebook for TOEFL test preparation. This book provides 17 strategies for Listening Comprehension Part A. Therefore, the current study examines whether the strategies provided in the book are applicable in a real test. The data were obtained from Part A of the TOEFL listening section in nine tests designed by Educational Testing Service (ETS). Each item in the test was tested against the proposed strategies. The results reveal that many items did not successfully match the strategies. Only two proposed strategies had matches of higher than 80%, while others were less than 10%. There are four strategies that did not have a match at all. All in all, very few strategies can be applied to answer the questions correctly without adequate comprehension. With comprehension, test takers can answer the questions without needing to rely on any of the proposed strategies. Therefore, the study concludes that most of the strategies proposed by Phillips (2003) are not successful in real test.Key words: strategies, PBT TOEFL, listening comprehension Abstrak: The Longman Preparation Course for the TOEFL yang ditulis oleh Phillips in 2003 paling banyak digunakan untuk pelatihan persiapan TOEFL. Buku tersebut berisi 17 strategi untuk menjawab soal TOEFP pada Part A. Oleh karena itu, penelitian ini menelaah apakah strategi dalam buku tersebut dapat digunakan untuk mengerjakan tes TOEFL yang sebenarnya. Data untuk penelitian ini diperoleh dari tes TOEFL bagian listening Part A yang diperoleh dari Educational Testing Service (ETS). Setiap soal pada tes tersebut dicocokkan dengan strategi yang terdapat dalam buku Phillips (2003). Hasil penelitian menunjukkan bahwa banyak soal tidak dapat dijawab dengan menggunakan strategi yang dianjurkan. Hanya ada dua strategi yang memiliki kecocokan lebih dari 80%, sedangkan yang lain kurang dari 10%. Ada dua strategi yang tidak cocok dengan satu soal manapun. Secara keseluruhan, sangat sedikit strategi yang dapat digunakan untuk menjawab soal TOEFL listening Part A tanpa pemahaman yang cukup. Dengan pemahaman, peserta tes bisa menjawab soal tanpa harus menggunakan strategi-strategi tersebut. Oleh karena itu, penelitian ini menyimpulkan bahwa kebanyakan strategi yang dikemukakan oleh Phillips (2003) tidak dapat digunakan untuk menjawab soal TOEFL asli.Kata kunci: strategi, PBT TOEFL, listening comprehension.

Using technology-assisted peer feedback to improve academic writing

The poster discusses the possibilities of technology-assisted peer feedback in English as a Foreign Language (EFL) writing classrooms among low proficiency students. It is a part of an ongoing research project developed after a study conducted in the first half of 2019 (Lazic & Tsuji, 2020a, 2020b). The first goal is to explore the effectiveness of in-class activities, which include technology-assisted peer feedback, in improving global aspects of writing, i.e. paragraph structure and content, and to examine the uptake of peer feedback delivered via an Automated Writing Evaluation tool (AWE), Educational Testing Service (ETS) Criterion®. Second, the study looks at students’ perceptions. Participants were 15 first-year students taking an academic writing class.

Machine Learning im Bildungskontext: Evidenz für die Genauigkeit der automatisierten Beurteilung von Essays im Fach Englisch

Zusammenfassung. Argumentatives Schreiben ist eine bedeutsame Kompetenz in der Fremdsprache Englisch. Entsprechende Schreibaufgaben sind Teil von Schulabschlussprüfungen in der Sekundarstufe II und von Zugangstests für Hochschulen (z.B. TOEFL®). Trotz ihrer Bedeutsamkeit wurden diese komplexen Schreibleistungen bisher im Kontext großer Schulleistungsuntersuchungen kaum empirisch untersucht. Ein Grund dafür ist die aufwendige Auswertung der Essays, für die eine große Anzahl speziell trainierter Kodiererinnen und Kodierer zur Beurteilung benötigt wird. Um den Aufwand der Auswertung zu reduzieren, können Machine Learning Verfahren eingesetzt werden, welche die Urteile der Kodiererinnen und Kodierer approximieren. Dabei werden linguistische Eigenschaften der Essays automatisiert erfasst, die dann genutzt werden, um mit Hilfe von statistischen Verfahren des maschinellen Lernens die menschlichen Urteile vorherzusagen. In der vorliegenden Arbeit soll dieses Vorgehen dargestellt und das Potenzial solcher automatisierten Prozeduren in Bezug auf die Vorhersagegenauigkeit untersucht werden. Dazu lagen Texte von N = 2179 Schülerinnen und Schülern der 11. Jahrgangsstufe in Deutschland und der Schweiz vor. Zur Kodierung der Texte wurde die open source-Software The Common Text Analysis Platform (CTAP) eingesetzt, die linguistische Textmerkmale automatisch kodiert. Auf Basis dieser Textmerkmale wurden die vorliegenden Urteile von trainierten Kodiererinnen und Kodierern des Educational Testing Service (ETS) vorhersagt. Die Genauigkeit der Vorhersage erwies sich als zufriedenstellend ( r = .75; Anteil genauer Übereinstimmung: 42%) und konnte im Vergleich mit einer etablierten kommerziellen Software des ETS (e-rater®; r = .81; Anteil genauer Übereinstimmung: 42%) bestehen. Es wurden vergleichbare Ergebnisse für die lineare Regression sowie Gradient Boosting als Analysestrategien zur Vorhersage der menschlichen Urteile gefunden. Möglichkeiten und Limitationen der automatisierten Textbeurteilung und deren Anwendung in Forschung und Praxis werden diskutiert.

The The Relationship between Speaking and Writing Performance in an Indonesian Senior High English Foreign Language (EFL) Classroom

Purpose - This research aims to figure out the relationship between those two productive skills focusing on four aforementioned crucial aspects in language learning. Design/methodology/approach - This study used a cross-sectional quantitative method with 32 English Education Department students in the 6th semester at Universitas Islam Negeri Sunan Ampel Surabaya who have already been experienced in spoken and written class to be the participants. In terms of gaining the data, It used pictured cued interviews and answer the open-ended question related to the best way in finding a job and give their opinion within 30 minutes. Both of the tests were adapted from the TOEIC sample test from the Educational Testing Service (ETS). Finally, use SPSS 2.2 for the data analysis. Findings - The result shows that the organization gains the highest rate or simply as the most related part among the other language learning. Furthermore, the correlation between the two productive skills also shows a moderate correlation. Originality/value - It simply said that speaking and writing show an insignificant correlation with the organization as the most related part of language learning. Paper type – Research paper

Examining consistency among different rubrics for assessing writing

The literature on using scoring rubrics in writing assessment denotes the significance of rubrics as practical and useful means to assess the quality of writing tasks. This study tries to investigate the agreement among rubrics endorsed and used for assessing the essay writing tasks by the internationally recognized tests of English language proficiency. To carry out this study, two hundred essays (task 2) from the academic IELTS test were randomly selected from about 800 essays from an official IELTS center, a representative of IDP Australia, which was taken between 2015 and 2016. The test takers were 19 to 42 years of age, 120 of them were female and 80 were males. Three raters were provided with four sets of rubrics used for scoring the essay writing task of tests developed by Educational Testing Service (ETS) and Cambridge English Language Assessment (i.e., Independent TOELF iBT, GRE, CPE, and CAE) to score the essays which had been previously scored officially by a certified IELTS examiner. The data analysis through correlation and factor analysis showed a general agreement among raters and scores; however, some deviant scorings were spotted by two of the raters. Follow-up interviews and a questionnaire survey revealed that the source of score deviations could be related to the raters’ interests and (un)familiarity with certain exams and their corresponding rubrics. Specifically, the results indicated that despite the significance which can be attached to rubrics in writing assessment, raters themselves can exceed them in terms of impact on scores.