Integrated writing scores based on holistic and multi-trait scales: A generalizability analysis

2018 ◽  
Vol 38 ◽  
pp. 21-36 ◽  
Author(s):  
Renka Ohta ◽  
Lia M. Plakans ◽  
Atta Gebril
2009 ◽  
Author(s):  
Walter P. Vispoel ◽  
Shuqin Tao ◽  
Linan Sun ◽  
Yi He ◽  
Paul A. Westrick

2020 ◽  
pp. 1-10
Author(s):  
Marilyn N. Ahun ◽  
Lamprini Psychogiou ◽  
Frédéric Guay ◽  
Michel Boivin ◽  
Richard E. Tremblay ◽  
...  

Abstract Background Maternal depressive symptoms (MDSs) are negatively associated with children's academic performance, with stronger effects sometimes reported in boys. However, few studies have tested the mechanisms of this association. We examined the mediating role of school engagement and peer victimization in this association and tested for sex differences. Methods Participants were 1173 families from a population-based longitudinal Canadian study. MDSs were self-reported annually using the Centre for Epidemiologic Studies Depression Scale (child's age: 5 months to 5 years). Data on mediators (peer victimization, cognitive, behavioral, and emotional school engagement) were reported annually from ages 6–10 by multiple informants including children, parents, and teachers using items from validated scales. Mathematics, reading, and writing exam scores at age 12 were obtained from standardized exams administered by Québec's Ministry of Education and Teaching. Structural equation modeling was used to test mediation by school experiences in boys and girls. Results Exposure to MDSs was negatively associated with mathematics, reading, and writing scores in girls and with mathematics only in boys. Cognitive and behavioral engagement significantly mediated the association between MDSs and mathematics, reading, and writing scores in girls. There were no significant mediators for boys. Conclusions Prevention and intervention strategies aiming to improve school engagement might be beneficial for daughters of mothers experiencing depressive symptoms. Further research is needed to replicate these findings and to identify the mechanisms explaining this association in boys.


SAGE Open ◽  
2021 ◽  
Vol 11 (2) ◽  
pp. 215824402110231
Author(s):  
Jui-Jung Tsao ◽  
Wen-Ta Tseng ◽  
Tsung-Yuan Hsiao ◽  
Chaochang Wang ◽  
Andy Xuesong Gao

Research has shown that the effectiveness of written corrective feedback (WCF) on writing performance depends on learners’ engagement with WCF and its associated motivational state. However, little research has examined the inner causal relationships between motivation, learner engagement with WCF, and writing performance. The current study fills the void in the existing literature by taking a structural equation modeling approach as the methodological framework. Two independent Chinese undergraduate samples partook in the pilot and formal phases of the study. The results showed that cultivation of an ideal self-image significantly promoted both intrinsic and extrinsic motivations and enhanced learners’ engagement with WCF, but ought-to self-image was found to have no such effects. Furthermore, both intrinsic motivation and learner engagement with WCF could directly influence writing scores, with the latter being more explanatory than the former. The implications of the research findings are provided and discussed.


2015 ◽  
Vol 27 (4) ◽  
pp. 395-403 ◽  
Author(s):  
Margaret W. Gerbase ◽  
Michèle Germond ◽  
Bernard Cerutti ◽  
Nu V. Vu ◽  
Anne Baroffio

2016 ◽  
Vol 125 (5) ◽  
pp. 1056-1065 ◽  
Author(s):  
Kiki M. J. M. H. Lombarts ◽  
Andrew Ferguson ◽  
Markus W. Hollmann ◽  
Bente Malling ◽  
Onyebuchi A. Arah ◽  
...  

Abstract Background Given the increasing international recognition of clinical teaching as a competency and regulation of residency training, evaluation of anesthesiology faculty teaching is needed. The System for Evaluating Teaching Qualities (SETQ) Smart questionnaires were developed for assessing teaching performance of faculty in residency training programs in different countries. This study investigated (1) the structure, (2) the psychometric qualities of the new tools, and (3) the number of residents’ evaluations needed per anesthesiology faculty to use the instruments reliably. Methods Two SETQ Smart questionnaires—for faculty self-evaluation and for resident evaluation of faculty—were developed. A multicenter survey was conducted among 399 anesthesiology faculty and 430 residents in six countries. Statistical analyses included exploratory factor analysis, reliability analysis using Cronbach α, item-total scale correlations, interscale correlations, comparison of composite scales to global ratings, and generalizability analysis to assess residents’ evaluations needed per faculty. Results In total, 240 residents completed 1,622 evaluations of 247 faculty. The SETQ Smart questionnaires revealed six teaching qualities consisting of 25 items. Cronbach α’s were very high (greater than 0.95) for the overall SETQ Smart questionnaires and high (greater than 0.80) for the separate teaching qualities. Interscale correlations were all within the acceptable range of moderate correlation. Overall, questionnaire and scale scores correlated moderately to highly with the global ratings. For reliable feedback to individual faculty, three to five resident evaluations are needed. Conclusions The first internationally piloted questionnaires for evaluating individual anesthesiology faculty teaching performance can be reliably, validly, and feasibly used for formative purposes in residency training.


2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Ali Khodi

AbstractThe present study attempted to to investigate  factors  which affect EFL writing scores through using generalizability theory (G-theory). To this purpose, one hundred and twenty students participated in one independent and one integrated writing tasks. Proceeding, their performances were scored by six raters: one self-rating,  three peers,-rating and two instructors-rating. The main purpose of the sudy was to determine the relative and absolute contributions of different facets such as student, rater, task, method of scoring, and background of education  to the validity of writing assessment scores. The results indicated three major sources of variance: (a) the student by task by method of scoring (nested in background of education) interaction (STM:B) with 31.8% contribution to the total variance, (b) the student by rater by task by method of scoring (nested in background of education) interaction (SRTM:B) with 26.5% of contribution to the total variance, and (c) the student by rater by method of scoring (nested in background of education) interaction (SRM:B) with 17.6% of the contribution. With regard to the G-coefficients in G-study (relative G-coefficient ≥ 0.86), it was also found that the result of the assessment was highly valid and reliable. The sources of error variance were detected as the student by rater (nested in background of education) (SR:B) and rater by background of education with 99.2% and 0.8% contribution to the error variance, respectively. Additionally, ten separate G-studies were conducted to investigate the contribution of different facets across rater, task, and methods of scoring as differentiation facet. These studies suggested that peer rating, analytical scoring method, and integrated writing tasks were the most reliable and generalizable designs of the writing assessments. Finally, five decision-making studies (D-studies) in optimization level were conducted and it was indicated that at least four raters (with G-coefficient = 0.80) are necessary for a valid and reliable assessment. Based on these results, to achieve the greatest gain in generalizability, teachers should have their students take two writing assessments and their performance should be rated on at least two scoring methods by at least four raters.


EDULANGUE ◽  
2019 ◽  
Vol 2 (1) ◽  
pp. 19-35
Author(s):  
Kasyfur Rahman ◽  
Imansyah Imansyah ◽  
I Made Permadi Utama

This study aimed to investigate the relative effect of Self-Regulated Strategy Development (SRSD) teaching approach on tertiary EFL students’ writing. During two months, sixty EFL University students enrolling in basic writing course took part in this project and were divided into experimental and control groups. Pre- and post-tests were carried out to garner the intended data pertinent to the students’ writing scores. A mixed-design ANOVA was carried out to analyze the changes of writing scores of the respective groups since the assumption of homogeneity of regression slopes for the planned Analysis of Covariance was not met. The statistical evidence showed that there was a significant difference between the two groups in terms of writing scores at the significant value sig .00 < .05. The data also suggested that students who were taught using SRSD relatively outperformed their counterparts in the control group. The empirical evidence demonstrates that the use of SRSD as an instructional approach to some extent positively affects EFL students’ writing skill.


E-Structural ◽  
2020 ◽  
Vol 3 (02) ◽  
pp. 82-96
Author(s):  
Qorinta Shinta

Abstract. The English Course in Public Health Faculty of Diponegoro University is expected to develop the reading skills of the students through various exercises. However, its main goal is to help students identify simple, complex and compound sentences and how to organize them into good paragraphs. As in the Faculty of Public Health, the English Course is only given once in the first Semester, the exercises in each of the lesson is very compact which consists of Reading Text, Comprehension Questions, Word Study, Grammar and Usage, and Writing, in short these are called integrated exercised. Some students did very well on the exercises while others did not. The purpose of the study is to examine the relationship between the students’ performance in integrated exercises and their academic achievements of Diponegoro University Students majoring in Public Health who took English Course. The academic achievements in this study refer to their Mid-Term test Scores. Method of the Study: in the study of Diponegoro University Freshmen majoring in Public Health, a descriptive qualitative method was used to determine whether or not there is a correlation between reading and writing exercises and their academic achievements. The instruments used are the scores of 3 taken from exercises in unit 1, 2 and 3 and 1 Mid-Term test. The population of this research is 73 Public Health students of Diponegoro University taking English 1 Course. The data analysis result shows that Sig score is 0,000 < 0,05 therefore Ho is declined and Ha is accepted, which means there is a correlation between reading and writing practices scores and mid test scores 0.403 which is positive. Therefore, it can be concluded that the higher their writing scores are, the higher mid test scores will be.Key words: correlation, English course, integrated exercises, mid-term testAbstrak.  Mata kuliah Bahasa Inggris di Fakultas Kesehatan Masyarakat diharapkan mampu mengembangkan ketrampilan membaca mahasiswa memalui berbagai Latihan soal-soal. Namun, tujuan utama dari mata kuliah ini adalah membantu para mahasiswa menindentifikasi kalimat – kalimat sederhana, kompleks dan majemuk serta bagaimana membentuk kalimat – kalimat tersebut dalam bentuk paragraph. Karena mata kuliah Bahasa Inggris hanya disajikan satu kali, maka latihan-latihan soal pada setiap unit dibuat sangat padat yang terdiri dari Teks Bacaan, Pertanyaan tentang bacaan, Kosakata, Grammar dan fungsinya, serta Latihan writing. Ada mahasiswa yang bisa mengerjakan Latihan soal dengan sangat baik tetapi ada juga yang kesulitan. Tujuan dari peneltian ini adalah untuk melihat korelasi antara Nilai dari Latihan soal reading dan writing serta prestasi akademik dari Mahasiwa baru Universitas Diponegoro jurusan Kesehatan Masyarakat. Prestasi akademik dalam hal ini adalah nilai Mid-term test mereka. Dalam penelitian pada mahasiswa baru Universitas Diponegoro jurusan Kesehatan Masyarakat, digunakan metode deskriptif kualitatif untuk menetukan apakah ada atau tidak ada korelasi antara Latihan soal reading dan writing dengan prestasi akademik mereka. Instrumen yang digunakan pada penelitian ini  adalah 3 nilai yang diambil dari 3 latihan soal Unit 1, 2 dan 3. Jumlah populasi adalah 73 Mahasiswa Kesehatan Masyarakat Universitas Diponegoro yang mengambil mata kuliah Bahasa Inggris 1. Hasil dari data analisis menunjukkan bahwa nilai Sig  0,000 < 0,05  sehingga Ho ditolak dan Ha diterima, yang berarti ada korelasi antara nilai Latihan writing dengan nilai ujian tengah semester 0.403 positif. Sehingga bisa disimpulkan bahwa semakin tinggi nilai mereka pada Latihan writing akan semakin tinggi juga nilai mereka dalam Ujian Tengah Semester.Kata kunci: korelasi, mata kuliah bahasa Inggris, nilai latihan soal terpadu, nilai Ujian Tengah Semester


Sign in / Sign up

Export Citation Format

Share Document