Establishing an Operational Model of Rating Scale Construction for English Writing Assessment

Rating scales for writing assessment are critical in that they determine directly the quality and fairness of such performance tests. However, in many EFL contexts, rating scales are made, to certain extent, based on the intuition of teachers who strongly need a feasible and scientific route to guide their construction of rating scales. This study aims to design an operational model of rating scale construction with English summary writing as an example. Altogether 325 university English teachers, 4 experts in language assessment and 60 English majors in China participated in the study. 20 textual attributes were extracted, through text analysis, from China’s Standards of English Language Ability (CSE), theoretical construct of summary writing, comments on sample summary writing essays from 8 English teachers and their personal judgement. The textual attributes were then investigated through a large-scale questionnaire survey. Exploratory factor analysis and expert judgement were employed to determine rating scale dimensions. Regression analysis and expert judgement were conducted to determine the weighting distribution across all dimensions. Based on such endeavors, a tentative operational model of rating scale construction was established, which can also be applied and adapted to develop rating scales in other writing assessment. 

Examining EFL learners’ source text use in summary writing

To avoid plagiarism, students have to learn the appropriate and effective ways of source text use, such as paraphrasing, summarizing, and citation. However, few studies have investigated how learners of English as a foreign language (EFL) employ copy and paraphrase as source text use strategies in completing writing tasks involving reading material, and how characteristics of the reading material, such as genre, affect such use. This article reports an empirical study that attempted to address the issue. Eighty students from an undergraduate program of a Chinese university wrote summaries for a narrative text and an expository text. Their drafts and the source texts are compared to identify instances of retention of strings of words from sources. Analysis of the summary scripts showed that participants heavily relied on the source texts when writing summaries. The expository text elicited a larger proportion of words than the narrative text in total text borrowed, and the former also led the participants to stronger reliance on its surface structure than the latter. The less proficient group used Exact Copy more but Paraphrase less than the more proficient group. Higher level of source use – Major Paraphrase and Maximal Paraphrase – seems to be immune to changes in terms of genre of source text and language proficiency. Implications of these findings are discussed with reference to the teaching, learning, and assessing appropriate source text use of university level EFL learners.

EFFECTS OF COMPLEMENTARY APPROACH ON ARABIC WRITING SKILLS FOR TRAINEE TEACHERS

The purpose of this article is to report a quasi-experimental research on Arabic writing skills AWS for the trainee teachers. The effect of complementary approach CA on achievements of AWS for grade one Arabic as a Second Language students was investigated. The null hypothesis (Ho); there is no signicant difference at 0.05 between students' achievements in the experimental cluster and students' achievements in the control cluster when CA is used to teach AWS was tested. The Randomized Solomon Four Group Design n=100 was used. Different statistical measures were applied to analyze the data including averages, standard deviations, degrees of freedom and independent t-tests. The ndings yielded the rejection of the null hypothesis (Ho) of no signicant difference between clusters' means when CA is used to teach AWS for the sample in question at: 1-2 =19.04, t=5.223, df (48)=.000, α2, p <.05. The alternative hypothesis (Ha) was accepted due to the following data abstractions: integrated reading and orthography: 1-2 =11.12, t=3.955, df (48)=.000, α2, p <.05; integrated reading and essay writing: 1-2 =31.24, t=5.630, df (48)=.000, α2, p <.05; integrated listening and summary writing: 1-2 =-6.461, t=-1.246, df (48)= .220, α2, p >.05 and integrated speaking and report writing: 1-2 =40.44, t=7.823, df (48)=.000, α2, p <.05. Other than the CA strategy of integrated listening and summary writing which yielded insignicant effects, the rest of the selected CA strategies positively inuenced AWS variables in question. These ndings can be interpreted in a number of implications, including building integrative AWS curricula for the purpose of Arabic writing benchmarking and standardization. They are also applicable in terms of predicting and controlling of relevant Arabic linguistic variables for different research and practical purposes. The signicance of these ndings is also feasible with the contemporary scientic call for the diversication of communication andragogy.

Development and validation of a rating scale for summarization as an integrated task

Summary writing is essential for academic success, and has attracted renewed interest in academic research and large-scale language test. However, less attention has been paid to the development and evaluation of the scoring scales of summary writing. This study reports on the validation of a summary rubric that represented an approach to scale development with limited resources out of consideration for practicality. Participants were 83 students and three raters. Diagnostic evaluation of the scale components and categories was based on raters’ perception of their use and the scores of students’ summaries which were analyzed using multifaceted Rasch measurement (MFRM). Correlation analysis revealed significant relationships among the scoring components, but the coefficients among some of the components were over high. MFRM analysis provided evidence in support of the usefulness of the scoring rubric, but also suggested the need of a refinement of the components and categories. According to the raters, the rubric was ambiguous in addressing some crucial text features. This study has implications for summarization task design, scoring scale development and validation in particular.

Concept Mapping and Summary Writing as Complementary Strategies for Developing EFL Content Comprehension

This is an exploratory study in an undergraduate EFL business-writing course studying participants' ability to read, comprehend, and represent text visually using concept mapping (CM), summary writing, and social network analysis techniques as complementary strategies. The idea with this experiment was to explore if students are capable of analyzing business and technology information from technical readings in a way to represent it graphically with CMs and social networks. Preliminary data from the case study showed that students were reasonably successful in processing texts on topics related to the Tesla electric car company's business and technology models. Multiple iterations and guided instructions when designing CMs demonstrated the interplay of various actors, processes, interactions, and contexts. Student performance indicated significant expertise with CM design and text summarization but inadequate performance designing social networks, indicating the necessity for more structured instructions and practice.

What instructions are available to health researchers for writing lay summaries? A scoping review.

Objective To better understand the characteristics of, and requirements for, lay summaries by reviewing journals, global health organisations, professional medical associations and multi-disciplinary organisations, consumer advocacy groups and funding bodies. Design Using a scoping review methodology, we searched the websites of each identified data source to determine if they require, suggest, or refer to lay summaries. Two reviewers extracted lay summary writing instructions from eligible data sources from Australia, USA, UK, Canada and New Zealand. Data sources were linked to the top 10 non-communicable diseases. Main Outcome Measures Using an inductive approach, we identified characteristics of lay summaries and lay summary writing instructions and extracted data on these characteristics. These characteristics are lay summary formats, audience, requirements, authorship and labels, and elements of lay summary writing instructions (e.g. word count/length). We also noted who was expected to write the lay summaries, whether they were mandatory or optional, and the terms used for to denote them. Results The websites of 526 data sources were searched. Of these, 124 published or mentioned lay summaries and 108 provided writing instructions. For lay summaries, most were in journals, written by the author of the published paper, and only half were mandatory. Thirty-three distinct labels for a lay summary were identified, the most common being graphical abstract, highlights and key points. From the lay summary writing instructions, the most common elements for written lay summaries referred to: structure (86%), content (80%) and word count/length (74%). The least common elements were readability (3%), use of jargon, acronyms and abbreviations (24%), and wording (29%). The target audience was unclear in 68 of 108 (63.0%) of lay summary instructions. Discussion Although we identified over 100 sources provided instructions for writing lay summaries, very few provided instructions related to readability, use of jargon, acronyms and abbreviations, and wording. Some instructions provided structured formats via subheadings or questions to guide content, but not all. Only half mandated the use of lay summaries. Conclusion For lay summaries to be effective, writing instructions should consider the intended audience, ideally incorporating consumer input into their development. Presently, lay summaries are likely to be inaccessible to many consumers, written at a high reading level, with jargon, acronyms and abbreviations. Ideally, all research articles will have an accompanying lay summary. Mandatory lay summaries, however, are of limited value without clear and thorough instructions to guide authors. Public and patient involvement statement Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research study. Protocol and registration We conducted a scoping review using methods outlined in the PRISMA extension for scoping reviews and information in the Joanna Briggs Institute Reviewers Manual for scoping reviews. A protocol for this study was completed prior to data analysis and is on Open Science Framework.

Problems Faced By Students in Writing English Academic Summary

Purpose: This research investigates the problems faced by college students in writing English academic summary. Methodology/Approach/Design: The method used includes surface strategy taxonomy by Heidi Dulay and non-linguistic problem using personal interview. The data were collected from 35 original summary writing documents for the linguistic sources and Zoom meeting video recorder for the non-linguistic sources. The samples were 7 students in 5th semester of English Education Department, Islamic State Institute of Kediri. To gather data, the researchers employed a random sampling technique. The data analysis used descriptive qualitative methods. Results: The result of the data analysis shows that the students committed four types of surface strategy taxonomy: omission, addition, misformation, and miss ordering. Misformation is the type of common problem found in the students’ summary writing. In the personal interview, it was found that misunderstands about advanced-vocabulary is the common non-linguistic problem. These represent the problems faced by students. Practical Implications: This research study has implications for teachers and students. The results can be of use for improving students’ writing and designing alternative teaching skills for this important skill. Originality/Value: Summary writing process helps the students to expand their view about proper academic writing. This situation suggests that summary writing is a good strategy technique to be applied in college students. However, the ability in writing an academic summary using surface strategy taxonomy needs to be improved