scholarly journals Assessment of the Analytic Scale of Argumentative Writing (ASAW)

2021 ◽  
Vol 29 (S3) ◽  
Author(s):  
Vahid Nimehchisalem ◽  
Jayakaran Mukundan ◽  
Shameem Rafik-Galea ◽  
Arshad Abd Samad
Keyword(s):  
Concurrent Validity ◽  
Field Testing ◽  
University Student ◽  
Argumentative Writing ◽  
Rater Reliability ◽  
Student Writers ◽  
Argumentative Essays ◽  
Development And Validation ◽  
Specific Scale ◽  
Reliability Coefficients

The Analytic Scale of Argumentative Writing (ASAW) was developed because of the need for a genre-specific scale to assess English as a Second Language (ESL) university student writers’ argumentative essays. The present study reports the findings of field-testing ASAW. For this purpose, argumentative samples (n = 110) were collected and remote-scored by experienced raters (n = 5) who used ASAW. Overall, moderate to high inter-rater reliability (r = 0.7-0.9), as well as high (r = 0.84-0.92) and moderate to high (r = 0.70-0.77) intra-rater reliability coefficients after short (6-week) and long (9-week) rating intervals were obtained, respectively. Some established instruments were used to score the same essays rated using ASAW to test the concurrent validity of the scale. The scores assigned by the raters using the scale demonstrated moderate (r = 0.51) to high (r = 0.77) correlations with the scores awarded using several other standard instruments. The raters who used ASAW were given a questionnaire to evaluate the scale itself, and on average, the results indicated that the raters were highly satisfied with it. It took an average of 5.5 minutes for the raters to evaluate an essay, indicating it was economical. The study has useful implications for refinement of ASAW and development and validation of similar scales and benchmarks in the future.

Pre-university Students’ Voice Construction in Argumentative Essays

RELC Journal ◽  
2017 ◽  
Vol 50 (2) ◽  
pp. 269-284 ◽  
Author(s):  
Yin Ling Cheung ◽  
Tze Hui Low
Keyword(s):  
University Students ◽  
Systemic Functional Linguistics ◽  
Appraisal Theory ◽  
Argumentative Writing ◽  
Appraisal System ◽  
Writing Teachers ◽  
Student Writers ◽  
University Level ◽  
Argumentative Essays ◽  
Writer's Voice

This article examines how writer’s voice is constructed in argumentative essays written at the pre-university level. The study focusses on the student writers’ control over evaluative resources that influence the realization of voice in the high-scoring and low-scoring scripts. Using the APPRAISAL system in Systemic Functional Linguistics, the study shows how voice is construed through APPRAISAL theory in the high-scoring and low-scoring general paper essays, respectively. The differences between the two categories of essays can be seen in the application of ENGAGEMENT, ATTITUDE, and GRADUATION resources. Findings indicated that the high scoring essays used richer ENGAGEMENT and ATTITUDE resources to accomplish more mature and sophisticated argumentative voices. These opportunities to make full use of the APPRAISAL resources were missed by the low-scoring students in their argumentative writing. The findings are pedagogically useful for writing teachers who find the notion of voice too abstract to teach but accept its significance in producing a good essay.

Development of the Teacher Attitudes Towards Self-Regulated Learning Scale

2018 ◽  
Vol 34 (3) ◽  
pp. 193-205 ◽  
Author(s):  
Julia Steinbach ◽  
Heidrun Stoeger
Keyword(s):  
Factor Analysis ◽  
Primary School ◽  
Factor Structure ◽  
Concurrent Validity ◽  
Binary Logistic Regression ◽  
Affective Component ◽  
Self Regulated Learning ◽  
Regulated Learning ◽  
Teachers Attitudes ◽  
Development And Validation

Abstract. We describe the development and validation of an instrument for measuring the affective component of primary school teachers’ attitudes towards self-regulated learning. The questionnaire assesses the affective component towards those cognitive and metacognitive strategies that are especially effective in primary school. In a first study (n = 230), the factor structure was verified via an exploratory factor analysis. A confirmatory factor analysis with data from a second study (n = 400) indicated that the theoretical factor structure is appropriate. A comparison with four alternative models identified the theoretically derived factor structure as the most appropriate. Concurrent validity was demonstrated by correlations with a scale that measures the degree to which teachers create learning environments that enable students to self-regulate their learning. Retrospective validity was demonstrated by correlations with a scale that measures teachers’ experiences with self-regulated learning. In a third study (n = 47), the scale’s concurrent validity was tested with scales measuring teachers’ evaluation of the desirability of different aspects of self-regulated learning in class. Additionally, predictive validity was demonstrated via a binary logistic regression, with teachers attitudes as predictor on their registration for a workshop on self-regulated learning and their willingness to implement a seven-week training program on self-regulated learning.

Development and Validation of an Instrument to Assess Horseback Mobility in Hippotherapy

2021 ◽  
pp. 003151252110365
Author(s):  
Alessandra V. Prieto ◽  
Kênnea Martins Almeida Ayupe ◽  
Ana C. A. Abreu ◽  
Paulo J. B. Gutierres Filho
Keyword(s):  
Correlation Coefficient ◽  
Factor Structure ◽  
Internal Consistency ◽  
Rater Reliability ◽  
Intra Class Correlation ◽  
Exploratory Factor Analyses ◽  
Excellent Internal Consistency ◽  
Development And Validation ◽  
Functional Gain ◽  
Intra Class Correlation Coefficient

Improvement in rider mobility represents an important functional gain for people with disabilities undergoing hippotherapy. However, there is no validated measuring instrument to track and document the rider's progress in riding activities. In this study, we aimed to develop and establish validity evidence for an instrument to assess hippotherapy participants’ mobility on horseback. We report on this development through the stages of: (a) content validation, (b) construct validation, (c) inter- and intra-rater reliability and (d) internal consistency analysis. We evaluated its factor structure with exploratory factor analyses, calculated values for inter- and intra-rater reliability using the intra-class correlation coefficient, and calculated its internal consistency using Cronbach's alpha. We followed recommendations by the Guidelines for Reporting Reliability and Agreement Studies. We found good inter-rater reliability (intra-class correlation coefficient – ICC = 0.991–0.999) and good intra-rater reliability (ICC = 0.997–1.0), and there was excellent internal consistency (Cronbach's α = 0.937–0.999). The instrument’s factor structure grouped its three domains into one factor. As this instrument is theoretically consistent and has been found to be appropriate and reliable for its intended use, it is now available for the measurement of horseback mobility among hippotherapy riders.

scholarly journals Development and Validation of the Health-Friendly Activity Index: An Assessment Tool to Improve Consumers’ Health-Related Outcomes

2020 ◽  
Author(s):  
Young Ho Yun ◽  
Sinae Oh ◽  
Jin Ah Sim ◽  
Sujee Lee ◽  
Eun-Jung Sohn
Keyword(s):  
Assessment Tool ◽  
Activity Index ◽  
Field Testing ◽  
Cross Sectional Study ◽  
Cross Sectional ◽  
Item Questionnaire ◽  
Three Phase ◽  
Health Related ◽  
D 20 ◽  
Development And Validation

Abstract Objectives We developed the Health-Friendly Activity Index (HFAI) to measure the health-friendly activity of corporations or organizations comprehensively. We validated the developed tool and reported on its use as an assessment tool to improve consumers’ health-related outcomes.Study Design This is a cross-sectional studyPUBH-D-20-02610 Development of the HFAI questionnaire followed a three-phase process: item generation, item construction, and validation with field testing. Using relevance and feasibility criteria, we developed a 105-item questionnaire with six domains (Governance and Infrastructure, Needs Assessment, Planning, Implementation, Monitoring and Feedback, and Outcomes). To assess the sensitivity and validity of the questionnaire, we recruited two different groups. We assessed Group One (31 companies) based on their recent sustainability reports and compared their HFAI scores with the Contribution Assessment Tool for Consumer’s Health (CATCH) scores from 400 people from the general Korean population. For Group Two, we recruited 19 worksites and asked them to complete the HFAI and CATCH.Results Each domain of HFAI exhibited a Cronbach’s α coefficient between 0.382 and 1.000 for Group One and a Cronbach’s α coefficient between 0.676 and 0.938 for Group Two.

scholarly journals Adaptation and Psychometric Evaluation of the Chinese Counseling Competencies Scale-Revised

2021 ◽  
Vol 12 ◽  
Author(s):  
Wei Xia ◽  
William Ho Cheung Li ◽  
Tingna Liang ◽  
Yuanhui Luo ◽  
Laurie Long Kwan Ho ◽  
...  
Keyword(s):  
Concurrent Validity ◽  
Convergent Validity ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Psychometric Evaluation ◽  
Counseling Competencies ◽  
Counselors In Training ◽  
Rater Reliability ◽  
Retest Reliability ◽  
Test Retest Reliability

Objectives: This study conducted a linguistic and psychometric evaluation of the Chinese Counseling Competencies Scale-Revised (CCS-R).Methods: The Chinese CCS-R was created from the original English version using a standard forward-backward translation process. The psychometric properties of the Chinese CCS-R were examined in a cohort of 208 counselors-in-training by two independent raters. Fifty-three counselors-in-training were asked to undergo another counseling performance evaluation for the test-retest. The confirmatory factor analysis (CFA) was conducted for the Chinese CCS-R, followed by internal consistency, test-retest reliability, inter-rater reliability, convergent validity, and concurrent validity.Results: The results of the CFA supported the factorial validity of the Chinese CCS-R, with adequate construct replicability. The scale had a McDonald's omega of 0.876, and intraclass correlation coefficients of 0.63 and 0.90 for test-retest reliability and inter-rater reliability, respectively. Significantly positive correlations were observed between the Chinese CCS-R score and scores of performance checklist (Pearson's γ = 0.781), indicating a large convergent validity, and knowledge on drug abuse (Pearson's γ = 0.833), indicating a moderate concurrent validity.Conclusion: The results support that the Chinese CCS-R is a valid and reliable measure of the counseling competencies.Practice implication: The CCS-R provides trainers with a reliable tool to evaluate counseling students' competencies and to facilitate discussions with trainees about their areas for growth.

scholarly journals Inter-rater reliability in Learner Corpus Research

2020 ◽  
Vol 6 (2) ◽  
pp. 237-251
Author(s):  
Tove Larsson ◽  
Magali Paquot ◽  
Luke Plonsky
Keyword(s):  
Internal Consistency ◽  
Collaborative Research ◽  
Instrument Design ◽  
Common Source ◽  
Study Quality ◽  
Linguistic Features ◽  
Rater Reliability ◽  
Reporting Practices ◽  
Learner Corpus ◽  
Reliability Coefficients

Abstract In Learner Corpus Research (LCR), a common source of errors stems from manual coding and annotation of linguistic features. To estimate the amount of error present in a coded dataset, coefficients of inter-rater reliability are used. However, despite the importance of reliability and internal consistency for validity and, by extension, study quality, interpretability and generalizability, it is surprisingly uncommon for studies in the field of LCR to report on such reliability coefficients. In this Methods Report, we use a recent collaborative research project to illustrate the pertinence of considering inter-rater reliability. In doing so, we hope to initiate methodological discussion on instrument design, piloting and evaluation. We also suggest some ways forward to encourage increased transparency in reporting practices.

scholarly journals Analysis of Metadiscourse Features in Argumentative Writing by Pakistani Undergraduate Students

2017 ◽  
Vol 7 (6) ◽  
pp. 78
Author(s):  
Rashid Mahmood ◽  
Ghadia Javaid ◽  
Asim Mahmood
Keyword(s):  
Undergraduate Students ◽  
Corpus Analysis ◽  
Considerable Importance ◽  
Argumentative Writing ◽  
Argumentative Essays ◽  
Interpersonal Model

Metadiscourse involves the interaction between the reader and the writer of the text in the overall process of communication. Metadiscourse not only guides the reader to understand the primary message of the text through structure and content, but also it intimates the reader with the particular slants and perspectives in the primary discourse. The students have to master the use of Metadiscourse in their writings. The purpose of this study is to examine the distribution and frequency of Metadiscourse features used by Pakistani undergraduate students in their argumentative essays and to analyze roles played by these particular features. Moreover, the research explores the extent of appropriateness and inappropriateness in this particular text as well. Hyland’s Interpersonal model of Metadiscourse (2005) was adapted to conduct the present study. AntConc 3.4.4 software is used for corpus analysis of the text. Findings show that Pakistani undergraduate students are more comfortable with using Interactional Metadiscourse 61% rather than Interactive dimension 39%. It has been observed that undergraduate students used high score of self-mentions 37% and engagement markers 37%. Endophoric markers were not used by these students 0%. Findings have considerable importance, as they assist the learners to figure out the problems of the students regarding the use of Metadiscourse. Trainings should be given to the students to use these features appropriately.

scholarly journals Psychometric properties of trunk impairment scale in children with spastic diplegia

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Vedasri Dasoju ◽  
Rakesh Krishna Kovela ◽  
Jaya Shanker Tedla ◽  
Devika Rani Sangadala ◽  
Ravi Shankar Reddy
Keyword(s):  
Cerebral Palsy ◽  
Motor Function ◽  
Concurrent Validity ◽  
Reliability And Validity ◽  
Spastic Diplegia ◽  
Gross Motor ◽  
Rater Reliability ◽  
Gross Motor Function ◽  
Gross Motor Function Measure ◽  
Function Measure

AbstractThe Trunk Impairment Scale (TIS) is a valid and reliable tool to assess trunk impairment in children with heterogeneous cerebral palsy. The purpose of this study was to determine the reliability and validity of the TIS in assessing children with spastic diplegic cerebral palsy. The sample was a total of 30 subjects (15 = boys, 15 = girls). All subjects underwent an assessment of the sitting component of the Gross Motor Function Measure-88 and TIS by rater 1. Rater one observed video recordings within 24 h and scored TIS for intra-rater reliability, while rater two did likewise after 48 h for inter-rater reliability. The mean and standard deviation of the TIS and sitting components of the Gross Motor Function Measure-88 were 15.66 ± 4.20 and 52.36 ± 6.26, respectively. We established intra-rater and inter-rater reliability of the TIS with Intra Class Correlation Coefficient 0.991 and 0.972, respectively. The concurrent validity of the TIS with the sitting component of the Gross Motor Function Measure-88 was good, with an r-value of 0.844 (p < 0.001). This study showed the excellent intra-rater and inter-rater reliability and high concurrent validity of the TIS in assessing children with spastic diplegic cerebral palsy.

Sign in / Sign up

Export Citation Format

Share Document