scholarly journals Measurement Quality and Rating Scale Functioning of a CSL Classroom Assessment Instrument

2019 ◽  
pp. 211-235
Author(s):  
Shuai Li ◽  
Yali Feng ◽  
Ting Wen
Keyword(s):  
Rating Scale ◽  
Classroom Assessment ◽  
Assessment Instrument ◽  
Measurement Quality

scholarly journals Heritability of an Age-Dependent Categorical Phenotype: Cognitive Dysfunction

2006 ◽  
Vol 9 (1) ◽  
pp. 17-23 ◽  
Author(s):  
Chandra A. Reynolds ◽  
Amy Fiske ◽  
Laura Fratiglioni ◽  
Nancy L. Pedersen ◽  
Margaret Gatz
Keyword(s):  
Cognitive Dysfunction ◽  
Ordinal Data ◽  
Rating Scale ◽  
Environmental Influences ◽  
Later Life ◽  
Relative Magnitude ◽  
Assessment Instrument ◽  
Best Fitting ◽  
Brief Cognitive Assessment ◽  
Cognitively Intact

AbstractWe investigated the extent to which cognitive dysfunction is shaped by genetic or environmental influences, and whether these factors differ in women and men. All members of the Swedish Twin Registry aged 65 and older were screened by telephone using the TELE, a brief cognitive assessment instrument (Gatz et al., 2002), and the Blessed Dementia Rating Scale (Blessed et al., 1968) from relatives of those who scored poorly on the TELE. Data were available for 4308 pairs where both members responded and 5070 pairs where only one member was alive and participated. To analyze all available data, we used a raw data method extended to ordinal data. As the prevalence of cognitive dysfunction increases with age, we incorporated age-adjusted thresholds. The best fitting model from biometric analyses indicated 35% of the variation in liability to cognitive dysfunction could be explained by heritable influences and the remaining 65% by nonfamilial environmental influences. Differences by gender were not significant. As this is a normative population including cognitively intact individuals, preclinical dementia cases and demented individuals, the relative magnitude of genetic and environmental effects is of particular interest in light of high heritabilities found for dementias such as Alzheimer's disease. The findings emphasize the extent to which research is needed to uncover nonfamilial environmental influences on cognitive dysfunction in later life.

The Effects of Enthusiasm Training on Selected Teacher and Student Behaviors in Preservice Physical Education Teachers

1984 ◽  
Vol 4 (1) ◽  
pp. 64-76 ◽  
Author(s):  
Cynthia Carlisle ◽  
D. Allen Phillips
Keyword(s):  
Preservice Teachers ◽  
Physical Education ◽  
Student Behavior ◽  
Teaching Effectiveness ◽  
Rating Scale ◽  
Assessment Instrument ◽  
Control Group ◽  
Student Behaviors ◽  
School Students ◽  
Trained Teachers

Teacher enthusiasm has long been considered an important part of the teaching process. However, empirical verification of enthusiasm as an indicator of teaching effectiveness is somewhat sparse. One problem is with measuring that complex variable, while another problem has been determining what to correlate it with to allow it to surface as such an indicator. Twenty-four preservice teachers participated in this study to determine the differences in teacher and student behavior between the levels of enthusiasm in trained and untrained teachers. The experimental group was given 6 hours of enthusiasm training whereas the control group received no such training. Both groups taught a 30-minute Experimental Teaching Unit (ETU) to a total of 120 middle-school students. The observation instrument in this study was the Physical Education Teaching Assessment Instrument (PETAI), while the Collins Enthusiasm Rating Scale was used to measure the teachers’ enthusiasm. The trained teachers received much higher ratings in enthusiasm during their ETU lessons and were significantly better on three of the PETAI items. The students of the trained teachers also had higher skill achievement gains over their counterparts under the untrained teachers.

Rating-Scale Labeling in Online Surveys: An Experimental Comparison of Verbal and Numeric Rating Scales with Respect to Measurement Quality and Respondents’ Cognitive Processes

2017 ◽  
Vol 49 (1) ◽  
pp. 79-107 ◽  
Author(s):  
Natalja Menold
Keyword(s):  
Cognitive Processes ◽  
Online Survey ◽  
Rating Scales ◽  
Rating Scale ◽  
Reliability And Validity ◽  
Experimental Comparison ◽  
Online Surveys ◽  
Cross Sectional ◽  
Measurement Quality ◽  
Numeric Rating

Unlike other data collection modes, the effect of labeling rating scales on reliability and validity, as relevant aspects of measurement quality, has seldom been addressed in online surveys. In this study, verbal and numeric rating scales were compared in split-ballot online survey experiments. In the first experiment, respondents’ cognitive processes were observed by means of eye tracking, that is, determining the respondent’s fixations in different areas of the screen. In the remaining experiments, data for reliability and validity analysis were collected from a German adult sample. The results show that respondents needed more fixations and more time to endorse a category when a rating scale had numeric labels. Cross-sectional reliability was lower and some hypotheses with respect to the criterion validity could not be supported when numeric rating scales were used. In conclusion, theoretical considerations and the empirical results contradict the current broad usage of numeric scales in online surveys.

scholarly journals Assessing risk of self-harm in acute paediatric settings: a multicentre exploratory evaluation of the CYP-MH SAPhE instrument

BMJ Open ◽  
2021 ◽  
Vol 11 (5) ◽  
pp. e043762
Author(s):  
Joseph C Manning ◽  
Tim Carter ◽  
Gemma Walker ◽  
Jane Coad ◽  
Aimee Aubeeluck
Keyword(s):  
Mental Health ◽  
Psychometric Properties ◽  
Internal Consistency ◽  
Rating Scale ◽  
Assessment Instrument ◽  
Face Validity ◽  
Secondary Outcome ◽  
Delphi Consensus ◽  
Healthcare Environments ◽  
Self Harm

ObjectiveTo psychometrically assess the Children and Young People-Mental Health Self-harm Assessment in Paediatric healthcare Environments (CYP-MH SAPhE) instrument for the identification of immediate risk of self-harm in CYP, aged 10–19 years, in acute paediatric wards or emergency departments.DesignThe CYP-MH SAPhE Instrument was developed through a robust scoping review and Delphi consensus with 30 clinicians/topic experts. To evaluate the psychometric properties, a multicentre exploratory study was conducted.SettingThree acute hospitals in the UK.Participants163 CYP presenting at acute hospital settings with primary mental health (cases) or physical health (non-cases) conditions.Primary and secondary outcome measuresPsychometric properties of the CYP-MH SAPhE instrument were evaluated through Principle Axis Factoring (PAF) with Oblimin (Kaiser normalisation) alongside measures of internal consistency (Cronbach’s α), convergent, discriminant and face validity.ResultsPAF of the dichotomous items (n=9) loaded onto three factors (1) behaviours and intentions; (2) suicidality and (3) self-harm. Factors 1 (Cronbach’s α=0.960) and 3 (Cronbach’s α=1) had high internal consistency. There was: good level of agreement between raters (kappa=0.65); a moderately positive correlation between the CYP-MH SAPhE instrument and the Columbia-Suicide Severity Rating Scale; and discrimination between cases and non-cases across the three factors (factor 1: m=88 vs 70; factor 2: m=102 vs 70; factor 3: m=104 vs 68). Assessment of face validity resulted in six items being removed, culminating in an eight question, rapid assessment instrument.ConclusionsThe results support the CYP-MH SAPhE Tool as a potentially reliable and valid instrument to identify immediate risk of self-harm in CYP presenting to acute paediatric healthcare environments, which is a burgeoning and significant global health issue.

scholarly journals Developing Needs Assessment Instrument On The Topic Of Classroom Guidance In Social Guidance And Counseling

2018 ◽  
Vol 4 (1) ◽  
pp. 47
Author(s):  
Nindya Ayu Pristanti ◽  
Muh Farozin
Keyword(s):  
Factor Analysis ◽  
Exploratory Factor Analysis ◽  
Content Validity ◽  
Rating Scale ◽  
Assessment Instrument ◽  
Counseling Services ◽  
Guidance And Counseling ◽  
Need Assessment ◽  
Classroom Guidance ◽  
Rating Scale Data

This study aimed to know the psychometry characteristic of need assessment instrument on the topic of classroom guidance in the social guidance and counseling services for students at SMP Negeri 1 Yogyakarta. This study was research and development followed Saifuddin Azwar model. Data collection method was a rating scale. Data analysis used content validity which was formulated by Aiken, construct validity used Exploratory Factor Analysis (EFA) and reliability analysis used Alpha Cronbach. The results of this development research were the need assessment instrument on the topic of classroom guidance in social guidance and counseling services for students at SMP N 1 Yogyakarta based on the content validity obtained 0,85, reliability 0,98 and found several new item groups from exploratory factor analysis result. It could be concluded that instrument of need assessment on the topic of social guidance and counseling was feasible to use

scholarly journals Persepsi Dampak Ujian dengan MCQ terhadap Proses Belajar Mahasiswa Fakultas Kedokteran

2014 ◽  
Vol 3 (3) ◽  
pp. 162
Author(s):  
Irwin Aras ◽  
Gandes Retno Rahayu ◽  
Yayi Suryo Prabandari
Keyword(s):  
Medical Students ◽  
Learning Outcomes ◽  
Rating Scale ◽  
Assessment Instrument ◽  
Free Response ◽  
Mind Mapping ◽  
Preliminary Study ◽  
Many Sources ◽  
Descriptive Survey ◽  
Content Information

Background: Multiple choice question (MCQ) has been used widely as a method to assess the achievement of learning outcomes. MCQ as an assessment instrument may give both expected and unexpected impact. The research’s objective is to identify the impacts of MCQs in term of the structure, content, information and regulation on the learning process of student at Medical Faculty Hasanuddin University Method: The study was a descriptive survey involving 505 medical students from Hasanuddin University who were still in the academic phase, class of 2010, 2011 and 2012. Preliminary study was carried out to explore learning impacts caused by MCQs. Based on the result of preliminary study a questionnaire using rating scale was developed, consisting 92 items of possible learning impacts. Open ended questions were added to get free response from the students. The answers were classified into expected and unexpected learning impacts.Results: The structure, content, information and regulation of the MCQ method gave expected impacts such as learning from many sources (74.4%), group studying (96.2%), mind mapping (37.9%) and re-discussing the exam materials (96.2%). It also gave unexpected impacts such as guessing the answer (44.8%), only studying previous exams (93.5%), cheating (33.7%) and taking pictures of the exam papers (38.2%). Conclusion: Unexpected impacts may occur from the MCQ method, which structurally consists of item flaws, such as only assessing memorizing skills rather than the application of knowledge and incomplete information in the stem. The regulation, in the form of summative exam, will encourage students to prepare themselves more seriously if compared to the formative exam.

scholarly journals Pengembangan perangkat pembelajaran dengan pendekatan pendidikan matematika realistik berorientasi pada minat dan prestasi

2018 ◽  
Vol 13 (1) ◽  
pp. 76-87
Author(s):  
Ivone Marlinda ◽  
Ariyadi Wijaya
Keyword(s):  
Mathematics Education ◽  
Junior High School ◽  
Rating Scale ◽  
Field Trials ◽  
Assessment Instrument ◽  
Realistic Mathematics Education ◽  
School Based ◽  
Realistic Mathematics ◽  
Learning Set ◽  
Category Score

Penelitian ini bertujuan untuk menghasilkan perangkat pembelajaran untuk siswa kelas VIII SMP dengan pendekatan pendidikan matematika realistik yang valid, praktis, dan efektif. Perangkat Pembelajaran ini terdiri atas RPP dan LKS yang berorientasi pada minat dan prestasi belajar siswa. Jenis penelitian ini adalah penelitian pengembangan (Research & Development). Penelitian ini mengembangkan perangkat pembelajaran dengan menggunakan model pengembangan 4D (Define, Design, Develop, Disseminate). Subjek uji coba penelitian adalah 21 siswa kelas uji coba. Instrumen  penelitian  yang digunakan terdiri dari lembar validasi, lembar penilaian kepraktisan siswa, lembar penilaian kepraktisan guru dan instrumen penilaian. Analisis data dilakukan dengan mengkonversi total skor data kuantitatif yang diperoleh menjadi data kualitatif skala lima. Kevalidan perangkat mencapai kriteria sangat baik. Kepraktisan pada uji coba lapangan mencapai kategori sangat baik untuk penilaian guru dan mencapai kategori baik untuk penilaian siswa. Keefektifan perangkat yang dikembangkan terlihat dari (1) persentase ketuntasan  tes prestasi belajar mencapai 80,95%; (2) berdasarkan hasil penilaian angket, persentase siswa yang memiliki minat minimal pada kategori tinggi mencapai 85,71%. Developing a learning set with realistic mathematics education approach oriented to the interest and achievementAbstractThis research is aimed at developing a learning set for the eight grade students of junior high school based on realistic mathematics education which is valid, practical, and effective. The instructional material consists of lesson plans and student’s worksheets, which are oriented to the interest and student’s learning achievements. This is a research and development study. It developed learning pack using the 4-D (Define, Design, Develop and Disseminate) development model.  The subjects of this research involved 21 students. The research instrument included validation sheets, student practicality assessment sheets and assessment instrument. The data analysis was done by converting the total score of the quantitative data into qualitative data in rating scale of five. The validity of learninng set get  very good criteria. The practicality on field trials get very good category from teacher assesment and get good category from student assesment. Effectiveness of learning set are developed by (1) the percentage of classical masttery of learning achievements test are 80.95%; (2) based on the result of the questioner the percentage of the student’s interest who have reached the high minimum category score is 85.71%.

51. Structured assessment format for evaluating operative reports (SAFE-OR) in general surgery

2007 ◽  
Vol 30 (4) ◽  
pp. 55
Author(s):  
A. Vergis ◽  
L. Gillman ◽  
M. Taylor ◽  
S. Minor ◽  
J. Park
Keyword(s):  
Construct Validity ◽  
Internal Consistency ◽  
General Surgery ◽  
Rating Scale ◽  
Assessment Instrument ◽  
Sigmoid Colectomy ◽  
Rater Reliability ◽  
Global Quality ◽  
Canadian Association ◽  
Assessment Format

This study determined the construct validity, inter-rater reliability and internal consistency of a “Structured Assessment Format for Evaluating Operative Reports” (SAFE-OR) in general surgery. The assessment instrument was developed using consensus criteria set forth by the Canadian Association of General Surgeons. It includes a structured assessment and a global quality rating scale. Residents divided into novice and experienced groups viewed and dictated a video-taped laparoscopic sigmoid colectomy. Transcriptions were then graded by blinded, independent faculty evaluators using SAFE-OR. Twenty-one residents participated in the study. Mean structured assessment scores (out of 44) were significantly lower for novice versus experienced residents (23.3 ± 5.2 vs 34.1 ± 6.0, t=0.001). Mean global quality scores (out of 45) were similarly lower for novice residents (25.6 ± 4.7 vs 35.9 ± 7.6, t=0.006). Inter-class correlation coefficients were 0.98 (95% CI 0.96-0.99) for structured assessment and 0.93 (95% CI 0.83-0.97) for global quality scales. Cronbach’s alpha coefficients for internal consistency were 0.85 for structured assessment and 0.96 for global quality assessment scales. SAFE-OR demonstrates significant construct validity, excellent inter-rater reliability and high internal consistency. This tool will allow educators to objectively evaluate the quality of trainee operative reports and ultimately provide a mechanism for implementing, monitoring, and refining curriculum for operative dictation communication skills. Moore R. The dictated operative note: important but is it being taught? Journal of the American College of Surgeons 2000; 190(5):639-40. Novitsky Y, Sing R, et al. Prospective, blinded evaluation of accuracy of operative reports dictated by surgical residents. The American Surgeon 2005; 71(8):627-31. Wanzel K, Ward R, et al. Teaching the surgical craft: From selection to certification. Current Problems in Surgery 2002; 39(6):573-659.

Structured evaluation of psychosocial factors

1991 ◽  
Vol 6 (6) ◽  
pp. 295-300
Author(s):  
P Kowal ◽  
JD Guelfi
Keyword(s):  
Social Integration ◽  
Social Functioning ◽  
Rating Scales ◽  
Rating Scale ◽  
Principal Component ◽  
The Elderly ◽  
Social Adaptation ◽  
Assessment Instrument ◽  
Structured Interviews ◽  
Self Rating

SummarySeveral reviews on social functioning measurements using structured interviews or questionnaires have been published in recent years. This report examines Weissman and Brugha’s report on these tools and the respective interest of self-rating scales or interviews. In the elderly, multidimensional tools such as MAI or CARE have proven to be very useful. Questionnaires may be the best tools for assessing social adaptation in the elderly: a short self-rating scale that aims to assess social adaptation in the elderly in the community has therefore been presented. The SII Q is derived from the Multilevel Assessment Instrument (MAI) and the Index of Social Support (ISS). A principal component analysis of this new Social Integration and Independence Questionnaire (17 items) elicited four factors accounting for 48.8% of the variance, representing four parameters of social functioning: independence, social integration, familial interaction and depression.

Sign in / Sign up

Export Citation Format

Share Document