Factor Analytic Study of Social Behavior in Children

1969 ◽  
Vol 25 (2) ◽  
pp. 399-406
Author(s):  
S. Thomas Friedman ◽  
Richard F. Purnell ◽  
Edward E. Gotts
Keyword(s):  
Interrater Reliability ◽  
Rating Scales ◽  
Rating Scale ◽  
Ego Strength ◽  
Personality Variables ◽  
Adult Participant ◽  
Factor Analytic Study ◽  
Living Together ◽  
Peer Orientation ◽  
Children And Young Adolescents

The purpose was to use adult participant observers to create a scale for assessing some salient personality variables of children and young adolescents living together in close quarters. The 91 children were summer campers of both sexes (8 to 15 yr.). Counselors of these children were the adult participant observers. At least two counselors rated each camper on a 49-item rating scale. Interrater reliability was determined and composite ratings of the campers were factor analyzed. Seven factors accounted for the behaviors on the rating scales. These factors were consistent with and comparable to the constructs that were introduced into the items on the rating scale, e.g., Peer Orientation, Ego Strength, Interaction Potential, Adult Orientation, Rebelliousness, and Rigidity.

scholarly journals Using Differential Item Functioning to Test for Interrater Reliability in Constructed Response Items

2020 ◽  
Vol 80 (4) ◽  
pp. 808-820
Author(s):  
Cindy M. Walker ◽  
Sakine Göçer Şahin
Keyword(s):  
Differential Item Functioning ◽  
Interrater Reliability ◽  
Rating Scales ◽  
Rating Scale ◽  
Intraclass Correlation ◽  
Kappa Statistic ◽  
Promising Alternative ◽  
Constructed Response ◽  
Polytomous Item ◽  
Item Functioning

The purpose of this study was to investigate a new way of evaluating interrater reliability that can allow one to determine if two raters differ with respect to their rating on a polytomous rating scale or constructed response item. Specifically, differential item functioning (DIF) analyses were used to assess interrater reliability and compared with traditional interrater reliability measures. Three different procedures that can be used as measures of interrater reliability were compared: (1) intraclass correlation coefficient (ICC), (2) Cohen’s kappa statistic, and (3) DIF statistic obtained from Poly-SIBTEST. The results of this investigation indicated that DIF procedures appear to be a promising alternative to assess the interrater reliability of constructed response items, or other polytomous types of items, such as rating scales. Furthermore, using DIF to assess interrater reliability does not require a fully crossed design and allows one to determine if a rater is either more severe, or more lenient, in their scoring of each individual polytomous item on a test or rating scale.

Speech-Language Pathologists' Ratings of Speech Accuracy in Children With Speech Sound Disorders

2021 ◽  
pp. 1-12
Author(s):  
Linye Jing ◽  
Maria I. Grigos
Keyword(s):  
Interrater Reliability ◽  
Rating Scales ◽  
Rating Scale ◽  
Speech Sound ◽  
Future Research ◽  
Training Procedure ◽  
Speech Sound Disorders ◽  
Speech Language Pathologists ◽  
Point Rating Scale ◽  
Whole Word

Purpose: Forming accurate and consistent speech judgments can be challenging when working with children with speech sound disorders who produce a large number and varied types of error patterns. Rating scales offer a systematic approach to assessing the whole word rather than individual sounds. Thus, these scales can be an efficient way for speech-language pathologists (SLPs) to monitor treatment progress. This study evaluated the interrater reliability of an existing 3-point rating scale using a large group of SLPs as raters. Method: Utilizing an online platform, 30 SLPs completed a brief training and then rated single words produced by children with typical speech patterns and children with speech sound disorders. Words were closely balanced across the three rating categories of the scale. The interrater reliability of the SLPs ratings to a consensus judgment was examined. Results: The majority of SLPs (87%) reached substantial interrater reliability to a consensus judgment using the 3-point rating scale. Correct productions had the highest interrater reliability. Productions with extensive errors had higher agreement than those with minor errors. Certain error types, such as vowel distortions, were especially challenging for SLPs to judge. Conclusions: This study demonstrated substantial interrater reliability to a consensus judgment among a large majority of 30 SLPs using a 3-point rating. The clinical implications of the findings are discussed along with proposed modifications to the training procedure to guide future research.

scholarly journals Validation of a Spanish Version of the Lille Apathy Rating Scale for Parkinson’s Disease

2014 ◽  
Vol 2014 ◽  
pp. 1-7 ◽  
Author(s):  
Rocio García-Ramos ◽  
Clara Villanueva Iza ◽  
María José Catalán ◽  
Abilio Reig-Ferrer ◽  
Jorge Matías-Guíu
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Interrater Reliability ◽  
Rating Scales ◽  
Rating Scale ◽  
Reliability And Validity ◽  
Spanish Version ◽  
Healthy Controls ◽  
Lars Score ◽  
Optimal Cutoff Point

Introduction. To date, no rating scales for detecting apathy in Parkinson’s disease (PD) patients have been validated in Spanish. For this reason, the aim of this study was to validate a Spanish version of Lille apathy rating scale (LARS) in a cohort of PD patients from Spain.Participants and Methods. 130 PD patients and 70 healthy controls were recruited to participate in the study. Apathy was measured using the Spanish version of LARS and the neuropsychiatric inventory (NPI). Reliability (internal consistency, test-retest, and interrater reliability) and validity (construct, content, and criterion validity) were measured.Results. Interrater reliability was 0.93. Cronbach’sαfor LARS was 0.81. The test-retest correlation coefficient was 0.97. The correlation between LARS and NPI scores was 0.61. The optimal cutoff point under the ROC curve was-14, whereas the value derived from healthy controls was-11. The prevalence of apathy in our population tested by LARS was 42%.Conclusions. The Spanish version of LARS is a reliable and useful tool for diagnosing apathy in PD patients. Total LARS score is influenced by the presence of depression and cognitive impairment. However, both disorders are independent identities with respect to apathy. The satisfactory reliability and validity of the scale make it an appropriate instrument for screening and diagnosing apathy in clinical practice or for research purposes.

The “appropriate” conditions for establishing interrater reliability of interview-based rating scales. A controversial issue

1994 ◽  
Vol 9 (3) ◽  
pp. 149-150
Author(s):  
A Hafkenscheid
Keyword(s):  
Interrater Reliability ◽  
Rating Scales ◽  
Rating Scale ◽  
Controversial Issue ◽  
Auditory Hallucinations ◽  
Brain Abnormalities ◽  
Psychopathological Symptoms ◽  
Schizophrenic Patients ◽  
Practical Implications ◽  
Structural Brain

For many of the conditions with which psychiatry is concerned, there is still no anatomical or physiological abnormality that is reliably found in those affected with the conditions and not in other people. Some evidence exists that, for instance, severe and chronical psychopathological symptoms such as persistent auditory hallucinations, may be linked to structural brain abnormalities in certain subgroups of schizophrenic patients. However, such associations are not well-established as yet and the research in this field does not have practical implications for psychiatric assessment so far.Because objective (physical) indices for measuring (the severity of) psychopathological states have hardly been available up to now, the rating scale is an essential instrument in clinical (psychopharmacological) research.

scholarly journals Preliminary Validity Evidence for a Milestones-Based Rating Scale for Chart-Stimulated Recall

2018 ◽  
Vol 10 (3) ◽  
pp. 269-275 ◽  
Author(s):  
Shalini T. Reddy ◽  
Ara Tekian ◽  
Steven J. Durning ◽  
Shanu Gupta ◽  
Justin Endo ◽  
...  
Keyword(s):  
Medical Education ◽  
Interrater Reliability ◽  
Rating Scales ◽  
Rating Scale ◽  
Intraclass Correlation ◽  
Substantial Improvement ◽  
Validity Evidence ◽  
Stimulated Recall ◽  
Rating Form ◽  
Group A

ABSTRACT Background  Minimally anchored Standard Rating Scales (SRSs), which are widely used in medical education, are hampered by suboptimal interrater reliability. Expert-derived frameworks, such as the Accreditation Council for Graduate Medical Education (ACGME) Milestones, may be helpful in defining level-specific anchors to use on rating scales. Objective  We examined validity evidence for a Milestones-Based Rating Scale (MBRS) for scoring chart-stimulated recall (CSR). Methods  Two 11-item scoring forms with either an MBRS or SRS were developed. Items and anchors for the MBRS were adapted from the ACGME Internal Medicine Milestones. Six CSR standardized videos were developed. Clinical faculty scored videos using either the MBRS or SRS and following a randomized crossover design. Reliability of the MBRS versus the SRS was compared using intraclass correlation. Results  Twenty-two faculty were recruited for instrument testing. Some participants did not complete scoring, leaving a response rate of 15 faculty (7 in the MBRS group and 8 in the SRS group). A total of 529 ratings (number of items × number of scores) using SRSs and 540 using MBRSs were available. Percent agreement was higher for MBRSs for only 2 of 11 items—use of consultants (92 versus 75, P = .019) and unique characteristics of patients (96 versus 79, P = .011)—and the overall score (89 versus 82, P < .001). Interrater agreement was 0.61 for MBRSs and 0.51 for SRSs. Conclusions  Adding milestones to our rating form resulted in significant, but not substantial, improvement in intraclass correlation coefficient. Improvement was inconsistent across items.

AAC Collaboration Using the Self-Anchored Rating Scales (SARS): An Aphasia Case Study

2012 ◽  
Vol 21 (4) ◽  
pp. 136-143
Author(s):  
Lynn E. Fox
Keyword(s):  
Augmentative And Alternative Communication ◽  
Rating Scales ◽  
Rating Scale ◽  
Therapeutic Process ◽  
Family Counseling ◽  
The Self ◽  
Alternative Communication ◽  
Her Family ◽  
Solution Focused

Abstract The self-anchored rating scale (SARS) is a technique that augments collaboration between Augmentative and Alternative Communication (AAC) interventionists, their clients, and their clients' support networks. SARS is a technique used in Solution-Focused Brief Therapy, a branch of systemic family counseling. It has been applied to treating speech and language disorders across the life span, and recent case studies show it has promise for promoting adoption and long-term use of high and low tech AAC. I will describe 2 key principles of solution-focused therapy and present 7 steps in the SARS process that illustrate how clinicians can use the SARS to involve a person with aphasia and his or her family in all aspects of the therapeutic process. I will use a case study to illustrate the SARS process and present outcomes for one individual living with aphasia.

Childbirth and Posttraumatic Stress Responses

2006 ◽  
Vol 22 (4) ◽  
pp. 259-267 ◽  
Author(s):  
Eelco Olde ◽  
Rolf J. Kleber ◽  
Onno van der Hart ◽  
Victor J.M. Pop
Keyword(s):  
Posttraumatic Stress Disorder ◽  
Posttraumatic Stress ◽  
Stress Responses ◽  
Rating Scales ◽  
Rating Scale ◽  
Stress Disorder ◽  
Traumatic Experience ◽  
Event Scale ◽  
Self Rating ◽  
The Impact

Childbirth has been identified as a possible traumatic experience, leading to traumatic stress responses and even to the development of posttraumatic stress disorder (PTSD). The current study investigated the psychometric properties of the Dutch version of the Impact of Event Scale-Revised (IES-R) in a group of women who recently gave birth (N = 435). In addition, a comparison was made between the original IES and the IES-R. The scale showed high internal consistency (α = 0.88). Using confirmatory factor analysis no support was found for a three-factor structure of an intrusion, an avoidance, and a hyperarousal factor. Goodness of fit was only reasonable, even after fitting one intrusion item on the hyperarousal scale. The IES-R correlated significantly with scores on depression and anxiety self-rating scales, as well as with scores on a self-rating scale of posttraumatic stress disorder. Although the IES-R can be used for studying posttraumatic stress reactions in women who recently gave birth, the original IES proved to be a better instrument compared to the IES-R. It is concluded that adding the hyperarousal scale to the IES-R did not make the scale stronger.

D-test: A New Test for Analyzing Scale Invariance Using Symbolic Dynamics and Symbolic Entropy

Methodology ◽  
2011 ◽  
Vol 7 (3) ◽  
pp. 88-95 ◽  
Author(s):  
Jose A. Martínez ◽  
Manuel Ruiz Marín
Keyword(s):  
Symbolic Dynamics ◽  
Scale Invariance ◽  
Rating Scales ◽  
Rating Scale ◽  
Marketing Research ◽  
Response Patterns ◽  
Measurement Scales ◽  
Improve Measurement ◽  
Entropy Measures ◽  
The Difference

The aim of this study is to improve measurement in marketing research by constructing a new, simple, nonparametric, consistent, and powerful test to study scale invariance. The test is called D-test. D-test is constructed using symbolic dynamics and symbolic entropy as a measure of the difference between the response patterns which comes from two measurement scales. We also give a standard asymptotic distribution of our statistic. Given that the test is based on entropy measures, it avoids smoothed nonparametric estimation. We applied D-test to a real marketing research to study if scale invariance holds when measuring service quality in a sports service. We considered a free-scale as a reference scale and then we compared it with three widely used rating scales: Likert-type scale from 1 to 5 and from 1 to 7, and semantic-differential scale from −3 to +3. Scale invariance holds for the two latter scales. This test overcomes the shortcomings of other procedures for analyzing scale invariance; and it provides researchers a tool to decide the appropriate rating scale to study specific marketing problems, and how the results of prior studies can be questioned.

Finding Talent Among Elementary English Learners: A Validity Study of the HOPE Teacher Rating Scale

2021 ◽  
pp. 001698622098594
Author(s):  
Nielsen Pereira
Keyword(s):  
English Language Learners ◽  
English Learners ◽  
English Language ◽  
Rating Scales ◽  
Rating Scale ◽  
Teacher Rating ◽  
Esl Teachers ◽  
Invariance Testing ◽  
Scale Scores ◽  
The One

The purpose of this study was to investigate the validity of the HOPE Scale for identifying gifted English language learners (ELs) and how classroom and English as a second language (ESL) teacher HOPE Scale scores differ. Seventy teachers completed the HOPE Scale on 1,467 students in grades K-5 and four ESL teachers completed the scale on 131 ELs. Measurement invariance tests indicated that the HOPE Scale yields noninvariant latent means across EL and English proficient (EP) samples. However, confirmatory factor analysis results support the use of the scale with ELs or EP students separately. Results also indicate that the rating patterns of classroom and ESL teachers were different and that the HOPE Scale does not yield valid data when used by ESL teachers. Caution is recommended when using the HOPE Scale and other teacher rating scales to compare ELs to EP students. The importance of invariance testing before using an instrument with a population that is different from the one(s) for which the instrument was developed is discussed.

Assessment of Agreement Between Human Ratings and Lexicon-Based Sentiment Ratings of Open-Ended Responses on a Behavioral Rating Scale

Assessment ◽  
2021 ◽  
pp. 107319112199646
Author(s):  
Olivia Gratz ◽  
Duncan Vos ◽  
Megan Burke ◽  
Neelkamal Soares
Keyword(s):  
Sentiment Analysis ◽  
Language Processing ◽  
Rating Scales ◽  
Clinical Decision Making ◽  
Rating Scale ◽  
Clinical Decision ◽  
Assessment System ◽  
Word Count ◽  
Positive Correlation ◽  
Sentiment Score

To date, there is a paucity of research conducting natural language processing (NLP) on the open-ended responses of behavior rating scales. Using three NLP lexicons for sentiment analysis of the open-ended responses of the Behavior Assessment System for Children-Third Edition, the researchers discovered a moderately positive correlation between the human composite rating and the sentiment score using each of the lexicons for strengths comments and a slightly positive correlation for the concerns comments made by guardians and teachers. In addition, the researchers found that as the word count increased for open-ended responses regarding the child’s strengths, there was a greater positive sentiment rating. Conversely, as word count increased for open-ended responses regarding child concerns, the human raters scored comments more negatively. The authors offer a proof-of-concept to use NLP-based sentiment analysis of open-ended comments to complement other data for clinical decision making.

Sign in / Sign up

Export Citation Format

Share Document