Classification Algorithm for the Determination of Suicide Attempt and Suicide (CAD-SAS)

Crisis ◽  
2012 ◽  
Vol 33 (3) ◽  
pp. 151-161 ◽  
Author(s):  
Izabela E. Fedyszyn ◽  
Meredith G. Harris ◽  
Jo Robinson ◽  
Susan J. Paxton
Keyword(s):  
Suicide Attempt ◽  
Interrater Reliability ◽  
Real Life ◽  
Classification Algorithm ◽  
Outcome Variable ◽  
Clinical Judgments ◽  
Retest Reliability ◽  
Test Retest Reliability

Background: One methodological difficulty in research into suicide attempts and suicide is distinguishing these phenomena from nonsuicidal self-harming behaviors and accidents. This is problematic because a reliable assessment of the presence or absence of the outcome variable is fundamental for the validity of the findings. Aims: To develop a standardized rating system, the Classification Algorithm for the Determination of Suicide Attempt and Suicide (CAD-SAS), and to investigate its psychometric properties. Methods: To examine the test-retest reliability, one investigator rated 217 narratives of real-life self-harming incidents at initial assessment and 4 weeks later. To establish the interrater reliability, three independent raters assessed a random sample of 70 narratives using the CAD-SAS. To examine the validity, one investigator using the CAD-SAS compared ratings to clinical judgments made by a consultant psychiatrist without the CAD-SAS on the same random set of 70 narratives. Results: Test-retest reliability was excellent (97.2% agreement) and interrater reliability was substantial (70.0% agreement, κ = 0.70). Agreement in the classification of incidents with the “real-world” clinical judgments supports the validity of the CAD-SAS (64.3% agreement, κ = 0.46). Conclusions: The reliability and validity of future studies can be enhanced through the standardized assessment and classification of incidents.

scholarly journals Reliability of Safe Maximum Lifting Determinations of a Functional Capacity Evaluation

2002 ◽  
Vol 82 (4) ◽  
pp. 364-371 ◽  
Author(s):  
Douglas P Gross ◽  
Michele C Battié
Keyword(s):  
Functional Capacity ◽  
Repeated Measures ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Functional Capacity Evaluation ◽  
Measurement Variability ◽  
Retest Reliability ◽  
Repeated Measures Design ◽  
Test Retest Reliability

Abstract Background and Purpose. Functional capacity evaluations (FCEs) are measurement tools used in predicting readiness to return to work following injury. The interrater and test-retest reliability of determinations of maximal safe lifting during kinesiophysical FCEs were examined in a sample of people who were off work and receiving workers' compensation. Subjects. Twenty-eight subjects with low back pain who had plateaued with treatment were enrolled. Five occupational therapists, trained and experienced in kinesiophysical methods, conducted testing. Methods. A repeated-measures design was used, with raters testing subjects simultaneously, yet independently. Subjects were rated on 2 occasions, separated by 2 to 4 days. Analyses included intraclass correlation coefficients (ICCs) and 95% confidence intervals. Results. The ICC values for interrater reliability ranged from .95 to .98. Test-retest values ranged from .78 to .94. Discussion and Conclusion. Inconsistencies in subjects' performance across sessions were the greatest source of FCE measurement variability. Overall, however, test-retest reliability was good and interrater reliability was excellent.

Psychometric Properties of Functional Balance Assessment in Children With Cerebral Palsy

2008 ◽  
Vol 22 (6) ◽  
pp. 745-753 ◽  
Author(s):  
Sue-Mae Gan ◽  
Li-Chen Tung ◽  
Yue-Her Tang ◽  
Chun-Hou Wang
Keyword(s):  
Cerebral Palsy ◽  
Motor Function ◽  
Convergent Validity ◽  
Interrater Reliability ◽  
Walking Speed ◽  
Retest Reliability ◽  
Functional Balance ◽  
Sit To Stand ◽  
Children With Cerebral Palsy ◽  
Test Retest Reliability

Background. Children with cerebral palsy often suffer from a lack of balance compared with typically developing children. Because balance capacity is relevant to functional activities, reliable and valid functional balance measures are crucial for the pediatric clinical setting. Objective. This study examined the reliability and validity of 3 functional balance measures. Methods. Thirty children aged 60 to 142 months with Gross Motor Function Classification System (GMFCS) levels of I to IV were recruited. For test-retest reliability, the same physical therapist administered the Functional Reach Test (FRT), Berg Balance Scale (BBS), and Timed Up and Go (TUG) twice. For interrater reliability, the testing processes were video recorded and later scored by another therapist. For convergent validity, children with cerebral palsy received the Gross Motor Function Measures (GMFM), walking speed, and 10-second sit-to-stand test within 1 week and the results evaluated. Results. The 3 functional balance measures had excellent test-retest reliability (intraclass correlation coefficient [ICC] >0.95) and interrater reliability (ICC = 0.98-1.00). With regard to convergent validity, the BBS and the TUG were highly correlated with GMFM total score, walking speed, and the 10-second sit-to-stand test. The discriminate validity indicates that the FRT can distinguish children with cerebral palsy with different GMFCS levels, whereas the BBS total score and TUG failed to distinguish between children with cerebral palsy with GMFCS levels of I and II. Conclusion. The 3 functional balance measures are simple, valid, and reliable for examining children with cerebral palsy and are thus suitable for clinical practice.

Reliability and Validity of the Hierarchic Dementia Scale

1994 ◽  
Vol 6 (1) ◽  
pp. 87-94 ◽  
Author(s):  
Lisa Rönnberg ◽  
Kjerstin Ericsson
Keyword(s):  
Cognitive Deficits ◽  
Concurrent Validity ◽  
Interrater Reliability ◽  
Mixed Type ◽  
Reliability And Validity ◽  
Kappa Coefficient ◽  
Severe Dementia ◽  
Valid Instrument ◽  
Test Retest Reliability

The aim of the study was to apply a method for measuring cognitive functioning in severely demented patients. Reliability and validity of the Hierarchic Dementia Scale (HDS) were tested. Fifty patients with dementia of the Alzheimer's type (DAT), multi-infarct dementia (MID), and dementia of mixed type (MIX) were studied. The interrater reliability was satisfactory as estimated by means of a kappa coefficient. The test retest reliability was rs = .96. The concurrent validity of the HDS, as measured by the correlation between the HDS and the MMSE, was rs = .86, and between the HDS and the CDR was rs = −.71. The results indicate that HDS is a useful and valid instrument for determination of the heterogeneous cognitive deficits in severe dementia.

Psychometric Properties of a Structured Diagnostic Interview for DSM-5 Anxiety, Mood, and Obsessive-Compulsive and Related Disorders

Assessment ◽  
2016 ◽  
Vol 25 (1) ◽  
pp. 3-13 ◽  
Author(s):  
David F. Tolin ◽  
Christina Gilliam ◽  
Bethany M. Wootton ◽  
William Bowe ◽  
Laura B. Bragdon ◽  
...  
Keyword(s):  
Convergent Validity ◽  
Interrater Reliability ◽  
Diagnostic Interview ◽  
Self Report ◽  
Obsessive Compulsive ◽  
Reliability Data ◽  
Retest Reliability ◽  
Dsm 5 ◽  
Structured Diagnostic Interview ◽  
Test Retest Reliability

Three hundred sixty-two adult patients were administered the Diagnostic Interview for Anxiety, Mood, and OCD and Related Neuropsychiatric Disorders (DIAMOND). Of these, 121 provided interrater reliability data, and 115 provided test–retest reliability data. Participants also completed a battery of self-report measures that assess symptoms of anxiety, mood, and obsessive-compulsive and related disorders. Interrater reliability of DIAMOND anxiety, mood, and obsessive-compulsive and related diagnoses ranged from very good to excellent. Test–retest reliability of DIAMOND diagnoses ranged from good to excellent. Convergent validity was established by significant between-group comparisons on applicable self-report measures for nearly all diagnoses. The results of the present study indicate that the DIAMOND is a promising semistructured diagnostic interview for DSM-5 disorders.

scholarly journals Test-Retest and Interrater Reliability of the Functional Movement Screen

2013 ◽  
Vol 48 (3) ◽  
pp. 331-336 ◽  
Author(s):  
Rebecca Shultz ◽  
Scott C. Anderson ◽  
Gordon O. Matheson ◽  
Brandon Marcello ◽  
Thor Besier
Keyword(s):  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Video Recording ◽  
Functional Movement Screen ◽  
Good Test ◽  
Good Reliability ◽  
Functional Movement ◽  
Retest Reliability ◽  
Test Retest Reliability ◽  
First Session

Context: The Functional Movement Screen (FMS) is a popular test to evaluate the degree of painful, dysfunctional, and asymmetric movement patterns. Despite great interest in the FMS, test-retest reliability data have not been published. Objective: To assess the test-retest and interrater reliability of the FMS and to compare the scoring by 1 rater during a live session and the same session on video. Design: Cross-sectional study. Setting: Human performance laboratory in the sports medicine center. Patients or Other Participants: A total of 21 female (age = 19.6 ± 1.5 years, height = 1.7 ± 0.1 m, mass = 64.4 ± 5.1 kg) and 18 male (age = 19.7 ± 1.0 years, height = 1.9 ± 0.1 m, mass = 80.1 ± 9.9 kg) National Collegiate Athletic Association Division IA varsity athletes volunteered. Intervention(s): Each athlete was tested and retested 1 week later by the same rater who also scored the athlete's first session from a video recording. Five other raters scored the video from the first session. Main Outcome Measure(s): The Krippendorff α (K α) was used to assess the interrater reliability, whereas intraclass correlation coefficients (ICCs) were used to assess the test-retest reliability and reliability of live-versus-video scoring. Results: Good reliability was found for the test-retest (ICC = 0.6), and excellent reliability was found for the live-versus-video sessions (ICC = 0.92). Poor reliability was found for the interrater reliability (K α = .38). Conclusions: The good test-retest and high live-versus-video session reliability show that the FMS is a usable tool within 1 rater. However, the low interrater K α values suggest that the FMS within the limits of generalization should not be used indiscriminately to detect deficiencies that place the athlete at greater risk for injury. The FMS interrater reliability may be improved with better training for the rater.

scholarly journals Virtual Exam for Parkinson's Disease Enables Frequent and Reliable Measurements of Motor Function

2021 ◽  
Author(s):  
Maximilien Burq ◽  
Erin Rainaldi ◽  
King Chung Ho ◽  
Chen Chen ◽  
Bastiaan R Bloem ◽  
...  
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Early Stage ◽  
Real Life ◽  
Dropout Rate ◽  
Therapeutic Interventions ◽  
Retest Reliability ◽  
Time Sensitivity ◽  
Remote Measurements ◽  
Test Retest Reliability

Sensor-based remote monitoring could help us better track Parkinson's disease (PD) progression, and measure patients' response to putative disease-modifying therapeutic interventions. To be useful, the remotely-collected measurements should be valid, reliable and sensitive to change, and people with PD must engage with the technology. We developed a smartwatch-based active assessment that enables unsupervised measurement of motor signs of PD. 388 study participants with early-stage PD (Personalized Parkinson Project, 64% men, average age 63 years) wore a smartwatch for a median of 390 days, allowing for continuous passive monitoring. Participants performed unsupervised motor tasks both in the clinic (once) and remotely (twice weekly for one year). Dropout rate was 2% at the end of follow-up. Median wear-time was 21.1 hours/day, and 59% of per-protocol remote assessments were completed. In-clinic performance of the virtual exam verified that most participants correctly followed watch-based instructions. Analytical validation was established for in-clinic measurements, which showed moderate-to-strong correlations with consensus MDS-UPDRS Part III ratings for rest tremor (R=0.70), bradykinesia (R=-0.62), and gait (R=-0.46). Test-retest reliability of remote measurements, aggregated monthly, was good-to-excellent (ICC: 0.75 - 0.96). Remote measurements were sensitive to the known effects of dopaminergic medication (on vs off Cohen's d: 0.19 - 0.54). Of note, in-clinic assessments often did not reflect the patients' typical status at home. This demonstrates the feasibility of using smartwatch-based unsupervised active tests, and establishes the analytical validity of associated digital measurements. Weekly measurements can create a more complete picture of patient functioning by providing a real-life distribution of disease severity, as it fluctuates over time. Sensitivity to medication-induced change, together with the improvement in test-retest reliability from temporal aggregation implies that these methods could help reduce sample sizes needed to demonstrate a response to therapeutic intervention or disease progression.

scholarly journals Psychometric properties of the Mental Health Recovery Star

2012 ◽  
Vol 201 (1) ◽  
pp. 65-70 ◽  
Author(s):  
Helen Killaspy ◽  
Sarah White ◽  
Tatiana L. Taylor ◽  
Michael King
Keyword(s):  
Mental Health ◽  
Psychometric Properties ◽  
Convergent Validity ◽  
Interrater Reliability ◽  
Pearson Correlation ◽  
Intraclass Correlation ◽  
Community Services ◽  
Mental Health Recovery ◽  
Retest Reliability ◽  
Test Retest Reliability

BackgroundThe Mental Health Recovery Star (MHRS) is a popular outcome measure rated collaboratively by staff and service users, but its psychometric properties are unknown.AimsTo assess the MHRS's acceptability, reliability and convergent validity.MethodA total of 172 services users and 120 staff from in-patient and community services participated. Interrater reliability of staff-only ratings and test–retest reliability of staff-only and collaborative ratings were assessed using intraclass correlation coefficients (ICCs). Convergent validity between MHRS ratings and standardised measures of social functioning and recovery was assessed using Pearson correlation. The influence of collaboration on ratings was assessed using descriptive statistics and ICCs.ResultsThe MHRS was relatively quick and easy to use and had good test–retest reliability, but interrater reliability was inadequate. Collaborative ratings were slightly higher than staff-only ratings. Convergent validity suggests it assesses social function more than recovery.ConclusionsThe MHRS cannot be recommended as a routine clinical outcome tool but may facilitate collaborative care planning.

scholarly journals Reliability of the Function in Sitting Test (FIST)

2014 ◽  
Vol 2014 ◽  
pp. 1-6 ◽  
Author(s):  
Sharon L. Gorman ◽  
Monica Rivera ◽  
Lise McCarthy
Keyword(s):  
Interrater Reliability ◽  
Physical Therapists ◽  
Physical Therapist ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Inpatient Rehabilitation ◽  
Neurological Dysfunction ◽  
Retest Reliability ◽  
Reliability Characteristics ◽  
Test Retest Reliability

The function in sitting test (FIST) is a newly developed, performance-based measure examining deficits in seated postural control. The FIST has been shown to be internally consistent and valid in persons with neurological dysfunction but intra- and interrater reliability and test-retest reliability have not been previously described. Seven patients with chronic neurologic dysfunction were tested and videotaped performing the FIST on two consecutive days. Seventeen acute care and inpatient rehabilitation physical therapist raters scored six of the videotaped performance of the FIST on two occasions at least 2 weeks apart. Intraclass correlation coefficients were used to calculate the test-retest and intra- and interrater reliability of the FIST. ICC of 0.97 (95% CI 0.847–0.995) indicated excellent test-retest reliability of the FIST. Intra- and interrater reliability was also excellent with ICCs of 0.99 (95% CI 0.994–0.997) and 0.99 (95% CI 0.988–0.994), respectively. Physical therapists and other rehabilitation professionals can confidently use the FIST in a variety of clinical practice and research settings due to its favorable reliability characteristics. More studies are needed to describe the responsiveness and minimal clinically important level of change in FIST scores to further enhance clinical usefulness of this measure.

The Scales for Assessing Emotional Disturbance—Third Edition: Internal Reliability, Interrater Reliability, and Test–Retest Reliability

2020 ◽  
pp. 019874292096915
Author(s):  
Jacqueline Huscroft-D’Angelo ◽  
Jessica Wery ◽  
Jodie Diane Martin ◽  
Corey Pierce ◽  
Lindy Crawford
Keyword(s):  
Emotional Disturbance ◽  
Interrater Reliability ◽  
Rating Scale ◽  
Composite Score ◽  
Future Research ◽  
Internal Reliability ◽  
Retest Reliability ◽  
Nationally Representative ◽  
Test Retest Reliability ◽  
Reliability Coefficients

The Scales for Assessing Emotional Disturbance—Third Edition Rating Scale (SAED-3 RS; Epstein et al.) is a standardized, norm-referenced measure designed to aid in the identification process by providing useful data to professionals determining eligibility of students with an emotional disturbance (ED). Three studies are reported to address the reliability of the SAED-3 RS. Study 1 investigated the internal reliability of the SAED-3 RS using data from a nationally representative sample of 1,430 students and 441 with ED. Study 2 examined interrater reliability between 123 pairs of educators who had worked with the student for at least 2 months. Study 3 assessed the test–retest reliability over a 2-week period to determine stability of the SAED-3 RS. Across all studies, scores collected from the SAED-3 RS were determined to be a reliable, stable for measuring the emotional and behavioral functioning of students. Specifically, the averaged coefficient alpha for internal consistency ranged from .79 to .92 for each subscale and .96 for the composite score; interrater reliability coefficients ranged from .77 to .89 for each subscale and .89 for the composite score, and test–retest reliability coefficients ranged from .79 to .92 for each subscale and .96 for the composite score. Limitations, future research and implications for use of the SAED-3 RS are discussed.

Sign in / Sign up

Export Citation Format

Share Document