Interrater Reliability of the New Sport-Specific Evidence-Based Classification System for Para Va'a

2020 ◽  
Vol 37 (3) ◽  
pp. 241-252
Author(s):  
Johanna S. Rosén ◽  
Victoria L. Goosey-Tolfrey ◽  
Keith Tolfrey ◽  
Anton Arndt ◽  
Anna Bjerkefors
Keyword(s):  
Classification System ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Test Battery ◽  
Evidence Based ◽  
Water Test ◽  
The Individual ◽  
New Evidence ◽  
Test Level ◽  
Total Agreement

The purpose of this study was to examine the interrater reliability of a new evidence-based classification system for Para Va'a. Twelve Para Va'a athletes were classified by three classifier teams each consisting of a medical and a technical classifier. Interrater reliability was assessed by calculating intraclass correlation for the overall class allocation and total scores of trunk, leg, and on-water test batteries and by calculating Fleiss’s kappa and percentage of total agreement in the individual tests of each test battery. All classifier teams agreed with the overall class allocation of all athletes, and all three test batteries exhibited excellent interrater reliability. At a test level, agreement between classifiers was almost perfect in 14 tests, substantial in four tests, moderate in four tests, and fair in one test. The results suggest that a Para Va'a athlete can expect to be allocated to the same class regardless of which classifier team conducts the classification.

scholarly journals Validation of an Evidence-Based Medicine Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET)

2013 ◽  
Vol 5 (2) ◽  
pp. 252-256 ◽  
Author(s):  
Hans B. Kersten ◽  
John G. Frohna ◽  
Erin L. Giudice
Keyword(s):  
Internal Consistency ◽  
Evidence Based Medicine ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Evidence Based ◽  
Evaluation Tool ◽  
Intraclass Correlation Coefficients ◽  
Excellent Internal Consistency ◽  
Based Medicine

Abstract Background Competence in evidence-based medicine (EBM) is an important clinical skill. Pediatrics residents are expected to acquire competence in EBM during their education, yet few validated tools exist to assess residents' EBM skills. Objective We sought to develop a reliable tool to evaluate residents' EBM skills in the critical appraisal of a research article, the development of a written EBM critically appraised topic (CAT) synopsis, and a presentation of the findings to colleagues. Methods Instrument development used a modified Delphi technique. We defined the skills to be assessed while reviewing (1) a written CAT synopsis and (2) a resident's EBM presentation. We defined skill levels for each item using the Dreyfus and Dreyfus model of skill development and created behavioral anchors using a frame-of-reference training technique to describe performance for each skill level. We evaluated the assessment instrument's psychometric properties, including internal consistency and interrater reliability. Results The EBM Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET) is composed of 14 items that assess residents' EBM and global presentation skills. Resident presentations (N  =  27) and the corresponding written CAT synopses were evaluated using the EBM C-PET. The EBM C-PET had excellent internal consistency (Cronbach α  =  0.94). Intraclass correlation coefficients were used to assess interrater reliability. Intraclass correlation coefficients for individual items ranged from 0.31 to 0.74; the average intraclass correlation coefficients for the 14 items was 0.67. Conclusions We identified essential components of an assessment tool for an EBM CAT synopsis and presentation with excellent internal consistency and a good level of interrater reliability across 3 different institutions. The EBM C-PET is a reliable tool to document resident competence in higher-level EBM skills.

Early intervention in psychosis: a new evidence based paradigm

2002 ◽  
Vol 11 (4) ◽  
pp. 237-247 ◽  
Author(s):  
Patrick D. McGorry ◽  
Eoin J. Killackey
Keyword(s):  
Mental Illness ◽  
Early Intervention ◽  
Evidence Based ◽  
Major Health ◽  
Indicated Prevention ◽  
The Individual ◽  
Early Intervention In Psychosis ◽  
New Evidence ◽  
Major Health Issue

SUMMARYObjective – Even in countries whose mental health services are comparatively well resourced, the care offered to those in the early stages of psychotic illnesses is not what it could be. Patients often have to progress to chronicity before receiving adequate interventions, by which stage there has been great potential for harm, not only through the psychosis, but also to the quality of life of the individual who has often missed or not completed adequately, several important developmental tasks. Further, evidence indicates that delay in treatment is positively associated with poorer outcome. This paper puts the case for early intervention in psychosis. Method – Based on the experience of the Early Psychosis Prevention and Intervention Centre in Melbourne, the paper reviews the evidence for and the criticisms of, early intervention. Using the concept of indicated prevention, it suggests ways in which clinicians can improve the interventions available to those experiencing the onset of psychosis and suggests that prepsychotic intervention may be possible. Results – Evidence discussed in this paper indicates that the development of mental illness is a major health issue in young people; that there is a positive correlation between duration of untreated psychosis and outcome; that it is possible to identify a proportion of those at high risk of developing mental illness; that through intervention it may be possible to reduce the transition rate to illness. Conclusion – Primary prevention is beyond the capacity of our present knowledge. Indicated prevention in the form of early intervention and optimal, sustained treatment is a paradigm for which there is increasing supportive evidence. It is a paradigm which is appealing to clinicians, patients, families and which has the potential to reduce the secondary impact of serious mental illness such as suicide, stigma, isolation and reduction in social status.

Change-of-Direction Ability of Para-Footballers With Cerebral Palsy Under a New Evidence-Based and Sport-Specific Classification System

2020 ◽  
pp. 1-6
Author(s):  
Iván Peña-González ◽  
Alba Roldan ◽  
Carlos Toledo ◽  
Tomás Urbán ◽  
Raúl Reina
Keyword(s):  
Cerebral Palsy ◽  
Classification System ◽  
Intraclass Correlation ◽  
Football Players ◽  
Validity And Reliability ◽  
New Classification ◽  
Change Of Direction ◽  
Speed Test ◽  
Highly Correlated ◽  
New Evidence

Purpose: This study aimed (1) to explore the validity and reliability of a new and specific change-of-direction (COD) test that requires dribbling skills to classify international footballers with cerebral palsy (CP) and compare it with another valid and reliable COD test without ball dribbling and (2) to probe whether both tests can discriminate between the new CP football classes (ie, FT1, FT2, and FT3) established worldwide in 2018. Methods: This study involved 180 international para-footballers with CP from 23 national teams at the 3 regional competitions held in 2018. They performed 2 COD tests, the modified agility test (no dribbling skills) and the dribbling speed test (DST). Results: Reliability was excellent for both the modified agility test (intraclass correlation coefficient [ICC]2,1 = .91, SEM = 5.75%) and the DST (ICC2,1 = .92, SEM = 4.66%). The modified agility test and DST results were highly to very highly correlated to one another for the whole group and considering the sport classes (r = .60–.80; P < .001). A 1-way analysis of variance showed significant differences between sport classes in both tests (P < .001). However, among classes, there were significant differences between FT1 and FT2 and FT3 (P < .01, effect size = large) and low to moderate effect sizes between FT2 and FT3 for either test. Conclusion: The DST appears to be valid and reliable to classify CP football players within the new classification system. Regression analysis revealed that 18.2% of the variance in the new sport classes could be explained by the 2 examined tests.

A Second Step in Development of a Checklist for Screening Risk for Violence in Acute Psychiatric Patients: Evaluation of Interrater Reliability of the Preliminary Scheme 33

2007 ◽  
Vol 101 (3_suppl) ◽  
pp. 1145-1161
Author(s):  
Stål Bjørkly ◽  
Tron A. Moger
Keyword(s):  
Interrater Reliability ◽  
Psychiatric Patients ◽  
Intraclass Correlation ◽  
Second Step ◽  
Psychiatric Admission ◽  
Clinical Vignettes ◽  
Management Scheme ◽  
Content Domain ◽  
The Individual ◽  
Further Development

The Acute Project is a research project conducted on acute psychiatric admission wards in Norway. The objective is to develop and validate a structured, easy-to-use screening checklist for assessment of risk for violence in patients both during their stay in the ward and after discharge. The Preliminary Scheme 33 is a 33-item screening checklist with content domain inspired by the Historical-Clinical-Risk Management Scheme (HCR–20), the Brøset Violence Checklist, and eight risk factors extracted from the literature on risk assessment. The Preliminary Scheme 33 was designed and tested in two steps by a research group which includes the authors. The common aim of both steps was to develop this into a time economical, reliable, and valid checklist. In the first step in 2006 the predictive validity of the individual items was tested. The present work presents results from the second step, a study conducted to assess the interrater reliability of the 33 items. Eight clinicians working in an acute psychiatric unit volunteered to be raters and were trained to score the 33 items on a three-point scale in relation to 15 clinical vignettes, which contained information from 15 acute psychiatric patients' files. Analysis showed high interrater reliability for the total score with an intraclass correlation coefficient (ICC) of .86 (95% CI: 0.74-0.94). However, a substantial proportion of the items had medium to low ICCs. Consequences of this finding for further development of these items into a brief screen are discussed.

scholarly journals Extended Version of a Test Battery for Visual Assessment of Postural Orientation Errors: Face Validity, Internal Consistency, and Reliability

2020 ◽  
Vol 100 (9) ◽  
pp. 1542-1556 ◽  
Author(s):  
Jenny Nae ◽  
Mark W Creaby ◽  
Eva Ageberg
Keyword(s):  
Internal Consistency ◽  
Interrater Reliability ◽  
Late Phase ◽  
Intraclass Correlation ◽  
Acl Injury ◽  
Visual Assessment ◽  
Test Battery ◽  
Face Validity ◽  
Extended Version ◽  
Postural Orientation

Abstract Objective Undesirable postural orientation may be a risk factor for a second anterior cruciate ligament (ACL) injury. The purpose of this study was to evaluate face validity, internal consistency, and interrater reliability of an extended version of a previous test battery for visual assessment of postural orientation errors (POEs) in patients during the late phase of rehabilitation following ACL reconstruction (ACLR) (ie, when they have initiated jumping exercises). Methods This study used a cross-sectional design. Fifty-three patients (45% women) in the late phase of ACLR rehabilitation performed 5 functional tasks of varying difficulty. POEs of the lower extremity and trunk were visually assessed from video and scored on a scale from 0 (good) to 2 (poor). Results The side-hop and 2 new POEs (femur medial to shank, femoral valgus) were added to the test battery after expert focus group discussions. Internal consistency was calculated for all tasks (α = .712–.823). Interrater reliability showed fair to substantial agreement for femur medial to shank and femoral valgus during all tasks (K = 0.31–0.815) and almost perfect agreement for side-hop (intraclass correlation coefficient = 0.88). Conclusions The good internal consistency and reliability after adding side-hop, femur medial to shank, and femoral valgus suggests that this test battery is a suitable tool to quantify postural orientation throughout ACLR rehabilitation. Impact This test battery for visual assessment of POEs was evaluated in a heterogeneous group of patients in different phases of ACLR battery and can be used in clinical practice to measure POEs in patients with ACLR, including in the late phase of rehabilitation to return to sport. This study encourages research on more demanding tasks and additional POEs to cover the entire rehabilitation period after ACL injury or reconstruction.

Validity and reliability of the Persian version of the Eating and Drinking Ability Classification System

2019 ◽  
Vol 26 (9) ◽  
pp. 1-11
Author(s):  
Azade Riyahi ◽  
Mehdi Rassafiani ◽  
Zahra Nobakht ◽  
Hosseinali Abdolrazaghi ◽  
Rahmatollah Moradzadeh
Keyword(s):  
Cerebral Palsy ◽  
Classification System ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Occupational Therapists ◽  
Retest Reliability ◽  
Persian Version ◽  
Children With Cerebral Palsy ◽  
Test Retest Reliability ◽  
Speech Therapists

Background/Aims The ability to eat and drink is the most important factor in health, survival and longevity. Children with cerebral palsy can suffer from severe dysphagia, which can lead to a high risk of aspiration and choking. As classifying eating and drinking ability in children with cerebral palsy is important in research and treatment, the aim of this study was to determine the cross-cultural validation and reliability of the Persian version of the Eating and Drinking Ability Classification System. Methods After translation procedures, the face validity, content validity, test–retest reliability and interrater reliability of the Eating and Drinking Ability Classification System were evaluated. In total, 130 parents of children with cerebral palsy and 34 therapists participated in the evaluation of the scale's reliability. The 73 boys and 57 girls with cerebral palsy were aged 3–20 years (mean age 4.4 years) and had various types of cerebral palsy. They were classified according to the Eating and Drinking Ability Classification System by both their parents and the therapists. Results The overall results indicated that the words and sentences used were simple, clear, understandable, relevant and necessary. The intraclass correlation coefficients for test–retest reliability for parents, occupational therapists and speech therapists were 0.98, 0.98 and 0.995, respectively. The interrater reliability between parents and occupational therapists was 0.96, between parents and speech therapists was 0.95, and between occupational therapists and speech therapists was 0.985. Conclusions The Persian version of the Eating and Drinking Ability Classification System seems to be valid and reliable. This system may be used to evaluate children with cerebral palsy.

scholarly journals Reflections on Offering a Therapeutic Creative Arts Intervention With Cult Survivors: A Collective Biography

2020 ◽  
Vol 1 (1) ◽  
pp. 47-60
Author(s):  
Ailsa Parsons ◽  
Maria Kefalogianni ◽  
Linda Dubrow-Marshall ◽  
Richard Turner ◽  
Hailee Ingleton ◽  
...  
Keyword(s):  
Research Ethics ◽  
Psychological Therapy ◽  
Evidence Based ◽  
Creative Arts ◽  
Research Project ◽  
Research And Practice ◽  
Collective Biography ◽  
The Individual ◽  
New Evidence ◽  
Insight Into

A new, evidence-based, multimodal, and creative psychological therapy, Arts for the Blues, was piloted with survivors of cultic abuse in a workshop within a conference setting. The five facilitators, who occupied diverse roles and perspectives within the workshop and research project, reflected on their experiences of introducing this novel intervention to the cult-survivor population. In this underreported territory of using structured, arts-based, psychological therapy with those who have survived cultic abuse, the authors used a process of collective biography to compile a firstperson, combined narrative based on those reflections. This approach allows for a visceral insight into the dynamics and obstacles encountered, and the countertransference responses of the facilitators. This reflexive process shined a light into aspects of research and practice that were not all visible to the individual researchers previously, with implications for research ethics, psychological therapy, and creative arts within the cult-survivor field.

Interobserver Reliability Using the Phonetic Level Evaluation With Severely and Profoundly Hearing-Impaired Children

1991 ◽  
Vol 34 (5) ◽  
pp. 989-999 ◽  
Author(s):  
Stephanie Shaw ◽  
Truman E. Coggins
Keyword(s):  
Interrater Reliability ◽  
Interobserver Reliability ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Hearing Impaired ◽  
Intraclass Correlation Coefficients ◽  
Assessment Measure ◽  
Impaired Children ◽  
Speech Assessment ◽  
Hearing Impaired Children

This study examines whether observers reliably categorize selected speech production behaviors in hearing-impaired children. A group of experienced speech-language pathologists was trained to score the elicited imitations of 5 profoundly and 5 severely hearing-impaired subjects using the Phonetic Level Evaluation (Ling, 1976). Interrater reliability was calculated using intraclass correlation coefficients. Overall, the magnitude of the coefficients was found to be considerably below what would be accepted in published behavioral research. Failure to obtain acceptably high levels of reliability suggests that the Phonetic Level Evaluation may not yet be an accurate and objective speech assessment measure for hearing-impaired children.

Sign in / Sign up

Export Citation Format

Share Document