Telemedicine in General Neurology: Interrater Reliability of Clinical Neurological Examination Via Audio-Visual Telemedicine

2018 ◽  
Vol 80 (5-6) ◽  
pp. 289-294 ◽  
Author(s):  
Mohamed Awadallah ◽  
Frank Janssen ◽  
Barbara Körber ◽  
Lorenz Breuer ◽  
Mateusz Scibor ◽  
...  
Keyword(s):  
Emergency Room ◽  
Neurological Examination ◽  
Interrater Reliability ◽  
Muscle Tone ◽  
Reliability And Validity ◽  
Observer Agreement ◽  
Kappa Statistics ◽  
Clinical Neurological Examination ◽  
Academic Teaching Hospital ◽  
Acute Patients

Introduction: While there are several studies on reliability of telemedicine in assessing stroke scales, little is known about the validity of a general neurological examination performed via telemedicine. Therefore, we sought to test the agreement between bedside and remote examination in acute patients of the emergency room. Methods: Acute patients at the emergency room of a 450-bed academic teaching hospital were included in this study. A clinical neurological examination consisting of 22 items was performed at bedside and also remotely via an audio-visual link by a different neurologist; both were experienced clinicians at the consultant level. Kappa statistics were calculated for each item of the examination. Results: Forty three patients completed both examinations (mean age 58.3 years, 56% female). Patients were seen between 8 and 72 min after admission (mean 36.3 min). Total time for remote examination was 12.6 min (8–21 min) and 8.9 min (5–18 min) for bedside examination. K-coefficients ranged from 0.32 (muscle tone) – 0.82 (language) indicating a fair to excellent agreement in most items. Conclusions: Remote examination via an audio-visual link produces comparable results to bedside performance even in acute patients of the emergency room. Compared to the scarce data available, inter-observer agreement is about the same as that between 2 examiners at bedside. However, more studies on reliability and validity of clinical neurological examination are required.

scholarly journals Comparison of the validity and reliability of two image classification systems for the assessment of mammogram quality

2005 ◽  
Vol 12 (1) ◽  
pp. 38-42 ◽  
Author(s):  
Conrad Moreira ◽  
Kate Svoboda ◽  
Ann Poulos ◽  
Richard Taylor ◽  
Andrew Page ◽  
...  
Keyword(s):  
Expert Panel ◽  
Reliability And Validity ◽  
Population Based ◽  
Classification Systems ◽  
Observer Agreement ◽  
Kappa Statistics ◽  
Reference Standard ◽  
Validity And Reliability ◽  
Suitable Alternative ◽  
South Wales

Objective: To compare the reliability and validity of two classification systems used to evaluate the quality of mammograms: PGMI ('perfect', 'good', 'moderate' and 'inadequate') and EAR ('excellent', 'acceptable' and 'repeat'). Setting: New South Wales (Australia)population-based mammography screening programme (BreastScreen NSW). Methods: Thirty sets of mammograms were rated by 21 radiographers and an expert panel. PGMI and EAR criteria were used to assign ratings to the medio-lateral oblique (MLO) and cranio-caudal (CC) views for each setof films. Inter-observer reliability and criterion validity (compared with expert panel ratings) were assessed using mean weighted observed agreement and kappa statistics. Results: Reliability : Kappa values for both classification systems were low (0.01–0.17). PGMI producedsignificantly higher values than EAR. Agreement between raters was higher using PGMI than EAR for the MLO view (77% versus 74%, P<0.05), but was similar for the CC view. Dichotomized ratings ('acceptable' or 'needs repeating') did not improve reliability estimates. Validity : Kappavalues between raters and the reference standard were low for both classification systems (0.05–0.15). Agreement between raters and the reference standard was higher using PGMI than EAR for the MLO view (74% versus 63%), but was similar for the CC view. Dichotomized ratings of the MLOview showed slightly higher observer agreement. Conclusions: Both PGMI and EAR have poor reliability and validity in evaluating mammogram quality. EAR is not a suitable alternative to PGMI, which must be improved if it is to be useful.

Abstract 2650: Reliability and Validity of Retrospective Extraction of the ABCD 2 Score From Medical Records

Stroke ◽  
2012 ◽  
Vol 43 (suppl_1) ◽  
Author(s):  
Koto Ishida ◽  
Jonathan M Raser ◽  
Christina A Wilson ◽  
Scott E Kasner ◽  
Michael T Mullen ◽  
...  
Keyword(s):  
Clinical Features ◽  
Medical Records ◽  
Interrater Reliability ◽  
Chart Review ◽  
Reliability And Validity ◽  
Interrater Agreement ◽  
Risk Category ◽  
Kappa Statistics ◽  
Retrospective Assessment ◽  
Record Review

Background and Purpose: The ABCD 2 score is increasingly used for risk stratification of TIA patients. Many of the studies assessing the utility of the ABCD 2 score have retrospectively extracted the score from medical records, but the reliability of estimation based on chart review has not been reported. We sought to determine whether the ABCD 2 score can be estimated retrospectively from medical records. Methods: We compared ABCD 2 scores that were prospectively determined by a board certified vascular neurologist to scores determined retrospectively from medical record review. All ED records and initial neurology consult resident notes were abstracted with explicit ABCD 2 scoring redacted. ABCD 2 scores were then estimated by two independent blinded trained raters using these records. Estimated component, total scores, and ABCD 2 risk category (0-3, 4-5, 6-7), were compared both between raters and with the prospectively obtained scores. Reliability was assessed using unweighted kappa statistics. Results: Records were analyzed for 55 consecutive TIA patients. For the two raters of retrospective ABCD 2 scores, interrater reliability was good with 75% exact agreement in total score between raters (κ=0.68) and 84% agreement for ABCD 2 category (κ=0.74). Interrater agreement on individual items was greatest for age and diabetes, and poorest for clinical features and duration. Retrospectively estimated total ABCD 2 scores were identical to prospective scores in 52% of patients. Agreement between raters and the prospectively obtained score was >90% for age, blood pressure, and diabetes, but was only ∼70% for clinical features and duration. Retrospectively estimated ABCD 2 category was identical to the prospective category in 64% of patients ( Table ). Dichotomizing at an ABCD 2 score ≥4, retrospective assessment had a sensitivity of 83% (CI 70-95%) and specificity of 67% (CI 40-95%) compared to the prospective score. Conclusions: The ABCD 2 score can be abstracted from medical records with good interrater reliability but limited validity, which may lead to misclassification of risk category in over one-third of patients.

scholarly journals Neurodevelopment in Children Exposed to Zika Virus: What Are the Consequences for Children Who Do Not Present with Microcephaly at Birth?

Viruses ◽  
2021 ◽  
Vol 13 (8) ◽  
pp. 1427
Author(s):  
Paula Sobral da Silva ◽  
Sophie Eickmann ◽  
Ricardo Ximenes ◽  
Celina Martelli ◽  
Elizabeth Brickley ◽  
...  
Keyword(s):  
Neurological Examination ◽  
Zika Virus ◽  
Control Group ◽  
P Value ◽  
Zikv Infection ◽  
Clinical Neurological Examination ◽  
Cognitive Delay ◽  
Increased Risk ◽  
Neurodevelopmental Impairment ◽  
The Impact

The relation of Zika virus (ZIKV) with microcephaly is well established. However, knowledge is lacking on later developmental outcomes in children with evidence of maternal ZIKV infection during pregnancy born without microcephaly. The objective of this analysis is to investigate the impact of prenatal exposure to ZIKV on neuropsychomotor development in children without microcephaly. We evaluated 274 children including 235 ZIKV exposed and 39 controls using the Bayley-III Scales of Infant and Toddler Development (BSIDIII) and neurological examination. We observed a difference in cognition with a borderline p-value (p = 0.052): 9.4% of exposed children and none of the unexposed control group had mild to moderate delays. The prevalence of delays in the language and motor domains did not differ significantly between ZIKV-exposed and unexposed children (language: 12.3% versus 12.8%; motor: 4.7% versus 2.6%). Notably, neurological examination results were predictive of neurodevelopmental delays in the BSIDIII assessments for exposed children: 46.7% of children with abnormalities on clinical neurological examination presented with delay in contrast to 17.8% among exposed children without apparent neurological abnormalities (p = 0.001). Overall, our findings suggest that relative to their unexposed peers, ZIKV-exposed children without microcephaly are not at considerably increased risk of neurodevelopmental impairment in the first 42 months of life, although a small group of children demonstrated higher frequencies of cognitive delay. It is important to highlight that in the group of exposed children, an abnormal neuroclinical examination may be a predictor of developmental delay. The article contributes to practical guidance and advances our knowledge about congenital Zika.

Development and Initial Validation of a Project-Based Rubric to Assess the Systems-Based Practice Competency of Residents in the Clinical Chemistry Rotation of a Pathology Residency

2014 ◽  
Vol 138 (6) ◽  
pp. 809-813
Author(s):  
Carolyn R. Vitek ◽  
Jane C. Dale ◽  
Henry A. Homburger ◽  
Sandra C. Bryant ◽  
Amy K. Saenger ◽  
...  
Keyword(s):  
Critical Thinking ◽  
Interrater Reliability ◽  
Clinical Chemistry ◽  
Core Competencies ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Thinking Skills ◽  
Project Evaluation ◽  
Critical Thinking Skills

Context.— Systems-based practice (SBP) is 1 of 6 core competencies required in all resident training programs accredited by the Accreditation Council for Graduate Medical Education. Reliable methods of assessing resident competency in SBP have not been described in the medical literature. Objective.— To develop and validate an analytic grading rubric to assess pathology residents' analyses of SBP problems in clinical chemistry. Design.— Residents were assigned an SBP project based upon unmet clinical needs in the clinical chemistry laboratories. Using an iterative method, we created an analytic grading rubric based on critical thinking principles. Four faculty raters used the SBP project evaluation rubric to independently grade 11 residents' projects during their clinical chemistry rotations. Interrater reliability and Cronbach α were calculated to determine the reliability and validity of the rubric. Project mean scores and range were also assessed to determine whether the rubric differentiated resident critical thinking skills related to the SBP projects. Results.— Overall project scores ranged from 6.56 to 16.50 out of a possible 20 points. Cronbach α ranged from 0.91 to 0.96, indicating that the 4 rubric categories were internally consistent without significant overlap. Intraclass correlation coefficients ranged from 0.63 to 0.81, indicating moderate to strong interrater reliability. Conclusions.— We report development and statistical analysis of a novel SBP project evaluation rubric. The results indicate the rubric can be used to reliably assess pathology residents' critical thinking skills in SBP.

Reliability and Validity Analyses of the Youth Level of Service/Case Management Inventory

2005 ◽  
Vol 32 (3) ◽  
pp. 329-344 ◽  
Author(s):  
Fred Schmidt ◽  
Robert D. Hoge ◽  
Lezlie Gomes
Keyword(s):  
Juvenile Offenders ◽  
Case Management ◽  
Interrater Reliability ◽  
Assessment Tool ◽  
Reliability And Validity ◽  
Level Of Service ◽  
Risk Level ◽  
Criminogenic Needs ◽  
Health Assessments ◽  
Males And Females

The Youth Level of Service/Case Management Inventory (YLS/CMI) is a structured assessment tool designed to facilitate the effective intervention and rehabilitation of juvenile offenders by assessing each youth’s risk level and criminogenic needs. The present study examined the YLS/CMI’s reliability and validity in a sample of 107 juvenile offenders who were court-referred for mental health assessments. Results demonstrated the YLS/CMI’s internal consistency and interrater reliability. Moreover, the instrument’s predictive validity was substantiated on a number of recidivism measures for both males and females. Limitations of the current findings are discussed.

scholarly journals Examining Interrater Reliability and Validity of a Paediatric Cardiopulmonary Physiotherapy Discharge Tool

2014 ◽  
Vol 66 (2) ◽  
pp. 153-159 ◽  
Author(s):  
Jamil Lati ◽  
Vanessa Pellow ◽  
Jeannine Sproule ◽  
Dina Brooks ◽  
Cindy Ellerton
Keyword(s):  
Interrater Reliability ◽  
Reliability And Validity

“You Say Severe, I Say Mild”: Toward an Empirical Classification of Dysarthria Severity

2021 ◽  
pp. 1-18
Author(s):  
Kaila L. Stipancic ◽  
Kira M. Palmer ◽  
Hannah P. Rowe ◽  
Yana Yunusova ◽  
James D. Berry ◽  
...  
Keyword(s):  
Speech Intelligibility ◽  
Interrater Reliability ◽  
Reliability And Validity ◽  
Roc Curves ◽  
Speaking Rate ◽  
Response Options ◽  
Cutoff Points ◽  
Healthy Control ◽  
Dysarthric Speech ◽  
Surrogate Measures

Purpose: The main purpose of this study was to create an empirical classification system for speech severity in patients with dysarthria secondary to amyotrophic lateral sclerosis (ALS) by exploring the reliability and validity of speech-language pathologists' (SLPs') ratings of dysarthric speech. Method: Ten SLPs listened to speech samples from 52 speakers with ALS and 20 healthy control speakers. SLPs were asked to rate the speech severity of the speakers using five response options: normal, mild, moderate, severe, and profound. Four severity-surrogate measures were also calculated: SLPs transcribed the speech samples for the calculation of speech intelligibility and rated the effort it took to understand the speakers on a visual analog scale. In addition, speaking rate and intelligible speaking rate were calculated for each speaker. Intrarater and interrater reliability were calculated for each measure. We explored the validity of clinician-based severity ratings by comparing them to the severity-surrogate measures. Receiver operating characteristic (ROC) curves were conducted to create optimal cutoff points for defining dysarthria severity categories. Results: Intrarater and interrater reliability for the clinician-based severity ratings were excellent and were comparable to reliability for the severity-surrogate measures explored. Clinician severity ratings were strongly associated with all severity-surrogate measures, suggesting strong construct validity. We also provided a range of values for each severity-surrogate measure within each severity category based on the cutoff points obtained from the ROC analyses. Conclusions: Clinician severity ratings of dysarthric speech are reliable and valid. We discuss the underlying challenges that arise when selecting a stratification measure and offer recommendations for a classification scheme when stratifying patients and research participants into speech severity categories.

Is multidetector computed tomography comparable to magnetic resonance imaging for assessment of lumbar foraminal stenosis?

2016 ◽  
Vol 58 (2) ◽  
pp. 197-203
Author(s):  
Woo Young Kang ◽  
Joong Mo Ahn ◽  
Joon Woo Lee ◽  
Eugene Lee ◽  
Yun Jung Bae ◽  
...  
Keyword(s):  
Magnetic Resonance Imaging ◽  
Computed Tomography ◽  
Magnetic Resonance ◽  
Multidetector Computed Tomography ◽  
Excellent Agreement ◽  
Observer Agreement ◽  
Kappa Statistics ◽  
Foraminal Stenosis ◽  
Resonance Imaging ◽  
Lumbar Foraminal Stenosis

Background Both multidetector computed tomography (MDCT) and magnetic resonance imaging (MRI) are used for assessment of lumbar foraminal stenosis (LFS). Therefore, it is relevant to assess agreement between these imaging modalities. Purpose To determine intermodality, inter-, and intra-observer agreement for assessment of LFS on MDCT and MRI. Material and Methods A total of 120 foramina in 20 patients who visited our institution in January and February 2014 were evaluated by six radiologists with different levels of experience. Radiologists evaluated presence and severity of LFS on sagittal CT and MR images according to a previously published LFS grading system. Intermodality agreement was analyzed by using weighted kappa statistics, while inter- and intra-observer agreement were analyzed by using intraclass correlation coefficients (ICCs) and kappa statistics. Results Overall intermodality agreement was moderate to good (kappa, 0.478–0.765). In particular, two professors and one fellow tended to overestimate the degree of LFS on CT compared with MRI. For inter-observer agreement of all six observers, ICCs indicated excellent agreement for both CT (0.774) and MRI (0.771), while Fleiss’ kappa values showed moderate agreement for CT (0.482) and MRI (0.575). There was better agreement between professors and fellows compared with residents. For intra-observer agreement, ICCs indicated excellent agreement, while kappa values showed good to excellent agreement for both CT and MRI. Conclusion MDCT was comparable to MRI for diagnosis and assessment of LFS, especially for experienced observers. However, there was a tendency to overestimate the degree of LFS on MDCT compared with MRI.

scholarly journals Diagnostic validity of different cephalometric analyses for assessment of the sagittal skeletal pattern

2018 ◽  
Vol 23 (5) ◽  
pp. 75-81 ◽  
Author(s):  
Maheen Ahmed ◽  
Attiya Shaikh ◽  
Mubassar Fida
Keyword(s):  
Reliability And Validity ◽  
Class I ◽  
Kappa Statistics ◽  
Class Iii ◽  
Validity And Reliability ◽  
Substantial Agreement ◽  
Lateral Cephalograms ◽  
Skeletal Pattern ◽  
Beta Angle ◽  
Final Group

Abstract Introduction: Numerous cephalometric analyses have been proposed to diagnose the sagittal discrepancy of the craniofacial structures. Objective: This study aimed at evaluating the reliability and validity of different skeletal analyses for the identification of sagittal skeletal pattern. Methods: A total of 146 subjects (males = 77; females = 69; mean age = 23.6 ± 4.6 years) were included. The ANB angle, Wits appraisal, Beta angle, AB plane angle, Downs angle of convexity and W angle were used to assess the anteroposterior skeletal pattern on lateral cephalograms. The sample was classified into Class I, II and III groups as determined by the diagnostic results of majority of the parameters. The validity and reliability of the aforementioned analyses were determined using Kappa statistics, sensitivity and positive predictive value (PPV). Results: A substantial agreement was present between ANB angle and the diagnosis made by the final group (k = 0.802). In the Class I group, Downs angle of convexity showed the highest sensitivity (0.968), whereas ANB showed the highest PPV (0.910). In the Class II group, ANB angle showed the highest sensitivity (0.928) and PPV (0.951). In the Class III group, the ANB angle, the Wits appraisal and the Beta angle showed the highest sensitivity (0.902), whereas the Downs angle of convexity and the ANB angle showed the highest PPV (1.00). Conclusion: The ANB angle was found to be the most valid and reliable indicator in all sagittal groups. Downs angle of convexity, Wits appraisal and Beta angle may be used as valid indicators to assess the Class III sagittal pattern.

Sign in / Sign up

Export Citation Format

Share Document